From dovecot at paz.bz Sun Mar 1 07:53:52 2015 From: dovecot at paz.bz (Jim Pazarena) Date: Sat, 28 Feb 2015 23:53:52 -0800 Subject: IP drop list Message-ID: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> I wonder if there is an easy way to provide dovecot a flat text file of ipv4 #'s which should be ignored or dropped? I have accumulated 45,000+ IPs which routinely try dictionary and 12345678 password attempts. The file is too big to create firewall drops, and I don't want to compile with wrappers *if* dovecot has an easy ability to do this. If dovecot could parse a flat text file of IPs and drop connections it would sure put a dent in these attempts. Thanks. From HFlor at gmx.de Sun Mar 1 08:42:39 2015 From: HFlor at gmx.de (Hardy Flor) Date: Sun, 01 Mar 2015 09:42:39 +0100 Subject: IP drop list In-Reply-To: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> Message-ID: <54F2D0FF.5020309@gmx.de> fail2ban blocked dynamically addresses for a period of time. It has a module for dovecot. > I wonder if there is an easy way to provide dovecot a flat text file > of ipv4 #'s which should be ignored or dropped? > > I have accumulated 45,000+ IPs which routinely try dictionary and > 12345678 password attempts. The file is too big to create firewall > drops, and I don't want to compile with wrappers *if* dovecot has an > easy ability to do this. If dovecot could parse a flat text file of > IPs and drop connections it would sure put a dent in these attempts. From h.reindl at thelounge.net Sun Mar 1 09:25:43 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Sun, 01 Mar 2015 10:25:43 +0100 Subject: IP drop list In-Reply-To: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> Message-ID: <54F2DB17.2090509@thelounge.net> Am 01.03.2015 um 08:53 schrieb Jim Pazarena: > I wonder if there is an easy way to provide dovecot a flat text file of > ipv4 #'s which should be ignored or dropped? > > I have accumulated 45,000+ IPs which routinely try dictionary and > 12345678 password attempts. The file is too big to create firewall > drops, and I don't want to compile with wrappers *if* dovecot has an > easy ability to do this. If dovecot could parse a flat text file of IPs > and drop connections it would sure put a dent in these attempts. hence i asked month ago for RBL support because such lists are easy to feed into http://www.corpit.ru/mjt/rbldnsd.html - sadly i got no reply than use fail2ban and what not irrelevant if there is already a local dnsbl i guess for a C-programmer it takes not much more than 10 minutens include a config option to list rbl servers and close connections absed on the DNS responses -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From karol at babioch.de Sun Mar 1 10:47:29 2015 From: karol at babioch.de (Karol Babioch) Date: Sun, 01 Mar 2015 11:47:29 +0100 Subject: Require certificate for external clients In-Reply-To: References: Message-ID: <54F2EE41.9050806@babioch.de> Hi, Am 28.02.2015 um 00:28 schrieb Joseph Tam: > That should be qualified as "Is it possible to have Dovecot imap/pop > daemons listening on multiple ports for a single running instance." Yes, exactly. > You can share libraries, binaries, > log files, but use separate configuration files, specifying different > ports/addresses/ssl-configs/auth/access parameters. Then you can fire > them both up > > dovecot -c /dovecot/etc/dovecot-1.conf > dovecot -c /dovecot/etc/dovecot-2.conf I will have to look into it. I'm afraid that I would have to fiddle around with the default unit files. Also I'm not completely sure how this would work with all of the configuration files that have been split off into small chunks and get included at some point. This is probably going to be messy rather quickly :'(. Thanks for your suggestion. Best regards, Karol Babioch -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From mail at marc-stuermer.de Sun Mar 1 13:34:43 2015 From: mail at marc-stuermer.de (Marc Stuermer) Date: Sun, 01 Mar 2015 14:34:43 +0100 Subject: IP drop list In-Reply-To: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> Message-ID: <54F31573.7080903@marc-stuermer.de> Am 01.03.2015 um 08:53 schrieb Jim Pazarena: > I have accumulated 45,000+ IPs which routinely try dictionary and > 12345678 password attempts. The file is too big to create firewall > drops, and I don't want to compile with wrappers *if* dovecot has an Have you ever tried using IP sets on Linux? From r at sys4.de Sun Mar 1 21:14:36 2015 From: r at sys4.de (Ralf Hildebrandt) Date: Sun, 1 Mar 2015 22:14:36 +0100 Subject: full text index "per user"? Message-ID: <20150301211436.GB2996@sys4.de> Is there any way of disabling the creation of a full text index on a per user basis? -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From mcguire at neurotica.com Sun Mar 1 22:16:30 2015 From: mcguire at neurotica.com (Dave McGuire) Date: Sun, 01 Mar 2015 17:16:30 -0500 Subject: IP drop list In-Reply-To: <54F2DB17.2090509@thelounge.net> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> Message-ID: <54F38FBE.3040108@neurotica.com> On 03/01/2015 04:25 AM, Reindl Harald wrote: >> I wonder if there is an easy way to provide dovecot a flat text >> file of ipv4 #'s which should be ignored or dropped? >> >> I have accumulated 45,000+ IPs which routinely try dictionary >> and 12345678 password attempts. The file is too big to create >> firewall drops, and I don't want to compile with wrappers *if* >> dovecot has an easy ability to do this. If dovecot could parse a >> flat text file of IPs and drop connections it would sure put a >> dent in these attempts. > > hence i asked month ago for RBL support because such lists are easy > to feed into http://www.corpit.ru/mjt/rbldnsd.html - sadly i got no > reply than use fail2ban and what not irrelevant if there is already > a local dnsbl > > i guess for a C-programmer it takes not much more than 10 minutens > include a config option to list rbl servers and close connections > absed on the DNS responses I've been asking for this off-and-on for years, and people immediately parrot back "just use fail2ban". I think fail2ban is a nice idea and all, but that suggestion assumes that I use iptables (I don't), I run firewalls on my servers (I don't; I run them on routers) and that I run Linux on my mail server (I don't). The other side of this equation, Postfix, has had this capability for years. Why it hasn't been added to dovecot is a mystery. It's the only thing (really, the ONLY thing!) that I dislike about dovecot. -Dave -- Dave McGuire, AK4HZ/3 New Kensington, PA From h.reindl at thelounge.net Sun Mar 1 22:20:09 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Sun, 01 Mar 2015 23:20:09 +0100 Subject: IP drop list In-Reply-To: <54F38FBE.3040108@neurotica.com> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> <54F38FBE.3040108@neurotica.com> Message-ID: <54F39099.2010006@thelounge.net> Am 01.03.2015 um 23:16 schrieb Dave McGuire: > On 03/01/2015 04:25 AM, Reindl Harald wrote: >>> I wonder if there is an easy way to provide dovecot a flat text >>> file of ipv4 #'s which should be ignored or dropped? >>> >>> I have accumulated 45,000+ IPs which routinely try dictionary >>> and 12345678 password attempts. The file is too big to create >>> firewall drops, and I don't want to compile with wrappers *if* >>> dovecot has an easy ability to do this. If dovecot could parse a >>> flat text file of IPs and drop connections it would sure put a >>> dent in these attempts. >> >> hence i asked month ago for RBL support because such lists are easy >> to feed into http://www.corpit.ru/mjt/rbldnsd.html - sadly i got no >> reply than use fail2ban and what not irrelevant if there is already >> a local dnsbl >> >> i guess for a C-programmer it takes not much more than 10 minutens >> include a config option to list rbl servers and close connections >> absed on the DNS responses > > I've been asking for this off-and-on for years, and people > immediately parrot back "just use fail2ban". I think fail2ban is a > nice idea and all, but that suggestion assumes that I use iptables (I > don't), I run firewalls on my servers (I don't; I run them on routers) > and that I run Linux on my mail server (I don't). > > The other side of this equation, Postfix, has had this capability > for years. Why it hasn't been added to dovecot is a mystery. It's > the only thing (really, the ONLY thing!) that I dislike about dovecot even if you use Linux, Firewalls and what not * postfix supports RBL's in several ways on the MTA * mod_security and so webservers support RBL's * RBL's are *centralized* * DNS queries, especially in a LAN, are cheap everybody answering with fail2ban if someone asks for RBL support has no clue what he is talking about because he did not get the question -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From gcr+dovecot at tharned.org Sun Mar 1 22:46:19 2015 From: gcr+dovecot at tharned.org (Greg Rivers) Date: Sun, 1 Mar 2015 16:46:19 -0600 (CST) Subject: dsync panic Message-ID: As per , I'm running the following command on a local dovecot server to replicate email for a single user from a remote IMAP server: doveadm -D \ -o imapc_host=remote.imap.server \ -o imapc_user=gcr \ -o imapc_password=XXXXXXXX \ -o imapc_list_prefix=IMAP \ -o imapc_features="rfc822.size fetch-headers" \ -o mail_prefetch_count=20 \ -o mail_fsync=never \ backup -R -u gcr imapc: This runs fine for a while and successfully copies quite a lot of mail, but always aborts before completion with the following error: dsync(gcr): Panic: file mail-transaction-log.c: line 271 (mail_transaction_log_rotate): assertion failed: (file->locked) The exit code is 262. Does anyone know why this might happen or how to fix it? -- Greg Rivers -------------- next part -------------- # 2.2.15: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.6 (3e924b1b6c5c+) # OS: FreeBSD 10.1-RELEASE-p6 amd64 auth_verbose = yes imap_id_log = * imap_id_send = name * version * os * os-version * mail_location = mdbox:~/.mdbox mail_plugins = " quota zlib" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate editheader vnd.dovecot.debug imapflags notify vnd.dovecot.duplicate vnd.dovecot.pipe vnd.dovecot.filter vnd.dovecot.execute namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = %s driver = pam } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve sieve_execute_bin_dir = /usr/local/etc/dovecot/sieve/execute sieve_execute_socket_dir = sieve-execute sieve_extensions = +notify +imapflags +editheader +vnd.dovecot.duplicate +vnd.dovecot.pipe +vnd.dovecot.filter +vnd.dovecot.execute +vnd.dovecot.debug sieve_filter_bin_dir = /usr/local/etc/dovecot/sieve/filter sieve_filter_socket_dir = sieve-filter sieve_global = /usr/local/etc/dovecot/sieve sieve_max_actions = 0 sieve_max_redirects = 16 sieve_max_script_size = 0 sieve_pipe_bin_dir = /usr/local/etc/dovecot/sieve/pipe sieve_pipe_socket_dir = sieve-pipe sieve_plugins = sieve_extprograms } postmaster_address = postmaster at local.domain protocols = imap lmtp sieve quota_full_tempfail = yes ssl_cert = References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> Message-ID: <14bd7992180.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> On March 1, 2015 10:26:40 AM Reindl Harald wrote: > i guess for a C-programmer it takes not much more than 10 minutens > include a config option to list rbl servers and close connections absed > on the DNS responses close pop3, set imap to listen only in lo interface, setup webmail with smtp auth, now then in apache install mod geoip, and only allow countrys with users in is imho the current most simplest, but maybe not the most usefull :( From h.reindl at thelounge.net Sun Mar 1 23:17:54 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 02 Mar 2015 00:17:54 +0100 Subject: IP drop list In-Reply-To: <14bd7992180.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> <14bd7992180.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> Message-ID: <54F39E22.30209@thelounge.net> Am 02.03.2015 um 00:08 schrieb Benny Pedersen: > On March 1, 2015 10:26:40 AM Reindl Harald wrote: > >> i guess for a C-programmer it takes not much more than 10 minutens >> include a config option to list rbl servers and close connections absed >> on the DNS responses > > close pop3, set imap to listen only in lo interface, setup webmail with > smtp auth, now then in apache install mod geoip, and only allow > countrys with users in what a foolish trolling as usual from you.... -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From me at junc.eu Sun Mar 1 23:34:55 2015 From: me at junc.eu (Benny Pedersen) Date: Mon, 02 Mar 2015 00:34:55 +0100 Subject: IP drop list In-Reply-To: <54F38FBE.3040108@neurotica.com> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> <54F38FBE.3040108@neurotica.com> Message-ID: <14bd7b14530.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> > The other side of this equation, Postfix, has had this capability > for years. Why it hasn't been added to dovecot is a mystery. It's > the only thing (really, the ONLY thing!) that I dislike about dovecot. http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/AllowNets then setup fail2ban to manage extrafields From jtam.home at gmail.com Mon Mar 2 03:06:44 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Sun, 1 Mar 2015 19:06:44 -0800 (PST) Subject: Require certificate for external clients In-Reply-To: References: Message-ID: Karol Babioch writes: >> You can share libraries, binaries, >> log files, but use separate configuration files, specifying different >> ports/addresses/ssl-configs/auth/access parameters. Then you can fire >> them both up >> >> dovecot -c /dovecot/etc/dovecot-1.conf >> dovecot -c /dovecot/etc/dovecot-2.conf > > I will have to look into it. I'm afraid that I would have to fiddle > around with the default unit files. What are "unit files"? > Also I'm not completely sure how this would work with all of the > configuration files that have been split off into small chunks and get > included at some point. This is probably going to be messy rather > quickly :'(. I don't see why it would be messy. If the conf.d/* are the same, you can use the same config directory. If they differ, you can can copy those files to another config directory (e.g. conf-d/* -> conf-2.d/), modify the snippets that differ, then include this alternate set of configurations !include conf-2.d/* > Thanks for your suggestion. You're welcome. Joseph Tam From dlasota at alaska.edu Mon Mar 2 05:03:20 2015 From: dlasota at alaska.edu (Dan LaSota) Date: Sun, 1 Mar 2015 20:03:20 -0900 Subject: Connect failed to database Message-ID: <0B461721-6E10-4290-A61C-B86146BC7798@alaska.edu> I have dovecot version 2.2.10 dovecot -n output below I am seeing connection errors being written to my dovecot error log: Mar 1 19:51:15 mail dovecot: auth-worker(2224): Error: mysql(localhost): Connect failed to database (servermail): Access denied for user 'usermail'@'localhost' (using password: YES) - waiting for 5 seconds before retry My connection script located at /etc/dovecot/dovecot-sql.conf.ext is like (password edited): driver = mysql connect = host=localhost dbname=servermail user='usermail' password='MY_SUPER_SECRET_PASSWORD' default_pass_scheme = SHA512-CRYPT password_query = SELECT email as user, password FROM virtual_users WHERE email='%u'; I have verified that I can gain access to the SQL database with # mysql -u usermail -p I can make select statements on the 'servermail' database and all of its tables. I've searched for similar errors from users, but most of the questions are unanswered, or answered incorrectly. I did follow one thread's suggest of setting the MYSQL password for the 'usermail' with OLD_PASSWORD instead of PASSWORD. I did that. I could still login from the shell using mysql -u usermail -p But dovecot still wrote the same error. (I did a flush privileges, and restarted mysql, and dovecot) I then set the password in SQL back using PASSWORD. (flush'd priveleges and restarte mysql and dovecot). Still Errors. Looking for leads. Thanks, Dan LaSota Instructional Designer, UAF eLearning (907) 451-4067 dan.lasota at alaska.edu http://elearning.uaf.edu dovecot -n # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-123.20.1.el7.x86_64 x86_64 CentOS Linux release 7.0.1406 (Core) xfs auth_debug = yes auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = plain mail_debug = yes mail_location = maildir:/var/mail/vhosts/%d/%n mail_privileged_group = mail mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocols = imap lmtp service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = dovecot } service imap-login { inet_listener imap { port = 0 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl = required ssl_cert = References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 27 Feb 2015, Tim Jones wrote: > userdb { > args = username_format=%n /home/%d/mail_users > default_fields = uid=%d gid=%d home=/home/%d/mail/%n > mail=maildir:/home/%d/mail/%n > driver = passwd-file > } > > Every time I try to authenticate via imap, I get the error > >> dovecot: auth: Fatal: passwd-file userdb: Invalid uid: %d Putting aside the question, whether or not %d is/should be expanded in default_fields = uid=%d gid=%d couldn't or shouldn't you place the correct numerical ids in the file anyway? > If I put fixed a uid and gid in the userdb default_fields line: > >> default_fields = uid=example.com gid=example.com home=/home/%d/mail/%n mail=maildir:/home/%d/mail/%n > > authentication passes without a problem, but of course, only for users > of example.com. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVPQTPXz1H7kL/d9rAQIXQAgAx2u5pds6c3YygbHwCj4lkK1DR3FVKx4+ u24cLNacU/AiqNYC4AN6gQ180EaqK3M3bPnfV0rsuyy6fOcHwXm6HpbYIUNkUbBV WZYZPu0K+FoLWLL0JnjoXAe2QWK0whb9h4BLeG9xIA6FpRKwwnJYv0MuEqk78rZv HHrwbZra1XEbqKJsMHxYyhZ+ZOA9cC/mz25iZhBdygPPuxVgr7RtFfppI2DwdU+n XeDsr7OExgMgetBFImEYnA9YzZ5P7kxpNguaNoMtB5MFKsfhtaeCkATOoBW8Mpcw KTfVQriL3TiyVfYFU4eruJBcz6XOOvkEcrWiZytJV5WDl8GDYfYRqA== =X8eE -----END PGP SIGNATURE----- From mail at oliwel.de Mon Mar 2 07:38:39 2015 From: mail at oliwel.de (Oliver Welter) Date: Mon, 02 Mar 2015 08:38:39 +0100 Subject: IP drop list In-Reply-To: <54F38FBE.3040108@neurotica.com> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> <54F38FBE.3040108@neurotica.com> Message-ID: <54F4137F.8050507@oliwel.de> Am 01.03.2015 um 23:16 schrieb Dave McGuire: > On 03/01/2015 04:25 AM, Reindl Harald wrote: >>> I wonder if there is an easy way to provide dovecot a flat text >>> file of ipv4 #'s which should be ignored or dropped? >>> >>> I have accumulated 45,000+ IPs which routinely try dictionary >>> and 12345678 password attempts. The file is too big to create >>> firewall drops, and I don't want to compile with wrappers *if* >>> dovecot has an easy ability to do this. If dovecot could parse a >>> flat text file of IPs and drop connections it would sure put a >>> dent in these attempts. >> >> hence i asked month ago for RBL support because such lists are easy >> to feed into http://www.corpit.ru/mjt/rbldnsd.html - sadly i got no >> reply than use fail2ban and what not irrelevant if there is already >> a local dnsbl >> >> i guess for a C-programmer it takes not much more than 10 minutens >> include a config option to list rbl servers and close connections >> absed on the DNS responses > > I've been asking for this off-and-on for years, and people > immediately parrot back "just use fail2ban". I think fail2ban is a > nice idea and all, but that suggestion assumes that I use iptables (I > don't), I run firewalls on my servers (I don't; I run them on routers) > and that I run Linux on my mail server (I don't). > > The other side of this equation, Postfix, has had this capability > for years. Why it hasn't been added to dovecot is a mystery. It's > the only thing (really, the ONLY thing!) that I dislike about dovecot. > Guys, dovecot is open source - if you desire a feature that the upstream programmer did not include, pay him a bounty to do so or send him a patch to be included. Period. We can discuss and mightbe somebody will fork if he is not willing to accept such a solutuion for any political reason. I am really tired of reading this kind of complaints on OSS lists. To make this not a "troll only" posting - it might be an suitable approach to let dovecot listen on the lo interface and put a proxy software in front, that supports RBLs. Oliver -- Protect your environment - close windows and adopt a penguin! -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4074 bytes Desc: S/MIME Cryptographic Signature URL: From mail at oliwel.de Mon Mar 2 07:42:52 2015 From: mail at oliwel.de (Oliver Welter) Date: Mon, 02 Mar 2015 08:42:52 +0100 Subject: Connect failed to database In-Reply-To: <0B461721-6E10-4290-A61C-B86146BC7798@alaska.edu> References: <0B461721-6E10-4290-A61C-B86146BC7798@alaska.edu> Message-ID: <54F4147C.4040708@oliwel.de> Am 02.03.2015 um 06:03 schrieb Dan LaSota: > I have dovecot version 2.2.10 > dovecot -n output below > > I am seeing connection errors being written to my dovecot error log: > Mar 1 19:51:15 mail dovecot: auth-worker(2224): Error: mysql(localhost): Connect failed to database (servermail): Access denied for user 'usermail'@'localhost' (using password: YES) - waiting for 5 seconds before retry > Just some quick ideas * check if the mysql socket file has rw permissions for the dovecot user * Try to run the mysql query as user dovecot (su dovecot) * Try to set the local ip instead of localhost (mysql makes a difference in the ACL checks if you come from localhost) Oliver -- Protect your environment - close windows and adopt a penguin! -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4074 bytes Desc: S/MIME Cryptographic Signature URL: From skdovecot at smail.inf.fh-brs.de Mon Mar 2 07:50:33 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 2 Mar 2015 08:50:33 +0100 (CET) Subject: Require certificate for external clients In-Reply-To: <54F07F6C.6050008@babioch.de> References: <54F07F6C.6050008@babioch.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 27 Feb 2015, Karol Babioch wrote: > I'm currently looking into ways of making use of client certificates. I > want to force external clients (i.e. anything outside the local subnet) > to use client certificates. It is my understanding that this in itself > can be achieved with the "ssl_require_client_cert" setting. > > However, I also want local clients (i.e. anything from a specific > subnet) to be able to authenticate by the usual means (i.e. password-based). There are local and remote IP blocks in Dovecot, however, I cannot find the Wiki page it is documented on. But see: http://wiki2.dovecot.org/SSL/DovecotConfiguration local means to match the local IP of the connection, remote matches the remote end, aka client IP address. You could try to use ssl_require_client_cert as default and add a remote { } block, in which you disable that feature. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVPQWSXz1H7kL/d9rAQITnQf+PrgqIyf98ZhF1TbL/7MAfEMYBZCHXvF4 iUScUxYyaUbeJ/h2RkeXjpVfrp9ktPXDmM+yge9U1fbDJ8ejQ+7nn0ZnSWqm8Cpm SlhnkYEBfdR1ht5fzGNj1hy9CA3vLZRzCoAtPBL58VZocyFnDDdtcgFpgBg0gKaE Cmf6BYs0AtvP6omUSj4myh4lW5trklebtxClZS2K6Zol+rpATofGTfE16wRrEnBK kt4N8ZKZ70vwt8wCiytcqddegIDm9uiiSfrK0W57o5n377oZtHzN2luCOQ3S4GdF aMh6ybDEN8NeS+3pbTQp/QXa1hm4x2UefEjI1KUJJSkniKGsv6knzA== =DmyK -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Mon Mar 2 07:58:47 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 2 Mar 2015 08:58:47 +0100 (CET) Subject: Connect failed to database In-Reply-To: <54F4147C.4040708@oliwel.de> References: <0B461721-6E10-4290-A61C-B86146BC7798@alaska.edu> <54F4147C.4040708@oliwel.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 2 Mar 2015, Oliver Welter wrote: > Am 02.03.2015 um 06:03 schrieb Dan LaSota: >> I have dovecot version 2.2.10 >> dovecot -n output below >> >> I am seeing connection errors being written to my dovecot error log: >> Mar 1 19:51:15 mail dovecot: auth-worker(2224): Error: mysql(localhost): >> Connect failed to database (servermail): Access denied for user >> 'usermail'@'localhost' (using password: YES) - waiting for 5 seconds before >> retry >> > > Just some quick ideas > * check if the mysql socket file has rw permissions for the dovecot user > * Try to run the mysql query as user dovecot (su dovecot) > * Try to set the local ip instead of localhost (mysql makes a difference in > the ACL checks if you come from localhost) (Y) in addition: * Did mysql logged something useful? - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVPQYN3z1H7kL/d9rAQLyRwgAlasqa/rDY86UmYHF2+e/Q5++oCC/8n0a 0sCyQdY8SVJA8jsZbL4+B/F9lwkMA+7gSkiSDuLQWM/c7VotBhQ5AvZKOXfEUmCZ DFH7J2dZMwPjAubcdjjp2lnA97NS4wt3+dqyo4ezCEcc+ZKjDh8QSuPAO8xRP1Dq pK/47DYi9yyz0dExQlQ1Fx1w792n4igCuPySThT03k+yRZpx4x5Va4/s0TM5ZwLP JaRZWo8IzzWjFWvCZQDGWCpy1+TWNTN1NUAfN2ngZSxWGq0mpPX9dFerXJdgyBzg LCYGkufOO1FjlT+bRDqezBf/ps5MJsObeJr/Z816u1JdCS2Uc49CbQ== =Z7XP -----END PGP SIGNATURE----- From tim.jones at fon.com Mon Mar 2 08:12:44 2015 From: tim.jones at fon.com (Tim Jones) Date: Mon, 2 Mar 2015 09:12:44 +0100 Subject: userdb passwd-file default_fields uid not expanding %variable In-Reply-To: References: Message-ID: > Putting aside the question, whether or not %d is/should be expanded in > default_fields = uid=%d gid=%d > couldn't or shouldn't you place the correct numerical ids in the file > anyway? The issue is, and maybe this was a bad requirement on my part, that each domains' mailbox is owned by the user associated with that domain. I don't have a global 'virtualmail' user that handles the mailboxes. Each 'domain' hosted on the server is a system user, with their own PHP instance, Python virtualenv etc, and maildir inside $HOME. So until the client issues a log in, I don't know which uid\gid to assign to them. Many thanks, Tim Jones Software Development +34 612345678 C/ Quintanavides 15, Edificio 2, Planta 1? Parque Empresarial V?a Norte de Metrovacesa Las Tablas 28050 Madrid Skype: tim.jones.fon All information in this email is confidential From paolo.cravero at csi.it Mon Mar 2 08:13:24 2015 From: paolo.cravero at csi.it (Paolo Cravero) Date: Mon, 2 Mar 2015 09:13:24 +0100 (CET) Subject: mdbox attachment errors In-Reply-To: <54F0A6CA.6020806@gmx.de> References: <54F0A6CA.6020806@gmx.de> Message-ID: <1121377779.753019.1425284004958.JavaMail.open-xchange@comunica.csi.it> > Il 27 febbraio 2015 alle 18.18 Hardy Flor: > copy file "f6f4f3b882bf3488af632389d4aaba8adc332b12" from backup to > "/var/mail/attachments/f6/f4/hashes/f6f4f3b882bf3488af632389d4aaba8adc332b12" > > and create hardlink to > "/var/mail/attachments/f6/f4/f6f4f3b882bf3488af632389d4aaba8adc332b12-60ab750a1aa4b554da1600009db5accb" Sure. That fix restores access to the mailbox/message/attachment. What if the file disappeared before the backup process copied it? What if 1% of 20'000 users open a ticket a day for the same reason? But ... why did it disappear at all? I got the same error on a test environment, where I am learning Dovecot (see my message sent 26/2/2015 at 14:48 CET). Nobody else has access to my server. I have not manually deleted files in the SIS partition. With a little script I ran through my mdbox m.* files, I extracted attachment hash-filenames and searched for them on the disk: 10 were missing. In some cases there is no hashes directory at all. Not even the 2nd level base path ( /attachments/a/b/hash .... stops at /attachments/a ). WHY??? Note that depending on how the IMAP client accesses the mailbox folder, you may be unable to see the whole folder at all! That's the case with Open-Xchange. Again: which tests can be run in order to confirm the behaviour and possibly a bug? Paolo Cravero From mcguire at neurotica.com Mon Mar 2 08:16:16 2015 From: mcguire at neurotica.com (Dave McGuire) Date: Mon, 02 Mar 2015 03:16:16 -0500 Subject: IP drop list In-Reply-To: <14bd7b14530.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> <54F38FBE.3040108@neurotica.com> <14bd7b14530.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> Message-ID: <54F41C50.308@neurotica.com> On 03/01/2015 06:34 PM, Benny Pedersen wrote: >> The other side of this equation, Postfix, has had this capability >> for years. Why it hasn't been added to dovecot is a mystery. It's >> the only thing (really, the ONLY thing!) that I dislike about dovecot. > > http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/AllowNets > > then setup fail2ban to manage extrafields Now that's a very interesting idea, thank you! I will investigate this. -Dave -- Dave McGuire, AK4HZ/3 New Kensington, PA From mcguire at neurotica.com Mon Mar 2 08:23:57 2015 From: mcguire at neurotica.com (Dave McGuire) Date: Mon, 02 Mar 2015 03:23:57 -0500 Subject: IP drop list In-Reply-To: <54F4137F.8050507@oliwel.de> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> <54F38FBE.3040108@neurotica.com> <54F4137F.8050507@oliwel.de> Message-ID: <54F41E1D.5030901@neurotica.com> On 03/02/2015 02:38 AM, Oliver Welter wrote: > Guys, dovecot is open source - if you desire a feature that the upstream > programmer did not include, pay him a bounty to do so or send him a > patch to be included. Period. We can discuss and mightbe somebody will > fork if he is not willing to accept such a solutuion for any political > reason. > > I am really tired of reading this kind of complaints on OSS lists. ....and this is perhaps the second most predictable knee-jerk response. I am certainly capable of writing such a patch, but there is no point in expending the effort if it would not be included in the code base. The extreme negative reactions to this idea from people in this community, every time it has come up over the years, with almost rabid ramming of fail2ban down posters' throats (Benny Pedersen's excellent suggestion not included) suggests that a patch implementing such functionality would not be well received. The idea here is not to whine until somebody pops up and assumes that I don't know how the open-source software world works. I assure you that I do. The idea is to mention, vocally, a different use case in which fail2ban (again, excepting Benny Pedersen's excellent suggestion) is not an appropriate solution, as many times as it takes to make people realize that some networks aren't exactly like theirs. In the 1980s and 1990s, we fought the great assumption of "all the world's a VAX running BSD", in which programmers everywhere wrote code that assumed EVERYONE was running that platform. Today we fight the "all the world's an x86_64 box with a gazillibyte of memory running Linux" mentality in exactly the same way. It's not any more palatable now than it was then. -Dave -- Dave McGuire, AK4HZ/3 New Kensington, PA From skdovecot at smail.inf.fh-brs.de Mon Mar 2 08:28:33 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 2 Mar 2015 09:28:33 +0100 (CET) Subject: IP drop list In-Reply-To: <54F41C50.308@neurotica.com> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> <54F38FBE.3040108@neurotica.com> <14bd7b14530.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> <54F41C50.308@neurotica.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 2 Mar 2015, Dave McGuire wrote: > On 03/01/2015 06:34 PM, Benny Pedersen wrote: >>> The other side of this equation, Postfix, has had this capability >>> for years. Why it hasn't been added to dovecot is a mystery. It's >>> the only thing (really, the ONLY thing!) that I dislike about dovecot. >> >> http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/AllowNets >> >> then setup fail2ban to manage extrafields > > Now that's a very interesting idea, thank you! I will investigate this. Does allownets support negative CIDRs? - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVPQfMXz1H7kL/d9rAQLP6Qf+KLmEwyVugxT5iXYRK5mVES5L8fsyKIM+ nZR0hMO2N2Aq30Sq6GFRc1+pJoICzP8t20X0yrOgR0pG7CfIIOwH6s/Z9RsBpFW6 WtuqPwRf5/K/KcL2IslIrvjvoYSuzlw4ny7/fLfBIwtuqlnIRhZz8L9CGAMmDWnK cPK2+qNDMGMDk9ueeriklO//BdvFcvlE9Rz/NlsmmbLXzXDN2OQdO9SqV67y7sIA pb7JSr+O2WNAIROm1tccTW22Z1YIYKjOboOHLCNr0MlPL8QDPDrSuy+z7gQpXtCC BDjXba2R/nWBAbwUR/+mJzErShCw48eERCCr7EGjQWYqd6+NHHgl6A== =xYN/ -----END PGP SIGNATURE----- From h.reindl at thelounge.net Mon Mar 2 09:00:36 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 02 Mar 2015 10:00:36 +0100 Subject: IP drop list In-Reply-To: <54F4137F.8050507@oliwel.de> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> <54F38FBE.3040108@neurotica.com> <54F4137F.8050507@oliwel.de> Message-ID: <54F426B4.8050102@thelounge.net> Am 02.03.2015 um 08:38 schrieb Oliver Welter: > I am really tired of reading this kind of complaints on OSS lists. and because it's free everybody has to shut up? that's your defintion of free? your definition is broken? as said on a other list: if the developer of the OSS sais "listen, i am not that interested but if you pay me ? xyz i would include it" the chances are good that one or more people sponsor it - ignore or complain about feature requests don't help that mich -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From skdovecot at smail.inf.fh-brs.de Mon Mar 2 09:06:41 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 2 Mar 2015 10:06:41 +0100 (CET) Subject: IP drop list In-Reply-To: <54F41E1D.5030901@neurotica.com> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> <54F38FBE.3040108@neurotica.com> <54F4137F.8050507@oliwel.de> <54F41E1D.5030901@neurotica.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 2 Mar 2015, Dave McGuire wrote: > On 03/02/2015 02:38 AM, Oliver Welter wrote: >> Guys, dovecot is open source - if you desire a feature that the upstream >> programmer did not include, pay him a bounty to do so or send him a >> patch to be included. Period. We can discuss and mightbe somebody will >> fork if he is not willing to accept such a solutuion for any political >> reason. >> >> I am really tired of reading this kind of complaints on OSS lists. > > ....and this is perhaps the second most predictable knee-jerk response. > > I am certainly capable of writing such a patch, but there is no point > in expending the effort if it would not be included in the code base. > The extreme negative reactions to this idea from people in this > community, every time it has come up over the years, with almost rabid Neither Timo nor dovecot.fi did responded with "use fail2ban", if I remember correctly. I actually wonder, why nobody replied with: "this is what tcpwrapper is for" :-) http://wiki2.dovecot.org/LoginProcess?highlight=%28tcp+wrapper%29 what had been ruled out by the OP with a conditional *if*. If you for instance add a passdb{} driver, that does not interfere with the remaining code base (much), so one can use: passdb { driver = ipdeny args = /matchpattern/action .... *** } in front of any other passdb{}. *** some sort of notation to configure IP source, matching and reaction. If such plugin(?) is available, I would expect immediate complains, it does not support: + local file lists with various sets of syntaxes + RBLs with a fine grained response matching + use the same RBL response for multiple match-action pairs + have it depended on protocol (POP3, IMAP, ManageSieve, ...) + have it depended on user (use that passdb for all-but or just-these) + have it to kick in after certain user-protocol-count-time patterns only There is this, too: http://article.gmane.org/gmane.mail.imap.dovecot/61570 http://article.gmane.org/gmane.mail.imap.dovecot/42512 Maybe an addition to the penalty service would be OK as well. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVPQoIXz1H7kL/d9rAQLHWwgAs+8TAw7i3qerJQHXD4GSDO0jPCDtqGg3 660CMHCilWNYP+AwM/wxRbBkhz6rtTZrMa3BjLlHo3jnc/kNnJu8YdPCiolQCiWX enU5576oeCikWcAQG/BJxrRTCtHVjzhenu/skCazD8vKncIUlJtn+kiAqpGC3NPe IAJg2FvZ0wgI+bzecZHFktVT8TF0JWtd8FNkD83rOJvNUW7ECrzyAMSUKQ+X54GH 6vcto6eeERY3DKpf/xUs1QBM/Pee1gdMTFU4clW2u9QZLf1aKuNaEVBAx4BaI5Ti hzL/UIXZ0+qHehxNCIyTFx0t4MZsPfJg9/dS3t2vmX9efSUFxe9bgg== =XjPT -----END PGP SIGNATURE----- From felix.schueren at heg.com Sun Mar 1 08:49:54 2015 From: felix.schueren at heg.com (=?windows-1252?Q?Felix_Sch=FCren?=) Date: Sun, 01 Mar 2015 09:49:54 +0100 Subject: IP drop list In-Reply-To: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> Message-ID: <54F2D2B2.4020908@heg.com> Hi Jim, you may want to simply try ipset. :) http://ipset.netfilter.org/ http://daemonkeeper.net/781/mass-blocking-ip-addresses-with-ipset/ Kind regards, Felix On 01.03.15 08:53, Jim Pazarena wrote: > I wonder if there is an easy way to provide dovecot a flat text file of > ipv4 #'s which should be ignored or dropped? > > I have accumulated 45,000+ IPs which routinely try dictionary and > 12345678 password attempts. The file is too big to create firewall > drops, and I don't want to compile with wrappers *if* dovecot has an > easy ability to do this. If dovecot could parse a flat text file of IPs > and drop connections it would sure put a dent in these attempts. > > Thanks. > -- Felix Sch?ren Group Director of Enterprise Architecture, HEG. Online: http://www.heg.com/ This email is subject to: http://www.heg.com/disclaimer. Please consider the environment before printing this email From h.reindl at thelounge.net Mon Mar 2 09:11:20 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 02 Mar 2015 10:11:20 +0100 Subject: IP drop list In-Reply-To: References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> <54F38FBE.3040108@neurotica.com> <54F4137F.8050507@oliwel.de> <54F41E1D.5030901@neurotica.com> Message-ID: <54F42938.7000000@thelounge.net> Am 02.03.2015 um 10:06 schrieb Steffen Kaiser: > If such plugin(?) is available, I would expect immediate complains, it > does not support: > > + local file lists with various sets of syntaxes > + RBLs with a fine grained response matching > + use the same RBL response for multiple match-action pairs or it could work just with no config, unconditional and in front of any authentication, frankly even without any response - connection -> RBL check -> close connection, done hence RBL's make sense in the core because *in front* of any other protocol specific code -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From tobster at brain-force.ch Mon Mar 2 09:14:36 2015 From: tobster at brain-force.ch (Tobi) Date: Mon, 02 Mar 2015 10:14:36 +0100 Subject: IP drop list In-Reply-To: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> Message-ID: <54F429FC.6010105@brain-force.ch> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Am 01.03.2015 um 08:53 schrieb Jim Pazarena: > I have accumulated 45,000+ IPs which routinely try dictionary and > 12345678 password attempts. The file is too big to create firewall > drops, Have you also checked ipset (http://ipset.netfilter.org/) Its extremely powerful even with huge block lists -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJU9Cn8AAoJEDUc5iWoaKTk3ToQAItYxio2z7BiGjpGD2KOztkQ LvD1yLoJyO2LQqM+8ItT7lFC1tXMfwxs1pMS0983f0H2r4k4w5DFaMtu6Nw1LWyD OTHvxpnkA95b/APn+02GDdXUTVdR9gdk7CWefm4undsuR20QX5b9xm7GmvYJL9Zl n8FyfedQBO1FaiaUEOLmXAJ/oNCx3XzNa4oHVNtV0F2uckAtHzQ+jTcjwgLPYiUm m48MQyYEk9BdXGYS0790zfYWUvfTymxGGBjiALlVRXA9k445OAsv0/PppvTBxH+S 4a3yF6CXh5vfb7bYSdcBhZz7nI5AnSDuFYKMSl+5VIMxFafLxN3N28TD5w7FAu12 ubpSMj52N8UO8axcFOoVuBi4o1fPoPODf46ztfKb5tC5inhpdnxEba1tExR4Eitn WHWu1y3HA9qUoZpG9iA97/bltQqqo0ZPw3run+j8HfR0eVkfBXogbahXxcWx7voq pnvDnL0HA6RUjA9d0wRmHpNvBfSxxzlcFaxV1uacoiZhcJcURilJgpufx0V0mys9 d+MOKVQ/4nxm4Rb2gXQXbaQiBr1TXMJNRRHFnox/lmuCRornHHVf3zDiCh5lM4vQ vnEO2qpVYfqBggTHeIQxC4rdfvmhcKZ3qtngmsQldXafph++n0mGIsu8Vkt7H4Zj 9inl3Wo4Mh84X0NEhZbj =lPnB -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Mon Mar 2 09:17:15 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 2 Mar 2015 10:17:15 +0100 (CET) Subject: Postfix - dovecot-lda -> Permission denied In-Reply-To: <497ED10F0C1FA8488397AB0162B18D330B209068@EX1.neos-it.local> References: <497ED10F0C1FA8488397AB0162B18D330B209068@EX1.neos-it.local> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 27 Feb 2015, W?ltje, Marcus wrote: > I'm trying to run Postix and Dovecot on a Mac OS X, but somehow, it doesn't work. I probably messed up the privileges or something else. Hopefully someone of you, can help me figure it out. > So far, Postfix is delivering the mail into ~/Maildir. But as soon as I add the line: > > mailbox_command = /Applications/Server.app/Contents/ServerRoot/usr/libexec/dovecot/dovecot-lda > > in /etc/postfix/main.cf > > I get the following entry in the mail.log. > > Feb 27 16:08:02 Nils-iMac.local local[53237]: fatal: execvp /Applications/Server.app/Contents/ServerRoot/usr/libexec/dovecot/dovecot-lda: Permission denied > Feb 27 16:08:02 Nils-iMac.local postfix/local[53236]: B246837BE40: to=, orig_to=, relay=local, delay=1185, delays=1185/0.01/0/0.01, dsn=4.3.0, status=deferred (temporary failure. Command output: local: fatal: execvp /Applications/Server.app/Contents/ServerRoot/usr/libexec/dovecot/dovecot-lda: Permission denied ) > > Here are the rights of dovecot-lda: > > -rwxrwx--- 1 root certusers 32144 Dec 5 04:41 /Applications/Server.app/Contents/ServerRoot/usr/libexec/dovecot/dovecot-lda ^^^^^^^^^^ Only root and users in group certusers may exec the program, as what user postfix tries to run the LDA? Maybe check any directory in path as well. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVPQqm3z1H7kL/d9rAQLpggf/bYcA4tPbo2lChmsuidjXGVp+li1mbQdI enSgxD2dZP/im4Jk9djGH6uPMaPaCUwiRrjR9xIVtMkEv8o0XqgGn3ba4imjOt3t YxZhEx8l8cQQYu/54ATZf8JgaqFhxGxdFGebd5JpR9P1U36y7ZUdH3ukJ+9Yzz9W J2loRSj2+Lvqi6yE4Tcg7HLvdQlM3vycS/9l8pokd+uH3PtiOILHe8Q9wM61CHRv pNlwK/GL8fyBCs8nr1AEd2nwrXx1h4B338lWqfGCTSLTxZoWsLMQCo51BCEITDWQ pzIic7OS1gkxi0DMJ6bQOhoOgQleXu6CE8eBEeDvkKPqrrkEFHERBw== =8Z7u -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Mon Mar 2 09:33:48 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 2 Mar 2015 10:33:48 +0100 (CET) Subject: IP drop list In-Reply-To: <54F42938.7000000@thelounge.net> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> <54F38FBE.3040108@neurotica.com> <54F4137F.8050507@oliwel.de> <54F41E1D.5030901@neurotica.com> <54F42938.7000000@thelounge.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 2 Mar 2015, Reindl Harald wrote: > Am 02.03.2015 um 10:06 schrieb Steffen Kaiser: >> If such plugin(?) is available, I would expect immediate complains, it >> does not support: >> >> + local file lists with various sets of syntaxes >> + RBLs with a fine grained response matching >> + use the same RBL response for multiple match-action pairs > > or it could work just with no config, unconditional and therefore I wrote, that I expect complains, if this feature would work like that > in front of any > authentication, what is that same as to place it as first passdb, with the overhead of parsing the config file and adding it into the passdb{} chain. > frankly even without any response - connection -> RBL check > -> close connection, done some external RBLs return certain information in the response, e.g. 127.0.0.2 is less problematic than 127.0.0.1, so "I expect complains" this or that RBL is not working correctly ;-) > hence RBL's make sense in the core because *in front* of any other protocol > specific code That's TCP wrapper or a firewall, IMHO. (for a file list, not RBL). However, there used to be a RBL patch for TCP wrapper and some distribution provide other implementations of a TCP wrapper with RBL, if this post correct: http://grokbase.com/t/centos/centos/143mg1wxsj/does-anyone-use-tcp-wrappers-hosts-allow-hosts-deny-anymore - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVPQufHz1H7kL/d9rAQKC3wf/ZuStrHInsV3OkgDC5EDBeSyvMOxlskiy xCNUeAxaqPt4DvgCHnXmXX3V2yi+hXvsFyWhIBcsJcgUvbi0sJWwy7Undw2Fs6Cf iaOD3+u1VV+7IwiiZIMNMpUcDisj9Ic3DBoDTx9SeyBS09i7lKAVORZw486LooWX uTCMZOEmzH43DEfHxmIMPMcyQBF4b7kzc3A/sabpc70bhrJAV8E2ZNpPzIyAiC3A PwjUR+YfdYoorqz79ymmzcngsUUSAXfiUAhJpRyVOL2UiMurjROdsU5vSpXJm71j lgELgKpo6DkIjX+qAPVtdPu/J6cRLUcfvysNezU2vV9KpgJk97cwmw== =2nvt -----END PGP SIGNATURE----- From h.reindl at thelounge.net Mon Mar 2 09:40:25 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 02 Mar 2015 10:40:25 +0100 Subject: IP drop list In-Reply-To: References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> <54F38FBE.3040108@neurotica.com> <54F4137F.8050507@oliwel.de> <54F41E1D.5030901@neurotica.com> <54F42938.7000000@thelounge.net> Message-ID: <54F43009.1040400@thelounge.net> Am 02.03.2015 um 10:33 schrieb Steffen Kaiser: >> hence RBL's make sense in the core because *in front* of any other >> protocol specific code > > That's TCP wrapper or a firewall, IMHO. (for a file list, not RBL). > However, there used to be a RBL patch for TCP wrapper and some > distribution provide other implementations of a TCP wrapper with RBL TCP wrapper is dying (more and more software in distributions is built without tcpwrapper support, more and more upstream packages remove support starting with openssh) and given that the author of tcpwrapper is the same person which wrote postfix if it would not make sense in the mail-daemon itself you can be sure it would not be in postfix one point is logging - frankly i want rejected mail connections in the maillog and not spread over the whole system logs EADSUP: OpenSSH 6.7 drops tcpwrapper support: https://www.cygwin.com/ml/cygwin/2014-08/msg00345.html https://rwmj.wordpress.com/tag/tcp-wrappers/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From me at junc.eu Mon Mar 2 09:56:56 2015 From: me at junc.eu (Benny Pedersen) Date: Mon, 02 Mar 2015 10:56:56 +0100 Subject: IP drop list In-Reply-To: References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> <54F38FBE.3040108@neurotica.com> <14bd7b14530.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> <54F41C50.308@neurotica.com> Message-ID: <14bd9eac240.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> On March 2, 2015 9:28:16 AM Steffen Kaiser wrote: > Does allownets support negative CIDRs? if order of ips is done in listed order imho yes Example:?allow_nets=127.0.0.0/8,192.168.0.0/16,!1.2.3.4,4.5.6.7 deny 1.2.3.4 but allow all others listed pr user this does not work with pam pr user, but allownets is genric pr login user if fields are in auth db From Jochen.Bern at LINworks.de Mon Mar 2 10:02:49 2015 From: Jochen.Bern at LINworks.de (Jochen Bern) Date: Mon, 02 Mar 2015 11:02:49 +0100 Subject: IP drop list In-Reply-To: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> Message-ID: <54F43549.1030803@LINworks.de> On 03/01/2015 08:53 AM, Jim Pazarena wrote: > I wonder if there is an easy way to provide dovecot a flat text file of > ipv4 #'s which should be ignored or dropped? > > I have accumulated 45,000+ IPs which routinely try dictionary and > 12345678 password attempts. The file is too big to create firewall > drops [...] The inherent assumption here is that dovecot, using a "flat file", will be able to process the block list more effectively than the firewall, which is a tool written for the *purpose* but supposedly unable to even *try* due to the list's size. That sounds ... counterintuitive. To clarify, the governing influence on performance of *most* firewalls is the average number of rules a packet has to be matched against, and the two main tools to help with that are (if I may use iptables lingo here) a) --state ESTABLISHED to get everything but the connection-initiating packets out of the way ASAP and b) branching tree-like into dedicated-purpose subchains, rather than building linear lists. Assuming that the IPs to be blocked are randomly distributed, I'ld try something along the following lines: [main chain] --state ESTABLISHED,RELATED -j ACCEPT -p tcp --dport pop3 -j dove-blocks -p tcp --dport imap -j dove-blocks [subchain dove-blocks] -d 1.0.0.0/8 -j sub-1 -d 2.0.0.0/8 -j sub-2 ... -d 254.0.0.0/8 -j sub-254 [subchain sub-1] -d 1.2.0.0/16 -j sub-1-2 # We've seen 1.2.3.4 and 1.2.2.1 ... [subchain sub-1-2] -d 1.2.2.1 -j DROP -d 1.2.3.4 -j DROP Regards, J. Bern -- *NEU* - NEC IT-Infrastruktur-Produkte im : Server--Storage--Virtualisierung--Management SW--Passion for Performance Jochen Bern, Systemingenieur --- LINworks GmbH Postfach 100121, 64201 Darmstadt | Robert-Koch-Str. 9, 64331 Weiterstadt PGP (1024D/4096g) FP = D18B 41B1 16C0 11BA 7F8C DCF7 E1D5 FAF4 444E 1C27 Tel. +49 6151 9067-231, Zentr. -0, Fax -299 - Amtsg. Darmstadt HRB 85202 Unternehmenssitz Weiterstadt, Gesch?ftsf?hrer Metin Dogan, Oliver Michel From h.reindl at thelounge.net Mon Mar 2 10:07:12 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 02 Mar 2015 11:07:12 +0100 Subject: IP drop list In-Reply-To: <54F43549.1030803@LINworks.de> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F43549.1030803@LINworks.de> Message-ID: <54F43650.5050200@thelounge.net> Am 02.03.2015 um 11:02 schrieb Jochen Bern: > On 03/01/2015 08:53 AM, Jim Pazarena wrote: >> I wonder if there is an easy way to provide dovecot a flat text file of >> ipv4 #'s which should be ignored or dropped? >> >> I have accumulated 45,000+ IPs which routinely try dictionary and >> 12345678 password attempts. The file is too big to create firewall >> drops [...] > > The inherent assumption here is that dovecot, using a "flat file", will > be able to process the block list more effectively than the firewall, > which is a tool written for the *purpose* but supposedly unable to even > *try* due to the list's size. That sounds ... counterintuitive * it's unmaintainable on firewall level * it's waste of ressources because it is *packet based* * hence a RBL would make so much more sense for rbldnsd it don't matter if 100, 1000, 10000, 10000000 addresses or even cidr-ranges are listed because the check is always *one* cheap dns request for the IP conencting at the moment -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From me at junc.eu Mon Mar 2 10:10:55 2015 From: me at junc.eu (Benny Pedersen) Date: Mon, 02 Mar 2015 11:10:55 +0100 Subject: IP drop list In-Reply-To: <54F429FC.6010105@brain-force.ch> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F429FC.6010105@brain-force.ch> Message-ID: <14bd9f78bb0.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> On March 2, 2015 10:15:22 AM Tobi wrote: > > I have accumulated 45,000+ IPs which routinely try dictionary and > > 12345678 password attempts. The file is too big to create firewall > > drops, > Have you also checked ipset (http://ipset.netfilter.org/) > Its extremely powerful even with huge block lists this is only usefull if real user have more then +45000 ips, and it why its not denynets in dovecot using xtables geoip here, and could let fail2ban create xtable csv datafile that can be included in xtable build, then just use geoip firewall rule to allow in all other ips if thats the goal of allow many ips default but i just default allow pr user country, all other is denyed connection From david.scheele2 at googlemail.com Mon Mar 2 10:14:03 2015 From: david.scheele2 at googlemail.com (David Scheele) Date: Mon, 2 Mar 2015 11:14:03 +0100 Subject: Dovecot & LDAP Take #2: Authentication failed and logging In-Reply-To: <624736716.726284.1425049259608.JavaMail.open-xchange@comunica.csi.it> References: <624736716.726284.1425049259608.JavaMail.open-xchange@comunica.csi.it> Message-ID: Ok I played around a bit and activated debugging correctly (Thanks to Steffen) Now I try to log in with the user johndoe (that is his cn and his uid) and i get the following message in syslog: Mar 2 11:03:32 mailserver dovecot: auth: Debug: master in: REQUEST#0111283457025#0117428#0111#011d139b5d372d882643bc995003c615c89 Mar 2 11:03:32 mailserver dovecot: auth: Debug: ldap(johndoe,127.0.0.1,): user search: base=ou=People,dc=[domainname],dc=de scope=subtree filter=(&(objectClass=inetOrgPerson)(cn=johndoe)) fields=uidNumber Mar 2 11:03:32 mailserver slapd[2465]: <= bdb_equality_candidates: (cn) not indexed Mar 2 11:03:32 mailserver dovecot: auth: Debug: ldap(johndoe,127.0.0.1,): result: uidNumber missing Mar 2 11:03:32 mailserver dovecot: auth: Debug: master out: USER#0111283457025#011johndoe Mar 2 11:03:32 mailserver dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=7450, secured, session= Mar 2 11:03:32 mailserver dovecot: imap(johndoe): Error: user johndoe: Couldn't drop privileges: User is missing UID (see mail_uid setting) Mar 2 11:03:32 mailserver dovecot: imap(johndoe): Error: Internal error occurred. Refer to server log for more information. I am confused what the line Mar 2 11:03:32 mailserver dovecot: imap(johndoe): Error: user johndoe: Couldn't drop privileges: User is missing UID (see mail_uid setting) is trying to tell me. doveconf -n: # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8 ext4 auth_debug = yes auth_mechanisms = plain login auth_verbose = yes default_login_user = vmail disable_plaintext_auth = no first_valid_gid = 2222 first_valid_uid = 2222 listen = * mail_access_groups = vmail mail_debug = yes mail_location = maildir:/var/vmail/%n passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } service imap-login { process_min_avail = 1 user = vmail } ssl = no userdb { args = /etc/dovecot/dovecot-ldap-userdb.conf.ext driver = ldap } grep -v '^ *\(#.*\)\?$' dovecot-ldap.conf.ext : hosts = mailserver.[domainname].de debug_level = 0 auth_bind = yes auth_bind_userdn = cn=%u,ou=People,dc=[domainname],dc=de base = ou=People,dc=[domainname],dc=de user_attrs = uidNumber=uid user_filter = (&(objectClass=inetOrgPerson)(cn=%u)) pass_attrs = userPassword=password pass_filter = (&(objectClass=inetOrgPerson)(uid=%u)) iterate_attrs = uid=user iterate_filter = (objectClass=inetOrgPerson) 2015-02-27 16:00 GMT+01:00 Paolo Cravero : > > This is the user DN: > > > cn=Klara Fall,ou=People,dc=[domainname],dc=de > > > According to your Dovecot configuration > > > auth_bind_userdn = cn=%u,ou=People,dc=**[domainname]**,dc=de > > if you login with "klarafall" it will be expanded into > > cn=klarafall,ou=People,dc=[domainname],dc=de > > which is not the correct DN for Mrs Klara. > > So if you login with "Klara Fall" it should work, but that will probably > mess up the things on Dovecot filesystem. > > > I am strongly against setting a static DN when dealing with LDAP > authentication. LDAP servers are optimized to serve search requests, so let > yours do the job. Allow Dovecot to lookup the correct DN based on the > attribute you supply (uid) and then authenticate. > > This should be achieved if you comment out the auth_bind_userdn line. > > Paolo Cravero > From jtam.home at gmail.com Mon Mar 2 10:34:26 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 2 Mar 2015 02:34:26 -0800 (PST) Subject: IP drop list In-Reply-To: References: Message-ID: Dave McGuire writes: >> http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/AllowNets >> >> then setup fail2ban to manage extrafields > > Now that's a very interesting idea, thank you! I will investigate this. If you don't expect yor firewall to handle 45K+ IPs, I'm not how you expect dovecot will handle a comma separated string with 45K+ entries any better. If you want to turn your global backlist into a per-user whitelist, that would be perfectly doable though. Joseph Tam From jean-francois.senechal at ac.marche.be Mon Mar 2 10:43:44 2015 From: jean-francois.senechal at ac.marche.be (=?UTF-8?B?SmVhbi1GcmFuw6dvaXMgU8OpbsOpY2hhbA==?=) Date: Mon, 02 Mar 2015 11:43:44 +0100 Subject: Quota and ldap Message-ID: <54F43EE0.3020506@ac.marche.be> Hello I try to install quota per usr, everything seems fine but the quota is not working Dovecot : 2.2.13 dovecot-ldap.conf.ext user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid,gosaMailQuota=quota_rule=*:bytes=%$ 10-mail.conf mail_location = maildir:~/Maildir:LAYOUT=fs mail_plugins = $mail_plugins quota /10-master.conf service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } 20-imapd.conf protocol imap { mail_plugins = $mail_plugins imap_quota } 20-lmtp.conf protocol lmtp { mail_plugins = $mail_plugins quota } 90-quota.conf plugin { quota_rule = *:storage=150M quota_rule2 = Trash:storage=+100M quota_grace = 10%% } plugin { quota = maildir:User quota } Command doveadm quota recalc return nothing (no error) My log Mar 2 11:41:58 domaine dovecot: imap-login: Login: user=, method=PLAIN, rip=::1, lip=::1, mpid=3695, secured, session= Mar 2 11:41:58 domaine dovecot: imap: Debug: Loading modules from directory: /usr/lib/dovecot/modules Mar 2 11:41:58 domaine dovecot: imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Mar 2 11:41:58 domaine dovecot: imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so Mar 2 11:41:58 domaine dovecot: imap: Debug: Added userdb setting: plugin/quota_rule=*:bytes=0 Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: Effective uid=5000, gid=5000, home=/var/spool/dovecot/mail/s/siroco Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: Quota root: name=User quota backend=maildir args= Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: Quota rule: root=User quota mailbox=* bytes=0 messages=0 Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: Quota rule: root=User quota mailbox=Trash bytes=+104857600 messages=0 Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: Quota grace: root=User quota bytes=0 (10%) Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir:LAYOUT=fs Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: fs: root=/var/spool/dovecot/mail/s/siroco/Maildir, index=, indexpvt=, control=, inbox=/var/spool/dovecot/mail/s/siroco/Maildir, From mihai at badici.ro Mon Mar 2 10:53:02 2015 From: mihai at badici.ro (Mihai Badici) Date: Mon, 02 Mar 2015 12:53:02 +0200 Subject: Dovecot & LDAP Take #2: Authentication failed and logging In-Reply-To: References: <624736716.726284.1425049259608.JavaMail.open-xchange@comunica.csi.it> Message-ID: <1974727.gzJ3lm3Q4u@arhivio> On Monday 02 March 2015 11:14:03 David Scheele wrote: > Ok I played around a bit and activated debugging correctly (Thanks to > Steffen) > > > Now I try to log in with the user johndoe (that is his cn and his uid) and > i get the following message in syslog: > Mar 2 11:03:32 mailserver dovecot: auth: Debug: master in: > REQUEST#0111283457025#0117428#0111#011d139b5d372d882643bc995003c615c89 > Mar 2 11:03:32 mailserver dovecot: auth: Debug: > ldap(johndoe,127.0.0.1,): user search: > base=ou=People,dc=[domainname],dc=de scope=subtree > filter=(&(objectClass=inetOrgPerson)(cn=johndoe)) fields=uidNumber > Mar 2 11:03:32 mailserver slapd[2465]: <= bdb_equality_candidates: (cn) > not indexed > Mar 2 11:03:32 mailserver dovecot: auth: Debug: > ldap(johndoe,127.0.0.1,): result: uidNumber missing There are two strategies: put the uid of each user in ldap or use the same uid for all accounts. for the second choice, you need to put something like mail_uid = 10000 mail_gid = 10000 in 10-mail.conf This user need some rights on dovecot storage folder. When using the first choice, you will need a mechanism to generate those uid's ( this should be implemented in the ldap management tool) > Mar 2 11:03:32 mailserver dovecot: auth: Debug: master out: > USER#0111283457025#011johndoe > Mar 2 11:03:32 mailserver dovecot: imap-login: Login: user=, > method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=7450, secured, > session= > Mar 2 11:03:32 mailserver dovecot: imap(johndoe): Error: user johndoe: > Couldn't drop privileges: User is missing UID (see mail_uid setting) > Mar 2 11:03:32 mailserver dovecot: imap(johndoe): Error: Internal error > occurred. Refer to server log for more information. > > > I am confused what the line Mar 2 11:03:32 mailserver dovecot: > imap(johndoe): Error: user johndoe: Couldn't drop privileges: User is > missing UID (see mail_uid setting) is trying to tell me. > > doveconf -n: > > # 2.1.7: /etc/dovecot/dovecot.conf > # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8 ext4 > auth_debug = yes > auth_mechanisms = plain login > auth_verbose = yes > default_login_user = vmail > disable_plaintext_auth = no > first_valid_gid = 2222 > first_valid_uid = 2222 > listen = * > mail_access_groups = vmail > mail_debug = yes > mail_location = maildir:/var/vmail/%n > passdb { > args = /etc/dovecot/dovecot-ldap.conf.ext > driver = ldap > } > protocols = imap > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > user = root > } > service imap-login { > process_min_avail = 1 > user = vmail > } > ssl = no > userdb { > args = /etc/dovecot/dovecot-ldap-userdb.conf.ext > driver = ldap > } > grep -v '^ *\(#.*\)\?$' dovecot-ldap.conf.ext : > > hosts = mailserver.[domainname].de > debug_level = 0 > auth_bind = yes > auth_bind_userdn = cn=%u,ou=People,dc=[domainname],dc=de > base = ou=People,dc=[domainname],dc=de > user_attrs = uidNumber=uid > user_filter = (&(objectClass=inetOrgPerson)(cn=%u)) > pass_attrs = userPassword=password > pass_filter = (&(objectClass=inetOrgPerson)(uid=%u)) > iterate_attrs = uid=user > iterate_filter = (objectClass=inetOrgPerson) > > 2015-02-27 16:00 GMT+01:00 Paolo Cravero : > > This is the user DN: > > > cn=Klara Fall,ou=People,dc=[domainname],dc=de > > > > According to your Dovecot configuration > > > > > auth_bind_userdn = cn=%u,ou=People,dc=**[domainname]**,dc=de > > > > if you login with "klarafall" it will be expanded into > > > > cn=klarafall,ou=People,dc=[domainname],dc=de > > > > which is not the correct DN for Mrs Klara. > > > > So if you login with "Klara Fall" it should work, but that will probably > > mess up the things on Dovecot filesystem. > > > > > > I am strongly against setting a static DN when dealing with LDAP > > authentication. LDAP servers are optimized to serve search requests, so > > let > > yours do the job. Allow Dovecot to lookup the correct DN based on the > > attribute you supply (uid) and then authenticate. > > > > This should be achieved if you comment out the auth_bind_userdn line. > > > > Paolo Cravero -- Mihai B?dici http://mihai.badici.ro From Christian.Schmidt at chemie.uni-hamburg.de Mon Mar 2 11:12:46 2015 From: Christian.Schmidt at chemie.uni-hamburg.de (Christian Schmidt) Date: Mon, 02 Mar 2015 12:12:46 +0100 Subject: Postfix - dovecot-lda -> Permission denied In-Reply-To: <497ED10F0C1FA8488397AB0162B18D330B209068@EX1.neos-it.local> References: <497ED10F0C1FA8488397AB0162B18D330B209068@EX1.neos-it.local> Message-ID: <54F445AE.6000409@chemie.uni-hamburg.de> On 27.02.2015 16:16, W?ltje, Marcus wrote: > So far, Postfix is delivering the mail into ~/Maildir. But as soon as I add the line: > mailbox_command = /Applications/Server.app/Contents/ServerRoot/usr/libexec/dovecot/dovecot-lda > in /etc/postfix/main.cf > I get the following entry in the mail.log. Why don't you let postfix hand over the mail data to dovecot using lmtp? Regards, Christian -- No signature available. From skdovecot at smail.inf.fh-brs.de Mon Mar 2 11:17:02 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 2 Mar 2015 12:17:02 +0100 (CET) Subject: Quota and ldap In-Reply-To: <54F43EE0.3020506@ac.marche.be> References: <54F43EE0.3020506@ac.marche.be> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: please post output of dovecont -n > > user_attrs = > homeDirectory=home,uidNumber=uid,gidNumber=gid,gosaMailQuota=quota_rule=*:bytes=%$ > > 10-mail.conf > > mail_location = maildir:~/Maildir:LAYOUT=fs > mail_plugins = $mail_plugins quota > > /10-master.conf > > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0600 > user = postfix > } > > > 20-imapd.conf > > protocol imap { > mail_plugins = $mail_plugins imap_quota > } > > > 20-lmtp.conf > > protocol lmtp { > mail_plugins = $mail_plugins quota > } This looks like quota is added twice. > 90-quota.conf > > plugin { > quota_rule = *:storage=150M > quota_rule2 = Trash:storage=+100M > quota_grace = 10%% > } > > plugin { > quota = maildir:User quota > } > > Command doveadm quota recalc > return nothing (no error) you need -u user or -A also try: doveadm quota get -u siroco > My log > > Mar 2 11:41:58 domaine dovecot: imap-login: Login: user=, > method=PLAIN, rip=::1, lip=::1, mpid=3695, secured, > session= > Mar 2 11:41:58 domaine dovecot: imap: Debug: Loading modules from directory: > /usr/lib/dovecot/modules > Mar 2 11:41:58 domaine dovecot: imap: Debug: Module loaded: > /usr/lib/dovecot/modules/lib10_quota_plugin.so > Mar 2 11:41:58 domaine dovecot: imap: Debug: Module loaded: > /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so > Mar 2 11:41:58 domaine dovecot: imap: Debug: Added userdb setting: > plugin/quota_rule=*:bytes=0 > Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: Effective uid=5000, > gid=5000, home=/var/spool/dovecot/mail/s/siroco > Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: Quota root: name=User > quota backend=maildir args= > Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: Quota rule: root=User > quota mailbox=* bytes=0 messages=0 > Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: Quota rule: root=User > quota mailbox=Trash bytes=+104857600 messages=0 > Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: Quota grace: root=User > quota bytes=0 (10%) > Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: Namespace inbox: > type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, > subscriptions=yes location=maildir:~/Maildir:LAYOUT=fs > Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: fs: > root=/var/spool/dovecot/mail/s/siroco/Maildir, index=, indexpvt=, control=, > inbox=/var/spool/dovecot/mail/s/siroco/Maildir, > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVPRGrnz1H7kL/d9rAQJC8wf+N74Pc8y4LsgrLtIEWvb7jkwbhbYIpijN lmZN2ZGZroDbfDB6ieXPP6lSz7c6dUfDmVDDlq1R280FzXRfyEU08jdxs4G3/NuM IA9kw2IPpyZdk4sKDDA2rNYxN7SJa+v8V3ab4TFP7LFEsLDZJlZVwo6+ugcjWhSe DErBwSKTsqKUnBwStQZyHow5hV8PhSDI3Dj4Mp2L3WncpUPsEIFeB/6PKDphnnN7 nFzADtS67rLDR1KMGvVv/7RKysbs9a7IhdN3YDywWfgoDlSEFNIhM4DFNBiPvm/P jhOGq25KpPnm2mI5MumNnZW4UlJN1KpEc6iko2XA4ABaWhQgE1FOLg== =91Zl -----END PGP SIGNATURE----- From jean-francois.senechal at ac.marche.be Mon Mar 2 11:21:19 2015 From: jean-francois.senechal at ac.marche.be (=?UTF-8?B?SmVhbi1GcmFuw6dvaXMgU8OpbsOpY2hhbA==?=) Date: Mon, 02 Mar 2015 12:21:19 +0100 Subject: Quota and ldap In-Reply-To: References: <54F43EE0.3020506@ac.marche.be> Message-ID: <54F447AF.7050800@ac.marche.be> doveadm quota get -u siroco give Quota name Type Value Limit % User quota STORAGE 177014 - 0 User quota MESSAGE 2033 - 0 Le 02/03/15 12:17, Steffen Kaiser a ?crit : > doveadm quota get -u siroco From jean-francois.senechal at ac.marche.be Mon Mar 2 11:28:46 2015 From: jean-francois.senechal at ac.marche.be (=?UTF-8?B?SmVhbi1GcmFuw6dvaXMgU8OpbsOpY2hhbA==?=) Date: Mon, 02 Mar 2015 12:28:46 +0100 Subject: Quota and ldap In-Reply-To: References: <54F43EE0.3020506@ac.marche.be> Message-ID: <54F4496E.7060700@ac.marche.be> # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8 auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = yes imapc_features = rfc822.size fetch-headers imapc_host = citoyen.marche.be imapc_list_prefix = INBOX imapc_master_user = cyrus imapc_password = homer imapc_user = %u mail_debug = yes mail_location = maildir:~/Maildir:LAYOUT=fs mail_plugins = " quota" mail_prefetch_count = 20 maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { quota = maildir:User quota quota_grace = 10%% quota_rule = *:storage=150M quota_rule2 = Trash:storage=+100M sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = jf at marche.be protocols = imap lmtp service auth { unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl_ca = -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: > > please post output of dovecont -n > >> >> user_attrs = >> homeDirectory=home,uidNumber=uid,gidNumber=gid,gosaMailQuota=quota_rule=*:bytes=%$ >> >> 10-mail.conf >> >> mail_location = maildir:~/Maildir:LAYOUT=fs >> mail_plugins = $mail_plugins quota >> >> /10-master.conf >> >> service lmtp { >> unix_listener /var/spool/postfix/private/dovecot-lmtp { >> group = postfix >> mode = 0600 >> user = postfix >> } >> >> >> 20-imapd.conf >> >> protocol imap { >> mail_plugins = $mail_plugins imap_quota >> } >> >> >> 20-lmtp.conf >> >> protocol lmtp { >> mail_plugins = $mail_plugins quota >> } > > This looks like quota is added twice. > >> 90-quota.conf >> >> plugin { >> quota_rule = *:storage=150M >> quota_rule2 = Trash:storage=+100M >> quota_grace = 10%% >> } >> >> plugin { >> quota = maildir:User quota >> } >> >> Command doveadm quota recalc >> return nothing (no error) > > you need -u user or -A > > also try: > > doveadm quota get -u siroco > > >> My log >> >> Mar 2 11:41:58 domaine dovecot: imap-login: Login: user=, >> method=PLAIN, rip=::1, lip=::1, mpid=3695, secured, >> session= >> Mar 2 11:41:58 domaine dovecot: imap: Debug: Loading modules from >> directory: /usr/lib/dovecot/modules >> Mar 2 11:41:58 domaine dovecot: imap: Debug: Module loaded: >> /usr/lib/dovecot/modules/lib10_quota_plugin.so >> Mar 2 11:41:58 domaine dovecot: imap: Debug: Module loaded: >> /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so >> Mar 2 11:41:58 domaine dovecot: imap: Debug: Added userdb setting: >> plugin/quota_rule=*:bytes=0 >> Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: Effective >> uid=5000, gid=5000, home=/var/spool/dovecot/mail/s/siroco >> Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: Quota root: >> name=User quota backend=maildir args= >> Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: Quota rule: >> root=User quota mailbox=* bytes=0 messages=0 >> Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: Quota rule: >> root=User quota mailbox=Trash bytes=+104857600 messages=0 >> Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: Quota grace: >> root=User quota bytes=0 (10%) >> Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: Namespace >> inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, >> subscriptions=yes location=maildir:~/Maildir:LAYOUT=fs >> Mar 2 11:41:58 domaine dovecot: imap(siroco): Debug: fs: >> root=/var/spool/dovecot/mail/s/siroco/Maildir, index=, indexpvt=, >> control=, inbox=/var/spool/dovecot/mail/s/siroco/Maildir, >> > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVPRGrnz1H7kL/d9rAQJC8wf+N74Pc8y4LsgrLtIEWvb7jkwbhbYIpijN > lmZN2ZGZroDbfDB6ieXPP6lSz7c6dUfDmVDDlq1R280FzXRfyEU08jdxs4G3/NuM > IA9kw2IPpyZdk4sKDDA2rNYxN7SJa+v8V3ab4TFP7LFEsLDZJlZVwo6+ugcjWhSe > DErBwSKTsqKUnBwStQZyHow5hV8PhSDI3Dj4Mp2L3WncpUPsEIFeB/6PKDphnnN7 > nFzADtS67rLDR1KMGvVv/7RKysbs9a7IhdN3YDywWfgoDlSEFNIhM4DFNBiPvm/P > jhOGq25KpPnm2mI5MumNnZW4UlJN1KpEc6iko2XA4ABaWhQgE1FOLg== > =91Zl > -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Mon Mar 2 12:37:24 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 2 Mar 2015 13:37:24 +0100 (CET) Subject: Quota and ldap In-Reply-To: <54F4496E.7060700@ac.marche.be> References: <54F43EE0.3020506@ac.marche.be> <54F4496E.7060700@ac.marche.be> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: > protocol lmtp { > mail_plugins = " quota quota" ^^^^^^^^ it's added twice > > Le 02/03/15 12:17, Steffen Kaiser a ?crit : >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: >> >> please post output of dovecont -n >> >>> >>> user_attrs = >>> homeDirectory=home,uidNumber=uid,gidNumber=gid,gosaMailQuota=quota_rule=*:bytes=%$ >>> Mar 2 11:41:58 domaine dovecot: imap: Debug: Added userdb setting: >>> plugin/quota_rule=*:bytes=0 what's the content of siroco's LDAP entry? Esp. gosaMailQuota. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVPRZhHz1H7kL/d9rAQL+FwgAv6T9+fp4JeVdox9Ct3SbNMZUvU7lMLZO MBuZQCMgcov72VR6Kx/c5EGC7qXZA8sIW6PZNQOuAtEWEDJu9XCmy5JA3xBhFvwx ox8v08AUu2WVF0rxu+ABFdv0jvXlp1f2ditaI/45dfynbn2HHfVclliFk5mU9gii UhBJ8w22m0of82RLMIEgcViJbH8swJHjOxY+dx6mxkq2IgJkxo6ST3s7btaibYUH Rxxjqij4NhRKl3Cq/yft4ygQRNytAEk9k7lPrLfGt+hnxhv2L8SFBWdC/ozm8olW psEoCkPJz9ihcW0Wd/5aQg+8OeWSrSHYY+XpnxK12VmOruaOvl8crg== =+N1X -----END PGP SIGNATURE----- From jean-francois.senechal at ac.marche.be Mon Mar 2 13:01:23 2015 From: jean-francois.senechal at ac.marche.be (=?ISO-8859-15?Q?Jean-Fran=E7ois_S=E9n=E9chal?=) Date: Mon, 02 Mar 2015 14:01:23 +0100 Subject: Quota and ldap In-Reply-To: References: <54F43EE0.3020506@ac.marche.be> <54F4496E.7060700@ac.marche.be> Message-ID: <54F45F23.6020005@ac.marche.be> now dovecot -n give protocol lmtp { mail_plugins = " quota" postmaster_address = jf at marche.be } But nothing change in gosaMailQuota I set size in ko bytes gosaMailQuota : 100000 Le 02/03/15 13:37, Steffen Kaiser a ?crit : > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: > >> protocol lmtp { >> mail_plugins = " quota quota" > > ^^^^^^^^ > it's added twice > >> >> Le 02/03/15 12:17, Steffen Kaiser a ?crit : >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: >>> >>> please post output of dovecont -n >>> >>>> >>>> user_attrs = >>>> homeDirectory=home,uidNumber=uid,gidNumber=gid,gosaMailQuota=quota_rule=*:bytes=%$ > >>>> Mar 2 11:41:58 domaine dovecot: imap: Debug: Added userdb setting: >>>> plugin/quota_rule=*:bytes=0 > > what's the content of siroco's LDAP entry? Esp. gosaMailQuota. > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVPRZhHz1H7kL/d9rAQL+FwgAv6T9+fp4JeVdox9Ct3SbNMZUvU7lMLZO > MBuZQCMgcov72VR6Kx/c5EGC7qXZA8sIW6PZNQOuAtEWEDJu9XCmy5JA3xBhFvwx > ox8v08AUu2WVF0rxu+ABFdv0jvXlp1f2ditaI/45dfynbn2HHfVclliFk5mU9gii > UhBJ8w22m0of82RLMIEgcViJbH8swJHjOxY+dx6mxkq2IgJkxo6ST3s7btaibYUH > Rxxjqij4NhRKl3Cq/yft4ygQRNytAEk9k7lPrLfGt+hnxhv2L8SFBWdC/ozm8olW > psEoCkPJz9ihcW0Wd/5aQg+8OeWSrSHYY+XpnxK12VmOruaOvl8crg== > =+N1X > -----END PGP SIGNATURE----- From juan at inti.gob.ar Mon Mar 2 13:21:24 2015 From: juan at inti.gob.ar (Juan Bernhard) Date: Mon, 02 Mar 2015 10:21:24 -0300 Subject: Quota and ldap In-Reply-To: <54F45F23.6020005@ac.marche.be> References: <54F43EE0.3020506@ac.marche.be> <54F4496E.7060700@ac.marche.be> <54F45F23.6020005@ac.marche.be> Message-ID: <54F463D4.1070101@inti.gob.ar> El 02/03/2015 a las 10:01 a.m., Jean-Fran?ois S?n?chal escibi?: > now dovecot -n give > > protocol lmtp { > mail_plugins = " quota" > postmaster_address = jf at marche.be > } > > But nothing change > > in gosaMailQuota I set size in ko bytes > > gosaMailQuota : 100000 > Maildir controls qutoa using a file, in your case will be ~/Maildir/maildirzise. The first line on the file tell you the mailbox limits of sapace (in bytes) and files. Check of this is enforced or not. Maybe dovecot only reads the ldap quota value when this file is not present, because it may be modified by another program (an mta for example) > > Le 02/03/15 13:37, Steffen Kaiser a ?crit : > On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: > >>>> protocol lmtp { >>>> mail_plugins = " quota quota" > > ^^^^^^^^ > it's added twice > >>>> >>>> Le 02/03/15 12:17, Steffen Kaiser a ?crit : >>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>> Hash: SHA1 >>>>> >>>>> On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: >>>>> >>>>> please post output of dovecont -n >>>>> >>>>>> >>>>>> user_attrs = >>>>>> homeDirectory=home,uidNumber=uid,gidNumber=gid,gosaMailQuota=quota_rule=*:bytes=%$ >>>>>> > >>>>>> Mar 2 11:41:58 domaine dovecot: imap: Debug: Added userdb setting: >>>>>> plugin/quota_rule=*:bytes=0 > > what's the content of siroco's LDAP entry? Esp. gosaMailQuota. > > -- Steffen Kaiser From jean-francois.senechal at ac.marche.be Mon Mar 2 13:39:30 2015 From: jean-francois.senechal at ac.marche.be (=?ISO-8859-15?Q?Jean-Fran=E7ois_S=E9n=E9chal?=) Date: Mon, 02 Mar 2015 14:39:30 +0100 Subject: Quota and ldap In-Reply-To: <54F463D4.1070101@inti.gob.ar> References: <54F43EE0.3020506@ac.marche.be> <54F4496E.7060700@ac.marche.be> <54F45F23.6020005@ac.marche.be> <54F463D4.1070101@inti.gob.ar> Message-ID: <54F46812.6020405@ac.marche.be> Content maildirsize : cat Maildir/maildirsize 0S 181243879 2032 19202 1 14239 1 31954 1 Le 02/03/15 14:21, Juan Bernhard a ?crit : > El 02/03/2015 a las 10:01 a.m., Jean-Fran?ois S?n?chal escibi?: >> now dovecot -n give >> >> protocol lmtp { >> mail_plugins = " quota" >> postmaster_address = jf at marche.be >> } >> >> But nothing change >> >> in gosaMailQuota I set size in ko bytes >> >> gosaMailQuota : 100000 >> > Maildir controls qutoa using a file, in your case will be > ~/Maildir/maildirzise. The first line on the file tell you the mailbox > limits of sapace (in bytes) and files. Check of this is enforced or not. > Maybe dovecot only reads the ldap quota value when this file is not > present, because it may be modified by another program (an mta for example) > > >> Le 02/03/15 13:37, Steffen Kaiser a ?crit : >> On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: >> >>>>> protocol lmtp { >>>>> mail_plugins = " quota quota" >> ^^^^^^^^ >> it's added twice >> >>>>> Le 02/03/15 12:17, Steffen Kaiser a ?crit : >>>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>>> Hash: SHA1 >>>>>> >>>>>> On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: >>>>>> >>>>>> please post output of dovecont -n >>>>>> >>>>>>> user_attrs = >>>>>>> homeDirectory=home,uidNumber=uid,gidNumber=gid,gosaMailQuota=quota_rule=*:bytes=%$ >>>>>>> >>>>>>> Mar 2 11:41:58 domaine dovecot: imap: Debug: Added userdb setting: >>>>>>> plugin/quota_rule=*:bytes=0 >> what's the content of siroco's LDAP entry? Esp. gosaMailQuota. >> >> -- Steffen Kaiser From skdovecot at smail.inf.fh-brs.de Mon Mar 2 13:45:50 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 2 Mar 2015 14:45:50 +0100 (CET) Subject: Quota and ldap In-Reply-To: <54F46812.6020405@ac.marche.be> References: <54F43EE0.3020506@ac.marche.be> <54F4496E.7060700@ac.marche.be> <54F45F23.6020005@ac.marche.be> <54F463D4.1070101@inti.gob.ar> <54F46812.6020405@ac.marche.be> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: > Content maildirsize : > > cat Maildir/maildirsize > 0S ^^^^^^^^^ > 181243879 2032 > 19202 1 > 14239 1 > 31954 1 > > > > Le 02/03/15 14:21, Juan Bernhard a ?crit : >> El 02/03/2015 a las 10:01 a.m., Jean-Fran?ois S?n?chal escibi?: >>> now dovecot -n give >>> >>> protocol lmtp { >>> mail_plugins = " quota" >>> postmaster_address = jf at marche.be >>> } >>> >>> But nothing change >>> >>> in gosaMailQuota I set size in ko bytes >>> >>> gosaMailQuota : 100000 >>> >> Maildir controls qutoa using a file, in your case will be >> ~/Maildir/maildirzise. The first line on the file tell you the mailbox >> limits of sapace (in bytes) and files. Check of this is enforced or not. >> Maybe dovecot only reads the ldap quota value when this file is not >> present, because it may be modified by another program (an mta for example) >> >> >>> Le 02/03/15 13:37, Steffen Kaiser a ?crit : >>> On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: >>> >>>>>> protocol lmtp { >>>>>> mail_plugins = " quota quota" >>> ^^^^^^^^ >>> it's added twice >>> >>>>>> Le 02/03/15 12:17, Steffen Kaiser a ?crit : >>>>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>>>> Hash: SHA1 >>>>>>> >>>>>>> On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: >>>>>>> >>>>>>> please post output of dovecont -n >>>>>>> >>>>>>>> user_attrs = >>>>>>>> homeDirectory=home,uidNumber=uid,gidNumber=gid,gosaMailQuota=quota_rule=*:bytes=%$ >>>>>>>> >>>>>>>> Mar 2 11:41:58 domaine dovecot: imap: Debug: Added userdb setting: >>>>>>>> plugin/quota_rule=*:bytes=0 >>> what's the content of siroco's LDAP entry? Esp. gosaMailQuota. >>> >>> -- Steffen Kaiser > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVPRpjnz1H7kL/d9rAQKKTQf/VbOUb2kLKWJgvaFAgPY7ze51VhE1Vxg0 WbgZJ366aTDXhGozPSpEtfJlLF0gFYl6pK5+TQWv+AOdfGbcuhFQVZXIcclm5u8R O2nsf9/Q9JQT3tDDHQ78XCSAsBb+gVMYBxL66Gqi0qddbwAcTfMNUaXRGSQn/0z2 ZLIPhRktnmpIqpqLDXU7MtGwXNa1lVxRtpE2EXqwe40CqyFWsmKw0bWCZHkG1zmr s+w85LBHFVJlW4sCVN5KOjqyTiC8c78Lfuv05eQW/qN9wM83RAz0S/DLv8AnYgAa YL8MhSb9s8tEbeyPOpMSi+hsxQpldiwqEx0ftoO09aNrccuVWpd+3Q== =sVwD -----END PGP SIGNATURE----- From ct at flyingcircus.io Mon Mar 2 13:46:38 2015 From: ct at flyingcircus.io (Christian Theune) Date: Mon, 2 Mar 2015 14:46:38 +0100 Subject: Weird sub-folder error Message-ID: <3D51972C-EC20-45CA-ACD4-1555C2FE4DA0@flyingcircus.io> Hi, I?m running dovecot 2.2.13 and I ran into a weird problem. A user migrated his mailboxes by letting users copy them from their old server to the new one through their MUA. All that went fine except for one very weird instance. The user is running Outlook 2010. The user has a sub-folder structure which dovecot places as dotted names beneath the Inbox. (E.g. Maildir/.my.sub.folders). After copying all data, that user came back to us and complained about missing mails in their folders. Apparently 3 of her folders disappeared. I found folders with similar (but different!) names in the dovecot storage but with no mails in them. After a while I noticed that something made dovecot create another level of nesting, like this: Maildir/.My.Folder/Subfolder In this case the Original folder structure was INBOX - My - Folder - Subfolder. When the user visited the subscription settings in her MUA she found the INBOX - My - Folder mailbox which was empty. The Subfolder was nowhere to be found. This appeared weird to me. I moved the Subfolder back directly under the Maildir directory and named it ?.My.Folder.Subfolder?. doveadm now lists this folder correctly. I could not find whether this is a known bug - so I?m letting you know. Maybe you can help understand what happened. Sounds like a weird server-side bug to me - this shouldn?t happen, right? Cheers, Christian ? Christian Theune ? ct at flyingcircus.io ? +49 345 219401 0 Flying Circus Internet Operations GmbH ? http://flyingcircus.io Forsterstra?e 29 ? 06112 Halle (Saale) ? Deutschland HR Stendal HRB 21169 ? Gesch?ftsf?hrer: Christian. Theune, Christian. Zagrodnick -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From juan at inti.gob.ar Mon Mar 2 13:56:44 2015 From: juan at inti.gob.ar (Juan Bernhard) Date: Mon, 02 Mar 2015 10:56:44 -0300 Subject: Quota and ldap In-Reply-To: <54F46812.6020405@ac.marche.be> References: <54F43EE0.3020506@ac.marche.be> <54F4496E.7060700@ac.marche.be> <54F45F23.6020005@ac.marche.be> <54F463D4.1070101@inti.gob.ar> <54F46812.6020405@ac.marche.be> Message-ID: <54F46C1C.1060506@inti.gob.ar> El 02/03/2015 a las 10:39 a.m., Jean-Fran?ois S?n?chal escibi?: > Content maildirsize : > > cat Maildir/maildirsize > 0S The quota is set to unlimited. (0S) Try to remove the file and check if the new values are taken when a new mail arrives (or do a doveadm quota recalc) The wiki will explain more about this file: http://wiki2.dovecot.org/Quota/Maildir Saludos, Juan. > 181243879 2032 > 19202 1 > 14239 1 > 31954 1 > > > > Le 02/03/15 14:21, Juan Bernhard a ?crit : >> El 02/03/2015 a las 10:01 a.m., Jean-Fran?ois S?n?chal escibi?: >>> now dovecot -n give >>> >>> protocol lmtp { >>> mail_plugins = " quota" >>> postmaster_address = jf at marche.be >>> } >>> >>> But nothing change >>> >>> in gosaMailQuota I set size in ko bytes >>> >>> gosaMailQuota : 100000 >>> >> Maildir controls qutoa using a file, in your case will be >> ~/Maildir/maildirzise. The first line on the file tell you the mailbox >> limits of sapace (in bytes) and files. Check of this is enforced or not. >> Maybe dovecot only reads the ldap quota value when this file is not >> present, because it may be modified by another program (an mta for >> example) >> >> >>> Le 02/03/15 13:37, Steffen Kaiser a ?crit : >>> On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: >>> >>>>>> protocol lmtp { >>>>>> mail_plugins = " quota quota" >>> ^^^^^^^^ >>> it's added twice >>> >>>>>> Le 02/03/15 12:17, Steffen Kaiser a ?crit : >>>>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>>>> Hash: SHA1 >>>>>>> >>>>>>> On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: >>>>>>> >>>>>>> please post output of dovecont -n >>>>>>> >>>>>>>> user_attrs = >>>>>>>> homeDirectory=home,uidNumber=uid,gidNumber=gid,gosaMailQuota=quota_rule=*:bytes=%$ >>>>>>>> >>>>>>>> >>>>>>>> Mar 2 11:41:58 domaine dovecot: imap: Debug: Added userdb setting: >>>>>>>> plugin/quota_rule=*:bytes=0 >>> what's the content of siroco's LDAP entry? Esp. gosaMailQuota. >>> >>> -- Steffen Kaiser From jean-francois.senechal at ac.marche.be Mon Mar 2 14:06:58 2015 From: jean-francois.senechal at ac.marche.be (=?UTF-8?B?SmVhbi1GcmFuw6dvaXMgU8OpbsOpY2hhbA==?=) Date: Mon, 02 Mar 2015 15:06:58 +0100 Subject: Quota and ldap In-Reply-To: <54F46C1C.1060506@inti.gob.ar> References: <54F43EE0.3020506@ac.marche.be> <54F4496E.7060700@ac.marche.be> <54F45F23.6020005@ac.marche.be> <54F463D4.1070101@inti.gob.ar> <54F46812.6020405@ac.marche.be> <54F46C1C.1060506@inti.gob.ar> Message-ID: <54F46E82.6040308@ac.marche.be> I deleted doveadm quota recalc -u siroco Send a new mail But nothing change it's always 0S Is there is error here : user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid,gosaMailQuota=quota_rule=*:bytes=%$ ? Thnaks Mar 2 15:02:12 domaine dovecot: auth: Debug: userdb out: USER#0111#011siroco#011home=/var/spool/dovecot/mail/s/siroco#011uid=5000#011gid=5000#011quota_rule=*:bytes=0 Mar 2 15:02:19 domaine dovecot: auth: Debug: master in: USER#0111#011siroco#011service=doveadm Mar 2 15:02:19 domaine dovecot: auth: Debug: ldap(siroco): user search: base=ou=Users,ou=Citoyens,dc=marche,dc=be scope=subtree filter=(&(objectClass=posixAccount)(uid=siroco)) fields=homeDirectory,uidNumber,gidNumber,gosaMailQuota Mar 2 15:02:19 domaine dovecot: auth: Debug: ldap(siroco): result: gosaMailQuota=0 homeDirectory=/var/spool/dovecot/mail/s/siroco gidNumber=5000 uidNumber=5000; homeDirectory,uidNumber,gidNumber,gosaMailQuota unused Mar 2 15:02:19 domaine dovecot: auth: Debug: ldap(siroco): result: gosaMailQuota=0 homeDirectory=/var/spool/dovecot/mail/s/siroco gidNumber=5000 uidNumber=5000 Mar 2 15:02:19 domaine dovecot: auth: Debug: userdb out: USER#0111#011siroco#011home=/var/spool/dovecot/mail/s/siroco#011uid=5000#011gid=5000#011quota_rule=*:bytes=0 Mar 2 15:05:01 domaine /USR/SBIN/CRON[6447]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) Mar 2 15:05:14 domaine postfix/anvil[6417]: statistics: max connection rate 1/60s for (smtp:172.17.1.244) at Mar 2 15:01:54 Mar 2 15:05:14 domaine postfix/anvil[6417]: statistics: max connection count 1 for (smtp:172.17.1.244) at Mar 2 15:01:54 Mar 2 15:05:14 domaine postfix/anvil[6417]: statistics: max cache size 1 at Mar 2 15:01:54 Mar 2 15:05:26 domaine dovecot: auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Mar 2 15:05:26 domaine dovecot: auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Mar 2 15:05:26 domaine dovecot: auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libauthdb_ldap.so Mar 2 15:05:26 domaine dovecot: auth: Debug: Read auth token secret from /var/run/dovecot/auth-token-secret.dat Mar 2 15:05:26 domaine dovecot: auth: Debug: master in: USER#0111#011siroco#011service=doveadm Mar 2 15:05:26 domaine dovecot: auth: Debug: ldap(siroco): user search: base=ou=Users,ou=Citoyens,dc=marche,dc=be scope=subtree filter=(&(objectClass=posixAccount)(uid=siroco)) fields=homeDirectory,uidNumber,gidNumber,gosaMailQuota Mar 2 15:05:26 citoyen2015 dovecot: auth: Debug: ldap(siroco): result: gosaMailQuota=0 homeDirectory=/var/spool/dovecot/mail/s/siroco gidNumber=5000 uidNumber=5000; homeDirectory,uidNumber,gidNumber,gosaMailQuota unused Mar 2 15:05:26 domaine dovecot: auth: Debug: ldap(siroco): result: gosaMailQuota=0 homeDirectory=/var/spool/dovecot/mail/s/siroco gidNumber=5000 uidNumber=5000 Mar 2 15:05:26 domaine dovecot: auth: Debug: userdb out: USER#0111#011siroco#011home=/var/spool/dovecot/mail/s/siroco#011uid=5000#011gid=5000#011quota_rule=*:bytes=0 Le 02/03/15 14:56, Juan Bernhard a ?crit : > El 02/03/2015 a las 10:39 a.m., Jean-Fran?ois S?n?chal escibi?: >> Content maildirsize : >> >> cat Maildir/maildirsize >> 0S > The quota is set to unlimited. (0S) > Try to remove the file and check if the new values are taken when a new > mail arrives (or do a doveadm quota recalc) > > The wiki will explain more about this file: > http://wiki2.dovecot.org/Quota/Maildir > > Saludos, Juan. > >> 181243879 2032 >> 19202 1 >> 14239 1 >> 31954 1 >> >> >> >> Le 02/03/15 14:21, Juan Bernhard a ?crit : >>> El 02/03/2015 a las 10:01 a.m., Jean-Fran?ois S?n?chal escibi?: >>>> now dovecot -n give >>>> >>>> protocol lmtp { >>>> mail_plugins = " quota" >>>> postmaster_address =jf at marche.be >>>> } >>>> >>>> But nothing change >>>> >>>> in gosaMailQuota I set size in ko bytes >>>> >>>> gosaMailQuota : 100000 >>>> >>> Maildir controls qutoa using a file, in your case will be >>> ~/Maildir/maildirzise. The first line on the file tell you the mailbox >>> limits of sapace (in bytes) and files. Check of this is enforced or not. >>> Maybe dovecot only reads the ldap quota value when this file is not >>> present, because it may be modified by another program (an mta for >>> example) >>> >>> >>>> Le 02/03/15 13:37, Steffen Kaiser a ?crit : >>>> On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: >>>> >>>>>>> protocol lmtp { >>>>>>> mail_plugins = " quota quota" >>>> ^^^^^^^^ >>>> it's added twice >>>> >>>>>>> Le 02/03/15 12:17, Steffen Kaiser a ?crit : >>>>>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>>>>> Hash: SHA1 >>>>>>>> >>>>>>>> On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: >>>>>>>> >>>>>>>> please post output of dovecont -n >>>>>>>> >>>>>>>>> user_attrs = >>>>>>>>> homeDirectory=home,uidNumber=uid,gidNumber=gid,gosaMailQuota=quota_rule=*:bytes=%$ >>>>>>>>> >>>>>>>>> >>>>>>>>> Mar 2 11:41:58 domaine dovecot: imap: Debug: Added userdb setting: >>>>>>>>> plugin/quota_rule=*:bytes=0 >>>> what's the content of siroco's LDAP entry? Esp. gosaMailQuota. >>>> >>>> -- Steffen Kaiser From gheskett at wdtv.com Mon Mar 2 14:26:09 2015 From: gheskett at wdtv.com (Gene Heskett) Date: Mon, 2 Mar 2015 09:26:09 -0500 Subject: IP drop list In-Reply-To: <54F43549.1030803@LINworks.de> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F43549.1030803@LINworks.de> Message-ID: <201503020926.09445.gheskett@wdtv.com> On Monday 02 March 2015 05:02:49 Jochen Bern wrote: > On 03/01/2015 08:53 AM, Jim Pazarena wrote: > > I wonder if there is an easy way to provide dovecot a flat text file > > of ipv4 #'s which should be ignored or dropped? > > > > I have accumulated 45,000+ IPs which routinely try dictionary and > > 12345678 password attempts. The file is too big to create firewall > > drops [...] > > The inherent assumption here is that dovecot, using a "flat file", > will be able to process the block list more effectively than the > firewall, which is a tool written for the *purpose* but supposedly > unable to even *try* due to the list's size. That sounds ... > counterintuitive. > > To clarify, the governing influence on performance of *most* firewalls > is the average number of rules a packet has to be matched against, and > the two main tools to help with that are (if I may use iptables lingo > here) a) --state ESTABLISHED to get everything but the > connection-initiating packets out of the way ASAP and b) branching > tree-like into dedicated-purpose subchains, rather than building > linear lists. Assuming that the IPs to be blocked are randomly > distributed, I'ld try something along the following lines: > > [main chain] > --state ESTABLISHED,RELATED -j ACCEPT > -p tcp --dport pop3 -j dove-blocks > -p tcp --dport imap -j dove-blocks > > [subchain dove-blocks] > -d 1.0.0.0/8 -j sub-1 > -d 2.0.0.0/8 -j sub-2 > ... > -d 254.0.0.0/8 -j sub-254 > > [subchain sub-1] > -d 1.2.0.0/16 -j sub-1-2 # We've seen 1.2.3.4 and 1.2.2.1 > ... > > [subchain sub-1-2] > -d 1.2.2.1 -j DROP > -d 1.2.3.4 -j DROP > > Regards, > J. Bern I rather like this idea, but let me point out that this list should be pre-sorted with something that puts them in numerical order, and that order then pre-processed again to condense them into sequential blocks. And those sequential blocks are what you would present to iptables of ipset. You might have to trigger a new sort & condense session each time a new address is harvested and added to the list, but on a busy server that would have to be much less of a cpu hog than just searching a flat random list for every access. I use pop3 for access to 3 accounts, with mailfilter in front of fetchmail here, and occasionally will sort the reference files, and if a given class d address block gets hit several times, I re-arrange the regex to kill on "[xx.xx.xx'" alone, killing the whole class D. I watch the logs, and I don't recall that this policy has cost me a single message I should have received. Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page From jean-francois.senechal at ac.marche.be Mon Mar 2 15:25:32 2015 From: jean-francois.senechal at ac.marche.be (=?ISO-8859-15?Q?Jean-Fran=E7ois_S=E9n=E9chal?=) Date: Mon, 02 Mar 2015 16:25:32 +0100 Subject: Quota and ldap In-Reply-To: <54F46C1C.1060506@inti.gob.ar> References: <54F43EE0.3020506@ac.marche.be> <54F4496E.7060700@ac.marche.be> <54F45F23.6020005@ac.marche.be> <54F463D4.1070101@inti.gob.ar> <54F46812.6020405@ac.marche.be> <54F46C1C.1060506@inti.gob.ar> Message-ID: <54F480EC.6070801@ac.marche.be> I have found I have replace gosaMailQuota=quota_rule=*:bytes=%$ by gosaMailQuota=quota_rule=*:storage=%$M Le 02/03/15 14:56, Juan Bernhard a ?crit : > El 02/03/2015 a las 10:39 a.m., Jean-Fran?ois S?n?chal escibi?: >> Content maildirsize : >> >> cat Maildir/maildirsize >> 0S > The quota is set to unlimited. (0S) > Try to remove the file and check if the new values are taken when a new > mail arrives (or do a doveadm quota recalc) > > The wiki will explain more about this file: > http://wiki2.dovecot.org/Quota/Maildir > > Saludos, Juan. > >> 181243879 2032 >> 19202 1 >> 14239 1 >> 31954 1 >> >> >> >> Le 02/03/15 14:21, Juan Bernhard a ?crit : >>> El 02/03/2015 a las 10:01 a.m., Jean-Fran?ois S?n?chal escibi?: >>>> now dovecot -n give >>>> >>>> protocol lmtp { >>>> mail_plugins = " quota" >>>> postmaster_address = jf at marche.be >>>> } >>>> >>>> But nothing change >>>> >>>> in gosaMailQuota I set size in ko bytes >>>> >>>> gosaMailQuota : 100000 >>>> >>> Maildir controls qutoa using a file, in your case will be >>> ~/Maildir/maildirzise. The first line on the file tell you the mailbox >>> limits of sapace (in bytes) and files. Check of this is enforced or not. >>> Maybe dovecot only reads the ldap quota value when this file is not >>> present, because it may be modified by another program (an mta for >>> example) >>> >>> >>>> Le 02/03/15 13:37, Steffen Kaiser a ?crit : >>>> On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: >>>> >>>>>>> protocol lmtp { >>>>>>> mail_plugins = " quota quota" >>>> ^^^^^^^^ >>>> it's added twice >>>> >>>>>>> Le 02/03/15 12:17, Steffen Kaiser a ?crit : >>>>>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>>>>> Hash: SHA1 >>>>>>>> >>>>>>>> On Mon, 2 Mar 2015, Jean-Fran?ois S?n?chal wrote: >>>>>>>> >>>>>>>> please post output of dovecont -n >>>>>>>> >>>>>>>>> user_attrs = >>>>>>>>> homeDirectory=home,uidNumber=uid,gidNumber=gid,gosaMailQuota=quota_rule=*:bytes=%$ >>>>>>>>> >>>>>>>>> >>>>>>>>> Mar 2 11:41:58 domaine dovecot: imap: Debug: Added userdb setting: >>>>>>>>> plugin/quota_rule=*:bytes=0 >>>> what's the content of siroco's LDAP entry? Esp. gosaMailQuota. >>>> >>>> -- Steffen Kaiser -- logomarche Jean-Fran?ois S?n?chal Centre de Support T?l?matique Webmaster Rue des Carmes, 22 6900 Marche-en-Famenne T?l: 084/32 70 55 - Fax: 084/32 70 72 jean-francois.senechal at ac.marche.be Retrouvez-nous sur www.marche.be et sur logofacebook Disclaimer | SVP, pensez ? notre environnement avant d'imprimer ce mail From rs at sys4.de Mon Mar 2 17:56:18 2015 From: rs at sys4.de (Robert Schetterer) Date: Mon, 02 Mar 2015 18:56:18 +0100 Subject: IP drop list In-Reply-To: References: Message-ID: <54F4A442.6040805@sys4.de> Am 02.03.2015 um 11:34 schrieb Joseph Tam: > Dave McGuire writes: > >>> http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/AllowNets >>> >>> then setup fail2ban to manage extrafields >> >> Now that's a very interesting idea, thank you! I will investigate this. > > If you don't expect yor firewall to handle 45K+ IPs, I'm not how you > expect dovecot will handle a comma separated string with 45K+ entries > any better. If you want to turn your global backlist into a per-user > whitelist, that would be perfectly doable though. > > Joseph Tam perhaps and i mean really "perhaps" go this way https://sys4.de/de/blog/2014/03/27/fighting-smtp-auth-brute-force-attacks/ https://sys4.de/de/blog/2012/12/28/botnets-mit-rsyslog-und-iptables-recent-modul-abwehren/ 45K+ IPs will work in a recent table i have them too but for smtp only like echo 10000000 > /sys/module/xt_recent/parameters/ip_list_tot combine with geoip might be a good idea too is ultra faster then fail2ban cause no log file parsing is needed or an other idea you might test, configure a syslog filter pumping in a recent table the direct way Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From h.reindl at thelounge.net Mon Mar 2 18:03:00 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 02 Mar 2015 19:03:00 +0100 Subject: IP drop list In-Reply-To: <54F4A442.6040805@sys4.de> References: <54F4A442.6040805@sys4.de> Message-ID: <54F4A5D4.6070303@thelounge.net> Am 02.03.2015 um 18:56 schrieb Robert Schetterer: > perhaps and i mean really "perhaps" go this way > > https://sys4.de/de/blog/2014/03/27/fighting-smtp-auth-brute-force-attacks/ > > https://sys4.de/de/blog/2012/12/28/botnets-mit-rsyslog-und-iptables-recent-modul-abwehren/ > > 45K+ IPs will work in a recent table > i have them too but for smtp only like > > echo 10000000 > /sys/module/xt_recent/parameters/ip_list_tot > > combine with geoip might be a good idea too > > is ultra faster then fail2ban cause no log file parsing is needed > > or an other idea > you might test, configure a syslog filter pumping in a recent table the > direct way that is all nice but the main benefit of RBL's is always ignored: * centralized * no log parsing at all * honeypot data are "delivered" to any host * it's cheap * it's easy to maintain * it don't need any root privileges anywhere we have a small honeypot network with a couple of ipranges detecting mass port-scans and so on and this data are available *everywhere* so if some IP hits there it takes 60 seconds and any service supportings DNS blacklists can block them *even before* the bot hits the real mailserver at all -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From ct at flyingcircus.io Mon Mar 2 18:25:34 2015 From: ct at flyingcircus.io (Christian Theune) Date: Mon, 2 Mar 2015 19:25:34 +0100 Subject: Weird sub-folder error In-Reply-To: <3D51972C-EC20-45CA-ACD4-1555C2FE4DA0@flyingcircus.io> References: <3D51972C-EC20-45CA-ACD4-1555C2FE4DA0@flyingcircus.io> Message-ID: <413DB470-BE6B-435D-898C-D171ED371EC1@flyingcircus.io> > On 02 Mar 2015, at 14:46, Christian Theune wrote: > > Hi, > > I?m running dovecot 2.2.13 and I ran into a weird problem. > > A user migrated his mailboxes by letting users copy them from their old server to the new one through their MUA. All that went fine except for one very weird instance. The user is running Outlook 2010. > > The user has a sub-folder structure which dovecot places as dotted names beneath the Inbox. (E.g. Maildir/.my.sub.folders). > After copying all data, that user came back to us and complained about missing mails in their folders. Apparently 3 of her folders disappeared. > > I found folders with similar (but different!) names in the dovecot storage but with no mails in them. After a while I noticed that something made dovecot create another level of nesting, like this: > > Maildir/.My.Folder/Subfolder > > In this case the Original folder structure was INBOX - My - Folder - Subfolder. When the user visited the subscription settings in her MUA she found the INBOX - My - Folder mailbox which was empty. The Subfolder was nowhere to be found. This appeared weird to me. I moved the Subfolder back directly under the Maildir directory and named it ?.My.Folder.Subfolder?. doveadm now lists this folder correctly. Actually, little correction. It didn?t fail to split a subfolder like this but some (to me unknown) character, maybe whitespace: So the original hierarchy was: ?INBOX? - ?Some Subfolder? which was turned into Maildir/.Some/Subfolder Christian ? Christian Theune ? ct at flyingcircus.io ? +49 345 219401 0 Flying Circus Internet Operations GmbH ? http://flyingcircus.io Forsterstra?e 29 ? 06112 Halle (Saale) ? Deutschland HR Stendal HRB 21169 ? Gesch?ftsf?hrer: Christian. Theune, Christian. Zagrodnick -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dlasota at alaska.edu Mon Mar 2 18:30:13 2015 From: dlasota at alaska.edu (Dan LaSota) Date: Mon, 2 Mar 2015 09:30:13 -0900 Subject: Connect failed to database In-Reply-To: References: <0B461721-6E10-4290-A61C-B86146BC7798@alaska.edu> <54F4147C.4040708@oliwel.de> Message-ID: >> >> Just some quick ideas >> * check if the mysql socket file has rw permissions for the dovecot user # ls -l /var/lib/mysql/mysql.sock srwxrwxrwx. 1 mysql mysql 0 Mar 1 19:33 /var/lib/mysql/mysql.sock >> >> * Try to run the mysql query as user dovecot (su dovecot) The dovecot user I set up is a non-interactive user /sbin/nologin I studied different setup guides, and all of them suggested setting up dovecot with a non-interactive shell. However the mysql client is world executable: # ls -l /bin/mysql -rwxr-xr-x. 1 root root 3533008 Feb 5 05:47 /bin/mysql >> * Try to set the local ip instead of localhost (mysql makes a difference in the ACL checks if you come from localhost) I have tried with connect = host=localhost dbname=servermail user='usermail' password='gjwslegosoghjshloehg$_jsdgh' and connect = host=127.0.0.1 dbname=servermail user='usermail' password='gjwslegosoghjshloehg$_jsdgh' still doesn't work: Mar 2 04:58:48 mail dovecot: auth-worker(5745): Error: mysql(127.0.0.1): Connect failed to database (servermail): Access denied for user 'usermail'@'localhost' (using password: YES) - waiting for 1 seconds before retry > > > * Did mysql logged something useful? I have turned on general-log in MySQL (actually mariadb): A few things to note: connection 221 is me manually logging into the server from the command line. The other numbers are dovecot. I'm using the same credentials as the dovecot config files. (not sure if this is useful or not) 221 Connect mailuser at localhost as anonymous on 221 Query select @@version_comment limit 1 150302 9:18:33 221 Query show databases 150302 9:18:38 221 Query SELECT DATABASE() 221 Init DB servermail 221 Query show databases 221 Query show tables 221 Field List virtual_aliases 221 Field List virtual_domains 221 Field List virtual_users 150302 9:18:43 221 Query show tables 150302 9:18:50 222 Connect usermail at localhost as anonymous on servermail 222 Connect Access denied for user 'usermail'@'localhost' (using password: YES) 223 Connect usermail at localhost as anonymous on servermail 223 Connect Access denied for user 'usermail'@'localhost' (using password: YES) 150302 9:18:56 221 Query select * from virtual_users 150302 9:19:19 224 Connect usermail at localhost as anonymous on servermail 224 Connect Access denied for user 'usermail'@'localhost' (using password: YES) 213 Quit 214 Quit 215 Quit 150302 9:19:20 225 Connect usermail at localhost as anonymous on servermail 225 Connect Access denied for user 'usermail'@'localhost' (using password: YES) 150302 9:19:24 221 Quit 150302 9:19:25 226 Connect usermail at localhost as anonymous on servermail 226 Connect Access denied for user 'usermail'@'localhost' (using password: YES) 150302 9:19:50 227 Connect usermail at localhost as anonymous on servermail 227 Connect Access denied for user 'usermail'@'localhost' (using password: YES) Dan LaSota Instructional Designer, UAF eLearning (907) 451-4067 dan.lasota at alaska.edu http://elearning.uaf.edu From h.reindl at thelounge.net Mon Mar 2 18:33:11 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 02 Mar 2015 19:33:11 +0100 Subject: Connect failed to database In-Reply-To: References: <0B461721-6E10-4290-A61C-B86146BC7798@alaska.edu> <54F4147C.4040708@oliwel.de> Message-ID: <54F4ACE7.7060408@thelounge.net> Am 02.03.2015 um 19:30 schrieb Dan LaSota: >>> Just some quick ideas >>> * check if the mysql socket file has rw permissions for the dovecot user > > # ls -l /var/lib/mysql/mysql.sock > srwxrwxrwx. 1 mysql mysql 0 Mar 1 19:33 /var/lib/mysql/mysql.sock that's not the problem > I have tried with > connect = host=localhost dbname=servermail user='usermail' password='gjwslegosoghjshloehg$_jsdgh' > and > connect = host=127.0.0.1 dbname=servermail user='usermail' password='gjwslegosoghjshloehg$_jsdgh' > > still doesn't work: > Mar 2 04:58:48 mail dovecot: auth-worker(5745): Error: mysql(127.0.0.1): Connect failed to database (servermail): Access denied for user 'usermail'@'localhost' (using password: YES) - waiting for 1 seconds before retry that is a pretty clear message the username / password / host is wrong, fix your mysql permissions and keep in mind that localhost != 127.0.0.1 in that context the user with that password from that host is not allowed - period -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From me at junc.eu Mon Mar 2 18:44:48 2015 From: me at junc.eu (Benny Pedersen) Date: Mon, 02 Mar 2015 19:44:48 +0100 Subject: IP drop list In-Reply-To: References: Message-ID: <14bdbce0130.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> On March 2, 2015 11:35:24 AM Joseph Tam wrote: > Dave McGuire writes: > > >> http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/AllowNets > >> > >> then setup fail2ban to manage extrafields > > > > Now that's a very interesting idea, thank you! I will investigate this. > > If you don't expect yor firewall to handle 45K+ IPs, I'm not how you > expect dovecot will handle a comma separated string with 45K+ entries > any better. If you want to turn your global backlist into a per-user > whitelist, that would be perfectly doable though. lets call it denynets so :) avises is bad when users does not understand why its allownets and still fokus on block bad ips then just keep list of good client ips where login is not fail, if dovecot is hard to understand try windows 10 then :) From dlasota at alaska.edu Mon Mar 2 18:53:26 2015 From: dlasota at alaska.edu (Dan LaSota) Date: Mon, 2 Mar 2015 09:53:26 -0900 Subject: Connect failed to database In-Reply-To: <54F4ACE7.7060408@thelounge.net> References: <0B461721-6E10-4290-A61C-B86146BC7798@alaska.edu> <54F4147C.4040708@oliwel.de> <54F4ACE7.7060408@thelounge.net> Message-ID: <2F423C46-5653-4D8F-A5B8-A24F8DBF41A8@alaska.edu> >> >> still doesn't work: >> Mar 2 04:58:48 mail dovecot: auth-worker(5745): Error: mysql(127.0.0.1): Connect failed to database (servermail): Access denied for user 'usermail'@'localhost' (using password: YES) - waiting for 1 seconds before retry > > that is a pretty clear message > > the username / password / host is wrong, fix your mysql permissions and keep in mind that localhost != 127.0.0.1 in that context which is the preferred address: localhost or 127.0.0.1 ? > > the user with that password from that host is not allowed - period Then why can I use the same credentials to log into mysql from the command line? Like so: # mysql -u mailuser -h localhost -p Dan LaSota Instructional Designer, UAF eLearning (907) 451-4067 dan.lasota at alaska.edu http://elearning.uaf.edu From h.reindl at thelounge.net Mon Mar 2 18:57:27 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 02 Mar 2015 19:57:27 +0100 Subject: Connect failed to database In-Reply-To: <2F423C46-5653-4D8F-A5B8-A24F8DBF41A8@alaska.edu> References: <0B461721-6E10-4290-A61C-B86146BC7798@alaska.edu> <54F4147C.4040708@oliwel.de> <54F4ACE7.7060408@thelounge.net> <2F423C46-5653-4D8F-A5B8-A24F8DBF41A8@alaska.edu> Message-ID: <54F4B297.3090009@thelounge.net> Am 02.03.2015 um 19:53 schrieb Dan LaSota: >>> >>> still doesn't work: >>> Mar 2 04:58:48 mail dovecot: auth-worker(5745): Error: mysql(127.0.0.1): Connect failed to database (servermail): Access denied for user 'usermail'@'localhost' (using password: YES) - waiting for 1 seconds before retry >> >> that is a pretty clear message >> >> the username / password / host is wrong, fix your mysql permissions and keep in mind that localhost != 127.0.0.1 in that context > > which is the preferred address: localhost or 127.0.0.1 ? depends if you prefer unix sockets: localhost if you prefer TCP: 127.0.0.1 >> the user with that password from that host is not allowed - period > > Then why can I use the same credentials to log into mysql from the command line? Like so: > > # mysql -u mailuser -h localhost -p that is unix-socket, -h 127.0.0.1 would be TCP just use "host=/var/lib/mysql/mysql.sock" or wherever your socket lives or give the 127.0.0.1 user the same permissions - these are mysql basics -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From rs at sys4.de Mon Mar 2 19:00:30 2015 From: rs at sys4.de (Robert Schetterer) Date: Mon, 02 Mar 2015 20:00:30 +0100 Subject: IP drop list In-Reply-To: <54F4A5D4.6070303@thelounge.net> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> Message-ID: <54F4B34E.6010205@sys4.de> Am 02.03.2015 um 19:03 schrieb Reindl Harald: > > Am 02.03.2015 um 18:56 schrieb Robert Schetterer: >> perhaps and i mean really "perhaps" go this way >> >> https://sys4.de/de/blog/2014/03/27/fighting-smtp-auth-brute-force-attacks/ >> >> >> https://sys4.de/de/blog/2012/12/28/botnets-mit-rsyslog-und-iptables-recent-modul-abwehren/ >> >> >> 45K+ IPs will work in a recent table >> i have them too but for smtp only like >> >> echo 10000000 > /sys/module/xt_recent/parameters/ip_list_tot >> >> combine with geoip might be a good idea too >> >> is ultra faster then fail2ban cause no log file parsing is needed >> >> or an other idea >> you might test, configure a syslog filter pumping in a recent table the >> direct way > > that is all nice > > but the main benefit of RBL's is always ignored: > > * centralized > * no log parsing at all > * honeypot data are "delivered" to any host > * it's cheap > * it's easy to maintain > * it don't need any root privileges anywhere > > we have a small honeypot network with a couple of ipranges detecting > mass port-scans and so on and this data are available *everywhere* > > so if some IP hits there it takes 60 seconds and any service supportings > DNS blacklists can block them *even before* the bot hits the real > mailserver at all > centralize may also work with syslog filters acting to a "grand" firewall/loadbalancers in front of all hosts, anyway depending to setups combine many solutions may goal the best results, your solution is fine too. At the end everything is fine what solves the task, and the admin has to decide which way he want to go MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From me at junc.eu Mon Mar 2 19:01:59 2015 From: me at junc.eu (Benny Pedersen) Date: Mon, 02 Mar 2015 20:01:59 +0100 Subject: IP drop list In-Reply-To: <54F4A442.6040805@sys4.de> References: <54F4A442.6040805@sys4.de> Message-ID: <14bdbddc458.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> > >>> http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/AllowNets rethink why its allownets not denynets > 45K+ IPs will work in a recent table > i have them too but for smtp only like have you seem a single user with 45k ips that does not make logs of login fails ? From rs at sys4.de Mon Mar 2 19:31:50 2015 From: rs at sys4.de (Robert Schetterer) Date: Mon, 02 Mar 2015 20:31:50 +0100 Subject: IP drop list In-Reply-To: <14bdbddc458.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> References: <54F4A442.6040805@sys4.de> <14bdbddc458.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> Message-ID: <54F4BAA6.6080103@sys4.de> Am 02.03.2015 um 20:01 schrieb Benny Pedersen: > >> >>> http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/AllowNets > > rethink why its allownets not denynets > >> 45K+ IPs will work in a recent table >> i have them too but for smtp only like > > have you seem a single user with 45k ips that does not make logs of > login fails ? the most problem may nat and false positves, with firewall or deny ip stuff you may ban wanted users too, so this should be only used in heavy cases, so there is no ultimate solution which fits every case on every setup Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From mail at oliwel.de Mon Mar 2 20:20:56 2015 From: mail at oliwel.de (Oliver Welter) Date: Mon, 02 Mar 2015 21:20:56 +0100 Subject: Connect failed to database In-Reply-To: References: <0B461721-6E10-4290-A61C-B86146BC7798@alaska.edu> <54F4147C.4040708@oliwel.de> Message-ID: <54F4C628.3050208@oliwel.de> Am 02.03.2015 um 19:30 schrieb Dan LaSota: >>> >>> Just some quick ideas >>> * check if the mysql socket file has rw permissions for the dovecot user > > # ls -l /var/lib/mysql/mysql.sock > srwxrwxrwx. 1 mysql mysql 0 Mar 1 19:33 /var/lib/mysql/mysql.sock >>> >>> * Try to run the mysql query as user dovecot (su dovecot) > > The dovecot user I set up is a non-interactive user > /sbin/nologin You can use "su dovecot -s /bin/bash" to override the configured shell (or temporary set the users shell using "usermod"). > However the mysql client is world executable: doevcot should not need the client (its build in) >>> * Try to set the local ip instead of localhost (mysql makes a difference in the ACL checks if you come from localhost) > > I have tried with > connect = host=localhost dbname=servermail user='usermail' password='gjwslegosoghjshloehg$_jsdgh' > and > connect = host=127.0.0.1 dbname=servermail user='usermail' password='gjwslegosoghjshloehg$_jsdgh' Do you really have a dollar sign in the password? This is always a suspicious candidate - try a password without the dollar sign (I dont know if/what dovecot requires escaping) Oliver -- Protect your environment - close windows and adopt a penguin! -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4074 bytes Desc: S/MIME Cryptographic Signature URL: From mcguire at neurotica.com Mon Mar 2 21:50:00 2015 From: mcguire at neurotica.com (Dave McGuire) Date: Mon, 02 Mar 2015 16:50:00 -0500 Subject: IP drop list In-Reply-To: References: Message-ID: <54F4DB08.5060209@neurotica.com> On 03/02/2015 05:34 AM, Joseph Tam wrote: >>> http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/AllowNets >>> >>> then setup fail2ban to manage extrafields >> >> Now that's a very interesting idea, thank you! I will investigate this. > > If you don't expect yor firewall to handle 45K+ IPs, I'm not how you > expect dovecot will handle a comma separated string with 45K+ entries > any better. My firewall can handle that without breaking a sweat. I just haven't found a way (that I'm comfortable with) to automatically inject rules into it from a machine on the network. Doing it via a DNSBL is an elegant solution to the problem, IMO. It offloads the IP address indexing to the DNS server; BIND (and most anything else I'd imagine, but I run BIND) uses a pretty respectable in-memory btree system which gives fast lookups. (well, at least that's what it used the last time I looked at its internals) I myself just want a mechanism to deny certain IP addresses when I spot them, regardless of the implementation. But anything that offloads my mail servers from anything that doesn't involve serving mail makes me happy. -Dave -- Dave McGuire, AK4HZ/3 New Kensington, PA From dlasota at alaska.edu Mon Mar 2 23:24:17 2015 From: dlasota at alaska.edu (Dan LaSota) Date: Mon, 2 Mar 2015 14:24:17 -0900 Subject: Connect failed to database In-Reply-To: <54F4C628.3050208@oliwel.de> References: <0B461721-6E10-4290-A61C-B86146BC7798@alaska.edu> <54F4147C.4040708@oliwel.de> <54F4C628.3050208@oliwel.de> Message-ID: Thanks for the auggestion. I did indeed have a dollar sign in there. No longer the case. I also changed 'localhost' to the pathname to the unix socket file as well. I'm now onto the next misconfiguration! > On Mar 2, 2015, at 11:20 AM, Oliver Welter wrote: > > Am 02.03.2015 um 19:30 schrieb Dan LaSota: >>>> >>>> Just some quick ideas >>>> * check if the mysql socket file has rw permissions for the dovecot user >> >> # ls -l /var/lib/mysql/mysql.sock >> srwxrwxrwx. 1 mysql mysql 0 Mar 1 19:33 /var/lib/mysql/mysql.sock >>>> >>>> * Try to run the mysql query as user dovecot (su dovecot) >> >> The dovecot user I set up is a non-interactive user >> /sbin/nologin > > You can use "su dovecot -s /bin/bash" to override the configured shell (or temporary set the users shell using "usermod"). > >> However the mysql client is world executable: > doevcot should not need the client (its build in) > >>>> * Try to set the local ip instead of localhost (mysql makes a difference in the ACL checks if you come from localhost) >> >> I have tried with >> connect = host=localhost dbname=servermail user='usermail' password='gjwslegosoghjshloehg$_jsdgh' >> and >> connect = host=127.0.0.1 dbname=servermail user='usermail' password='gjwslegosoghjshloehg$_jsdgh' > > Do you really have a dollar sign in the password? This is always a suspicious candidate - try a password without the dollar sign (I dont know if/what dovecot requires escaping) > > Oliver > > -- > Protect your environment - close windows and adopt a penguin! > Dan LaSota Instructional Designer, UAF eLearning (907) 451-4067 dan.lasota at alaska.edu http://elearning.uaf.edu From me at junc.eu Mon Mar 2 23:34:54 2015 From: me at junc.eu (Benny Pedersen) Date: Tue, 03 Mar 2015 00:34:54 +0100 Subject: IP drop list In-Reply-To: <54F4BAA6.6080103@sys4.de> References: <54F4A442.6040805@sys4.de> <14bdbddc458.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> <54F4BAA6.6080103@sys4.de> Message-ID: <14bdcd7a130.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> On March 2, 2015 8:32:35 PM Robert Schetterer wrote: > the most problem may nat and false positves, with firewall or deny ip > stuff you may ban wanted users too, so this should be only used in heavy > cases, so there is no ultimate solution which fits every case on every setup yep pop-before-smtp was simple once, here i just allow in country users with xtables geoip, out of country is vacation users with no life :) geoip have here low mem footprint, so i prefer it so From me at junc.eu Mon Mar 2 23:45:46 2015 From: me at junc.eu (Benny Pedersen) Date: Tue, 03 Mar 2015 00:45:46 +0100 Subject: IP drop list In-Reply-To: <54F4DB08.5060209@neurotica.com> References: <54F4DB08.5060209@neurotica.com> Message-ID: <14bdce19410.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> On March 2, 2015 10:50:59 PM Dave McGuire wrote: > On 03/02/2015 05:34 AM, Joseph Tam wrote: > >>> http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/AllowNets its not a big hint its not called denynets is it ? > I myself just want a mechanism to deny certain IP addresses when I > spot them, regardless of the implementation. But anything that offloads > my mail servers from anything that doesn't involve serving mail makes me > happy. fokus on not blocking 500000 ips, but that users not have 500000 ips i will stop saying this again From kremels at kreme.com Mon Mar 2 23:47:53 2015 From: kremels at kreme.com (@lbutlr) Date: Mon, 2 Mar 2015 16:47:53 -0700 Subject: IP drop list In-Reply-To: <14bdcd7a130.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> References: <54F4A442.6040805@sys4.de> <14bdbddc458.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> <54F4BAA6.6080103@sys4.de> <14bdcd7a130.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> Message-ID: <1B2DA95F-D936-4953-8C99-8BD8070C3965@kreme.com> On 02 Mar 2015, at 16:34 , Benny Pedersen wrote: > On March 2, 2015 8:32:35 PM Robert Schetterer wrote: > >> the most problem may nat and false positves, with firewall or deny ip >> stuff you may ban wanted users too, so this should be only used in heavy >> cases, so there is no ultimate solution which fits every case on every setup > > yep pop-before-smtp was simple once, here i just allow in country users with xtables geoip, out of country is vacation users with no life :) You must have a small user base if you can block users who are out of the country. Even with my very small server I?ve had legitimate users connect from at least a dozen countries in the last year. Not everyone who travels abroad is doing it for vacation. I have one user who is often in the far east or India and another who is often in numerous African countries, all for business. The proper thing to do is to setup authentication on port 587 and only use that for submitting mail (that is, do not allow submission on port 25 at all) and then use something like sshguard or fail2ban to blacklist repeated unauthorized connections. I simply block ssh access at all unless it is from inside the LAN or from one specific IP address, so to get to my servers I have to ssh to ServerA which is the only server allowed external access to ssh on my mail web and DNS servers. -- I intend to live forever -- so far, so good! From h.reindl at thelounge.net Mon Mar 2 23:57:28 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Tue, 03 Mar 2015 00:57:28 +0100 Subject: IP drop list In-Reply-To: <14bdce19410.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> References: <54F4DB08.5060209@neurotica.com> <14bdce19410.28c1.d475fad7b14312f5d8424e35e39f7512@junc.eu> Message-ID: <54F4F8E8.2010103@thelounge.net> Am 03.03.2015 um 00:45 schrieb Benny Pedersen: > On March 2, 2015 10:50:59 PM Dave McGuire wrote: > >> On 03/02/2015 05:34 AM, Joseph Tam wrote: >> >>> http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/AllowNets > > its not a big hint its not called denynets is it ? > >> I myself just want a mechanism to deny certain IP addresses when I >> spot them, regardless of the implementation. But anything that offloads >> my mail servers from anything that doesn't involve serving mail makes me >> happy. > > fokus on not blocking 500000 ips, but that users not have 500000 ips if the server is just for you, your brother and his wife > i will stop saying this again better so -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From dlasota at alaska.edu Tue Mar 3 02:09:47 2015 From: dlasota at alaska.edu (Dan LaSota) Date: Mon, 2 Mar 2015 17:09:47 -0900 Subject: problem with dovecot/lmtp Message-ID: I am getting this in my errorlog for postfix/dovecot Mar 2 16:42:14 mail postfix/error[5074]: A689580F236: to=, relay=none, delay=21420, delays=21420/0.05/0/0, dsn=4.4.1, status=deferred (delivery temporarily suspended: connect to mail.example.com[private/dovecot-lmtp]: Connection refused) From my reading, I believe this has something to do with virtual_transport which I have set to: virtual_transport = lmtp:unix:private/dovecot-lmtp The socket file, exists: [root at mail ~]# ls -l /var/spool/postfix/private/dovecot-lmtp srw-------. 1 postfix postfix 0 Mar 2 09:14 /var/spool/postfix/private/dovecot-lmtp But I'm a little stumped as how to proceed. The only thing I think I understand is that postfix is trying to talk to dovecot-lmtp and dovecot-lmtp is refusing the connection. Am I reading that correct? What can I do about it? Thanks, Dan LaSota Instructional Designer, UAF eLearning (907) 451-4067 dan.lasota at alaska.edu http://elearning.uaf.edu From jtam.home at gmail.com Tue Mar 3 02:41:51 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 2 Mar 2015 18:41:51 -0800 (PST) Subject: IP drop list In-Reply-To: References: Message-ID: Dave McGuire writes: >>>> then setup fail2ban to manage extrafields >>> >>> Now that's a very interesting idea, thank you! I will investigate this. >> >> If you don't expect yor firewall to handle 45K+ IPs, I'm not how you >> expect dovecot will handle a comma separated string with 45K+ entries >> any better. > > My firewall can handle that without breaking a sweat. I just haven't > found a way (that I'm comfortable with) to automatically inject rules > into it from a machine on the network. > > Doing it via a DNSBL is an elegant solution to the problem, IMO. I'm agnostic as far as which method you want to use. All I'm saying is that using dovecot's allow_net facility is as difficult, if not more so, than letting your firewall handle it. Joseph Tam From skdovecot at smail.inf.fh-brs.de Tue Mar 3 07:25:14 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Tue, 3 Mar 2015 08:25:14 +0100 (CET) Subject: Weird sub-folder error In-Reply-To: <413DB470-BE6B-435D-898C-D171ED371EC1@flyingcircus.io> References: <3D51972C-EC20-45CA-ACD4-1555C2FE4DA0@flyingcircus.io> <413DB470-BE6B-435D-898C-D171ED371EC1@flyingcircus.io> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 2 Mar 2015, Christian Theune wrote: > Actually, little correction. It didn?t fail to split a subfolder like this but some (to me unknown) character, maybe whitespace: > > So the original hierarchy was: > > ?INBOX? - ?Some Subfolder? which was turned into > > Maildir/.Some/Subfolder is it even more possible, that the slash was in place before, too? E.g. "Jahr 2014/15" ? Please show us your config, because the Maildir code forbids slashes normally, any CREATE command should reject the request. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVPVh2nz1H7kL/d9rAQLXBQf+Mb2BKzHtGquEsDEGBMwlhO1EXZvYetXL UYydKiCXU3vWKoKMaKWfotqbyvBYUnPC4cxfO7XodQ+yYMPTDOEVWzwjT2ewYGg8 mDeYgSN5ks86yqKUDikZ9swVh/1zKeqZbV4vcKfkU7YfY2TmCtw2Fb1ubhLY9RUO mqLXpxnkr/0UreTbme2lqCeF8mTiKkSQ4dv5A2eSNFxPF6M0C+8qn66N37IQE5T1 hH+36r9nZbwUGJu4bECnmyZyXsEQ1R4ivRcGACaD2W3dmmoqMo790XQk6oyMgpIT 5Rq8+2CPPgoyMQMhyphVZrBwxjXSVVLsjnXpQkrA9HLV8GvbOX+mkg== =pfT3 -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Tue Mar 3 07:36:27 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Tue, 3 Mar 2015 08:36:27 +0100 (CET) Subject: Connect failed to database In-Reply-To: References: <0B461721-6E10-4290-A61C-B86146BC7798@alaska.edu> <54F4147C.4040708@oliwel.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 2 Mar 2015, Dan LaSota wrote: > connect = host=localhost dbname=servermail user='usermail' password='gjwslegosoghjshloehg$_jsdgh' Honestely, I do not know whether and how you have / can quote strings, but try: 1) to not quote at all 2) use " 3) use a password without $ - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVPVke3z1H7kL/d9rAQJMpwf/c23CpNdb2jRtjw3LcMdDZmO19ylMJdZJ 9rBtse0zWpyCLZ60pCLvhWh/jZSHmXGofV3oTaJl9CXuK8/jVMSzfYzCpV4gPU4l fBT+QT6q6HnmfhtWH746wV/l18arp0uOa/D/sbI10Zjx/m16mj6md0FAVxAGklnw m2vzKXWPOj7HBXQwlDoZdPt8S08JOZE9Q6WSkhXpjsF2oGH6i61yckdrtEWs/N0n U2NtTZTGmMOm+8jRq1ljSdPEJYT6k93e1pw2wmA2zH5ODr9AQspsfyWbL3dmChKl 2RtzUsajfQwiwnaIOzgur8KHoFDS8Tm+sFLmiSAqrr8AeQyRBlLWbg== =5UDi -----END PGP SIGNATURE----- From ct at flyingcircus.io Tue Mar 3 08:55:10 2015 From: ct at flyingcircus.io (Christian Theune) Date: Tue, 3 Mar 2015 09:55:10 +0100 Subject: Weird sub-folder error In-Reply-To: References: <3D51972C-EC20-45CA-ACD4-1555C2FE4DA0@flyingcircus.io> <413DB470-BE6B-435D-898C-D171ED371EC1@flyingcircus.io> Message-ID: Hi, interesting idea about ?/? being in the folder name. I?ll ask whether that was the case. Here?s the doveconf -n output: (minus some privacy) # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.17.2-gentoo i686 Gentoo Base System release 2.2 auth_mechanisms = plain login auth_socket_path = /run/dovecot/auth-userdb disable_plaintext_auth = no first_valid_uid = 119 mail_location = maildir:~/Maildir mail_plugins = quota mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate namespace inbox { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/deny-users deny = yes driver = passwd-file } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { quota = maildir sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service managesieve-login { executable = /usr/libexec/dovecot/managesieve-login inet_listener sieve { address = xxx.xxx.xxx.xxx port = 4190 reuse_port = no ssl = no } } service managesieve { executable = /usr/libexec/dovecot/managesieve } ssl_cert = On 03 Mar 2015, at 08:25, Steffen Kaiser wrote: > > PS: weird, your mail was shown in this way to me on OS X Mail. Haven?t seen that before. :) ? Christian Theune ? ct at flyingcircus.io ? +49 345 219401 0 Flying Circus Internet Operations GmbH ? http://flyingcircus.io Forsterstra?e 29 ? 06112 Halle (Saale) ? Deutschland HR Stendal HRB 21169 ? Gesch?ftsf?hrer: Christian. Theune, Christian. Zagrodnick -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mcguire at neurotica.com Tue Mar 3 11:40:09 2015 From: mcguire at neurotica.com (Dave McGuire) Date: Tue, 03 Mar 2015 06:40:09 -0500 Subject: IP drop list In-Reply-To: References: Message-ID: <54F59D99.4070507@neurotica.com> On 03/02/2015 09:41 PM, Joseph Tam wrote: >>>>> then setup fail2ban to manage extrafields >>>> >>>> Now that's a very interesting idea, thank you! I will investigate >>>> this. >>> >>> If you don't expect yor firewall to handle 45K+ IPs, I'm not how you >>> expect dovecot will handle a comma separated string with 45K+ entries >>> any better. >> >> My firewall can handle that without breaking a sweat. I just haven't >> found a way (that I'm comfortable with) to automatically inject rules >> into it from a machine on the network. >> >> Doing it via a DNSBL is an elegant solution to the problem, IMO. > > I'm agnostic as far as which method you want to use. All I'm saying is > that using dovecot's allow_net facility is as difficult, if not > more so, than letting your firewall handle it. I'm not disagreeing with you. As I stated above, getting new rules into my firewall in an automated way is not something I've found a good way to do yet. Granted, it has been a couple of years since I've googled around to see if anyone has been able to do it in a reasonably secure way. (Perhaps it's time for me to revisit that.) -Dave -- Dave McGuire, AK4HZ/3 New Kensington, PA From nick.z.edwards at gmail.com Tue Mar 3 12:35:16 2015 From: nick.z.edwards at gmail.com (Nick Edwards) Date: Tue, 3 Mar 2015 22:35:16 +1000 Subject: IP drop list In-Reply-To: References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> <54F38FBE.3040108@neurotica.com> <54F4137F.8050507@oliwel.de> <54F41E1D.5030901@neurotica.com> <54F42938.7000000@thelounge.net> Message-ID: daemontools On 3/2/15, Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Mon, 2 Mar 2015, Reindl Harald wrote: >> Am 02.03.2015 um 10:06 schrieb Steffen Kaiser: >>> If such plugin(?) is available, I would expect immediate complains, it >>> does not support: >>> >>> + local file lists with various sets of syntaxes >>> + RBLs with a fine grained response matching >>> + use the same RBL response for multiple match-action pairs >> >> or it could work just with no config, unconditional and > > therefore I wrote, that I expect complains, if this feature would work > like that > >> in front of any >> authentication, > > what is that same as to place it as first passdb, with the overhead of > parsing the config file and adding it into the passdb{} chain. > >> frankly even without any response - connection -> RBL >> check >> -> close connection, done > > some external RBLs return certain information in the response, e.g. > 127.0.0.2 is less problematic than 127.0.0.1, so "I expect complains" this > or that RBL is not working correctly ;-) > >> hence RBL's make sense in the core because *in front* of any other >> protocol >> specific code > > That's TCP wrapper or a firewall, IMHO. (for a file list, not RBL). > However, there used to be a RBL patch for TCP wrapper and some > distribution provide other implementations of a TCP wrapper with RBL, if > this post correct: > http://grokbase.com/t/centos/centos/143mg1wxsj/does-anyone-use-tcp-wrappers-hosts-allow-hosts-deny-anymore > > - -- > Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVPQufHz1H7kL/d9rAQKC3wf/ZuStrHInsV3OkgDC5EDBeSyvMOxlskiy > xCNUeAxaqPt4DvgCHnXmXX3V2yi+hXvsFyWhIBcsJcgUvbi0sJWwy7Undw2Fs6Cf > iaOD3+u1VV+7IwiiZIMNMpUcDisj9Ic3DBoDTx9SeyBS09i7lKAVORZw486LooWX > uTCMZOEmzH43DEfHxmIMPMcyQBF4b7kzc3A/sabpc70bhrJAV8E2ZNpPzIyAiC3A > PwjUR+YfdYoorqz79ymmzcngsUUSAXfiUAhJpRyVOL2UiMurjROdsU5vSpXJm71j > lgELgKpo6DkIjX+qAPVtdPu/J6cRLUcfvysNezU2vV9KpgJk97cwmw== > =2nvt > -----END PGP SIGNATURE----- > From paolo.cravero at csi.it Tue Mar 3 15:36:37 2015 From: paolo.cravero at csi.it (Paolo Cravero) Date: Tue, 3 Mar 2015 16:36:37 +0100 (CET) Subject: acl and lazy_expunge plugins are incompatible? Message-ID: <1684658858.809710.1425396997470.JavaMail.open-xchange@comunica.csi.it> Hello list. Following two previous unanswered requests (http://www.dovecot.org/list/dovecot/2014-August/097449.html and http://www.dovecot.org/list/dovecot/2014-May/096261.html), I fell into the same problem: ACL and lazy_expunge plugins do not work together. I had lazy_expunge enabled working OK. Then I activated ACL plugin, and gave dovecot.2 visibily over paolo.cravero's Inbox: the IMAP connection fails on server-side. Relevant debug log is: Mar 3 15:59:02 tst-msg03 dovecot: imap(dovecot.2 at csi.it): Debug: Quota grace: root=User quota bytes=576716800 (10%) Mar 3 15:59:02 tst-msg03 dovecot: imap(dovecot.2 at csi.it): Debug: dict quota: user=paolo.cravero at csi.it, uri=file:/maildata1/home/csi.it/p/paolo.cravero/dovecot-quota, noenforcing=0 Mar 3 15:59:02 tst-msg03 dovecot: imap(dovecot.2 at csi.it): Debug: fs: root=/maildata1/main/csi.it/p/paolo.cravero, index=/maildata1/indexes/csi.it/p/paolo.cravero, indexpvt=, control=, inbox=, alt=/maildata1/altstorage/csi.it/p/paolo.cravero Mar 3 15:59:02 tst-msg03 dovecot: imap(dovecot.2 at csi.it): Debug: acl: initializing backend with data: vfile Mar 3 15:59:02 tst-msg03 dovecot: imap(dovecot.2 at csi.it): Debug: acl: acl username = paolo.cravero at csi.it Mar 3 15:59:02 tst-msg03 dovecot: imap(dovecot.2 at csi.it): Debug: acl: owner = 1 Mar 3 15:59:02 tst-msg03 dovecot: imap(dovecot.2 at csi.it): Debug: acl vfile: Global ACLs disabled Mar 3 15:59:02 tst-msg03 dovecot: imap(dovecot.2 at csi.it): Fatal: lazy_expunge: Unknown namespace: '.EXPUNGED/' Removing the lazy_expunge, ACLs do work as expected and I can add/remove/use them. Is there a known compatibility issue? Or there's something wrong in my config and someone else is running with both plugins enabled? Thanks, Paolo PS: output of dovecot -n: # 2.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.5 (Santiago) ext3 auth_mechanisms = plain login disable_plaintext_auth = no imap_client_workarounds = tb-extra-mailbox-sep lmtp_save_to_detail_mailbox = yes mail_attachment_dir = /maildata1/attachments mail_attachment_hash = %{sha256} mail_debug = yes mail_gid = 500 mail_location = mdbox:/maildata1/main/%d/%1n/%n:ALT=/maildata1/altstorage/%d/%1n/%n:INDEX=/maildata1/indexes/%d/%1n/%n mail_plugins = quota mail_log notify lazy_expunge acl mail_uid = 500 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate mbox_write_locks = fcntl mdbox_rotate_size = 50 M namespace { hidden = no list = yes location = mdbox:/maildata1/main/%d/%1n/%n:INDEX=/maildata1/indexes/%d/%1n/%n:MAILBOXDIR=expunged prefix = .EXPUNGED/ subscriptions = no } namespace { list = children location = mdbox:/maildata1/main/%%d/%%1n/%%n:ALT=/maildata1/altstorage/%%d/%%1n/%%n:INDEX=/maildata1/indexes/%%d/%%1n/%%n prefix = user/%%u/ separator = / type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile acl_shared_dict = file:/maildata1/db/shared-mailboxes.db lazy_expunge = .EXPUNGED/ mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_fields = uid box msgid size flags vsize quota = dict:User quota::file:%h/dovecot-quota quota_grace = 10%% quota_rule = *:storage=10G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = vmail user = vmail } } service lmtp { inet_listener lmtp { address = 10.102.42.114 port = 24 } process_min_avail = 2 user = vmail } ssl_cert = This seems simple enough...I'm just not script wizard. If someone can throw together a starting point I can test and tweak it from there. It seems to me: 1. Read /var/mail/mail.err or specified logfile 2. For each "failed: read(/var/mail/attachments/aa/bb/attachmentHash-userHash" line, a. Confirm /var/mail/attachments/aa/bb/hashes/attachmentHash exists i. If attachmentHash is missing display such for possible backup searching. b. create link attachmentHash-userHash to hashes/attachmentHash 3. Continue to end of file Can this be done via "pure" BASH? Need sed/awk as well? -- Daniel From mail at oliwel.de Tue Mar 3 21:31:12 2015 From: mail at oliwel.de (Oliver Welter) Date: Tue, 03 Mar 2015 22:31:12 +0100 Subject: IP drop list In-Reply-To: <54F59D99.4070507@neurotica.com> References: <54F59D99.4070507@neurotica.com> Message-ID: <54F62820.6040304@oliwel.de> Am 03.03.2015 um 12:40 schrieb Dave McGuire: > On 03/02/2015 09:41 PM, Joseph Tam wrote: >>>>>> then setup fail2ban to manage extrafields >>>>> >>>>> Now that's a very interesting idea, thank you! I will investigate >>>>> this. >>>> >>>> If you don't expect yor firewall to handle 45K+ IPs, I'm not how you >>>> expect dovecot will handle a comma separated string with 45K+ entries >>>> any better. >>> >>> My firewall can handle that without breaking a sweat. I just haven't >>> found a way (that I'm comfortable with) to automatically inject rules >>> into it from a machine on the network. >>> >>> Doing it via a DNSBL is an elegant solution to the problem, IMO. >> >> I'm agnostic as far as which method you want to use. All I'm saying is >> that using dovecot's allow_net facility is as difficult, if not >> more so, than letting your firewall handle it. > > I'm not disagreeing with you. As I stated above, getting new rules > into my firewall in an automated way is not something I've found a good > way to do yet. Granted, it has been a couple of years since I've > googled around to see if anyone has been able to do it in a reasonably > secure way. (Perhaps it's time for me to revisit that.) > I did a quick hack for exactly this purpose - send offending IPs from my mail server to the firewall "in a secure way". Its a python script that uses the fail2ban syntax on the one end and feeds a (patched) pfSense on the other end. You can find the scripts on github: https://github.com/oliwel/fail2sense - be warned, its a first draft - but it does the job here...For the unblock feature you need this patch against pfsense https://github.com/pfsense/pfsense/pull/1444/ Oli -- Protect your environment - close windows and adopt a penguin! -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4074 bytes Desc: S/MIME Cryptographic Signature URL: From h.reindl at thelounge.net Tue Mar 3 21:43:24 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Tue, 03 Mar 2015 22:43:24 +0100 Subject: IP drop list In-Reply-To: <54F62820.6040304@oliwel.de> References: <54F59D99.4070507@neurotica.com> <54F62820.6040304@oliwel.de> Message-ID: <54F62AFC.8010200@thelounge.net> Am 03.03.2015 um 22:31 schrieb Oliver Welter: > I did a quick hack for exactly this purpose - send offending IPs from my > mail server to the firewall "in a secure way". Its a python script that > uses the fail2ban syntax on the one end and feeds a (patched) pfSense on > the other end. You can find the scripts on github: > https://github.com/oliwel/fail2sense - be warned, its a first draft - > but it does the job here...For the unblock feature you need this patch > against pfsense https://github.com/pfsense/pfsense/pull/1444/ the problem is the "in a secure way" that's not really possible when you mangle firewall rules which implies root permissions - as RBL request is just a DNS request which don't need *any* permissions on the machine which does the request the other problem is mangle firewall rules in context of existing infrastructures is error prone - you may interfere existing rulesets - it's a bad idea to start with -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From voytek at sbt.net.au Wed Mar 4 00:44:51 2015 From: voytek at sbt.net.au (voytek at sbt.net.au) Date: Wed, 4 Mar 2015 11:44:51 +1100 Subject: ot: maildir has overdrawn his diskspace quota Message-ID: <9cdaf6abe4d8a4b6b8596f62d993e514.squirrel@sbt.net.au> I have Postfix/Dovecot with virtual domains, same setup unaltered since server was set up quite a while ago last month, added a new virtual domain, 'just like before'. but, today noticed this in the queue/log 'overdrawn his diskspace quota'[1]: user's Maildir cur has like 48,762,696 bytes (lot less than other users) tried some searches, found similar probs, but, no solution as yet any ideas ? after physically removing some emails from this user's 'cur', I re-queued 'stuck' message, it was then accepted # quota -g vmail # quota -u vmail # [1] # grep 719855FC42 /var/log/maillog Mar 3 15:20:58 emu postfix/smtpd[18535]: 719855FC42: client=mail-pa0-f51.google.com[209.85.220.51] Mar 3 15:20:58 emu postfix/cleanup[18631]: 719855FC42: message-id=<54f536ad.21da440a.6572.3c07 at mx.google.com> Mar 3 15:21:01 emu postfix/qmgr[1906]: 719855FC42: from=, size=5487190, nrcpt=1 (queue active) Mar 3 15:21:01 emu postfix/virtual[20248]: 719855FC42: to=, relay=virtual, delay=3.5, delays=3.4/0/0/0.08, dsn=4.2.2, status=deferred (maildir delivery failed: Sorry, the user's maildir has overdrawn his diskspace quota, please try again later.) Mar 3 15:27:53 emu postfix/qmgr[1906]: 719855FC42: from=, size=5487190, nrcpt=1 (queue active) Mar 3 15:27:53 emu postfix/virtual[23352]: 719855FC42: to=, relay=virtual, delay=415, delays=415/0.01/0/0.07, dsn=4.2.2, status=deferred (maildir delivery failed: Sorry, the user's maildir has overdrawn his diskspace quota, please try again later.) From dovecot at lists.killian.com Wed Mar 4 04:03:52 2015 From: dovecot at lists.killian.com (Earl Killian) Date: Tue, 03 Mar 2015 20:03:52 -0800 Subject: IP drop list In-Reply-To: <54F4A5D4.6070303@thelounge.net> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> Message-ID: <54F68428.9060700@lists.killian.com> On 2015/3/2 10:03, Reindl Harald wrote: > > that is all nice > > but the main benefit of RBL's is always ignored: > > * centralized > * no log parsing at all > * honeypot data are "delivered" to any host > * it's cheap > * it's easy to maintain > * it don't need any root privileges anywhere > > we have a small honeypot network with a couple of ipranges detecting > mass port-scans and so on and this data are available *everywhere* > > so if some IP hits there it takes 60 seconds and any service > supportings DNS blacklists can block them *even before* the bot hits > the real mailserver at all > I would like to reiterate Reindl Harald's point above, since subsequent discussion has gotten away from it. If Dovecot had DNS RBL support similar to Postfix, I think quite a few people would use it, and thereby defeat the scanners far more effectively than any other method. It is good that other people are suggesting things that will work today, but in terms of what new feature would be the best solution, I can't think of one better than a DNS RBL. From dmiller at amfes.com Wed Mar 4 04:26:06 2015 From: dmiller at amfes.com (Daniel Miller) Date: Tue, 03 Mar 2015 20:26:06 -0800 Subject: Rebuilding SIS attachment links from log In-Reply-To: <54F622EA.9050203@amfes.com> References: <54F622EA.9050203@amfes.com> Message-ID: Well, with no guarantees or promises whatsoever...here's my first attempt. I'm certain someone else can come up with a much more robust solution but it's a starting point. Since I typically see a batch of the error messages during an FTS update, my current usage is: 1. doveadm fts rescan -u user-to-fix 2. Perform a fts search to do a full mailbox scan 3. Check the mail log to see the date/time of the errors (I usually have a window open with tail -f) 4. Something like "grep 'Mar 3 20:17' mail.log > mail.err" gives me a starting point execute dovesisfix and see if it helps. -- Daniel On 3/3/2015 1:08 PM, Daniel Miller wrote: > This seems simple enough...I'm just not script wizard. If someone can > throw together a starting point I can test and tweak it from there. > It seems to me: > > 1. Read /var/mail/mail.err or specified logfile > 2. For each "failed: > read(/var/mail/attachments/aa/bb/attachmentHash-userHash" line, > a. Confirm /var/mail/attachments/aa/bb/hashes/attachmentHash exists > i. If attachmentHash is missing display such for possible > backup searching. > b. create link attachmentHash-userHash to hashes/attachmentHash > 3. Continue to end of file > > Can this be done via "pure" BASH? Need sed/awk as well? > -------------- next part -------------- #!/bin/bash # These variables need to be customized for your particular installation LOGFILE='/var/log/mail.err' ATTACHMENT_STORAGE_BASE='/var/mail/attachments' # These variables are based on current Dovecot behaviour and should not require changing HASH_FOLDER='hashes' # Initialization PREVIOUS_ERR='' ERR='' function usage { echo "Dovecot Single-Instance-Storage Attachment Repair" echo "usage: dovesisfix [-d] [-t] [-v] [-h]" echo " -t | --test-only perform logfile analysis and show steps to be taken without any on-disk modification" echo " -v | --verbose provide verbose messages at each step" echo " -d | --debug provide additional debug messages" echo " -h | --help this screen" } while [ "$1" != "" ]; do case $1 in -d | --debug ) DEBUG=1 VERBOSE=1 ;; -t | --test-only ) TESTMODE=1 ;; -v | --verbose ) VERBOSE=1 ;; -h | --help ) usage exit ;; * ) usage exit 1 esac shift done while read -r LINE do ERR=$LINE # Format of log line has date, host, process, user, mail storage file, and then the # attachment path failure, followed by a duplicate of the path as an argument to open, # and then final details. # Verify this line is indeed a dovecot attachment error. Don't look for "dovecot" specifically # in case that name was changed - but the individual worker names are probably safe searches. # So we test against "attachments-connector" - hopefully that's good enough. TEST=$(echo "$ERR" | sed -n "s|.*attachments-connector.*|1|p") if [ "$TEST" != "1" ]; then # Not found - not relevant if [ "$DEBUG" = 1 ]; then echo "Skipping non-relevant log line. $ERR" fi continue fi # Remove prefacing details from log line - find start of attachment path within log line # This is a greedy match - so the second attachment path is returned along with the trailing info ATTACH_LINE_FILTER="s|.*$ATTACHMENT_STORAGE_BASE||" ATTACH_LINE=$(echo "$ERR" | sed "$ATTACH_LINE_FILTER") # Now extract the aa/bb/ prefix, the base attachment file name, and user hash CATEGORY_PATH="${ATTACH_LINE:1:5}" BASE_HASH="${ATTACH_LINE:7:40}" USER_HASH="${ATTACH_LINE:48:32}" ATTACH_SOURCE="$ATTACHMENT_STORAGE_BASE/$CATEGORY_PATH/$HASH_FOLDER/$BASE_HASH" ATTACH_TARGET="$ATTACHMENT_STORAGE_BASE/$CATEGORY_PATH/$BASE_HASH-$USER_HASH" # There appear to be duplicate lines - so to try to filter some out. if [ "$PREVIOUS_TARGET" = "$ATTACH_TARGET" ]; then if [ "$DEBUG" = 1 ]; then echo "Skipping duplicate log line for $ATTACH_SOURCE-$ATTACH_TARGET" fi continue fi PREVIOUS_TARGET=$ATTACH_TARGET # If in debug/verbose mode show operation about to occur if [ "$VERBOSE" = 1 ]; then echo "The file $ATTACH_SOURCE must be linked to $ATTACH_TARGET" fi # Verify that source exists if [ ! -f "$ATTACH_SOURCE" ]; then echo "ERROR: File $ATTACH_SOURCE does not exist. You must restore this from a backup and run this utility again." fi # This is a Good Thing. if [ "$DEBUG" = 1 ]; then echo "The file $ATTACH_SOURCE appears to be a valid file." fi # Check if user link mysteriously reappeared if [ -f "$ATTACH_TARGET" ]; then echo "INFO: File $ATTACH_TARGET exists. This may mean the fault has been previously corrected. Clearing/rotating the logfile $LOGFILE is appropriate now." continue fi # Prepare to create user link LINK_LINE="$ATTACH_SOURCE $ATTACH_TARGET" if [ "$DEBUG" = 1 ]; then echo "About to execute command: ln $LINK_LINE" fi # If test mode, do nothing if [ "$TESTMODE" = 1 ]; then continue fi # There's probably more tests I could/should do - but I don't know how # So...if we're not in test mode...time to do it to it. LINK_CREATED=$(ln $LINK_LINE) if [ "$VERBOSE" = 1 ]; then echo "Repair result for $ATTACH_TARGET - $LINK_CREATED" fi done < "$LOGFILE" From paolo.cravero at csi.it Wed Mar 4 08:23:51 2015 From: paolo.cravero at csi.it (Paolo Cravero) Date: Wed, 4 Mar 2015 09:23:51 +0100 (CET) Subject: Rebuilding SIS attachment links from log In-Reply-To: References: Message-ID: <2061025091.820810.1425457431330.JavaMail.open-xchange@comunica.csi.it> Daniel, please help me understand, since I met your same problem on 2015-02-26 as you did. > 2. For each "failed: > read(/var/mail/attachments/aa/bb/attachmentHash-userHash" line, > a. Confirm /var/mail/attachments/aa/bb/hashes/attachmentHash exists > i. If attachmentHash is missing display such for possible Is this a FEATURE of Dovecot SIS? Or a known bug described somewhere? Loosing detached attachments, I mean. > Can this be done via "pure" BASH? Need sed/awk as well? Well, you need a way to strip out the "-userHash" part from the string, so a little bit of sed and regex is needed. You also need to work out source and destination files. Still, personally I think the problem is somewhere else and should be fixed. Else, stop using SIS and let the SAN do the deduplication. Am I wrong? Paolo From david.myers.24j74 at gmail.com Wed Mar 4 08:30:15 2015 From: david.myers.24j74 at gmail.com (David Myers) Date: Wed, 4 Mar 2015 09:30:15 +0100 Subject: IP drop list In-Reply-To: <54F68428.9060700@lists.killian.com> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> Message-ID: hi all I've been reading this thread with interest. As a rather novice programmer. I'm not being humble here, I really am not very good, I can do stuff, but it takes a LONG time. My spaghetti code even has meatballs in it ! Not being a great programmer I'm not really able to code something up, but it occurred to me something could be scripted, are the other posters suggesting something like the following .... It does use fail2ban, which I understand isn't the ideal solution, but in brief.... extract the IP's from the fail to ban log file (or any other log file if you so desire). Use these to push up to the firewall or insert into your dovecot 'if' statement (which programmatically even I could probably manage ;) ) I understand that this wasn't exactly what the OP was looking for but creating the 'if' on the fly, as it were , is certainly better than putting the values in manually . An outline for the first part, extracting the ips from a log file, if anyone is interested can be found here. http://www.the-art-of-web.com/system/fail2ban-log/ The second bit, adding in the values to the if statement, shouldn't be that hard... I could probably push something out in Java (but that would obviously not be any good for anyone!), maybe even PERL it would take me longer, at a push even a bash script... (I feel like my hair is going grey ;) ... Maybe even a good bash project for me as a beginner. Just a question to see if I am understanding the general preposition of this thread. thanks for you time, and to helping me to learn this stuff. David On 4 Mar 2015 05:04, "Earl Killian" wrote: > On 2015/3/2 10:03, Reindl Harald wrote: > >> >> that is all nice >> >> but the main benefit of RBL's is always ignored: >> >> * centralized >> * no log parsing at all >> * honeypot data are "delivered" to any host >> * it's cheap >> * it's easy to maintain >> * it don't need any root privileges anywhere >> >> we have a small honeypot network with a couple of ipranges detecting mass >> port-scans and so on and this data are available *everywhere* >> >> so if some IP hits there it takes 60 seconds and any service supportings >> DNS blacklists can block them *even before* the bot hits the real >> mailserver at all >> >> I would like to reiterate Reindl Harald's point above, since subsequent > discussion has gotten away from it. If Dovecot had DNS RBL support similar > to Postfix, I think quite a few people would use it, and thereby defeat the > scanners far more effectively than any other method. It is good that other > people are suggesting things that will work today, but in terms of what new > feature would be the best solution, I can't think of one better than a DNS > RBL. > From nick.z.edwards at gmail.com Wed Mar 4 14:24:45 2015 From: nick.z.edwards at gmail.com (Nick Edwards) Date: Thu, 5 Mar 2015 00:24:45 +1000 Subject: IP drop list In-Reply-To: References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> Message-ID: You can script fail2ban to send the entries to a rbldnsd file on a remote server, I know someone who does it based on apache, since it uses fail2ban, i shouldnt matter if its apache, or dovecot. I thought Timo once said dovecot had tarpitting, its useless if it is there, and if it is, it needs user configurable timings, or maybe its one of those things thats been in the gunna happen list for a long time, like other stuff On 3/4/15, David Myers wrote: > hi all > > I've been reading this thread with interest. As a rather novice programmer. > I'm not being humble here, I really am not very good, I can do stuff, but > it takes a LONG time. My spaghetti code even has meatballs in it ! > > Not being a great programmer I'm not really able to code something up, but > it occurred to me something could be scripted, are the other posters > suggesting something like the following .... > > It does use fail2ban, which I understand isn't the ideal solution, but in > brief.... > > extract the IP's from the fail to ban log file (or any other log file if > you so desire). > Use these to push up to the firewall or insert into your dovecot 'if' > statement (which programmatically even I could probably manage ;) ) > > I understand that this wasn't exactly what the OP was looking for but > creating the 'if' on the fly, as it were , is certainly better than putting > the values in manually . > > An outline for the first part, extracting the ips from a log file, if > anyone is interested can be found here. > > http://www.the-art-of-web.com/system/fail2ban-log/ > > The second bit, adding in the values to the if statement, shouldn't be that > hard... I could probably push something out in Java (but that would > obviously not be any good for anyone!), maybe even PERL it would take me > longer, at a push even a bash script... (I feel like my hair is going grey > ;) ... > > Maybe even a good bash project for me as a beginner. > > Just a question to see if I am understanding the general preposition of > this thread. > > thanks for you time, and to helping me to learn this stuff. > > David > > > On 4 Mar 2015 05:04, "Earl Killian" wrote: > >> On 2015/3/2 10:03, Reindl Harald wrote: >> >>> >>> that is all nice >>> >>> but the main benefit of RBL's is always ignored: >>> >>> * centralized >>> * no log parsing at all >>> * honeypot data are "delivered" to any host >>> * it's cheap >>> * it's easy to maintain >>> * it don't need any root privileges anywhere >>> >>> we have a small honeypot network with a couple of ipranges detecting >>> mass >>> port-scans and so on and this data are available *everywhere* >>> >>> so if some IP hits there it takes 60 seconds and any service supportings >>> DNS blacklists can block them *even before* the bot hits the real >>> mailserver at all >>> >>> I would like to reiterate Reindl Harald's point above, since subsequent >> discussion has gotten away from it. If Dovecot had DNS RBL support >> similar >> to Postfix, I think quite a few people would use it, and thereby defeat >> the >> scanners far more effectively than any other method. It is good that >> other >> people are suggesting things that will work today, but in terms of what >> new >> feature would be the best solution, I can't think of one better than a >> DNS >> RBL. >> > From nick.z.edwards at gmail.com Wed Mar 4 14:30:38 2015 From: nick.z.edwards at gmail.com (Nick Edwards) Date: Thu, 5 Mar 2015 00:30:38 +1000 Subject: IP drop list In-Reply-To: <54F68428.9060700@lists.killian.com> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> Message-ID: On 3/4/15, Earl Killian wrote: > On 2015/3/2 10:03, Reindl Harald wrote: >> >> that is all nice >> >> but the main benefit of RBL's is always ignored: >> >> * centralized >> * no log parsing at all >> * honeypot data are "delivered" to any host >> * it's cheap >> * it's easy to maintain >> * it don't need any root privileges anywhere >> >> we have a small honeypot network with a couple of ipranges detecting >> mass port-scans and so on and this data are available *everywhere* >> >> so if some IP hits there it takes 60 seconds and any service >> supportings DNS blacklists can block them *even before* the bot hits >> the real mailserver at all >> > I would like to reiterate Reindl Harald's point above, since subsequent > discussion has gotten away from it. If Dovecot had DNS RBL support > similar to Postfix, I think quite a few people would use it, and thereby > defeat the scanners far more effectively than any other method. It is > good that other people are suggesting things that will work today, but > in terms of what new feature would be the best solution, I can't think > of one better than a DNS RBL. > A people argued for this before, like a few argued for postfix-style failover for mysql database lookups, both are welcome features by Timo from memory, but only if someone else codes it up, otherwise, it will probably snow in Dubai before Timo does it, just look at how scarce he is here, we were getting bugzilla 18 months ago, and as the song goes "still waiting, still waiting" :-> From Jochen.Bern at LINworks.de Wed Mar 4 16:06:54 2015 From: Jochen.Bern at LINworks.de (Jochen Bern) Date: Wed, 04 Mar 2015 17:06:54 +0100 Subject: IP drop list In-Reply-To: <54F68428.9060700@lists.killian.com> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> Message-ID: <54F72D9E.6010702@LINworks.de> On 03/04/2015 05:03 AM, Earl Killian wrote: > I would like to reiterate Reindl Harald's point above, since subsequent > discussion has gotten away from it. If Dovecot had DNS RBL support > similar to Postfix, I think quite a few people would use it, and thereby > defeat the scanners far more effectively than any other method. It is > good that other people are suggesting things that will work today, but > in terms of what new feature would be the best solution, I can't think > of one better than a DNS RBL. I've *seen* mailservers after an external DNSBL configured into them became defunct or unreachable, and "better", much less "the best solution", is not how *I* would rank the result in comparison to local rate limiting. (Note that, unlike in the case of spam and SMTP, allowing a couple POP/IMAP connection attempts until the limit strikes is unlikely to become visible to the legit userbase.) Which is not to say that such a feature should not be implemented - after all, Jim said that he compiled the 45k list *himself*, so it would be a *locally administered* DNSBL for him. On 03/03/2015 10:43 PM, Reindl Harald wrote: > the problem is the "in a secure way" > > that's not really possible when you mangle firewall rules which implies > root permissions - as RBL request is just a DNS request which don't need > *any* permissions on the machine which does the request > > the other problem is mangle firewall rules in context of existing > infrastructures is error prone - you may interfere existing rulesets > - it's a bad idea to start with That's a lot of smoke you're blowing at a firewall that hasn't been specified beyond "it's *not* iptables". FWIW, *if* it were iptables, something along the lines of "-d myserver --dport 993 --state NEW -j (NF)QUEUE" would happily pass *only* the incoming IMAPS connections to a decision-maker running in userspace. Regards, J. Bern -- *NEU* - NEC IT-Infrastruktur-Produkte im : Server--Storage--Virtualisierung--Management SW--Passion for Performance Jochen Bern, Systemingenieur --- LINworks GmbH Postfach 100121, 64201 Darmstadt | Robert-Koch-Str. 9, 64331 Weiterstadt PGP (1024D/4096g) FP = D18B 41B1 16C0 11BA 7F8C DCF7 E1D5 FAF4 444E 1C27 Tel. +49 6151 9067-231, Zentr. -0, Fax -299 - Amtsg. Darmstadt HRB 85202 Unternehmenssitz Weiterstadt, Gesch?ftsf?hrer Metin Dogan, Oliver Michel From adrian.minta at gmail.com Wed Mar 4 16:13:31 2015 From: adrian.minta at gmail.com (Adrian Minta) Date: Wed, 04 Mar 2015 18:13:31 +0200 Subject: New FREAK SSL Attack CVE-2015-0204 Message-ID: <54F72F2B.6000503@gmail.com> Hello, about the CVE-2015-0204, in apache the following config seems to disable this vulnerability: SSLProtocol All -SSLv2 -SSLv3 SSLCipherSuite HIGH:MEDIUM:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4 Is something similar possible with dovecot ? If yes, what are the implications with old mail clients ? -- Best regards, Adrian Minta From manu at netbsd.org Wed Mar 4 16:19:47 2015 From: manu at netbsd.org (Emmanuel Dreyfus) Date: Wed, 4 Mar 2015 16:19:47 +0000 Subject: New FREAK SSL Attack CVE-2015-0204 In-Reply-To: <54F72F2B.6000503@gmail.com> References: <54F72F2B.6000503@gmail.com> Message-ID: <20150304161947.GL15625@homeworld.netbsd.org> On Wed, Mar 04, 2015 at 06:13:31PM +0200, Adrian Minta wrote: > Hello, > about the CVE-2015-0204, in apache the following config seems to disable > this vulnerability: > SSLProtocol All -SSLv2 -SSLv3 > SSLCipherSuite > HIGH:MEDIUM:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4 > > Is something similar possible with dovecot ? I use this with some succes: # dovecot has built-in protection against BEAST, therefore no need # to remove -SSLv2-SHA1:-TLSv10-SHA1 ssl_protocols = !SSLv2 !SSLv3 ssl_cipher_list = ECDH at STRENGTH:DH at STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNULL I only had a single report of an old client being locked out. Oddly it was a recent Windows Phone that was perfectly capable of using latest protocol and ciphers. While there, I will self advertise my own paper on TLS hardening: http://arxiv.org/abs/1407.2168 -- Emmanuel Dreyfus manu at netbsd.org From adrian.minta at gmail.com Wed Mar 4 16:36:07 2015 From: adrian.minta at gmail.com (Adrian Minta) Date: Wed, 04 Mar 2015 18:36:07 +0200 Subject: New FREAK SSL Attack CVE-2015-0204 In-Reply-To: <20150304161947.GL15625@homeworld.netbsd.org> References: <54F72F2B.6000503@gmail.com> <20150304161947.GL15625@homeworld.netbsd.org> Message-ID: <54F73477.3070001@gmail.com> On 04.03.2015 18:19, Emmanuel Dreyfus wrote: > On Wed, Mar 04, 2015 at 06:13:31PM +0200, Adrian Minta wrote: >> Hello, >> about the CVE-2015-0204, in apache the following config seems to disable >> this vulnerability: >> SSLProtocol All -SSLv2 -SSLv3 >> SSLCipherSuite >> HIGH:MEDIUM:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4 >> >> Is something similar possible with dovecot ? > I use this with some succes: > > # dovecot has built-in protection against BEAST, therefore no need > # to remove -SSLv2-SHA1:-TLSv10-SHA1 > ssl_protocols = !SSLv2 !SSLv3 > ssl_cipher_list = ECDH at STRENGTH:DH at STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNULL > > I only had a single report of an old client being locked out. Oddly it > was a recent Windows Phone that was perfectly capable of using > latest protocol and ciphers. > > While there, I will self advertise my own paper on TLS hardening: > http://arxiv.org/abs/1407.2168 > Thank you for the answer. The "!EXPORT" part is included in "ECDH at STRENGTH:DH at STRENGTH:HIGH", or it must be added as well ? -- Best regards, Adrian Minta From h.reindl at thelounge.net Wed Mar 4 16:36:19 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 04 Mar 2015 17:36:19 +0100 Subject: IP drop list In-Reply-To: <54F72D9E.6010702@LINworks.de> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> <54F72D9E.6010702@LINworks.de> Message-ID: <54F73483.6080105@thelounge.net> Am 04.03.2015 um 17:06 schrieb Jochen Bern: > On 03/04/2015 05:03 AM, Earl Killian wrote: >> I would like to reiterate Reindl Harald's point above, since subsequent >> discussion has gotten away from it. If Dovecot had DNS RBL support >> similar to Postfix, I think quite a few people would use it, and thereby >> defeat the scanners far more effectively than any other method. It is >> good that other people are suggesting things that will work today, but >> in terms of what new feature would be the best solution, I can't think >> of one better than a DNS RBL. > > I've *seen* mailservers after an external DNSBL configured into them > became defunct or unreachable, and "better", much less "the best > solution", is not how *I* would rank the result in comparison to local > rate limiting. (Note that, unlike in the case of spam and SMTP, allowing > a couple POP/IMAP connection attempts until the limit strikes is > unlikely to become visible to the legit userbase.) > > Which is not to say that such a feature should not be implemented - > after all, Jim said that he compiled the 45k list *himself*, so it would > be a *locally administered* DNSBL for him. surely - and *that* was my whole point, nobody talked about using spamhaus or DUL RBL's on a IMAP/POP3 my feature request last year was *because i have* already a rbldnsd which is used in postfix and on webserver with mod_security and i find it strange that i can't stop a dictionary attack faced on SMTP to continue on POP3/IMAP after locked out from postfix without write firewall rules the whole point of a *locally administered* RBL is that you don't need to care about hown many mailservers you have and where they are nor need you to open security holes between them for sharing data > On 03/03/2015 10:43 PM, Reindl Harald wrote: >> the problem is the "in a secure way" >> >> that's not really possible when you mangle firewall rules which implies >> root permissions - as RBL request is just a DNS request which don't need >> *any* permissions on the machine which does the request >> >> the other problem is mangle firewall rules in context of existing >> infrastructures is error prone - you may interfere existing rulesets >> - it's a bad idea to start with > > That's a lot of smoke you're blowing at a firewall that hasn't been > specified beyond "it's *not* iptables". > > FWIW, *if* it were iptables, something along the lines of "-d myserver > --dport 993 --state NEW -j (NF)QUEUE" would happily pass *only* the > incoming IMAPS connections to a decision-maker running in userspace. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From manu at netbsd.org Wed Mar 4 16:53:10 2015 From: manu at netbsd.org (Emmanuel Dreyfus) Date: Wed, 4 Mar 2015 16:53:10 +0000 Subject: New FREAK SSL Attack CVE-2015-0204 In-Reply-To: <54F73477.3070001@gmail.com> References: <54F72F2B.6000503@gmail.com> <20150304161947.GL15625@homeworld.netbsd.org> <54F73477.3070001@gmail.com> Message-ID: <20150304165310.GM15625@homeworld.netbsd.org> On Wed, Mar 04, 2015 at 06:36:07PM +0200, Adrian Minta wrote: > Thank you for the answer. > The "!EXPORT" part is included in "ECDH at STRENGTH:DH at STRENGTH:HIGH", or it > must be added as well ? This is not the cipher list I sent. It was: ECDH at STRENGTH:DH at STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNUL Mine does not contain any export cipher, yours does. You can use openssl ciphers to compare cipher lists: $ openssl ciphers EXPORT|tr ':' '\n' |sort > export $ openssl ciphers ECDH at STRENGTH:DH at STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNULL \ |tr ':' '\n' |sort> manu $ openssl ciphers ECDH at STRENGTH:DH at STRENGTH:HIGH |tr ':' '\n' |sort > adrian $ join export manu (nothing) $ join export adrian EXP-ADH-DES-CBC-SHA EXP-ADH-RC4-MD5 EXP-EDH-DSS-DES-CBC-SHA EXP-EDH-RSA-DES-CBC-SHA -- Emmanuel Dreyfus manu at netbsd.org From adrian.minta at gmail.com Wed Mar 4 17:14:28 2015 From: adrian.minta at gmail.com (Adrian Minta) Date: Wed, 04 Mar 2015 19:14:28 +0200 Subject: New FREAK SSL Attack CVE-2015-0204 In-Reply-To: <20150304165310.GM15625@homeworld.netbsd.org> References: <54F72F2B.6000503@gmail.com> <20150304161947.GL15625@homeworld.netbsd.org> <54F73477.3070001@gmail.com> <20150304165310.GM15625@homeworld.netbsd.org> Message-ID: <54F73D74.7060401@gmail.com> On 04.03.2015 18:53, Emmanuel Dreyfus wrote: > On Wed, Mar 04, 2015 at 06:36:07PM +0200, Adrian Minta wrote: >> Thank you for the answer. >> The "!EXPORT" part is included in "ECDH at STRENGTH:DH at STRENGTH:HIGH", or it >> must be added as well ? > This is not the cipher list I sent. It was: > ECDH at STRENGTH:DH at STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNUL > > Mine does not contain any export cipher, yours does. > You can use openssl ciphers to compare cipher lists: > > $ openssl ciphers EXPORT|tr ':' '\n' |sort > export > $ openssl ciphers ECDH at STRENGTH:DH at STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNULL \ > |tr ':' '\n' |sort> manu > $ openssl ciphers ECDH at STRENGTH:DH at STRENGTH:HIGH |tr ':' '\n' |sort > adrian > $ join export manu > (nothing) > $ join export adrian > EXP-ADH-DES-CBC-SHA > EXP-ADH-RC4-MD5 > EXP-EDH-DSS-DES-CBC-SHA > EXP-EDH-RSA-DES-CBC-SHA > > I was using HIGH:MEDIUM:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4 in apache. You are using ECDH at STRENGTH:DH at STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNULL for dovecot. I didn't know how to compare both settings. Now I know, and I see that my "!EXPORT" part is covered by your "ECDH at STRENGTH:DH at STRENGTH:HIGH" part. $openssl ciphers HIGH:MEDIUM:\!aNULL:\!eNULL:\!EXPORT:\!CAMELLIA:\!DES:\!MD5:\!PSK:\!RC4 | tr ':' '\n' |sort > /tmp/adrian $openssl ciphers ECDH at STRENGTH:DH at STRENGTH:HIGH:\!RC4:\!MD5:\!DES:\!aNULL:\!eNULL | tr ':' '\n' |sort > /tmp/manu $diff adrian manu 6a7,8 > CAMELLIA128-SHA > CAMELLIA256-SHA 13a16,17 > DHE-DSS-CAMELLIA128-SHA > DHE-DSS-CAMELLIA256-SHA 20a25,26 > DHE-RSA-CAMELLIA128-SHA > DHE-RSA-CAMELLIA256-SHA 52c58,60 < SEED-SHA --- > PSK-3DES-EDE-CBC-SHA > PSK-AES128-CBC-SHA > PSK-AES256-CBC-SHA The main difference is the support for CAMELLIA and PSK. Unfortunately I don't now enough to say if is good or bad to support any of those two. Thank you ! -- Best regards, Adrian Minta From HFlor at gmx.de Wed Mar 4 17:29:00 2015 From: HFlor at gmx.de (Hardy Flor) Date: Wed, 04 Mar 2015 18:29:00 +0100 Subject: Rebuilding SIS attachment links from log In-Reply-To: References: Message-ID: <54F740DC.9030903@gmx.de> I use this script for review. For a correct result Dovecot should not run. #!/bin/bash attdir="/var/files/attachments" maildir="/var/mail" cd "$attdir" || exit 1 find -type f -printf "%P\n" | grep -v "hashes" | sort -u >"$maildir/attachments.s" cd "$maildir" || exit 2 rm "$maildir/files.ok" "$maildir/files.fail" 2>/dev/null for f in $(find -type f -name "m.*" -printf "%P\n"); do doveadm dump -t dbox "$f" | egrep "^msg.ext-ref" | while read z; do set -- $z while [ -n "$1" ]; do if [[ $1 == */* ]]; then test -r "$attdir/$1" && echo "$1" >>"$maildir/files.ok" || echo "$1" >>"$maildir/files.fail" fi shift done done done sort -u "$maildir/files.ok" >"$maildir/files.s" diff -Nu "$maildir/attachments.s" "$maildir/files.s" | tee "$maildir/files.diff" | egrep "^\-" Am 03.03.2015 um 22:08 schrieb Daniel Miller: > This seems simple enough...I'm just not script wizard. If someone can > throw together a starting point I can test and tweak it from there. > It seems to me: > > 1. Read /var/mail/mail.err or specified logfile > 2. For each "failed: > read(/var/mail/attachments/aa/bb/attachmentHash-userHash" line, > a. Confirm /var/mail/attachments/aa/bb/hashes/attachmentHash exists > i. If attachmentHash is missing display such for possible > backup searching. > b. create link attachmentHash-userHash to hashes/attachmentHash > 3. Continue to end of file > > Can this be done via "pure" BASH? Need sed/awk as well? > From emailbuilder88 at yahoo.com Wed Mar 4 19:04:40 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Wed, 4 Mar 2015 11:04:40 -0800 Subject: [OT] Extdata / Extprograms Plugins on CentOS 7? Message-ID: <1425495880.57566.YahooMailBasic@web142401.mail.bf1.yahoo.com> Hello, I want to install Dovecot Pigeonhole and use the Extdata and Extprograms plugins on CentOS 7. I prefer to install software via yum, and a reasonably new version of Dovecot is available in the CentOS repo. But according to the dovecot documentation, these plugins need to be compiled, so I don't know if yum is out of the question? * Does anyone know if either of these plugins built into the Pigeonhole that is available in the CentOS 7 extras repo? OR if there is a version of Pigeonhole available in a different 3rd party repo with what I want? (like EPEL, DAG, etc) * Is it possible to use these plugins with Pigeonhole installed from yum? Does the compilation requirement mean that I have to remove not only Pigeonhole, but also Dovecot itself from my system and build everything by hand? :-( THANK YOU!!!!!! From michael at orlitzky.com Wed Mar 4 19:12:20 2015 From: michael at orlitzky.com (Michael Orlitzky) Date: Wed, 04 Mar 2015 14:12:20 -0500 Subject: IP drop list In-Reply-To: <54F68428.9060700@lists.killian.com> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> Message-ID: <54F75914.5070001@orlitzky.com> On 03/03/2015 11:03 PM, Earl Killian wrote: > On 2015/3/2 10:03, Reindl Harald wrote: >> >> that is all nice >> >> but the main benefit of RBL's is always ignored: >> >> * centralized >> * no log parsing at all >> * honeypot data are "delivered" to any host >> * it's cheap >> * it's easy to maintain >> * it don't need any root privileges anywhere >> >> we have a small honeypot network with a couple of ipranges detecting >> mass port-scans and so on and this data are available *everywhere* >> >> so if some IP hits there it takes 60 seconds and any service >> supportings DNS blacklists can block them *even before* the bot hits >> the real mailserver at all >> > I would like to reiterate Reindl Harald's point above, since subsequent > discussion has gotten away from it. If Dovecot had DNS RBL support > similar to Postfix, I think quite a few people would use it, and thereby > defeat the scanners far more effectively than any other method. It is > good that other people are suggesting things that will work today, but > in terms of what new feature would be the best solution, I can't think > of one better than a DNS RBL. Please add this support to iptables instead of Dovecot. It's a waste of effort to code it into every application that listens on the network. Combined with "--ctstate NEW" and a chain for IMAP packets, it would be no less efficient. From felix at zandanel.me Wed Mar 4 19:19:54 2015 From: felix at zandanel.me (Felix Zandanel) Date: Wed, 4 Mar 2015 20:19:54 +0100 Subject: IP drop list In-Reply-To: <54F2DB17.2090509@thelounge.net> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> Message-ID: <7D4AE057-CC9F-4D86-A1C1-C644B2A1EB95@zandanel.me> > Am 01.03.2015 um 10:25 schrieb Reindl Harald : > Am 01.03.2015 um 08:53 schrieb Jim Pazarena: >> I wonder if there is an easy way to provide dovecot a flat text file of >> ipv4 #'s which should be ignored or dropped? >> >> I have accumulated 45,000+ IPs which routinely try dictionary and >> 12345678 password attempts. The file is too big to create firewall >> drops, and I don't want to compile with wrappers *if* dovecot has an >> easy ability to do this. If dovecot could parse a flat text file of IPs >> and drop connections it would sure put a dent in these attempts. > > hence i asked month ago for RBL support because such lists are easy to feed into http://www.corpit.ru/mjt/rbldnsd.html - sadly i got no reply than use fail2ban and what not irrelevant if there is already a local dnsbl I absolutely agree with Harald Reindl's findings on the advantages of DNSBL, but you have to see the big picture. Though I?ll speak about DNSBL a lot in this text, this is about blocking IPs in general. In my opinion, the *only* valid setup to use DNSBL are MTAs that accept mail from unauthenticated clients. That is because in such scenarios there are several heuristics you can safely use to distinguish the good from the bad. One of the most important aspects has to do with the distinction between mail submission and transmission. If you don?t want an open relay, you normally let your users authenticate before they can submit their mail. In any other case it?s safe to assume that the client is another MTA wanting to push a message over to you. We are talking about server systems here, not end users. Servers that should have a valid hostname, a static IP with no NAT in between etc. Blocking one IP in this case *should* really only block that one bad computer system. In the end, it?s perfectly OK to block clients that are either not authenticated, have no valid hostname, use dynamically assigned IPs etc. from accessing your MTA. Once having checked that one may put single IPs on a private block list to speed things up. In the case of HTTP, IMAP, etc. things are not so easy. Just think about NAT and CGN. You as a service provider *can* never know that there?s no collateral damage when you block an IP address. Every single IP out there could be a gateway to a private or even carrier grade network with hundreds or thousands of computers behind it. Some of them might be infected by malware or controlled by a bad guy. Some others might be those your clients use to download their mail. You?d lock them all out?just because you want to safe some server resources? Is it really worth it? Imagine one of your customers traveling abroad, using unusual POPs to access your dovecot instance. If the gateway IP that your server sees is blocked, you lock out your own customer. It?s the old tale. Some words of advice: (1) There?s no point in listing thousands of IPs without proper TTL. And that TTL should be short! If there really were 45 000 single /32 IPs that were behaving rude at some point in the past, how can you be sure these addresses are still doing so? Moreover, with IPv4 addresses being rare and IPv6 only being deployed slowly, CGN happens to be used more often than in the past. Even with IPv6, where prefixes were initially meant to be static, there are many ISPs that don?t give their line customers static IPv6 prefixes. That means attackers as well as your customers might end up using many different addresses over time. (2) If you run your own block list and were to add another IP, there should be sufficient knowledge about the origin of the attack. Always check the RIR whois databases, look at the delegated address range the IP is in, the country, the owner of the network, hostnames... Monitor your log files and try to detect patterns. [Honestly, I?d not be willing to invest the time ;-) ] (3) Use a scoring system. If there are other DNSBLs that list the IP or network in question, the likelihood of causing more harm than good is a bit lower than if you are the only one suffering an attack. Community based DNSBLs are commonly a good thing. You see, blocking IPs just because it?s simple and effective (for the moment) might not be what you want. I?d rather let my users choose stronger passwords, strongly enforce TLS and scale up my server systems to handle the bad traffic. Surely, it depends on your own case, just don?t be na?ve and think that blocking IPs is a general solution to anything nowadays. It might very well work for you if you don?t have a lot of customers, though. Speaking about dovecot, I doubt direct DNSBL integration will happen upstream because dovecot already supports access lookups. You can use dovecot's tcpwrap and configure your /etc/hosts.deny to lookup an external ACL program that in turn consults your DNSBL. See man hosts_options, section RUNNING OTHER COMMANDS. Look for ?aclexec?.[1] I guess that should get you on track. Just be warned that this solution (a) spawns a new tcpwrap instance for each new client connection, and (b) also spawns a new process of your custom acl program. Cheers, Felix [1] http://manpages.ubuntu.com/manpages/quantal/man5/hosts_options.5.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dmiller at amfes.com Wed Mar 4 19:15:17 2015 From: dmiller at amfes.com (Daniel Miller) Date: Wed, 04 Mar 2015 11:15:17 -0800 Subject: Rebuilding SIS attachment links from log In-Reply-To: <54F740DC.9030903@gmx.de> References: <54F622EA.9050203@amfes.com> <54F740DC.9030903@gmx.de> Message-ID: That works brilliantly! I revised my script now to take advantage of yours - #!/bin/bash # These variables need to be customized for your particular installation LISTFILE='/var/mail/files.fail' ATTACHMENT_STORAGE_BASE='/var/files/attachments' # These variables are based on current Dovecot behaviour and should not require changing HASH_FOLDER='hashes' function usage { echo "Dovecot Single-Instance-Storage Attachment Repair" echo "usage: dovesisfix [-d] [-t] [-v] [-h]" echo " -t | --test-only perform logfile analysis and show steps to be taken without any on-disk modification" echo " -v | --verbose provide verbose messages at each step" echo " -d | --debug provide additional debug messages" echo " -h | --help this screen" } while [ "$1" != "" ]; do case $1 in -d | --debug ) DEBUG=1 VERBOSE=1 ;; -t | --test-only ) TESTMODE=1 ;; -v | --verbose ) VERBOSE=1 ;; -h | --help ) usage exit ;; * ) usage exit 1 esac shift done while read -r LINE do # Now extract the aa/bb/ prefix, the base attachment file name, and user hash CATEGORY_PATH="${LINE:0:5}" BASE_HASH="${LINE:6:40}" USER_HASH="${LINE:47:32}" ATTACH_SOURCE="$ATTACHMENT_STORAGE_BASE/$CATEGORY_PATH/$HASH_FOLDER/$BASE_HASH" ATTACH_TARGET="$ATTACHMENT_STORAGE_BASE/$CATEGORY_PATH/$BASE_HASH-$USER_HASH" # If in debug/verbose mode show operation about to occur if [ "$VERBOSE" = 1 ]; then echo "The file $ATTACH_SOURCE must be linked to $ATTACH_TARGET" fi # Verify that source exists if [ ! -f "$ATTACH_SOURCE" ]; then echo "ERROR: File $ATTACH_SOURCE does not exist. You must restore this from a backup and run this utility again." fi # This is a Good Thing. if [ "$DEBUG" = 1 ]; then echo "The file $ATTACH_SOURCE appears to be a valid file." fi # Check if user link mysteriously reappeared if [ -f "$ATTACH_TARGET" ]; then echo "INFO: File $ATTACH_TARGET exists. This may mean the fault has been previously corrected. Clearing/rotating the logfile $LOGFILE is appropriate now." continue fi # Prepare to create user link LINK_LINE="$ATTACH_SOURCE $ATTACH_TARGET" if [ "$DEBUG" = 1 ]; then echo "About to execute command: ln $LINK_LINE" fi # If test mode, do nothing if [ "$TESTMODE" = 1 ]; then continue fi # There's probably more tests I could/should do - but I don't know how # So...if we're not in test mode...time to do it to it. LINK_CREATED=$(ln $LINK_LINE) if [ "$VERBOSE" = 1 ]; then echo "Repair result for $ATTACH_TARGET - $LINK_CREATED" fi done < "$LISTFILE" On 3/4/2015 9:29 AM, Hardy Flor wrote: > I use this script for review. For a correct result Dovecot should not > run. > > #!/bin/bash > > attdir="/var/files/attachments" > maildir="/var/mail" > > cd "$attdir" || exit 1 > find -type f -printf "%P\n" | grep -v "hashes" | sort -u > >"$maildir/attachments.s" > cd "$maildir" || exit 2 > rm "$maildir/files.ok" "$maildir/files.fail" 2>/dev/null > for f in $(find -type f -name "m.*" -printf "%P\n"); do > doveadm dump -t dbox "$f" | egrep "^msg.ext-ref" | while read z; do > set -- $z > while [ -n "$1" ]; do > if [[ $1 == */* ]]; then > test -r "$attdir/$1" && echo "$1" >>"$maildir/files.ok" || > echo "$1" >>"$maildir/files.fail" > fi > shift > done > done > done > sort -u "$maildir/files.ok" >"$maildir/files.s" > diff -Nu "$maildir/attachments.s" "$maildir/files.s" | tee > "$maildir/files.diff" | egrep "^\-" > > > > Am 03.03.2015 um 22:08 schrieb Daniel Miller: >> This seems simple enough...I'm just not script wizard. If someone >> can throw together a starting point I can test and tweak it from >> there. It seems to me: >> >> 1. Read /var/mail/mail.err or specified logfile >> 2. For each "failed: >> read(/var/mail/attachments/aa/bb/attachmentHash-userHash" line, >> a. Confirm /var/mail/attachments/aa/bb/hashes/attachmentHash exists >> i. If attachmentHash is missing display such for possible >> backup searching. >> b. create link attachmentHash-userHash to hashes/attachmentHash >> 3. Continue to end of file >> >> Can this be done via "pure" BASH? Need sed/awk as well? >> -- Daniel L. Miller, VP - Engineering, SET AM Fire & Electronic Services, Inc. [AMFES] dmiller at amfes.com 702-312-5276 From h.reindl at thelounge.net Wed Mar 4 19:31:49 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 04 Mar 2015 20:31:49 +0100 Subject: IP drop list In-Reply-To: <54F75914.5070001@orlitzky.com> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> <54F75914.5070001@orlitzky.com> Message-ID: <54F75DA5.5020406@thelounge.net> Am 04.03.2015 um 20:12 schrieb Michael Orlitzky: > On 03/03/2015 11:03 PM, Earl Killian wrote: >> On 2015/3/2 10:03, Reindl Harald wrote: >>> >>> that is all nice >>> >>> but the main benefit of RBL's is always ignored: >>> >>> * centralized >>> * no log parsing at all >>> * honeypot data are "delivered" to any host >>> * it's cheap >>> * it's easy to maintain >>> * it don't need any root privileges anywhere >>> >>> we have a small honeypot network with a couple of ipranges detecting >>> mass port-scans and so on and this data are available *everywhere* >>> >>> so if some IP hits there it takes 60 seconds and any service >>> supportings DNS blacklists can block them *even before* the bot hits >>> the real mailserver at all >>> >> I would like to reiterate Reindl Harald's point above, since subsequent >> discussion has gotten away from it. If Dovecot had DNS RBL support >> similar to Postfix, I think quite a few people would use it, and thereby >> defeat the scanners far more effectively than any other method. It is >> good that other people are suggesting things that will work today, but >> in terms of what new feature would be the best solution, I can't think >> of one better than a DNS RBL. > > Please add this support to iptables instead of Dovecot. It's a waste of > effort to code it into every application that listens on the network. > > Combined with "--ctstate NEW" and a chain for IMAP packets, it would be > no less efficient you don't want a dns client in a kernel module with full permissions and you will never convince any sane kernel developer doing that nor does it much help for the users on a different operating system dovecot is not linux only ____________________________________ > In the case of HTTP, IMAP, etc. things are not so easy. > Just think about NAT and CGN that don't matter if i blacklist a client because he starts a dictionary attack in SMTP i want it also bock on IMAP without use a dozen of different tools because teh via IMAP now catched account password will be used for send spam later when the SMTP RBL entry expires and frankly that 100% trustable RBL lives *before* "permit_sasl_authenticated" because it would be pointless anywhere else ordinary blacklists are score based on the MX, that is a complete differet machine with no business for POP3/IMAP or even outgoing mail -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From mcguire at neurotica.com Wed Mar 4 20:03:58 2015 From: mcguire at neurotica.com (Dave McGuire) Date: Wed, 04 Mar 2015 15:03:58 -0500 Subject: IP drop list In-Reply-To: <54F75914.5070001@orlitzky.com> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> <54F75914.5070001@orlitzky.com> Message-ID: <54F7652E.8010902@neurotica.com> On 03/04/2015 02:12 PM, Michael Orlitzky wrote: >> I would like to reiterate Reindl Harald's point above, since subsequent >> discussion has gotten away from it. If Dovecot had DNS RBL support >> similar to Postfix, I think quite a few people would use it, and thereby >> defeat the scanners far more effectively than any other method. It is >> good that other people are suggesting things that will work today, but >> in terms of what new feature would be the best solution, I can't think >> of one better than a DNS RBL. > > Please add this support to iptables instead of Dovecot. It's a waste of > effort to code it into every application that listens on the network. Would you care to integrate it into IOS on my Cisco as well? There are things connected to the Internet that aren't PCs running Linux, you know. It may be hard to accept, but that's the way it is. -Dave -- Dave McGuire, AK4HZ/3 New Kensington, PA From mail at oliwel.de Wed Mar 4 20:37:16 2015 From: mail at oliwel.de (Oliver Welter) Date: Wed, 04 Mar 2015 21:37:16 +0100 Subject: IP drop list In-Reply-To: <54F7652E.8010902@neurotica.com> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> <54F75914.5070001@orlitzky.com> <54F7652E.8010902@neurotica.com> Message-ID: <54F76CFC.3010207@oliwel.de> Am 04.03.2015 um 21:03 schrieb Dave McGuire: > On 03/04/2015 02:12 PM, Michael Orlitzky wrote: >>> I would like to reiterate Reindl Harald's point above, since subsequent >>> discussion has gotten away from it. If Dovecot had DNS RBL support >>> similar to Postfix, I think quite a few people would use it, and thereby >>> defeat the scanners far more effectively than any other method. It is >>> good that other people are suggesting things that will work today, but >>> in terms of what new feature would be the best solution, I can't think >>> of one better than a DNS RBL. >> >> Please add this support to iptables instead of Dovecot. It's a waste of >> effort to code it into every application that listens on the network. > > > > Would you care to integrate it into IOS on my Cisco as well? > > There are things connected to the Internet that aren't PCs running > Linux, you know. It may be hard to accept, but that's the way it is. > I assume your dovecot runs on some kind of *nix so there should be some sort of netfilter available which you can put in front of your listening ports. It might be also an option to create some kind of "hooks" in dovecot that can be used to connect to a DNSBL checker - so configuration can happen outside of dovecot. Oliver -- Protect your environment - close windows and adopt a penguin! -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4074 bytes Desc: S/MIME Cryptographic Signature URL: From mcguire at neurotica.com Wed Mar 4 20:45:45 2015 From: mcguire at neurotica.com (Dave McGuire) Date: Wed, 04 Mar 2015 15:45:45 -0500 Subject: IP drop list In-Reply-To: <54F76CFC.3010207@oliwel.de> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> <54F75914.5070001@orlitzky.com> <54F7652E.8010902@neurotica.com> <54F76CFC.3010207@oliwel.de> Message-ID: <54F76EF9.70206@neurotica.com> On 03/04/2015 03:37 PM, Oliver Welter wrote: >>>> I would like to reiterate Reindl Harald's point above, since subsequent >>>> discussion has gotten away from it. If Dovecot had DNS RBL support >>>> similar to Postfix, I think quite a few people would use it, and >>>> thereby >>>> defeat the scanners far more effectively than any other method. It is >>>> good that other people are suggesting things that will work today, but >>>> in terms of what new feature would be the best solution, I can't think >>>> of one better than a DNS RBL. >>> >>> Please add this support to iptables instead of Dovecot. It's a waste of >>> effort to code it into every application that listens on the network. >> >> >> >> Would you care to integrate it into IOS on my Cisco as well? >> >> There are things connected to the Internet that aren't PCs running >> Linux, you know. It may be hard to accept, but that's the way it is. >> > I assume your dovecot runs on some kind of *nix Of course. I run it under Solaris. > so there should be some > sort of netfilter available which you can put in front of your listening > ports. There is. But I already have a firewall, running on bulletproof hardware that doesn't depend on spinning disks. I don't want to add ANOTHER firewall when I already have a perfectly good one. Besides, my mail server is built for...serving mail. Not being a firewall. -Dave -- Dave McGuire, AK4HZ/3 New Kensington, PA From mail at oliwel.de Wed Mar 4 20:51:44 2015 From: mail at oliwel.de (Oliver Welter) Date: Wed, 04 Mar 2015 21:51:44 +0100 Subject: IP drop list In-Reply-To: <54F76EF9.70206@neurotica.com> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> <54F75914.5070001@orlitzky.com> <54F7652E.8010902@neurotica.com> <54F76CFC.3010207@oliwel.de> <54F76EF9.70206@neurotica.com> Message-ID: <54F77060.8020609@oliwel.de> Am 04.03.2015 um 21:45 schrieb Dave McGuire: > On 03/04/2015 03:37 PM, Oliver Welter wrote: >>>>> I would like to reiterate Reindl Harald's point above, since subsequent >>>>> discussion has gotten away from it. If Dovecot had DNS RBL support >>>>> similar to Postfix, I think quite a few people would use it, and >>>>> thereby >>>>> defeat the scanners far more effectively than any other method. It is >>>>> good that other people are suggesting things that will work today, but >>>>> in terms of what new feature would be the best solution, I can't think >>>>> of one better than a DNS RBL. >>>> >>>> Please add this support to iptables instead of Dovecot. It's a waste of >>>> effort to code it into every application that listens on the network. >>> >>> >>> >>> Would you care to integrate it into IOS on my Cisco as well? >>> >>> There are things connected to the Internet that aren't PCs running >>> Linux, you know. It may be hard to accept, but that's the way it is. >>> >> I assume your dovecot runs on some kind of *nix > > Of course. I run it under Solaris. > >> so there should be some >> sort of netfilter available which you can put in front of your listening >> ports. > > There is. But I already have a firewall, running on bulletproof > hardware that doesn't depend on spinning disks. I don't want to add > ANOTHER firewall when I already have a perfectly good one. Besides, my > mail server is built for...serving mail. Not being a firewall. > Well, from an academic point of view, a network service that denies connection on the ip layer is also an ip firewall. Oliver -- Protect your environment - close windows and adopt a penguin! -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4074 bytes Desc: S/MIME Cryptographic Signature URL: From mcguire at neurotica.com Wed Mar 4 20:58:31 2015 From: mcguire at neurotica.com (Dave McGuire) Date: Wed, 04 Mar 2015 15:58:31 -0500 Subject: IP drop list In-Reply-To: <54F77060.8020609@oliwel.de> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> <54F75914.5070001@orlitzky.com> <54F7652E.8010902@neurotica.com> <54F76CFC.3010207@oliwel.de> <54F76EF9.70206@neurotica.com> <54F77060.8020609@oliwel.de> Message-ID: <54F771F7.4000907@neurotica.com> On 03/04/2015 03:51 PM, Oliver Welter wrote: >>>>>> I would like to reiterate Reindl Harald's point above, since >>>>>> subsequent >>>>>> discussion has gotten away from it. If Dovecot had DNS RBL support >>>>>> similar to Postfix, I think quite a few people would use it, and >>>>>> thereby >>>>>> defeat the scanners far more effectively than any other method. It is >>>>>> good that other people are suggesting things that will work today, >>>>>> but >>>>>> in terms of what new feature would be the best solution, I can't >>>>>> think >>>>>> of one better than a DNS RBL. >>>>> >>>>> Please add this support to iptables instead of Dovecot. It's a >>>>> waste of >>>>> effort to code it into every application that listens on the network. >>>> >>>> >>>> >>>> Would you care to integrate it into IOS on my Cisco as well? >>>> >>>> There are things connected to the Internet that aren't PCs running >>>> Linux, you know. It may be hard to accept, but that's the way it is. >>>> >>> I assume your dovecot runs on some kind of *nix >> >> Of course. I run it under Solaris. >> >>> so there should be some >>> sort of netfilter available which you can put in front of your listening >>> ports. >> >> There is. But I already have a firewall, running on bulletproof >> hardware that doesn't depend on spinning disks. I don't want to add >> ANOTHER firewall when I already have a perfectly good one. Besides, my >> mail server is built for...serving mail. Not being a firewall. >> > Well, from an academic point of view, a network service that denies > connection on the ip layer is also an ip firewall. In a real-world datacenter at 3AM, academic points of view seldom, if ever, come into play. -Dave -- Dave McGuire, AK4HZ/3 New Kensington, PA From h.reindl at thelounge.net Wed Mar 4 21:29:47 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 04 Mar 2015 22:29:47 +0100 Subject: IP drop list In-Reply-To: <54F77060.8020609@oliwel.de> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> <54F75914.5070001@orlitzky.com> <54F7652E.8010902@neurotica.com> <54F76CFC.3010207@oliwel.de> <54F76EF9.70206@neurotica.com> <54F77060.8020609@oliwel.de> Message-ID: <54F7794B.4000303@thelounge.net> Am 04.03.2015 um 21:51 schrieb Oliver Welter: >>>>> Please add this support to iptables instead of Dovecot. It's a >>>>> waste of >>>>> effort to code it into every application that listens on the network. >>>> >>>> >>>> >>>> Would you care to integrate it into IOS on my Cisco as well? >>>> >>>> There are things connected to the Internet that aren't PCs running >>>> Linux, you know. It may be hard to accept, but that's the way it is. >>>> >>> I assume your dovecot runs on some kind of *nix >> >> Of course. I run it under Solaris. >> >>> so there should be some >>> sort of netfilter available which you can put in front of your listening >>> ports. >> >> There is. But I already have a firewall, running on bulletproof >> hardware that doesn't depend on spinning disks. I don't want to add >> ANOTHER firewall when I already have a perfectly good one. Besides, my >> mail server is built for...serving mail. Not being a firewall. >> > Well, from an academic point of view, a network service that denies > connection on the ip layer is also an ip firewall. nonsense a service using RBL's don't reject on IP layer -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From professa at dementianati.com Wed Mar 4 21:33:47 2015 From: professa at dementianati.com (Professa Dementia) Date: Wed, 04 Mar 2015 13:33:47 -0800 Subject: IP drop list In-Reply-To: <54F76EF9.70206@neurotica.com> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> <54F75914.5070001@orlitzky.com> <54F7652E.8010902@neurotica.com> <54F76CFC.3010207@oliwel.de> <54F76EF9.70206@neurotica.com> Message-ID: <54F77A3B.8050205@dementianati.com> On 3/4/2015 12:45 PM, Dave McGuire wrote: > There is. But I already have a firewall, running on bulletproof > hardware that doesn't depend on spinning disks. I don't want to add > ANOTHER firewall when I already have a perfectly good one. Besides, my > mail server is built for...serving mail. Not being a firewall. You can implement whatever type of security you are comfortable with, however, best practices is to have layered security, also known as the "belt and suspenders" method of keeping your pants up. A perimeter firewall and local firewalls (iptables usually) on each machine is the minimum level of security I set up. A perimeter firewall alone does not protect you from an attacker who is able to compromise one machine and install a scanner which then scan all the systems on your internal network looking for exploitable weaknesses. All the while the perimeter firewall is oblivious to the attack going on internally and utterly incapable of mitigating it even if it were aware. Dem From felix at zandanel.me Wed Mar 4 22:00:42 2015 From: felix at zandanel.me (Felix Zandanel) Date: Wed, 4 Mar 2015 23:00:42 +0100 Subject: IP drop list In-Reply-To: <54F75DA5.5020406@thelounge.net> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> <54F75914.5070001@orlitzky.com> <54F75DA5.5020406@thelounge.net> Message-ID: <8E9EDE0E-9F8B-4D83-AD3A-DC54AD279E6F@zandanel.me> > Am 04.03.2015 um 20:31 schrieb Reindl Harald : > > > In the case of HTTP, IMAP, etc. things are not so easy. > > Just think about NAT and CGN > > that don't matter > > if i blacklist a client because he starts a dictionary attack in SMTP i want it also bock on IMAP without use a dozen of different tools because teh via IMAP now catched account password will be used for send spam later when the SMTP RBL entry expires That?s the point why DNSBLs are good: You can use them for many different services at once. However, the idea is to block attackers before they are able to succeed identifying a valid login credential AND not to block your customers that expect a service that just works. This is a trade off. If both the attacker (or a malware infected computer etc.) and your valid user sit behind the same CGN gateway then it does matter and that scenario is not uncommon. Blocking a rude boy for some time from continuing with the attack will likely cause him to stop entirely, at least for a much longer time than you blocking the address. If he proceeds afterwards, then you have no other choice than blocking the IP for longer anyway and maybe tell your users you are suffering an attack. I am not against block lists. I just say their use should be justified as they may decrease overall service quality as well. There is another solution for auth based services: As soon as you detect a possible attack (# auth reqs > x etc.), keep the connection open, slow it down and just never let it succeed regardless of the credentials provided. This is done on a per-connection basis. No block list needed. Can be accomplished with fail2ban and iptables and therefore uses minimal server resources. Cheers, Felix -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mcguire at neurotica.com Wed Mar 4 22:02:11 2015 From: mcguire at neurotica.com (Dave McGuire) Date: Wed, 04 Mar 2015 17:02:11 -0500 Subject: IP drop list In-Reply-To: <54F77A3B.8050205@dementianati.com> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> <54F75914.5070001@orlitzky.com> <54F7652E.8010902@neurotica.com> <54F76CFC.3010207@oliwel.de> <54F76EF9.70206@neurotica.com> <54F77A3B.8050205@dementianati.com> Message-ID: <54F780E3.8070609@neurotica.com> On 03/04/2015 04:33 PM, Professa Dementia wrote: > On 3/4/2015 12:45 PM, Dave McGuire wrote: >> There is. But I already have a firewall, running on bulletproof >> hardware that doesn't depend on spinning disks. I don't want to add >> ANOTHER firewall when I already have a perfectly good one. Besides, my >> mail server is built for...serving mail. Not being a firewall. > > You can implement whatever type of security you are comfortable with, > however, best practices is to have layered security, also known as the > "belt and suspenders" method of keeping your pants up. > > A perimeter firewall and local firewalls (iptables usually) on each > machine is the minimum level of security I set up. A perimeter firewall > alone does not protect you from an attacker who is able to compromise > one machine and install a scanner which then scan all the systems on > your internal network looking for exploitable weaknesses. All the while > the perimeter firewall is oblivious to the attack going on internally > and utterly incapable of mitigating it even if it were aware. Yes, I have some experience in these matters, thank you. You've made my point for me. This is why I want Dovecot to handle the next layer, either via big flat files, a mysql/pgsql table, or DNS queries. -Dave -- Dave McGuire, AK4HZ/3 New Kensington, PA From Jochen.Bern at LINworks.de Wed Mar 4 23:12:02 2015 From: Jochen.Bern at LINworks.de (Jochen Bern) Date: Thu, 05 Mar 2015 00:12:02 +0100 Subject: IP drop list In-Reply-To: <54F76EF9.70206@neurotica.com> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> <54F75914.5070001@orlitzky.com> <54F7652E.8010902@neurotica.com> <54F76CFC.3010207@oliwel.de> <54F76EF9.70206@neurotica.com> Message-ID: <54F79142.7070607@LINworks.de> On 03/04/2015 09:45 PM, Dave McGuire wrote: > On 03/04/2015 03:37 PM, Oliver Welter wrote: >> Am 04.03.2015 um 21:03 schrieb Dave McGuire: >>> Am 04.03.2015 um 20:12 schrieb Michael Orlitzky: >>>> Please add [DNSBL] support to iptables instead of Dovecot. It's a waste of >>>> effort to code it into every application that listens on the network. (FWIW, I agree that DNSBL hooks have no business being in kernel space. A standard *userland* DNSBL client communicating with iptables and similar by means of libnetfilter_queue would sound quite promising, however ...) >>> Would you care to integrate it into IOS on my Cisco as well? [...] >> so there should be some >> sort of netfilter available which you can put in front of your listening >> ports. > > There is. But I already have a firewall, running on bulletproof > hardware that doesn't depend on spinning disks. I don't want to add > ANOTHER firewall when I already have a perfectly good one. Besides, my > mail server is built for...serving mail. Not being a firewall. You're contradicting yourself here. If it's "a perfectly good" firewall, why would you care whether an additional feature (might or) might not get added to it? And if you don't trust those disks to keep spinning, why do you allow them to hold your e-mail? For what it's worth, the host firewall functionality *already is* in the kernel, and kernel memory gets locked into RAM. Apart from bootup and local logging, firewalling may well just keep running after the HDD died in mid-operation (yes, I've seen (iptables-based) firewalls do that; the customers typically complain that the webUI or CLI turned unresponsive). Good luck getting the co-located dovecot to live up to that level of resilience. :-} Regards, J. Bern -- *NEU* - NEC IT-Infrastruktur-Produkte im : Server--Storage--Virtualisierung--Management SW--Passion for Performance Jochen Bern, Systemingenieur --- LINworks GmbH Postfach 100121, 64201 Darmstadt | Robert-Koch-Str. 9, 64331 Weiterstadt PGP (1024D/4096g) FP = D18B 41B1 16C0 11BA 7F8C DCF7 E1D5 FAF4 444E 1C27 Tel. +49 6151 9067-231, Zentr. -0, Fax -299 - Amtsg. Darmstadt HRB 85202 Unternehmenssitz Weiterstadt, Gesch?ftsf?hrer Metin Dogan, Oliver Michel From h.reindl at thelounge.net Thu Mar 5 00:56:35 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Thu, 05 Mar 2015 01:56:35 +0100 Subject: IP drop list In-Reply-To: <8E9EDE0E-9F8B-4D83-AD3A-DC54AD279E6F@zandanel.me> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> <54F75914.5070001@orlitzky.com> <54F75DA5.5020406@thelounge.net> <8E9EDE0E-9F8B-4D83-AD3A-DC54AD279E6F@zandanel.me> Message-ID: <54F7A9C3.4080906@thelounge.net> Am 04.03.2015 um 23:00 schrieb Felix Zandanel: > I am not against block lists. I just say their use should be justified as they may decrease overall service quality as well. There is another solution for auth based services: As soon as you detect a possible attack (# auth reqs > x etc.), keep the connection open, slow it down and just never let it succeed regardless of the credentials provided. This is done on a per-connection basis. No block list needed. Can be accomplished with fail2ban and iptables and therefore uses minimal server resources. well, i have iptables rate controls which blocks most dictionary attacks and small DOS-attacks perfectly well but that won't change the fact that if from an IP address starts a large dictionary attack and that IP is a CGN it *would* affect users from the same IP anyways and since this is fact it is reasonable to * enter that IP in the wbeinterface feeding rbldnsd * enter in the scond field 1800 seconds or whatever value * apply it that way for any service supporting RBL's * release that lock automatically after X seconds security and defense is always layered but such things don't work well if half or mail-subsytems needs sepcial handling -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From michael at orlitzky.com Thu Mar 5 01:13:10 2015 From: michael at orlitzky.com (Michael Orlitzky) Date: Wed, 04 Mar 2015 20:13:10 -0500 Subject: IP drop list In-Reply-To: <54F79142.7070607@LINworks.de> References: <54F4A442.6040805@sys4.de> <54F4A5D4.6070303@thelounge.net> <54F68428.9060700@lists.killian.com> <54F75914.5070001@orlitzky.com> <54F7652E.8010902@neurotica.com> <54F76CFC.3010207@oliwel.de> <54F76EF9.70206@neurotica.com> <54F79142.7070607@LINworks.de> Message-ID: <54F7ADA6.3010207@orlitzky.com> On 03/04/2015 06:12 PM, Jochen Bern wrote: > On 03/04/2015 09:45 PM, Dave McGuire wrote: >> On 03/04/2015 03:37 PM, Oliver Welter wrote: >>> Am 04.03.2015 um 21:03 schrieb Dave McGuire: >>>> Am 04.03.2015 um 20:12 schrieb Michael Orlitzky: >>>>> Please add [DNSBL] support to iptables instead of Dovecot. It's a waste of >>>>> effort to code it into every application that listens on the network. > > (FWIW, I agree that DNSBL hooks have no business being in kernel space. > A standard *userland* DNSBL client communicating with iptables and > similar by means of libnetfilter_queue would sound quite promising, > however ...) > This is what I had in mind. Here's a proof of concept. First, the iptables rule: iptables -A tcp_packets -p tcp --dport 443 -j NFQUEUE --queue-num 1 (the details aren't important, just send something to NFQUEUE #1). Then create the queue as root, and drop privileges. After that you can make accept/drop decisions in userspace. This took maybe 15 minutes using NetfilterQueue from pypi. It would be easy to replace the if ipp.src == badguy test with a real RBL lookup. But then you'd need to make the RBL list configurable, and implement a scoring system, and document it, etc. (i.e. all the /actual/ work). ------ import os, pwd, grp from netfilterqueue import NetfilterQueue from scapy.all import IP def drop_privileges(uid_name='dovecot', gid_name='dovecot'): """ Drop user/group privileges from root/root to the given ones. """ if os.getuid() != 0: # We're not root *shrug*. return # Get the uid/gid from the name running_uid = pwd.getpwnam(uid_name).pw_uid running_gid = grp.getgrnam(gid_name).gr_gid # Remove group privileges os.setgroups([]) # Try setting the new uid/gid os.setgid(running_gid) os.setuid(running_uid) # Ensure a very conservative umask old_umask = os.umask(077) def callback(packet): """ Callback function registered through netfilter. Will be called on every packet passed to the netfilter queue. """ badguy = "127.0.0.1" ipp = IP(packet.get_payload()) if ipp.src == badguy: print("Dropping packet from %s..." % badguy) packet.drop() else: packet.accept() nfqueue = NetfilterQueue() nfqueue.bind(1, callback) drop_privileges() try: nfqueue.run() except KeyboardInterrupt: print("Bailing...") From jtam.home at gmail.com Thu Mar 5 03:06:36 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 4 Mar 2015 19:06:36 -0800 (PST) Subject: IP drop list In-Reply-To: References: Message-ID: Nick Edwards writes: > I thought Timo once said dovecot had tarpitting, its useless if it is > there, and if it is, it needs user configurable timings, or maybe its > one of those things thats been in the gunna happen list > for a long time, like other stuff If I remember correctly, I think this was the "auth_failure_delay" feature. However, these delays are only inserted into the same session -- no IP tracking is done so a BFD attacking host can just keep opening up new connections. Dave McGuire writes: >>>> Please add this support to iptables instead of Dovecot. It's a >>>> waste of effort to code it into every application that listens on >>>> the network. >>> >>> >>> >>> Would you care to integrate it into IOS on my Cisco as well? >>> >>> There are things connected to the Internet that aren't PCs running >>> Linux, you know. It may be hard to accept, but that's the way it is. >>> >> I assume your dovecot runs on some kind of *nix > > Of course. I run it under Solaris. Oddly enough, if you run some versions of Solaris, it uses IPFilter as the native firewall, and it *does* have userland hooks so that you can make pass/block decisions based on userland executable. Not well documented though (see auth rules). You would need a firewall rule like auth in proto tcp from any to any port = 143 flags S/SA keep state then write a program that does ioctl(ipauth) calls to inspect and accept/reject packets; not for the faint of heart. However, you can use whatever weird and wonderful methods you want to determine IMAP/POP/SMTP network access policy, including DNSRBL or parsing a text file. This thread seems to be spinning into non-dovecot subjects, and I'm not helping, so I'll stop. Joseph Tam From dovecot at paz.bz Thu Mar 5 04:46:37 2015 From: dovecot at paz.bz (Jim Pazarena) Date: Wed, 04 Mar 2015 20:46:37 -0800 Subject: Fwd: Re: IP drop list Message-ID: <54F7DFAD.3080104@paz.bz> On 2015-03-02 2:02 AM, Jochen Bern wrote: > On 03/01/2015 08:53 AM, Jim Pazarena wrote: >> I wonder if there is an easy way to provide dovecot a flat text file of >> ipv4 #'s which should be ignored or dropped? >> >> I have accumulated 45,000+ IPs which routinely try dictionary and >> 12345678 password attempts. The file is too big to create firewall >> drops [...] > > The inherent assumption here is that dovecot, using a "flat file", will > be able to process the block list more effectively than the firewall, > which is a tool written for the *purpose* but supposedly unable to even > *try* due to the list's size. That sounds ... counterintuitive. I am the original poster and just came back to this thread. When the first couple replies were "fail2ban" I lost interest. The reason I contemplated a flat text scan by dovecot is because, for the most part, my dovecot is low volume. So even if parsing a flat text file is less 'efficient' than a firewall insertion, it WOULD serve to defeat dictionary attacks rather readily. I already have a routine which scans my dovecot logs for goofy attacks such as dictionary or 12345 attempts. And since the attacks are pop/IMAP only, that is the only avenue which I wanted to defeat. This question garnered lots and lots of responses and I appreciate them all and read them all. And out of all the responses I think I will pursue the ipset routine. It seems easy enough and can act at the firewall level. The DNS RBL would be cool. I am also cognizant that 45,000 SHOULD have a TTL. However, these were IPs attempting to fetch email with obviously hacker type passwords. If, later, a given IP is re-assigned to a 'legitimate' person, they would still be able to send an email to me ' postmaster@ ' asking about an inability to fetch email. But parsing the flat text file would STILL be my preference. I'll look at the source and see if I can figure out where to inject such code. Like I said, my dovecot is low volume, so a fraction of a second at connection time is low impact. Considering that the flat text file may hang around in the memory cache it could even be less impact than low. From superinterstellar at gmail.com Thu Mar 5 08:11:21 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Thu, 5 Mar 2015 15:11:21 +0700 Subject: Dovecot Full Text Search results in SolrException: undefined field text [SERIOUS] Message-ID: Hello, My dovecot constantly runs into this error. I want to fix this one last time, I am tired of troubleshooting so please someone give me a lasting and proper solution for this error. I think its a problem with the dovecot-solr module. Please tell me how do I find the root of this problem with Dovecot. There is a problem with the body search text field. It always fails(with no result), other searches work(ie. search date, subject etc, ) The field-text I believe is missing. Please help. Desperate here! 2/25/2015, 11:32:30 PM ERROR SolrCore org.apache.solr.common. SolrException: undefined field text org.apache.solr.common.SolrException: undefined field text at org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1269) at org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:434) at org.apache.lucene.analysis.DelegatingAnalyzerWrapper$DelegatingReuseStrategy.getReusableComponents(DelegatingAnalyzerWrapper.java:74) at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:175) at org.apache.lucene.util.QueryBuilder.createFieldQuery(QueryBuilder.java:207) at org.apache.solr.parser.SolrQueryParserBase.newFieldQuery(SolrQueryParserBase.java:374) at org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:742) at org.apache.solr.parser.SolrQueryParserBase.handleBareTokenQuery(SolrQueryParserBase.java:541) at org.apache.solr.parser.QueryParser.Term(QueryParser.java:299) at org.apache.solr.parser.QueryParser.Clause(QueryParser.java:185) at org.apache.solr.parser.QueryParser.Query(QueryParser.java:107) at org.apache.solr.parser.QueryParser.TopLevelQuery(QueryParser.java:96) at org.apache.solr.parser.SolrQueryParserBase.parse(SolrQueryParserBase.java:151) at org.apache.solr.search.LuceneQParser.parse(LuceneQParser.java:50) at org.apache.solr.search.QParser.getQuery(QParser.java:141) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:148) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:197) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1967) at org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:64) at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1739) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) From tolga at ozses.net Thu Mar 5 08:46:45 2015 From: tolga at ozses.net (Muzaffer Tolga Ozses) Date: Thu, 5 Mar 2015 10:46:45 +0200 Subject: Dovecot Full Text Search results in SolrException: undefined field text [SERIOUS] In-Reply-To: References: Message-ID: Paste your xml file here. On 5 March 2015 at 10:11, Kevin Laurie wrote: > Hello, > My dovecot constantly runs into this error. > I want to fix this one last time, I am tired of troubleshooting so > please someone give me a lasting and proper solution for this error. I > think its a problem with the dovecot-solr module. > > Please tell me how do I find the root of this problem with Dovecot. > There is a problem with the body search text field. It always > fails(with no result), other searches work(ie. search date, subject > etc, ) The field-text I believe is missing. Please help. Desperate > here! > > > > > 2/25/2015, 11:32:30 PM ERROR SolrCore > org.apache.solr.common. > SolrException: undefined field text > > org.apache.solr.common.SolrException: undefined field text > at > org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1269) > at > org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:434) > at > org.apache.lucene.analysis.DelegatingAnalyzerWrapper$DelegatingReuseStrategy.getReusableComponents(DelegatingAnalyzerWrapper.java:74) > at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:175) > at > org.apache.lucene.util.QueryBuilder.createFieldQuery(QueryBuilder.java:207) > at > org.apache.solr.parser.SolrQueryParserBase.newFieldQuery(SolrQueryParserBase.java:374) > at > org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:742) > at > org.apache.solr.parser.SolrQueryParserBase.handleBareTokenQuery(SolrQueryParserBase.java:541) > at org.apache.solr.parser.QueryParser.Term(QueryParser.java:299) > at org.apache.solr.parser.QueryParser.Clause(QueryParser.java:185) > at org.apache.solr.parser.QueryParser.Query(QueryParser.java:107) > at org.apache.solr.parser.QueryParser.TopLevelQuery(QueryParser.java:96) > at > org.apache.solr.parser.SolrQueryParserBase.parse(SolrQueryParserBase.java:151) > at org.apache.solr.search.LuceneQParser.parse(LuceneQParser.java:50) > at org.apache.solr.search.QParser.getQuery(QParser.java:141) > at > org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:148) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:197) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1967) > at > org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:64) > at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1739) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > -- mto From leon at dexterous.org Thu Mar 5 08:48:50 2015 From: leon at dexterous.org (Leon Kyneur) Date: Thu, 5 Mar 2015 19:48:50 +1100 Subject: Dovecot Full Text Search results in SolrException: undefined field text [SERIOUS] In-Reply-To: References: Message-ID: In your schema.XML check you have defined: On 05/03/2015 7:11 PM, "Kevin Laurie" wrote: > Hello, > My dovecot constantly runs into this error. > I want to fix this one last time, I am tired of troubleshooting so > please someone give me a lasting and proper solution for this error. I > think its a problem with the dovecot-solr module. > > Please tell me how do I find the root of this problem with Dovecot. > There is a problem with the body search text field. It always > fails(with no result), other searches work(ie. search date, subject > etc, ) The field-text I believe is missing. Please help. Desperate > here! > > > > > 2/25/2015, 11:32:30 PM ERROR SolrCore > org.apache.solr.common. > SolrException: undefined field text > > org.apache.solr.common.SolrException: undefined field text > at > org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1269) > at > org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:434) > at > org.apache.lucene.analysis.DelegatingAnalyzerWrapper$DelegatingReuseStrategy.getReusableComponents(DelegatingAnalyzerWrapper.java:74) > at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:175) > at > org.apache.lucene.util.QueryBuilder.createFieldQuery(QueryBuilder.java:207) > at > org.apache.solr.parser.SolrQueryParserBase.newFieldQuery(SolrQueryParserBase.java:374) > at > org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:742) > at > org.apache.solr.parser.SolrQueryParserBase.handleBareTokenQuery(SolrQueryParserBase.java:541) > at org.apache.solr.parser.QueryParser.Term(QueryParser.java:299) > at org.apache.solr.parser.QueryParser.Clause(QueryParser.java:185) > at org.apache.solr.parser.QueryParser.Query(QueryParser.java:107) > at org.apache.solr.parser.QueryParser.TopLevelQuery(QueryParser.java:96) > at > org.apache.solr.parser.SolrQueryParserBase.parse(SolrQueryParserBase.java:151) > at org.apache.solr.search.LuceneQParser.parse(LuceneQParser.java:50) > at org.apache.solr.search.QParser.getQuery(QParser.java:141) > at > org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:148) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:197) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1967) > at > org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:64) > at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1739) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > From superinterstellar at gmail.com Thu Mar 5 08:51:53 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Thu, 5 Mar 2015 15:51:53 +0700 Subject: Dovecot Full Text Search results in SolrException: undefined field text [SERIOUS] In-Reply-To: References: Message-ID: Below is my schema.xml id body On Thu, Mar 5, 2015 at 3:48 PM, Leon Kyneur wrote: > In your schema.XML check you have defined: > > multiValued="true"/> > > On 05/03/2015 7:11 PM, "Kevin Laurie" wrote: >> >> Hello, >> My dovecot constantly runs into this error. >> I want to fix this one last time, I am tired of troubleshooting so >> please someone give me a lasting and proper solution for this error. I >> think its a problem with the dovecot-solr module. >> >> Please tell me how do I find the root of this problem with Dovecot. >> There is a problem with the body search text field. It always >> fails(with no result), other searches work(ie. search date, subject >> etc, ) The field-text I believe is missing. Please help. Desperate >> here! >> >> >> >> >> 2/25/2015, 11:32:30 PM ERROR SolrCore >> org.apache.solr.common. >> SolrException: undefined field text >> >> org.apache.solr.common.SolrException: undefined field text >> at >> org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1269) >> at >> org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:434) >> at >> org.apache.lucene.analysis.DelegatingAnalyzerWrapper$DelegatingReuseStrategy.getReusableComponents(DelegatingAnalyzerWrapper.java:74) >> at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:175) >> at >> org.apache.lucene.util.QueryBuilder.createFieldQuery(QueryBuilder.java:207) >> at >> org.apache.solr.parser.SolrQueryParserBase.newFieldQuery(SolrQueryParserBase.java:374) >> at >> org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:742) >> at >> org.apache.solr.parser.SolrQueryParserBase.handleBareTokenQuery(SolrQueryParserBase.java:541) >> at org.apache.solr.parser.QueryParser.Term(QueryParser.java:299) >> at org.apache.solr.parser.QueryParser.Clause(QueryParser.java:185) >> at org.apache.solr.parser.QueryParser.Query(QueryParser.java:107) >> at org.apache.solr.parser.QueryParser.TopLevelQuery(QueryParser.java:96) >> at >> org.apache.solr.parser.SolrQueryParserBase.parse(SolrQueryParserBase.java:151) >> at org.apache.solr.search.LuceneQParser.parse(LuceneQParser.java:50) >> at org.apache.solr.search.QParser.getQuery(QParser.java:141) >> at >> org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:148) >> at >> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:197) >> at >> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) >> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1967) >> at >> org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:64) >> at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1739) >> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) From superinterstellar at gmail.com Thu Mar 5 10:14:51 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Thu, 5 Mar 2015 17:14:51 +0700 Subject: Dovecot Full Text Search: HTTP 500 : Unknown fieldType 'text_general' specified on field text. [SERIOUS] Message-ID: Hi Muzzafer, I get the error as specified below when i try to added it in as a field:- I dont think text_general is a valid field? HTTP ERROR 500 Problem accessing /solr/. Reason: {msg=SolrCore 'collection1' is not available due to init failure: Could not load conf for core collection1: Unknown fieldType 'text_general' specified on field text. Schema file is /opt/solr/solr/collection1/conf/schema.xml,trace=org.apache.solr.common.SolrException: SolrCore 'collection1' is not available due to init failure: Could not load conf for core collection1: Unknown fieldType 'text_general' specified on field text. Schema file is /opt/solr/solr/collection1/conf/schema.xml at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:745) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:307) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:368) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.solr.common.SolrException: Could not load conf for core collection1: Unknown fieldType 'text_general' specified on field text. Schema file is /opt/solr/solr/collection1/conf/schema.xml at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:66) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:489) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ... 1 more Caused by: org.apache.solr.common.SolrException: Unknown fieldType 'text_general' specified on field text. Schema file is /opt/solr/solr/collection1/conf/schema.xml at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:595) at org.apache.solr.schema.IndexSchema.(IndexSchema.java:166) at org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:55) at org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69) at org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:90) at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:62) ... 7 more Caused by: org.apache.solr.common.SolrException: Unknown fieldType 'text_general' specified on field text at org.apache.solr.schema.IndexSchema.loadFields(IndexSchema.java:638) at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:489) ... 12 more ,code=500} On Thu, Mar 5, 2015 at 5:04 PM, Muzaffer Tolga Ozses wrote: > Sure thing > > On 5 March 2015 at 11:52, Kevin Laurie wrote: >> >> No i dont have it. >> there is body field though. I think text is needed. Let me add it in and >> see. >> Thanks >> >> >> On Thu, Mar 5, 2015 at 4:42 PM, Muzaffer Tolga Ozses >> wrote: >> > Leon gave you the answer. Check if you have it, define if you don't, and >> > restart the service. >> > >> > On 5 March 2015 at 11:24, Kevin Laurie >> > wrote: >> >> >> >> Hi Muzaffer, >> >> Could you help by checking this? Should I provide my solrconfig.xml >> >> too? >> >> Please advise urgently. I really need this to work man.Thanks! >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > >> omitNorms="true"/> >> >> >> >> >> >> >> >> > >> positionIncrementGap="100"> >> >> >> >> >> >> > >> words="stopwords.txt"/> >> >> > >> generateWordParts="1" generateNumberParts="1" catenateWords="1" >> >> catenateNumbers="1" catenateAll="0"/> >> >> >> >> >> >> > >> protected="protwords.txt"/> >> >> > >> maxGramSize="15"/> >> >> >> >> >> >> >> >> >> >> > >> synonyms="synonyms.txt" ignoreCase="true" expand="true"/> >> >> > >> words="stopwords.txt"/> >> >> > >> generateWordParts="1" generateNumberParts="1" catenateWords="0" >> >> catenateNumbers="0" catenateAll="0"/> >> >> >> >> >> >> > >> protected="protwords.txt"/> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > >> required="true" /> >> >> > >> required="true" /> >> >> > >> required="true" /> >> >> > >> required="true" /> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> id >> >> body >> >> >> >> >> >> >> >> >> >> On Thu, Mar 5, 2015 at 3:48 PM, Leon Kyneur wrote: >> >> > In your schema.XML check you have defined: >> >> > >> >> > > >> > multiValued="true"/> >> >> > >> >> > On 05/03/2015 7:11 PM, "Kevin Laurie" >> >> > wrote: >> >> >> >> >> >> Hello, >> >> >> My dovecot constantly runs into this error. >> >> >> I want to fix this one last time, I am tired of troubleshooting so >> >> >> please someone give me a lasting and proper solution for this error. >> >> >> I >> >> >> think its a problem with the dovecot-solr module. >> >> >> >> >> >> Please tell me how do I find the root of this problem with Dovecot. >> >> >> There is a problem with the body search text field. It always >> >> >> fails(with no result), other searches work(ie. search date, subject >> >> >> etc, ) The field-text I believe is missing. Please help. Desperate >> >> >> here! >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> 2/25/2015, 11:32:30 PM ERROR SolrCore >> >> >> org.apache.solr.common. >> >> >> SolrException: undefined field text >> >> >> >> >> >> org.apache.solr.common.SolrException: undefined field text >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1269) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:434) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.lucene.analysis.DelegatingAnalyzerWrapper$DelegatingReuseStrategy.getReusableComponents(DelegatingAnalyzerWrapper.java:74) >> >> >> at >> >> >> org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:175) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.lucene.util.QueryBuilder.createFieldQuery(QueryBuilder.java:207) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.parser.SolrQueryParserBase.newFieldQuery(SolrQueryParserBase.java:374) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:742) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.parser.SolrQueryParserBase.handleBareTokenQuery(SolrQueryParserBase.java:541) >> >> >> at org.apache.solr.parser.QueryParser.Term(QueryParser.java:299) >> >> >> at org.apache.solr.parser.QueryParser.Clause(QueryParser.java:185) >> >> >> at org.apache.solr.parser.QueryParser.Query(QueryParser.java:107) >> >> >> at >> >> >> >> >> >> org.apache.solr.parser.QueryParser.TopLevelQuery(QueryParser.java:96) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.parser.SolrQueryParserBase.parse(SolrQueryParserBase.java:151) >> >> >> at org.apache.solr.search.LuceneQParser.parse(LuceneQParser.java:50) >> >> >> at org.apache.solr.search.QParser.getQuery(QParser.java:141) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:148) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:197) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) >> >> >> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1967) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:64) >> >> >> at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1739) >> >> >> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >> >> >> at >> >> >> >> >> >> >> >> >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> >> >> at >> >> >> >> >> >> >> >> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> >> >> at java.lang.Thread.run(Thread.java:745) >> > >> > >> > >> > >> > -- >> > mto > > > > > -- > mto From superinterstellar at gmail.com Thu Mar 5 10:45:23 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Thu, 5 Mar 2015 17:45:23 +0700 Subject: Dovecot Full Text Search: HTTP 500 : Unknown fieldType 'text_general' specified on field text. [SERIOUS] In-Reply-To: References: Message-ID: Anyone here can enlighten me on this? On Thu, Mar 5, 2015 at 5:14 PM, Kevin Laurie wrote: > Hi Muzzafer, > I get the error as specified below when i try to added it in as a field:- > I dont think text_general is a valid field? > > HTTP ERROR 500 > > Problem accessing /solr/. Reason: > > {msg=SolrCore 'collection1' is not available due to init failure: > Could not load conf for core collection1: Unknown fieldType > 'text_general' specified on field text. Schema file is > /opt/solr/solr/collection1/conf/schema.xml,trace=org.apache.solr.common.SolrException: > SolrCore 'collection1' is not available due to init failure: Could not > load conf for core collection1: Unknown fieldType 'text_general' > specified on field text. Schema file is > /opt/solr/solr/collection1/conf/schema.xml > at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:745) > at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:307) > at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207) > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) > at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) > at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) > at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) > at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) > at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) > at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) > at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) > at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) > at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) > at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) > at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) > at org.eclipse.jetty.server.Server.handle(Server.java:368) > at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) > at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) > at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) > at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) > at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) > at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) > at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.solr.common.SolrException: Could not load conf > for core collection1: Unknown fieldType 'text_general' specified on > field text. Schema file is /opt/solr/solr/collection1/conf/schema.xml > at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:66) > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:489) > at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255) > at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > ... 1 more > Caused by: org.apache.solr.common.SolrException: Unknown fieldType > 'text_general' specified on field text. Schema file is > /opt/solr/solr/collection1/conf/schema.xml > at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:595) > at org.apache.solr.schema.IndexSchema.(IndexSchema.java:166) > at org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:55) > at org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69) > at org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:90) > at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:62) > ... 7 more > Caused by: org.apache.solr.common.SolrException: Unknown fieldType > 'text_general' specified on field text > at org.apache.solr.schema.IndexSchema.loadFields(IndexSchema.java:638) > at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:489) > ... 12 more > ,code=500} > > > On Thu, Mar 5, 2015 at 5:04 PM, Muzaffer Tolga Ozses wrote: >> Sure thing >> >> On 5 March 2015 at 11:52, Kevin Laurie wrote: >>> >>> No i dont have it. >>> there is body field though. I think text is needed. Let me add it in and >>> see. >>> Thanks >>> >>> >>> On Thu, Mar 5, 2015 at 4:42 PM, Muzaffer Tolga Ozses >>> wrote: >>> > Leon gave you the answer. Check if you have it, define if you don't, and >>> > restart the service. >>> > >>> > On 5 March 2015 at 11:24, Kevin Laurie >>> > wrote: >>> >> >>> >> Hi Muzaffer, >>> >> Could you help by checking this? Should I provide my solrconfig.xml >>> >> too? >>> >> Please advise urgently. I really need this to work man.Thanks! >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >> >> omitNorms="true"/> >>> >> >>> >> >>> >> >>> >> >> >> positionIncrementGap="100"> >>> >> >>> >> >>> >> >> >> words="stopwords.txt"/> >>> >> >> >> generateWordParts="1" generateNumberParts="1" catenateWords="1" >>> >> catenateNumbers="1" catenateAll="0"/> >>> >> >>> >> >>> >> >> >> protected="protwords.txt"/> >>> >> >> >> maxGramSize="15"/> >>> >> >>> >> >>> >> >>> >> >>> >> >> >> synonyms="synonyms.txt" ignoreCase="true" expand="true"/> >>> >> >> >> words="stopwords.txt"/> >>> >> >> >> generateWordParts="1" generateNumberParts="1" catenateWords="0" >>> >> catenateNumbers="0" catenateAll="0"/> >>> >> >>> >> >>> >> >> >> protected="protwords.txt"/> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >> >> required="true" /> >>> >> >> >> required="true" /> >>> >> >> >> required="true" /> >>> >> >> >> required="true" /> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> id >>> >> body >>> >> >>> >> >>> >> >>> >> >>> >> On Thu, Mar 5, 2015 at 3:48 PM, Leon Kyneur wrote: >>> >> > In your schema.XML check you have defined: >>> >> > >>> >> > >> >> > multiValued="true"/> >>> >> > >>> >> > On 05/03/2015 7:11 PM, "Kevin Laurie" >>> >> > wrote: >>> >> >> >>> >> >> Hello, >>> >> >> My dovecot constantly runs into this error. >>> >> >> I want to fix this one last time, I am tired of troubleshooting so >>> >> >> please someone give me a lasting and proper solution for this error. >>> >> >> I >>> >> >> think its a problem with the dovecot-solr module. >>> >> >> >>> >> >> Please tell me how do I find the root of this problem with Dovecot. >>> >> >> There is a problem with the body search text field. It always >>> >> >> fails(with no result), other searches work(ie. search date, subject >>> >> >> etc, ) The field-text I believe is missing. Please help. Desperate >>> >> >> here! >>> >> >> >>> >> >> >>> >> >> >>> >> >> >>> >> >> 2/25/2015, 11:32:30 PM ERROR SolrCore >>> >> >> org.apache.solr.common. >>> >> >> SolrException: undefined field text >>> >> >> >>> >> >> org.apache.solr.common.SolrException: undefined field text >>> >> >> at >>> >> >> >>> >> >> >>> >> >> org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1269) >>> >> >> at >>> >> >> >>> >> >> >>> >> >> org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:434) >>> >> >> at >>> >> >> >>> >> >> >>> >> >> org.apache.lucene.analysis.DelegatingAnalyzerWrapper$DelegatingReuseStrategy.getReusableComponents(DelegatingAnalyzerWrapper.java:74) >>> >> >> at >>> >> >> org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:175) >>> >> >> at >>> >> >> >>> >> >> >>> >> >> org.apache.lucene.util.QueryBuilder.createFieldQuery(QueryBuilder.java:207) >>> >> >> at >>> >> >> >>> >> >> >>> >> >> org.apache.solr.parser.SolrQueryParserBase.newFieldQuery(SolrQueryParserBase.java:374) >>> >> >> at >>> >> >> >>> >> >> >>> >> >> org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:742) >>> >> >> at >>> >> >> >>> >> >> >>> >> >> org.apache.solr.parser.SolrQueryParserBase.handleBareTokenQuery(SolrQueryParserBase.java:541) >>> >> >> at org.apache.solr.parser.QueryParser.Term(QueryParser.java:299) >>> >> >> at org.apache.solr.parser.QueryParser.Clause(QueryParser.java:185) >>> >> >> at org.apache.solr.parser.QueryParser.Query(QueryParser.java:107) >>> >> >> at >>> >> >> >>> >> >> org.apache.solr.parser.QueryParser.TopLevelQuery(QueryParser.java:96) >>> >> >> at >>> >> >> >>> >> >> >>> >> >> org.apache.solr.parser.SolrQueryParserBase.parse(SolrQueryParserBase.java:151) >>> >> >> at org.apache.solr.search.LuceneQParser.parse(LuceneQParser.java:50) >>> >> >> at org.apache.solr.search.QParser.getQuery(QParser.java:141) >>> >> >> at >>> >> >> >>> >> >> >>> >> >> org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:148) >>> >> >> at >>> >> >> >>> >> >> >>> >> >> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:197) >>> >> >> at >>> >> >> >>> >> >> >>> >> >> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) >>> >> >> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1967) >>> >> >> at >>> >> >> >>> >> >> >>> >> >> org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:64) >>> >> >> at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1739) >>> >> >> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>> >> >> at >>> >> >> >>> >> >> >>> >> >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>> >> >> at >>> >> >> >>> >> >> >>> >> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>> >> >> at java.lang.Thread.run(Thread.java:745) >>> > >>> > >>> > >>> > >>> > -- >>> > mto >> >> >> >> >> -- >> mto From pw at wk-serv.de Thu Mar 5 10:46:06 2015 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 05 Mar 2015 11:46:06 +0100 Subject: fast sync vs. full sync Message-ID: <54F833EE.1010708@wk-serv.de> Hi everyone, can anybody explain the difference between Dovecots fast sync and full sync in replication mode? Regards Patrick From ct at flyingcircus.io Thu Mar 5 11:46:12 2015 From: ct at flyingcircus.io (Christian Theune) Date: Thu, 5 Mar 2015 12:46:12 +0100 Subject: Weird sub-folder error In-Reply-To: References: <3D51972C-EC20-45CA-ACD4-1555C2FE4DA0@flyingcircus.io> <413DB470-BE6B-435D-898C-D171ED371EC1@flyingcircus.io> Message-ID: <27A9B66B-6037-46FB-BBDA-156DBE855BA5@flyingcircus.io> Hi, > On 03 Mar 2015, at 09:55, Christian Theune wrote: > > Hi, > > interesting idea about ?/? being in the folder name. I?ll ask whether that was the case. Feedback from the user was ?probably not?. Anything I can help debugging this further? Christian ? Christian Theune ? ct at flyingcircus.io ? +49 345 219401 0 Flying Circus Internet Operations GmbH ? http://flyingcircus.io Forsterstra?e 29 ? 06112 Halle (Saale) ? Deutschland HR Stendal HRB 21169 ? Gesch?ftsf?hrer: Christian. Theune, Christian. Zagrodnick -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tolga at ozses.net Thu Mar 5 12:58:04 2015 From: tolga at ozses.net (Muzaffer Tolga Ozses) Date: Thu, 5 Mar 2015 14:58:04 +0200 Subject: Dovecot Full Text Search: HTTP 500 : Unknown fieldType 'text_general' specified on field text. [SERIOUS] In-Reply-To: References: Message-ID: Make that *text* instead of *text_general* On 5 March 2015 at 12:14, Kevin Laurie wrote: > Hi Muzzafer, > I get the error as specified below when i try to added it in as a field:- > I dont think text_general is a valid field? > > HTTP ERROR 500 > > Problem accessing /solr/. Reason: > > {msg=SolrCore 'collection1' is not available due to init failure: > Could not load conf for core collection1: Unknown fieldType > 'text_general' specified on field text. Schema file is > > /opt/solr/solr/collection1/conf/schema.xml,trace=org.apache.solr.common.SolrException: > SolrCore 'collection1' is not available due to init failure: Could not > load conf for core collection1: Unknown fieldType 'text_general' > specified on field text. Schema file is > /opt/solr/solr/collection1/conf/schema.xml > at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:745) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:307) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) > at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) > at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) > at org.eclipse.jetty.server.Server.handle(Server.java:368) > at > org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) > at > org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) > at > org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) > at > org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) > at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) > at > org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) > at > org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) > at > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.solr.common.SolrException: Could not load conf > for core collection1: Unknown fieldType 'text_general' specified on > field text. Schema file is /opt/solr/solr/collection1/conf/schema.xml > at > org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:66) > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:489) > at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255) > at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > ... 1 more > Caused by: org.apache.solr.common.SolrException: Unknown fieldType > 'text_general' specified on field text. Schema file is > /opt/solr/solr/collection1/conf/schema.xml > at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:595) > at org.apache.solr.schema.IndexSchema.(IndexSchema.java:166) > at > org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:55) > at > org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69) > at > org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:90) > at > org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:62) > ... 7 more > Caused by: org.apache.solr.common.SolrException: Unknown fieldType > 'text_general' specified on field text > at org.apache.solr.schema.IndexSchema.loadFields(IndexSchema.java:638) > at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:489) > ... 12 more > ,code=500} > > > On Thu, Mar 5, 2015 at 5:04 PM, Muzaffer Tolga Ozses > wrote: > > Sure thing > > > > On 5 March 2015 at 11:52, Kevin Laurie > wrote: > >> > >> No i dont have it. > >> there is body field though. I think text is needed. Let me add it in and > >> see. > >> Thanks > >> > >> > >> On Thu, Mar 5, 2015 at 4:42 PM, Muzaffer Tolga Ozses > >> wrote: > >> > Leon gave you the answer. Check if you have it, define if you don't, > and > >> > restart the service. > >> > > >> > On 5 March 2015 at 11:24, Kevin Laurie > >> > wrote: > >> >> > >> >> Hi Muzaffer, > >> >> Could you help by checking this? Should I provide my solrconfig.xml > >> >> too? > >> >> Please advise urgently. I really need this to work man.Thanks! > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> >> >> omitNorms="true"/> > >> >> omitNorms="true"/> > >> >> omitNorms="true"/> > >> >> > >> >> >> >> positionIncrementGap="100"> > >> >> > >> >> > >> >> >> >> words="stopwords.txt"/> > >> >> >> >> generateWordParts="1" generateNumberParts="1" catenateWords="1" > >> >> catenateNumbers="1" catenateAll="0"/> > >> >> > >> >> > >> >> >> >> protected="protwords.txt"/> > >> >> >> >> maxGramSize="15"/> > >> >> > >> >> > >> >> > >> >> > >> >> >> >> synonyms="synonyms.txt" ignoreCase="true" expand="true"/> > >> >> >> >> words="stopwords.txt"/> > >> >> >> >> generateWordParts="1" generateNumberParts="1" catenateWords="0" > >> >> catenateNumbers="0" catenateAll="0"/> > >> >> > >> >> > >> >> >> >> protected="protwords.txt"/> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> >> >> required="true" /> > >> >> >> >> required="true" /> > >> >> >> >> required="true" /> > >> >> >> >> required="true" /> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> id > >> >> body > >> >> > >> >> > >> >> > >> >> > >> >> On Thu, Mar 5, 2015 at 3:48 PM, Leon Kyneur > wrote: > >> >> > In your schema.XML check you have defined: > >> >> > > >> >> > stored="false" > >> >> > multiValued="true"/> > >> >> > > >> >> > On 05/03/2015 7:11 PM, "Kevin Laurie" > > >> >> > wrote: > >> >> >> > >> >> >> Hello, > >> >> >> My dovecot constantly runs into this error. > >> >> >> I want to fix this one last time, I am tired of troubleshooting so > >> >> >> please someone give me a lasting and proper solution for this > error. > >> >> >> I > >> >> >> think its a problem with the dovecot-solr module. > >> >> >> > >> >> >> Please tell me how do I find the root of this problem with > Dovecot. > >> >> >> There is a problem with the body search text field. It always > >> >> >> fails(with no result), other searches work(ie. search date, > subject > >> >> >> etc, ) The field-text I believe is missing. Please help. Desperate > >> >> >> here! > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> 2/25/2015, 11:32:30 PM ERROR SolrCore > >> >> >> org.apache.solr.common. > >> >> >> SolrException: undefined field text > >> >> >> > >> >> >> org.apache.solr.common.SolrException: undefined field text > >> >> >> at > >> >> >> > >> >> >> > >> >> >> > org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1269) > >> >> >> at > >> >> >> > >> >> >> > >> >> >> > org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:434) > >> >> >> at > >> >> >> > >> >> >> > >> >> >> > org.apache.lucene.analysis.DelegatingAnalyzerWrapper$DelegatingReuseStrategy.getReusableComponents(DelegatingAnalyzerWrapper.java:74) > >> >> >> at > >> >> >> org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:175) > >> >> >> at > >> >> >> > >> >> >> > >> >> >> > org.apache.lucene.util.QueryBuilder.createFieldQuery(QueryBuilder.java:207) > >> >> >> at > >> >> >> > >> >> >> > >> >> >> > org.apache.solr.parser.SolrQueryParserBase.newFieldQuery(SolrQueryParserBase.java:374) > >> >> >> at > >> >> >> > >> >> >> > >> >> >> > org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:742) > >> >> >> at > >> >> >> > >> >> >> > >> >> >> > org.apache.solr.parser.SolrQueryParserBase.handleBareTokenQuery(SolrQueryParserBase.java:541) > >> >> >> at org.apache.solr.parser.QueryParser.Term(QueryParser.java:299) > >> >> >> at org.apache.solr.parser.QueryParser.Clause(QueryParser.java:185) > >> >> >> at org.apache.solr.parser.QueryParser.Query(QueryParser.java:107) > >> >> >> at > >> >> >> > >> >> >> > org.apache.solr.parser.QueryParser.TopLevelQuery(QueryParser.java:96) > >> >> >> at > >> >> >> > >> >> >> > >> >> >> > org.apache.solr.parser.SolrQueryParserBase.parse(SolrQueryParserBase.java:151) > >> >> >> at > org.apache.solr.search.LuceneQParser.parse(LuceneQParser.java:50) > >> >> >> at org.apache.solr.search.QParser.getQuery(QParser.java:141) > >> >> >> at > >> >> >> > >> >> >> > >> >> >> > org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:148) > >> >> >> at > >> >> >> > >> >> >> > >> >> >> > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:197) > >> >> >> at > >> >> >> > >> >> >> > >> >> >> > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > >> >> >> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1967) > >> >> >> at > >> >> >> > >> >> >> > >> >> >> > org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:64) > >> >> >> at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1739) > >> >> >> at java.util.concurrent.FutureTask.run(FutureTask.java:262) > >> >> >> at > >> >> >> > >> >> >> > >> >> >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > >> >> >> at > >> >> >> > >> >> >> > >> >> >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > >> >> >> at java.lang.Thread.run(Thread.java:745) > >> > > >> > > >> > > >> > > >> > -- > >> > mto > > > > > > > > > > -- > > mto > -- mto From gintare.ragaisiene at gmail.com Thu Mar 5 15:18:27 2015 From: gintare.ragaisiene at gmail.com (Gintare Ragaisiene) Date: Thu, 5 Mar 2015 17:18:27 +0200 Subject: Receiving emails problem Message-ID: Hello, I have DirectAdmin system installed with CentOS OS. I created a user and a user email account. Then I tried to send email from my website (which is written on PHP5) to my mailbox. Everything seemed ok, but I did not received letter on the mailbox. The same is wehen I am trying to send through webmail SquirrelMail. I got a record on var/log/mailog: Mar 5 17:07:15 432282 dovecot[535]: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=7134, secured, session= Mar 5 17:07:15 432282 dovecot[535]: imap(gintare): Disconnected: Logged out in=79 out=766 Please, help me. Versions: DirectAdmin - 1.4 CentOS - 6.6 Dovecot - 2.2.15 SquirrelMail (wbmail) - 1.4.23 Thanks, Gintare From skdovecot at smail.inf.fh-brs.de Thu Mar 5 15:38:06 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 5 Mar 2015 16:38:06 +0100 (CET) Subject: Receiving emails problem In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 5 Mar 2015, Gintare Ragaisiene wrote: > I have DirectAdmin system installed with CentOS OS. I created a user and a > user email account. Then I tried to send email from my website (which is > written on PHP5) to my mailbox. Everything seemed ok, but I did not Check the logs of your MTA. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVPh4Xnz1H7kL/d9rAQIN5wgAo6hyMHe9ZsIOPMOchYQejYugmmsC8Ub2 DFxPwb7BzhpNaGmZv2I7KAIrfWffMNhw39Y1JZ0lGjPNfyAibpu+CCAyqDXKFJpF 17uJ6RGH70U4RDydKb1/d4Q3SRcGXNkyYc/v3dGuew7fRb39h8Y+P9VohqoCuYyE b1hLlah3VYgWMg5Ki5jSP7ooU5STK/W3eMV6bIreKTbYjD+x4tfYBPg/9Gt3/J4Z OJX5eVtcYfk+CfcOOJBM4J6ckieycW/mnLTY2Abrbv4q8eda8GYWW3ZWmcspmoed kpuvaaYztO8KmmkxSAupnt3Bi4+gVDuQ92RD/AOtOYeTKGSKns7mgA== =yQ6H -----END PGP SIGNATURE----- From kremels at kreme.com Thu Mar 5 19:23:08 2015 From: kremels at kreme.com (@lbutlr) Date: Thu, 5 Mar 2015 12:23:08 -0700 Subject: IP drop list In-Reply-To: <54F7DFAD.3080104@paz.bz> References: <54F7DFAD.3080104@paz.bz> Message-ID: <4DEF9635-19E8-4C10-9AA6-7494FB5E79FB@kreme.com> On 04 Mar 2015, at 21:46 , Jim Pazarena wrote: > On 2015-03-02 2:02 AM, Jochen Bern wrote: >> On 03/01/2015 08:53 AM, Jim Pazarena wrote: >>> I wonder if there is an easy way to provide dovecot a flat text file of >>> ipv4 #'s which should be ignored or dropped? >>> >>> I have accumulated 45,000+ IPs which routinely try dictionary and >>> 12345678 password attempts. The file is too big to create firewall >>> drops [...] >> >> The inherent assumption here is that dovecot, using a "flat file", will >> be able to process the block list more effectively than the firewall, >> which is a tool written for the *purpose* but supposedly unable to even >> *try* due to the list's size. That sounds ... counterintuitive. > > I am the original poster and just came back to this thread. When the > first couple replies were "fail2ban" I lost interest. Why? Fail2ban is simple to install, simple to setup, and then (and here?s the best part) then you never have to look at it again. -- Death is caused by swallowing small amounts of saliva over a long period of time. From h.reindl at thelounge.net Thu Mar 5 19:34:52 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Thu, 05 Mar 2015 20:34:52 +0100 Subject: IP drop list In-Reply-To: <4DEF9635-19E8-4C10-9AA6-7494FB5E79FB@kreme.com> References: <54F7DFAD.3080104@paz.bz> <4DEF9635-19E8-4C10-9AA6-7494FB5E79FB@kreme.com> Message-ID: <54F8AFDC.40400@thelounge.net> Am 05.03.2015 um 20:23 schrieb @lbutlr: > On 04 Mar 2015, at 21:46 , Jim Pazarena wrote: >> On 2015-03-02 2:02 AM, Jochen Bern wrote: >>> On 03/01/2015 08:53 AM, Jim Pazarena wrote: >>>> I wonder if there is an easy way to provide dovecot a flat text file of >>>> ipv4 #'s which should be ignored or dropped? >>>> >>>> I have accumulated 45,000+ IPs which routinely try dictionary and >>>> 12345678 password attempts. The file is too big to create firewall >>>> drops [...] >>> >>> The inherent assumption here is that dovecot, using a "flat file", will >>> be able to process the block list more effectively than the firewall, >>> which is a tool written for the *purpose* but supposedly unable to even >>> *try* due to the list's size. That sounds ... counterintuitive. >> >> I am the original poster and just came back to this thread. When the >> first couple replies were "fail2ban" I lost interest. > > Why? Fail2ban is simple to install, simple to setup, and then (and here?s the best part) then you never have to look at it again fail2ban is simple to install and to setup? *lol* yes if you have 99% out-of-the-box distribution configurations, igave it a try not so long ago and honestly the whole config snippets and log-parsing is a mess where i call it insane to give that stuff root permissions even on my private testserver -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From rs at sys4.de Thu Mar 5 21:18:20 2015 From: rs at sys4.de (Robert Schetterer) Date: Thu, 05 Mar 2015 22:18:20 +0100 Subject: IP drop list In-Reply-To: <4DEF9635-19E8-4C10-9AA6-7494FB5E79FB@kreme.com> References: <54F7DFAD.3080104@paz.bz> <4DEF9635-19E8-4C10-9AA6-7494FB5E79FB@kreme.com> Message-ID: <54F8C81C.6040208@sys4.de> Am 05.03.2015 um 20:23 schrieb @lbutlr: > On 04 Mar 2015, at 21:46 , Jim Pazarena wrote: >> On 2015-03-02 2:02 AM, Jochen Bern wrote: >>> On 03/01/2015 08:53 AM, Jim Pazarena wrote: >>>> I wonder if there is an easy way to provide dovecot a flat text file of >>>> ipv4 #'s which should be ignored or dropped? >>>> >>>> I have accumulated 45,000+ IPs which routinely try dictionary and >>>> 12345678 password attempts. The file is too big to create firewall >>>> drops [...] >>> >>> The inherent assumption here is that dovecot, using a "flat file", will >>> be able to process the block list more effectively than the firewall, >>> which is a tool written for the *purpose* but supposedly unable to even >>> *try* due to the list's size. That sounds ... counterintuitive. >> >> I am the original poster and just came back to this thread. When the >> first couple replies were "fail2ban" I lost interest. > > Why? Fail2ban is simple to install, simple to setup, and then (and here?s the best part) then you never have to look at it again. > I like fail2ban, but related to its design it is slow, i.e it was never fast enough to drop massive smtp botnets in time at one of my servers. Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From skdovecot at smail.inf.fh-brs.de Thu Mar 5 21:45:57 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen) Date: Thu, 05 Mar 2015 22:45:57 +0100 Subject: RBL with stock Dovecot 2.2.15 (was Re: IP drop list) In-Reply-To: References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> <54F38FBE.3040108@neurotica.com> <54F4137F.8050507@oliwel.de> <54F41E1D.5030901@neurotica.com> Message-ID: <54F8CE95.1080602@smail.inf.fh-brs.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Steffen Kaiser wrote: > passdb { driver = ipdeny args = /matchpattern/action .... > *** } > With next passdb{} as 1st in chain: passdb { driver = checkpassword args = "/tmp/chktst ip=%r service=%s" result_success = continue result_failure = return-fail } and this script BEGIN /tmp/chktst #!/bin/bash echo "$@" >>/tmp/chktst.log # return OK exit 0 # return FAIL exit 1 END I get the log entry: ip=127.0.0.1 service=imap /usr/local/dovecot-2.2.15/libexec/dovecot/checkpassword-reply and with exit 0, the next passdb{} let me login, and with exit 1, all logins fail. So, with the current stock Dovecot you can make RBL calls and decissions with a script. ;-) - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQEVAwUBVPjOlXz1H7kL/d9rAQIDFggAtDGl8rgN3zpOa8QQ1JVgVne5alAzBShN JfWm/4rDLBqPfAeqLX8OGUja19dxru0rJFAZPr673v7I4GfGVu2XHgEFV7qWag/m r32B//ADgvyBc0hwYOy2IQ4Zc2BW7K7Xx9hvbA5ZzmlDwbkIg1fBQ8SDHP7EoPso Io/OD8ADvyGJf0RC6lDF+shhpu1mPGg9YVx+jiUD2EOlnq06JDo51sbaQ0BUGfK3 3TmiWr+yFLALrJAYTkoNbonGioGwPPfSqGwmj5/l0ch4N/k9vAf06IbNyFYTzqh+ apjDUNrTVzTnlUeeadoFNDpqkNCGpZDfEe/C/OImxsmNwQoe9fXjbg== =NQ5g -----END PGP SIGNATURE----- From h.reindl at thelounge.net Thu Mar 5 21:50:39 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Thu, 05 Mar 2015 22:50:39 +0100 Subject: RBL with stock Dovecot 2.2.15 (was Re: IP drop list) In-Reply-To: <54F8CE95.1080602@smail.inf.fh-brs.de> References: <1ba4d67085a5f26c7fabc8c539f15f66@paz.bz> <54F2DB17.2090509@thelounge.net> <54F38FBE.3040108@neurotica.com> <54F4137F.8050507@oliwel.de> <54F41E1D.5030901@neurotica.com> <54F8CE95.1080602@smail.inf.fh-brs.de> Message-ID: <54F8CFAF.5000209@thelounge.net> Am 05.03.2015 um 22:45 schrieb Steffen: > Steffen Kaiser wrote: > >> passdb { driver = ipdeny args = /matchpattern/action .... >> *** } > > With next passdb{} as 1st in chain: > > passdb { > driver = checkpassword > args = "/tmp/chktst ip=%r service=%s" > result_success = continue > result_failure = return-fail > } > > and this script > BEGIN /tmp/chktst > #!/bin/bash > > echo "$@" >>/tmp/chktst.log > # return OK > exit 0 > # return FAIL > exit 1 > END > > I get the log entry: > ip=127.0.0.1 service=imap > /usr/local/dovecot-2.2.15/libexec/dovecot/checkpassword-reply > > and with exit 0, the next passdb{} let me login, and with exit 1, all > logins fail. > > So, with the current stock Dovecot you can make RBL calls and > decissions with a script. ;-) * with a terrible overhead starting a full process * no handling for DNS temp errors and so on * i don't see any RBL handling above, you just call a random script -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From r at sys4.de Fri Mar 6 13:44:08 2015 From: r at sys4.de (Ralf Hildebrandt) Date: Fri, 6 Mar 2015 14:44:08 +0100 Subject: LMTP error: Too many concurrent deliveries for user (in reply to end of DATA command) Message-ID: <20150306134408.GH11501@sys4.de> I updated dovecot today and all over a sudden I'm getting: Mar 6 14:40:46 mail postfix/lmtp[3150]: 3kz95y3nX3zCtTS: to=, relay=127.0.0.1[private/dovecot-lmtp], delay=88, delays=87/0.94/0.01/0.01, dsn=4.3.0, status=deferred (host 127.0.0.1[private/dovecot-lmtp] said: 451 4.3.0 Too many concurrent deliveries for user (in reply to end of DATA command)) Why is that? Which setting must I tweak? -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From r at sys4.de Fri Mar 6 13:46:47 2015 From: r at sys4.de (Ralf Hildebrandt) Date: Fri, 6 Mar 2015 14:46:47 +0100 Subject: LMTP error: Too many concurrent deliveries for user (in reply to end of DATA command) In-Reply-To: <20150306134408.GH11501@sys4.de> References: <20150306134408.GH11501@sys4.de> Message-ID: <20150306134647.GI11501@sys4.de> * Ralf Hildebrandt : > > I updated dovecot today and all over a sudden I'm getting: Old: dovecot-lmtpd:amd64 2:2.2.15-1~auto+136 New: dovecot-lmtpd:amd64 2:2.2.15-1~auto+148 >From Stephan Bosch's repos. -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From r at sys4.de Fri Mar 6 13:56:08 2015 From: r at sys4.de (Ralf Hildebrandt) Date: Fri, 6 Mar 2015 14:56:08 +0100 Subject: LMTP error: Too many concurrent deliveries for user (in reply to end of DATA command) In-Reply-To: <20150306134647.GI11501@sys4.de> References: <20150306134408.GH11501@sys4.de> <20150306134647.GI11501@sys4.de> Message-ID: <20150306135608.GJ11501@sys4.de> * Ralf Hildebrandt : > * Ralf Hildebrandt : > > > > I updated dovecot today and all over a sudden I'm getting: > > Old: > dovecot-lmtpd:amd64 2:2.2.15-1~auto+136 > > New: > dovecot-lmtpd:amd64 2:2.2.15-1~auto+148 > > From Stephan Bosch's repos. Found the issue: # doveconf |grep concurr lmtp_user_concurrency_limit = 0 -> not working # doveconf |grep concurr lmtp_user_concurrency_limit = 10000 -> working again! I guess 0 should disable the limit. It doesn't (anymore). -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From h.reindl at thelounge.net Fri Mar 6 13:57:14 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 06 Mar 2015 14:57:14 +0100 Subject: LMTP error: Too many concurrent deliveries for user (in reply to end of DATA command) In-Reply-To: <20150306134408.GH11501@sys4.de> References: <20150306134408.GH11501@sys4.de> Message-ID: <54F9B23A.8080507@thelounge.net> Am 06.03.2015 um 14:44 schrieb Ralf Hildebrandt: > I updated dovecot today and all over a sudden I'm getting: > > Mar 6 14:40:46 mail postfix/lmtp[3150]: 3kz95y3nX3zCtTS: to=, relay=127.0.0.1[private/dovecot-lmtp], > delay=88, delays=87/0.94/0.01/0.01, dsn=4.3.0, status=deferred (host 127.0.0.1[private/dovecot-lmtp] said: 451 4.3.0 > Too many concurrent deliveries for user (in reply to end of DATA command)) > > Why is that? Which setting must I tweak? lmtp_destination_concurrency_limit on postfix side lmtp_destination_concurrency_limit = $default_destination_concurrency_limit postconf -d default_destination_concurrency_limit default_destination_concurrency_limit = 20 don't overload your storage by a high busy queue combined with to much parallel delivery to the mailstorage - thins don't get really faster that way -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From r at sys4.de Fri Mar 6 13:59:04 2015 From: r at sys4.de (Ralf Hildebrandt) Date: Fri, 6 Mar 2015 14:59:04 +0100 Subject: LMTP error: Too many concurrent deliveries for user (in reply to end of DATA command) In-Reply-To: <54F9B23A.8080507@thelounge.net> References: <20150306134408.GH11501@sys4.de> <54F9B23A.8080507@thelounge.net> Message-ID: <20150306135904.GK11501@sys4.de> * Reindl Harald : > lmtp_destination_concurrency_limit on postfix side It's not a postfix issue. postfix is merely reporting what Dovecot said. -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From h.reindl at thelounge.net Fri Mar 6 14:00:42 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 06 Mar 2015 15:00:42 +0100 Subject: LMTP error: Too many concurrent deliveries for user (in reply to end of DATA command) In-Reply-To: <20150306135904.GK11501@sys4.de> References: <20150306134408.GH11501@sys4.de> <54F9B23A.8080507@thelounge.net> <20150306135904.GK11501@sys4.de> Message-ID: <54F9B30A.9030707@thelounge.net> Am 06.03.2015 um 14:59 schrieb Ralf Hildebrandt: > * Reindl Harald : > >> lmtp_destination_concurrency_limit on postfix side > > It's not a postfix issue. postfix is merely reporting what Dovecot said i know that on my own since i can read maillogs :-) anyways, we even use a lmtp concurrency level of 1 since years -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From tss at iki.fi Fri Mar 6 14:37:54 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 6 Mar 2015 16:37:54 +0200 Subject: LMTP error: Too many concurrent deliveries for user (in reply to end of DATA command) In-Reply-To: <20150306134408.GH11501@sys4.de> References: <20150306134408.GH11501@sys4.de> Message-ID: On 06 Mar 2015, at 15:44, Ralf Hildebrandt wrote: > > I updated dovecot today and all over a sudden I'm getting: > > Mar 6 14:40:46 mail postfix/lmtp[3150]: 3kz95y3nX3zCtTS: to=, relay=127.0.0.1[private/dovecot-lmtp], > delay=88, delays=87/0.94/0.01/0.01, dsn=4.3.0, status=deferred (host 127.0.0.1[private/dovecot-lmtp] said: 451 4.3.0 > Too many concurrent deliveries for user (in reply to end of DATA command)) Oops. Fixed: http://hg.dovecot.org/dovecot-2.2/rev/9c69c197b527 From emmanuelchanel at gmail.com Fri Mar 6 15:12:26 2015 From: emmanuelchanel at gmail.com (Emmanuel Chanel) Date: Sat, 07 Mar 2015 00:12:26 +0900 Subject: dovecot auth-worker error happens when I enabled the dovecot to consider both the system account and the MySQL virtual mailbox databases. Message-ID: <54F9C3DA.6060703@gmail.com> Version: 2.2.9 & 2.2.15 ( http://ppa.launchpad.net/malte.swart/dovecot-2.2/ubuntu ) OS: Ubuntu 14.04 Server LTS I installed Ubuntu 14.04.2 LTS newly on my new server... And I try to configure the mail server considering both my system account and the virtual mailboxes, by seeing https://www.exratione.com/2014/05/a-mailserver-on-ubuntu-1404-postfix-dovecot-mysql/ But when I configured by uncommenting #!include auth-sql.conf.ext , it doesn't auth... Possibly, it's a trouble with the configurations of postfix and of dovecot. But I don't understand why... When I enable either auth-system.conf.ext or auth-sql.conf.ext , it works... My doveconf -n http://pastebin.com/MU1LL63d dovecot-sql.conf.ext http://pastebin.com/PJkTnPB3 tail /var/log/mail.log by debug mode / dovecot 2.2.15 http://pastebin.com/1TRQL49d Mar 6 07:45:12 gateway dovecot: auth: Debug: client in: AUTH#0114#011PLAIN#011service=imap#011secured#011session=xS6uSpIQvQDAqAAC#011lip=192.168.0.1#011rip=192.168.0.2#011lport=143#011rport=38333 Mar 6 07:45:12 gateway dovecot: auth: Debug: client passdb out: CONT#0114#011 Mar 6 07:45:12 gateway dovecot: auth: Debug: client in: CONT Mar 6 07:45:12 gateway dovecot: auth-worker(22933): Debug: pam(emmanuel,192.168.0.2): lookup service=dovecot Mar 6 07:45:12 gateway dovecot: auth: Error: auth worker: Aborted PASSV request for emmanuel: Worker process died unexpectedly Mar 6 07:45:12 gateway dovecot: auth-worker(22936): Debug: sql(emmanuel,192.168.0.2): query: SELECT username as user, password, '/home/vmail//emmanuel' as userdb_home, 'maildir:/home/vmail//emmanuel' as userdb_mail, 65000 as userdb_uid, 8 as userdb_gid FROM mailbox WHERE username = 'emmanuel' AND active = '1' Mar 6 07:45:12 gateway dovecot: auth-worker(22933): Fatal: master: service(auth-worker): child 22933 killed with signal 11 (core dumped) Mar 6 07:45:12 gateway dovecot: auth-worker(22936): sql(emmanuel,192.168.0.2): unknown user Mar 6 07:45:12 gateway dovecot: auth-worker(23755): Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Mar 6 07:45:12 gateway dovecot: auth-worker(23755): Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so Mar 6 07:45:14 gateway dovecot: auth: Debug: client passdb out: FAIL#0114#011user=emmanuel#011temp Mar 6 07:45:14 gateway dovecot: auth: Debug: client in: AUTH#0115#011LOGIN#011service=imap#011secured#011session=xS6uSpIQvQDAqAAC#011lip=192.168.0.1#011rip=192.168.0.2#011lport=143#011rport=38333 Mar 6 07:45:14 gateway dovecot: auth: Debug: client passdb out: CONT#0115#011VXNlcm5hbWU6 Mar 6 07:45:14 gateway dovecot: auth: Debug: client in: CONT Mar 6 07:45:14 gateway dovecot: auth: Debug: client passdb out: CONT#0115#011UGFzc3dvcmQ6 Mar 6 07:45:14 gateway dovecot: auth: Debug: client in: CONT Mar 6 07:45:14 gateway dovecot: auth-worker(22936): Debug: pam(emmanuel,192.168.0.2): lookup service=dovecot Mar 6 07:45:14 gateway dovecot: auth: Error: auth worker: Aborted PASSV request for emmanuel: Worker process died unexpectedly Mar 6 07:45:14 gateway dovecot: auth-worker(23755): Debug: sql(emmanuel,192.168.0.2): query: SELECT username as user, password, '/home/vmail//emmanuel' as userdb_home, 'maildir:/home/vmail//emmanuel' as userdb_mail, 65000 as userdb_uid, 8 as userdb_gid FROM mailbox WHERE username = 'emmanuel' AND active = '1' Mar 6 07:45:14 gateway dovecot: auth-worker(22936): Fatal: master: service(auth-worker): child 22936 killed with signal 11 (core dumped) Mar 6 07:45:14 gateway dovecot: auth-worker(23755): sql(emmanuel,192.168.0.2): unknown user Mar 6 07:45:14 gateway dovecot: auth-worker(23764): Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Mar 6 07:45:14 gateway dovecot: auth-worker(23764): Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so Mar 6 07:45:16 gateway dovecot: auth: Debug: client passdb out: FAIL#0115#011user=emmanuel#011temp Mar 6 07:45:16 gateway dovecot: auth: Debug: client in: AUTH#0116#011PLAIN#011service=imap#011secured#011session=xS6uSpIQvQDAqAAC#011lip=192.168.0.1#011rip=192.168.0.2#011lport=143#011rport=38333#011resp= Mar 6 07:45:16 gateway dovecot: auth-worker(23755): Debug: pam(emmanuel,192.168.0.2): lookup service=dovecot Mar 6 07:45:16 gateway dovecot: auth: Error: auth worker: Aborted PASSV request for emmanuel: Worker process died unexpectedly Mar 6 07:45:16 gateway dovecot: auth-worker(23764): Debug: sql(emmanuel,192.168.0.2): query: SELECT username as user, password, '/home/vmail//emmanuel' as userdb_home, 'maildir:/home/vmail//emmanuel' as userdb_mail, 65000 as userdb_uid, 8 as userdb_gid FROM mailbox WHERE username = 'emmanuel' AND active = '1' Mar 6 07:45:16 gateway dovecot: auth-worker(23755): Fatal: master: service(auth-worker): child 23755 killed with signal 11 (core dumped) Mar 6 07:45:16 gateway dovecot: auth-worker(23764): sql(emmanuel,192.168.0.2): unknown user Mar 6 07:45:16 gateway dovecot: auth-worker(23767): Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Mar 6 07:45:16 gateway dovecot: auth-worker(23767): Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so Mar 6 07:45:18 gateway dovecot: auth: Debug: client passdb out: FAIL#0116#011user=emmanuel#011temp -- Emmanuel Chanel( emmanuelchanel at gmail.com ) From tss at iki.fi Fri Mar 6 15:53:10 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 6 Mar 2015 17:53:10 +0200 Subject: v2.2.16 release candidate released Message-ID: http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz.sig Looks like it's been a long time since v2.2.15. There have been a ton of changes since it was released though, so here's a release candidate first to find out if somebody can find any bugs before the final v2.2.16. Unfortunately I haven't had time/energy to read Dovecot mailing list for a while now. I'm hoping this will change, but I don't really expect it to happen anytime soon. On the positive side for Dovecot, it's now becoming used in more and more multi-million user installations, which brings all kinds of nice new improvements. The largest changes since v2.2.15: * dbox: Resyncing (e.g. doveadm force-resync) no longer deletes dovecot.index.cache file. The cache file was rarely the problem so this just caused unnecessary slowness. * Mailbox name limits changed during mailbox creation: Each part of a hierarchical name (e.g. "x" or "y" in "x/y") can now be up to 255 chars long (instead of 200). This also reduces the max number of hierarchical levels to 16 (instead of 20) to keep the maximum name length 4096 (a common PATH_MAX limit). The 255 char limit is hopefully large enough for migrations from all existing systems. It's also the limit on many filesystems. + director: Added director_consistent_hashing setting to enable consistent hashing (instead of the mostly-random MD5 hashing). This causes fewer user moves between backends when backend counts are changed, which may improve performance (mainly due to caching). + director: Added support for "tags", which allows one director ring to serve multiple backend clusters with different sets of users. + LMTP server: Added lmtp_user_concurrency_limit setting to limit how many LMTP deliveries can be done concurrently for a single user. + LMTP server: Added support for STARTTLS command. + If logging data is generated faster than it can be written, log a warning about it and show information about it in log process's process title in ps output. Also don't allow a single service to flood too long at the cost of delaying other services' logging. + stats: Added support for getting global statistics. + stats: Use the same session IDs as the rest of Dovecot. + stats: Plugins can now create their own statistics fields + doveadm server: Non-mail related commands can now also be used via doveadm server (TCP socket). + doveadm proxying: passdb lookup can now override doveadm_port and change the username. + doveadm: Search query supports now "oldestonly" parameter to stop immediately on the first non-match. This can be used to optimize: doveadm expunge mailbox Trash savedbefore 30d oldestonly + doveadm: Added "save" command to directly save mails to specified mailbox (bypassing Sieve). + doveadm fetch: Added body.snippet field, which returns the first 100 chars of a message without whitespace or HTML tags. The result is stored into dovecot.index.cache, so it can be fetched efficiently. + dsync: Added -t parameter to sync only mails newer than the given received-timestamp. + dsync: Added -F [-] parameter to sync only mails with[out] the given flag/keyword. + dsync: Added -a parameter to specify the virtual mailbox containing user's all mails. If this mailbox is already found to contain the wanted mail (by its GUID), the message is copied from there instead of being re-saved. (This isn't efficient enough yet for incremental replication.) + dsync: -m parameter can now specify \Special-use names for mailboxes. + imapc: Added imapc_features=gmail-migration to help migrations from GMail. See http://wiki2.dovecot.org/Migration/Gmail + imapc: Added imapc_features=search to support IMAP SEARCH command. (Currently requires ESEARCH support from remote server.) + expire plugin: Added expire_cache=yes setting to cache most of the database lookups in dovecot index files. + quota: If overquota-flag in userdb doesn't match the current quota usage, execute a configured script. + redis dict: Added support for expiring keys (:expire_secs=n) and specifying the database number (:db=n) - auth: Don't crash if master user login is attempted without any configured master=yes passdbs - Parsing UTF-8 text for mails could have caused broken results sometimes if buffering was split in the middle of a UTF-8 character. This affected at least searching messages. - String sanitization for some logged output wasn't done properly: UTF-8 text could have been truncated wrongly or the truncation may not have happened at all. - fts-lucene: Lookups from virtual mailbox consisting of over 32 physical mailboxes could have caused crashes. From lee at hexkey.co.uk Wed Mar 4 23:05:43 2015 From: lee at hexkey.co.uk (Lee Maguire) Date: Wed, 4 Mar 2015 23:05:43 +0000 Subject: Sieve editheader should support adding Received and Auto-Submitted headers Message-ID: <37039C5C-750F-4DCD-9F09-DEBD904A5D56@hexkey.co.uk> In order to simplify auto-response suppression and other filtering, I want to synthesise an Auto-Submitted header on notification mails that should have them but do not. In a sieve file I add the following: if anyof ( header :contains ["From","Sender"] ?info at example.com" ) { if header :contains "Subject" [" just joined "," just left "] { if not exists "Auto-Submitted" { addheader "Auto-Submitted" "auto-generated (via sieve)"; } } However this doesn?t work and results in the following error > main script: line 205: warning: addheader action: specified header field `Auto-Submitted' is protected; modification denied. (Workaround for this specific issue would probably be to use a List-* header instead.) Looking at http://hg.rename-it.nl/dovecot-2.2-pigeonhole/file/c8edece267cd/src/lib-sieve/plugins/editheader/ext-editheader-common.c I can see that "Auto-Submitted? and ?Received? are hardcoded to be blocked without reference to whether they are to be added or removed. However, the hard restriction in RFC 5293 on Auto-Submitted and Received only applies to the use of deleteheader and not addheader. https://tools.ietf.org/html/rfc5293#section-6 > As a matter of local policy, implementations MAY limit which header > fields may be deleted and which header fields may be added. However, > implementations MUST NOT permit attempts to delete "Received" and > "Auto-Submitted" header fields and MUST permit both addition and > deletion of the "Subject" header field. Using Dovecot version 2.1.7 but I assume it?s not version specific issue. From r at sys4.de Fri Mar 6 16:47:34 2015 From: r at sys4.de (Ralf Hildebrandt) Date: Fri, 6 Mar 2015 17:47:34 +0100 Subject: v2.2.16 release candidate released In-Reply-To: References: Message-ID: <20150306164734.GL11501@sys4.de> * Timo Sirainen : > + LMTP server: Added lmtp_user_concurrency_limit setting to limit how > many LMTP deliveries can be done concurrently for a single user. That's broken! 0 doesn't remove the limit :( -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From tss at iki.fi Fri Mar 6 16:54:49 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 6 Mar 2015 18:54:49 +0200 Subject: v2.2.16 release candidate released In-Reply-To: <20150306164734.GL11501@sys4.de> References: <20150306164734.GL11501@sys4.de> Message-ID: On 06 Mar 2015, at 18:47, Ralf Hildebrandt wrote: > > * Timo Sirainen : > >> + LMTP server: Added lmtp_user_concurrency_limit setting to limit how >> many LMTP deliveries can be done concurrently for a single user. > > That's broken! 0 doesn't remove the limit :( Should have been fixed in the rc1 already: http://hg.dovecot.org/dovecot-2.2/rev/9c69c197b527 From stephan at rename-it.nl Fri Mar 6 19:05:42 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 06 Mar 2015 20:05:42 +0100 Subject: Sieve editheader should support adding Received and Auto-Submitted headers In-Reply-To: <37039C5C-750F-4DCD-9F09-DEBD904A5D56@hexkey.co.uk> References: <37039C5C-750F-4DCD-9F09-DEBD904A5D56@hexkey.co.uk> Message-ID: <54F9FA86.3090104@rename-it.nl> On 3/5/2015 12:05 AM, Lee Maguire wrote: > However, the hard restriction in RFC 5293 on Auto-Submitted and > Received only applies to the use of deleteheader and not addheader. > https://tools.ietf.org/html/rfc5293#section-6 >> As a matter of local policy, implementations MAY limit which header >> fields may be deleted and which header fields may be added. However, >> implementations MUST NOT permit attempts to delete "Received" and >> "Auto-Submitted" header fields and MUST permit both addition and >> deletion of the "Subject" header field. > Using Dovecot version 2.1.7 but I assume it?s not version specific issue. Oh, looks like I missed that part. Fixed: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/c13471f102be I've split up the configuration to allow separate configuration of addheader/deleteheader. Updated documentation: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/raw-file/c13471f102be/doc/extensions/editheader.txt Regards, Stephan. From dmiller at amfes.com Fri Mar 6 19:58:49 2015 From: dmiller at amfes.com (Daniel Miller) Date: Fri, 06 Mar 2015 11:58:49 -0800 Subject: v2.2.16 release candidate released In-Reply-To: References: Message-ID: On 3/6/2015 7:53 AM, Timo Sirainen wrote: > http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz > http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz.sig > > Looks like it's been a long time since v2.2.15. There have been a ton of changes since it was released though, so here's a release candidate first to find out if somebody can find any bugs before the final v2.2.16. > > Unfortunately I haven't had time/energy to read Dovecot mailing list for a while now. I'm hoping this will change, but I don't really expect it to happen anytime soon. On the positive side for Dovecot, it's now becoming used in more and more multi-million user installations, which brings all kinds of nice new improvements. Great to hear both Dovecot and you are doing well. I do need to ask you to check the list for two threads: mdbox attachment errors Rebuilding SIS attachment links from log A few of us have been having SIS problems. -- Daniel From dmiller at amfes.com Fri Mar 6 20:13:46 2015 From: dmiller at amfes.com (Daniel Miller) Date: Fri, 06 Mar 2015 12:13:46 -0800 Subject: Identify mail for attachment hash Message-ID: How can I find the message associated with a given attachment hash? For example, if I know that: /ff/23/ff23873c712474ad0f48cbd81485eaeaba3d8468-e1b4513a2eb8eb53b16400009db5accb is missing, I want to find the message associated with it so I can determine if it's critical and I need to recover it from another source, or if I can simply delete the old message. -- Daniel From tss at iki.fi Sat Mar 7 16:33:46 2015 From: tss at iki.fi (Timo Sirainen) Date: Sat, 7 Mar 2015 18:33:46 +0200 Subject: v2.2.16 release candidate released In-Reply-To: References: Message-ID: <23FB93FC-B8F9-4C94-9BDE-45260EF6A8C7@iki.fi> On 06 Mar 2015, at 21:58, Daniel Miller wrote: > > On 3/6/2015 7:53 AM, Timo Sirainen wrote: >> http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz >> http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz.sig >> >> Looks like it's been a long time since v2.2.15. There have been a ton of changes since it was released though, so here's a release candidate first to find out if somebody can find any bugs before the final v2.2.16. >> >> Unfortunately I haven't had time/energy to read Dovecot mailing list for a while now. I'm hoping this will change, but I don't really expect it to happen anytime soon. On the positive side for Dovecot, it's now becoming used in more and more multi-million user installations, which brings all kinds of nice new improvements. > > Great to hear both Dovecot and you are doing well. I do need to ask you to check the list for two threads: > > mdbox attachment errors > Rebuilding SIS attachment links from log > > A few of us have been having SIS problems. Unless there's a way to reproduce a bug I don't think I can do anything about it (I could spend hours looking at the code or trying to reproduce it and come up with nothing). But a while ago I did think about a SIS redesign that would make it much less likely to break - just need to get it actually implemented: Currently single instance storage works by having one global directory that contains all the attachments. They are hashed by the attachment content, so for example /var/attachments/ac/7d/ac7d1274891248912489124 would be the attachment. Then each instance would have its own hard link to it, e.g. /var/attachments/ac/7d/hashes/ac7d1274891248912489124-1234567890. sdbox and mdbox can use these by containing the "ac7d1274891248912489124-1234567890" in the header metadata. When mail is deleted, the hard link is deleted. If the link count had been 2, the original attachment file was deleted also. (There's of course some race conditions here, but in those rare situations the attachment would just be duplicated, which isn't too bad.) The main problem with the old design is that all the users' attachments are dumped into a single global directory. It's difficult to take backups and in general it seems too difficult to manage correctly so I haven't really recommended using it in any bigger installations. So here's the new idea, which is nearly the same as the old, but with a small change that makes it much nicer I think: Instead of storing the attachment hard links to a global dir, store the hard links under the user's mail dir. This way taking backups doesn't require anything complicated, just tar the user's mail dir. You can rm -rf the user without forever leaving the user's attachments lying around in the global dir (assuming there's a job that periodically cleans out attachments with link count=1). In general there's no easy way to accidentally break things. The only new complication here is that if users are split to multiple filesystems, hard linking across them isn't going to work. So this would then require not only having a per-user mail directory but also per-user attachment directory (which would actually be the per-filesystem attachment dir). The SIS is implemented as lib-fs backend wrapper, so a new one could be implemented easily without breaking the old one. From absolutely_free at libero.it Sat Mar 7 18:20:00 2015 From: absolutely_free at libero.it (absolutely_free at libero.it) Date: Sat, 7 Mar 2015 19:20:00 +0100 (CET) Subject: Expunge messages older than Message-ID: <9571164.1711991425752400948.JavaMail.httpd@webmail-39.iol.local> Hi, I am using Dovecot 2.2.13 on cPanel server. Is there a way to delete messages older than some date for a number of users (in every subfolder)? Thank you From gedalya at gedalya.net Sat Mar 7 18:42:14 2015 From: gedalya at gedalya.net (Gedalya) Date: Sat, 07 Mar 2015 13:42:14 -0500 Subject: Expunge messages older than In-Reply-To: <9571164.1711991425752400948.JavaMail.httpd@webmail-39.iol.local> References: <9571164.1711991425752400948.JavaMail.httpd@webmail-39.iol.local> Message-ID: <54FB4686.3050404@gedalya.net> On 03/07/2015 01:20 PM, absolutely_free at libero.it wrote: > Hi, I am using Dovecot 2.2.13 on cPanel server. > Is there a way to delete messages older than some date for a number of users (in every subfolder)? > Thank you http://wiki2.dovecot.org/Plugins/Expire From david at davrom.com Sun Mar 8 04:20:48 2015 From: david at davrom.com (David.M.Clark) Date: Sun, 08 Mar 2015 14:20:48 +1000 Subject: Outlook 2013/2010 nightmare Message-ID: <54FBCE20.9010305@davrom.com> Hi All, This is my first post so forgive me if this hits the wrong list. I have been using dovecot for years happily with SendMail on mainly CentOS servers, and my customers are starting to more and more use Thunderbird, so my perfect model is: Linux --> Dovecot --> SendMail --> Thunderbird (or mobile phone e-mail app). I do have some customers using Outlook or Windows Live Mail, and these are for the most part working fine with IMAP - I don't do POP. My mod to the dovecot.conf file: disable_plaintext_auth=no My mod to conf.d/10-mail.conf: #mail_location = mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u My mod to conf.d/10-master.conf: #default_internal_user = dovecot service imap-login { inet_listener imap { port = 143 } inet_listener imap_second { port = 10143 } inet_listener imaps { #port = 993 #ssl = yes } - I do the 10143 bit so I can pseudo hide port 143 for clients with devices that access both internally and externally - port forwarded on MikroTik routers for my customers. All goes well for many years with the above configs, and continues to be awesome as always. Under the user's ${HOME} directory they end up with a lovely 'mail' directory and all of the e-mail client folders live in harmony under there. This weekend I am helping a customer cut over his customer from legacy POP based e-mail uses with splatterings of Outlook 2010 and 2013 throughout the company, to using Linux with IMAP in the setup as described above as per my standard configs. The issue starts when you add an IMAP user to the Outlook client and upon opening it, initially, it tries to find a "Sent" Items folder under IMAP to send from. To this end I have traditionally gone into the dialogue box that opens, click on the IMAP account to receive a list of possible folders, but it now immediately crashes indicating Outlook has had an error, and it all goes to hell from there. Outlook then cannot open but keeps crashing with its "unknown error". Up to last week at another site with Windows Live Mail, this was not an issue. By modifying the e-mail account via Control Panel, I can delete or modify the account and try again. Here is what I found: If I don't specify a 'root' folder under the "Advanced" tab which also contains the port selections for 143 and 25, it fails. If I follow the MS recommended root folder example from Microsoft and use INBOX, or Inbox, it 'does' stop the error but you can't create subfolders - I am assuming as this is picking up the .imap/INBOX folder or something and makes sense that it can't create folders under INBOX/Inbox, particularly if this is a file rather than a directory that dovecot/Outlook are pointing to. What I have found is I can 'lie' to Outlook and put in a 'mail' folder. This results in a ${HOME}/login_name/mail/mail folder, but I don't care, as it happily puts files under mail/mail. Now here is something totally strange: If I try to put this same account on another PC so users can 'share' the same e-mail account, even with my 'mail' root folder work-around, it crashes again. All through this, Thuderbird works fine and so does RoundCube - SOGo will be up and working a bit later once I get the initial part up and going... but I digress but good to know it is only Outlook doing this. To share multiple accounts I am currently getting the subsequent PCs that need to share a universal account to select a root folder mail2, mail3 etc. I then delete these folders and symbolic link the mail2, mail3 etc to the mail directory - yucky fix, but kind of works with some auto scripting of .subscriptions to follow so all of the Outlook clients sync to the one 'real' root folder. I have never seen this before and have customers running all kinds of IMAP e-mail clients to dovecot on Linux. I saw an MS posted bug on something for IMAP and they recommend rolling back MS updates to fix it - but I am not sure the client can or will do that. This kind of random issue that only affects this site at present is why I only recommend people use e-mail clients like Thunderbird because it just 'works'. The users at this site would not switch and will cite they have been working in a POP situation on their other Linux box since Adam was a boy. Any help would be very much appreciated. I like to think I can do this with my eyes shut by now, but this one has got me stumped. -- As always, I remain at your service. Kindest Regards, David.M.Clark (Director - Senior Linux/UNIX Consultant) =----------------------------------------------------------------------= Davrom Consulting Pty Ltd E-mail : david at davrom.com PO Box 1644, Sunnybank Hills, 4109 Twitter: @DavidClark1961 ABN: 81 096 990 804 MSN: david at davrom.com Phone/Fax: 61-7-32720267 Skype: dmc1961 Mobile: 0418-763124 Google: dmc1961 at gmail.com Podcast: http://www.davrom.com/ldup/ldup_rss.xml =---------------------- http://www.davrom.com -------------------------= Specialising in: Linux (Fedora/RedHat/CentOS), UNIX, SCO, MikroTik, Networking/Internet, E-mail/Web Technologies =----------------------------------------------------------------------= Please note: Any e-mail communication bearing this signature is for the exclusive purpose of the sender and is not for publication without the expressed permission of the sender or respective sender's organisation. =----------------------------------------------------------------------= From emailbuilder88 at yahoo.com Sun Mar 8 07:54:18 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Sat, 7 Mar 2015 23:54:18 -0800 Subject: Sieve can't find Extprograms or Extdata Message-ID: <1425801258.56989.YahooMailBasic@web142406.mail.bf1.yahoo.com> Hi, On a new install-from-source with Dovecot 2.2.16rc1, Piegeonhole 0.4.6 and a grab of the newest Extdata code, I confirmed basic Sieve functionality is working (made a simple sieve script with a test on message subject resulting in a fileinto action). But I cannot get Sieve to see Extdata or Extprograms. sievec reports "Warning: sieve: ignored unknown extension 'vnd.dovecot.filter' while configuring available extensions" and the same for vnd.dovecot.extdata. And of course for the user script, "error: require command: unknown Sieve capability `vnd.dovecot.filter'" (same with extdata) I'm pretty sure this is a library path problem, but I've tried symlinking and restarting dovecot with as many iterations of library paths as I can think of but still same problem. The lib90_sieve_extdata_plugin.so and lib90_sieve_extprograms_plugin.so are in /usr/local/lib/dovecot/sieve and all the rest of the normal dovecot libraries are one level above in /usr/local/lib/dovecot. I tried creating symlinks for a "modules" directory and a "modules/sieve" directory, I tried symlinking these two librarie files themselves into the top level dovecot library directory, and a few other ideas, but no luck. Is there a way I can tell where Sieve expects its libraries to reside? To see where it is looking? I didn't provide any arguments to the configure command for pigeonhole as well as extdata and it seems like the libs got placed somewhere sensible, but I don't know what else to do. From emailbuilder88 at yahoo.com Sun Mar 8 08:08:34 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Sun, 8 Mar 2015 00:08:34 -0800 Subject: Sieve can't find Extprograms or Extdata In-Reply-To: <1425801258.56989.YahooMailBasic@web142406.mail.bf1.yahoo.com> Message-ID: <1425802114.43477.YahooMailBasic@web142401.mail.bf1.yahoo.com> Also, of course I have entered this in 90-plugins.conf plugin { seive_plugins = sieve_extdata sieve_extprograms sieve_extensions = +vnd.dovecot.filter +vnd.dovecot.extdata } And enabled sieve for lmtp (as I noted, I have tested that simple sieve scripting is working OK) protocol lmtp { mail_plugins = " sieve" } > On a new install-from-source with Dovecot 2.2.16rc1, > Piegeonhole 0.4.6 and a grab of the newest Extdata code, I > confirmed basic Sieve functionality is working (made a > simple sieve script with a test on message subject resulting > in a fileinto action). > > But I cannot get Sieve to see Extdata or Extprograms. sievec > reports "Warning: sieve: ignored unknown extension > 'vnd.dovecot.filter' while configuring available extensions" > and the same for vnd.dovecot.extdata.? And of course > for the user script, "error: require command: unknown Sieve > capability `vnd.dovecot.filter'" (same with extdata) > > I'm pretty sure this is a library path problem, but I've > tried symlinking and restarting dovecot with as many > iterations of library paths as I can think of but still same > problem. > > The lib90_sieve_extdata_plugin.so and > lib90_sieve_extprograms_plugin.so are in > /usr/local/lib/dovecot/sieve and all the rest of the normal > dovecot libraries are one level above in > /usr/local/lib/dovecot. > > I tried creating symlinks for a "modules" directory and a > "modules/sieve" directory, I tried symlinking these two > librarie files themselves into the top level dovecot library > directory, and a few other ideas, but no luck. > > Is there a way I can tell where Sieve expects its libraries > to reside? To see where it is looking? I didn't provide any > arguments to the configure command for pigeonhole as well as > extdata and it seems like the libs got placed somewhere > sensible, but I don't know what else to do. From j.echter at echter-kuechen-elektro.de Sun Mar 8 08:46:26 2015 From: j.echter at echter-kuechen-elektro.de (J. Echter) Date: Sun, 08 Mar 2015 09:46:26 +0100 Subject: Sieve can't find Extprograms or Extdata In-Reply-To: <1425802114.43477.YahooMailBasic@web142401.mail.bf1.yahoo.com> References: <1425802114.43477.YahooMailBasic@web142401.mail.bf1.yahoo.com> Message-ID: <54FC0C62.3090303@echter-kuechen-elektro.de> Am 08.03.2015 um 09:08 schrieb E.B.: > Also, of course I have entered this in 90-plugins.conf > > plugin { > seive_plugins = sieve_extdata sieve_extprograms > maybe you have a typo... seive != sieve From HFlor at gmx.de Sun Mar 8 13:40:56 2015 From: HFlor at gmx.de (Hardy Flor) Date: Sun, 08 Mar 2015 14:40:56 +0100 Subject: v2.2.16 release candidate released In-Reply-To: <23FB93FC-B8F9-4C94-9BDE-45260EF6A8C7@iki.fi> References: <23FB93FC-B8F9-4C94-9BDE-45260EF6A8C7@iki.fi> Message-ID: <54FC5168.7010500@gmx.de> Instead of deleting it, it should be marked "deleted" by renaming only. If it is then required, so it can be recovered. This directory can be deleted by another control process that searches the mbox files for references. What has become of the question that attachments are not only saved as separate files? (mdbox-files over 2 MB from 17.03.2014) Am 07.03.2015 um 17:33 schrieb Timo Sirainen: > ... When mail is deleted, the hard link is deleted. If the link count > had been 2, the original attachment file was deleted also. From emmanuelchanel at gmail.com Sun Mar 8 15:45:11 2015 From: emmanuelchanel at gmail.com (Emmanuel Chanel) Date: Mon, 09 Mar 2015 00:45:11 +0900 Subject: dovecot auth-worker error happens when I enabled the dovecot to consider both the system account and the MySQL virtual mailbox databases. In-Reply-To: <54F9C3DA.6060703@gmail.com> References: <54F9C3DA.6060703@gmail.com> Message-ID: <54FC6E87.9040400@gmail.com> This is the configuration... ------- $ sudo doveconf -n # 2.2.15: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.6 (3e924b1b6c5c+) # OS: Linux 3.16.0-31-generic x86_64 Ubuntu 14.04.2 LTS auth_debug = yes auth_mechanisms = plain login auth_verbose = yes first_valid_uid = 1000 mail_debug = yes mail_location = maildir:~/Maildir namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } postmaster_address = emmanuelchanel at gmail.com protocols = " imap" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } ssl_cert = Version: 2.2.9 & 2.2.15 ( > http://ppa.launchpad.net/malte.swart/dovecot-2.2/ubuntu ) > OS: Ubuntu 14.04 Server LTS > I installed Ubuntu 14.04.2 LTS newly on my new server... > And I try to configure the mail server considering both my system > account and the virtual mailboxes, by seeing > https://www.exratione.com/2014/05/a-mailserver-on-ubuntu-1404-postfix-dovecot-mysql/ > But when I configured by uncommenting #!include auth-sql.conf.ext , it > doesn't auth... > Possibly, it's a trouble with the configurations of postfix and of > dovecot. But I don't understand why... > When I enable either auth-system.conf.ext or auth-sql.conf.ext , it > works... > > My doveconf -n http://pastebin.com/MU1LL63d > dovecot-sql.conf.ext http://pastebin.com/PJkTnPB3 > tail /var/log/mail.log by debug mode / dovecot 2.2.15 > http://pastebin.com/1TRQL49d > > Mar 6 07:45:12 gateway dovecot: auth: Debug: client > in: > AUTH#0114#011PLAIN#011service=imap#011secured#011session=xS6uSpIQvQDAqAAC#011lip=192.168.0.1#011rip=192.168.0.2#011lport=143#011rport=38333 > > > Mar 6 07:45:12 gateway dovecot: auth: Debug: client passdb out: > CONT#0114#011 > Mar 6 07:45:12 gateway dovecot: auth: Debug: client in: CONT > Mar 6 07:45:12 gateway dovecot: auth-worker(22933): Debug: > pam(emmanuel,192.168.0.2): lookup service=dovecot > Mar 6 07:45:12 gateway dovecot: auth: Error: auth worker: Aborted > PASSV request for emmanuel: Worker process died unexpectedly > Mar 6 07:45:12 gateway dovecot: auth-worker(22936): Debug: > sql(emmanuel,192.168.0.2): query: SELECT username as user, password, > '/home/vmail//emmanuel' as userdb_home, > 'maildir:/home/vmail//emmanuel' as userdb_mail, 65000 as userdb_uid, 8 > as userdb_gid FROM mailbox WHERE username = 'emmanuel' AND active = '1' > Mar 6 07:45:12 gateway dovecot: auth-worker(22933): Fatal: master: > service(auth-worker): child 22933 killed with signal 11 (core dumped) > Mar 6 07:45:12 gateway dovecot: auth-worker(22936): > sql(emmanuel,192.168.0.2): unknown user > Mar 6 07:45:12 gateway dovecot: auth-worker(23755): Debug: Loading > modules from directory: /usr/lib/dovecot/modules/auth > Mar 6 07:45:12 gateway dovecot: auth-worker(23755): Debug: Module > loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so > Mar 6 07:45:14 gateway dovecot: auth: Debug: client passdb out: > FAIL#0114#011user=emmanuel#011temp > Mar 6 07:45:14 gateway dovecot: auth: Debug: client in: > AUTH#0115#011LOGIN#011service=imap#011secured#011session=xS6uSpIQvQDAqAAC#011lip=192.168.0.1#011rip=192.168.0.2#011lport=143#011rport=38333 > Mar 6 07:45:14 gateway dovecot: auth: Debug: client passdb out: > CONT#0115#011VXNlcm5hbWU6 > Mar 6 07:45:14 gateway dovecot: auth: Debug: client in: CONT > Mar 6 07:45:14 gateway dovecot: auth: Debug: client passdb out: > CONT#0115#011UGFzc3dvcmQ6 > Mar 6 07:45:14 gateway dovecot: auth: Debug: client in: CONT > Mar 6 07:45:14 gateway dovecot: auth-worker(22936): Debug: > pam(emmanuel,192.168.0.2): lookup service=dovecot > Mar 6 07:45:14 gateway dovecot: auth: Error: auth worker: Aborted > PASSV request for emmanuel: Worker process died unexpectedly > Mar 6 07:45:14 gateway dovecot: auth-worker(23755): Debug: > sql(emmanuel,192.168.0.2): query: SELECT username as user, password, > '/home/vmail//emmanuel' as userdb_home, > 'maildir:/home/vmail//emmanuel' as userdb_mail, 65000 as userdb_uid, 8 > as userdb_gid FROM mailbox WHERE username = 'emmanuel' AND active = '1' > Mar 6 07:45:14 gateway dovecot: auth-worker(22936): Fatal: master: > service(auth-worker): child 22936 killed with signal 11 (core dumped) > Mar 6 07:45:14 gateway dovecot: auth-worker(23755): > sql(emmanuel,192.168.0.2): unknown user > Mar 6 07:45:14 gateway dovecot: auth-worker(23764): Debug: Loading > modules from directory: /usr/lib/dovecot/modules/auth > Mar 6 07:45:14 gateway dovecot: auth-worker(23764): Debug: Module > loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so > Mar 6 07:45:16 gateway dovecot: auth: Debug: client passdb out: > FAIL#0115#011user=emmanuel#011temp > Mar 6 07:45:16 gateway dovecot: auth: Debug: client in: > AUTH#0116#011PLAIN#011service=imap#011secured#011session=xS6uSpIQvQDAqAAC#011lip=192.168.0.1#011rip=192.168.0.2#011lport=143#011rport=38333#011resp= > Mar 6 07:45:16 gateway dovecot: auth-worker(23755): Debug: > pam(emmanuel,192.168.0.2): lookup service=dovecot > Mar 6 07:45:16 gateway dovecot: auth: Error: auth worker: Aborted > PASSV request for emmanuel: Worker process died unexpectedly > Mar 6 07:45:16 gateway dovecot: auth-worker(23764): Debug: > sql(emmanuel,192.168.0.2): query: SELECT username as user, password, > '/home/vmail//emmanuel' as userdb_home, > 'maildir:/home/vmail//emmanuel' as userdb_mail, 65000 as userdb_uid, 8 > as userdb_gid FROM mailbox WHERE username = 'emmanuel' AND active = '1' > Mar 6 07:45:16 gateway dovecot: auth-worker(23755): Fatal: master: > service(auth-worker): child 23755 killed with signal 11 (core dumped) > Mar 6 07:45:16 gateway dovecot: auth-worker(23764): > sql(emmanuel,192.168.0.2): unknown user > Mar 6 07:45:16 gateway dovecot: auth-worker(23767): Debug: Loading > modules from directory: /usr/lib/dovecot/modules/auth > Mar 6 07:45:16 gateway dovecot: auth-worker(23767): Debug: Module > loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so > Mar 6 07:45:18 gateway dovecot: auth: Debug: client passdb out: > FAIL#0116#011user=emmanuel#011temp > -- Emmanuel Chanel( emmanuelchanel at gmail.com ) From emmanuelchanel at gmail.com Sun Mar 8 16:35:28 2015 From: emmanuelchanel at gmail.com (Emmanuel Chanel) Date: Mon, 09 Mar 2015 01:35:28 +0900 Subject: dovecot auth-worker error happens when I enabled the dovecot to consider both the system account and the MySQL virtual mailbox databases. In-Reply-To: <54FC6E87.9040400@gmail.com> References: <54F9C3DA.6060703@gmail.com> <54FC6E87.9040400@gmail.com> Message-ID: <54FC7A50.3040501@gmail.com> Core Dump(?) from /var/log/apport.log on Ubuntu Server 14.04 LTS... ---------------------------------- ERROR: apport (pid 29249) Mon Mar 9 01:26:31 2015: called for pid 29239, signal 11, core limit 0 ERROR: apport (pid 29249) Mon Mar 9 01:26:31 2015: executable: /usr/lib/dovecot/auth (command line "dovecot/auth -w") ERROR: apport (pid 29249) Mon Mar 9 01:26:31 2015: Unhandled exception: Traceback (most recent call last): File "/usr/share/apport/apport", line 391, in if is_closing_session(pid, pidstat.st_uid): File "/usr/share/apport/apport", line 213, in is_closing_session env = e.read().split('\0') File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xab in position 0: ordinal not in range(128) ERROR: apport (pid 29249) Mon Mar 9 01:26:31 2015: pid: 29249, uid: 0, gid: 0, euid: 0, egid: 0 ERROR: apport (pid 29249) Mon Mar 9 01:26:31 2015: environment: environ({}) ERROR: apport (pid 29260) Mon Mar 9 01:26:37 2015: called for pid 29241, signal 11, core limit 0 ERROR: apport (pid 29260) Mon Mar 9 01:26:37 2015: executable: /usr/lib/dovecot/auth (command line "dovecot/auth -w") ERROR: apport (pid 29260) Mon Mar 9 01:26:37 2015: Unhandled exception: Traceback (most recent call last): File "/usr/share/apport/apport", line 391, in if is_closing_session(pid, pidstat.st_uid): File "/usr/share/apport/apport", line 213, in is_closing_session env = e.read().split('\0') File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xab in position 0: ordinal not in range(128) ERROR: apport (pid 29260) Mon Mar 9 01:26:37 2015: pid: 29260, uid: 0, gid: 0, euid: 0, egid: 0 ERROR: apport (pid 29260) Mon Mar 9 01:26:37 2015: environment: environ({}) ERROR: apport (pid 29264) Mon Mar 9 01:26:43 2015: called for pid 29250, signal 11, core limit 0 ERROR: apport (pid 29264) Mon Mar 9 01:26:43 2015: executable: /usr/lib/dovecot/auth (command line "dovecot/auth -w") ERROR: apport (pid 29264) Mon Mar 9 01:26:43 2015: Unhandled exception: Traceback (most recent call last): File "/usr/share/apport/apport", line 391, in if is_closing_session(pid, pidstat.st_uid): File "/usr/share/apport/apport", line 213, in is_closing_session env = e.read().split('\0') File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xab in position 0: ordinal not in range(128) ERROR: apport (pid 29264) Mon Mar 9 01:26:43 2015: pid: 29264, uid: 0, gid: 0, euid: 0, egid: 0 ERROR: apport (pid 29264) Mon Mar 9 01:26:43 2015: environment: environ({}) -- Emmanuel Chanel( emmanuelchanel at gmail.com ) From emailbuilder88 at yahoo.com Sun Mar 8 18:11:19 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Sun, 8 Mar 2015 11:11:19 -0700 Subject: Sieve can't find Extprograms or Extdata In-Reply-To: <54FC0C62.3090303@echter-kuechen-elektro.de> Message-ID: <1425838279.90062.YahooMailBasic@web142406.mail.bf1.yahoo.com> > > Also, of course I have entered this in 90-plugins.conf > > > > > > plugin { > >???seive_plugins = sieve_extdata sieve_extprograms > > maybe you have a typo... > seive != sieve Oh yes that was problem!! Thank you!! Tricky because it's the plugins section, dovecot cannot lint it so typos won't be detected. Thank you one time more! And to Stephan Bosch, consider this a vote to keep Extdata alive. It's better in many ways than spawning a script to do data lookups. I hope the dovecot dict mechanism can be enhanced some day to allow more elaborate controls and customization of lookups. From emmanuelchanel at gmail.com Sun Mar 8 21:23:51 2015 From: emmanuelchanel at gmail.com (Emmanuel Chanel) Date: Mon, 09 Mar 2015 06:23:51 +0900 Subject: dovecot auth-worker error happens when I enabled the dovecot to consider both the system account and the MySQL virtual mailbox databases. In-Reply-To: <54FC7A50.3040501@gmail.com> References: <54F9C3DA.6060703@gmail.com> <54FC6E87.9040400@gmail.com> <54FC7A50.3040501@gmail.com> Message-ID: <54FCBDE7.4090208@gmail.com> I've found this for a long time. But I didn't examined "sudo apt-get libpam-smbpass" till now... Doing that, my problem is "solved"... http://ubuntuforums.org/showthread.php?t=2246951&highlight=dovecot+auth+worker libpam-smbpass has a bug of memory leak. I cannot tell surely....But it returns the error... So the auth-worker would get down by that. Dovecot uses both my system accounts and MySQL virtual mailboxes now... -- Emmanuel Chanel( emmanuelchanel at gmail.com ) From emailbuilder88 at yahoo.com Mon Mar 9 03:17:24 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Sun, 8 Mar 2015 20:17:24 -0700 Subject: Overriding dovecot.conf from Userdb Extras Message-ID: <1425871044.31600.YahooMailBasic@web142403.mail.bf1.yahoo.com> Hi, I thought I read that anything from dovecot.conf can be overridden in a userdb lookup. Or a passdb lookup with "userdb_" prefix. But I tried for fun change log_path but it never worked. Is that because logging is special, already started logging before it comes to the passdb/userdb lookups? So are there some dovecot.conf settings that cannot be overridden? Thanks! From jtam.home at gmail.com Mon Mar 9 05:27:02 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Sun, 8 Mar 2015 22:27:02 -0700 (PDT) Subject: Outlook 2013/2010 nightmare In-Reply-To: References: Message-ID: "David.M.Clark" writes: > I do have some customers using Outlook or Windows Live Mail, and these > are for the most part working fine with IMAP - I don't do POP. > ... > The issue starts when you add an IMAP user to the Outlook client and > upon opening it, initially, it tries to find a "Sent" Items folder under > IMAP to send from. To this end I have traditionally gone into the > dialogue box that opens, click on the IMAP account to receive a list of > possible folders, but it now immediately crashes indicating Outlook has > had an error, and it all goes to hell from there. Outlook then cannot > open but keeps crashing with its "unknown error". > Up to last week at another site with Windows Live Mail, this was not an > issue. Just a shot in the dark, but have you enabled the workarounds? http://wiki2.dovecot.org/Clients Some clients like, like Outlook, have known problems which dovecot can dance around. in 20-imap.conf # Workarounds for various client bugs: # delay-newmail: # Send EXISTS/RECENT new mail notifications only when replying to NOOP # and CHECK commands. Some clients ignore them otherwise, for example OSX # Mail ( From skdovecot at smail.inf.fh-brs.de Mon Mar 9 07:41:21 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 9 Mar 2015 08:41:21 +0100 (CET) Subject: Expunge messages older than In-Reply-To: <9571164.1711991425752400948.JavaMail.httpd@webmail-39.iol.local> References: <9571164.1711991425752400948.JavaMail.httpd@webmail-39.iol.local> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sat, 7 Mar 2015, absolutely_free at libero.it wrote: > Hi, I am using Dovecot 2.2.13 on cPanel server. > Is there a way to delete messages older than some date for a number of users (in every subfolder)? Do you have shell access to the server, in order to run doveadm commands? If so, check out doveadm expunge: "This command can be used to expunge mails matching the given search query. It is typically used to expunge old mails from users' Trash and/or Spam mailboxes. To test which messages a given search query would match, you can use doveadm fetch or doveadm search commands." - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVP1OoXz1H7kL/d9rAQLWsAf/cDliyU1OhTPrcMPtz0Yvp51VU5FHB/Fk fLSghkDr8ZWNA8s0aBeLO9eQVYlZDDXG6bqZSZZtjYMfDaVtqaAQd7Sj30ZRH1kK TuTiiqFnltyLnstBXzkOwVS+qNSs+3hRdFLlkS3CGOTG7hZ7owXRpo1zCDAACe3P Y6RlB+42wb5wgFUqhrVs4KGBSEywTrndadpR8MvlFqZVXHUE9nzKwLmDFQQKEj6H Y6ecRJzuVvlmp5IRRqPsYRYOQHg9tqoKQ6vC/UwEyPz9u+hLtRqP3qijlMaN9JG3 U7IA9uMQxcP9tVK3nP44alvQfuJbpiOkdIH/5obHacvzxIJW7oypZA== =qy24 -----END PGP SIGNATURE----- From bernard at bachfreund.nl Sat Mar 7 16:47:56 2015 From: bernard at bachfreund.nl (Bernard Spil) Date: Sat, 07 Mar 2015 17:47:56 +0100 Subject: [PATCH] Fix dovecot 1.2 build with LibreSSL Message-ID: <7027153bc8bfdd31427fa4ae96a95936@bachfreund.nl> Hi All, mail/dovecot build fails when linked against LibreSSL. This is due to LibreSSL no longer including comp.h from ssl.h/ssl3.h. See https://wiki.freebsd.org/LibreSSL as well. This patch fixes the build failure. Build log attached as well (not any more, too large! Get it via link just above). Please commit this fix to the 1.2 branch (2.1 branch is not affected) Kind regards, Bernard Spil. -------------- next part -------------- A non-text attachment was scrubbed... Name: patch-src_login-common_ssl-proxy-openssl.c Type: text/x-diff Size: 409 bytes Desc: not available URL: From jc at info-systems.de Mon Mar 9 09:43:32 2015 From: jc at info-systems.de (Jakob Curdes) Date: Mon, 09 Mar 2015 10:43:32 +0100 Subject: Outlook 2013/2010 nightmare In-Reply-To: <54FBCE20.9010305@davrom.com> References: <54FBCE20.9010305@davrom.com> Message-ID: <54FD6B44.1080008@info-systems.de> Just a basic comment: Outlook is well known for its bad IMAP support. It is really not a recommendable IMAP client. My experience is similar to yours: it may work, then it may not work, while other clients can use the same account without problems. Essentially I think Outlook always beleives what it has on-disk for the account rather than believing the IMAP server, which leads to very strange situations. The issue you describe could be related to this. Regards, Jakob From HFlor at gmx.de Mon Mar 9 11:07:24 2015 From: HFlor at gmx.de (Hardy Flor) Date: Mon, 09 Mar 2015 12:07:24 +0100 Subject: v2.2.16 release candidate released In-Reply-To: References: Message-ID: <54FD7EEC.5090500@gmx.de> This easy to correct errors is unfortunately still exists. I once hung my solution, but I can only test on a system. # dovecot --version 2.2.16.rc1 # doveadm -f table mailbox status "messages guid" -u flor_hardy "*" mailbox messages guid Trash 0 60a619171f66fd54035800004b126f5b Entw?rfe 0 62a619171f66fd54035800004b126f5b INBOX 0 61a619171f66fd54035800004b126f5b -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-2.2.16rc1-table.diff Type: text/x-patch Size: 1426 bytes Desc: not available URL: -------------- next part -------------- An embedded message was scrubbed... From: Hardy Flor Subject: incorrect column width for multi-byte characters by'doveadm -f table' Date: Wed, 03 Dec 2014 22:57:24 +0100 Size: 5012 URL: From HFlor at gmx.de Mon Mar 9 17:53:29 2015 From: HFlor at gmx.de (Hardy Flor) Date: Mon, 09 Mar 2015 18:53:29 +0100 Subject: rawlog with -i not worked Message-ID: <54FDDE19.2080401@gmx.de> Hello, I have the rawlog enabled according to the manual:http://wiki2.dovecot.org/Debugging/Rawlog Without -i everything is wonderful, a coredump is with the -i parameter created **************************************** service imap-postlogin { executable = script-login -d rawlog -i user = vmail } **************************************** GNU gdb (GDB) 7.4.1-debian Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/lib/dovecot/rawlog...(no debugging symbols found)...done. [New LWP 29312] warning: Can't read pathname for load map: Eingabe-/Ausgabefehler. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/lib/dovecot/rawlog -i /usr/lib/dovecot/script-login'. Program terminated with signal 11, Segmentation fault. #0 0x00007f66bd2fe6c7 in master_instance_list_find_by_name () from /usr/lib/dovecot/libdovecot.so.0 (gdb) bt full #0 0x00007f66bd2fe6c7 in master_instance_list_find_by_name () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #1 0x00007f66bd300ea5 in master_service_parse_option () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #2 0x00007f66bd300fd4 in master_getopt () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #3 0x0000000000401e15 in main () No symbol table info available. (gdb) From jfs.world at gmail.com Mon Mar 9 18:15:29 2015 From: jfs.world at gmail.com (Jeffrey 'jf' Lim) Date: Tue, 10 Mar 2015 02:15:29 +0800 Subject: Dovecot fails to compile --with-cdb (and attempted fix) Message-ID: Dovecot version: 2.2.15 Tinycdb: 0.78 I'm attempting to compile dovecot --with-cdb, but keep getting into a problem with cdb: ======================== libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -Wl,--as-needed -o .libs/anvil main.o anvil-connection.o anvil-settings.o connect-limit.o penalty.o -Wl,--export-dynamic ../../src/lib-dovecot/.libs/libdovecot.so -ldl -lrt -Wl,-rpath -Wl,/usr/local/lib/dovecot ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to `cdb_read' ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to `cdb_find' ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to `cdb_free' ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to `cdb_init' collect2: ld returned 1 exit status make[3]: *** [anvil] Error 1 make[3]: Leaving directory `/home/jf/dovecot-2.2.15/src/anvil' ========================== After some troubleshooting, I've finally managed to get dovecot to compile, albeit with a hack: --- src/lib-dovecot/Makefile.old 2015-03-09 17:36:15.690179802 +0000 +++ src/lib-dovecot/Makefile 2015-03-09 18:01:43.075719700 +0000 @@ -230,7 +230,7 @@ LIBDOVECOT_STORAGE_DEPS = $(top_builddir)/src/lib-storage/ libdovecot-storage.la $(top_builddir)/src/lib-imap-storage/ libimap-storage.la LIBICONV = LIBOBJS = -LIBS = -lrt +LIBS = -lrt -lcdb LIBTOOL = $(SHELL) $(top_builddir)/libtool LIBWRAP_LIBS = LINKED_STORAGE_LDADD = This isn't the best fix, admittedly (I wish I were more familiar with autotools), but I'll say that the configure process isn't right yet for cdb. Can anybody produce a better fix? Would love to learn more. thanks, -jf -- He who settles on the idea of the intelligent man as a static entity only shows himself to be a fool. Mensan / Full-Stack Technical Polymath / System Administrator 12 years over the entire web stack: Performance, Sysadmin, Ruby and Frontend From jfs.world at gmail.com Mon Mar 9 18:22:34 2015 From: jfs.world at gmail.com (Jeffrey 'jf' Lim) Date: Tue, 10 Mar 2015 02:22:34 +0800 Subject: Dovecot fails to compile --with-cdb (and attempted fix) In-Reply-To: References: Message-ID: On Tue, Mar 10, 2015 at 2:15 AM, Jeffrey 'jf' Lim wrote: > Dovecot version: 2.2.15 > Tinycdb: 0.78 > > I'm attempting to compile dovecot --with-cdb, but keep getting into a > problem with cdb: > ======================== > libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes > -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 > -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 > -Wl,--as-needed -o .libs/anvil main.o anvil-connection.o anvil-settings.o > connect-limit.o penalty.o -Wl,--export-dynamic > ../../src/lib-dovecot/.libs/libdovecot.so -ldl -lrt -Wl,-rpath > -Wl,/usr/local/lib/dovecot > ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to > `cdb_read' > ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to > `cdb_find' > ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to > `cdb_free' > ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to > `cdb_init' > collect2: ld returned 1 exit status > make[3]: *** [anvil] Error 1 > make[3]: Leaving directory `/home/jf/dovecot-2.2.15/src/anvil' > ========================== > > After some troubleshooting, I've finally managed to get dovecot to > compile, albeit with a hack: > > --- src/lib-dovecot/Makefile.old 2015-03-09 17:36:15.690179802 +0000 > +++ src/lib-dovecot/Makefile 2015-03-09 18:01:43.075719700 +0000 > @@ -230,7 +230,7 @@ > LIBDOVECOT_STORAGE_DEPS = $(top_builddir)/src/lib-storage/ > libdovecot-storage.la $(top_builddir)/src/lib-imap-storage/ > libimap-storage.la > LIBICONV = > LIBOBJS = > -LIBS = -lrt > +LIBS = -lrt -lcdb > LIBTOOL = $(SHELL) $(top_builddir)/libtool > LIBWRAP_LIBS = > LINKED_STORAGE_LDADD = > > This isn't the best fix, admittedly (I wish I were more familiar with > autotools), but I'll say that the configure process isn't right yet for > cdb. Can anybody produce a better fix? Would love to learn more. > > ok, well I guess that "fixed" the compilation, but as I've just found out, it does *not* link in libcdb (I've got libcdb in /usr/local/lib) properly! Has anybody tried to compile dovecot with cdb support? -jf -- He who settles on the idea of the intelligent man as a static entity only shows himself to be a fool. Mensan / Full-Stack Technical Polymath / System Administrator 12 years over the entire web stack: Performance, Sysadmin, Ruby and Frontend From david at davrom.com Mon Mar 9 23:48:33 2015 From: david at davrom.com (David.M.Clark) Date: Tue, 10 Mar 2015 09:48:33 +1000 Subject: Outlook 2013/2010 nightmare #2 Message-ID: <54FE3151.8000103@davrom.com> Hi All and my sincere thanks to Jakob and Joseph for your responses. I got around the nightmare for this site but it is far from satisfactory and given both Thunderbird, Roundcube and the Android e-mail client work perfectly as expected, this following links comments enforce what I experienced over the weekend: http://comments.gmane.org/gmane.mail.imap.dovecot/79231 I have not struck this for other sites using Outlook 2013 (though I continually banter that Outlook version 'anything' is designed for MS Exchange and nothing else), but then again I have not had Outlook needing to share the same e-mail accounts across PCs. So the bottom line for this particular site is: Set the "Root Folder" for IMAP in outlook to "mail". This is messy from my beloved Linux command line perspective in that you end up with ${HOME}/login_name/mail/mail. But it does work and stops the Outlook crashes. If you set it to the Outlook recommended "INBOX" or "Inbox", you can't see or access subfolders. Now here comes the ugly part of Outlook 2013 using IMAP to Linux (at least for this site): If you want to share a single e-mail account across multiple PCs running Outlook 2013, you _cannot_ use the "Root Folder" of "mail" as I have indicated above. The workaround is to create each subsequent PC with a "mail2", "mail3" etc folder (without the quote marks of course). If you set up two PCs with the same Root Folder, the new PC crashes out of Outlook and eventually so does the original PC. The only way around this is to delete the identity and PST files in Outlook and strictly set them up again to different "mail" something folders. Almost reminds me of the old MS "Share Violation" issue :-) So after the user is set to the mail2/mail3 folder and it appears under the user's original "mail" folder, you then have to blow away the mail2/mail3 folder and then do a symbolic link to the mail folder: ln -s mail mail2 In this way, both users on the PCs will be seeing the same IMAP folders, being mail and mail2 on two different PCs. Naturally to add a third PC to this would be "mail3" etc. Now the other ugly part of this is you also need to update their .subscriptions file in ${HOME}/login_name/mail to show folders like: mail/Sent Items mail/Junk E-mails mail2/Sent Items mail2/Junk E-mails Given you would need to manually do this above, and is just not suitable, I have written two scripts to automate this process and it works cleanly so far. The first script does the automatic removal of the mail2/mail3 etc directory and creates the symbolic link, the second script is used to redo the .subscriptions file if the contents of the 'mail' directory change. You may need to mod these but I have a cron running as root to do this. Script 1 (mail_root_enforce): =----------------------------------------------------------------------------------= COMPLETE_HOME=/u/home cd ${COMPLETE_HOME} for USERHOME in * do ls -d ${USERHOME}/mail/mail? | while read CHECKME do JUST_CHECKME=`basename ${CHECKME}` if [ -d ${CHECKME} -a -L ${CHECKME} ] then #echo "${CHECKME} is ok" : else echo "${CHECKME} is _not_ ok" cd ${USERHOME}/mail rm -rf ${COMPLETE_HOME}/${CHECKME} ln -s mail ${JUST_CHECKME} fi done cd ${COMPLETE_HOME} done =----------------------------------------------------------------------------------= Script 2 (mail_multi_users): =----------------------------------------------------------------------------------= cd /u/home for THIS_USER in * do MORE_THAN_1=`ls -d ${THIS_USER}/mail/mail* | wc -l` if [ "${MORE_THAN_1}" -gt "1" ] then # echo "${THIS_USER} has more than one" ls ${THIS_USER}/mail/mail >${THIS_USER}/subcount if [ ! -f ${THIS_USER}/subcount_last ] then >${THIS_USER}/subcount_last fi ISDIFF=`diff ${THIS_USER}/subcount ${THIS_USER}/subcount_last | wc -l | awk '{print $1}'` if [ "${ISDIFF}" -gt "0" ] then # echo "Is Different" ls ${THIS_USER}/mail/mail >${THIS_USER}/subscription_tmp >${THIS_USER}/subscription_new for ROOT_MAIL_DIR in ${THIS_USER}/mail/mail* do MAIL_BASE_NAME=`basename ${ROOT_MAIL_DIR}` cat ${THIS_USER}/subscription_tmp | sed "s:^:$MAIL_BASE_NAME/:g" >>${THIS_USER}/subscription_new done cp ${THIS_USER}/subscription_new ${THIS_USER}/mail/.subscriptions chown ${THIS_USER}:popusers ${THIS_USER}/mail/.subscriptions cp ${THIS_USER}/subcount ${THIS_USER}/subcount_last fi fi done =----------------------------------------------------------------------------------= You will note the ${HOME} directories are all under /u/home so if you do sadly need to use these scripts, please change the /u/home to suite the ${HOME} location set on your server. I prefer to run this as root to enforce things and not have any funny permission errors, and also to save needing to run multiple instances for multiples users. One annoying caveat that the site will have to get used to: the mail2, mail3 folders do show up when using good e-mail clients like Thunderbird, SOGo, Roundcube or Android e-mail client. This is because they are seeing the contents of the .subscriptions file. I haven't looked into Dovecot doco for possible multiple .subscription files to make the folders bit look 'nicer'. I will look at something like 'namespace' to see if this might help. Please bear in mind this was in an emergency to get things at least 'working' so the users were happy when they hit the office on Monday. A documented story for your perusal and use if you are 'stuck' like I was. I will continue to try and see if there is a more elegant work-around but for now, this is all I can come up with. PS: I notice everyone suppresses their e-mail footer so doing the same here :-) -- As always, I remain at your service. Kindest Regards, David.M.Clark From david at davrom.com Tue Mar 10 00:35:39 2015 From: david at davrom.com (David.M.Clark) Date: Tue, 10 Mar 2015 10:35:39 +1000 Subject: Outlook 2013/2010 nightmare #3 Message-ID: <54FE3C5B.9070009@davrom.com> Hi All, Sorry for the extra e-mail. Given I am helping roll out this identical setup again I am thinking I will test another approach which I am hoping will work but given having two or more PCs trying to access the 'mail' folder crashed at this site on the weekend, I went away from this. But in my thinking perhaps it 'might' work if same login ID is not used to access the exact same area. Example: 1. Setup user "accounts" and open the permissions of its folders. (${HOME} = /u/home/accounts) 2. Create "accounts2" but have its ${HOME} directory set to the same as "accounts". (${HOME} = /u/home/accounts) Make sure accounts2 can fully access everything - perhaps I will need to enforce a central GID (group ID) for each user to belong to (eg., accounts). 3. See if the same crash issue happens if using a different login name. This is using the reverse approach of not having separate Root Folder paths for the same Linux login ID. If this does work I can then reverse engineer the issue at the site over the weekend as well. Will keep you posted. -- As always, I remain at your service. Kindest Regards, David.M.Clark From david at davrom.com Tue Mar 10 00:44:34 2015 From: david at davrom.com (David.M.Clark) Date: Tue, 10 Mar 2015 10:44:34 +1000 Subject: Outlook 2013/2010 nightmare #2 In-Reply-To: <54FE393B.7020701@whitehorsetc.com> References: <54FE3151.8000103@davrom.com> <54FE393B.7020701@whitehorsetc.com> Message-ID: <54FE3E72.2010903@davrom.com> Yes Eric, Outlook also has a declaration that as of either version 2010 or 2013, they no longer download IMAP headers, they download the whole message - thank God for faster Internet connections these days - could you imagine that in the older dialup days? Still it is a waste of bandwidth and disk space to do this. I am so tired of how they claim to use the RFC and indicate they are 'compatible' with a certain protocol/procedure, but really just move the goal posts so you can only talk to M$ stuff at the back end using their enforcements. All part of locking people into the "perpetual upgrade" path I guess. On 03/10/2015 10:22 AM, Eric Broch wrote: > Not only does M$ Outlook not work well with others, it strips headers I > use for training spam filters. > > On 3/9/2015 5:48 PM, David.M.Clark wrote: >> Hi All and my sincere thanks to Jakob and Joseph for your responses. >> >> I got around the nightmare for this site but it is far from >> satisfactory and given both Thunderbird, Roundcube and the Android >> e-mail client work perfectly as expected, this following links >> comments enforce what I experienced over the weekend: >> >> http://comments.gmane.org/gmane.mail.imap.dovecot/79231 >> >> I have not struck this for other sites using Outlook 2013 (though I >> continually banter that Outlook version 'anything' is designed for MS >> Exchange and nothing else), but then again I have not had Outlook >> needing to share the same e-mail accounts across PCs. >> >> So the bottom line for this particular site is: >> >> Set the "Root Folder" for IMAP in outlook to "mail". This is messy >> from my beloved Linux command line perspective in that you end up with >> ${HOME}/login_name/mail/mail. But it does work and stops the Outlook >> crashes. >> >> If you set it to the Outlook recommended "INBOX" or "Inbox", you can't >> see or access subfolders. >> >> Now here comes the ugly part of Outlook 2013 using IMAP to Linux (at >> least for this site): >> >> If you want to share a single e-mail account across multiple PCs >> running Outlook 2013, you _cannot_ use the "Root Folder" of "mail" as >> I have indicated above. The workaround is to create each subsequent PC >> with a "mail2", "mail3" etc folder (without the quote marks of >> course). If you set up two PCs with the same Root Folder, the new PC >> crashes out of Outlook and eventually so does the original PC. The >> only way around this is to delete the identity and PST files in >> Outlook and strictly set them up again to different "mail" something >> folders. Almost reminds me of the old MS "Share Violation" issue :-) >> >> So after the user is set to the mail2/mail3 folder and it appears >> under the user's original "mail" folder, you then have to blow away >> the mail2/mail3 folder and then do a symbolic link to the mail folder: >> >> ln -s mail mail2 >> >> In this way, both users on the PCs will be seeing the same IMAP >> folders, being mail and mail2 on two different PCs. Naturally to add a >> third PC to this would be "mail3" etc. >> >> Now the other ugly part of this is you also need to update their >> .subscriptions file in ${HOME}/login_name/mail to show folders like: >> >> mail/Sent Items >> mail/Junk E-mails >> mail2/Sent Items >> mail2/Junk E-mails >> >> Given you would need to manually do this above, and is just not >> suitable, I have written two scripts to automate this process and it >> works cleanly so far. >> >> The first script does the automatic removal of the mail2/mail3 etc >> directory and creates the symbolic link, the second script is used to >> redo the .subscriptions file if the contents of the 'mail' directory >> change. >> >> You may need to mod these but I have a cron running as root to do this. >> >> Script 1 (mail_root_enforce): >> >> =----------------------------------------------------------------------------------= >> >> COMPLETE_HOME=/u/home >> cd ${COMPLETE_HOME} >> >> for USERHOME in * >> do >> ls -d ${USERHOME}/mail/mail? | while read CHECKME >> do >> JUST_CHECKME=`basename ${CHECKME}` >> if [ -d ${CHECKME} -a -L ${CHECKME} ] >> then >> #echo "${CHECKME} is ok" >> : >> else >> echo "${CHECKME} is _not_ ok" >> cd ${USERHOME}/mail >> rm -rf ${COMPLETE_HOME}/${CHECKME} >> ln -s mail ${JUST_CHECKME} >> fi >> done >> cd ${COMPLETE_HOME} >> done >> =----------------------------------------------------------------------------------= >> >> >> Script 2 (mail_multi_users): >> >> =----------------------------------------------------------------------------------= >> >> cd /u/home >> >> for THIS_USER in * >> do >> MORE_THAN_1=`ls -d ${THIS_USER}/mail/mail* | wc -l` >> if [ "${MORE_THAN_1}" -gt "1" ] >> then >> # echo "${THIS_USER} has more than one" >> ls ${THIS_USER}/mail/mail >${THIS_USER}/subcount >> if [ ! -f ${THIS_USER}/subcount_last ] >> then >> >${THIS_USER}/subcount_last >> fi >> ISDIFF=`diff ${THIS_USER}/subcount ${THIS_USER}/subcount_last | >> wc -l | awk '{print $1}'` >> if [ "${ISDIFF}" -gt "0" ] >> then >> # echo "Is Different" >> ls ${THIS_USER}/mail/mail >${THIS_USER}/subscription_tmp >> >${THIS_USER}/subscription_new >> for ROOT_MAIL_DIR in ${THIS_USER}/mail/mail* >> do >> MAIL_BASE_NAME=`basename ${ROOT_MAIL_DIR}` >> cat ${THIS_USER}/subscription_tmp | sed >> "s:^:$MAIL_BASE_NAME/:g" >>${THIS_USER}/subscription_new >> done >> cp ${THIS_USER}/subscription_new >> ${THIS_USER}/mail/.subscriptions >> chown ${THIS_USER}:popusers ${THIS_USER}/mail/.subscriptions >> cp ${THIS_USER}/subcount ${THIS_USER}/subcount_last >> fi >> fi >> done >> =----------------------------------------------------------------------------------= >> >> >> You will note the ${HOME} directories are all under /u/home so if you >> do sadly need to use these scripts, please change the /u/home to suite >> the ${HOME} location set on your server. I prefer to run this as root >> to enforce things and not have any funny permission errors, and also >> to save needing to run multiple instances for multiples users. >> >> One annoying caveat that the site will have to get used to: the mail2, >> mail3 folders do show up when using good e-mail clients like >> Thunderbird, SOGo, Roundcube or Android e-mail client. This is because >> they are seeing the contents of the .subscriptions file. I haven't >> looked into Dovecot doco for possible multiple .subscription files to >> make the folders bit look 'nicer'. I will look at something like >> 'namespace' to see if this might help. Please bear in mind this was in >> an emergency to get things at least 'working' so the users were happy >> when they hit the office on Monday. >> >> A documented story for your perusal and use if you are 'stuck' like I >> was. I will continue to try and see if there is a more elegant >> work-around but for now, this is all I can come up with. >> >> PS: I notice everyone suppresses their e-mail footer so doing the same >> here :-) >> -- As always, I remain at your service. Kindest Regards, David.M.Clark From slusarz at curecanti.org Tue Mar 10 01:16:23 2015 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 09 Mar 2015 19:16:23 -0600 Subject: Outlook 2013/2010 nightmare #2 In-Reply-To: <54FE3E72.2010903@davrom.com> References: <54FE3151.8000103@davrom.com> <54FE393B.7020701@whitehorsetc.com> <54FE3E72.2010903@davrom.com> Message-ID: <20150309191623.Horde.oxSA2qo95Se1GM5rve6u1v0@bigworm.curecanti.org> Quoting David.M.Clark : > Outlook also has a declaration that as of either version 2010 or > 2013, they no longer download IMAP headers, they download the whole > message - thank God for faster Internet connections these days - > could you imagine that in the older dialup days? Still it is a waste > of bandwidth and disk space to do this. Except that's not anything specific to Outlook... most (all) of the desktop clients do this. Thunderbird most certainly does, for example. michael From ebroch at whitehorsetc.com Tue Mar 10 02:04:21 2015 From: ebroch at whitehorsetc.com (Eric Broch) Date: Mon, 09 Mar 2015 20:04:21 -0600 Subject: Outlook 2013/2010 nightmare #2 In-Reply-To: <20150309191623.Horde.oxSA2qo95Se1GM5rve6u1v0@bigworm.curecanti.org> References: <54FE3151.8000103@davrom.com> <54FE393B.7020701@whitehorsetc.com> <54FE3E72.2010903@davrom.com> <20150309191623.Horde.oxSA2qo95Se1GM5rve6u1v0@bigworm.curecanti.org> Message-ID: <54FE5125.4010702@whitehorsetc.com> On 3/9/2015 7:16 PM, Michael M Slusarz wrote: > Quoting David.M.Clark : > >> Outlook also has a declaration that as of either version 2010 or >> 2013, they no longer download IMAP headers, they download the whole >> message - thank God for faster Internet connections these days - >> could you imagine that in the older dialup days? Still it is a waste >> of bandwidth and disk space to do this. > > Except that's not anything specific to Outlook... most (all) of the > desktop clients do this. Thunderbird most certainly does, for example. > > michael Stripping headers is particular to M$ Outlook. M$ Outlook 2013 strips headers when moving email between IMAP folders. This is a known issue: https://social.msdn.microsoft.com/Forums/en-US/8eafe714-8a8c-44bc-9228-d6a68731494f/outlook-2013-behavior-as-an-imap-client?forum=os_exchangeprotocols From stephan at rename-it.nl Tue Mar 10 02:27:57 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 10 Mar 2015 03:27:57 +0100 Subject: Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1 Message-ID: <54FE56AD.7060402@rename-it.nl> Hello Dovecot users, Last time I had a few stupid problems in the releases, so I'll follow Timo's example and I release an RC first. The highlights include the implementation of the index and metadata extensions. Quite a few bugs are fixed as well. Changelog v0.4.7: * editheader extension: Made protection against addition and deletion of headers configurable separately. Also, the `Received' and `Auto-Submitted' headers are no longer protected against addition by default. * Turned message envelope address parse errors into warnings. * The interpreter now accepts non-standard domain names, e.g. containing '_'. + Implemented the Sieve index extension (RFC 5260). + Implemented support for the mboxmetadata and servermetadata extensions (RFC 5490). + Implemented new sieve commands for the doveadm command line utility. These commands are currently limited to ManageSieve operations, but the other current sieve tools will be migrated to doveadm in the near future as well. + Added more debug output to binary up-to-date checking. - The Sieve interpreter now flushes duplicate database during start phase of result execution rather than commit phase. This makes sure locks on the duplicate database are released as soon as possible, preventing contention. - Performed a few optimizations in the lexical scanner of the language. - Fixed bug in `:matches' match-type that made a pattern without wildcards match as if there were a '*' at the beginning. - file storage: Restructured storage initialization to address backwards compatibility issues. - Fixed crash in validation of the string parameter of the comparator tag. - extprograms extension: Made sure supplemental group privileges are also dropped. This was a problem reported by Debian lintian. - Fixed bug in handling of binary errors for action side-effects and message overrides. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.7.rc1.tar.gz http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.7.rc1.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From skdovecot at smail.inf.fh-brs.de Tue Mar 10 07:38:56 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Tue, 10 Mar 2015 08:38:56 +0100 (CET) Subject: rawlog with -i not worked In-Reply-To: <54FDDE19.2080401@gmx.de> References: <54FDDE19.2080401@gmx.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 9 Mar 2015, Hardy Flor wrote: > I have the rawlog enabled according to the > manual:http://wiki2.dovecot.org/Debugging/Rawlog > > Without -i everything is wonderful, a coredump is with the -i parameter > created - -i is: v2.0 and older don't expect a bugfix ;-) - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVP6fkHz1H7kL/d9rAQLDNAgAnI7XBMpegoYcxCRdOFRMk5nbVMbZjLUD sLQInTa6ggBvxVy+1iSS9U1R8//BqgvIum7Sgy6kAjzk59Vjtr5i5a+cCBv/+3aj KBAylmaf008/zD6lPJQ5B0fjVYdJB+vx/px6e14FgAuDZb52yqC0hh3Ychs1xDmN xdcZePg5MSAooaXLLdQ6PzKhwmeSclYFHaIzkbnqZvPCXHYjbs37bVCmSP+HHo8p 16MmwuyKZQ3YpAh+TK/k0k16Vb5Lle/5tTbPZE2U0QT/gvPaYCp+FS4eZOiRjjai GUgnABcxSNUeXnpFTWArXHatKXuxI/tp8BQxVYJaYFIGv0nDwCbUlw== =9iHQ -----END PGP SIGNATURE----- From alessio at skye.it Tue Mar 10 08:26:19 2015 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 10 Mar 2015 09:26:19 +0100 Subject: Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1 In-Reply-To: <54FE56AD.7060402@rename-it.nl> References: <54FE56AD.7060402@rename-it.nl> Message-ID: <54FEAAAB.1000508@skye.it> Hi Stephan, I'm running the current stable version (Dovecot 2.2.15 and Pigeonhole 0.4.6), all works fine except an issue. In my configuration I have this default sieve script (I'm using Maildir): === require ["vnd.dovecot.duplicate", "fileinto", "mailbox"]; if header :matches "X-Spam-Status" "Yes, *" { fileinto "Spam"; } if duplicate :seconds 60 { fileinto "Trash"; } === when in the Maildir/cur/ there are more than (about) 100.000 messages dovecot-lda crash and the email is not delivered (remains in the queue) and in the log I found this error: =========== Feb 16 03:23:41 mx03 dovecot: lda(user at domain.com): Panic: file mail-transaction-log.c: line 271 (mail_transaction_log_rotate): assertion failed: (file->locked) Feb 16 03:23:41 mx03 dovecot: lda(user at domain.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x7319a) [0x7f27cef8b19a] -> /usr/lib64/dovecot/libdovecot.so.0(i_syslog_fatal_handler+0x33) [0x7f27cef8b253] -> /usr/lib64/dovecot/libdovecot.so.0(+0x24b1a) [0x7f27cef3cb1a] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0xd3965) [0x7f27cf2ce965] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_write+0x1f5) [0x7f27cf2cd7a5] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x2c4) [0x7f27cf2c5214] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x510e8) [0x7f27cf24c0e8] -> /usr/lib64/dovecot/libdovecot-storage.so.0(maildir_transaction_save_commit_pre+0x40f) [0x7f27cf2477ef] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0xb39c8) [0x7f27cf2ae9c8] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_transaction_commit_full+0x9f) [0x7f27cf2bc62f] -> /usr/lib64/dovecot/libdovecot-storage.so.0(index_transaction_commit+0xa5) [0x7f27cf2ae585] -> /usr/lib64/dovecot/lib20_zlib_plugin.so(+0x302f) [0x7f27cd90d02f] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0xac52) [0x7f27cdf34c52] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x96b46) [0x7f27cf291b46] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x4f) [0x7f27cf279def] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit+0x16) [0x7f27cf279ea6] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(+0x418c7) [0x7f27cca328c7] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(+0x382a1) [0x7f27cca292a1] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(sieve_result_execute+0x1e2) [0x7f27cca295f2] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(sieve_execute+0x56) [0x7f27cca3a346] -> /usr/lib64/dovecot/lib90_sieve_plugin.so(+0x2e1e) [0x7f27ccc8de1e] -> /usr/lib64/dovecot/libdovecot-lda.so.0(mail_deliver+0x45) [0x7f27cf51fda5] -> /usr/libexec/dovecot/dovecot-lda(main+0x6ff) [0x402caf] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f27ceba2d5d] -> /usr/libexec/dovecot/dovecot-lda() [0x402339] =========== If I remove from default.sieve "if duplicate" rule all works fine. Do you think this problem can be solved in the new version? Thanks Il 10/03/2015 03:27, Stephan Bosch ha scritto: > Hello Dovecot users, > > Last time I had a few stupid problems in the releases, so I'll follow > Timo's example and I release an RC first. > > The highlights include the implementation of the index and metadata > extensions. Quite a few bugs are fixed as well. > > Changelog v0.4.7: From HFlor at gmx.de Tue Mar 10 08:58:40 2015 From: HFlor at gmx.de (Hardy Flor) Date: Tue, 10 Mar 2015 09:58:40 +0100 Subject: rawlog with -i not worked In-Reply-To: References: <54FDDE19.2080401@gmx.de> Message-ID: <54FEB240.6000004@gmx.de> This option was added at Mon Jul 21 10:53:19 2014 +0300 http://hg.dovecot.org/dovecot-2.2/diff/fbf434ad2485/src/util/rawlog.c Hardy Am 10.03.2015 um 08:38 schrieb Steffen Kaiser: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Mon, 9 Mar 2015, Hardy Flor wrote: > >> I have the rawlog enabled according to the >> manual:http://wiki2.dovecot.org/Debugging/Rawlog >> >> Without -i everything is wonderful, a coredump is with the -i >> parameter created > > - -i is: v2.0 and older > don't expect a bugfix ;-) > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVP6fkHz1H7kL/d9rAQLDNAgAnI7XBMpegoYcxCRdOFRMk5nbVMbZjLUD > sLQInTa6ggBvxVy+1iSS9U1R8//BqgvIum7Sgy6kAjzk59Vjtr5i5a+cCBv/+3aj > KBAylmaf008/zD6lPJQ5B0fjVYdJB+vx/px6e14FgAuDZb52yqC0hh3Ychs1xDmN > xdcZePg5MSAooaXLLdQ6PzKhwmeSclYFHaIzkbnqZvPCXHYjbs37bVCmSP+HHo8p > 16MmwuyKZQ3YpAh+TK/k0k16Vb5Lle/5tTbPZE2U0QT/gvPaYCp+FS4eZOiRjjai > GUgnABcxSNUeXnpFTWArXHatKXuxI/tp8BQxVYJaYFIGv0nDwCbUlw== > =9iHQ > -----END PGP SIGNATURE----- From stephan at rename-it.nl Tue Mar 10 09:14:09 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 10 Mar 2015 10:14:09 +0100 Subject: Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1 In-Reply-To: <54FEAAAB.1000508@skye.it> References: <54FE56AD.7060402@rename-it.nl> <54FEAAAB.1000508@skye.it> Message-ID: <54FEB5E1.2030401@rename-it.nl> Alessio Cecchi schreef op 10-3-2015 om 9:26: > > =========== > Feb 16 03:23:41 mx03 dovecot: lda(user at domain.com): Panic: file > mail-transaction-log.c: line 271 (mail_transaction_log_rotate): > assertion failed: (file->locked) > Feb 16 03:23:41 mx03 dovecot: lda(user at domain.com): Error: Raw > backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x7319a) > [0x7f27cef8b19a] -> > /usr/lib64/dovecot/libdovecot.so.0(i_syslog_fatal_handler+0x33) > [0x7f27cef8b253] -> /usr/lib64/dovecot/libdovecot.so.0(+0x24b1a) > [0x7f27cef3cb1a] -> > /usr/lib64/dovecot/libdovecot-storage.so.0(+0xd3965) [0x7f27cf2ce965] > -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_write+0x1f5) > [0x7f27cf2cd7a5] -> > /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x2c4) > [0x7f27cf2c5214] -> > /usr/lib64/dovecot/libdovecot-storage.so.0(+0x510e8) [0x7f27cf24c0e8] > -> > /usr/lib64/dovecot/libdovecot-storage.so.0(maildir_transaction_save_commit_pre+0x40f) > [0x7f27cf2477ef] -> > /usr/lib64/dovecot/libdovecot-storage.so.0(+0xb39c8) [0x7f27cf2ae9c8] > -> > /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_transaction_commit_full+0x9f) > [0x7f27cf2bc62f] -> > /usr/lib64/dovecot/libdovecot-storage.so.0(index_transaction_commit+0xa5) > [0x7f27cf2ae585] -> /usr/lib64/dovecot/lib20_zlib_plugin.so(+0x302f) > [0x7f27cd90d02f] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0xac52) > [0x7f27cdf34c52] -> > /usr/lib64/dovecot/libdovecot-storage.so.0(+0x96b46) [0x7f27cf291b46] > -> > /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x4f) > [0x7f27cf279def] -> > /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit+0x16) > [0x7f27cf279ea6] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(+0x418c7) > [0x7f27cca328c7] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(+0x382a1) > [0x7f27cca292a1] -> > /usr/lib64/dovecot/libdovecot-sieve.so.0(sieve_result_execute+0x1e2) > [0x7f27cca295f2] -> > /usr/lib64/dovecot/libdovecot-sieve.so.0(sieve_execute+0x56) > [0x7f27cca3a346] -> /usr/lib64/dovecot/lib90_sieve_plugin.so(+0x2e1e) > [0x7f27ccc8de1e] -> > /usr/lib64/dovecot/libdovecot-lda.so.0(mail_deliver+0x45) > [0x7f27cf51fda5] -> /usr/libexec/dovecot/dovecot-lda(main+0x6ff) > [0x402caf] -> /lib64/libc.so.6(__libc_start_main+0xfd) > [0x7f27ceba2d5d] -> /usr/libexec/dovecot/dovecot-lda() [0x402339] > =========== > > If I remove from default.sieve "if duplicate" rule all works fine. > > Do you think this problem can be solved in the new version? Oh, looks like you reported this before. Will discuss this with Timo. Regards, Stephan. From tss at iki.fi Tue Mar 10 09:40:49 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Mar 2015 11:40:49 +0200 Subject: Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1 In-Reply-To: <54FEAAAB.1000508@skye.it> References: <54FE56AD.7060402@rename-it.nl> <54FEAAAB.1000508@skye.it> Message-ID: <1998155B-252C-4F6E-8F05-1AB27642BCF2@iki.fi> On 10 Mar 2015, at 10:26, Alessio Cecchi wrote: > > Hi Stephan, > > I'm running the current stable version (Dovecot 2.2.15 and Pigeonhole 0.4.6), all works fine except an issue. > > Feb 16 03:23:41 mx03 dovecot: lda(user at domain.com): Panic: file mail-transaction-log.c: line 271 (mail_transaction_log_rotate): assertion failed: (file->locked) This is fixed by v2.2.16.rc1. I'll make v2.2.16 release this week. > If I remove from default.sieve "if duplicate" rule all works fine. I think it's pure chance that it happens to crash that way. I don't see how the duplicate checking could have affected this crash. From frido at 0tten.nl Tue Mar 10 11:15:20 2015 From: frido at 0tten.nl (Frido Otten) Date: Tue, 10 Mar 2015 12:15:20 +0100 Subject: Migrating from NetApp to ZFS Message-ID: <54FED248.4040705@0tten.nl> Hi All, We're currently in the process of migrating from a NetApp storage to a ZFS storage, both shared over NFS. We've ran into some problems which were solved by switching to dotlocks instead of flock. But now there's one problem left with the dotlock method. When connecting to Dovecot/IMAP, and moving messages from one folder to another, it is very slow and I'm suspecting this has all to do with the dotlock method. During this process of moving mail, the dotlock file is created and stays there for about 90 seconds. The mail client is unresponsive in this time period. This happens also when a new message is delivered to the mailbox. The dovecot example config talks about that dotlocking is using some tricks which may create more disk I/O than other locking methods. What are these tricks and can it be the cause of the slowness? We've tested it on different ZFS storages and the problem seems to only exist on ZFS storages which have their ZIL on SSD's Just creating and removing a file from the CLI is fast though. Regards, Frido -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: OpenPGP digital signature URL: From jfs.world at gmail.com Tue Mar 10 14:24:57 2015 From: jfs.world at gmail.com (Jeffrey 'jf' Lim) Date: Tue, 10 Mar 2015 22:24:57 +0800 Subject: Dovecot fails to compile --with-cdb (and attempted fix) In-Reply-To: References: Message-ID: On Tue, Mar 10, 2015 at 2:22 AM, Jeffrey 'jf' Lim wrote: > On Tue, Mar 10, 2015 at 2:15 AM, Jeffrey 'jf' Lim > wrote: > >> Dovecot version: 2.2.15 >> Tinycdb: 0.78 >> >> I'm attempting to compile dovecot --with-cdb, but keep getting into a >> problem with cdb: >> ======================== >> libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes >> -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 >> -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 >> -Wl,--as-needed -o .libs/anvil main.o anvil-connection.o anvil-settings.o >> connect-limit.o penalty.o -Wl,--export-dynamic >> ../../src/lib-dovecot/.libs/libdovecot.so -ldl -lrt -Wl,-rpath >> -Wl,/usr/local/lib/dovecot >> ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to >> `cdb_read' >> ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to >> `cdb_find' >> ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to >> `cdb_free' >> ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to >> `cdb_init' >> collect2: ld returned 1 exit status >> make[3]: *** [anvil] Error 1 >> make[3]: Leaving directory `/home/jf/dovecot-2.2.15/src/anvil' >> ========================== >> >> After some troubleshooting, I've finally managed to get dovecot to >> compile, albeit with a hack: >> >> --- src/lib-dovecot/Makefile.old 2015-03-09 17:36:15.690179802 +0000 >> +++ src/lib-dovecot/Makefile 2015-03-09 18:01:43.075719700 +0000 >> @@ -230,7 +230,7 @@ >> LIBDOVECOT_STORAGE_DEPS = $(top_builddir)/src/lib-storage/ >> libdovecot-storage.la $(top_builddir)/src/lib-imap-storage/ >> libimap-storage.la >> LIBICONV = >> LIBOBJS = >> -LIBS = -lrt >> +LIBS = -lrt -lcdb >> LIBTOOL = $(SHELL) $(top_builddir)/libtool >> LIBWRAP_LIBS = >> LINKED_STORAGE_LDADD = >> >> This isn't the best fix, admittedly (I wish I were more familiar with >> autotools), but I'll say that the configure process isn't right yet for >> cdb. Can anybody produce a better fix? Would love to learn more. >> >> > ok, well I guess that "fixed" the compilation, but as I've just found out, > it does *not* link in libcdb (I've got libcdb in /usr/local/lib) properly! > Has anybody tried to compile dovecot with cdb support? > For anybody following: sorry, my bad. It turns out I hadn't set CPPFLAGS and LDFLAGS during the configure to use /usr/local/{include,lib}. When that's done, this works (in the sense that libdovecot.so is now linked with libcdb!). I'm not so sure that this is the best place to link libcdb in, though, but for an expedient fix, it works.... Would appreciate somebody more familiar with the code weighing in on this one. thanks, -jf -- He who settles on the idea of the intelligent man as a static entity only shows himself to be a fool. Mensan / Full-Stack Technical Polymath / System Administrator 12 years over the entire web stack: Performance, Sysadmin, Ruby and Frontend > From cma at cmadams.net Tue Mar 10 17:20:37 2015 From: cma at cmadams.net (Chris Adams) Date: Tue, 10 Mar 2015 12:20:37 -0500 Subject: Different realm for different listeners? Message-ID: <20150310172037.GA22564@cmadams.net> I see this has been asked a few times over the years (but not in several years), and the response was along the lines of "maybe someday", so I figured I'd see if someday was here yet... I have a need to have a different default realm for different listeners. Basically, I've a bunch of different domains, and a few of them insist their users not be required to authenticate with "user at domain" and just use "user", and I'm trying to make this work without separate daemons and/or VMs. I'll be using Dovecot 2.2 (most likely 2.2.10 as provided by RHEL/CentOS 7). Thanks. If there's not a way to do this directly in Dovecot, does anybody have any suggestion that doesn't require several different VMs? -- Chris Adams From tom at whyscream.net Tue Mar 10 17:38:40 2015 From: tom at whyscream.net (Tom Hendrikx) Date: Tue, 10 Mar 2015 18:38:40 +0100 Subject: Different realm for different listeners? In-Reply-To: <20150310172037.GA22564@cmadams.net> References: <20150310172037.GA22564@cmadams.net> Message-ID: <54FF2C20.8080207@whyscream.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 10-03-15 18:20, Chris Adams wrote: > I see this has been asked a few times over the years (but not in > several years), and the response was along the lines of "maybe > someday", so I figured I'd see if someday was here yet... > > I have a need to have a different default realm for different > listeners. Basically, I've a bunch of different domains, and a few > of them insist their users not be required to authenticate with > "user at domain" and just use "user", and I'm trying to make this work > without separate daemons and/or VMs. > > I'll be using Dovecot 2.2 (most likely 2.2.10 as provided by > RHEL/CentOS 7). > > Thanks. If there's not a way to do this directly in Dovecot, does > anybody have any suggestion that doesn't require several different > VMs? > You could drop the default realm completely, and create a second passdb lookup which uses only the username part to lookup credentials. This means that, as long as you have no conflicts as usernames 'john at foo.com' vs 'john at bar.com', you'd be able to support username 'john'. When you're able to use the password in your sql WHERE statement [1] too, you could even work around that, because you'd only have mismatches when 'john at foo.com' and 'john at bar.com' have the same passphrase. [1] http://wiki2.dovecot.org/AuthDatabase/SQL#Password_verification_by_SQL_server Tom -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCAAGBQJU/ywgAAoJEJPfMZ19VO/1mzQP/Anz44a5c7J6ir8mUgTVZEOn bFOrkXHkntLSruOZ9FHrMJX4RYXTSbZTe9fUUx1zDXiU7Aj2VQ6LqueXt7LFUW2Z jmPsxaHXBL8fSR8suwpzZ06/blz563Qi06HEZb2ixjITyvd3iVJCSPzVu3NLdsSH oxAE50I2vzuFHqfN9JbXD98HuEdGedhOldjlw9MsXmUFthfAYN8N4AhCgzm7G1C8 wk0909pvUAjfDuY7mJU6Q9smDBRdvP4i/aZtKrs/G1ZGq8SPbClbjwLedQy88q76 F/5DcY0M+Z2hYHEZrBQimfUByANY0YvMlQubqQ/s/lsUSD2IdDX6K3lvYKIuKNSe SHgnt9p6/yTAmX3gbsnB7fbbYQQNRrTHM8Uk8KWJhkxhWZpKt3BOoPMphMa5TCKR 9sBRZqIdN1vSd31Gu/QylKv0+zDeIXay7T+HVCf82KPHq/+zFzSZ1iR14V4TdOyz SXNprdxRik49tiAHRTSMQwRmOHDel9+guhozSJVj0ISnTCeRAC5vSvGActA3Gl8l 8aMBLDf/VyS7B+Ss0aWOs8MwIrre9oAxMa937lLc3sNh48A7uw38wJmIB6L20q2n QPlsYCm39wMgWvWi84rEhi663jBdRsZKBq+/Ou7oFHXkQmVmt1ToXw0mC5Bst9Ew y832vTrhtp1Sl2Siv1RV =KWRg -----END PGP SIGNATURE----- From mihai at badici.ro Tue Mar 10 17:44:35 2015 From: mihai at badici.ro (Mihai Badici) Date: Tue, 10 Mar 2015 19:44:35 +0200 Subject: Different realm for different listeners? In-Reply-To: <54FF2C20.8080207@whyscream.net> References: <20150310172037.GA22564@cmadams.net> <54FF2C20.8080207@whyscream.net> Message-ID: <3087875.1KSElmt2VW@arhivio> On Tuesday 10 March 2015 18:38:40 Tom Hendrikx wrote: > On 10-03-15 18:20, Chris Adams wrote: > > I see this has been asked a few times over the years (but not in > > several years), and the response was along the lines of "maybe > > someday", so I figured I'd see if someday was here yet... > > > > I have a need to have a different default realm for different > > listeners. Basically, I've a bunch of different domains, and a few > > of them insist their users not be required to authenticate with > > "user at domain" and just use "user", and I'm trying to make this work > > without separate daemons and/or VMs. > > > > I'll be using Dovecot 2.2 (most likely 2.2.10 as provided by > > RHEL/CentOS 7). > > > > Thanks. If there's not a way to do this directly in Dovecot, does > > anybody have any suggestion that doesn't require several different > > VMs? > > You could drop the default realm completely, and create a second > passdb lookup which uses only the username part to lookup credentials. > This means that, as long as you have no conflicts as usernames > 'john at foo.com' vs 'john at bar.com', you'd be able to support username > 'john'. Well, I think "conflict" is the keyword here. Everybody want an e-mail address called "contact" or "office" :) > > When you're able to use the password in your sql WHERE statement [1] > too, you could even work around that, because you'd only have > mismatches when 'john at foo.com' and 'john at bar.com' have the same > passphrase. > > [1] > http://wiki2.dovecot.org/AuthDatabase/SQL#Password_verification_by_SQL_serve > r > > Tom -- Mihai B?dici http://mihai.badici.ro From cma at cmadams.net Tue Mar 10 17:49:45 2015 From: cma at cmadams.net (Chris Adams) Date: Tue, 10 Mar 2015 12:49:45 -0500 Subject: Different realm for different listeners? In-Reply-To: <54FF2C20.8080207@whyscream.net> References: <20150310172037.GA22564@cmadams.net> <54FF2C20.8080207@whyscream.net> Message-ID: <20150310174945.GB22564@cmadams.net> Once upon a time, Tom Hendrikx said: > You could drop the default realm completely, and create a second > passdb lookup which uses only the username part to lookup credentials. > This means that, as long as you have no conflicts as usernames > 'john at foo.com' vs 'john at bar.com', you'd be able to support username > 'john'. The domains I need to handle are separate telephone company ISPs, so they all want common things like "sales", so that won't work. All my users are in MySQL. Reading the docs, I see that "%l" expands to the local IP address "almost everywhere". If that's available in the userdb SQL expansion, I could write a (somewhat convoluted) SQL statement that adds a domain based on the local IP. Does that sound like it'll work? Not quite as "elegant" as just setting auth_default_realm in separate listeners, but should work I guess. -- Chris Adams From emailbuilder88 at yahoo.com Tue Mar 10 15:54:35 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Tue, 10 Mar 2015 08:54:35 -0700 Subject: Overriding dovecot.conf from Userdb Extras In-Reply-To: <1425871044.31600.YahooMailBasic@web142403.mail.bf1.yahoo.com> Message-ID: <1426002875.57441.YahooMailBasic@web142403.mail.bf1.yahoo.com> > I thought I read that anything from dovecot.conf can be > overridden in a > userdb lookup. Or a passdb lookup with "userdb_" prefix. > > But I tried for fun change log_path but it never worked. Is > that because > logging is special, already started logging before it comes > to the > passdb/userdb lookups?? So are there some dovecot.conf > settings > that cannot be overridden? Any takers? From emailbuilder88 at yahoo.com Tue Mar 10 19:11:14 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Tue, 10 Mar 2015 12:11:14 -0700 Subject: Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1 In-Reply-To: <54FE56AD.7060402@rename-it.nl> Message-ID: <1426014674.94690.YahooMailBasic@web142404.mail.bf1.yahoo.com> > Last time I had a few stupid problems in the releases, so > I'll follow > Timo's example and I release an RC first. > > The highlights include the implementation of the index and > metadata > extensions. Quite a few bugs are fixed as well. When I compiled and installed this, Sieve scripts were being ignored. Not sure if it's my own stupid mistake, but when I put v0.4.6 back in place, it worked fine. No configuration changes, only make install on the different sources and restart dovecot. From stephan at rename-it.nl Tue Mar 10 19:52:34 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 10 Mar 2015 20:52:34 +0100 Subject: Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1 In-Reply-To: <1426014674.94690.YahooMailBasic@web142404.mail.bf1.yahoo.com> References: <1426014674.94690.YahooMailBasic@web142404.mail.bf1.yahoo.com> Message-ID: <54FF4B82.80403@rename-it.nl> On 3/10/2015 8:11 PM, E.B. wrote: >> Last time I had a few stupid problems in the releases, so >> I'll follow >> Timo's example and I release an RC first. >> >> The highlights include the implementation of the index and >> metadata >> extensions. Quite a few bugs are fixed as well. > When I compiled and installed this, Sieve scripts were being ignored. Not > sure if it's my own stupid mistake, but when I put v0.4.6 back in place, it > worked fine. No configuration changes, only make install on the different > sources and restart dovecot. Could you show your dovecot -n output? Also, if you enable mail_debug, what sieve-related debug lines are shown? Regards, Stephan. From bobber at kc0dxf.net Tue Mar 10 20:06:33 2015 From: bobber at kc0dxf.net (Bobber) Date: Tue, 10 Mar 2015 15:06:33 -0500 Subject: Deduplicate Public Folder Message-ID: <54FF4EC9.3040802@kc0dxf.net> How do you perform the doveadm deduplicate command on public folders? If I specify a user, nothing is found. Using -A also doesn't seem to work. -- *Bob Wooldridge* Blog: http://kc0dxf.net/blog/ From jtam.home at gmail.com Wed Mar 11 00:17:31 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 10 Mar 2015 17:17:31 -0700 (PDT) Subject: Outlook 2013/2010 nightmare #2 In-Reply-To: References: Message-ID: "David.M.Clark" writes: > So the bottom line for this particular site is: > > Set the "Root Folder" for IMAP in outlook to "mail". This is messy from > my beloved Linux command line perspective in that you end up with > ${HOME}/login_name/mail/mail. But it does work and stops the Outlook > crashes. If "root folder" is Outlook's parlance for IMAP prefix, you might find it helpful to configure a namespace alias. For examples, see http://wiki2.dovecot.org/Namespaces (Section Backwards Compatibility: UW-IMAP) It will map different npamespaces to the same folder so you don't have this mail/mail/ goofiness. > If you want to share a single e-mail account across multiple PCs running > Outlook 2013, you _cannot_ use the "Root Folder" of "mail" as I have > indicated above. The workaround is to create each subsequent PC with a > "mail2", "mail3" etc folder (without the quote marks of course). If you > set up two PCs with the same Root Folder, the new PC crashes out of > Outlook and eventually so does the original PC. The only way around this > is to delete the identity and PST files in Outlook and strictly set them > up again to different "mail" something folders. Almost reminds me of the > old MS "Share Violation" issue :-) > > So after the user is set to the mail2/mail3 folder and it appears under > the user's original "mail" folder, you then have to blow away the > mail2/mail3 folder and then do a symbolic link to the mail folder: > > ln -s mail mail2 Again, namespace aliases might help: you can configure as many as you like. It's a kludge though -- the behaviour you report is really bizarre. Joseph Tam From emailbuilder88 at yahoo.com Wed Mar 11 01:10:14 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Tue, 10 Mar 2015 18:10:14 -0700 Subject: Why is Sieve trying to re-compile global scripts? Message-ID: <1426036214.25972.YahooMailBasic@web142404.mail.bf1.yahoo.com> I have some global scripts that were running nicely. Then I opened one in an editor and (probably, but not 100% sure) mindlessly saved the file, even though I hadn't made any changes. Shortly after, Sieve errors started showing in the log: Error: 4k5JA74R/1TlIwABG/SpMA: sieve: binary save: failed to create temporary file: open(/usr/local/var/dovecot/sieve/script2.svbin.example.com.4139.) failed: Permission denied... Error: 4k5JA74R/1TlIwABG/SpMA: sieve: The LDA Sieve plugin does not have permission to save global Sieve script binaries; global Sieve scripts like `/usr/local/var/dovecot/sieve/script2.sieve' need to be pre-compiled using the sievec tool Well, OK, is it going by the timestamp on the files? Fine. I recompiled it by hand. Yet, I STILL got these errors! I triple and quadruple checked that the timestamp on the svbin files was more recent. And Sieve was only complaining about one of the two scripts in the directory. I restarted dovecot. No change. So I removed read permission on the .sieve files and only left read permission on the .svbin files. THIS WORKED. No more error. I can live with that, but why was it not complaining before, why was it only complaining about one of my scripts and why would it complain at all when the timestamps on the svbin should have indicated on compilation is needed? From stephan at rename-it.nl Wed Mar 11 01:16:36 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 11 Mar 2015 02:16:36 +0100 Subject: Why is Sieve trying to re-compile global scripts? In-Reply-To: <1426036214.25972.YahooMailBasic@web142404.mail.bf1.yahoo.com> References: <1426036214.25972.YahooMailBasic@web142404.mail.bf1.yahoo.com> Message-ID: <54FF9774.8070006@rename-it.nl> On 3/11/2015 2:10 AM, E.B. wrote: > I have some global scripts that were running nicely. > > Then I opened one in an editor and (probably, but not 100% sure) > mindlessly saved the file, even though I hadn't made any changes. > > Shortly after, Sieve errors started showing in the log: > > Error: 4k5JA74R/1TlIwABG/SpMA: sieve: binary save: failed to create temporary file: open(/usr/local/var/dovecot/sieve/script2.svbin.example.com.4139.) failed: Permission denied... > Error: 4k5JA74R/1TlIwABG/SpMA: sieve: The LDA Sieve plugin does not have permission to save global Sieve script binaries; global Sieve scripts like `/usr/local/var/dovecot/sieve/script2.sieve' need to be pre-compiled using the sievec tool > > Well, OK, is it going by the timestamp on the files? Fine. I recompiled > it by hand. Yet, I STILL got these errors! > > I triple and quadruple checked that the timestamp on the svbin files was > more recent. And Sieve was only complaining about one of the two > scripts in the directory. > > I restarted dovecot. No change. > > So I removed read permission on the .sieve files and only left read > permission on the .svbin files. THIS WORKED. No more error. > I can live with that, but why was it not complaining before, why was it > only complaining about one of my scripts and why would it complain > at all when the timestamps on the svbin should have indicated on > compilation is needed? I've heard about this problem before. Do you have the opportunity to test this with the 0.4.7.rc1 release? That adds a few extra debug lines (shown when mail_debug=yes) that would indicate why Sieve is thinking the global script is not up-to-date. Regards, Stephan. From emailbuilder88 at yahoo.com Wed Mar 11 01:23:50 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Tue, 10 Mar 2015 18:23:50 -0700 Subject: Why is Sieve trying to re-compile global scripts? In-Reply-To: <54FF9774.8070006@rename-it.nl> Message-ID: <1426037030.2770.YahooMailBasic@web142403.mail.bf1.yahoo.com> > > I have some global scripts that were running nicely. > > > > Then I opened one in an editor and (probably, but not 100% sure) > > mindlessly saved the file, even though I hadn't made any changes. > > > > Shortly after, Sieve errors started showing in the log: > > > > Error: 4k5JA74R/1TlIwABG/SpMA: sieve: binary save: failed to create temporary file: > open(/usr/local/var/dovecot/sieve/script2.svbin.example.com.4139.) failed: Permission denied... > > Error: 4k5JA74R/1TlIwABG/SpMA: sieve: The LDA Sieve plugin does not have permission to save global > Sieve script binaries; global Sieve scripts like `/usr/local/var/dovecot/sieve/script2.sieve' > need to be pre-compiled using the sievec tool > > > > Well, OK, is it going by the timestamp on the files? Fine. I recompiled > > it by hand. Yet, I STILL got these errors! > > > > I triple and quadruple checked that the timestamp on the svbin files was > > more recent. And Sieve was only complaining about one of the two > > scripts in the directory. > > > > I restarted dovecot. No change. > > > > So I removed read permission on the .sieve files and only left read > > permission on the .svbin files. THIS WORKED. No more error. > > I can live with that, but why was it not complaining before, why was it > > only complaining about one of my scripts and why would it complain > > at all when the timestamps on the svbin should have indicated on > > compilation is needed? > > I've heard about this problem before. Do you have the opportunity to > test this with the 0.4.7.rc1 release? That adds a few extra debug lines > (shown when mail_debug=yes) that would indicate why Sieve is thinking > the global script is not up-to-date. Yes, I do as a matter of fact. I was just going to put in the RC in order to answer your email on the thread about the RC. Don't have the full answers yet, but when I installed the RC and restarted, I now get an error where Sieve doesn't like that I won't give it read permission on the .sieve file, so now I'm back to square one with this particular issue. OTOH, regarding my earlier post about the RC ignoring seive files, at least it is seeing global scripts (or trying to). Not sure about personal scripts yet. Error: TiQJHH2X/1S5UuAAM/SpMA: sieve: file script: Failed to open sieve script: open(/usr/local/var/dovecot/sieve/script1.sieve) failed: Permission denied... I will do some more testing and report what I find. From emailbuilder88 at yahoo.com Wed Mar 11 01:33:35 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Tue, 10 Mar 2015 18:33:35 -0700 Subject: Why is Sieve trying to re-compile global scripts? In-Reply-To: <1426037030.2770.YahooMailBasic@web142403.mail.bf1.yahoo.com> Message-ID: <1426037615.9810.YahooMailBasic@web142406.mail.bf1.yahoo.com> > > > I have some global scripts that were running nicely. > > > > > > Then I opened one in an editor and (probably, but not 100% sure) > > > mindlessly saved the file, even though I hadn't made any changes. > > > > > > Shortly after, Sieve errors started showing in the log: > > > > > > Error: 4k5JA74R/1TlIwABG/SpMA: sieve: binary save: failed to create temporary file: > > open(/usr/local/var/dovecot/sieve/script2.svbin.example.com.4139.) failed: Permission denied... > > > Error: 4k5JA74R/1TlIwABG/SpMA: sieve: The LDA Sieve plugin does not have permission to save global > > Sieve script binaries; global Sieve scripts like `/usr/local/var/dovecot/sieve/script2.sieve' > > need to be pre-compiled using the sievec tool > > > > > > Well, OK, is it going by the timestamp on the files? Fine. I recompiled > > > it by hand. Yet, I STILL got these errors! > > > > > > I triple and quadruple checked that the timestamp on the svbin files was > > > more recent. And Sieve was only complaining about one of the two > > > scripts in the directory. > > > > > > I restarted dovecot. No change. > > > > > > So I removed read permission on the .sieve files and only left read > > > permission on the .svbin files. THIS WORKED. No more error. > > > I can live with that, but why was it not complaining before, why was it > > > only complaining about one of my scripts and why would it complain > > > at all when the timestamps on the svbin should have indicated on > > > compilation is needed? > > > > I've heard about this problem before. Do you have the opportunity to > > test this with the 0.4.7.rc1 release? That adds a few extra debug lines > > (shown when mail_debug=yes) that would indicate why Sieve is thinking > > the global script is not up-to-date. > > Yes, I do as a matter of fact. I was just going to put in the RC in > order to answer your email on the thread about the RC. Don't have the > full answers yet, but when I installed the RC and restarted, I now get > an error where Sieve doesn't like that I won't give it read permission > on the .sieve file, so now I'm back to square one with this particular > issue. > > OTOH, regarding my earlier post about the RC ignoring seive files, at > least it is seeing global scripts (or trying to). Not sure about > personal scripts yet. > > Error: TiQJHH2X/1S5UuAAM/SpMA: sieve: file script: Failed to open sieve script: > open(/usr/local/var/dovecot/sieve/script1.sieve) failed: Permission denied... > > I will do some more testing and report what I find. I gave read permission to the .sieve files and the same original error happens as with .0.4.6. Now it's complaining about both scripts in my global directory. That it was working without these errors for a while and then complained only about one of the scripts, now both scripts seems to say something but I'm not sure what. Maybe I'll try to recreate the files for fun. I'll test personal scripts again too... From emailbuilder88 at yahoo.com Wed Mar 11 02:16:56 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Tue, 10 Mar 2015 19:16:56 -0700 Subject: Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1 In-Reply-To: <54FF4B82.80403@rename-it.nl> Message-ID: <1426040216.60360.YahooMailBasic@web142406.mail.bf1.yahoo.com> > >> Last time I had a few stupid problems in the releases, so > >> I'll follow > >> Timo's example and I release an RC first. > >> > >> The highlights include the implementation of the index and > >> metadata > >> extensions. Quite a few bugs are fixed as well. > > When I compiled and installed this, Sieve scripts were being ignored. Not > > sure if it's my own stupid mistake, but when I put v0.4.6 back in place, it > > worked fine. No configuration changes, only make install on the different > > sources and restart dovecot. > > Could you show your dovecot -n output? > > Also, if you enable mail_debug, what sieve-related debug lines are shown? OK, I re-tested and it's still ignoring personal scripts (but not global ones). No .svbin gets generated, no errors, just nothing. However, I do see that Sieve was accessing the user home directory because for some reason now it just created a ".pki" directory therein, which inside of it has an empty "nssdb" directory. That never happened before...? Not a big problem, but I'd prefer not to have that there. Re: mail_debug, this relates to another post I made that didn't get any replies - can I not override settings such as that (and log_path) from a userdb lookup? Hmm, I WAS able to override mail_debug from userdb, but not log_path? Sieve-related mail_debug, then? This looks like the relevant log info: dovecot: lmtp(testuser at example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: file storage: Storage path `/vmail/example.com/testuser/sieve' not found dovecot: lmtp(testuser at example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: No default script configured for user dovecot: lmtp(testuser at example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: User has no personal script I'll check on 0.4.6 and report if I see anything interesting, but I will assume for the moment that since personal scripts work in 0.4.6 that this log info won't be there. It is correct that there is no "sieve" file or directory in the user's home dir. This wasn't a problem in 0.4.6. Is it a requirement? Also, if you didn't see my post a few days ago, while I have your attention, I thank you for the extdata plugin and vote to keep it alive. Only caveat is I hope to see the dict mechanism in Dovecot become more flexible... at a minimum, would like to be able to pass at least one parameter (beside the implicit username) and indicate what field to test it against (I would like at least a simply-configured WHERE clause) if not full query customization. :) Perhaps not easy stuff, but extdata as is still helps me out. doveconf -n # 2.2.16.rc1: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.7.rc1 (c74220e16e0f+) # OS: Linux 3.10.0-123.20.1.el7.x86_64 x86_64 CentOS Linux release 7.0.1406 (Core) xfs dict { sieve = mysql:/usr/local/etc/dovecot/pigeonhole-sql.dict.ext } mail_location = mdbox:/vmail/%d/%1Mn/%1.1Mn/%n mail_plugins = " zlib" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /usr/local/var/dovecot/sieve/ sieve_execute_bin_dir = /usr/local/var/dovecot/sieve-extscripts/ sieve_extdata_dict_uri = proxy::sieve sieve_global_extensions = +vnd.dovecot.extdata +vnd.dovecot.execute sieve_plugins = sieve_extdata sieve_extprograms zlib_save = gz zlib_save_level = 9 } service auth-worker { user = $default_internal_user } service dict { unix_listener dict { group = vmail mode = 0660 } } service imap-login { inet_listener imap { address = 127.0.0.1 } } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } } ssl_cert = Message-ID: <1426042595.88992.YahooMailBasic@web142404.mail.bf1.yahoo.com> > > > > I have some global scripts that were running nicely. > > > > > > > > Then I opened one in an editor and (probably, but not 100% sure) > > > > mindlessly saved the file, even though I hadn't made any changes. > > > > > > > > Shortly after, Sieve errors started showing in the log: > > > > > > > > Error: 4k5JA74R/1TlIwABG/SpMA: sieve: binary save: failed to create temporary file: > > > open(/usr/local/var/dovecot/sieve/script2.svbin.example.com.4139.) failed: Permission denied... > > > > Error: 4k5JA74R/1TlIwABG/SpMA: sieve: The LDA Sieve plugin does not have permission to save global > > > Sieve script binaries; global Sieve scripts like `/usr/local/var/dovecot/sieve/script2.sieve' > > > need to be pre-compiled using the sievec tool > > > > > > > > Well, OK, is it going by the timestamp on the files? Fine. I recompiled > > > > it by hand. Yet, I STILL got these errors! > > > > > > > > I triple and quadruple checked that the timestamp on the svbin files was > > > > more recent. And Sieve was only complaining about one of the two > > > > scripts in the directory. > > > > > > > > I restarted dovecot. No change. > > > > > > > > So I removed read permission on the .sieve files and only left read > > > > permission on the .svbin files. THIS WORKED. No more error. > > > > I can live with that, but why was it not complaining before, why was it > > > > only complaining about one of my scripts and why would it complain > > > > at all when the timestamps on the svbin should have indicated on > > > > compilation is needed? > > > > > > I've heard about this problem before. Do you have the opportunity to > > > test this with the 0.4.7.rc1 release? That adds a few extra debug lines > > > (shown when mail_debug=yes) that would indicate why Sieve is thinking > > > the global script is not up-to-date. > > > > Yes, I do as a matter of fact. I was just going to put in the RC in > > order to answer your email on the thread about the RC. Don't have the > > full answers yet, but when I installed the RC and restarted, I now get > > an error where Sieve doesn't like that I won't give it read permission > > on the .sieve file, so now I'm back to square one with this particular > > issue. > > > > OTOH, regarding my earlier post about the RC ignoring seive files, at > > least it is seeing global scripts (or trying to). Not sure about > > personal scripts yet. > > > > Error: TiQJHH2X/1S5UuAAM/SpMA: sieve: file script: Failed to open sieve script: > > open(/usr/local/var/dovecot/sieve/script1.sieve) failed: Permission denied... > > > > I will do some more testing and report what I find. > > I gave read permission to the .sieve files and the same > original error happens as with .0.4.6. Now it's complaining > about both scripts in my global directory. That it was > working without these errors for a while and then complained > only about one of the scripts, now both scripts seems to say > something but I'm not sure what. Maybe I'll try to recreate the > files for fun. The relevant mail_debug lines seem to be these: dovecot: lmtp(testuser at example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: Opening script 1 of 2 from `/usr/local/var/dovecot/sieve/script1.sieve' dovecot: lmtp(testuser at example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: Loading script /usr/local/var/dovecot/sieve/script1.sieve dovecot: lmtp(testuser at example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: binary open: binary /usr/local/var/dovecot/sieve/script1.svbin stored with different binary version 1.2 (!= 1.3; automatically fixed when re-compiled) dovecot: lmtp(testuser at example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: Script `script1' from /usr/local/var/dovecot/sieve/script1.sieve successfully compiled Is this possibly due to a mixing of 0.4.6 and 0.4.7 sievec command? Well, I'm not sure that would be it because when I started getting ther error, I recompiled the sieve scrips and restarted dovecot which presumably would have made software versions match up. On the other hand, I don't know exactly what's happening: I downgraded to 0.4.6 again, intentionally triggered the error by updating the timestamp on the .sieve file, recompiled the script and now the error went away. From emailbuilder88 at yahoo.com Wed Mar 11 03:30:36 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Tue, 10 Mar 2015 20:30:36 -0700 Subject: Extending dict lookups (SQL)? Message-ID: <1426044636.95080.YahooMailBasic@web142404.mail.bf1.yahoo.com> Hi, How difficult would it be to try to hack some extension to the Dovecot dict mechanism for someone unfamiliar with the code? I'm using SQL as a backend and am looking for, at a minimum, the ability to specify a WHERE clause in addition to the built-in one that feeds the current username. The field name to test the additional value against would be necessary as well of course. Naturally, full query customization would be great to have. Perhaps providing a mechanism for the caller to pass data for the extra WHERE clause is the hardest part, not sure... but it can't hurt to ask. Don't take this as lack of gratitude - I can work with what's there and greatly appreciate it. From emailbuilder88 at yahoo.com Wed Mar 11 04:25:44 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Tue, 10 Mar 2015 21:25:44 -0700 Subject: Why is Sieve trying to re-compile global scripts? In-Reply-To: <1426042595.88992.YahooMailBasic@web142404.mail.bf1.yahoo.com> Message-ID: <1426047944.36841.YahooMailBasic@web142404.mail.bf1.yahoo.com> > > > > > I have some global scripts that were running nicely. > > > > > > > > > > Then I opened one in an editor and (probably, but not 100% sure) > > > > > mindlessly saved the file, even though I hadn't made any changes. > > > > > > > > > > Shortly after, Sieve errors started showing in the log: > > > > > > > > > > Error: 4k5JA74R/1TlIwABG/SpMA: sieve: binary save: failed to create temporary file: > > > > open(/usr/local/var/dovecot/sieve/script2.svbin.example.com.4139.) failed: Permission denied... > > > > > Error: 4k5JA74R/1TlIwABG/SpMA: sieve: The LDA Sieve plugin does not have permission to save global > > > > Sieve script binaries; global Sieve scripts like `/usr/local/var/dovecot/sieve/script2.sieve' > > > > need to be pre-compiled using the sievec tool > > > > > > > > > > Well, OK, is it going by the timestamp on the files? Fine. I recompiled > > > > > it by hand. Yet, I STILL got these errors! > > > > > > > > > > I triple and quadruple checked that the timestamp on the svbin files was > > > > > more recent. And Sieve was only complaining about one of the two > > > > > scripts in the directory. > > > > > > > > > > I restarted dovecot. No change. > > > > > > > > > > So I removed read permission on the .sieve files and only left read > > > > > permission on the .svbin files. THIS WORKED. No more error. > > > > > I can live with that, but why was it not complaining before, why was it > > > > > only complaining about one of my scripts and why would it complain > > > > > at all when the timestamps on the svbin should have indicated on > > > > > compilation is needed? > > > > > > > > I've heard about this problem before. Do you have the opportunity to > > > > test this with the 0.4.7.rc1 release? That adds a few extra debug lines > > > > (shown when mail_debug=yes) that would indicate why Sieve is thinking > > > > the global script is not up-to-date. > > > > > > Yes, I do as a matter of fact. I was just going to put in the RC in > > > order to answer your email on the thread about the RC. Don't have the > > > full answers yet, but when I installed the RC and restarted, I now get > > > an error where Sieve doesn't like that I won't give it read permission > > > on the .sieve file, so now I'm back to square one with this particular > > > issue. > > > > > > OTOH, regarding my earlier post about the RC ignoring seive files, at > > > least it is seeing global scripts (or trying to). Not sure about > > > personal scripts yet. > > > > > > Error: TiQJHH2X/1S5UuAAM/SpMA: sieve: file script: Failed to open sieve script: > > > open(/usr/local/var/dovecot/sieve/script1.sieve) failed: Permission denied... > > > > > > I will do some more testing and report what I find. > > > > I gave read permission to the .sieve files and the same > > original error happens as with .0.4.6. Now it's complaining > > about both scripts in my global directory. That it was > > working without these errors for a while and then complained > > only about one of the scripts, now both scripts seems to say > > something but I'm not sure what. Maybe I'll try to recreate the > > files for fun. > > The relevant mail_debug lines seem to be these: > > dovecot: lmtp(testuser example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: Opening script 1 of 2 > from `/usr/local/var/dovecot/sieve/script1.sieve' > dovecot: lmtp(testuser example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: Loading script /usr/local/var/dovecot/sieve/script1.sieve > dovecot: lmtp(testuser example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: binary open: binary > /usr/local/var/dovecot/sieve/script1.svbin stored with different binary version 1.2 (!= 1.3; > automatically fixed when re-compiled) > dovecot: lmtp(testuser example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: Script `script1' from > /usr/local/var/dovecot/sieve/script1.sieve successfully compiled > > Is this possibly due to a mixing of 0.4.6 and 0.4.7 sievec command? > Well, I'm not sure that would be it because when I started getting > ther error, I recompiled the sieve scrips and restarted dovecot > which presumably would have made software versions match up. > > On the other hand, I don't know exactly what's happening: I downgraded > to 0.4.6 again, intentionally triggered the error by updating the > timestamp on the .sieve file, recompiled the script and now the > error went away. After editing one of the global scripts (and compiling it), I am able to get the error back again (and not able to get it to go away). The previous log info I found may have been unrelated and more to do with haivng switched to 0.4.7 without recompiling the scripts. I'm back with 0.4.6 and the only thing I see in the log now is this: Debug: lRgL3tO1/1RvOyA6M/SpMA: sieve: Script binary /usr/local/var/dovecot/sieve/script2.svbin is not up-to-date Debug: lRgL3tO1/1RvOyA6M/SpMA: sieve: Script `script2' from /usr/local/var/dovecot/sieve/script2.sieve successfully compiled Error: lRgL3tO1/1RvOyA6M/SpMA: sieve: binary save: failed to create temporary file: open(... All I can think is that when I initially triggered the error, I noticed that I exited my editor and compiled the script within the *same minute* thus creating timestamps that were equal when compared without seconds. But now, even after recompiling to get a much later timestamp on the binary, I can't get the error to go away. I upgraded to 0.4.7, and re-compiled one of my two scripts in the same way (during the same minute), and indeed, the first script (that I DID NOT recompile) gets the previous error I saw with the mismatched binary version notice -- that seems irrelevant then, only reltaed to the upgrade. The script that I did recompile (during the same minute as I saved it) after upgrading causes the same error, so the bug seems consistent across versions. However, there is one additional debug line in version 0.4.7: sieve: binary up-to-date: script metadata indicates that binary /usr/local/var/dovecot/sieve/script2.svbin is not up-to-date Doesn't say which metadata. Downgraded back to 0.4.6, deleted the svbin files, compiled again, and now the error persists. Tried deleting the .sieve source files, re-created them, waited until the next minute, compiled them. Error still in logs. Not sure how I got it to go away last time. Something being cached somewhere? From emailbuilder88 at yahoo.com Wed Mar 11 03:37:28 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Tue, 10 Mar 2015 20:37:28 -0700 Subject: Sieve extprograms socket vs. direct execution Message-ID: <1426045048.26340.YahooMailBasic@web142405.mail.bf1.yahoo.com> Hi, I'm hoping to get some clarification of the differences between calling a script using the Sieve extprograms plugin execute method via direct execution or using the socket feature. Being naive, I see the socket option and think that way you tell Dovecot to spawn a daemon and I think that's going to be far superior in performance. But if that really was the difference, why would direct execution even be an option? (And doesn't the daemon still have to spawn a new shell every time if the target is a shell script? Does that defeat the purpose of having a daemon?) So I don't really know what the difference is, and under what circumstances you'd want to use one or the other. Can someone please help clarify? From emailbuilder88 at yahoo.com Wed Mar 11 05:22:44 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Tue, 10 Mar 2015 22:22:44 -0700 Subject: Sieve reject with ORIG-TO vs TO Message-ID: <1426051364.22765.YahooMailBasic@web142403.mail.bf1.yahoo.com> Hi, The bounce message generated by the reject extension has what looks like a hard coded message prefix that comes before the configurable reason text: "Your message to was automatically rejected:" In some cases, the is NOT the original-to address, which can cause confusion to the sender or expose private aliasing data that some people might want to hide. Is there a way to make the reject extension use the original-to address in that hard-coded message prefix? Is there a way to completely customize the FULL message? Is there a way to customize the headers for such messages? From emailbuilder88 at yahoo.com Wed Mar 11 05:52:01 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Tue, 10 Mar 2015 22:52:01 -0700 Subject: Sieve security: Any way to protect credentials used in extprograms? Message-ID: <1426053121.88304.YahooMailBasic@web142406.mail.bf1.yahoo.com> I need to connect to a database in a script called using Sieve extprograms plugin. When delivering mail, Sieve is running as the mail recipient user, which means any files, either the sieve script or the extprograms it invokes, are run under that user's permissions. What would be a way to hide the database credentials in a more restricted file? I can think of... * Store the credentials in the database itself, return them from a DICT lookup that is fed to the sieve script. The credentials would have to be duplicated for every user record in the database. * Somehow put the value in the Sieve environment, but I'm not sure how to get a value into the environment (just return it in a userdb lookup?) and then how to get at that value once I'm executing the sieve script (can I retrieve it with the "environment" extension?) Are there better ways? [Of course, if I could do more elaborate things like call a stored procedure from Dovecot DICT then I could avoid this situation] From emailbuilder88 at yahoo.com Wed Mar 11 06:17:45 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Tue, 10 Mar 2015 23:17:45 -0700 Subject: Why is Sieve trying to re-compile global scripts? In-Reply-To: <1426047944.36841.YahooMailBasic@web142404.mail.bf1.yahoo.com> Message-ID: <1426054665.13781.YahooMailBasic@web142405.mail.bf1.yahoo.com> > Not sure how I got it to go away last time. Might have gotten it to go away by deleting the scripts, causing an email delivery, THEN creating the scripts again. Although I think my ideas are all flawed: I can delete the scripts and recreate and recompile all in the same minute and I don't get errors. I can cause the error to happen again by editing and recompiling one of the files, *whether or not in the same minute* and I do get the error. This time around, deleting the files and recreating/recompiling them even without an email delivery in between seems to fix the error. Might be unpredictable caching. Might be the error didn't go away last time I recreated due to different methods of creating the files. Who knows, I think I should give up and stop spamming the list with uneducated guesswork. From HFlor at gmx.de Wed Mar 11 08:35:17 2015 From: HFlor at gmx.de (Hardy Flor) Date: Wed, 11 Mar 2015 09:35:17 +0100 Subject: Deduplicate Public Folder In-Reply-To: <54FF4EC9.3040802@kc0dxf.net> References: <54FF4EC9.3040802@kc0dxf.net> Message-ID: <54FFFE45.9010701@gmx.de> use the -o for doveadm: doveadm -o "mail_location=mdbox:/:LAYOUT=fs" ... Am 10.03.2015 um 21:06 schrieb Bobber: > How do you perform the doveadm deduplicate command on public folders? > If I specify a user, nothing is found. Using -A also doesn't seem to > work. > > From Olaf.Hopp at kit.edu Wed Mar 11 10:10:38 2015 From: Olaf.Hopp at kit.edu (Olaf Hopp) Date: Wed, 11 Mar 2015 11:10:38 +0100 Subject: Why is Sieve trying to re-compile global scripts? In-Reply-To: <1426054665.13781.YahooMailBasic@web142405.mail.bf1.yahoo.com> References: <1426054665.13781.YahooMailBasic@web142405.mail.bf1.yahoo.com> Message-ID: <5500149E.2010309@kit.edu> On 03/11/2015 07:17 AM, E.B. wrote: > > Might be unpredictable caching. Might be the error didn't go away > last time I recreated due to different methods of creating the files. > Who knows, I think I should give up and stop spamming the list > with uneducated guesswork. No - no spam at least for me. Please see the thread with subject "Sieve permissions issue following update" I tested sucessfully a developper issue last month on the hint of Stephan. Yesterday I started to test the currenr RCs. First I was disappointed, because the error seems to persist. So I double checked everything, recreated / recompiled everything an the error went away. So I thought it was mistake on my side. I gave Spephan postive feedback. And I'm waiting for the final release for my production server. But when I read your mails, I'm not feeling happy. I think it's a kink of luck/voodoo/whatever. What you must do, I think, is to compile the sieve script with the exact version running afterwards. And I think you should the remove the compiled .svbin files before recreating them again. Don't overwrite them with the compiler. I think I'll also dig into this any further today. Olaf -- Karlsruher Institut f?r Technologie (KIT) ATIS - Abt. Technische Infrastruktur, Fakult?t f?r Informatik Dipl.-Geophys. Olaf Hopp - Leitung IT-Dienste - Am Fasanengarten 5, Geb?ude 50.34, Raum 009 76131 Karlsruhe Telefon: +49 721 608-43973 Fax: +49 721 608-46699 E-Mail: Olaf.Hopp at kit.edu atis.informatik.kit.edu www.kit.edu KIT - Universit?t des Landes Baden-W?rttemberg und nationales Forschungszentrum in der Helmholtz-Gemeinschaft Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5214 bytes Desc: S/MIME Cryptographic Signature URL: From HFlor at gmx.de Wed Mar 11 10:50:08 2015 From: HFlor at gmx.de (Hardy Flor) Date: Wed, 11 Mar 2015 11:50:08 +0100 Subject: rawlog with -i not worked In-Reply-To: <54FDDE19.2080401@gmx.de> References: <54FDDE19.2080401@gmx.de> Message-ID: <55001DE0.7000001@gmx.de> When I change the option 'i' to the letter 'a' is everything as designed. Why is 'i' at this location the problem? ******************************** diff -Nurb dovecot-2.2.16rc1/src/util/rawlog.c dovecot-2.2.16rc1.build/src/util/rawlog.c --- dovecot-2.2.16rc1/src/util/rawlog.c 2015-03-04 13:49:28.000000000 +0100 +++ dovecot-2.2.16rc1.build/src/util/rawlog.c 2015-03-11 11:31:46.090201199 +0100 @@ -362,7 +362,7 @@ int c; master_service = master_service_init("rawlog", 0, - &argc, &argv, "+f:obit"); + &argc, &argv, "+f:obat"); while ((c = master_getopt(master_service)) > 0) { switch (c) { case 'f': @@ -376,7 +376,7 @@ case 'b': flags |= RAWLOG_FLAG_LOG_BOUNDARIES; break; - case 'i': + case 'a': flags |= RAWLOG_FLAG_LOG_IP_IN_FILENAME; break; case 't': @@ -390,7 +390,7 @@ argv += optind; if (argc < 1) - i_fatal("Usage: rawlog [-f in|out] [-i] [-b] [-t] "); + i_fatal("Usage: rawlog [-f in|out] [-a] [-b] [-t] "); master_service_init_log(master_service, "rawlog: "); master_service_init_finish(master_service); Am 09.03.2015 um 18:53 schrieb Hardy Flor: > > Without -i everything is wonderful, a coredump is with the -i > parameter created > From HFlor at gmx.de Wed Mar 11 13:46:59 2015 From: HFlor at gmx.de (Hardy Flor) Date: Wed, 11 Mar 2015 14:46:59 +0100 Subject: dec2str ... Message-ID: <55004753.7070307@gmx.de> a very stupid question: What reason is there for an output with printf until dec2str to convert the numeric value to a string and not to use the format identifier "%d" or "u%"? From r at sys4.de Wed Mar 11 14:29:55 2015 From: r at sys4.de (Ralf Hildebrandt) Date: Wed, 11 Mar 2015 15:29:55 +0100 Subject: full text index "per user"? In-Reply-To: <20150301211436.GB2996@sys4.de> References: <20150301211436.GB2996@sys4.de> Message-ID: <20150311142955.GD3020@sys4.de> * Ralf Hildebrandt : > Is there any way of disabling the creation of a full text index on a > per user basis? userdb lookup which returns "fts="! -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From kaniggl at gmail.com Wed Mar 11 14:37:48 2015 From: kaniggl at gmail.com (kaniggl) Date: Wed, 11 Mar 2015 15:37:48 +0100 Subject: libdriver_msql.so Message-ID: Hello, i installed dovecot 2.11.1 on ubuntu Linux dadd3041 3.16.0-30-generic #40-Ubuntu SMP Mon Jan 12 22:07:11 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux To make it clear, architecture is PowerPC 64bit Then i installed dovecot via apt-get. But the file /usr/lib/dovecot/modules/auth/libdriver_mysql.so is missing. So i compiled dovecot myself on this system and got a file /usr/lib/dovecot/modules/auth/libdriver_mysql.so Then i restartetd the server and try to do a login via a webmailer (roundcube), but i get no connection to the database server. This is found in /var/log/mail.log Mar 11 15:19:04 dadd3041 dovecot: master: Dovecot v2.2.9 starting up (core dumps disabled) Mar 11 15:19:04 dadd3041 postfix/master[3367]: reload -- version 2.11.1, configuration /etc/postfix Mar 11 15:19:04 dadd3041 postfix/master[3367]: terminating on signal 15 Mar 11 15:19:04 dadd3041 postfix/master[3597]: daemon started -- version 2.11.1, configuration /etc/postfix Mar 11 15:19:25 dadd3041 dovecot: auth: Error: dlopen(/usr/lib/dovecot/modules/auth/libdriver_mysql.so) failed: /usr/lib/dovecot/modules/auth/libdriver_mysql.so: wrong ELF class: ELFCLASS32 Mar 11 15:19:25 dadd3041 dovecot: auth: Fatal: Unknown database driver 'mysql' Mar 11 15:19:25 dadd3041 dovecot: master: Error: service(auth): command startup failed, throttling for 2 secs Mar 11 15:19:25 dadd3041 dovecot: imap-login: Disconnected: Auth process broken (disconnected before auth was ready, waited 0 secs): user=<>, rip=::1, lip=::1, TLS handshaking, session= I know this means it is not a 64bit object but how could i achieve to get right one? best regards, kaniggl From h.reindl at thelounge.net Wed Mar 11 14:41:43 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 11 Mar 2015 15:41:43 +0100 Subject: libdriver_msql.so In-Reply-To: References: Message-ID: <55005427.4070500@thelounge.net> Am 11.03.2015 um 15:37 schrieb kaniggl: > To make it clear, architecture is PowerPC 64bit > Then i installed dovecot via apt-get. But the file > /usr/lib/dovecot/modules/auth/libdriver_mysql.so > is missing. install the sub-package "dovecot-mysql" and the next time *ask before* you ruin your system > So i compiled dovecot myself on this system and got a file > /usr/lib/dovecot/modules/auth/libdriver_mysql.so stupid idea, i guess without install the package proper > Mar 11 15:19:25 dadd3041 dovecot: auth: Error: > dlopen(/usr/lib/dovecot/modules/auth/libdriver_mysql.so) failed: > /usr/lib/dovecot/modules/auth/libdriver_mysql.so: wrong ELF class: > ELFCLASS32 > Mar 11 15:19:25 dadd3041 dovecot: auth: Fatal: Unknown database driver > 'mysql' likely the result of mix package and self compile instead use google https://packages.debian.org/de/sid/dovecot-mysql -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From kaniggl at gmail.com Wed Mar 11 14:43:20 2015 From: kaniggl at gmail.com (kaniggl) Date: Wed, 11 Mar 2015 15:43:20 +0100 Subject: libdriver_msql.so In-Reply-To: <55005427.4070500@thelounge.net> References: <55005427.4070500@thelounge.net> Message-ID: of course i installed dovecot-mysql before, but no file libdriver_msql.so was in there 2015-03-11 15:41 GMT+01:00 Reindl Harald : > > > Am 11.03.2015 um 15:37 schrieb kaniggl: > >> To make it clear, architecture is PowerPC 64bit >> Then i installed dovecot via apt-get. But the file >> /usr/lib/dovecot/modules/auth/libdriver_mysql.so >> is missing. >> > > install the sub-package "dovecot-mysql" and the next time *ask before* you > ruin your system > > So i compiled dovecot myself on this system and got a file >> /usr/lib/dovecot/modules/auth/libdriver_mysql.so >> > > stupid idea, i guess without install the package proper > > Mar 11 15:19:25 dadd3041 dovecot: auth: Error: >> dlopen(/usr/lib/dovecot/modules/auth/libdriver_mysql.so) failed: >> /usr/lib/dovecot/modules/auth/libdriver_mysql.so: wrong ELF class: >> ELFCLASS32 >> Mar 11 15:19:25 dadd3041 dovecot: auth: Fatal: Unknown database driver >> 'mysql' >> > > likely the result of mix package and self compile instead use google > https://packages.debian.org/de/sid/dovecot-mysql > > From h.reindl at thelounge.net Wed Mar 11 14:47:43 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 11 Mar 2015 15:47:43 +0100 Subject: libdriver_msql.so In-Reply-To: References: <55005427.4070500@thelounge.net> Message-ID: <5500558F.2040203@thelounge.net> Am 11.03.2015 um 15:43 schrieb kaniggl: > of course i installed dovecot-mysql before, but no file libdriver_msql.so > was in there i doubt that you are the only person using debian with dovecot and mysql however, that's not a dovecot question ask on your OS list how to fix your ruined setup > 2015-03-11 15:41 GMT+01:00 Reindl Harald : >> >> Am 11.03.2015 um 15:37 schrieb kaniggl: >> >>> To make it clear, architecture is PowerPC 64bit >>> Then i installed dovecot via apt-get. But the file >>> /usr/lib/dovecot/modules/auth/libdriver_mysql.so >>> is missing. >>> >> >> install the sub-package "dovecot-mysql" and the next time *ask before* you >> ruin your system >> >> So i compiled dovecot myself on this system and got a file >>> /usr/lib/dovecot/modules/auth/libdriver_mysql.so >>> >> >> stupid idea, i guess without install the package proper >> >> Mar 11 15:19:25 dadd3041 dovecot: auth: Error: >>> dlopen(/usr/lib/dovecot/modules/auth/libdriver_mysql.so) failed: >>> /usr/lib/dovecot/modules/auth/libdriver_mysql.so: wrong ELF class: >>> ELFCLASS32 >>> Mar 11 15:19:25 dadd3041 dovecot: auth: Fatal: Unknown database driver >>> 'mysql' >>> >> >> likely the result of mix package and self compile instead use google >> https://packages.debian.org/de/sid/dovecot-mysql -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From teemu.huovila at dovecot.fi Wed Mar 11 14:52:52 2015 From: teemu.huovila at dovecot.fi (Teemu Huovila) Date: Wed, 11 Mar 2015 16:52:52 +0200 Subject: dec2str ... In-Reply-To: <55004753.7070307@gmx.de> References: <55004753.7070307@gmx.de> Message-ID: <550056C4.1070303@dovecot.fi> On 03/11/2015 03:46 PM, Hardy Flor wrote: > a very stupid question: What reason is there for an output with printf until dec2str to convert the numeric value to a string > and not to use the format identifier "%d" or "u%"? The length of types such as pid_t or time_t can, at least in theory, vary from operating system to system. Thus no length modifier in the format can be correct on all systems. br, Teemu Huovila From mailinglist at darac.org.uk Wed Mar 11 15:05:56 2015 From: mailinglist at darac.org.uk (Darac Marjal) Date: Wed, 11 Mar 2015 15:05:56 +0000 Subject: libdriver_msql.so In-Reply-To: References: Message-ID: <20150311150555.GA18365@darac.org.uk> On Wed, Mar 11, 2015 at 03:37:48PM +0100, kaniggl wrote: > Hello, > > i installed dovecot 2.11.1 on ubuntu > Linux dadd3041 3.16.0-30-generic #40-Ubuntu SMP Mon Jan 12 22:07:11 UTC > 2015 ppc64le ppc64le ppc64le GNU/Linux > > To make it clear, architecture is PowerPC 64bit > Then i installed dovecot via apt-get. But the file > /usr/lib/dovecot/modules/auth/libdriver_mysql.so > is missing. According to https://launchpad.net/ubuntu/+source/dovecot, Ubuntu is still on Dovecot 2.2.9. According to http://dovecot.org/, upstream is only at 2.2.15. Where did you find dovecot version 2.11.1 and can you tell us the Lotto numbers, please? Try "dpkg -L dovecot-mysql" to list files installed by the dovecot-mysql package. > > So i compiled dovecot myself on this system and got a file > /usr/lib/dovecot/modules/auth/libdriver_mysql.so > > Then i restartetd the server and try to do a login via a webmailer > (roundcube), but i get no connection to the database server. > > This is found in /var/log/mail.log > Mar 11 15:19:04 dadd3041 dovecot: master: Dovecot v2.2.9 starting up (core > dumps disabled) > Mar 11 15:19:04 dadd3041 postfix/master[3367]: reload -- version 2.11.1, > configuration /etc/postfix > Mar 11 15:19:04 dadd3041 postfix/master[3367]: terminating on signal 15 > Mar 11 15:19:04 dadd3041 postfix/master[3597]: daemon started -- version > 2.11.1, configuration /etc/postfix > Mar 11 15:19:25 dadd3041 dovecot: auth: Error: > dlopen(/usr/lib/dovecot/modules/auth/libdriver_mysql.so) failed: > /usr/lib/dovecot/modules/auth/libdriver_mysql.so: wrong ELF class: > ELFCLASS32 > Mar 11 15:19:25 dadd3041 dovecot: auth: Fatal: Unknown database driver > 'mysql' > Mar 11 15:19:25 dadd3041 dovecot: master: Error: service(auth): command > startup failed, throttling for 2 secs > Mar 11 15:19:25 dadd3041 dovecot: imap-login: Disconnected: Auth process > broken (disconnected before auth was ready, waited 0 secs): user=<>, > rip=::1, lip=::1, TLS handshaking, > session= > > I know this means it is not a 64bit object but how could i achieve to get > right one? > > best regards, > kaniggl -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Digital signature URL: From gessel at blackrosetech.com Wed Mar 11 17:12:29 2015 From: gessel at blackrosetech.com (David Gessel) Date: Wed, 11 Mar 2015 20:12:29 +0300 Subject: FTS search with CLucene giving odd results/exposing config issue Message-ID: <5500777D.8040100@blackrosetech.com> I recently got FTS search with Clucene working on FreeBSD 10.1 (I had some trouble with 9.3, postponed solving it, then restarted with 10.1 by manually installing libtextcat, a listed dependency on http://wiki2.dovecot.org/Plugins/FTS/Lucene, but not part of the FreeBSD makefile: maybe that helped, maybe it was irrelevant). However, I now get some anomalous results. I'm putting them together as they may be related. Or, maybe not. 1) Searching from Thunderbird returns 35x as many results for the same term as searching from the command line. I'm not sure if the indexes are created or enumerable. 2) There seems to be a glitch in the maildir naming structure that results in a double slash (//) which doesn't seem to impact IMAP serving or Sieve, but maybe is causing the problem above with Clucene. Any hints? If I run a search from Thunderbird and "run search on server" for a somewhat infrequently used term like "titanium," I get results in more or less every archive folder one by one over about 20 minutes of searching and get about 516 results. It seems to run a little faster without the "run search on server" checkbox selected, but Thunderbird on Windows is abandonware and the only useful datapoint is 516 results. Maybe it is time for Claws. Anyway, if I run: # doveadm search -u user at domain.com mailbox '*' body titanium doveadm(user at domain.com): Error: stat(/mail/domain.com/user//.dovecot.sieve/tmp) failed: Not a directory doveadm(user at domain.com): Error: Syncing mailbox dovecot.sieve failed: Internal error occurred. Refer to server log for more information. [2015-03-11 09:11:16] bcadc922ba26b953a25701002812f29a 812478 bcadc922ba26b953a25701002812f29a 9338037 bcadc922ba26b953a25701002812f29a 9342918 bcadc922ba26b953a25701002812f29a 9343205 bcadc922ba26b953a25701002812f29a 9346096 bcadc922ba26b953a25701002812f29a 9350977 bcadc922ba26b953a25701002812f29a 9351379 bcadc922ba26b953a25701002812f29a 9352318 bcadc922ba26b953a25701002812f29a 9352441 bcadc922ba26b953a25701002812f29a 9352501 3d5d982e3a89b853c12c01002812f29a 1901940 3d5d982e3a89b853c12c01002812f29a 1908073 3d5d982e3a89b853c12c01002812f29a 1941453 3d5d982e3a89b853c12c01002812f29a 2036962 3d5d982e3a89b853c12c01002812f29a 2038360 I get 15 results. # doveadm fetch -u user at domain.com "mailbox" mailbox-guid 3d5d982e3a89b853c12c01002812f29a uid 1908073 mailbox: INBOX All are from the inbox. This takes roughly 30 seconds (much faster, that's good, though still not super fast) and in the logs I see promising messages like: Mar 11 09:27:52 host dovecot: indexer-worker(user at domain.com): Indexed 3708 messages in Commerce but also slightly less promising messages like: Mar 11 09:27:52 host dovecot: indexer-worker(user at domain.com): Warning: Maildir /mail/domain.com/user//.Bulkmail: Expunged message reappeared, giving a new UID (old uid=9335155, file=1426089637.M350915P91305.host,S=23845,W=24282:2,) If I execute # doveadm index -A '*' I get doveadm(user at domain.com): Error: stat(/mail/domain.com/user//.dovecot.sieve/tmp) failed: Not a directory doveadm(user at domain.com): Error: Syncing mailbox dovecot.sieve failed: Internal error occurred. Refer to server log for more information. [2015-03-11 09:51:26] (there's nothing in the server log) But note in both previous results a double slash (//) in the directory structure. I don't think that's right. So: Does # doveadm search search subfolders? The internets say either '*' or INBOX should search the hierarchy and the search command does set off indexing operations on every subfolder as indicated by the logs, but the results are only from the top level folder.... perhaps that is related to... Why do some errors report a double slash (//) in directory names? I don't seem to be having any other symptoms (if, indeed, this is one) of incompatibilities. My sieve filters work just fine and are accessible from the managesieve plugin in Thunderbird. My config file is below: # 2.2.15: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.6 (3e924b1b6c5c+) # OS: FreeBSD 10.1-RELEASE amd64 auth_default_realm = domain.com auth_mechanisms = plain login auth_realms = domain.com domain2.org disable_plaintext_auth = no first_valid_gid = 5000 first_valid_uid = 5000 hostname = domain.com imap_idle_notify_interval = 29 mins last_valid_gid = 5000 last_valid_uid = 5000 lda_mailbox_autocreate = yes listen = * mail_gid = 5000 mail_location = maildir:~:CONTROL=/var/no-quota/%u mail_plugins = quota mail_log notify fts fts_lucene mail_temp_dir = /var/tmp mail_uid = 5000 maildir_broken_filename_sizes = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate spamtest spamtestplus editheader mbox_lazy_writes = no namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { expire = Trash fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars=@. mime_parts mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota_exceeded_message = Storage quota for this account has been exceeded, please try again later. quota_rule = *:storage=50G quota_rule2 = Trash:storage=+30%% quota_rule3 = Sent:storage=+30%% quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=75%% quota-warning 75 %u sieve = ~/.dovecot.sieve sieve_before = /usr/local/etc/dovecot/sieve/ sieve_dir = ~/sieve sieve_extensions = +spamtest +spamtestplus +relational +comparator-i;ascii-numeric +editheader sieve_global_dir = /usr/local/etc/dovecot/sieve } postmaster_address = user at domain.com protocols = imap lmtp sieve sieve sendmail_path = /usr/local/sbin/sendmail service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 vsz_limit = 128 M } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = required ssl_cert = Hi, I noticed that dovecot's MySQL driver doesn't play well with UTF-8 encoded strings. I presume this issue has been around for a while. However, it still fails with the current release candidate 2.2.16-rc1. I have been using the sieve extdata plugin with a MySQL-backed dict to retrieve a folder name that is then passed to fileinto: fileinto :create "${extdata.folder_name_drafts}"; Whenever this dict returned a string with non-7byte characters, a line such as the following popped up in sieve.log. (This example query should return "Entw?rfe".) error: folder name specified for fileinto command is not utf-8: Entw?rfe. Although my MySQL installation is set up to use UTF-8 as the default charset on every level (config, database, table and field), and the character_set_* runtime variables all yield the value "utf8", it still seems that the mysql client library must be instructed to actually use UTF-8 explicitly. Adding the following statement to driver_mysql_connect() fixes the issue for me: mysql_options(db->mysql, MYSQL_SET_CHARSET_NAME, "utf8"); I checked this against mysql 5.5.41. Regards, Felix -------------- next part -------------- A non-text attachment was scrubbed... Name: mysql_utf8.patch Type: application/octet-stream Size: 559 bytes Desc: not available URL: From stephan at rename-it.nl Wed Mar 11 21:38:04 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 11 Mar 2015 22:38:04 +0100 Subject: Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1 In-Reply-To: <1426040216.60360.YahooMailBasic@web142406.mail.bf1.yahoo.com> References: <1426040216.60360.YahooMailBasic@web142406.mail.bf1.yahoo.com> Message-ID: <5500B5BC.1010508@rename-it.nl> On 3/11/2015 3:16 AM, E.B. wrote: >>>> Last time I had a few stupid problems in the releases, so >>>> I'll follow >>>> Timo's example and I release an RC first. >>>> >>>> The highlights include the implementation of the index and >>>> metadata >>>> extensions. Quite a few bugs are fixed as well. >>> When I compiled and installed this, Sieve scripts were being ignored. Not >>> sure if it's my own stupid mistake, but when I put v0.4.6 back in place, it >>> worked fine. No configuration changes, only make install on the different >>> sources and restart dovecot. >> Could you show your dovecot -n output? >> >> Also, if you enable mail_debug, what sieve-related debug lines are shown? > OK, I re-tested and it's still ignoring personal scripts (but not > global ones). No .svbin gets generated, no errors, just nothing. > However, I do see that Sieve was accessing the user home directory > because for some reason now it just created a ".pki" directory > therein, which inside of it has an empty "nssdb" directory. That > never happened before...? Not a big problem, but I'd prefer not > to have that there. Sieve doesn't do that. I don't think Dovecot does that either, but I am not sure. > Re: mail_debug, this relates to another post I made that didn't get any > replies - can I not override settings such as that (and log_path) from > a userdb lookup? Hmm, I WAS able to override mail_debug from userdb, > but not log_path? > > Sieve-related mail_debug, then? > > This looks like the relevant log info: > > dovecot: lmtp(testuser at example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: file storage: Storage path `/vmail/example.com/testuser/sieve' not found > dovecot: lmtp(testuser at example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: No default script configured for user > dovecot: lmtp(testuser at example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: User has no personal script > > I'll check on 0.4.6 and report if I see anything interesting, > but I will assume for the moment that since personal scripts > work in 0.4.6 that this log info won't be there. It is correct that > there is no "sieve" file or directory in the user's home dir. This > wasn't a problem in 0.4.6. Is it a requirement? Well, since your config says the following: sieve = file:~/sieve;active=~/.dovecot.sieve It expects a sieve storage directory at ~/sieve (created when ManageSieve is used to upload a script). Also, a symbolic link pointing to the active script will be located at ~/.dovecot.sieve once a script is activated (i.e. through ManageSieve or doveadm sieve). I wonder how this would have worked before with 0.4.6. Is the ~/.dovecot.sieve a normal script file perhaps (rather than a symlink)? This would mean that the following config would work (e.g. if you don't use ManageSieve): sieve = file:~/.dovecot.sieve > Also, if you didn't see my post a few days ago, while I have your > attention, I thank you for the extdata plugin and vote to keep it > alive. Only caveat is I hope to see the dict mechanism in Dovecot > become more flexible... at a minimum, would like to be able to pass > at least one parameter (beside the implicit username) and indicate > what field to test it against (I would like at least a simply-configured > WHERE clause) if not full query customization. :) Perhaps not easy > stuff, but extdata as is still helps me out. I haven't looked at that plugin in a while. I could add it to the main pigeonhole package, since it doesn't depend on anything else. It is still a work in progress though, so don't expect the syntax of the extensions to remain the same in the future when I would continue development. I don't know whether it is acceptable to change the dict functionality like that. Timo will know... Regards, Stephan. From stephan at rename-it.nl Wed Mar 11 21:50:13 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 11 Mar 2015 22:50:13 +0100 Subject: Sieve reject with ORIG-TO vs TO In-Reply-To: <1426051364.22765.YahooMailBasic@web142403.mail.bf1.yahoo.com> References: <1426051364.22765.YahooMailBasic@web142403.mail.bf1.yahoo.com> Message-ID: <5500B895.1020700@rename-it.nl> On 3/11/2015 6:22 AM, E.B. wrote: > Hi, > > The bounce message generated by the reject extension > has what looks like a hard coded message prefix that > comes before the configurable reason text: > > "Your message to was automatically rejected:" > > In some cases, the is NOT the original-to > address, which can cause confusion to the sender or > expose private aliasing data that some people might > want to hide. > > Is there a way to make the reject extension use the > original-to address in that hard-coded message prefix? There is currently no substitution variable for that available in the settings mentioned below. But that should be easy to add. I'll ask Timo. > Is there a way to completely customize the FULL > message? Those settings are at your disposal: http://hg.dovecot.org/dovecot-2.2/file/910727378a16/doc/example-config/conf.d/15-lda.conf#l25 > Is there a way to customize the headers for such > messages? Not that I know of. Regards, Stephan. From stephan at rename-it.nl Wed Mar 11 21:54:30 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 11 Mar 2015 22:54:30 +0100 Subject: Sieve security: Any way to protect credentials used in extprograms? In-Reply-To: <1426053121.88304.YahooMailBasic@web142406.mail.bf1.yahoo.com> References: <1426053121.88304.YahooMailBasic@web142406.mail.bf1.yahoo.com> Message-ID: <5500B996.4080505@rename-it.nl> On 3/11/2015 6:52 AM, E.B. wrote: > I need to connect to a database in a script called using Sieve > extprograms plugin. When delivering mail, Sieve is running > as the mail recipient user, which means any files, either the > sieve script or the extprograms it invokes, are run under that > user's permissions. > > What would be a way to hide the database credentials in a > more restricted file? I can think of... > > * Store the credentials in the database itself, return them > from a DICT lookup that is fed to the sieve script. The > credentials would have to be duplicated for every user > record in the database. > > * Somehow put the value in the Sieve environment, > but I'm not sure how to get a value into the environment > (just return it in a userdb lookup?) and then how to get > at that value once I'm executing the sieve script (can I > retrieve it with the "environment" extension?) > > Are there better ways? I would put that script invocation in a script service running as a different user. That should also answer much of your other e-mail. Regards, Stephan. From stephan at rename-it.nl Wed Mar 11 23:02:35 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 12 Mar 2015 00:02:35 +0100 Subject: Why is Sieve trying to re-compile global scripts? In-Reply-To: <5500149E.2010309@kit.edu> References: <1426054665.13781.YahooMailBasic@web142405.mail.bf1.yahoo.com> <5500149E.2010309@kit.edu> Message-ID: <5500C98B.70808@rename-it.nl> On 3/11/2015 11:10 AM, Olaf Hopp wrote: > Please see the thread with subject > "Sieve permissions issue following update" > I tested sucessfully a developper issue last month > on the hint of Stephan. Yesterday I started to test the currenr RCs. > > First I was disappointed, because the error seems to persist. > So I double checked everything, recreated / recompiled everything > an the error went away. So I thought it was mistake on my side. > I gave Spephan postive feedback. And I'm waiting for the final release > for my production server. > > But when I read your mails, I'm not feeling happy. > I think it's a kink of luck/voodoo/whatever. > > What you must do, I think, is to compile the sieve script with the > exact version running afterwards. > And I think you should the remove the compiled .svbin files > before recreating them again. Don't overwrite them with the compiler. > > I think I'll also dig into this any further today. Please do. I cannot reproduce this so far. Since E.B. still got an obscure debug message about metadata not being up to date, I added debug lines to the remaining places where this could emerge (currently only available from hg). Regards, Stephan. From emailbuilder88 at yahoo.com Thu Mar 12 03:23:20 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Wed, 11 Mar 2015 20:23:20 -0700 Subject: Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1 In-Reply-To: <5500B5BC.1010508@rename-it.nl> Message-ID: <1426130600.57505.YahooMailBasic@web142405.mail.bf1.yahoo.com> > >>> When I compiled and installed this, Sieve scripts were being ignored. Not > >>> sure if it's my own stupid mistake, but when I put v0.4.6 back in place, it > >>> worked fine. No configuration changes, only make install on the different > >>> sources and restart dovecot. > >> Could you show your dovecot -n output? > >> > >> Also, if you enable mail_debug, what sieve-related debug lines are shown? > > OK, I re-tested and it's still ignoring personal scripts (but not > > global ones). No .svbin gets generated, no errors, just nothing. > > However, I do see that Sieve was accessing the user home directory > > because for some reason now it just created a ".pki" directory > > therein, which inside of it has an empty "nssdb" directory. That > > never happened before...? Not a big problem, but I'd prefer not > > to have that there. > > Sieve doesn't do that. I don't think Dovecot does that either, but I am > not sure. Odd. Some lib Sieve uses? These directories do not appear in user home directories unless I install the newest Sieve (and not until a delivery via LMTP happens). No other changes. No other software is currently accessing user home locations at all. > > Re: mail_debug, this relates to another post I made that didn't get any > > replies - can I not override settings such as that (and log_path) from > > a userdb lookup? Hmm, I WAS able to override mail_debug from userdb, > > but not log_path? > > > > Sieve-related mail_debug, then? > > > > This looks like the relevant log info: > > > > dovecot: lmtp(testuser at example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: file storage: Storage path `/vmail/example.com/testuser/sieve' not found > > dovecot: lmtp(testuser at example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: No default script configured for user > > dovecot: lmtp(testuser at example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: User has no personal script > > > > I'll check on 0.4.6 and report if I see anything interesting, > > but I will assume for the moment that since personal scripts > > work in 0.4.6 that this log info won't be there. It is correct that > > there is no "sieve" file or directory in the user's home dir. This > > wasn't a problem in 0.4.6. Is it a requirement? > > Well, since your config says the following: > > sieve = file:~/sieve;active=~/.dovecot.sieve > > It expects a sieve storage directory at ~/sieve (created when > ManageSieve is used to upload a script). > Also, a symbolic link pointing to the active script will be located at > ~/.dovecot.sieve once a script is activated (i.e. through ManageSieve or > doveadm sieve). > > I wonder how this would have worked before with 0.4.6. Is the > ~/.dovecot.sieve a normal script file perhaps (rather than a symlink)? > This would mean that the following config would work (e.g. if you don't > use ManageSieve): > > sieve = file:~/.dovecot.sieve The configuration for that was not of my doing (doesn't that mean it shouldn't have shown up in doveconf -n?). Yes, the .sieve scripts in user home are regular files. Strange 0.4.6 didn't mind this situation, but seems easy to put the configuration right and move on. From emailbuilder88 at yahoo.com Thu Mar 12 03:39:13 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Wed, 11 Mar 2015 20:39:13 -0700 Subject: Why is Sieve trying to re-compile global scripts? In-Reply-To: <5500C98B.70808@rename-it.nl> Message-ID: <1426131553.57427.YahooMailBasic@web142402.mail.bf1.yahoo.com> > Since E.B. still got an obscure debug message about metadata not being > up to date, I added debug lines to the remaining places where this could > emerge (currently only available from hg). Using hg from just now - first line looks like what you want: dovecot: lmtp(testuser at example.com): Debug: U5ZtLH8IAVXydgNAM/SpMA: sieve: file script: Binary reports different script location (`script2.sieve' rather than `/usr/local/var/dovecot/sieve/script2.sieve') dovecot: lmtp(testuser at example.com): Debug: U5ZtLH8IAVXydgNAM/SpMA: sieve: binary up-to-date: script metadata indicates that binary /usr/local/var/dovecot/sieve/script2.svbin is not up-to-date dovecot: lmtp(testuser at example.com): Debug: U5ZtLH8IAVXydgNAM/SpMA: sieve: Script binary /usr/local/var/dovecot/sieve/script2.svbin is not up-to-date From emailbuilder88 at yahoo.com Thu Mar 12 03:46:26 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Wed, 11 Mar 2015 20:46:26 -0700 Subject: Why is Sieve trying to re-compile global scripts? In-Reply-To: <1426131553.57427.YahooMailBasic@web142402.mail.bf1.yahoo.com> Message-ID: <1426131986.90564.YahooMailBasic@web142404.mail.bf1.yahoo.com> > Using hg from just now - first line looks like what you want: > > dovecot: lmtp(testuser at example.com): Debug: U5ZtLH8IAVXydgNAM/SpMA: sieve: file script: Binary reports different script location (`script2.sieve' rather than `/usr/local/var/dovecot/sieve/script2.sieve') > dovecot: lmtp(testuser at example.com): Debug: U5ZtLH8IAVXydgNAM/SpMA: sieve: binary up-to-date: script metadata indicates that binary /usr/local/var/dovecot/sieve/script2.svbin is not up-to-date > dovecot: lmtp(testuser at example.com): Debug: U5ZtLH8IAVXydgNAM/SpMA: sieve: Script binary /usr/local/var/dovecot/sieve/script2.svbin is not up-to-date > Also, it does appear that blowing away *everything* in my global script location (is removing the svbin file the key?) and re-creating it all seems to fix the problem. From emailbuilder88 at yahoo.com Thu Mar 12 03:48:36 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Wed, 11 Mar 2015 20:48:36 -0700 Subject: Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1 In-Reply-To: <1426130600.57505.YahooMailBasic@web142405.mail.bf1.yahoo.com> Message-ID: <1426132116.33319.YahooMailBasic@web142405.mail.bf1.yahoo.com> > > > However, I do see that Sieve was accessing the user home directory > > > because for some reason now it just created a ".pki" directory > > > therein, which inside of it has an empty "nssdb" directory. That > > > never happened before...? Not a big problem, but I'd prefer not > > > to have that there. > > > > Sieve doesn't do that. I don't think Dovecot does that either, but I am > > not sure. > > Odd. Some lib Sieve uses? These directories do not appear in user > home directories unless I install the newest Sieve (and not until > a delivery via LMTP happens). No other changes. No other software > is currently accessing user home locations at all. Using today's hg version, these .pki directories aren't created. I wonder if Timo could shine a light on this. From andreas.kasenides at cs.ucy.ac.cy Thu Mar 12 07:31:50 2015 From: andreas.kasenides at cs.ucy.ac.cy (Andreas Kasenides) Date: Thu, 12 Mar 2015 09:31:50 +0200 Subject: Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1 In-Reply-To: <1998155B-252C-4F6E-8F05-1AB27642BCF2@iki.fi> References: <54FE56AD.7060402@rename-it.nl> <54FEAAAB.1000508@skye.it> <1998155B-252C-4F6E-8F05-1AB27642BCF2@iki.fi> Message-ID: <550140E6.3030404@cs.ucy.ac.cy> On 10/03/15 11:40, Timo Sirainen wrote: > I think it's pure chance that it happens to crash that way. I don't > see how the duplicate checking could have affected this crash. Oh! Do you mean Timo, that Computer Science is non-deterministic! Is there chance, therefore luck, to it? Just kidding! Thanks for all the the good work you and your associates are doing. For us in the education sector and with dwindling money supplies you are a life saver. Thanks. Andreas -- Andreas Kasenides Senior IT Officer Dept. of Computer Science, University of Cyprus Tel: 22892714, Fax: 22892701 (5B4ANK) From tolga at ozses.net Thu Mar 12 07:47:19 2015 From: tolga at ozses.net (Muzaffer Tolga Ozses) Date: Thu, 12 Mar 2015 09:47:19 +0200 Subject: Expunge (?) Message-ID: Hi, I don't know if this is the correct term. Let me elaborate. From time to time, I want to expunge (?) the mail folder so that users' Trash'd mails don't occupy space. So, here are my questions: 1. Is this the correct term? 2. Is it a Good Idea? to do it? Thanks, -- mto From skdovecot at smail.inf.fh-brs.de Thu Mar 12 08:17:36 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 12 Mar 2015 09:17:36 +0100 (CET) Subject: Expunge (?) In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 12 Mar 2015, Muzaffer Tolga Ozses wrote: > I don't know if this is the correct term. Let me elaborate. From time to > time, I want to expunge (?) the mail folder so that users' Trash'd mails > don't occupy space. So, here are my questions: > > 1. Is this the correct term? Well, perhaps. Why you clinge on the term? > 2. Is it a Good Idea? to do it? That depends on your users. Some users of mine use "Trash" as some sort "I do not need the message, but wait, in some days it might become sort of interessting". Therefore check out the expire plugin to remove messages that idle for an amount of days in Trash. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVQFLoHz1H7kL/d9rAQIExgf/eBdQVeJfpDNtRwXO/CeihybV+C09nRq4 jasutSbZU2yDQi/8XhqRdob76frvhhj3kPRX6z8epkIMNHiD7NGKbaXsfe+ACibq Sn81WRBhcUDrPPB3CJmkS0IqVdEuMLs9O0YAeK1WyT+Dh48Xcn+o0uYCjLkYM2By n6+vX1AD0k5BHqlWz157zZuUAlgZJRtlzJnWSIAbCBpKP5nKLmDZ9uZAEqc1Goj9 DBpGs24HL4kGddENk4bhfy/NFFqaVINOiEM/Q/bqZAEb9nA9dQSEWQQFjyGnAuq9 UOI0/Ijo/Hj/8H2fqTm+oC3EK2+C90E0QFCidjrp3b4B2pRQNjPBtQ== =XErt -----END PGP SIGNATURE----- From stephan at rename-it.nl Thu Mar 12 08:29:05 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 12 Mar 2015 09:29:05 +0100 Subject: Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1 In-Reply-To: <1426130600.57505.YahooMailBasic@web142405.mail.bf1.yahoo.com> References: <1426130600.57505.YahooMailBasic@web142405.mail.bf1.yahoo.com> Message-ID: <55014E51.7040208@rename-it.nl> On 3/12/2015 4:23 AM, E.B. wrote: > I'll check on 0.4.6 and report if I see anything interesting, > but I will assume for the moment that since personal scripts > work in 0.4.6 that this log info won't be there. It is correct that > there is no "sieve" file or directory in the user's home dir. This > wasn't a problem in 0.4.6. Is it a requirement? >> Well, since your config says the following: >> >> sieve = file:~/sieve;active=~/.dovecot.sieve >> >> It expects a sieve storage directory at ~/sieve (created when >> ManageSieve is used to upload a script). >> Also, a symbolic link pointing to the active script will be located at >> ~/.dovecot.sieve once a script is activated (i.e. through ManageSieve or >> doveadm sieve). >> >> I wonder how this would have worked before with 0.4.6. Is the >> ~/.dovecot.sieve a normal script file perhaps (rather than a symlink)? >> This would mean that the following config would work (e.g. if you don't >> use ManageSieve): >> >> sieve = file:~/.dovecot.sieve > The configuration for that was not of my doing (doesn't that mean > it shouldn't have shown up in doveconf -n?). Yes, the .sieve scripts > in user home are regular files. Strange 0.4.6 didn't mind this > situation, but seems easy to put the configuration right and move on. Well, even though your config is a bit strange, Pigeonhole should recognize this rather than fail. I will fix this. Regards, Stephan. From tolga at ozses.net Thu Mar 12 08:34:38 2015 From: tolga at ozses.net (Muzaffer Tolga Ozses) Date: Thu, 12 Mar 2015 10:34:38 +0200 Subject: Expunge (?) In-Reply-To: References: Message-ID: On 12 March 2015 at 10:17, Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Thu, 12 Mar 2015, Muzaffer Tolga Ozses wrote: > > I don't know if this is the correct term. Let me elaborate. From time to >> time, I want to expunge (?) the mail folder so that users' Trash'd mails >> don't occupy space. So, here are my questions: >> >> 1. Is this the correct term? >> > > Well, perhaps. Why you clinge on the term? > ?Because if it is, I'm going to search on that keyword.? > > 2. Is it a Good Idea? to do it? >> > > That depends on your users. Some users of mine use "Trash" as some sort "I > do not need the message, but wait, in some days it might become sort of > interessting". Therefore check out the expire plugin to remove messages > that idle for an amount of days in Trash. > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVQFLoHz1H7kL/d9rAQIExgf/eBdQVeJfpDNtRwXO/CeihybV+C09nRq4 > jasutSbZU2yDQi/8XhqRdob76frvhhj3kPRX6z8epkIMNHiD7NGKbaXsfe+ACibq > Sn81WRBhcUDrPPB3CJmkS0IqVdEuMLs9O0YAeK1WyT+Dh48Xcn+o0uYCjLkYM2By > n6+vX1AD0k5BHqlWz157zZuUAlgZJRtlzJnWSIAbCBpKP5nKLmDZ9uZAEqc1Goj9 > DBpGs24HL4kGddENk4bhfy/NFFqaVINOiEM/Q/bqZAEb9nA9dQSEWQQFjyGnAuq9 > UOI0/Ijo/Hj/8H2fqTm+oC3EK2+C90E0QFCidjrp3b4B2pRQNjPBtQ== > =XErt > -----END PGP SIGNATURE----- ?Thanks,? -- mto From HFlor at gmx.de Thu Mar 12 10:37:40 2015 From: HFlor at gmx.de (Hardy Flor) Date: Thu, 12 Mar 2015 11:37:40 +0100 Subject: location of dovecot.rawlog-directory Message-ID: <55016C74.6030208@gmx.de> How is it possible to have the dovecot.rawlog-directory outside the mail_location? A link to another directory is not because it is checked in rawlog.c S_ISDIR to directory. From Olaf.Hopp at kit.edu Thu Mar 12 10:56:18 2015 From: Olaf.Hopp at kit.edu (Olaf Hopp) Date: Thu, 12 Mar 2015 11:56:18 +0100 Subject: Why is Sieve trying to re-compile global scripts? In-Reply-To: <5500C98B.70808@rename-it.nl> References: <1426054665.13781.YahooMailBasic@web142405.mail.bf1.yahoo.com> <5500149E.2010309@kit.edu> <5500C98B.70808@rename-it.nl> Message-ID: <550170D2.60300@kit.edu> On 03/12/2015 12:02 AM, Stephan Bosch wrote: > On 3/11/2015 11:10 AM, Olaf Hopp wrote: >> Please see the thread with subject >> "Sieve permissions issue following update" >> I tested sucessfully a developper issue last month >> on the hint of Stephan. Yesterday I started to test the currenr RCs. >> >> First I was disappointed, because the error seems to persist. >> So I double checked everything, recreated / recompiled everything >> an the error went away. So I thought it was mistake on my side. >> I gave Spephan postive feedback. And I'm waiting for the final release >> for my production server. >> >> But when I read your mails, I'm not feeling happy. >> I think it's a kink of luck/voodoo/whatever. >> >> What you must do, I think, is to compile the sieve script with the >> exact version running afterwards. >> And I think you should the remove the compiled .svbin files >> before recreating them again. Don't overwrite them with the compiler. >> >> I think I'll also dig into this any further today. > > Please do. I cannot reproduce this so far. > > Since E.B. still got an obscure debug message about metadata not being > up to date, I added debug lines to the remaining places where this could > emerge (currently only available from hg). > > Regards, > > Stephan. > Hi, I'm still trying but currently I can not reproduce the bug. But I will keep on hammering on it. Olaf -- Karlsruher Institut f?r Technologie (KIT) ATIS - Abt. Technische Infrastruktur, Fakult?t f?r Informatik Dipl.-Geophys. Olaf Hopp - Leitung IT-Dienste - Am Fasanengarten 5, Geb?ude 50.34, Raum 009 76131 Karlsruhe Telefon: +49 721 608-43973 Fax: +49 721 608-46699 E-Mail: Olaf.Hopp at kit.edu atis.informatik.kit.edu www.kit.edu KIT - Universit?t des Landes Baden-W?rttemberg und nationales Forschungszentrum in der Helmholtz-Gemeinschaft Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5214 bytes Desc: S/MIME Cryptographic Signature URL: From skdovecot at smail.inf.fh-brs.de Thu Mar 12 11:07:14 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 12 Mar 2015 12:07:14 +0100 (CET) Subject: location of dovecot.rawlog-directory In-Reply-To: <55016C74.6030208@gmx.de> References: <55016C74.6030208@gmx.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 12 Mar 2015, Hardy Flor wrote: > How is it possible to have the dovecot.rawlog-directory outside the > mail_location? A link to another directory is not because it is checked in > rawlog.c S_ISDIR to directory. Patch rawlog.c and recompile. :) - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVQFzYnz1H7kL/d9rAQJM5gf/YKr5UP1Z1UlfjCzxgX0sYNUgo4Syd6wD 2lDrlxjkjtWaNgIOY5Kiv2NBJwU579rL+VMqwd1CpkUSmBGbJZGY/YNgDxLfsJkM +EgpmPbFB1+TeZjKygu5F1i9B51MECVRUvWWKI3BA/WQjFMGH7PiK+vSkuUzAi+g XhqvoaKx5doInX5nJibgeZv9W9kF2dLXPFYu79DI6go5laE4VFVSIaeiyVzo+Myl 6DHT1kRV7AHFhTe2wYc2YiIJcJUoiL9x8s5c5nCUKpzjdw5F95Mh5nMcyc1pu8Wz Gg9Po8tr2nNxpimlP/F7krJQKJFPltjZMOi7qJL03C0ILLTKqha35Q== =l/zF -----END PGP SIGNATURE----- From tss at iki.fi Thu Mar 12 12:45:06 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Mar 2015 14:45:06 +0200 Subject: Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1 In-Reply-To: <1426132116.33319.YahooMailBasic@web142405.mail.bf1.yahoo.com> References: <1426132116.33319.YahooMailBasic@web142405.mail.bf1.yahoo.com> Message-ID: <4714A7F2-6B21-4E33-B02D-5891ADAEE13D@iki.fi> On 12 Mar 2015, at 05:48, E.B. wrote: > >>>> However, I do see that Sieve was accessing the user home directory >>>> because for some reason now it just created a ".pki" directory >>>> therein, which inside of it has an empty "nssdb" directory. That >>>> never happened before...? Not a big problem, but I'd prefer not >>>> to have that there. >>> >>> Sieve doesn't do that. I don't think Dovecot does that either, but I am >>> not sure. >> >> Odd. Some lib Sieve uses? These directories do not appear in user >> home directories unless I install the newest Sieve (and not until >> a delivery via LMTP happens). No other changes. No other software >> is currently accessing user home locations at all. > > Using today's hg version, these .pki directories aren't created. > I wonder if Timo could shine a light on this. Probably some PAM plugin or some other strange automation. Dovecot definitely doesn't create those itself. From tss at iki.fi Thu Mar 12 13:34:15 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Mar 2015 15:34:15 +0200 Subject: v2.2.16 release candidate released In-Reply-To: <54FD7EEC.5090500@gmx.de> References: <54FD7EEC.5090500@gmx.de> Message-ID: <67A83113-77C3-4DE6-925F-811A83AC2958@iki.fi> On 09 Mar 2015, at 13:07, Hardy Flor wrote: > > This easy to correct errors is unfortunately still exists. > I once hung my solution, but I can only test on a system. > > > # dovecot --version > 2.2.16.rc1 > # doveadm -f table mailbox status "messages guid" -u flor_hardy "*" > mailbox messages guid > Trash 0 60a619171f66fd54035800004b126f5b > Entw?rfe 0 62a619171f66fd54035800004b126f5b > INBOX 0 61a619171f66fd54035800004b126f5b > > Thanks, committed with some coding style changes: http://hg.dovecot.org/dovecot-2.2/rev/8780ca0fbf22 From tss at iki.fi Thu Mar 12 13:43:47 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Mar 2015 15:43:47 +0200 Subject: Dovecot fails to compile --with-cdb (and attempted fix) In-Reply-To: References: Message-ID: <2320D160-9F86-4A17-8F58-00DA4DD27D60@iki.fi> On 10 Mar 2015, at 16:24, Jeffrey 'jf' Lim wrote: > > On Tue, Mar 10, 2015 at 2:22 AM, Jeffrey 'jf' Lim > wrote: > >> On Tue, Mar 10, 2015 at 2:15 AM, Jeffrey 'jf' Lim >> wrote: >> >>> Dovecot version: 2.2.15 >>> Tinycdb: 0.78 >>> >>> I'm attempting to compile dovecot --with-cdb, but keep getting into a >>> problem with cdb: >>> ======================== >>> libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes >>> -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 >>> -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 >>> -Wl,--as-needed -o .libs/anvil main.o anvil-connection.o anvil-settings.o >>> connect-limit.o penalty.o -Wl,--export-dynamic >>> ../../src/lib-dovecot/.libs/libdovecot.so -ldl -lrt -Wl,-rpath >>> -Wl,/usr/local/lib/dovecot >>> ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to >>> `cdb_read' >>> ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to >>> `cdb_find' >>> ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to >>> `cdb_free' >>> ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to >>> `cdb_init' >>> collect2: ld returned 1 exit status >>> make[3]: *** [anvil] Error 1 >>> make[3]: Leaving directory `/home/jf/dovecot-2.2.15/src/anvil' >>> ========================== >>> >>> After some troubleshooting, I've finally managed to get dovecot to >>> compile, albeit with a hack: >>> >>> --- src/lib-dovecot/Makefile.old 2015-03-09 17:36:15.690179802 +0000 >>> +++ src/lib-dovecot/Makefile 2015-03-09 18:01:43.075719700 +0000 >>> @@ -230,7 +230,7 @@ >>> LIBDOVECOT_STORAGE_DEPS = $(top_builddir)/src/lib-storage/ >>> libdovecot-storage.la $(top_builddir)/src/lib-imap-storage/ >>> libimap-storage.la >>> LIBICONV = >>> LIBOBJS = >>> -LIBS = -lrt >>> +LIBS = -lrt -lcdb >>> LIBTOOL = $(SHELL) $(top_builddir)/libtool >>> LIBWRAP_LIBS = >>> LINKED_STORAGE_LDADD = >>> >>> This isn't the best fix, admittedly (I wish I were more familiar with >>> autotools), but I'll say that the configure process isn't right yet for >>> cdb. Can anybody produce a better fix? Would love to learn more. >>> >>> >> ok, well I guess that "fixed" the compilation, but as I've just found out, >> it does *not* link in libcdb (I've got libcdb in /usr/local/lib) properly! >> Has anybody tried to compile dovecot with cdb support? >> > > For anybody following: sorry, my bad. It turns out I hadn't set CPPFLAGS > and LDFLAGS during the configure to use /usr/local/{include,lib}. When > that's done, this works (in the sense that libdovecot.so is now linked with > libcdb!). > > I'm not so sure that this is the best place to link libcdb in, though, but > for an expedient fix, it works.... Would appreciate somebody more familiar > with the code weighing in on this one. Well, it was never supposed to be linked to anything except the dict binary, allowing it to be used via the dict proxy. Changed: http://hg.dovecot.org/dovecot-2.2/rev/16ff063e3588 From tss at iki.fi Thu Mar 12 13:51:29 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Mar 2015 15:51:29 +0200 Subject: rawlog with -i not worked In-Reply-To: <55001DE0.7000001@gmx.de> References: <54FDDE19.2080401@gmx.de> <55001DE0.7000001@gmx.de> Message-ID: On 11 Mar 2015, at 12:50, Hardy Flor wrote: > > When I change the option 'i' to the letter 'a' is everything as designed. Why is 'i' at this location the problem? Fix & explanation: http://hg.dovecot.org/dovecot-2.2/rev/cd5fc6ff8027 Oops, also http://hg.dovecot.org/dovecot-2.2/rev/aed1e8340ea7 From tss at iki.fi Thu Mar 12 13:55:30 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Mar 2015 15:55:30 +0200 Subject: Fixing broken UTF-8 handling with MySQL driver In-Reply-To: <71AB8C5D-70A5-44B3-81B9-50D2429F9155@zandanel.me> References: <71AB8C5D-70A5-44B3-81B9-50D2429F9155@zandanel.me> Message-ID: On 11 Mar 2015, at 21:31, Felix Zandanel wrote: > > Although my MySQL installation is set up to use UTF-8 as the default charset on every level (config, database, table and field), and the character_set_* runtime variables all yield the value "utf8", it still seems that the mysql client library must be instructed to actually use UTF-8 explicitly. Adding the following statement to driver_mysql_connect() fixes the issue for me: > > mysql_options(db->mysql, MYSQL_SET_CHARSET_NAME, "utf8"); I think you can also add to /etc/my.cnf : [client] default-character-set = utf8 From dlasota at alaska.edu Thu Mar 12 14:07:19 2015 From: dlasota at alaska.edu (Dan LaSota) Date: Thu, 12 Mar 2015 06:07:19 -0800 Subject: doveconf -a Segmentation Fault Message-ID: <-1765799872240539539@unknownmsgid> Getting "Segmentation Fault" When I run doveconf -a Dan LaSota Instructional Designer, UAF eLearning (907) 451-4067 dan.lasota at alaska.edu http://elearning.uaf.edu From h.reindl at thelounge.net Thu Mar 12 14:13:31 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Thu, 12 Mar 2015 15:13:31 +0100 Subject: doveconf -a Segmentation Fault In-Reply-To: <-1765799872240539539@unknownmsgid> References: <-1765799872240539539@unknownmsgid> Message-ID: <55019F0B.60306@thelounge.net> Am 12.03.2015 um 15:07 schrieb Dan LaSota: > Getting "Segmentation Fault" When I run doveconf -a i don't in other words: bad for you but what's the purpose of the information without any debugging like strace? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From HFlor at gmx.de Thu Mar 12 14:18:44 2015 From: HFlor at gmx.de (Hardy Flor) Date: Thu, 12 Mar 2015 15:18:44 +0100 Subject: location of dovecot.rawlog-directory In-Reply-To: References: <55016C74.6030208@gmx.de> Message-ID: <5501A044.3090807@gmx.de> I want running servers, not with each new version have to compile. Am 12.03.2015 um 12:07 schrieb Steffen Kaiser: > Patch rawlog.c and recompile. :) From h.reindl at thelounge.net Thu Mar 12 14:21:22 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Thu, 12 Mar 2015 15:21:22 +0100 Subject: location of dovecot.rawlog-directory In-Reply-To: <5501A044.3090807@gmx.de> References: <55016C74.6030208@gmx.de> <5501A044.3090807@gmx.de> Message-ID: <5501A0E2.6020101@thelounge.net> Am 12.03.2015 um 15:18 schrieb Hardy Flor: > I want running servers, not with each new version have to compile. well, rpm-SPECs allow including of patches if you rely on distribution packages you won't see a update even if upstream would introduce a config option for years > Am 12.03.2015 um 12:07 schrieb Steffen Kaiser: >> Patch rawlog.c and recompile. :) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From jfs.world at gmail.com Thu Mar 12 14:49:15 2015 From: jfs.world at gmail.com (Jeffrey 'jf' Lim) Date: Thu, 12 Mar 2015 22:49:15 +0800 Subject: Dovecot fails to compile --with-cdb (and attempted fix) In-Reply-To: <2320D160-9F86-4A17-8F58-00DA4DD27D60@iki.fi> References: <2320D160-9F86-4A17-8F58-00DA4DD27D60@iki.fi> Message-ID: On Thu, Mar 12, 2015 at 9:43 PM, Timo Sirainen wrote: > On 10 Mar 2015, at 16:24, Jeffrey 'jf' Lim wrote: > > > > On Tue, Mar 10, 2015 at 2:22 AM, Jeffrey 'jf' Lim > > wrote: > > > >> On Tue, Mar 10, 2015 at 2:15 AM, Jeffrey 'jf' Lim > >> wrote: > >> > >>> Dovecot version: 2.2.15 > >>> Tinycdb: 0.78 > >>> > >>> I'm attempting to compile dovecot --with-cdb, but keep getting into a > >>> problem with cdb: > >>> ======================== > >>> libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes > >>> -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 > >>> -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 > >>> -Wl,--as-needed -o .libs/anvil main.o anvil-connection.o > anvil-settings.o > >>> connect-limit.o penalty.o -Wl,--export-dynamic > >>> ../../src/lib-dovecot/.libs/libdovecot.so -ldl -lrt -Wl,-rpath > >>> -Wl,/usr/local/lib/dovecot > >>> ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to > >>> `cdb_read' > >>> ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to > >>> `cdb_find' > >>> ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to > >>> `cdb_free' > >>> ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to > >>> `cdb_init' > >>> collect2: ld returned 1 exit status > >>> make[3]: *** [anvil] Error 1 > >>> make[3]: Leaving directory `/home/jf/dovecot-2.2.15/src/anvil' > >>> ========================== > >>> > >>> After some troubleshooting, I've finally managed to get dovecot to > >>> compile, albeit with a hack: > >>> > >>> --- src/lib-dovecot/Makefile.old 2015-03-09 17:36:15.690179802 +0000 > >>> +++ src/lib-dovecot/Makefile 2015-03-09 18:01:43.075719700 +0000 > >>> @@ -230,7 +230,7 @@ > >>> LIBDOVECOT_STORAGE_DEPS = $(top_builddir)/src/lib-storage/ > >>> libdovecot-storage.la $(top_builddir)/src/lib-imap-storage/ > >>> libimap-storage.la > >>> LIBICONV = > >>> LIBOBJS = > >>> -LIBS = -lrt > >>> +LIBS = -lrt -lcdb > >>> LIBTOOL = $(SHELL) $(top_builddir)/libtool > >>> LIBWRAP_LIBS = > >>> LINKED_STORAGE_LDADD = > >>> > >>> This isn't the best fix, admittedly (I wish I were more familiar with > >>> autotools), but I'll say that the configure process isn't right yet for > >>> cdb. Can anybody produce a better fix? Would love to learn more. > >>> > >>> > >> ok, well I guess that "fixed" the compilation, but as I've just found > out, > >> it does *not* link in libcdb (I've got libcdb in /usr/local/lib) > properly! > >> Has anybody tried to compile dovecot with cdb support? > >> > > > > For anybody following: sorry, my bad. It turns out I hadn't set CPPFLAGS > > and LDFLAGS during the configure to use /usr/local/{include,lib}. When > > that's done, this works (in the sense that libdovecot.so is now linked > with > > libcdb!). > > > > I'm not so sure that this is the best place to link libcdb in, though, > but > > for an expedient fix, it works.... Would appreciate somebody more > familiar > > with the code weighing in on this one. > > Well, it was never supposed to be linked to anything except the dict > binary, allowing it to be used via the dict proxy. Changed: > http://hg.dovecot.org/dovecot-2.2/rev/16ff063e3588 > > Thanks, Timo. Is this the only patch that needs to go in for the current version (2.2.15)? I've applied this to 2.2.15, and configured and compiled as before: CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib ./configure --with-cdb make and I still get the same errors. -jf From HFlor at gmx.de Thu Mar 12 14:54:45 2015 From: HFlor at gmx.de (Hardy Flor) Date: Thu, 12 Mar 2015 15:54:45 +0100 Subject: location of dovecot.rawlog-directory In-Reply-To: <55016C74.6030208@gmx.de> References: <55016C74.6030208@gmx.de> Message-ID: <5501A8B5.7080701@gmx.de> When I set in the home-variable in the userdb-section on my request directory and not used the home-dirctory in the configuration, have what can be expected for other effects? From tss at iki.fi Thu Mar 12 15:17:14 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Mar 2015 17:17:14 +0200 Subject: Dovecot fails to compile --with-cdb (and attempted fix) In-Reply-To: References: <2320D160-9F86-4A17-8F58-00DA4DD27D60@iki.fi> Message-ID: On 12 Mar 2015, at 16:49, Jeffrey 'jf' Lim wrote: > > On Thu, Mar 12, 2015 at 9:43 PM, Timo Sirainen wrote: > >> >> Well, it was never supposed to be linked to anything except the dict >> binary, allowing it to be used via the dict proxy. Changed: >> http://hg.dovecot.org/dovecot-2.2/rev/16ff063e3588 >> >> > Thanks, Timo. Is this the only patch that needs to go in for the current > version (2.2.15)? I've applied this to 2.2.15, and configured and compiled > as before: > > CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib ./configure > --with-cdb > make > > and I still get the same errors. Did you run autogen.sh again before configure? From tss at iki.fi Thu Mar 12 17:30:34 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Mar 2015 19:30:34 +0200 Subject: v2.2.16 released Message-ID: <0DC5FD73-B4A7-4811-BD7F-7B053BAF47E8@iki.fi> http://dovecot.org/releases/2.2/dovecot-2.2.16.tar.gz http://dovecot.org/releases/2.2/dovecot-2.2.16.tar.gz.sig A few fixes and some imapc improvements since the release candidate. * dbox: Resyncing (e.g. doveadm force-resync) no longer deletes dovecot.index.cache file. The cache file was rarely the problem so this just caused unnecessary slowness. * Mailbox name limits changed during mailbox creation: Each part of a hierarchical name (e.g. "x" or "y" in "x/y") can now be up to 255 chars long (instead of 200). This also reduces the max number of hierarchical levels to 16 (instead of 20) to keep the maximum name length 4096 (a common PATH_MAX limit). The 255 char limit is hopefully large enough for migrations from all existing systems. It's also the limit on many filesystems. + director: Added director_consistent_hashing setting to enable consistent hashing (instead of the mostly-random MD5 hashing). This causes fewer user moves between backends when backend counts are changed, which may improve performance (mainly due to caching). + director: Added support for "tags", which allows one director ring to serve multiple backend clusters with different sets of users. + LMTP server: Added lmtp_user_concurrency_limit setting to limit how many LMTP deliveries can be done concurrently for a single user. + LMTP server: Added support for STARTTLS command. + If logging data is generated faster than it can be written, log a warning about it and show information about it in log process's process title in ps output. Also don't allow a single service to flood too long at the cost of delaying other services' logging. + stats: Added support for getting global statistics. + stats: Use the same session IDs as the rest of Dovecot. + stats: Plugins can now create their own statistics fields + doveadm server: Non-mail related commands can now also be used via doveadm server (TCP socket). + doveadm proxying: passdb lookup can now override doveadm_port and change the username. + doveadm: Search query supports now "oldestonly" parameter to stop immediately on the first non-match. This can be used to optimize: doveadm expunge mailbox Trash savedbefore 30d oldestonly + doveadm: Added "save" command to directly save mails to specified mailbox (bypassing Sieve). + doveadm fetch: Added body.snippet field, which returns the first 100 chars of a message without whitespace or HTML tags. The result is stored into dovecot.index.cache, so it can be fetched efficiently. + dsync: Added -t parameter to sync only mails newer than the given received-timestamp. + dsync: Added -F [-] parameter to sync only mails with[out] the given flag/keyword. + dsync: Added -a parameter to specify the virtual mailbox containing user's all mails. If this mailbox is already found to contain the wanted mail (by its GUID), the message is copied from there instead of being re-saved. (This isn't efficient enough yet for incremental replication.) + dsync: -m parameter can now specify \Special-use names for mailboxes. + imapc: Added imapc_features=gmail-migration to help migrations from GMail. See http://wiki2.dovecot.org/Migration/Gmail + imapc: Added imapc_features=search to support IMAP SEARCH command. (Currently requires ESEARCH support from remote server.) + expire plugin: Added expire_cache=yes setting to cache most of the database lookups in dovecot index files. + quota: If overquota-flag in userdb doesn't match the current quota usage, execute a configured script. + redis dict: Added support for expiring keys (:expire_secs=n) and specifying the database number (:db=n) - auth: Don't crash if master user login is attempted without any configured master=yes passdbs - Parsing UTF-8 text for mails could have caused broken results sometimes if buffering was split in the middle of a UTF-8 character. This affected at least searching messages. - String sanitization for some logged output wasn't done properly: UTF-8 text could have been truncated wrongly or the truncation may not have happened at all. - fts-lucene: Lookups from virtual mailbox consisting of over 32 physical mailboxes could have caused crashes. From HFlor at gmx.de Thu Mar 12 18:50:40 2015 From: HFlor at gmx.de (Hardy Flor) Date: Thu, 12 Mar 2015 19:50:40 +0100 Subject: v2.2.16 released In-Reply-To: <0DC5FD73-B4A7-4811-BD7F-7B053BAF47E8@iki.fi> References: <0DC5FD73-B4A7-4811-BD7F-7B053BAF47E8@iki.fi> Message-ID: <5501E000.2000206@gmx.de> For this no man-page available. Am 12.03.2015 um 18:30 schrieb Timo Sirainen: > + doveadm: Added "save" command to directly save mails to specified > mailbox (bypassing Sieve). From h.reindl at thelounge.net Thu Mar 12 19:09:34 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Thu, 12 Mar 2015 20:09:34 +0100 Subject: v2.2.16 released In-Reply-To: <0DC5FD73-B4A7-4811-BD7F-7B053BAF47E8@iki.fi> References: <0DC5FD73-B4A7-4811-BD7F-7B053BAF47E8@iki.fi> Message-ID: <5501E46E.5030107@thelounge.net> /usr/lib64/dovecot/stats/libstats_mail.so why in the world a new sub-directory containing just one so-file enforcing pakcage buildsers to change SPEC files? Am 12.03.2015 um 18:30 schrieb Timo Sirainen: > http://dovecot.org/releases/2.2/dovecot-2.2.16.tar.gz > http://dovecot.org/releases/2.2/dovecot-2.2.16.tar.gz.sig > > A few fixes and some imapc improvements since the release candidate. > > * dbox: Resyncing (e.g. doveadm force-resync) no longer deletes > dovecot.index.cache file. The cache file was rarely the problem > so this just caused unnecessary slowness. > * Mailbox name limits changed during mailbox creation: Each part of > a hierarchical name (e.g. "x" or "y" in "x/y") can now be up to 255 > chars long (instead of 200). This also reduces the max number of > hierarchical levels to 16 (instead of 20) to keep the maximum name > length 4096 (a common PATH_MAX limit). The 255 char limit is > hopefully large enough for migrations from all existing systems. > It's also the limit on many filesystems. > > + director: Added director_consistent_hashing setting to enable > consistent hashing (instead of the mostly-random MD5 hashing). > This causes fewer user moves between backends when backend counts > are changed, which may improve performance (mainly due to caching). > + director: Added support for "tags", which allows one director ring > to serve multiple backend clusters with different sets of users. > + LMTP server: Added lmtp_user_concurrency_limit setting to limit how > many LMTP deliveries can be done concurrently for a single user. > + LMTP server: Added support for STARTTLS command. > + If logging data is generated faster than it can be written, log a > warning about it and show information about it in log process's > process title in ps output. Also don't allow a single service to > flood too long at the cost of delaying other services' logging. > + stats: Added support for getting global statistics. > + stats: Use the same session IDs as the rest of Dovecot. > + stats: Plugins can now create their own statistics fields > + doveadm server: Non-mail related commands can now also be used > via doveadm server (TCP socket). > + doveadm proxying: passdb lookup can now override doveadm_port and > change the username. > + doveadm: Search query supports now "oldestonly" parameter to stop > immediately on the first non-match. This can be used to optimize: > doveadm expunge mailbox Trash savedbefore 30d oldestonly > + doveadm: Added "save" command to directly save mails to specified > mailbox (bypassing Sieve). > + doveadm fetch: Added body.snippet field, which returns the first > 100 chars of a message without whitespace or HTML tags. The result > is stored into dovecot.index.cache, so it can be fetched efficiently. > + dsync: Added -t parameter to sync only mails newer than > the given received-timestamp. > + dsync: Added -F [-] parameter to sync only mails with[out] the > given flag/keyword. > + dsync: Added -a parameter to specify the virtual mailbox > containing user's all mails. If this mailbox is already found to > contain the wanted mail (by its GUID), the message is copied from > there instead of being re-saved. (This isn't efficient enough yet > for incremental replication.) > + dsync: -m parameter can now specify \Special-use names for mailboxes. > + imapc: Added imapc_features=gmail-migration to help migrations from > GMail. See http://wiki2.dovecot.org/Migration/Gmail > + imapc: Added imapc_features=search to support IMAP SEARCH command. > (Currently requires ESEARCH support from remote server.) > + expire plugin: Added expire_cache=yes setting to cache most of the > database lookups in dovecot index files. > + quota: If overquota-flag in userdb doesn't match the current quota > usage, execute a configured script. > + redis dict: Added support for expiring keys (:expire_secs=n) and > specifying the database number (:db=n) > - auth: Don't crash if master user login is attempted without > any configured master=yes passdbs > - Parsing UTF-8 text for mails could have caused broken results > sometimes if buffering was split in the middle of a UTF-8 character. > This affected at least searching messages. > - String sanitization for some logged output wasn't done properly: > UTF-8 text could have been truncated wrongly or the truncation may > not have happened at all. > - fts-lucene: Lookups from virtual mailbox consisting of over 32 > physical mailboxes could have caused crashes. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From chayes at afo.net Thu Mar 12 20:35:48 2015 From: chayes at afo.net (Cliff Hayes) Date: Thu, 12 Mar 2015 15:35:48 -0500 Subject: How to detect out-of-sync condition Message-ID: <5501F8A4.6080306@afo.net> Hello list, I recently had a user whose mailbox had gone out of sync. Webmail and client inbox totals had become vastly different and response times had become unacceptable. A doveadm force-resync fixed the problem nicely. This will no doubt happen again as we grow the service and I would like to have some type of indication at the server level so I can run a resync on a per-user basis as required. However, I don't know what to look for. I did not see any signs of the problem on the server - but perhaps I did not know what to look for. The only other alternative is for me to do a global resync in the cron.daily but my fear is that this won't scale well for thousands of users. Please advise. Thanks in advance. From laeeth at laeeth.com Thu Mar 12 20:55:44 2015 From: laeeth at laeeth.com (Laeeth Isharc) Date: Thu, 12 Mar 2015 20:55:44 +0000 Subject: indexer-worker panics with latest mercurial Message-ID: <5501FD50.8060702@laeeth.com> Hi. I am seeing the following in my logs. Happy to downgrade to an earlier version if you think this might be the problem, but dovecot.org is extremely slow (and has been for months) so I found it easier just to pull the latest from mercurial. Thanks. Laeeth Mar 12 20:48:39 indexer: Error: Indexer worker disconnected, discarding 1 reques ts for laeeth at laeeth.com Mar 12 20:48:39 indexer-worker(laeeth at laeeth.com): Fatal: master: service(indexe r-worker): child 24003 killed with signal 6 (core dumps disabled) Mar 12 20:49:01 dsync-local(laeeth at laeeth.com): Error: Couldn't lock /home/mail/ laeeth_laeeth_com/.dovecot-sync.lock: Timed out after 30 seconds Mar 12 20:49:16 indexer-worker(laeeth at laeeth.com): Error: fts_tika: PUT http://l ocalhost:9997/tika/ failed: 500 Server Error Mar 12 20:49:17 indexer-worker(rosie at kaleidicassociates.com): Warning: I/O leak: 0x7fc47d60fcf0 (line 127, fd 25) Mar 12 20:49:17 indexer-worker(rosie at kaleidicassociates.com): Panic: file ioloop .c: line 39 (io_add_file): assertion failed: (callback != NULL) Mar 12 20:49:17 indexer-worker(rosie at kaleidicassociates.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x76c40) [0x7fc47d63fc40] -> /usr/lib64/dovecot/libdovecot.so.0(+0x76d1e) [0x7fc47d63fd1e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7fc47d5f02e5] -> /usr/lib64/dovecot/libdovecot.so.0(+0x87bd0) [0x7fc47d650bd0] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_move_io+0x3e) [0x7fc47d65183e] -> /usr/lib64/dovecot/libdovecot.so.0(connection_switch_ioloop+0x14) [0x7fc47d63db14] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_switch_ioloop+0x28) [0x7fc47d614448] -> /usr/lib64/dovecot/libdovecot.so.0(+0x43876) [0x7fc47d60c876] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(solr_connection_post_more+0x29) [0x7fc47c733159] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x3637) [0x7fc47c72f637] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0x6d83) [0x7fc47cf5dd83] -> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_build_mail+0x614) [0x7fc47cf5e3c4] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xc14e) [0x7fc47cf6314e] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_precache+0x19) [0x7fc47d8ef429] -> dovecot/indexer-worker() [0x40237c] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7fc47d65155c] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb) [0x7fc47d65249b] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7fc47d6515c9] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fc47d651648] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fc47d5f5703] -> dovecot/indexer-worker(main+0xcb) [0x401e7b] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x32c6021d65] -> dovecot/indexer-worker() [0x401f25] Mar 12 20:49:17 indexer: Error: Indexer worker disconnected, discarding 15 reque sts for rosie at kaleidicassociates.com Mar 12 20:49:17 indexer-worker(rosie at kaleidicassociates.com): Fatal: master: service(indexer-worker): child 24022 killed with signal 6 (core dumps disabled) Mar 12 20:49:17 indexer-worker(laeeth at kaleidicassociates.com): Warning: I/O leak: 0x7f22c4d4dcf0 (line 127, fd 23) Mar 12 20:49:17 indexer-worker(laeeth at kaleidicassociates.com): Panic: file ioloop.c: line 829 (io_loop_move_io): assertion failed: ((old_io->condition & IO_NOTIFY) == 0) Mar 12 20:49:17 indexer-worker(laeeth at kaleidicassociates.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x76c40) [0x7f22c4d7dc40] -> /usr/lib64/dovecot/libdovecot.so.0(+0x76d1e) [0x7f22c4d7dd1e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f22c4d2e2e5] -> /usr/lib64/dovecot/libdovecot.so.0(+0x888ba) [0x7f22c4d8f8ba] -> /usr/lib64/dovecot/libdovecot.so.0(connection_switch_ioloop+0x14) [0x7f22c4d7bb14] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_switch_ioloop+0x28) [0x7f22c4d52448] -> /usr/lib64/dovecot/libdovecot.so.0(+0x43876) [0x7f22c4d4a876] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(solr_connection_post_more+0x29) [0x7f22c3e71159] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x3637) [0x7f22c3e6d637] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0x6d83) [0x7f22c469bd83] -> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_build_mail+0x123) [0x7f22c469bed3] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xc14e) [0x7f22c46a114e] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_precache+0x19) [0x7f22c502d429] -> dovecot/indexer-worker() [0x40237c] -> /usr/lib64/dovecot/libdovecot.so.0(io _loop_call_io+0x4c) [0x7f22c4d8f55c] -> /usr/lib64/dovecot/libdovecot.s o.0(io_loop_handler_run_internal+0xbb) [0x7f22c4d9049b] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f22c4d8f5c9] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f22c4d8f648] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f22c4d33703] -> dovecot/indexer-worker(main+0xcb) [0x401e7b] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x32c6021d65] -> dovecot/indexer-worker() [0x401f25] Mar 12 20:49:17 indexer: Error: Indexer worker disconnected, discarding 6 requests for laeeth at kaleidicassociates.com Mar 12 20:49:17 indexer-worker(laeeth at kaleidicassociates.com): Fatal: master: service(indexer-worker): child 24020 killed with signal 6 (core dumps disabled) Mar 12 20:49:26 indexer-worker(laeeth at laeeth.com): Warning: I/O leak: 0x7f0d5aa00cf0 (line 127, fd 23) Mar 12 20:49:26 indexer-worker(laeeth at laeeth.com): Panic: file ioloop-iolist.c: line 22: unreached Mar 12 20:49:26 indexer-worker(laeeth at laeeth.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x76c40) [0x7f0d5aa30c40] -> /usr/lib64/dovecot/libdovecot.so.0(+0x76d1e) [0x7f0d5aa30d1e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0d5a9e12e5] -> /usr/lib64/dovecot/libdovecot.so.0(+0x88a0d) [0x7f0d5aa42a0d] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handle_add+0x3b) [0x7f0d5aa4319b] -> /usr/lib64/dovecot/libdovecot.so.0(+0x87b61) [0x7f0d5aa41b61] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_move_io+0x3e) [0x7f0d5aa4283e] -> /usr/lib64/dovecot/libdovecot.so.0(connection_switch_ioloop+0x14) [0x7f0d5aa2eb14] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_switch_ioloop+0x28) [0x7f0d5aa0 From laeeth at laeeth.com Thu Mar 12 21:04:20 2015 From: laeeth at laeeth.com (Laeeth Isharc) Date: Thu, 12 Mar 2015 21:04:20 +0000 Subject: looks like a repeat of this problem Message-ID: <5501FF54.3060503@laeeth.com> http://permalink.gmane.org/gmane.mail.imap.dovecot/80062 From user+dovecot at localhost.localdomain.org Thu Mar 12 22:36:10 2015 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Thu, 12 Mar 2015 22:36:10 +0000 Subject: doveconf -a Segmentation Fault In-Reply-To: <-1765799872240539539@unknownmsgid> References: <-1765799872240539539@unknownmsgid> Message-ID: <550214DA.1040003@localhost.localdomain.org> On 03/12/2015 02:07 PM, Dan LaSota wrote: > Getting "Segmentation Fault" When I run doveconf -a Have a look at http://dovecot.org/bugreport.html and try again. Regards, Pascal -- The trapper recommends today: f007ba11.1507102 at localdomain.org From stephan at rename-it.nl Thu Mar 12 22:53:22 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 12 Mar 2015 23:53:22 +0100 Subject: Why is Sieve trying to re-compile global scripts? In-Reply-To: <550170D2.60300@kit.edu> References: <1426054665.13781.YahooMailBasic@web142405.mail.bf1.yahoo.com> <5500149E.2010309@kit.edu> <5500C98B.70808@rename-it.nl> <550170D2.60300@kit.edu> Message-ID: <550218E2.6050308@rename-it.nl> On 3/12/2015 11:56 AM, Olaf Hopp wrote: > On 03/12/2015 12:02 AM, Stephan Bosch wrote: >> On 3/11/2015 11:10 AM, Olaf Hopp wrote: >>> Please see the thread with subject >>> "Sieve permissions issue following update" >>> I tested sucessfully a developper issue last month >>> on the hint of Stephan. Yesterday I started to test the currenr RCs. >>> >>> First I was disappointed, because the error seems to persist. >>> So I double checked everything, recreated / recompiled everything >>> an the error went away. So I thought it was mistake on my side. >>> I gave Spephan postive feedback. And I'm waiting for the final release >>> for my production server. >>> >>> But when I read your mails, I'm not feeling happy. >>> I think it's a kink of luck/voodoo/whatever. >>> >>> What you must do, I think, is to compile the sieve script with the >>> exact version running afterwards. >>> And I think you should the remove the compiled .svbin files >>> before recreating them again. Don't overwrite them with the compiler. >>> >>> I think I'll also dig into this any further today. >> >> Please do. I cannot reproduce this so far. >> >> Since E.B. still got an obscure debug message about metadata not being >> up to date, I added debug lines to the remaining places where this could >> emerge (currently only available from hg). >> >> Regards, >> >> Stephan. >> > > Hi, > I'm still trying but currently I can not reproduce the bug. > But I will keep on hammering on it. Looks like I found the bug. Will need some time to fix this properly. Regards, Stephan. From dlasota at alaska.edu Thu Mar 12 23:04:33 2015 From: dlasota at alaska.edu (Dan LaSota) Date: Thu, 12 Mar 2015 18:04:33 -0500 Subject: doveconf -a Segmentation Fault In-Reply-To: <550214DA.1040003@localhost.localdomain.org> References: <-1765799872240539539@unknownmsgid> <550214DA.1040003@localhost.localdomain.org> Message-ID: <-6251548836583427568@unknownmsgid> I will try. thanks. Dan LaSota Instructional Designer, UAF eLearning (907) 451-4067 dan.lasota at alaska.edu http://elearning.uaf.edu > On Mar 12, 2015, at 5:34 PM, Pascal Volk wrote: > >> On 03/12/2015 02:07 PM, Dan LaSota wrote: >> Getting "Segmentation Fault" When I run doveconf -a > > Have a look at http://dovecot.org/bugreport.html and try again. > > > Regards, > Pascal > -- > The trapper recommends today: f007ba11.1507102 at localdomain.org From jtam.home at gmail.com Thu Mar 12 23:35:14 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Thu, 12 Mar 2015 16:35:14 -0700 (PDT) Subject: How to detect out-of-sync condition In-Reply-To: References: Message-ID: Cliff Hayes writes: > I recently had a user whose mailbox had gone out of sync. > Webmail and client inbox totals had become vastly different and response > times had become unacceptable. > A doveadm force-resync fixed the problem nicely. > > This will no doubt happen again as we grow the service and I would like > to have some type of indication at the server level so I can run a > resync on a per-user basis as required. Perhaps what you ought to look for is how are they getting out of sync in the first place. Assuming your webmail is a IMAP/POP client, I would look at ways that your user's INBOX can be modified out of Dovecot's sight. For example, does your LDA use dovecot's lda/lmtp, or do your users run mail clients that directly manipulate mailbox files, or do you use procmail/mail filters? By closing off other avenues other than dovecot imap/pop/lda/etc., the indices will stay sync'd. If you really have to do manual resyncing, comparing index/mailbox mtimes might be one way to do it. Joseph Tam From chayes at afo.net Fri Mar 13 05:30:58 2015 From: chayes at afo.net (Cliff Hayes) Date: Fri, 13 Mar 2015 00:30:58 -0500 Subject: How to detect out-of-sync condition Message-ID: <55027612.4070002@afo.net> Thanks. I use dovecot's lda and dovecot's sieve filter. So it looks like I need to compare the index/mailbox mtimes as you suggest. What am I looking for? I see that the indexes are updated when I run the resync. I checked my mailbox (that was not resynced) and noticed that dovecot.index last update was 16 days ago. So am I resyncing if the gap is over x days? If so, is there a way to resync just those mailboxes with a doveadm command or do I have to write a program look for that condition and run doveadm when matched? Cliff Hayes writes: > I recently had a user whose mailbox had gone out of sync. > Webmail and client inbox totals had become vastly different and response > times had become unacceptable. > A doveadm force-resync fixed the problem nicely. > > This will no doubt happen again as we grow the service and I would like to have some type of indication at the server level so I can run a resync on a per-user basis as required. Perhaps what you ought to look for is how are they getting out of sync in the first place. Assuming your webmail is a IMAP/POP client, I would look at ways that your user's INBOX can be modified out of Dovecot's sight. For example, does your LDA use dovecot's lda/lmtp, or do your users run mail clients that directly manipulate mailbox files, or do you use procmail/mail filters? By closing off other avenues other than dovecot imap/pop/lda/etc., the indices will stay sync'd. If you really have to do manual resyncing, comparing index/mailbox mtimes might be one way to do it. Joseph Tam From jernej.porenta at arnes.si Fri Mar 13 07:03:34 2015 From: jernej.porenta at arnes.si (Jernej Porenta) Date: Fri, 13 Mar 2015 08:03:34 +0100 Subject: doveadm sync subscribed folders only Message-ID: <55028BC6.5010008@arnes.si> Heya, is there a simple way to use "doveadm sync" for subscribed folders only? Since the migration from UW-IMAP, our users can actually subscribe to non-mbox files and now, when we are pushing new servers out, we are migrating our mboxes to dbox format and we would like to migrate only subscribed folders. I know, there is a -m switch which can be used to transfer subscribed folders for each user, but it makes it quite unusable with escaping and all other problems with such volume, so just wondering if there is a simple way to migrate subscribed folders only. Thank you in advance, cheers, Jernej From skdovecot at smail.inf.fh-brs.de Fri Mar 13 08:05:19 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 13 Mar 2015 09:05:19 +0100 (CET) Subject: How to detect out-of-sync condition In-Reply-To: <5501F8A4.6080306@afo.net> References: <5501F8A4.6080306@afo.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 12 Mar 2015, Cliff Hayes wrote: > I recently had a user whose mailbox had gone out of sync. There are no log entries about broken sync or something something like that? - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVQKaP3z1H7kL/d9rAQKkCQf/VwbytrDRGDJT8Ado8J5L/melZ751jqME pmRQ51McVo80SFvpNuQe93tLlJcGwltgr4He2scxBMelzBlfmQ9Vqrl0EH9R+RpE UxPGHbhWa6Dm23gRMgeYiLx9YlZXnnKtnpUL4MriMwIrrTGaN1hq7yX4Ot2k+ImM vHI3aFSTg35lklKfrzfDwKaQKQMMVf8fz4fZJICL2Vas2df5XaSEyQY1wM/PUqoj Bev0POPfvKhba8adY5yAM9fWpBmStwspIwKR1ZWqflOlz41WfOKeQBWkMqXK7hpV LpF5dY1omVE0BORE5+4T9hY03RXtBcepVczfS4MhX4WrNkJ3TMrjvw== =gjO3 -----END PGP SIGNATURE----- From jfs.world at gmail.com Fri Mar 13 08:18:53 2015 From: jfs.world at gmail.com (Jeffrey 'jf' Lim) Date: Fri, 13 Mar 2015 16:18:53 +0800 Subject: Dovecot fails to compile --with-cdb (and attempted fix) In-Reply-To: References: <2320D160-9F86-4A17-8F58-00DA4DD27D60@iki.fi> Message-ID: On Thu, Mar 12, 2015 at 11:17 PM, Timo Sirainen wrote: > On 12 Mar 2015, at 16:49, Jeffrey 'jf' Lim wrote: > > > > On Thu, Mar 12, 2015 at 9:43 PM, Timo Sirainen wrote: > > > >> > >> Well, it was never supposed to be linked to anything except the dict > >> binary, allowing it to be used via the dict proxy. Changed: > >> http://hg.dovecot.org/dovecot-2.2/rev/16ff063e3588 > >> > >> > > Thanks, Timo. Is this the only patch that needs to go in for the current > > version (2.2.15)? I've applied this to 2.2.15, and configured and > compiled > > as before: > > > > CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib ./configure > > --with-cdb > > make > > > > and I still get the same errors. > > Did you run autogen.sh again before configure? > > Sorry, but I dont see an autogen.sh anywhere in the tarball - not for 2.2.15, nor for the newly released 2.2.16. -jf From tss at iki.fi Fri Mar 13 10:23:41 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 13 Mar 2015 12:23:41 +0200 Subject: v2.2.16 released In-Reply-To: <5501E46E.5030107@thelounge.net> References: <0DC5FD73-B4A7-4811-BD7F-7B053BAF47E8@iki.fi> <5501E46E.5030107@thelounge.net> Message-ID: <2C52D958-1701-4FBF-B332-DC81CCD01C34@iki.fi> On 12 Mar 2015, at 21:09, Reindl Harald wrote: > > /usr/lib64/dovecot/stats/libstats_mail.so > > why in the world a new sub-directory containing just one so-file enforcing pakcage buildsers to change SPEC files? So that external plugins can add more files in there and extend the available statistics. From h.reindl at thelounge.net Fri Mar 13 10:34:01 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 13 Mar 2015 11:34:01 +0100 Subject: v2.2.16 released In-Reply-To: <2C52D958-1701-4FBF-B332-DC81CCD01C34@iki.fi> References: <0DC5FD73-B4A7-4811-BD7F-7B053BAF47E8@iki.fi> <5501E46E.5030107@thelounge.net> <2C52D958-1701-4FBF-B332-DC81CCD01C34@iki.fi> Message-ID: <5502BD19.3060304@thelounge.net> Am 13.03.2015 um 11:23 schrieb Timo Sirainen: > On 12 Mar 2015, at 21:09, Reindl Harald wrote: >> >> /usr/lib64/dovecot/stats/libstats_mail.so >> >> why in the world a new sub-directory containing just one so-file enforcing pakcage buildsers to change SPEC files? > > So that external plugins can add more files in there and extend the available statistics but they can also go to /usr/lib64/dovecot/ http://fedoraproject.org/wiki/Packaging:Guidelines#Beware_of_Rpath hence cat /etc/ld.so.conf.d/dovecot-x86_64.conf /usr/lib64/dovecot i just "rm -rf" the folder and other stuff for private builds in environments where dovecot is running only as proxy rm -rf %{buildroot}%{_sysconfdir}/%{name}/README \ %{buildroot}%{_docdir}/%{name}-%{version} \ %{buildroot}%{_includedir}/%{name}/ \ %{buildroot}%{_mandir}/man1/ \ %{buildroot}%{_mandir}/man7/ \ %{buildroot}%{_datarootdir}/aclocal/ \ %{buildroot}%{_bindir}/dsync \ %{buildroot}%{_libdir}/%{name}/*_plugin.so \ %{buildroot}%{_libdir}/%{name}/doveadm/*_plugin.so \ %{buildroot}%{_libdir}/%{name}/lib%{name}-lda.so \ %{buildroot}%{_libdir}/%{name}/lib%{name}-lda.so.0 \ %{buildroot}%{_libdir}/%{name}/lib%{name}-lda.so.0.0.0 \ %{buildroot}%{_libdir}/%{name}/lib%{name}-compression.so \ %{buildroot}%{_libdir}/%{name}/lib%{name}-compression.so.0 \ %{buildroot}%{_libdir}/%{name}/lib%{name}-compression.so.0.0.0 \ %{buildroot}%{_libdir}/%{name}/stats \ %{buildroot}%{_libexecdir}/%{name}/%{name}-lda \ %{buildroot}%{_libexecdir}/%{name}/gdbhelper \ %{buildroot}%{_libexecdir}/%{name}/quota-status \ %{buildroot}%{_libexecdir}/%{name}/deliver \ %{buildroot}%{_libexecdir}/%{name}/lmtp -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From james at lottspot.com Fri Mar 13 12:48:19 2015 From: james at lottspot.com (james at lottspot.com) Date: Fri, 13 Mar 2015 05:48:19 -0700 Subject: Configuring Shared Mailbox Dictionaries Message-ID: Hello, list! I am in the midst of configuring a new Dovecot IMAP server, and I'm preparing to setup shard mailboxes as part of the ocnfiguration. In order to get these setup the way I want, I've figured out that I'm going to have to setup an acl_shared_dict. This is where things start to get confusing for me, as I have had difficulty finding thorough explanations of dictionary configuration. My primary concern is that all my user details are stored in a sqlite database in which the '%n' and '%d' components of username/email addresses are stored in separate columns. The SQL dictionary example in the Shared Mailboxes document [http://wiki2.dovecot.org/SharedMailboxes/Shared] assumes that the username is stored as a singular '%n@%d' key. In my authdb lookups, I use sqlite syntax to concatenate the '%n' and '%d' columns around a '@' character to find the user. Is it possible to implement an acl_shared_dict in some similar fashion? My second question, which is completely unrelated and more out of morbid curiosity than anything (although it could affect my configuration decisions) is that I noticed every dictionary which shows up in the documentation has a "pattern" field. In the case of the acl_shared_dict example, the pattern is 'shared/shared-boxes/user/$to/$from'. I couldn't figure out by studying the example where this value was derived from, or indeed where it is derived from in any example dictionaries which are documented. Can anyone explain the meaning of the "pattern" field and what exactly its value means? Thanks in advance for all your helpful answers! :) From chayes at afo.net Fri Mar 13 13:29:09 2015 From: chayes at afo.net (Cliff Hayes) Date: Fri, 13 Mar 2015 08:29:09 -0500 Subject: How to detect out-of-sync condition In-Reply-To: References: <5501F8A4.6080306@afo.net> Message-ID: <5502E625.6080003@afo.net> I looked in the place where dovecot logs everything ... the maillog. I didn't see anything but the log is huge and I could have easily missed it. Is there a certain error or phrase I should look for? If so please advise. On 3/13/2015 3:05 AM, Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Thu, 12 Mar 2015, Cliff Hayes wrote: > >> I recently had a user whose mailbox had gone out of sync. > > There are no log entries about broken sync or something something like > that? > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVQKaP3z1H7kL/d9rAQKkCQf/VwbytrDRGDJT8Ado8J5L/melZ751jqME > pmRQ51McVo80SFvpNuQe93tLlJcGwltgr4He2scxBMelzBlfmQ9Vqrl0EH9R+RpE > UxPGHbhWa6Dm23gRMgeYiLx9YlZXnnKtnpUL4MriMwIrrTGaN1hq7yX4Ot2k+ImM > vHI3aFSTg35lklKfrzfDwKaQKQMMVf8fz4fZJICL2Vas2df5XaSEyQY1wM/PUqoj > Bev0POPfvKhba8adY5yAM9fWpBmStwspIwKR1ZWqflOlz41WfOKeQBWkMqXK7hpV > LpF5dY1omVE0BORE5+4T9hY03RXtBcepVczfS4MhX4WrNkJ3TMrjvw== > =gjO3 > -----END PGP SIGNATURE----- > From tss at iki.fi Fri Mar 13 13:36:20 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 13 Mar 2015 15:36:20 +0200 Subject: v2.2.16 released In-Reply-To: <5502BD19.3060304@thelounge.net> References: <0DC5FD73-B4A7-4811-BD7F-7B053BAF47E8@iki.fi> <5501E46E.5030107@thelounge.net> <2C52D958-1701-4FBF-B332-DC81CCD01C34@iki.fi> <5502BD19.3060304@thelounge.net> Message-ID: <6065F789-2D2C-4701-B2AC-9D66B9CEC9BB@iki.fi> On 13 Mar 2015, at 12:34, Reindl Harald wrote: > > > Am 13.03.2015 um 11:23 schrieb Timo Sirainen: >> On 12 Mar 2015, at 21:09, Reindl Harald wrote: >>> >>> /usr/lib64/dovecot/stats/libstats_mail.so >>> >>> why in the world a new sub-directory containing just one so-file enforcing pakcage buildsers to change SPEC files? >> >> So that external plugins can add more files in there and extend the available statistics > > but they can also go to /usr/lib64/dovecot/ > http://fedoraproject.org/wiki/Packaging:Guidelines#Beware_of_Rpath These are plugins, not libraries. They don't need rpaths. The reason for the new directory's existence is exactly the same as for the already existing plugin directories: $lib/dovecot/auth/ $lib/dovecot/doveadm/ $lib/dovecot/settings/ $lib/dovecot/sieve/ So auth, doveadm, settings and sieve all work by loading all the plugins from inside the directories. If I changed the stats to work differently, e.g. load $lib/dovecot/libstats_*.so it would work differently than everything else, which would be confusing. For consistency it would be better then to get rid of the above directories as well. But changing that would break backwards compatibility with external plugins (e.g. with Sieve). From h.reindl at thelounge.net Fri Mar 13 13:36:49 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 13 Mar 2015 14:36:49 +0100 Subject: How to detect out-of-sync condition In-Reply-To: <5502E625.6080003@afo.net> References: <5501F8A4.6080306@afo.net> <5502E625.6080003@afo.net> Message-ID: <5502E7F1.8020902@thelounge.net> Am 13.03.2015 um 14:29 schrieb Cliff Hayes: > I looked in the place where dovecot logs everything ... the maillog. > I didn't see anything but the log is huge and I could have easily missed > it. > Is there a certain error or phrase I should look for? > If so please advise. man grep grep -i 'sync' maillog grep -i 'fail' maillog grep -i 'error' maillog grep -i 'warn' maillog > On 3/13/2015 3:05 AM, Steffen Kaiser wrote: > On Thu, 12 Mar 2015, Cliff Hayes wrote: > >>>> I recently had a user whose mailbox had gone out of sync. > > There are no log entries about broken sync or something something like > that? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From cumc-4361-2 at chguadalquivir.es Fri Mar 13 13:44:19 2015 From: cumc-4361-2 at chguadalquivir.es (Fran) Date: Fri, 13 Mar 2015 14:44:19 +0100 Subject: Active directory bind and quota stuff Message-ID: <5502E9B3.3030403@chguadalquivir.es> Hi list, first of all, please excuse me for any grammar mistake on my text, I'm still learning english. I'm making a Dovecot installation using my Active Directory like pass and user DB. I'm setting the per-user quota from an AD field (Initials) and everything seems to work fine, but I would like to know if you could help me to improve a couple of things. --------------------------------------------------------------------------- --------------------------------------------------------------------------- I'm using CentOS 7, this is my Dovecot build: --------------------------------------------------------------------------- # dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw fail SQL driver plugins: mysql postgresql sqlite Passdb: checkpassword ldap pam passwd passwd-file shadow sql Userdb: checkpassword ldap(plugin) nss passwd prefetch passwd-file sql # dovecot --version 2.2.10 And this is my /etc/dovecot/dovecot-ldap.conf.ext file: --------------------------------------------------------------------------- hosts = adserver1.dom, adserver2.dom base = DC=dom ldap_version = 3 auth_bind = yes auth_bind_userdn = dom\%u dn = cn=imapUser,cn=Users,dc=dom dnpass = ******** user_filter = (&(samaccountname=%u)(objectClass=person)(|(mail=%u at dom.com)(othermailbox=%u at dom.com))) pass_filter = (&(samaccountname=%u)(objectClass=person)(|(mail=%u at dom.com)(othermailbox=%u at dom.com))) pass_attrs = userPassword=password user_attrs = Initials=quota_rule=*:storage=%$MB --------------------------------------------------------------------------- --------------------------------------------------------------------------- My questions: 1) My AD doesn't allow anonymous searches and I was thinking in a way to avoid the need of have a specific user account with his password stored in a configuration file to do the bind. And I don't know if this is posible, but, does it exist a way to use the same user you are trying to validate to do bind with the AD server? 2) As you can see (user_attrs = Initials=quota_rule=*:storage=%$MB), to set the per-user quota value, I require a numeric value in "Initials" field of my AD. I've realized that if some wrong value is introduced in that field, Dovecot throws an error when you try to validate the user, and hence, imap logon fails. Does it exist any way to bypass that and to use the default quota value if something is wrong with the per-user quota set? Error log with wrong quota values (I wrote the value e34 in the Initials field of user1 in AD, instead of a valid numeric value): * quota refresh command: [root at dmail02 dovecot]# doveadm quota recalc -u user1 doveadm(user1): Error: user user1: Initialization failed: Failed to initialize quota: Invalid quota root quota: Invalid rule *:storage=e34MB: Invalid rule limit value 'storage=e34MB': Unknown unit: e34MB * imap login: Mar 13 14:12:10 dmail02 dovecot: imap(user1): Error: user user1: Initialization failed: Failed to initialize quota: Invalid quota root quota: Invalid rule *:storage=e34MB: Invalid rule limit value 'storage=e34MB': Unknown unit: e34MB Mar 13 14:12:10 dmail02 dovecot: imap(user1): Error: Invalid user settings. Refer to server log for more information. Thank you very much in advance, Fran From skdovecot at smail.inf.fh-brs.de Fri Mar 13 14:20:21 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 13 Mar 2015 15:20:21 +0100 (CET) Subject: How to detect out-of-sync condition In-Reply-To: <5502E7F1.8020902@thelounge.net> References: <5501F8A4.6080306@afo.net> <5502E625.6080003@afo.net> <5502E7F1.8020902@thelounge.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 13 Mar 2015, Reindl Harald wrote: > Am 13.03.2015 um 14:29 schrieb Cliff Hayes: >> I looked in the place where dovecot logs everything ... the maillog. Are you sure about maillog? I mean: doveadm log find gives you that file? >> I didn't see anything but the log is huge and I could have easily missed >> it. >> Is there a certain error or phrase I should look for? >> If so please advise. > > man grep > > grep -i 'sync' maillog > grep -i 'fail' maillog > grep -i 'error' maillog > grep -i 'warn' maillog (Y) >> On 3/13/2015 3:05 AM, Steffen Kaiser wrote: >> On Thu, 12 Mar 2015, Cliff Hayes wrote: >> >>>>> I recently had a user whose mailbox had gone out of sync. >> >> There are no log entries about broken sync or something something like >> that? > > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVQLyJXz1H7kL/d9rAQK7YggAtWJz0FEylF3I8r7qUZsRjraPuXcHUAHM bDGkrr/9BVUTeFF+wVK51k8iBBb7P2VJo8tefJ1gho00gfJpZo2EE8VG/LEt2LeH chq7v8iCNBaTRMXLlQwzUMii5JjSfoKuFRFEhphuv0yLacWT5AICPiqTqrLz+wQh UW7KMfxZuXDxR4zNj58Da8HQmiTSNm7BQv4N1GpExkGNxcE14X7By4iPFxkKq1OI O24iPL1WSwwhOftdBGuXFO2oCOqIQKHu/aZdoElezobxlNXgegDMk6ui9LPu9V1T 44IF1sICPEjpQyf7SgEvJcpBarXeaUy2NtZGMkWHWU0LNl5XQrETcA== =OGb3 -----END PGP SIGNATURE----- From paolo.cravero at csi.it Fri Mar 13 14:43:52 2015 From: paolo.cravero at csi.it (Paolo Cravero) Date: Fri, 13 Mar 2015 15:43:52 +0100 (CET) Subject: Overriding dovecot.conf from Userdb Extras In-Reply-To: <1425871044.31600.YahooMailBasic@web142403.mail.bf1.yahoo.com> References: <1425871044.31600.YahooMailBasic@web142403.mail.bf1.yahoo.com> Message-ID: <244597769.1097702.1426257832120.JavaMail.open-xchange@comunica.csi.it> Il 9 marzo 2015 alle 4.17 "E.B." ha scritto: > I thought I read that anything from dovecot.conf can be overridden in a > userdb lookup. Or a passdb lookup with "userdb_" prefix. > > But I tried for fun change log_path but it never worked. Is that because > logging is special, already started logging before it comes to the > passdb/userdb lookups? So are there some dovecot.conf settings > that cannot be overridden? To my understanding only these extra parameters can be tweaked through the userdb/passdb: http://wiki2.dovecot.org/PasswordDatabase/ExtraFields http://wiki2.dovecot.org/UserDatabase/ExtraFields (+ mail and quota_rule) Paolo From cma at cmadams.net Fri Mar 13 14:56:51 2015 From: cma at cmadams.net (Chris Adams) Date: Fri, 13 Mar 2015 09:56:51 -0500 Subject: Different realm for different listeners? In-Reply-To: <20150310174945.GB22564@cmadams.net> References: <20150310172037.GA22564@cmadams.net> <54FF2C20.8080207@whyscream.net> <20150310174945.GB22564@cmadams.net> Message-ID: <20150313145651.GC15980@cmadams.net> Once upon a time, Chris Adams said: > All my users are in MySQL. Reading the docs, I see that "%l" expands to > the local IP address "almost everywhere". If that's available in the > userdb SQL expansion, I could write a (somewhat convoluted) SQL > statement that adds a domain based on the local IP. Does that sound > like it'll work? Not quite as "elegant" as just setting > auth_default_realm in separate listeners, but should work I guess. Just a follow-up for the archives: this worked. This is my MySQL password_query: password_query = \ SELECT username AS user, password, \ concat('/srv/mail/',maildir) AS userdb_home, 1000 AS userdb_uid, \ 1000 AS userdb_gid \ FROM mailbox WHERE active = 1 AND username = case \ when '%d' <> '' then '%u' \ when '%l' = '10.0.9.73' then '%u at domain1.com' \ when '%l' = '10.0.9.74' then '%u at domain2.net' \ else '%u' end -- Chris Adams From chayes at afo.net Fri Mar 13 17:51:50 2015 From: chayes at afo.net (Cliff Hayes) Date: Fri, 13 Mar 2015 12:51:50 -0500 Subject: How to detect out-of-sync condition In-Reply-To: <5502E7F1.8020902@thelounge.net> References: <5501F8A4.6080306@afo.net> <5502E625.6080003@afo.net> <5502E7F1.8020902@thelounge.net> Message-ID: <550323B6.2020608@afo.net> I searched with your recommended criteria and there was nothing relevant returned. On 3/13/2015 8:36 AM, Reindl Harald wrote: > > Am 13.03.2015 um 14:29 schrieb Cliff Hayes: >> I looked in the place where dovecot logs everything ... the maillog. >> I didn't see anything but the log is huge and I could have easily missed >> it. >> Is there a certain error or phrase I should look for? >> If so please advise. > > man grep > > grep -i 'sync' maillog > grep -i 'fail' maillog > grep -i 'error' maillog > grep -i 'warn' maillog > >> On 3/13/2015 3:05 AM, Steffen Kaiser wrote: >> On Thu, 12 Mar 2015, Cliff Hayes wrote: >> >>>>> I recently had a user whose mailbox had gone out of sync. >> >> There are no log entries about broken sync or something something like >> that? > From chayes at afo.net Fri Mar 13 17:55:01 2015 From: chayes at afo.net (Cliff Hayes) Date: Fri, 13 Mar 2015 12:55:01 -0500 Subject: How to detect out-of-sync condition In-Reply-To: References: <5501F8A4.6080306@afo.net> <5502E625.6080003@afo.net> <5502E7F1.8020902@thelounge.net> Message-ID: <55032475.9090609@afo.net> Yes ... it is maillog per below: # doveadm log find Looking for log files from /var/log Debug: /var/log/maillog Info: /var/log/maillog Warning: /var/log/maillog Error: /var/log/maillog Fatal: /var/log/maillog On 3/13/2015 9:20 AM, Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Fri, 13 Mar 2015, Reindl Harald wrote: >> Am 13.03.2015 um 14:29 schrieb Cliff Hayes: >>> I looked in the place where dovecot logs everything ... the maillog. > > Are you sure about maillog? I mean: > > doveadm log find > > gives you that file? > >>> I didn't see anything but the log is huge and I could have easily missed >>> it. >>> Is there a certain error or phrase I should look for? >>> If so please advise. >> >> man grep >> >> grep -i 'sync' maillog >> grep -i 'fail' maillog >> grep -i 'error' maillog >> grep -i 'warn' maillog > > (Y) > >>> On 3/13/2015 3:05 AM, Steffen Kaiser wrote: >>> On Thu, 12 Mar 2015, Cliff Hayes wrote: >>> >>>>>> I recently had a user whose mailbox had gone out of sync. >>> >>> There are no log entries about broken sync or something something like >>> that? >> >> > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVQLyJXz1H7kL/d9rAQK7YggAtWJz0FEylF3I8r7qUZsRjraPuXcHUAHM > bDGkrr/9BVUTeFF+wVK51k8iBBb7P2VJo8tefJ1gho00gfJpZo2EE8VG/LEt2LeH > chq7v8iCNBaTRMXLlQwzUMii5JjSfoKuFRFEhphuv0yLacWT5AICPiqTqrLz+wQh > UW7KMfxZuXDxR4zNj58Da8HQmiTSNm7BQv4N1GpExkGNxcE14X7By4iPFxkKq1OI > O24iPL1WSwwhOftdBGuXFO2oCOqIQKHu/aZdoElezobxlNXgegDMk6ui9LPu9V1T > 44IF1sICPEjpQyf7SgEvJcpBarXeaUy2NtZGMkWHWU0LNl5XQrETcA== > =OGb3 > -----END PGP SIGNATURE----- > From list at airstreamcomm.net Fri Mar 13 19:15:07 2015 From: list at airstreamcomm.net (List) Date: Fri, 13 Mar 2015 14:15:07 -0500 Subject: JMAP support Message-ID: <5503373B.7060505@airstreamcomm.net> Just found http://jmap.io/ which is a JSON based RPC protocol for synchronizing messages/contacts/calendars. Any plans to support this protocol in Dovecot? From jtam.home at gmail.com Fri Mar 13 23:00:56 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Fri, 13 Mar 2015 16:00:56 -0700 (PDT) Subject: How to detect out-of-sync condition In-Reply-To: References: Message-ID: On Fri, 13 Mar 2015, Cliff Hayes writes: >> By closing off other avenues other than dovecot imap/pop/lda/etc., >> the indices will stay sync'd. > > I use dovecot's lda and dovecot's sieve filter. Then I'm not sure how mailboxes ever get out of sync. > So it looks like I need to compare the index/mailbox mtimes as you suggest. > > What am I looking for? > I see that the indexes are updated when I run the resync. > > I checked my mailbox (that was not resynced) and noticed that > dovecot.index last update was 16 days ago. Just to be clear, you're talking "doveadm index ...", yes? You might have to check dovecot.index.log* as well as they could contain the latest changes that have yet to propagate to the main index file. http://wiki2.dovecot.org/Design/Indexes/TransactionLog > So am I resyncing if the gap is over x days? > If so, is there a way to resync just those mailboxes with a doveadm > command or do I have to write a program look for that condition and run > doveadm when matched? It's possible that "doveadm index" checks modification times as an optimization measure. My recommended game plan: 1) Are your indices *really* out of date (checks logs as Steffen recommends)? 2) If so, how do they get out of date and can you avoid it? 3) If you can't avoid it, does it cause real problems? Most of the time, dovecot will seemlessly rebuild it and it's transparent to the user. 4) If you got this far, run a trial "doveadm index -A INBOX" (assuming you're just concerned about INBOXs) to see if it's really such a bad operation. As Knuth (hallowed be thy name!) said, "premature optimization is the root of all evil." Only after all these steps are found to be unsatisfactory would I consider writing your own scan and fix tool. Joseph Tam From jtam.home at gmail.com Fri Mar 13 23:21:34 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Fri, 13 Mar 2015 16:21:34 -0700 (PDT) Subject: How to detect out-of-sync condition In-Reply-To: References: Message-ID: Cliff Hayes writes: > I didn't see anything but the log is huge and I could have easily missed > it. > Is there a certain error or phrase I should look for? > If so please advise. In my setup using mbox, typical log entries would be like (Common) Warning: UIDVALIDITY changed ({int} -> {int}) in mbox file ... Error: Next message unexpectedly corrupted in mbox file ... (Rare) Error: Corrupted index cache file {cache-file}: Broken physical size for mail UID ... Error: read({file}) failed: Next message unexpectedly lost from mbox file ... Error: read({file}) failed: Cached message size larger than expected ... Joseph Tam From chayes at afo.net Sat Mar 14 02:00:23 2015 From: chayes at afo.net (Cliff Hayes) Date: Fri, 13 Mar 2015 21:00:23 -0500 Subject: How to detect out-of-sync condition In-Reply-To: References: Message-ID: <55039637.2040609@afo.net> I tried to look at a dovecot.index.log and it was unreadable in a text editor. I didn't see anything in the wiki link about how to view the log. So I guess I'll just be guilty of excessive optimizing. Should I run a daily doveadm index or resync? On 3/13/2015 6:00 PM, Joseph Tam wrote: > On Fri, 13 Mar 2015, Cliff Hayes writes: > >>> By closing off other avenues other than dovecot imap/pop/lda/etc., >>> the indices will stay sync'd. >> >> I use dovecot's lda and dovecot's sieve filter. > > Then I'm not sure how mailboxes ever get out of sync. > >> So it looks like I need to compare the index/mailbox mtimes as you >> suggest. >> >> What am I looking for? >> I see that the indexes are updated when I run the resync. >> >> I checked my mailbox (that was not resynced) and noticed that >> dovecot.index last update was 16 days ago. > > Just to be clear, you're talking "doveadm index ...", yes? > > You might have to check dovecot.index.log* as well as they could contain > the latest changes that have yet to propagate to the main index file. > > http://wiki2.dovecot.org/Design/Indexes/TransactionLog > >> So am I resyncing if the gap is over x days? >> If so, is there a way to resync just those mailboxes with a doveadm >> command or do I have to write a program look for that condition and run >> doveadm when matched? > > It's possible that "doveadm index" checks modification times as an > optimization measure. My recommended game plan: > > 1) Are your indices *really* out of date (checks logs as > Steffen recommends)? > > 2) If so, how do they get out of date and can you avoid it? > > 3) If you can't avoid it, does it cause real problems? Most > of the time, dovecot will seemlessly rebuild it and > it's transparent to the user. > > 4) If you got this far, run a trial "doveadm index -A INBOX" > (assuming you're just concerned about INBOXs) to see if it's > really such a bad operation. As Knuth (hallowed be thy > name!) said, "premature optimization is the root of all evil." > > Only after all these steps are found to be unsatisfactory would I consider > writing your own scan and fix tool. > > Joseph Tam > From emailbuilder88 at yahoo.com Sat Mar 14 02:18:33 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Fri, 13 Mar 2015 19:18:33 -0700 Subject: Overriding dovecot.conf from Userdb Extras In-Reply-To: <244597769.1097702.1426257832120.JavaMail.open-xchange@comunica.csi.it> Message-ID: <1426299513.39043.YahooMailBasic@web142405.mail.bf1.yahoo.com> > > I thought I read that anything from dovecot.conf can be overridden in a > > userdb lookup. Or a passdb lookup with "userdb_" prefix. > > > > But I tried for fun change log_path but it never worked. Is that because > > logging is special, already started logging before it comes to the > > passdb/userdb lookups? So are there some dovecot.conf settings > > that cannot be overridden? > > > To my understanding only these extra parameters can be tweaked through the > userdb/passdb: > http://wiki2.dovecot.org/PasswordDatabase/ExtraFields > http://wiki2.dovecot.org/UserDatabase/ExtraFields (+ mail and quota_rule) Quoting from your second link: "It's possible to override settings from dovecot.conf" (quota rules being a common example, yes). I've successfully overridden a few different dovecot.conf settings, mostly for lda and sieve, but also mail_debug from 10-logging.conf. So it does work to override *some* settings from dovecot.conf -- but there seem to be a few like log_path that are immune to overrides I guess. From stephan at rename-it.nl Sat Mar 14 23:33:30 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 15 Mar 2015 00:33:30 +0100 Subject: Released Pigeonhole v0.4.7.rc2 for Dovecot v2.2.16 Message-ID: <5504C54A.9010502@rename-it.nl> Hello Dovecot users, The rc1 release showed a few problems with the up-to-date checking of on-disk binary files. The binary files contain some metadata about the script these were compiled from. If this doesn't match the current script anymore, the script is recompiled. One of these metadata items is the script location. However, this location wasn't normalized, so when the binary would be compiled with sievec it depended on the path provided at the command line. When dumping a script binary, the metadata is now printed as well. Also, I added more debug log lines about up-to-date checking. I improved backwards compatibility for the file script storage a bit relative to the last version. It now properly recognizes a regular file at the symlink path, even when the storage path itself doesn't exist. Changelog v0.4.7 (updated): * editheader extension: Made protection against addition and deletion of headers configurable separately. Also, the `Received' and `Auto-Submitted' headers are no longer protected against addition by default. * Turned message envelope address parse errors into warnings. * The interpreter now accepts non-standard domain names, e.g. containing '_'. + Implemented the Sieve index extension (RFC 5260). + Implemented support for the mboxmetadata and servermetadata extensions (RFC 5490). + Implemented new sieve commands for the doveadm command line utility. These commands are currently limited to ManageSieve operations, but the other current sieve tools will be migrated to doveadm in the near future as well. + Added more debug output about binary up-to-date checking. + Added script metadata to binary dump output. - Fixed Sieve script binary up-to-date checking by normalizing the script location. - The Sieve interpreter now flushes the duplicate database during start phase of result execution rather than commit phase. This makes sure locks on the duplicate database are released as soon as possible, preventing contention. - Performed a few optimizations in the lexical scanner of the language. - Fixed bug in `:matches' match-type that made a pattern without wildcards match as if there were a '*' at the beginning. - Fixed crash in validation of the string parameter of the comparator tag. - extprograms extension: Made sure supplemental group privileges are also dropped. This was a problem reported by Debian lintian. - Fixed bug in handling of binary errors for action side-effects and message overrides. - file script storage: Restructured storage initialization to address backwards compatibility issues. - dict script storage: Fixed small memory allocation bug. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.7.rc2.tar.gz http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.7.rc2.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From stephan at rename-it.nl Sat Mar 14 23:37:48 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 15 Mar 2015 00:37:48 +0100 Subject: Why is Sieve trying to re-compile global scripts? In-Reply-To: <550218E2.6050308@rename-it.nl> References: <1426054665.13781.YahooMailBasic@web142405.mail.bf1.yahoo.com> <5500149E.2010309@kit.edu> <5500C98B.70808@rename-it.nl> <550170D2.60300@kit.edu> <550218E2.6050308@rename-it.nl> Message-ID: <5504C64C.4020803@rename-it.nl> On 3/12/2015 11:53 PM, Stephan Bosch wrote: > On 3/12/2015 11:56 AM, Olaf Hopp wrote: >> On 03/12/2015 12:02 AM, Stephan Bosch wrote: >>> >>> Please do. I cannot reproduce this so far. >>> >>> Since E.B. still got an obscure debug message about metadata not being >>> up to date, I added debug lines to the remaining places where this could >>> emerge (currently only available from hg). >>> >>> Regards, >>> >>> Stephan. >>> >> Hi, >> I'm still trying but currently I can not reproduce the bug. >> But I will keep on hammering on it. > Looks like I found the bug. Will need some time to fix this properly. I released rc2. Please check whether this resolves the issues. Regards, Stephan. From thomas at preissler.co.uk Sun Mar 15 09:35:56 2015 From: thomas at preissler.co.uk (Thomas Preissler) Date: Sun, 15 Mar 2015 09:35:56 +0000 Subject: Dovecot 2.1.7 still accepting SSLv3 though disabled? Message-ID: <20150315093555.GC1513@blackhole.tracing.lan> Hello, I came across a strange problem with my Dovecot 2.1.7 installation (updated Debian Wheezy) in regards to SSL/TLS connections. My configuration is as follows: $ dovecot -n | grep ssl service imap-login { ssl = yes ... } ssl_cert = <...... ssl_cipher_list = EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA ssl_key = <...... ssl_protocols = !SSLv3 !SSLv2 This cipherstring has been taken from https://bettercrypto.org/static/applied-crypto-hardening.pdf. But this is not the problem, when I comment it out, Dovecot still behaves the same way. When I enable verbose_ssl I get this: 2015-03-15 08:27:39 imap-login: Warning: SSL: where=0x10, ret=1: before/accept initialization [$CLIENTIP] 2015-03-15 08:27:39 imap-login: Warning: SSL: where=0x2001, ret=1: before/accept initialization [$CLIENTIP] 2015-03-15 08:27:39 imap-login: Warning: SSL: where=0x2002, ret=-1: unknown state [$CLIENTIP] 2015-03-15 08:27:39 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 read client hello A [$CLIENTIP] 2015-03-15 08:27:39 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 write server hello A [$CLIENTIP] 2015-03-15 08:27:39 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 write certificate A [$CLIENTIP] 2015-03-15 08:27:39 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 write server done A [$CLIENTIP] 2015-03-15 08:27:39 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 flush data [$CLIENTIP] 2015-03-15 08:27:39 imap-login: Warning: SSL: where=0x2002, ret=-1: SSLv3 read client certificate A [$CLIENTIP] 2015-03-15 08:27:39 imap-login: Warning: SSL: where=0x2002, ret=-1: SSLv3 read client certificate A [$CLIENTIP] 2015-03-15 08:27:39 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 read client key exchange A [$CLIENTIP] 2015-03-15 08:27:39 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 read finished A [$CLIENTIP] 2015-03-15 08:27:39 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 write change cipher spec A [$CLIENTIP] 2015-03-15 08:27:39 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 write finished A [$CLIENTIP] 2015-03-15 08:27:39 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 flush data [$CLIENTIP] 2015-03-15 08:27:39 imap-login: Warning: SSL: where=0x20, ret=1: SSL negotiation finished successfully [$CLIENTIP] 2015-03-15 08:27:39 imap-login: Warning: SSL: where=0x2002, ret=1: SSL negotiation finished successfully [$CLIENTIP] Is this right? Is SSLv3 used on this connection? But when I explicitely test for SSLv3 support I get $ openssl s_client -connect $SERVERIP:993 -ssl3 CONNECTED(00000003) 140683835029160:error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure:s3_pkt.c:1260:SSL alert number 40 140683835029160:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:s3_pkt.c:598: --- no peer certificate available --- No client certificate CA names sent --- SSL handshake has read 7 bytes and written 0 bytes --- New, (NONE), Cipher is (NONE) Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE SSL-Session: Protocol : SSLv3 Cipher : 0000 Session-ID: Session-ID-ctx: Master-Key: Key-Arg : None PSK identity: None PSK identity hint: None SRP username: None Start Time: 1426411304 Timeout : 7200 (sec) Verify return code: 0 (ok) --- Where I got this from says "if you you get a handshake failure, then you don't support SSLv3". But in my case the following output kinda says, that I do support it - with a ciphers of (NONE)? In regards to libraries $ ldd /usr/lib/dovecot/imap-login | grep ssl libssl.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f1f55025000) $ dpkg -l | grep ssl ii libcrypt-openssl-bignum-perl 0.04-3 amd64 Access OpenSSL multiprecision integer arithmetic libraries ii libcrypt-openssl-dsa-perl 0.13-6 amd64 module which implements the DSA signature verification system ii libcrypt-openssl-rsa-perl 0.28-1 amd64 module for RSA encryption using OpenSSL ii libcrypt-ssleay-perl 0.58-1 amd64 OpenSSL support for LWP ii libio-socket-ssl-perl 1.76-2 all Perl module implementing object oriented interface to SSL sockets ii libnet-ssleay-perl 1.48-1+b1 amd64 Perl module for Secure Sockets Layer (SSL) rc libssl0.9.8 0.9.8o-4squeeze14 amd64 SSL shared libraries ii libssl1.0.0:amd64 1.0.1e-2+deb7u14 amd64 SSL shared libraries ii openssl 1.0.1e-2+deb7u14 amd64 Secure Socket Layer (SSL) binary and related cryptographic tools ii openssl-blacklist 0.5-3 all Blacklists for OpenSSL RSA keys and tools ii python-openssl 0.13-2+deb7u1 amd64 Python 2 wrapper around the OpenSSL library ii ssl-cert 1.0.32 all simple debconf wrapper for OpenSSL ii ssl-cert-check 3.22-1 all proactively handling X.509 certificate expiration ii sslmate 0.6.2-1 all Buy and manage SSL certificates from the command line My NginX is using the same library, and this does indeed support TLSv2, so what I am doing wrong in my Dovecot configuration? Any clues? Regards Thomas -- www.preissler.co.uk | Twitter: @module0x90 | PGP-Key: 75889415 GPG Fingerprint: CCBD 153A D257 CA7E A217 FDF7 5928 03D1 7588 9415 From sca at andreasschulze.de Sun Mar 15 13:42:00 2015 From: sca at andreasschulze.de (A. Schulze) Date: Sun, 15 Mar 2015 14:42:00 +0100 Subject: Dovecot 2.1.7 still accepting SSLv3 though disabled? In-Reply-To: <20150315093555.GC1513@blackhole.tracing.lan> Message-ID: <20150315144200.Horde.hUl8CfdWQMVR3ndNIO887w1@horde.andreasschulze.de> Thomas Preissler: > ssl_protocols = !SSLv3 !SSLv2 that disable SSLv3 > When I enable verbose_ssl I get this: > 2015-03-15 08:27:39 imap-login: Warning: SSL: where=0x2001, > ret=1: SSLv3 flush data [$CLIENTIP] > ... > Is this right? Is SSLv3 used on this connection? The logging is right, but SSLv3 isn't used. Today it's not uncommon that application /log/ SSLv3, where they /mean/ TLS1.x Some days ago where TLSv1 became available there wasn't a great difference between SSLv3 and TLSv1 So Developers reused large portions of code. That's what you see here.. > But when I explicitely test for SSLv3 support I get > > $ openssl s_client -connect $SERVERIP:993 -ssl3 > > CONNECTED(00000003) > 140683835029160:error:14094410:SSL > routines:SSL3_READ_BYTES:sslv3 alert handshake > failure:s3_pkt.c:1260:SSL alert number 40 > 140683835029160:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl > handshake failure:s3_pkt.c:598: That is the ultimate prove your server have SSLv3 disabled. Andreas From Olaf.Hopp at kit.edu Mon Mar 16 07:18:05 2015 From: Olaf.Hopp at kit.edu (Olaf Hopp) Date: Mon, 16 Mar 2015 08:18:05 +0100 Subject: Why is Sieve trying to re-compile global scripts? In-Reply-To: <5504C64C.4020803@rename-it.nl> References: <1426054665.13781.YahooMailBasic@web142405.mail.bf1.yahoo.com> <5500149E.2010309@kit.edu> <5500C98B.70808@rename-it.nl> <550170D2.60300@kit.edu> <550218E2.6050308@rename-it.nl> <5504C64C.4020803@rename-it.nl> Message-ID: <550683AD.8020003@kit.edu> On 03/15/2015 12:37 AM, Stephan Bosch wrote: > On 3/12/2015 11:53 PM, Stephan Bosch wrote: >> On 3/12/2015 11:56 AM, Olaf Hopp wrote: >>> On 03/12/2015 12:02 AM, Stephan Bosch wrote: >>>> >>>> Please do. I cannot reproduce this so far. >>>> >>>> Since E.B. still got an obscure debug message about metadata not being >>>> up to date, I added debug lines to the remaining places where this could >>>> emerge (currently only available from hg). >>>> >>>> Regards, >>>> >>>> Stephan. >>>> >>> Hi, >>> I'm still trying but currently I can not reproduce the bug. >>> But I will keep on hammering on it. >> Looks like I found the bug. Will need some time to fix this properly. > > I released rc2. Please check whether this resolves the issues. > With RC2 everything looks good ! And finally I could reproduce the bug: with 0.4.5 and 0.4.7 RC1 you can trigger it when you compile the master sieve script with a *relative* path: cd /etc/dovecot /usr/bin/sievec -D ./sieve-master will trigger it. Whereas /usr/bin/sievec -D /etc/dovecot/sieve-master even with 0.4.5 will run fine. With 0.4.7 RC2 it makes no difference, wether you use an absolute or a relative path to the sieve-master script. Olaf -- Karlsruher Institut f?r Technologie (KIT) ATIS - Abt. Technische Infrastruktur, Fakult?t f?r Informatik Dipl.-Geophys. Olaf Hopp - Leitung IT-Dienste - Am Fasanengarten 5, Geb?ude 50.34, Raum 009 76131 Karlsruhe Telefon: +49 721 608-43973 Fax: +49 721 608-46699 E-Mail: Olaf.Hopp at kit.edu www.atis.informatik.kit.edu www.kit.edu KIT - Universit?t des Landes Baden-W?rttemberg und nationales Forschungszentrum in der Helmholtz-Gemeinschaft Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5214 bytes Desc: S/MIME Cryptographic Signature URL: From jtam.home at gmail.com Mon Mar 16 07:52:06 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 16 Mar 2015 00:52:06 -0700 (PDT) Subject: How to detect out-of-sync condition In-Reply-To: References: Message-ID: > I tried to look at a dovecot.index.log and it was unreadable in a text > editor. I didn't see anything in the wiki link about how to view the log. It's not supposed to be human readable -- the point of this exercise was to check whether your indices are really out of date. You can perhaps compare the mtime of the index files and the mailbox. If they're close or the same, then your indices are not out of sync and don't require sync'ing. > So I guess I'll just be guilty of excessive optimizing. > Should I run a daily doveadm index or resync? You've skipped over the first 3 steps (is it really out of sync, is there a better way to avoid it, does the un-sync really cause probems) and went right to a solution for a possibly non-existent problem. Be that as it may, the frequency of re-syncing is up to you. I have no idea how large your userbase is or how intensive this operation will be. If it takes only a minute, you could run it more frequently. If you're going through terabytes of data, less frequently. I'm not quite convince you need it at all: I never had to do a global re-indexing except for a one-time dovecot initializing, or dealing with the odd crash that left indices in a unfixable state (which is more a bug than an unsync problem). Joseph Tam From skdovecot at smail.inf.fh-brs.de Mon Mar 16 08:12:48 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 16 Mar 2015 09:12:48 +0100 (CET) Subject: Overriding dovecot.conf from Userdb Extras In-Reply-To: <1426299513.39043.YahooMailBasic@web142405.mail.bf1.yahoo.com> References: <1426299513.39043.YahooMailBasic@web142405.mail.bf1.yahoo.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 13 Mar 2015, E.B. wrote: >>> I thought I read that anything from dovecot.conf can be overridden in a >>> userdb lookup. Or a passdb lookup with "userdb_" prefix. >>> >>> But I tried for fun change log_path but it never worked. Is that because >>> logging is special, already started logging before it comes to the >>> passdb/userdb lookups? So are there some dovecot.conf settings >>> that cannot be overridden? > > I've successfully overridden a few different > dovecot.conf settings, mostly for lda and sieve, > but also mail_debug from 10-logging.conf. So > it does work to override *some* settings from ^^ I would say "most" :) > dovecot.conf -- but there seem to be a few like > log_path that are immune to overrides I guess. My understanding is that you can override all settings, but overriding does not perform any action. You said yourself: the log is already open and Dovecot does not implement to re-open the log. If it would, the setting would be the overriden, I guess. The same applies to port and service settings, after an user authentificated or the userdb is queried, the basic configuration of the demons is not changed, hence, the override has no effect. Maybe you should raise a feature request for changing the log path or offer a patch. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVQaQgHz1H7kL/d9rAQKMSwf/XszBy8C1HUd2xHg/tC1VaTJq5DR7APne l39euVkRLpT9l2as2m/aEiHb08eackuaixH6OkBf8WlBhCHmOjsR3bdQiU3P3dmH DRAohK8nmuMRdx8WIZooUU5r62h2rniaKCqWctElWLyHYETJEMnmFB0/re5YG7Qa 7qio9/GVss/YPiOjUG3r5lVD2RoCQ9SYYlXzAE5ZtSfL6lxSkwSyDlbzhT0DjdBX +omt0YaJHWuuAvITqH+FOi3I6W0pxDAd9Y1kdKWWHHCDA1UzXy2z8swh+aNpIUQu bGyDHGpqu4b8bWoQ5RBqxOqP3zvNsNuTekK5Qz2N8hRY35bxhTHj2A== =ybB9 -----END PGP SIGNATURE----- From vanja at pobox.com Mon Mar 16 08:12:31 2015 From: vanja at pobox.com (Vanja) Date: Mon, 16 Mar 2015 15:12:31 +0700 Subject: Problem with replication ("Fatal: -N parameter requires syncing with remote host") Message-ID: <5506906F.5030406@pobox.com> I have been fighting, for 2 days, with a Dovecot setup and I now officially need help :) Basically, I am trying to setup replication between 2 identical Dovecot servers, on different physical servers (different hostnames/etc, of course), and although I've managed to solve many problems I can't overcome the final hurdle. When a mail comes in to one of the servers, replication over TCP should kick in and mail should be replicated, but instead I get this in the mail log: ------------------------------------------------------------------------------------------------- dovecot2 dovecot: doveadm(user1 at test1.int): Fatal: -N parameter requires syncing with remote host ------------------------------------------------------------------------------------------------- Mail is delivered to dovecot2 server, Postfix delivers it via LMTP, and everything seems to work ok. However, I can't figure out the cause for the error that I am seeing. I have looked into source code and found where this error is triggered (line 834 in doveadm/dsync/doveadm-dsync.c, inside Dovecot 2.2.9 source code), but I am afraid that I don't understand the context (ctx->run_type == DSYNC_RUN_TYPE_LOCAL) and it would take me a lot of time to debug the whole thing from the scratch. I've never looked into Dovecot internals before, so it's all a mystery to me at this point. I've tried stracing the process but the flow does not make sense to me, I can't understand what exactly happens before error is displayed. Also, there is no traffic to port 12345 on other node, so execution fails before syncing even starts (and error message is not related with remote node config/setup/etc). Triggering replication manually, from command line, works as expected: # doveadm sync -A tcp:192.168.1.100 (at this point user1's mail folder is properly updated on remote server) I've tried searching around, but it appears that noone else has this problem(or isn't complaining about it :). I am not sure if I am doing something wrong (apart from replication). I will need to support multiple virtual domains and I am basically using passwd-file for authentication, so I am using a single passwd file for all virtual domain users which allows user lookup (doveadm user '*') to work, which is required by replication. This is configuration file from dovecot2 server (which I use for testing and delivering mails): --- config start --- # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.16.0-30-generic x86_64 Ubuntu 14.04.2 LTS ext4 debug_log_path = /var/log/dovecot_debug.log doveadm_password = 67890 doveadm_port = 12345 mail_debug = yes mail_location = maildir:/vmail/mail/%d/%n/Maildir mail_plugins = " notify replication" namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = username_format=%u /vmail/auth/passwd driver = passwd-file } plugin { mail_replica = 192.168.1.100 } protocols = imap lmtp service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth-worker { unix_listener auth-worker { user = dovecot-auth } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = dovecot-auth } service doveadm { inet_listener { port = 12345 } } service imap-login { inet_listener imap { port = 0 } } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 10024 } process_min_avail = 3 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } user = vmail } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 } } ssl_cert = References: <5506906F.5030406@pobox.com> Message-ID: <55069B8E.7030402@pobox.com> Philipp, I already tried explicitly specifying port, it does not make any difference. I tried quite many 'obvious' (to put it this way) configuration changes and tried changing many things in setup, but it would be too much to list all of them in a post. Basically, I was hoping that someone would know what exactly triggers this behavior, so I could fix it. At this point in time, I would like to see whether I can make this work with 2.2.9 that comes with Ubuntu 14.04, to avoid having to compile anything from sources (for easier maintenance, later on, since I probably won't be the one maintaining this). I am also not sure whether there is a reliable (semi-official?) PPA with latest Dovecot. I saw this one ( https://launchpad.net/~mamarley/+archive/ubuntu/updates ) but I am not sure whether I can rely on it not breaking anything at some point. Thanks. On 03/16/2015 03:23 PM, Philipp wrote: > Am 16.03.2015 09:12 schrieb Vanja: >> doveadm_port = 12345 > >> plugin { >> mail_replica = 192.168.1.100 >> } > > would you try: > mail_replica = tcp:192.168.1.100:12345 > which works for me. > > check for firewall settings (both ways) anyway - and I can only advise > to go for 2.2.16; there have been > numerous fixes after 2.2.9 to the whole dsync/replica parts. From vanja at pobox.com Mon Mar 16 09:24:55 2015 From: vanja at pobox.com (Vanja) Date: Mon, 16 Mar 2015 16:24:55 +0700 Subject: Problem with replication ("Fatal: -N parameter requires syncing with remote host") In-Reply-To: References: <5506906F.5030406@pobox.com> Message-ID: <5506A167.9050006@pobox.com> Sorry to post to the list again, but just to confirm that I am seeing same problem with 2.2.16 that I compiled from sources. Same error message appears in the log, can't figure out what in the setup makes replication break. Philipp, can you tell me which auth backend are you using? passwd-file, SQL database or something else? Would you consider posting your 'sanitized' config file here, or emailing it to me privately? Thanks. Vanja On 03/16/2015 03:23 PM, Philipp wrote: > Am 16.03.2015 09:12 schrieb Vanja: >> doveadm_port = 12345 > >> plugin { >> mail_replica = 192.168.1.100 >> } > > would you try: > mail_replica = tcp:192.168.1.100:12345 > which works for me. > > check for firewall settings (both ways) anyway - and I can only advise > to go for 2.2.16; there have been > numerous fixes after 2.2.9 to the whole dsync/replica parts. From vanja at pobox.com Mon Mar 16 09:55:58 2015 From: vanja at pobox.com (Vanja) Date: Mon, 16 Mar 2015 16:55:58 +0700 Subject: Problem with replication ("Fatal: -N parameter requires syncing with remote host") In-Reply-To: <5506906F.5030406@pobox.com> References: <5506906F.5030406@pobox.com> Message-ID: <5506A8AE.3000903@pobox.com> As it usually happens, I found out the answer after reading my own mail to mailing list multiple times. I knew it had to be simple. Somehow, I managed to screw up config and mail_plugin section had the entry "192.168.1.100" instead of "tcp:192.168.1.100". Replication appears to be working now (at least few test mails that I got through triggered it). I am aware that I need to prepend tcp: and I am 100% sure I did it initially, but at some point I removed it and never noticed it (and never paid attention to it). Whoops. So... If anyone runs into this problem, fix it easy :) On 03/16/2015 03:12 PM, Vanja wrote: > I have been fighting, for 2 days, with a Dovecot setup and I now > officially need help :) > > Basically, I am trying to setup replication between 2 identical Dovecot > servers, on different physical servers (different hostnames/etc, of > course), and although I've managed to solve many problems I can't > overcome the final hurdle. > > When a mail comes in to one of the servers, replication over TCP should > kick in and mail should be replicated, but instead I get this in the > mail log: > > ------------------------------------------------------------------------------------------------- > dovecot2 dovecot: doveadm(user1 at test1.int): Fatal: -N parameter requires > syncing with remote host > ------------------------------------------------------------------------------------------------- > > Mail is delivered to dovecot2 server, Postfix delivers it via LMTP, and > everything seems to work ok. However, I can't figure out the cause for > the error that I am seeing. I have looked into source code and found > where this error is triggered (line 834 in > doveadm/dsync/doveadm-dsync.c, inside Dovecot 2.2.9 source code), but I > am afraid that I don't understand the context (ctx->run_type == > DSYNC_RUN_TYPE_LOCAL) and it would take me a lot of time to debug the > whole thing from the scratch. I've never looked into Dovecot internals > before, so it's all a mystery to me at this point. > > I've tried stracing the process but the flow does not make sense to me, > I can't understand what exactly happens before error is displayed. Also, > there is no traffic to port 12345 on other node, so execution fails > before syncing even starts (and error message is not related with remote > node config/setup/etc). > > Triggering replication manually, from command line, works as expected: > > # doveadm sync -A tcp:192.168.1.100 > > (at this point user1's mail folder is properly updated on remote server) > > I've tried searching around, but it appears that noone else has this > problem(or isn't complaining about it :). > > I am not sure if I am doing something wrong (apart from replication). I > will need to support multiple virtual domains and I am basically using > passwd-file for authentication, so I am using a single passwd file for > all virtual domain users which allows user lookup (doveadm user '*') to > work, which is required by replication. > > This is configuration file from dovecot2 server (which I use for testing > and delivering mails): > > --- config start --- > > # 2.2.9: /etc/dovecot/dovecot.conf > # OS: Linux 3.16.0-30-generic x86_64 Ubuntu 14.04.2 LTS ext4 > debug_log_path = /var/log/dovecot_debug.log > doveadm_password = 67890 > doveadm_port = 12345 > mail_debug = yes > mail_location = maildir:/vmail/mail/%d/%n/Maildir > mail_plugins = " notify replication" > namespace inbox { > inbox = yes > location = > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > prefix = > } > passdb { > args = username_format=%u /vmail/auth/passwd > driver = passwd-file > } > plugin { > mail_replica = 192.168.1.100 > } > protocols = imap lmtp > service aggregator { > fifo_listener replication-notify-fifo { > user = vmail > } > unix_listener replication-notify { > user = vmail > } > } > service auth-worker { > unix_listener auth-worker { > user = dovecot-auth > } > } > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > user = dovecot-auth > } > service doveadm { > inet_listener { > port = 12345 > } > } > service imap-login { > inet_listener imap { > port = 0 > } > } > service lmtp { > inet_listener lmtp { > address = 127.0.0.1 > port = 10024 > } > process_min_avail = 3 > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0660 > user = postfix > } > user = vmail > } > service replicator { > process_min_avail = 1 > unix_listener replicator-doveadm { > mode = 0666 > } > } > ssl_cert = ssl_key = userdb { > args = username_format=%u /vmail/auth/passwd > default_fields = uid=vmail gid=vmail home=/vmail/mail/%d/%n > driver = passwd-file > } > protocol lmtp { > postmaster_address = root at localhost > } > > > --- config end --- > > Configuration file on other node is exactly the same, except different > IP address in mail_replica statement. > > Any help is much appreciated. > > Thank you. > From tcstone at caseystone.com Mon Mar 16 10:08:57 2015 From: tcstone at caseystone.com (Casey Stone) Date: Mon, 16 Mar 2015 10:08:57 +0000 Subject: Problem with replication ("Fatal: -N parameter requires syncing with remote host") Message-ID: <7B3D1ADB-899F-4943-BC80-34D243FEBE20@caseystone.com> Switching to official dovecot-ee version solved my sync/backup issues, also on Ubuntu. The 12.04 package works on 14.04. It's free and should update fine. Backup your config files first (dovecot and postfix if you use that)! http://shop.dovecot.fi/home/8-dovecot-ee-repository-access.html From james at lottspot.com Mon Mar 16 13:17:18 2015 From: james at lottspot.com (james at lottspot.com) Date: Mon, 16 Mar 2015 06:17:18 -0700 Subject: Configuring Shared Mailbox Dictionaries In-Reply-To: References: Message-ID: <680912199856c3eb80ffb822d00d2fb9@lottspot.com> After doing some digging through old mailing list threads, I fonud the following information on the 'path' directive for dicts. http://www.dovecot.org/list/dovecot/2009-April/038922.html It looks like paths are hard coded per plugin, and you simply select one of the hard coded paths (this is what I gathered from the thread, I have not verified the accuracy of this). After taking a romp through the Dovecot source, I was able to gather that as far as I can tell, the lvalue of anything inside of the fields{} block will be evaluated as a SQL expression, unless the expression includes characters '/' or '\\'. So in my case, the following configuration will *theoretically* meet my needs. map { pattern = shared/shared-boxes/user/$to/$from table = mailbox_user_map value_field = mailbox_user fields { mailbox_user||'@'||mailbox_domain = $from user_name||'@'||user_domain = $to } } This is untested though, and I have almost certainly overlooked some devilish detail. Will report back further once I have time to test. On 2015-03-13 05:48, james at lottspot.com wrote: > Hello, list! > > I am in the midst of configuring a new Dovecot IMAP server, and I'm > preparing to setup shard mailboxes as part of the ocnfiguration. In > order to get these setup the way I want, I've figured out that I'm > going to have to setup an acl_shared_dict. This is where things start > to get confusing for me, as I have had difficulty finding thorough > explanations of dictionary configuration. > > My primary concern is that all my user details are stored in a sqlite > database in which the '%n' and '%d' components of username/email > addresses are stored in separate columns. The SQL dictionary example > in the Shared Mailboxes document > [http://wiki2.dovecot.org/SharedMailboxes/Shared] assumes that the > username is stored as a singular '%n@%d' key. In my authdb lookups, I > use sqlite syntax to concatenate the '%n' and '%d' columns around a > '@' character to find the user. Is it possible to implement an > acl_shared_dict in some similar fashion? > > My second question, which is completely unrelated and more out of > morbid curiosity than anything (although it could affect my > configuration decisions) is that I noticed every dictionary which > shows up in the documentation has a "pattern" field. In the case of > the acl_shared_dict example, the pattern is > 'shared/shared-boxes/user/$to/$from'. I couldn't figure out by > studying the example where this value was derived from, or indeed > where it is derived from in any example dictionaries which are > documented. Can anyone explain the meaning of the "pattern" field and > what exactly its value means? > > Thanks in advance for all your helpful answers! :) From vsevostiyanov at gmail.com Mon Mar 16 17:08:00 2015 From: vsevostiyanov at gmail.com (=?UTF-8?B?0JLQsNGB0LjQu9C40Lkg0KHQtdCy0L7RgdGC0YzRj9C90L7Qsg==?=) Date: Mon, 16 Mar 2015 20:08:00 +0300 Subject: Warning: sieve: file storage: Active sieve script symlink is broken: Invalid/unknown path to storage Message-ID: dovecot version: 2.2.16 (3d8a054a93a9) sieve configuration: plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } For each of the users, ~/.dovecot.sieve is a symlink to ~/sieve/roundcube.sieve Until yesterday, everything was working find. But starting from yesterday's update, the log is filling up with messages like: Mar 16 19:12:32 cs12986 dovecot: lda(postmaster at contoso.com ): Warning: sieve: file storage: Active sieve script symlink /home/virtualmail/contoso.com/it/.dovecot.sieve is broken: Invalid/unknown path to storage (points to /home/virtualmail/contoso.com/it/sieve ). Nothing was changed in the config files. What might have gone wrong? From rs at sys4.de Mon Mar 16 17:16:03 2015 From: rs at sys4.de (Robert Schetterer) Date: Mon, 16 Mar 2015 18:16:03 +0100 Subject: Warning: sieve: file storage: Active sieve script symlink is broken: Invalid/unknown path to storage In-Reply-To: References: Message-ID: <55070FD3.8020500@sys4.de> Am 16.03.2015 um 18:08 schrieb ??????? ???????????: > dovecot version: > 2.2.16 (3d8a054a93a9) > > sieve configuration: > plugin { > sieve = ~/.dovecot.sieve > sieve_dir = ~/sieve > } > > For each of the users, ~/.dovecot.sieve is a symlink to > ~/sieve/roundcube.sieve > > Until yesterday, everything was working find. But starting from yesterday's > update, the log is filling up with messages like: > Mar 16 19:12:32 cs12986 dovecot: lda(postmaster at contoso.com > ): Warning: sieve: file storage: Active sieve > script symlink /home/virtualmail/contoso.com/it/.dovecot.sieve > is broken: Invalid/unknown path to > storage (points to /home/virtualmail/contoso.com/it/sieve > ). > > Nothing was changed in the config files. > > What might have gone wrong? > latest is 15.03.2015 00:33 Released Pigeonhole v0.4.7.rc2 for Dovecot v2.2.16 do you have this ? Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From a.helwig at heinlein-support.de Mon Mar 16 17:24:59 2015 From: a.helwig at heinlein-support.de (Andre Helwig) Date: Mon, 16 Mar 2015 18:24:59 +0100 Subject: Patch SERVICE_FIRST_STATUS_TIMEOUT Message-ID: <550711EB.6070503@heinlein-support.de> Hi list, we had a lot of trouble with our Dovecot Replicating-Cluster installation authenticating against LDAP Service. We ran in a timeout and what happened was that we got more and more search requests running against on our ldap server. And on the dovecot site the process got killed after a couple of seconds. So we ran into a race condition and our LDAP Server was under heavy load. We got a log entry like this: "Initial status notification not received in 30 seconds, killing the process" So we had to apply the attached patch to our Source based Dovecot Cluster. Could you please make a configurable Parameter for our Patched value? That would be great to get this parameter Configurable. Thanks Andre Helwig -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-0 Fax: 030 / 405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin -------------- next part -------------- A non-text attachment was scrubbed... Name: increase_master_timeout.patch Type: text/x-patch Size: 461 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: OpenPGP digital signature URL: From vsevostiyanov at gmail.com Mon Mar 16 17:30:08 2015 From: vsevostiyanov at gmail.com (=?UTF-8?B?0JLQsNGB0LjQu9C40Lkg0KHQtdCy0L7RgdGC0YzRj9C90L7Qsg==?=) Date: Mon, 16 Mar 2015 20:30:08 +0300 Subject: Warning: sieve: file storage: Active sieve script symlink is broken: Invalid/unknown path to storage In-Reply-To: <55070FD3.8020500@sys4.de> References: <55070FD3.8020500@sys4.de> Message-ID: pigeonhole-version: line 9: info: DEBUG: Sieve name is Pigeonhole Sieve. pigeonhole-version: line 14: info: DEBUG: Sieve version 0.4.7.rc2. 2015-03-16 20:16 GMT+03:00 Robert Schetterer : > Am 16.03.2015 um 18:08 schrieb ??????? ???????????: > > dovecot version: > > 2.2.16 (3d8a054a93a9) > > > > sieve configuration: > > plugin { > > sieve = ~/.dovecot.sieve > > sieve_dir = ~/sieve > > } > > > > For each of the users, ~/.dovecot.sieve is a symlink to > > ~/sieve/roundcube.sieve > > > > Until yesterday, everything was working find. But starting from > yesterday's > > update, the log is filling up with messages like: > > Mar 16 19:12:32 cs12986 dovecot: lda(postmaster at contoso.com > > ): Warning: sieve: file storage: Active > sieve > > script symlink /home/virtualmail/contoso.com/it/.dovecot.sieve > > is broken: Invalid/unknown path to > > storage (points to /home/virtualmail/contoso.com/it/sieve > > ). > > > > Nothing was changed in the config files. > > > > What might have gone wrong? > > > > latest is > 15.03.2015 00:33 > Released Pigeonhole v0.4.7.rc2 for Dovecot v2.2.16 > > do you have this ? > > > Best Regards > MfG Robert Schetterer > > -- > [*] sys4 AG > > http://sys4.de, +49 (89) 30 90 46 64 > Franziskanerstra?e 15, 81669 M?nchen > > Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 > Vorstand: Patrick Ben Koetter, Marc Schiffbauer > Aufsichtsratsvorsitzender: Florian Kirstein > From stephan at rename-it.nl Mon Mar 16 18:14:39 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 16 Mar 2015 19:14:39 +0100 Subject: Warning: sieve: file storage: Active sieve script symlink is broken: Invalid/unknown path to storage In-Reply-To: References: <55070FD3.8020500@sys4.de> Message-ID: <55071D8F.3040800@rename-it.nl> On 3/16/2015 6:30 PM, ??????? ??????????? wrote: > pigeonhole-version: line 9: info: DEBUG: Sieve name is Pigeonhole Sieve. > pigeonhole-version: line 14: info: DEBUG: Sieve version 0.4.7.rc2. > > 2015-03-16 20:16 GMT+03:00 Robert Schetterer : >> Am 16.03.2015 um 18:08 schrieb ??????? ???????????: >>> dovecot version: >>> 2.2.16 (3d8a054a93a9) >>> >>> sieve configuration: >>> plugin { >>> sieve = ~/.dovecot.sieve >>> sieve_dir = ~/sieve >>> } >>> >>> For each of the users, ~/.dovecot.sieve is a symlink to >>> ~/sieve/roundcube.sieve >>> >>> Until yesterday, everything was working find. But starting from >> yesterday's >>> update, the log is filling up with messages like: >>> Mar 16 19:12:32 cs12986 dovecot: lda(postmaster at contoso.com >>> ): Warning: sieve: file storage: Active >> sieve >>> script symlink /home/virtualmail/contoso.com/it/.dovecot.sieve >>> is broken: Invalid/unknown path to >>> storage (points to /home/virtualmail/contoso.com/it/sieve >>> ). >>> >>> Nothing was changed in the config files. >>> >>> What might have gone wrong? First of all, what are those and parts in the log? Is your log mangled somehow? It is very difficult for me to explain how those would emerge at those locations in the logs. To find your actual problem, could you show your logs while mail_debug=yes is enabled? Your full dovecot -n output may also be helpful. Regards, Stephan. From benv-dovecot at junerules.com Sun Mar 15 15:08:04 2015 From: benv-dovecot at junerules.com (Wouter de Geus) Date: Sun, 15 Mar 2015 16:08:04 +0100 Subject: [Dovecot-news] Released Pigeonhole v0.4.7.rc2 for Dovecot v2.2.16 In-Reply-To: <5504C54A.9010502@rename-it.nl> References: <5504C54A.9010502@rename-it.nl> Message-ID: <20150315150803.GA14550@uil.winnipeg.nl> Hej folks, Today I upgraded from Dovecot 2.2.15 to 2.2.16 and while I was at it also went from pigeonhole 0.4.5 to 0.4.7-rc2. After upgrading I started getting these errors in the dovecot-deliver-error log: ===== lda(test at domain.nl): Error: sieve: file storage: Failed to normalize active script directory (path=/home/vpopmail/domains/domain.nl/test/.sieve): No such file or directory lda(test at domain.nl): Error: sieve: Failed to access user storage (temporary failure) ===== Is this due to a configuration change or is this a bug? Above problem is as it says due to the missing .sieve directory (not all my users have them), but it worked fine in 0.4.5 without those directories. Thanks, Wouter. From joan at ayma.cat Sun Mar 15 21:57:27 2015 From: joan at ayma.cat (=?UTF-8?B?Sm9hbiBBeW3DoA==?=) Date: Sun, 15 Mar 2015 22:57:27 +0100 Subject: sieve script not applied Message-ID: <55060047.9010406@ayma.cat> Hi everyone, I installed all-new server postfix+amavis+dovecot and pigeonhole. Clients, thunderbird and roundcube creates sieve scripts ok and changed ones are seen by eatch other, so server at 4190 accepts the scripts and ensure they are created. But filters onto new mail isn't applied. Example: # rule:[guifinet] if header :contains :comparator "i;octet" "subject" "[guifi-" { fileinto "INBOX.guifinet"; } Isn't the rule correct? Didn't find any troubleshooting on this topic. Any ideas? Thanks. From tss at iki.fi Mon Mar 16 20:36:55 2015 From: tss at iki.fi (Timo Sirainen) Date: Mon, 16 Mar 2015 22:36:55 +0200 Subject: Patch SERVICE_FIRST_STATUS_TIMEOUT In-Reply-To: <550711EB.6070503@heinlein-support.de> References: <550711EB.6070503@heinlein-support.de> Message-ID: <676D43BC-D2EF-43E7-902F-B5CA6038F5BC@iki.fi> On 16 Mar 2015, at 19:24, Andre Helwig wrote: > > Hi list, > we had a lot of trouble with our Dovecot Replicating-Cluster > installation authenticating against LDAP Service. > > We ran in a timeout and what happened was that we got more and more > search requests running against on our ldap server. > > And on the dovecot site the process got killed after a couple of > seconds. So we ran into a race condition and our LDAP Server was under > heavy load. > We got a log entry like this: "Initial status notification not received > in 30 seconds, killing the process" What process? auth process? > So we had to apply the attached patch to our Source based Dovecot Cluster. > > Could you please make a configurable Parameter for our Patched value? > > That would be great to get this parameter Configurable. That's the wrong solution though. A process initialization is supposed to take less than a second always. If something like LDAP initialization is taking minutes, this waiting should be done after the process initialization has finished. I thought the LDAP initialization code was asynchronous though, unless you're using sasl_bind=yes or tls=yes? From stephan at rename-it.nl Mon Mar 16 21:10:38 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 16 Mar 2015 22:10:38 +0100 Subject: Warning: sieve: file storage: Active sieve script symlink is broken: Invalid/unknown path to storage In-Reply-To: <55071D8F.3040800@rename-it.nl> References: <55070FD3.8020500@sys4.de> <55071D8F.3040800@rename-it.nl> Message-ID: <550746CE.2010006@rename-it.nl> On 3/16/2015 7:14 PM, Stephan Bosch wrote: > On 3/16/2015 6:30 PM, ??????? ??????????? wrote: >> pigeonhole-version: line 9: info: DEBUG: Sieve name is Pigeonhole Sieve. >> pigeonhole-version: line 14: info: DEBUG: Sieve version 0.4.7.rc2. >> >> 2015-03-16 20:16 GMT+03:00 Robert Schetterer : >>> Am 16.03.2015 um 18:08 schrieb ??????? ???????????: >>>> dovecot version: >>>> 2.2.16 (3d8a054a93a9) >>>> >>>> sieve configuration: >>>> plugin { >>>> sieve = ~/.dovecot.sieve >>>> sieve_dir = ~/sieve >>>> } >>>> >>>> For each of the users, ~/.dovecot.sieve is a symlink to >>>> ~/sieve/roundcube.sieve >>>> >>>> Until yesterday, everything was working find. But starting from >>> yesterday's >>>> update, the log is filling up with messages like: >>>> Mar 16 19:12:32 cs12986 dovecot: lda(postmaster at contoso.com >>>> ): Warning: sieve: file storage: Active >>> sieve >>>> script symlink /home/virtualmail/contoso.com/it/.dovecot.sieve >>>> is broken: Invalid/unknown path to >>>> storage (points to /home/virtualmail/contoso.com/it/sieve >>>> ). >>>> >>>> Nothing was changed in the config files. >>>> >>>> What might have gone wrong? > First of all, what are those > > > > and > > > > parts in the log? Is your log mangled somehow? It is very difficult for > me to explain how those would emerge at those locations in the logs. > > To find your actual problem, could you show your logs while > mail_debug=yes is enabled? Your full dovecot -n output may also be helpful. Ok, this was a stupid one. Fixed: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/07e25d04d8f2 Regards, Stephan. From stephan at rename-it.nl Mon Mar 16 21:28:27 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 16 Mar 2015 22:28:27 +0100 Subject: [Dovecot-news] Released Pigeonhole v0.4.7.rc2 for Dovecot v2.2.16 In-Reply-To: <20150315150803.GA14550@uil.winnipeg.nl> References: <5504C54A.9010502@rename-it.nl> <20150315150803.GA14550@uil.winnipeg.nl> Message-ID: <55074AFB.8010309@rename-it.nl> On 3/15/2015 4:08 PM, Wouter de Geus wrote: > Hej folks, > > Today I upgraded from Dovecot 2.2.15 to 2.2.16 and while I was at it also went from pigeonhole 0.4.5 to 0.4.7-rc2. > After upgrading I started getting these errors in the dovecot-deliver-error log: > ===== > lda(test at domain.nl): Error: sieve: file storage: Failed to normalize active script directory (path=/home/vpopmail/domains/domain.nl/test/.sieve): No such file or directory > lda(test at domain.nl): Error: sieve: Failed to access user storage (temporary failure) > ===== > > Is this due to a configuration change or is this a bug? > Above problem is as it says due to the missing .sieve directory (not all my users have them), but it worked fine in 0.4.5 without those directories. Fixed: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/cf8007c9a74f Regards, Stephan. From tss at iki.fi Mon Mar 16 21:30:55 2015 From: tss at iki.fi (Timo Sirainen) Date: Mon, 16 Mar 2015 23:30:55 +0200 Subject: Patch SERVICE_FIRST_STATUS_TIMEOUT In-Reply-To: <676D43BC-D2EF-43E7-902F-B5CA6038F5BC@iki.fi> References: <550711EB.6070503@heinlein-support.de> <676D43BC-D2EF-43E7-902F-B5CA6038F5BC@iki.fi> Message-ID: On 16 Mar 2015, at 22:36, Timo Sirainen wrote: > >> So we had to apply the attached patch to our Source based Dovecot Cluster. >> >> Could you please make a configurable Parameter for our Patched value? >> >> That would be great to get this parameter Configurable. > > That's the wrong solution though. A process initialization is supposed to take less than a second always. If something like LDAP initialization is taking minutes, this waiting should be done after the process initialization has finished. > > I thought the LDAP initialization code was asynchronous though, unless you're using sasl_bind=yes or tls=yes? http://hg.dovecot.org/dovecot-2.2/rev/0a17875f0ece should help with this. I did a bunch of other cleanup commits also, which are more or less required to avoid invalid errors from rapidly recreating auth processes that just die immediately. From chayes at afo.net Tue Mar 17 01:14:15 2015 From: chayes at afo.net (Cliff Hayes) Date: Mon, 16 Mar 2015 20:14:15 -0500 Subject: passwd file for quota Message-ID: <55077FE7.40701@afo.net> I need to implement quota. I read as much as I could including http://wiki2.dovecot.org/HowTo/SimpleVirtualInstall But my current /etc/passwd file does not have the password in it as shown in dovecot docs ... there is an x where the password should be. I created a program that creates a passwd file in the proper format with the proper extra quota fields for all users (except it has an x where the password should be). I plan to store it in /etc/dovecot How do I get dovecot to stop looking at /etc/passwd and start looking at /etc/dovecot/passwd? And hopefully that will not affect how it gets the password (pam) because when users change their password from webmail of course dovecot will need to be looking in the same place. # dovecot -n output # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.29.2.el6.x86_64 x86_64 Scientific Linux release 6.5 (Carbon) auth_failure_delay = 5 secs auth_username_format = %Ln auth_verbose_passwords = plain first_valid_uid = 9 mail_access_groups = mail mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_after = /dovecotSieveAfter/ sieve_dir = ~/sieve } protocols = imap lmtp sieve service managesieve-login { inet_listener sieve { port = 4190 } } ssl_ca = Hello Dovecot users, The rc2 release had a couple of new, rather small, but very annoying bugs. First of all, the path for a file storage wasn't actually normalized, due to a stupid last-minute change before the release. This caused the personal script to be skipped with a warning when there were e.g. duplicate slashes in the path. Second, due to recent changes, the file storage would complain with a temporary error if the directory where the active script symlink would be located was missing. Changelog v0.4.7 (unchanged): * editheader extension: Made protection against addition and deletion of headers configurable separately. Also, the `Received' and `Auto-Submitted' headers are no longer protected against addition by default. * Turned message envelope address parse errors into warnings. * The interpreter now accepts non-standard domain names, e.g. containing '_'. + Implemented the Sieve index extension (RFC 5260). + Implemented support for the mboxmetadata and servermetadata extensions (RFC 5490). + Implemented new sieve commands for the doveadm command line utility. These commands are currently limited to ManageSieve operations, but the other current sieve tools will be migrated to doveadm in the near future as well. + Added more debug output about binary up-to-date checking. + Added script metadata to binary dump output. - Fixed Sieve script binary up-to-date checking by normalizing the script location. - The Sieve interpreter now flushes the duplicate database during start phase of result execution rather than commit phase. This makes sure locks on the duplicate database are released as soon as possible, preventing contention. - Performed a few optimizations in the lexical scanner of the language. - Fixed bug in `:matches' match-type that made a pattern without wildcards match as if there were a '*' at the beginning. - Fixed crash in validation of the string parameter of the comparator tag. - extprograms extension: Made sure supplemental group privileges are also dropped. This was a problem reported by Debian lintian. - Fixed bug in handling of binary errors for action side-effects and message overrides. - file script storage: Restructured storage initialization to address backwards compatibility issues. - dict script storage: Fixed small memory allocation bug. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.7.rc3.tar.gz http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.7.rc3.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From dovecot.org at veggiechinese.net Tue Mar 17 04:54:49 2015 From: dovecot.org at veggiechinese.net (Will Yardley) Date: Mon, 16 Mar 2015 21:54:49 -0700 Subject: Dovecot 2.1.7 still accepting SSLv3 though disabled? In-Reply-To: <20150315144200.Horde.hUl8CfdWQMVR3ndNIO887w1@horde.andreasschulze.de> References: <20150315093555.GC1513@blackhole.tracing.lan> <20150315144200.Horde.hUl8CfdWQMVR3ndNIO887w1@horde.andreasschulze.de> Message-ID: <20150317045449.GB84921@aura.veggiechinese.net> On Sun, Mar 15, 2015 at 02:42:00PM +0100, A. Schulze wrote: > Thomas Preissler: > The logging is right, but SSLv3 isn't used. > Today it's not uncommon that application /log/ SSLv3, where they /mean/ TLS1.x > > Some days ago where TLSv1 became available there wasn't a great > difference between SSLv3 and TLSv1 > So Developers reused large portions of code. That's what you see here.. > > > But when I explicitely test for SSLv3 support I get > > > > $ openssl s_client -connect $SERVERIP:993 -ssl3 > > > > CONNECTED(00000003) > > 140683835029160:error:14094410:SSL > > routines:SSL3_READ_BYTES:sslv3 alert handshake > > failure:s3_pkt.c:1260:SSL alert number 40 > > 140683835029160:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl > > handshake failure:s3_pkt.c:598: > > That is the ultimate prove your server have SSLv3 disabled. Another fun trick for testing is nmap -p 993 --script ssl-enum-ciphers foo.example.com You'll then see (if you've got a new enough version) something like: [...] 993/tcp open imaps | ssl-enum-ciphers: | TLSv1.0: | ciphers: | TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA - strong | TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA - strong | TLS_RSA_WITH_3DES_EDE_CBC_SHA - strong | TLS_RSA_WITH_AES_128_CBC_SHA - strong | TLS_RSA_WITH_AES_256_CBC_SHA - strong | TLS_RSA_WITH_RC4_128_MD5 - strong | TLS_RSA_WITH_RC4_128_SHA - strong [...] w From skdovecot at smail.inf.fh-brs.de Tue Mar 17 07:20:46 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Tue, 17 Mar 2015 08:20:46 +0100 (CET) Subject: passwd file for quota In-Reply-To: <55077FE7.40701@afo.net> References: <55077FE7.40701@afo.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 16 Mar 2015, Cliff Hayes wrote: > password should be). I plan to store it in /etc/dovecot > How do I get dovecot to stop looking at /etc/passwd and start looking at > /etc/dovecot/passwd? Do you mean: http://wiki2.dovecot.org/AuthDatabase/PasswdFile - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVQfVznz1H7kL/d9rAQJfawf9FgQwwMDCAtbKzMG9vCD5hzSJPUB941h2 F+xLzeja3SyShrxstw15lwOeb+fJWQYzpjJlPVdu+UpO9wKascsli5aLBO1tBfHD tMM1wsJMbB8RCesjCrHcJ+/kVv3Rou1nGHo5L4FoAXmA13G9hWj3auiWBgrR0lzZ Z2YY1jYTcHfXivZ9lj4zWdmvvSBKIVguHXBwQdngfHQzvr4WCmZpk+LkHNOWbXr9 klXFEoEZ6cntKT/G7R8vcgb+mIOT8hu9EDHD0aoTk9exdVeB8ic8u+kaGkW3TBlR bLXsUxgpTBcHo27RjJOQGpY1S+/DbP8nmfIdt5VVEXcp+13+6CR2gQ== =Wi+d -----END PGP SIGNATURE----- From a.helwig at heinlein-support.de Tue Mar 17 09:31:21 2015 From: a.helwig at heinlein-support.de (Andre Helwig) Date: Tue, 17 Mar 2015 10:31:21 +0100 Subject: Patch SERVICE_FIRST_STATUS_TIMEOUT In-Reply-To: References: <550711EB.6070503@heinlein-support.de> <676D43BC-D2EF-43E7-902F-B5CA6038F5BC@iki.fi> Message-ID: <5507F469.50705@heinlein-support.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 03/16/2015 10:30 PM, Timo Sirainen wrote: > On 16 Mar 2015, at 22:36, Timo Sirainen wrote: >> >>> So we had to apply the attached patch to our Source based >>> Dovecot Cluster. >>> >>> Could you please make a configurable Parameter for our Patched >>> value? >>> >>> That would be great to get this parameter Configurable. >> >> That's the wrong solution though. A process initialization is >> supposed to take less than a second always. If something like >> LDAP initialization is taking minutes, this waiting should be >> done after the process initialization has finished. >> >> I thought the LDAP initialization code was asynchronous though, >> unless you're using sasl_bind=yes or tls=yes? > > http://hg.dovecot.org/dovecot-2.2/rev/0a17875f0ece should help with > this. I did a bunch of other cleanup commits also, which are more > or less required to avoid invalid errors from rapidly recreating > auth processes that just die immediately. > Wow that was a fast response also with patching. We will test this asap. Thanks a lot Timo. Cheers, Andre Helwig - -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-0 Fax: 030 / 405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJVB/RpAAoJEAoTNwRDnEhRM5MH+QFq+RY95LpZr9qoVmQ+ABnb BH46N5nJxcKc2zRjATfotmEwpxjeVLEH0YrziTkvkTHUd4ehMrqBalxbGVpe/Y1T fjVB4iddM3uHrqzpUSweiS8D4l3Rh6xUL1m36pHtajPKfO6V+bauM0APpogFEXHc GNuUNLCh8IYAEezHTOiMhDSCob4Gx/cr5XOTqRMw/w093nkp1gyfoUUCCj6ZwMVy rgTgBmjebZlx2Kf70G6Q5fa2QXKhDwpWThPqlSrsiC6it10rPAKMjTC/mJ9O2/8/ 9jRGeFYqKVzMCUvPlnD568kXxsLTsXOZRkUDspgQmrJ06Y4VuXCz5KqEfDW/KsI= =Zk4U -----END PGP SIGNATURE----- From linuxmail at 4lin.net Tue Mar 17 14:13:18 2015 From: linuxmail at 4lin.net (Denny Fuchs) Date: Tue, 17 Mar 2015 15:13:18 +0100 Subject: http://xi.rename-it.nl 2.2.16-1~auto+10/11/12/13: segfault /var/lib/dovecot/auth OR (db_ldap_connect_delayed): Message-ID: <7AB9C174-DF75-46F4-95A8-BC2315874CF5@4lin.net> hi, I testing around with Kolab with LDAP and have some strange problems, with the auto generated packages for Wheezy. On my first test VM, I have 2:2.2.16~rc1-1~auto+4 installed, and everything works as expected. On a new VM I have 2.2.16-1~auto+13_amd64.deb packages and if I include LDAP backend, I get a segmentation fault on /usr/lib/dovecot/auth: root at kolab:/etc/dovecot/conf.d# /usr/lib/dovecot/auth Segmentation fault with auto+12 and auto+13 If I downgrade to auto+11 or auto+10 I get: Mar 17 14:42:59 kolab dovecot: auth: Panic: file db-ldap.c: line 1228 (db_ldap_connect_delayed): assertion failed: (conn->to == NULL) Mar 17 14:42:59 kolab dovecot: auth: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7b5ef) [0x7f57d815c5ef] -> /usr/lib/dovecot/libdovecot.so.0(+0x7b64e) [0x7f57d815c64e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f57d810b0d5] -> /usr/lib/dovecot/modules/auth/libauthdb_ldap.so(+0x5455) [0x7f57d70e2455] -> dovecot/auth(userdb_init+0x1a) [0x42765a] -> dovecot/auth(auths_init+0xc9) [0x40d979] -> dovecot/auth(main+0x2b5) [0x40cc85] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f57d7523ead] -> dovecot/auth() [0x40cee9] Mar 17 14:42:59 kolab dovecot: auth: Fatal: master: service(auth): child 1161 killed with signal 6 (core dumps disabled) Mar 17 14:42:59 kolab dovecot: master: Error: service(auth): command startup failed, throttling for 2 secs Mar 17 14:42:59 kolab dovecot: pop3-login: Disconnected: Auth process broken (disconnected before auth was ready, waited 0 secs): user=<>, rip=::1, lip=::1, secured, session=<1QMUJXwRTQAAAAAAAAAAAAAAAAAAAAAB> ii dovecot-sieve 2:2.2.16~rc1-1~auto+4 amd64 secure POP3/IMAP server - Sieve filters support root at kolab:/# ldd /usr/lib/dovecot/modules/auth/libauthdb_ldap.so linux-vdso.so.1 => (0x00007ffc4a1c6000) libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007effe2d1a000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007effe2b12000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007effe2786000) liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007effe2577000) libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007effe2361000) libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007effe2145000) libgnutls.so.26 => /usr/lib/x86_64-linux-gnu/libgnutls.so.26 (0x00007effe1e85000) libgcrypt.so.11 => /lib/x86_64-linux-gnu/libgcrypt.so.11 (0x00007effe1c06000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007effe19e9000) /lib64/ld-linux-x86-64.so.2 (0x00007effe3180000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007effe17e5000) libtasn1.so.3 => /usr/lib/x86_64-linux-gnu/libtasn1.so.3 (0x00007effe15d3000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007effe13bc000) libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007effe11aa000) libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007effe0fa6000) Then I copied from my first VM the libauthdb_ldap.so file on the new test VM to /usr/lib/dovecot/modules/auth/libauthdb_ldap.so (remember: ~rc1-1~auto+4) and the result: root at kolab:/etc/dovecot/conf.d# /usr/lib/dovecot/auth Error: LDAP: binding failed (dn cn=Directory Manager): Invalid credentials and, it works again ... linux-vdso.so.1 => (0x00007fffa00c6000) libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007f94b7ec3000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f94b7cbb000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f94b792f000) liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007f94b7720000) libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f94b750a000) libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007f94b72ee000) libgnutls.so.26 => /usr/lib/x86_64-linux-gnu/libgnutls.so.26 (0x00007f94b702e000) libgcrypt.so.11 => /lib/x86_64-linux-gnu/libgcrypt.so.11 (0x00007f94b6daf000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f94b6b92000) /lib64/ld-linux-x86-64.so.2 (0x00007f94b8328000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f94b698e000) libtasn1.so.3 => /usr/lib/x86_64-linux-gnu/libtasn1.so.3 (0x00007f94b677c000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f94b6565000) libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f94b6353000) libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007f94b614f000) So, it seems, that there is something broken with the autogenerated packages, or with my fresh installed Wheey VM (KVM AMD64) in combination with LDAP. any suggestions? cu denny -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 163 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jaimeventura at gmail.com Tue Mar 17 16:13:31 2015 From: jaimeventura at gmail.com (Jaime Ventura) Date: Tue, 17 Mar 2015 16:13:31 +0000 Subject: Dovecot current number of connections being used. Message-ID: Hey, Is there an easy way to get how many connections are being handled at a moment? I wanted to have that number over the time and see: - Trends on usage during the day - how close is it from reaching the maximum number of connections. - ... I'm searching for a solution without recurring to something like netstat | grep imap | wc -l. Thanks, Jaime From a.helwig at heinlein-support.de Tue Mar 17 16:17:43 2015 From: a.helwig at heinlein-support.de (Andre Helwig) Date: Tue, 17 Mar 2015 17:17:43 +0100 Subject: Dovecot current number of connections being used. In-Reply-To: References: Message-ID: <550853A7.7060909@heinlein-support.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 doveadm who should help On 03/17/2015 05:13 PM, Jaime Ventura wrote: > Hey, Is there an easy way to get how many connections are being > handled at a moment? > > I wanted to have that number over the time and see: - Trends on > usage during the day - how close is it from reaching the maximum > number of connections. - ... > > I'm searching for a solution without recurring to something like > netstat | grep imap | wc -l. > > Thanks, Jaime > - -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-0 Fax: 030 / 405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJVCFOnAAoJEAoTNwRDnEhRJhIIAIHCCVaJmYO/qIfWjdDNb4EB qDuh+ssiMucMzMZFM7wsvYEYc3P0ERb3F8nMdUgyhJWDWU/RJTVniTRrXUkpdPDb 7ozDDaiXAQ3vZ747+RTtoOW7UgSZ08hdUJeQdIEU74Hy2Mf7tWvsqFdpj3qfkXAL piPHnHzl4HFZkuQisrqMyhUJXfICbJHMsH+FtDrpERI4WbVRd0tWtnRgH805Bsf+ 7GVj7kT67jM0rkdp3yD/6fcCt1n8nAdlK5TnjQOrbXHwOMmdV55EsQusgqlou44/ 6H4ZsokrJolFCmA9ayle6bR8qN5YfMzVtGvmkvpc1mUfSi9ANt6vZLfT5A5t9vc= =IZu3 -----END PGP SIGNATURE----- From chayes at afo.net Tue Mar 17 17:55:47 2015 From: chayes at afo.net (Cliff Hayes) Date: Tue, 17 Mar 2015 12:55:47 -0500 Subject: passwd file for quota In-Reply-To: References: <55077FE7.40701@afo.net> Message-ID: <55086AA3.2070600@afo.net> Yes and No. It's confusing to me which is why I ask. Per my initial email my password source is PAM. It's the userdb I'm concerned about ... which dovecot is using /etc/passwd. So dovecot is getting user information from passwd file; password information from PAM. I need to add extra fields for qouta but can't add them to /etc/passwd so I have to create a passwd with the extra fields for dovecot to use. How do I get dovecot to look at the new file for user info? I looked at your link and then did a grep on passdb which appears in ... auth-checkpassword.conf.ext auth-deny.conf.ext auth-ldap.conf.ext auth-master.conf.ext auth-passwdfile.conf.ext auth-sql.conf.ext:passdb auth-static.conf.ext auth-system.conf.ext auth-vpopmail.conf.ext: I'm guessing I need to look in auth-system.conf.ext since I use system users. In there it says ... passdb { driver = pam ... so I don't think I should change that. On 3/17/2015 2:20 AM, Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Mon, 16 Mar 2015, Cliff Hayes wrote: > >> password should be). I plan to store it in /etc/dovecot >> How do I get dovecot to stop looking at /etc/passwd and start looking >> at /etc/dovecot/passwd? > > Do you mean: http://wiki2.dovecot.org/AuthDatabase/PasswdFile > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVQfVznz1H7kL/d9rAQJfawf9FgQwwMDCAtbKzMG9vCD5hzSJPUB941h2 > F+xLzeja3SyShrxstw15lwOeb+fJWQYzpjJlPVdu+UpO9wKascsli5aLBO1tBfHD > tMM1wsJMbB8RCesjCrHcJ+/kVv3Rou1nGHo5L4FoAXmA13G9hWj3auiWBgrR0lzZ > Z2YY1jYTcHfXivZ9lj4zWdmvvSBKIVguHXBwQdngfHQzvr4WCmZpk+LkHNOWbXr9 > klXFEoEZ6cntKT/G7R8vcgb+mIOT8hu9EDHD0aoTk9exdVeB8ic8u+kaGkW3TBlR > bLXsUxgpTBcHo27RjJOQGpY1S+/DbP8nmfIdt5VVEXcp+13+6CR2gQ== > =Wi+d > -----END PGP SIGNATURE----- > From andre.peters at debinux.de Tue Mar 17 18:40:10 2015 From: andre.peters at debinux.de (=?windows-1252?Q?Andr=E9_Peters?=) Date: Tue, 17 Mar 2015 19:40:10 +0100 Subject: http://xi.rename-it.nl 2.2.16-1~auto+10/11/12/13: segfault /var/lib/dovecot/auth OR (db_ldap_connect_delayed): In-Reply-To: <7AB9C174-DF75-46F4-95A8-BC2315874CF5@4lin.net> References: <7AB9C174-DF75-46F4-95A8-BC2315874CF5@4lin.net> Message-ID: <5508750A.3020809@debinux.de> Hi, I mailed Timo S. about this problem this morning. He fixed it minutes later. Thanks again! :-) http://hg.dovecot.org/dovecot-2.2/rev/870cb73e5960 Andr? Am 17.03.2015 um 15:13 schrieb Denny Fuchs: > hi, > > I testing around with Kolab with LDAP and have some strange problems, with the auto generated packages for Wheezy. > > On my first test VM, I have 2:2.2.16~rc1-1~auto+4 installed, and everything works as expected. On a new VM I have 2.2.16-1~auto+13_amd64.deb packages and if I include LDAP backend, I get a segmentation fault on /usr/lib/dovecot/auth: > > root at kolab:/etc/dovecot/conf.d# /usr/lib/dovecot/auth > Segmentation fault > > with auto+12 and auto+13 > > If I downgrade to auto+11 or auto+10 I get: > > Mar 17 14:42:59 kolab dovecot: auth: Panic: file db-ldap.c: line 1228 (db_ldap_connect_delayed): assertion failed: (conn->to == NULL) > Mar 17 14:42:59 kolab dovecot: auth: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7b5ef) [0x7f57d815c5ef] -> /usr/lib/dovecot/libdovecot.so.0(+0x7b64e) [0x7f57d815c64e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f57d810b0d5] -> /usr/lib/dovecot/modules/auth/libauthdb_ldap.so(+0x5455) [0x7f57d70e2455] -> dovecot/auth(userdb_init+0x1a) [0x42765a] -> dovecot/auth(auths_init+0xc9) [0x40d979] -> dovecot/auth(main+0x2b5) [0x40cc85] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f57d7523ead] -> dovecot/auth() [0x40cee9] > Mar 17 14:42:59 kolab dovecot: auth: Fatal: master: service(auth): child 1161 killed with signal 6 (core dumps disabled) > Mar 17 14:42:59 kolab dovecot: master: Error: service(auth): command startup failed, throttling for 2 secs > Mar 17 14:42:59 kolab dovecot: pop3-login: Disconnected: Auth process broken (disconnected before auth was ready, waited 0 secs): user=<>, rip=::1, lip=::1, secured, session=<1QMUJXwRTQAAAAAAAAAAAAAAAAAAAAAB> > > ii dovecot-sieve 2:2.2.16~rc1-1~auto+4 amd64 secure POP3/IMAP server - Sieve filters support > root at kolab:/# ldd /usr/lib/dovecot/modules/auth/libauthdb_ldap.so > linux-vdso.so.1 => (0x00007ffc4a1c6000) > libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007effe2d1a000) > librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007effe2b12000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007effe2786000) > liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007effe2577000) > libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007effe2361000) > libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007effe2145000) > libgnutls.so.26 => /usr/lib/x86_64-linux-gnu/libgnutls.so.26 (0x00007effe1e85000) > libgcrypt.so.11 => /lib/x86_64-linux-gnu/libgcrypt.so.11 (0x00007effe1c06000) > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007effe19e9000) > /lib64/ld-linux-x86-64.so.2 (0x00007effe3180000) > libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007effe17e5000) > libtasn1.so.3 => /usr/lib/x86_64-linux-gnu/libtasn1.so.3 (0x00007effe15d3000) > libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007effe13bc000) > libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007effe11aa000) > libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007effe0fa6000) > > > Then I copied from my first VM the libauthdb_ldap.so file on the new test VM to /usr/lib/dovecot/modules/auth/libauthdb_ldap.so (remember: ~rc1-1~auto+4) and the result: > > root at kolab:/etc/dovecot/conf.d# /usr/lib/dovecot/auth > Error: LDAP: binding failed (dn cn=Directory Manager): Invalid credentials > > and, it works again ... > > linux-vdso.so.1 => (0x00007fffa00c6000) > libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007f94b7ec3000) > librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f94b7cbb000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f94b792f000) > liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007f94b7720000) > libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f94b750a000) > libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007f94b72ee000) > libgnutls.so.26 => /usr/lib/x86_64-linux-gnu/libgnutls.so.26 (0x00007f94b702e000) > libgcrypt.so.11 => /lib/x86_64-linux-gnu/libgcrypt.so.11 (0x00007f94b6daf000) > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f94b6b92000) > /lib64/ld-linux-x86-64.so.2 (0x00007f94b8328000) > libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f94b698e000) > libtasn1.so.3 => /usr/lib/x86_64-linux-gnu/libtasn1.so.3 (0x00007f94b677c000) > libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f94b6565000) > libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f94b6353000) > libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007f94b614f000) > > > So, it seems, that there is something broken with the autogenerated packages, or with my fresh installed Wheey VM (KVM AMD64) in combination with LDAP. > > any suggestions? > > cu denny > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5622 bytes Desc: S/MIME Cryptographic Signature URL: From jtam.home at gmail.com Tue Mar 17 19:39:27 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 17 Mar 2015 12:39:27 -0700 (PDT) Subject: Dovecot current number of connections being used. In-Reply-To: References: Message-ID: Jaime Ventura writes: > Is there an easy way to get how many connections are being handled at a > moment? This works: doveadm who -1 2>/dev/null | wc -l If yopu only want to coutn IMAP connections (not POP3), doveadm who -1 2>/dev/null | grep -Fc imap Joseph Tam From roland at micite.net Tue Mar 17 22:09:25 2015 From: roland at micite.net (Roland van Laar) Date: Tue, 17 Mar 2015 23:09:25 +0100 Subject: FreeBSD ZFS maildir to mdbox Message-ID: <5508A615.3060501@micite.net> Hello, I'm converting my mailbox from Maildir to mdbox.. The Maildir is from an 1.2 server. The new server is a virtual FreeBSD ZFS server. The conversion is not working, I'm getting either segfaults or the sync just quits with errors. I also did set with mmap_disable=yes. This didn't resolve the problem. How can I fix this? The same conversion on a virtual ubuntu 14.04 box works with mentions of "filename has the wrong S value" and Corrupted index errors. The FreeBSD errors and dovecot -n are included below: dsync(vagrant): Error: read(./Maildir/cur/1296038598.29562.mail.micite.net,S=20542:2,Sa) failed: Cached message size smaller than expected (20542 < 20640, box=INBOX, UID=475) dsync(vagrant): Error: Maildir filename has wrong S value, renamed the file from ./Maildir/cur/1296038598.29562.mail.micite.net,S=20542:2,Sa to ./Maildir/cur/1296038598.29562.mail.micite.net,S=20640:2,Sa dsync(vagrant): Error: Corrupted index cache file ./Maildir/dovecot.index.cache: Broken physical size for mail UID 475 dsync(vagrant): Error: write(/home/vagrant/mdbox/storage/m.12) failed: Invalid argument dsync(vagrant): Error: copy: i_stream_read(./Maildir/cur/1296038598.29562.mail.micite.net,S=20542:2,Sa) failed: Cached message size smaller than expected (20542 < 20640, box=INBOX, UID=475) dsync(vagrant): Error: read(./Maildir/cur/1296038598.29562.mail.micite.net,S=20542:2,Sa) failed: Cached message size smaller than expected (20542 < 20640, box=INBOX, UID=475) (uid=475, box=INBOX) dsync(vagrant): Error: Mailbox INBOX: read(msg input) failed: Cached message size smaller than expected (20542 < 20640, box=INBOX, UID=475) dsync(vagrant): Error: read(./Maildir/cur/1296038598.29562.mail.micite.net,S=20542:2,Sa) failed: Cached message size smaller than expected (20542 < 20640, box=INBOX, UID=475) (uid=475, box=INBOX) dsync(vagrant): Panic: file mail-index-transaction-update.c: line 964 (mail_index_update_ext): assertion failed: (seq > 0 && (seq <= mail_index_view_get_messages_count(t->view) || seq <= t->last_new_seq)) Abort (core dumped) (gdb) core doveadm.core Core was generated by `doveadm'. Program terminated with signal 6, Aborted. #0 0x00000008013f8a1a in ?? () (gdb) bt #0 0x00000008013f8a1a in ?? () #1 0x00000008013f7149 in ?? () #2 0x0000000000000000 in ?? () or with no core dump: % doveadm sync maildir:./Maildir dsync(vagrant): Error: read(./Maildir/cur/1296466748.64525.mail.micite.net,S=7629:2,Sa) failed: Cached message size smaller than expected (7629 < 7694, box=INBOX, UID=899) dsync(vagrant): Error: Maildir filename has wrong S value, renamed the file from ./Maildir/cur/1296466748.64525.mail.micite.net,S=7629:2,Sa to ./Maildir/cur/1296466748.64525.mail.micite.net,S=7694:2,Sa dsync(vagrant): Error: Corrupted index cache file ./Maildir/dovecot.index.cache: Broken physical size for mail UID 899 dsync(vagrant): Error: write(/home/vagrant/mdbox/storage/m.11) failed: Invalid argument dsync(vagrant): Error: copy: i_stream_read(./Maildir/cur/1296466748.64525.mail.micite.net,S=7629:2,Sa) failed: Cached message size smaller than expected (7629 < 7694, box=INBOX, UID=899) dsync(vagrant): Error: read(./Maildir/cur/1296466748.64525.mail.micite.net,S=7629:2,Sa) failed: Cached message size smaller than expected (7629 < 7694, box=INBOX, UID=899) (uid=899, box=INBOX) dsync(vagrant): Error: Mailbox INBOX: read(msg input) failed: Cached message size smaller than expected (7629 < 7694, box=INBOX, UID=899) dsync(vagrant): Error: read(./Maildir/cur/1296466748.64525.mail.micite.net,S=7629:2,Sa) failed: Cached message size smaller than expected (7629 < 7694, box=INBOX, UID=899) (uid=899, box=INBOX) % dovecot -n # 2.2.16: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 10.1-RELEASE-p6 amd64 mail_location = mdbox:~/mdbox namespace { inbox = yes location = prefix = separator = . } passdb { args = /usr/local/etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } protocols = lmtp imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = vmail } ssl = required ssl_cert = I am getting "Connection to storage server failed." in roundcubemail. On doing a google search If find: http://help.directadmin.com/item.php?id=473 Which puts the problem at dovecot. I did get a dovecot update back on 3-5 and that would match to my one roundcube mail user complaining and the delays I am seeing with thunderbird and outlook users. So with my setup, I am trying to figure what are the index files to delete. Say for myself, I am seeing: /home/vmail/htt-consult.com/rgm/dovecot.index /home/vmail/htt-consult.com/rgm/dovecot.index.cache /home/vmail/htt-consult.com/rgm/dovecot.index.log /home/vmail/htt-consult.com/rgm/.Drafts/dovecot.index /home/vmail/htt-consult.com/rgm/.Drafts/dovecot.index.cache /home/vmail/htt-consult.com/rgm/.Drafts/dovecot.index.log /home/vmail/htt-consult.com/rgm/.Sent/dovecot.index /home/vmail/htt-consult.com/rgm/.Sent/dovecot.index.cache /home/vmail/htt-consult.com/rgm/.Sent/dovecot.index.log /home/vmail/htt-consult.com/rgm/.Spam/dovecot.index /home/vmail/htt-consult.com/rgm/.Spam/dovecot.index.cache /home/vmail/htt-consult.com/rgm/.Spam/dovecot.index.log /home/vmail/htt-consult.com/rgm/.Trash/dovecot.index /home/vmail/htt-consult.com/rgm/.Trash/dovecot.index.cache /home/vmail/htt-consult.com/rgm/.Trash/dovecot.index.log Do I delete all of these? The index.cache files are quite large. From tss at iki.fi Tue Mar 17 23:47:31 2015 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Mar 2015 01:47:31 +0200 Subject: Proxying of non "plain" SASL mechnisms. In-Reply-To: <1424890760.8096.42.camel@one.com> References: <1424890760.8096.42.camel@one.com> Message-ID: On 25 Feb 2015, at 20:59, Peter Mogensen wrote: > So, why not just extend the support for proxy authentication forwarding > to any single-handskake SASL-IR mechanism, which doesn't use > channel-binding? (which includes PLAIN, but also GS2-KRB5, and possibly > others). Yeah, I guess it would work for several of the auth mechanisms. It's a lot of work though and requires some larger changes to how authentication works. I don't currently see it being worth the effort, but I wouldn't mind if somebody else implements it. I guess the parts would be: - Some flag to auth mechanisms that allow proxying based on their initial SASL response. - A new auth setting to enable auth proxying for mechanisms that support it. - If auth proxying is enabled, perform passdb lookup on non-plaintext auth on the initial SASL response. Return "finished" to the auth client with some "mech-proxy=y" extra field, so it knows to start proxying the SASL session to the destination server. - Implementation of the above for all the mechanisms that support it.. - login-common to support sending the same initial response to the target server and proxying the rest of the authentication. (Possibly somehow integrate this with Dovecot's lib-sasl, but not sure if this is needed/useful.) From tss at iki.fi Wed Mar 18 00:28:03 2015 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Mar 2015 02:28:03 +0200 Subject: Dovecot 2.2.15 issues with global ACL In-Reply-To: <617e67c2c6754e528c3b93fa89ebb912@mmambx4.global.ad> References: <195b4828cef3457090bac045659187eb@mmambx4.global.ad> <617e67c2c6754e528c3b93fa89ebb912@mmambx4.global.ad> Message-ID: <9CCE5036-D840-499F-B03B-2EE80009F1EB@iki.fi> Does this work correctly in v2.2.16? > On 10 Feb 2015, at 14:07, Ovidiu Moldovan wrote: > > It seems after checking the code that global ACL file functionality was changed at 2.2.14 so that only 1 rule is used, not multiple rules. > > This is not documented or said anywhere into change logs. > > Br, > Ova > > -----Original Message----- > From: dovecot [mailto:dovecot-bounces at dovecot.org] On Behalf Of Ovidiu Moldovan > Sent: 10. helmikuuta 2015 10:26 > To: dovecot at dovecot.org > Subject: Dovecot 2.2.15 issues with global ACL > > Hello, > > We have upgraded from Dovecot 2.2.13 to 2.2.15 and we are using global ACL file. > > The content of the ACL file is as following: > > * owner r > INBOX owner lrwstipekxa > INBOX/* owner lrwstipekxa > user owner rwstipekxa > user/* owner rwstipekxa > > > This worked fine but after update any user cannot see any folders from under the INBOX, also they cannot create any new folders. > > Error into the logs are like this: > > Debug: acl: Mailbox not in dovecot-acl-list: INBOX > Debug: acl: Mailbox not in dovecot-acl-list: INBOX/Chats > Debug: acl: Mailbox not in dovecot-acl-list: INBOX/Draft > Debug: acl: Mailbox not in dovecot-acl-list: INBOX/INBOX .... > > > I can only see one acl change log at version 2.2.14: http://www.dovecot.org/list/dovecot-news/2014-October/000276.html > > acl: Global ACL file now supports "quotes" around patterns. > > > But I could not find the reason why the global acl file does not work anymore? > > Br, > Ova From skdovecot at smail.inf.fh-brs.de Wed Mar 18 06:17:30 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 18 Mar 2015 07:17:30 +0100 (CET) Subject: passwd file for quota In-Reply-To: <55086AA3.2070600@afo.net> References: <55077FE7.40701@afo.net> <55086AA3.2070600@afo.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 17 Mar 2015, Cliff Hayes wrote: > Yes and No. > It's confusing to me which is why I ask. > Per my initial email my password source is PAM. > It's the userdb I'm concerned about ... which dovecot is using /etc/passwd. > So dovecot is getting user information from passwd file; password information > from PAM. > I need to add extra fields for qouta but can't add them to /etc/passwd so I > have to create a passwd with the extra fields for dovecot to use. How do I > get dovecot to look at the new file for user info? > > I looked at your link and then did a grep on passdb which appears in ... > auth-checkpassword.conf.ext > auth-deny.conf.ext > auth-ldap.conf.ext > auth-master.conf.ext > auth-passwdfile.conf.ext > auth-sql.conf.ext:passdb > auth-static.conf.ext > auth-system.conf.ext > auth-vpopmail.conf.ext: > I'm guessing I need to look in auth-system.conf.ext since I use system users. > In there it says ... > passdb { > driver = pam > ... so I don't think I should change that. See http://wiki2.dovecot.org/Authentication Dovecot utilizes separate password and user databases and you can have multiple of each. That means, too, that the auth-*.ext files are more samples than fixed configuration options. You probably have included auth-system.conf.ext by enabling: 10-auth.conf:#!include auth-system.conf.ext ^ no # here in auth-system.conf.ext you'll find the passdb { driver = pam ..}, which you'll keep unchanged, and a userdb {} section. Comment the current userdb section, which most probably has driver = /etc/passwd, and add the proper userdb { driver = passwd-file .... } You could create your own conf file, too. > > On 3/17/2015 2:20 AM, Steffen Kaiser wrote: >> On Mon, 16 Mar 2015, Cliff Hayes wrote: >> >>> password should be). I plan to store it in /etc/dovecot >>> How do I get dovecot to stop looking at /etc/passwd and start looking >>> at /etc/dovecot/passwd? >> >> Do you mean: http://wiki2.dovecot.org/AuthDatabase/PasswdFile - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVQkYenz1H7kL/d9rAQKD3AgAtbyqpF4ZpCeCbF4Uw+enznWX3XacxOVQ 5/bbXIynZwRykSzc4aLs9dfmgTPZ1l8pUalamObqz8Epqn1nEuVRzb/ivEd+AJk8 dT1U+b3fxdIz0IlT4lIu4BxG7pqvbxBemiP4D3VenEreQSqSRTb/yIJu0eM34QtF GF3qZOpuKOEwF0fve6vYXtgx9CUw6Ifp+dUHawDPV6xAWzc5+cxIUtqGinIX0Z8O wuzunpqC0Gcc2k4pMTYFwuKs9mkkG5KSZKmvNUoE7AiD1yO8I0QKQFVynrcc8dmM ZOPnFMi2JH567KuIiMIcHspclyhFp7znQYsLAcPxgYIAP5auWfRv6Q== =YV0d -----END PGP SIGNATURE----- From toni at solu.fi Wed Mar 18 07:26:37 2015 From: toni at solu.fi (Toni Mattila) Date: Wed, 18 Mar 2015 09:26:37 +0200 Subject: FreeBSD ZFS maildir to mdbox In-Reply-To: <5508A615.3060501@micite.net> References: <5508A615.3060501@micite.net> Message-ID: <550928AD.10807@solu.fi> Hi, On 18-Mar-15 00:09, Roland van Laar wrote: > I'm converting my mailbox from Maildir to mdbox.. > The Maildir is from an 1.2 server. > The same conversion on a virtual ubuntu 14.04 box works with mentions of > "filename has the wrong S value" and Corrupted index errors. You should fix the Maildir files first to have correct S= (size) on them. Older maildrops and qmail likes to create wrong sizes and newer dovecots rely on that S= to be correct. You can use http://www.dovecot.org/tools/maildir-size-fix.pl or similar script to fix your existing maildirs. Best Regards, Toni From apm at one.com Wed Mar 18 07:50:33 2015 From: apm at one.com (Peter Mogensen) Date: Wed, 18 Mar 2015 08:50:33 +0100 Subject: Proxying of non "plain" SASL mechnisms. In-Reply-To: References: <1424890760.8096.42.camel@one.com> Message-ID: <55092E49.5070801@one.com> On 2015-03-18 00:47, Timo Sirainen wrote: > - If auth proxying is enabled, perform passdb lookup on non-plaintext > auth on the initial SASL response. Return "finished" to the auth > client with some "mech-proxy=y" extra field, so it knows to start > proxying the SASL session to the destination server. This is actually the tricky part. To perform a problemer passdb lookup, the proxy will have to be able to decode the user from the SASL IR even though it might not be able to authenticate. This requires knowledge of the SASL IR format (like extracting authz-id/authn-id from the PLAIN argument). That might not be possible for all SASL mechanisms. With GS2-KRB5 you can always get authz-id. On the other hand, mechanisms like GSSAPI (which would work for other reasons) requires the actually perform the authentication before authz-id can be known. So ... it might be a bit difficult to precisely define which mechanism such a feature covers and which it doesn't. /Peter From jaimeventura at gmail.com Wed Mar 18 09:32:08 2015 From: jaimeventura at gmail.com (Jaime Ventura) Date: Wed, 18 Mar 2015 09:32:08 +0000 Subject: Dovecot current number of connections being used. In-Reply-To: References: Message-ID: great! Thank you all for the replies. On Tue, Mar 17, 2015 at 7:39 PM, Joseph Tam wrote: > Jaime Ventura writes: > > Is there an easy way to get how many connections are being handled at a >> moment? >> > > This works: > > doveadm who -1 2>/dev/null | wc -l > > If yopu only want to coutn IMAP connections (not POP3), > > doveadm who -1 2>/dev/null | grep -Fc imap > > Joseph Tam > From rgm at htt-consult.com Wed Mar 18 13:12:27 2015 From: rgm at htt-consult.com (Robert Moskowitz) Date: Wed, 18 Mar 2015 09:12:27 -0400 Subject: Solved - Re: Roundcubemail problem with Dovecot In-Reply-To: <5508B779.9090206@htt-consult.com> References: <5508B779.9090206@htt-consult.com> Message-ID: <550979BB.90407@htt-consult.com> From: http://www.roundcubeforum.net/index.php?topic=9886.0 I saw the comment about telneting from the host to its domain, and for some reason I recalled that I had to hardcode the server's IP address into /etc/hosts. And I recently moved the host to a new IP address. QED On 03/17/2015 07:23 PM, Robert Moskowitz wrote: > I am getting "Connection to storage server failed." in roundcubemail. > On doing a google search If find: > > http://help.directadmin.com/item.php?id=473 > > Which puts the problem at dovecot. I did get a dovecot update back on > 3-5 and that would match to my one roundcube mail user complaining and > the delays I am seeing with thunderbird and outlook users. > > So with my setup, I am trying to figure what are the index files to > delete. Say for myself, I am seeing: > > /home/vmail/htt-consult.com/rgm/dovecot.index > /home/vmail/htt-consult.com/rgm/dovecot.index.cache > /home/vmail/htt-consult.com/rgm/dovecot.index.log > /home/vmail/htt-consult.com/rgm/.Drafts/dovecot.index > /home/vmail/htt-consult.com/rgm/.Drafts/dovecot.index.cache > /home/vmail/htt-consult.com/rgm/.Drafts/dovecot.index.log > /home/vmail/htt-consult.com/rgm/.Sent/dovecot.index > /home/vmail/htt-consult.com/rgm/.Sent/dovecot.index.cache > /home/vmail/htt-consult.com/rgm/.Sent/dovecot.index.log > /home/vmail/htt-consult.com/rgm/.Spam/dovecot.index > /home/vmail/htt-consult.com/rgm/.Spam/dovecot.index.cache > /home/vmail/htt-consult.com/rgm/.Spam/dovecot.index.log > /home/vmail/htt-consult.com/rgm/.Trash/dovecot.index > /home/vmail/htt-consult.com/rgm/.Trash/dovecot.index.cache > /home/vmail/htt-consult.com/rgm/.Trash/dovecot.index.log > > Do I delete all of these? The index.cache files are quite large. > > From chayes at afo.net Wed Mar 18 16:15:52 2015 From: chayes at afo.net (Cliff Hayes) Date: Wed, 18 Mar 2015 11:15:52 -0500 Subject: passwd file for quota In-Reply-To: References: <55077FE7.40701@afo.net> <55086AA3.2070600@afo.net> Message-ID: <5509A4B8.8060009@afo.net> Thanks! That Worked :) On 3/18/2015 1:17 AM, Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Tue, 17 Mar 2015, Cliff Hayes wrote: > >> Yes and No. >> It's confusing to me which is why I ask. >> Per my initial email my password source is PAM. >> It's the userdb I'm concerned about ... which dovecot is using >> /etc/passwd. >> So dovecot is getting user information from passwd file; password >> information from PAM. >> I need to add extra fields for qouta but can't add them to /etc/passwd >> so I have to create a passwd with the extra fields for dovecot to use. >> How do I get dovecot to look at the new file for user info? >> >> I looked at your link and then did a grep on passdb which appears in ... >> auth-checkpassword.conf.ext >> auth-deny.conf.ext >> auth-ldap.conf.ext >> auth-master.conf.ext >> auth-passwdfile.conf.ext >> auth-sql.conf.ext:passdb >> auth-static.conf.ext >> auth-system.conf.ext >> auth-vpopmail.conf.ext: >> I'm guessing I need to look in auth-system.conf.ext since I use system >> users. In there it says ... >> passdb { >> driver = pam >> ... so I don't think I should change that. > > See http://wiki2.dovecot.org/Authentication > > Dovecot utilizes separate password and user databases and you can have > multiple of each. That means, too, that the auth-*.ext files are more > samples than fixed configuration options. > > You probably have included auth-system.conf.ext by enabling: > > 10-auth.conf:#!include auth-system.conf.ext > ^ no # here > > in auth-system.conf.ext you'll find the passdb { driver = pam ..}, which > you'll keep unchanged, and a userdb {} section. Comment the current > userdb section, which most probably has driver = /etc/passwd, and add > the proper > > userdb { > driver = passwd-file > .... > } > > You could create your own conf file, too. > >> >> On 3/17/2015 2:20 AM, Steffen Kaiser wrote: >>> On Mon, 16 Mar 2015, Cliff Hayes wrote: >>> >>>> password should be). I plan to store it in /etc/dovecot >>>> How do I get dovecot to stop looking at /etc/passwd and start looking >>>> at /etc/dovecot/passwd? >>> >>> Do you mean: http://wiki2.dovecot.org/AuthDatabase/PasswdFile > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVQkYenz1H7kL/d9rAQKD3AgAtbyqpF4ZpCeCbF4Uw+enznWX3XacxOVQ > 5/bbXIynZwRykSzc4aLs9dfmgTPZ1l8pUalamObqz8Epqn1nEuVRzb/ivEd+AJk8 > dT1U+b3fxdIz0IlT4lIu4BxG7pqvbxBemiP4D3VenEreQSqSRTb/yIJu0eM34QtF > GF3qZOpuKOEwF0fve6vYXtgx9CUw6Ifp+dUHawDPV6xAWzc5+cxIUtqGinIX0Z8O > wuzunpqC0Gcc2k4pMTYFwuKs9mkkG5KSZKmvNUoE7AiD1yO8I0QKQFVynrcc8dmM > ZOPnFMi2JH567KuIiMIcHspclyhFp7znQYsLAcPxgYIAP5auWfRv6Q== > =YV0d > -----END PGP SIGNATURE----- > From ck+dovecot at bl4ckb0x.de Wed Mar 18 19:40:12 2015 From: ck+dovecot at bl4ckb0x.de (Conrad Kostecki) Date: Wed, 18 Mar 2015 20:40:12 +0100 Subject: Support for multiple =?UTF-8?Q?passwords=3F?= Message-ID: Hi! Currently, the passwords are stored in plaintext for my dovecot, as I am still using cram-md5 AND digest-md5. I have still to offer that, as I have some deprecated clients, therefore, I am unable to hash at least those passwords for that accounts. I've found on the Wiki: > In future it's possible that Dovecot could support multiple passwords > in different schemes for a single user. Is there any news about this? Are there still any plans to support this maybe in future? For my understanding, that would solve my problem, that I could define a password in both schemes (cram and digest) and don't have to use plaintext password? Cheers Conrad From h.reindl at thelounge.net Wed Mar 18 19:46:02 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 18 Mar 2015 20:46:02 +0100 Subject: Support for multiple passwords? In-Reply-To: References: Message-ID: <5509D5FA.5080802@thelounge.net> Am 18.03.2015 um 20:40 schrieb Conrad Kostecki: > Hi! > Currently, the passwords are stored in plaintext for my dovecot, as I am > still using cram-md5 AND digest-md5. > I have still to offer that, as I have some deprecated clients, > therefore, I am unable to hash at least those passwords for that accounts. > > I've found on the Wiki: >> In future it's possible that Dovecot could support multiple passwords >> in different schemes for a single user. > > Is there any news about this? Are there still any plans to support this > maybe in future? > For my understanding, that would solve my problem, that I could define a > password in both schemes (cram and digest) and don't have to use > plaintext password? if you would read http://en.wikipedia.org/wiki/CRAM-MD5 and understand how CRAM-MD5 works you would know that you just can't store cram because the whole purpose is that it changes all the time -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From ck+dovecot at bl4ckb0x.de Wed Mar 18 19:56:57 2015 From: ck+dovecot at bl4ckb0x.de (Conrad Kostecki) Date: Wed, 18 Mar 2015 20:56:57 +0100 Subject: Support for multiple =?UTF-8?Q?passwords=3F?= In-Reply-To: <5509D5FA.5080802@thelounge.net> References: <5509D5FA.5080802@thelounge.net> Message-ID: <548b03877b75cb61da5af9f34338686a@bl4ckb0x.de> Am 2015-03-18 20:46, schrieb Reindl Harald: > Am 18.03.2015 um 20:40 schrieb Conrad Kostecki: >> Hi! >> Currently, the passwords are stored in plaintext for my dovecot, as I >> am >> still using cram-md5 AND digest-md5. >> I have still to offer that, as I have some deprecated clients, >> therefore, I am unable to hash at least those passwords for that >> accounts. >> >> I've found on the Wiki: >>> In future it's possible that Dovecot could support multiple passwords >>> in different schemes for a single user. >> >> Is there any news about this? Are there still any plans to support >> this >> maybe in future? >> For my understanding, that would solve my problem, that I could define >> a >> password in both schemes (cram and digest) and don't have to use >> plaintext password? > > if you would read http://en.wikipedia.org/wiki/CRAM-MD5 and understand > how CRAM-MD5 works you would know that you just can't store cram > because the whole purpose is that it changes all the time Maybe I am totally wrong, but according to the Wiki, if I would be use using CRAM-MD5 without DIGEST-MD5, the password could be stored not in plain text but instead in a cram-md5 scheme? At least, that had worked for me in a test setup. But I will have a look. > http://wiki.dovecot.org/Authentication/PasswordSchemes > For example if you're going to use CRAM-MD5 authentication, the > password needs to be stored in either PLAIN or CRAM-MD5 scheme. Cheers Conrad From h.reindl at thelounge.net Wed Mar 18 20:00:02 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 18 Mar 2015 21:00:02 +0100 Subject: Support for multiple passwords? In-Reply-To: <548b03877b75cb61da5af9f34338686a@bl4ckb0x.de> References: <5509D5FA.5080802@thelounge.net> <548b03877b75cb61da5af9f34338686a@bl4ckb0x.de> Message-ID: <5509D942.90403@thelounge.net> Am 18.03.2015 um 20:56 schrieb Conrad Kostecki: > Am 2015-03-18 20:46, schrieb Reindl Harald: >> Am 18.03.2015 um 20:40 schrieb Conrad Kostecki: >>> Hi! >>> Currently, the passwords are stored in plaintext for my dovecot, as I am >>> still using cram-md5 AND digest-md5. >>> I have still to offer that, as I have some deprecated clients, >>> therefore, I am unable to hash at least those passwords for that >>> accounts. >>> >>> I've found on the Wiki: >>>> In future it's possible that Dovecot could support multiple passwords >>>> in different schemes for a single user. >>> >>> Is there any news about this? Are there still any plans to support this >>> maybe in future? >>> For my understanding, that would solve my problem, that I could define a >>> password in both schemes (cram and digest) and don't have to use >>> plaintext password? >> >> if you would read http://en.wikipedia.org/wiki/CRAM-MD5 and understand >> how CRAM-MD5 works you would know that you just can't store cram >> because the whole purpose is that it changes all the time > > Maybe I am totally wrong, > but according to the Wiki, if I would be use using CRAM-MD5 without > DIGEST-MD5, the password could be stored not in plain text but instead > in a cram-md5 scheme? > At least, that had worked for me in a test setup. But I will have a look. only in a broken and unsecure implementation - or how do you store "arbitrary string of random digits, a timestamp"? http://en.wikipedia.org/wiki/CRAM-MD5 Challenge: The server sends a base64-encoded string to the client. Before encoding, it could be any random string, but the standard that currently defines CRAM-MD5 says that it is in the format of a Message-ID email header value (including angle brackets) and includes an arbitrary string of random digits, a timestamp, and the server's fully qualified domain name. >> http://wiki.dovecot.org/Authentication/PasswordSchemes >> For example if you're going to use CRAM-MD5 authentication, the >> password needs to be stored in either PLAIN or CRAM-MD5 scheme -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From rick at havokmon.com Wed Mar 18 20:10:59 2015 From: rick at havokmon.com (Rick Romero) Date: Wed, 18 Mar 2015 15:10:59 -0500 Subject: Support for multiple passwords? In-Reply-To: <5509D942.90403@thelounge.net> References: <5509D5FA.5080802@thelounge.net> <548b03877b75cb61da5af9f34338686a@bl4ckb0x.de> <5509D942.90403@thelounge.net> Message-ID: <20150318151059.Horde.kK-9omJNSWnND9i0RsMFvg1@www.vfemail.net> Quoting Reindl Harald : > Am 18.03.2015 um 20:56 schrieb Conrad Kostecki: >> Am 2015-03-18 20:46, schrieb Reindl Harald: >>> Am 18.03.2015 um 20:40 schrieb Conrad Kostecki: >>>> Hi! >>>> Currently, the passwords are stored in plaintext for my dovecot, as I >>>> am >>>> still using cram-md5 AND digest-md5. >>>> I have still to offer that, as I have some deprecated clients, >>>> therefore, I am unable to hash at least those passwords for that >>>> accounts. >>>> >>>> I've found on the Wiki: >>>>> In future it's possible that Dovecot could support multiple passwords >>>>> in different schemes for a single user. >>>> >>>> Is there any news about this? Are there still any plans to support this >>>> maybe in future? >>>> For my understanding, that would solve my problem, that I could >>>> define a >>>> password in both schemes (cram and digest) and don't have to use >>>> plaintext password? >>> >>> if you would read http://en.wikipedia.org/wiki/CRAM-MD5 and understand >>> how CRAM-MD5 works you would know that you just can't store cram >>> because the whole purpose is that it changes all the time >> >> Maybe I am totally wrong, >> but according to the Wiki, if I would be use using CRAM-MD5 without >> DIGEST-MD5, the password could be stored not in plain text but instead >> in a cram-md5 scheme? >> At least, that had worked for me in a test setup. But I will have a look. > > only in a broken and unsecure implementation - or how do you store > "arbitrary string of random digits, a timestamp"? > > http://en.wikipedia.org/wiki/CRAM-MD5 > > Challenge: The server sends a base64-encoded string to the client. > Before encoding, it could be any random string, but the standard that > currently defines CRAM-MD5 says that it is in the format of a Message-ID > email header value (including angle brackets) and includes an arbitrary > string of random digits, a timestamp, and the server's fully qualified > domain name. > Too much irrelevant information. Goal: Don't store cleartext passwords. Question: Do your clients support PLAIN authentication and SSL? The authentication method is (mostly) independent of the storage method. Only CRAM requires either a clear text password or Dovecot's CRAM 'cleartext workaround'. If you use PLAIN, then your passwords can be stored encrypted. If the clients support SSL, then you can require SSL/TLS and encrypt the password (and ALL the content) at the transport layer. Rick From ck+dovecot at bl4ckb0x.de Wed Mar 18 20:56:01 2015 From: ck+dovecot at bl4ckb0x.de (Conrad Kostecki) Date: Wed, 18 Mar 2015 21:56:01 +0100 Subject: Support for multiple =?UTF-8?Q?passwords=3F?= In-Reply-To: <20150318151059.Horde.kK-9omJNSWnND9i0RsMFvg1@www.vfemail.net> References: <5509D5FA.5080802@thelounge.net> <548b03877b75cb61da5af9f34338686a@bl4ckb0x.de> <5509D942.90403@thelounge.net> <20150318151059.Horde.kK-9omJNSWnND9i0RsMFvg1@www.vfemail.net> Message-ID: Am 2015-03-18 21:10, schrieb Rick Romero: > Quoting Reindl Harald : > >> Am 18.03.2015 um 20:56 schrieb Conrad Kostecki: >>> Am 2015-03-18 20:46, schrieb Reindl Harald: >>>> Am 18.03.2015 um 20:40 schrieb Conrad Kostecki: >>>>> Hi! >>>>> Currently, the passwords are stored in plaintext for my dovecot, as >>>>> I >>>>> am >>>>> still using cram-md5 AND digest-md5. >>>>> I have still to offer that, as I have some deprecated clients, >>>>> therefore, I am unable to hash at least those passwords for that >>>>> accounts. >>>>> >>>>> I've found on the Wiki: >>>>>> In future it's possible that Dovecot could support multiple >>>>>> passwords >>>>>> in different schemes for a single user. >>>>> >>>>> Is there any news about this? Are there still any plans to support > this >>>>> maybe in future? >>>>> For my understanding, that would solve my problem, that I could >>>>> define a >>>>> password in both schemes (cram and digest) and don't have to use >>>>> plaintext password? >>>> >>>> if you would read http://en.wikipedia.org/wiki/CRAM-MD5 and >>>> understand >>>> how CRAM-MD5 works you would know that you just can't store cram >>>> because the whole purpose is that it changes all the time >>> >>> Maybe I am totally wrong, >>> but according to the Wiki, if I would be use using CRAM-MD5 without >>> DIGEST-MD5, the password could be stored not in plain text but >>> instead >>> in a cram-md5 scheme? >>> At least, that had worked for me in a test setup. But I will have a > look. >> >> only in a broken and unsecure implementation - or how do you store >> "arbitrary string of random digits, a timestamp"? >> >> http://en.wikipedia.org/wiki/CRAM-MD5 >> >> Challenge: The server sends a base64-encoded string to the client. >> Before encoding, it could be any random string, but the standard that >> currently defines CRAM-MD5 says that it is in the format of a >> Message-ID >> email header value (including angle brackets) and includes an >> arbitrary >> string of random digits, a timestamp, and the server's fully qualified >> domain name. >> > > Too much irrelevant information. Goal: Don't store cleartext > passwords. > > Question: Do your clients support PLAIN authentication and SSL? Not all. I know, that this just sucks in the year 2015. > If you use PLAIN, then your passwords can be stored encrypted. If the > clients support SSL, then you can require SSL/TLS and encrypt the > password (and ALL the content) at the transport layer. That's what I am planning. Only enable PLAIN and force SSL/TLS. I hope, that till end of the year, I can shutdown that finally and switch to force SSL/TLS and disable CRAM-MD5/DIGEST-MD5, so my passwords can be encrypted. From rick at havokmon.com Wed Mar 18 21:08:50 2015 From: rick at havokmon.com (Rick Romero) Date: Wed, 18 Mar 2015 16:08:50 -0500 Subject: Support for multiple passwords? In-Reply-To: References: <5509D5FA.5080802@thelounge.net> <548b03877b75cb61da5af9f34338686a@bl4ckb0x.de> <5509D942.90403@thelounge.net> <20150318151059.Horde.kK-9omJNSWnND9i0RsMFvg1@www.vfemail.net> Message-ID: <20150318160850.Horde.x9OkvlNz9ZfRaiQXEs3xoA2@www.vfemail.net> Quoting Conrad Kostecki : > Am 2015-03-18 21:10, schrieb Rick Romero: >> Quoting Reindl Harald : >> >>> Am 18.03.2015 um 20:56 schrieb Conrad Kostecki: >>>> Am 2015-03-18 20:46, schrieb Reindl Harald: >>>>> Am 18.03.2015 um 20:40 schrieb Conrad Kostecki: >>>>>> Hi! >>>>>> Currently, the passwords are stored in plaintext for my dovecot, as I >>>>>> am >>>>>> still using cram-md5 AND digest-md5. >>>>>> I have still to offer that, as I have some deprecated clients, >>>>>> therefore, I am unable to hash at least those passwords for that >>>>>> accounts. >>>>>> >>>>>> I've found on the Wiki: >>>>>>> In future it's possible that Dovecot could support multiple >>>>>>> passwords >>>>>>> in different schemes for a single user. >>>>>> >>>>>> Is there any news about this? Are there still any plans to support >> >> this >>>>>> maybe in future? >>>>>> For my understanding, that would solve my problem, that I could >>>>>> define a >>>>>> password in both schemes (cram and digest) and don't have to use >>>>>> plaintext password? >>>>> >>>>> if you would read http://en.wikipedia.org/wiki/CRAM-MD5 and understand >>>>> how CRAM-MD5 works you would know that you just can't store cram >>>>> because the whole purpose is that it changes all the time >>>> >>>> Maybe I am totally wrong, >>>> but according to the Wiki, if I would be use using CRAM-MD5 without >>>> DIGEST-MD5, the password could be stored not in plain text but instead >>>> in a cram-md5 scheme? >>>> At least, that had worked for me in a test setup. But I will have a >> >> look. >>> only in a broken and unsecure implementation - or how do you store >>> "arbitrary string of random digits, a timestamp"? >>> >>> http://en.wikipedia.org/wiki/CRAM-MD5 >>> >>> Challenge: The server sends a base64-encoded string to the client. >>> Before encoding, it could be any random string, but the standard that >>> currently defines CRAM-MD5 says that it is in the format of a Message-ID >>> email header value (including angle brackets) and includes an arbitrary >>> string of random digits, a timestamp, and the server's fully qualified >>> domain name. >> >> Too much irrelevant information.? Goal: Don't store cleartext passwords. >> >> Question: Do your clients support PLAIN authentication and SSL? > > Not all. I know, that this just sucks in the year 2015. > >> If you use PLAIN, then your passwords can be stored encrypted.? If the >> clients support SSL, then you can require SSL/TLS and encrypt the >> password (and ALL the content) at the transport layer. > > That's what I am planning. Only enable PLAIN and force SSL/TLS.I hope, > that till end of the year, I can shutdown that finally and switch to > force SSL/TLS and disable CRAM-MD5/DIGEST-MD5, so my passwords can be > encrypted. Depending on your options, this could be a good stop-gap solution: https://www.stunnel.org/downloads.html Use stunnel to establish the encrypted connection between the client PC and your server, then the client connects to localhost. From jtam.home at gmail.com Thu Mar 19 00:37:58 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 18 Mar 2015 17:37:58 -0700 (PDT) Subject: Patch for "doveadm -f table" nit (was Re: Dovecot current number of connections being used.) In-Reply-To: References: Message-ID: > doveadm who -1 2>/dev/null | wc -l You have to redirect stderr to /dev/null because that's where the first header line is written to. The default format style (table) is inconsistent with the other formats (flow,pager,tab) that write headers and data to stdout. The following patch will pick this nit. This patch will require modifications to scripts that rely on doveadm writing headers to stderr. For example, the above doveadm command could be modified to doveadm who -1 | grep -vc '^username' -------------------------------------------------------------------------------- --- a/dovecot-2.2.16.rc1/src/doveadm/doveadm-print-table.c Wed Mar 18 15:40:40 2015 +++ b/dovecot-2.2.16.rc1/src/doveadm/doveadm-print-table.c Wed Mar 18 15:41:18 2015 @@ -151,3 +151,3 @@ for (i = 0; i < count; i++) { - if (i > 0) fprintf(stderr, " "); + if (i > 0) printf(" "); @@ -155,3 +155,3 @@ DOVEADM_PRINT_HEADER_FLAG_RIGHT_JUSTIFY) == 0) { - fprintf(stderr, "%-*s", (int)headers[i].length, + printf("%-*s", (int)headers[i].length, headers[i].title); @@ -158,3 +158,3 @@ } else { - fprintf(stderr, "%*s", (int)headers[i].length, + printf("%*s", (int)headers[i].length, headers[i].title); @@ -162,3 +162,3 @@ } - fprintf(stderr, "\n"); + printf("\n"); } -------------------------------------------------------------------------------- Joseph Tam From tss at iki.fi Thu Mar 19 00:49:33 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Mar 2015 02:49:33 +0200 Subject: Patch for "doveadm -f table" nit (was Re: Dovecot current number of connections being used.) In-Reply-To: References: Message-ID: <78D069B1-3D22-456B-B13E-1FD9AC871157@iki.fi> On 19 Mar 2015, at 02:37, Joseph Tam wrote: > > >> doveadm who -1 2>/dev/null | wc -l > > You have to redirect stderr to /dev/null because that's where the > first header line is written to. The default format style (table) > is inconsistent with the other formats (flow,pager,tab) that write > headers and data to stdout. > > The following patch will pick this nit. This patch will require > modifications to scripts that rely on doveadm writing headers to stderr. > For example, the above doveadm command could be modified to > > doveadm who -1 | grep -vc '^username' There's no reason why flow and pager should write headers to stderr because it would always result only in a mess. But instead of changing table headers to write to stdout, I think a better fix would be to make tab formatter write headers to stderr. Including headers in stdout makes it more difficult to write scripts that access the actual data. For example now you can do "doveadm who -1 | sort" and the output will work. If headers were written to stdout you'd have to make it more complicated. Also you can now easily specify what you want to do with the headers, 2>/dev/null if you don't care about them or 2>&1 if you want to include them in stdout (which works even after |sort). So, I'll add in my v2.3 TODO that tab formatter should write to stderr.. From jtam.home at gmail.com Thu Mar 19 01:01:49 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 18 Mar 2015 18:01:49 -0700 (PDT) Subject: Patch for "doveadm -f table" nit (was Re: Dovecot current number of connections being used.) In-Reply-To: <78D069B1-3D22-456B-B13E-1FD9AC871157@iki.fi> References: <78D069B1-3D22-456B-B13E-1FD9AC871157@iki.fi> Message-ID: On Thu, 19 Mar 2015, Timo Sirainen wrote: >> You have to redirect stderr to /dev/null because that's where the >> first header line is written to. The default format style (table) >> is inconsistent with the other formats (flow,pager,tab) that write >> headers and data to stdout. > > There's no reason why flow and pager should write headers to stderr > because it would always result only in a mess. Right. > But instead of changing table headers to write to stdout, I think a > better fix would be to make tab formatter write headers to stderr. That would at least make the behaviour consistent. > Including headers in stdout makes it more difficult to write scripts > that access the actual data. For example now you can do "doveadm who > -1 | sort" and the output will work. If headers were written to stdout > you'd have to make it more complicated. Also you can now easily > specify what you want to do with the headers, 2>/dev/null if you don't > care about them or 2>&1 if you want to include them in stdout (which > works even after |sort). I agree with the easier parsing thing for scripts. It just a little weird when you're doing piping interactively and the header pops out. Another option is to add another flag/formatter that will squelch headers. I'm cool with whatever you implement. Joseph Tam From gedalya at gedalya.net Thu Mar 19 08:30:21 2015 From: gedalya at gedalya.net (Gedalya) Date: Thu, 19 Mar 2015 04:30:21 -0400 Subject: Patch for "doveadm -f table" nit (was Re: Dovecot current number of connections being used.) In-Reply-To: <78D069B1-3D22-456B-B13E-1FD9AC871157@iki.fi> References: <78D069B1-3D22-456B-B13E-1FD9AC871157@iki.fi> Message-ID: <550A891D.8040108@gedalya.net> On 03/18/2015 08:49 PM, Timo Sirainen wrote: > There's no reason why flow and pager should write headers to stderr because it would always result only in a mess. But instead of changing table headers to write to stdout, I think a better fix would be to make tab formatter write headers to stderr. Including headers in stdout makes it more difficult to write scripts that access the actual data. For example now you can do "doveadm who -1 | sort" and the output will work. If headers were written to stdout you'd have to make it more complicated. Also you can now easily specify what you want to do with the headers, 2>/dev/null if you don't care about them or 2>&1 if you want to include them in stdout (which works even after |sort). > > So, I'll add in my v2.3 TODO that tab formatter should write to stderr.. I've been using "-f flow fetch text | sed s/^text=//" when training spamassasin. Couldn't find a straightforward 'fetch raw message'. Seems unnecessarily awkward. Moving headers to stderr would help this, though. From skdovecot at smail.inf.fh-brs.de Thu Mar 19 10:25:47 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 19 Mar 2015 11:25:47 +0100 (CET) Subject: How about an option to disbale headers? (was Re: Patch for "doveadm -f table" nit) In-Reply-To: <550A891D.8040108@gedalya.net> References: <78D069B1-3D22-456B-B13E-1FD9AC871157@iki.fi> <550A891D.8040108@gedalya.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 19 Mar 2015, Gedalya wrote: > On 03/18/2015 08:49 PM, Timo Sirainen wrote: >> There's no reason why flow and pager should write headers to stderr because >> it would always result only in a mess. But instead of changing table >> headers to write to stdout, I think a better fix would be to make tab >> formatter write headers to stderr. Including headers in stdout makes it >> more difficult to write scripts that access the actual data. For example >> now you can do "doveadm who -1 | sort" and the output will work. If headers >> were written to stdout you'd have to make it more complicated. Also you can >> now easily specify what you want to do with the headers, 2>/dev/null if you >> don't care about them or 2>&1 if you want to include them in stdout (which >> works even after |sort). doveadm .... 2> /dev/null will suppress headers _and_ error messages >> So, I'll add in my v2.3 TODO that tab formatter should write to stderr.. > > I've been using "-f flow fetch text | sed s/^text=//" when training > spamassasin. Couldn't find a straightforward 'fetch raw message'. Seems > unnecessarily awkward. Moving headers to stderr would help this, though. > How about a formatter option to enable/disable headers at all? Like, who -H ps h ps --headers Most probably it useful for scripts only, so add an option to suppress all headers will fit most case. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVQqkK3z1H7kL/d9rAQJ/cwgApl5SvuwOcLTDqn0X/A9pCCQvuzX50oyW FZH43AzCVcAShxX5SE8wRhmxTfI71mjQrJkvQu1zbCfh7FAY5lNP7dtTNWZRCxtI PG1B5qfHrB/56SdnEqckYNysrF8XSoKi+0mCzzB3GaSbAWeahvVu93qh1OItfDiA 5Ago3v9l2SlJq/TM9Bo54P5jwemgI9jg9t+5PHBF7RMJY1eQ1Dd4E4IoO/PnqnzX IKL2Xgp3YBm7eNksQWsSowfPntgb56UfLQZrQWG+7pjkDpDsKxZxTStAYcVIH6pj //vTYrDnQP0JPwMe2iqhhrl3NyK4Wk7R1obWcezEcXhxEKFP5fn/Vg== =m6yz -----END PGP SIGNATURE----- From tss at iki.fi Thu Mar 19 10:26:48 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Mar 2015 12:26:48 +0200 Subject: Dovecot Oy merger with Open-Xchange AG Message-ID: <8250CA2C-936F-468D-B5C6-88954DC6FB2F@iki.fi> Hi all, Today I can finally announce that Dovecot Oy company has merged with Open-Xchange AG. This helps us to get more Dovecot developers, support people and so on. Most importantly, eventually it should allow me to get back to doing what I like the most: Designing new and interesting stuff for Dovecot and perfecting the old stuff :) OX is a great match to Dovecot going forward. They also really like open source and share our plans for the future. Nothing big will change as a result of this merger: Dovecot will stay Dovecot with its own name and release schedules. We're not going to force OX and Dovecot to be the same product, other than having a somewhat deeper integration between them. Here are the press release links about it: http://www.dovecot.fi/open-xchange-and-dovecot-announce-merger-to-create-worlds-leading-open-source-messaging-software-provider/ http://www.open-xchange.com/dovecot http://www.open-xchange.com/announcements/18 From wjw at digiware.nl Thu Mar 19 10:51:04 2015 From: wjw at digiware.nl (Willem Jan Withagen) Date: Thu, 19 Mar 2015 11:51:04 +0100 Subject: Patch for "doveadm -f table" nit (was Re: Dovecot current number of connections being used.) In-Reply-To: <550A891D.8040108@gedalya.net> References: <78D069B1-3D22-456B-B13E-1FD9AC871157@iki.fi> <550A891D.8040108@gedalya.net> Message-ID: <550AAA18.2070608@digiware.nl> On 19-3-2015 9:30, Gedalya wrote: > On 03/18/2015 08:49 PM, Timo Sirainen wrote: >> There's no reason why flow and pager should write headers to stderr >> because it would always result only in a mess. But instead of changing >> table headers to write to stdout, I think a better fix would be to >> make tab formatter write headers to stderr. Including headers in >> stdout makes it more difficult to write scripts that access the actual >> data. For example now you can do "doveadm who -1 | sort" and the >> output will work. If headers were written to stdout you'd have to make >> it more complicated. Also you can now easily specify what you want to >> do with the headers, 2>/dev/null if you don't care about them or 2>&1 >> if you want to include them in stdout (which works even after |sort). >> >> So, I'll add in my v2.3 TODO that tab formatter should write to stderr.. > > I've been using "-f flow fetch text | sed s/^text=//" when training > spamassasin. Couldn't find a straightforward 'fetch raw message'. Seems > unnecessarily awkward. Moving headers to stderr would help this, though. I think that that is sort of forgoing the pupsoe of stderr. Moving things to stderr for reasons of parsing and other trivia, just complicates other sysadmin scripts where it is expected that only errors are written to stderr. I would suggest to write all std-info just to regular stdout, and deal with reporting tools just there. just my 2 cts, --WjW From vvu at mcra.fr Thu Mar 19 11:11:13 2015 From: vvu at mcra.fr (Vu Ngoc VU) Date: Thu, 19 Mar 2015 12:11:13 +0100 (CET) Subject: fast doveadm search results Message-ID: hello, I've posted same question here months ago. But obtaine no answer, even not a "you question is stupid". So I don't know what to think about that. My goal is to be able to find mails quickly with "doveadm search". We are using dovecot in a corporate. And some managers, sometimes ask to delete some messages. These messages are "unwanted" ones: mistakenly sent, defamatory... So, that situations appear rarely, but when asked, they expected me to deleted quickly these messages from few to thousands of users mailboxes. Is that possible to change behaviour of Dovecot, some headers like sender, message-id, date are stored in cache for at least 48h? (7 days would be better). I've read this page : http://wiki2.dovecot.org/Design/Indexes/Cache but cannot understand every parts. "A specific decision can be forced by ORing it with MAIL_CACHE_DECISION_FORCED." Is OR in this sentence like AND/OR comparators? Where to put "MAIL_CACHE_DECISION_FORCED"? "mail-cache-decisions.c file contains the rules how Dovecot changes the decisions." <-- is it in this file? My setup is 2.2.9 on Ubuntu 14.04 running on OpenVZ containers with 2 Director and 2 imap/pop3 backends. The storage is shared via NFS. Is this the performance problem? If dovecot has metadata I want in its cache, I cannot see how this is a problem. Can these settings be useful for my case? mail_always_cache_fields = mail_cache_fields = flags mail_cache_min_mail_count = 0 mail_never_cache_fields = imap.envelope Thanks. From lista at xdrv.co.uk Thu Mar 19 13:07:14 2015 From: lista at xdrv.co.uk (James) Date: Thu, 19 Mar 2015 13:07:14 +0000 Subject: core not dumped Message-ID: <550ACA02.5080704@xdrv.co.uk> There is some underlying problem with imap-login but in order to debug it I'd like to see a core file. On error dovecot reports "core not dumped". If I put "abort()" in the code at the point it is reporting then a core file is written, so there is no OS reason why a core can not be written. It appears to be dovecot's choice to control the child. In an attempt to obtain a core file I have: service imap-login { chroot = drop_priv_before_exec = yes user = dovecot } How can I make imap-login to produce a core file? Syslog error report: Mar 18 09:22:28 MAILHOST dovecot: [ID 583609 mail.crit] imap-login: Fatal: master: service(imap-login): child 10812 killed with signal 11 (core not dumped) [last ip=111.222.333.444] Should anyone care at this point (before I've looked at a core file) the error is related to; imap-login: Disconnected (no auth attempts in 0 secs): user=<>, James. From dovecot at list-post.mks-mail.de Thu Mar 19 13:35:04 2015 From: dovecot at list-post.mks-mail.de (=?UTF-8?B?TWFya3VzIFNjaMO2bmhhYmVy?=) Date: Thu, 19 Mar 2015 14:35:04 +0100 Subject: core not dumped In-Reply-To: <550ACA02.5080704@xdrv.co.uk> References: <550ACA02.5080704@xdrv.co.uk> Message-ID: <550AD088.6090705@list-post.mks-mail.de> Am 19.03.2015 um 14:07 schrieb James: > There is some underlying problem with imap-login but in order to debug > it I'd like to see a core file. On error dovecot reports "core not > dumped". If I put "abort()" in the code at the point it is reporting > then a core file is written, so there is no OS reason why a core can not > be written. It appears to be dovecot's choice to control the child. In > an attempt to obtain a core file I have: > > service imap-login { > chroot = > drop_priv_before_exec = yes > user = dovecot > } > > How can I make imap-login to produce a core file? Maybe this helps: http://dovecot.org/bugreport.html#coredumps -- Regards mks From skdovecot at smail.inf.fh-brs.de Thu Mar 19 14:08:42 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 19 Mar 2015 15:08:42 +0100 (CET) Subject: fast doveadm search results In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 19 Mar 2015, Vu Ngoc VU wrote: > My goal is to be able to find mails quickly with "doveadm search". What information you are search for? Message-ID, subject, body text? If your data is in the headers, check out: doveconf -a|grep -i cache mail_always_cache_fields = mail_cache_fields = flags mail_cache_min_mail_count = 0 mail_never_cache_fields = imap.envelope e.g. put message-id into mail_always_cache_fields - From http://dovecot.org/pipermail/dovecot-cvs/2013-May/023261.html lib-storage: Allow mail_*cache_fields settings to specify any hdr.* fields - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVQrYanz1H7kL/d9rAQInGwf9HSkQr7JhGI/GYH27qiEez2Adnp/7f8Qv PfgcH+PqKpUcvXNM1xRrVXEq4yswNnkEIYjT7gs0NAY/Yr72cT5NuoU6s37yrPYW moC/timqEx8UXafNmB3v+L2U0o0owvfxd4yK93EmUxbe7i91jeYBnKXHKoNA7uNF p3V6ey7qf+CNmFg++rUnmH69iNG12Ia28Gg8ynzc1jZXEDFB2OCB+tFJwjJx2/Y6 Ujgis2JrtI5w086dfMwGnVexqGd697PgSmgYDxX5h8wPe6hd4oUc6dxV1cTlmzwj cjJVVPNHhq37NFr8Uze+eYR3mqTBnex4I37WMCbuAhMgAn4BNOp8Kg== =dwNy -----END PGP SIGNATURE----- From vvu at mcra.fr Thu Mar 19 14:18:38 2015 From: vvu at mcra.fr (Vu Ngoc VU) Date: Thu, 19 Mar 2015 15:18:38 +0100 (CET) Subject: fast doveadm search results In-Reply-To: References: Message-ID: > Date: Thu, 19 Mar 2015 15:08:42 > From: Steffen Kaiser > Reply-To: dovecot at dovecot.org > To: Vu Ngoc VU > Cc: dovecot at dovecot.org > Subject: Re: fast doveadm search results > > >> My goal is to be able to find mails quickly with "doveadm search". > > What information you are search for? Message-ID, subject, body text? Thanks for your answer. Yes, I only need doveadm to search for fields that are in envelope or headers. Not the whole body part or the attachments. > If your data is in the headers, check out: > > doveconf -a|grep -i cache > > mail_always_cache_fields = > mail_cache_fields = flags > mail_cache_min_mail_count = 0 > mail_never_cache_fields = imap.envelope > > e.g. put message-id into mail_always_cache_fields Yes, as I didn't yet set anything, it looks like this. If I record correctly, I've pasted it in my initial post. OK, I'll try to change mail_always_cache_fields. From lista at xdrv.co.uk Thu Mar 19 14:53:31 2015 From: lista at xdrv.co.uk (James) Date: Thu, 19 Mar 2015 14:53:31 +0000 Subject: core not dumped In-Reply-To: <550AD088.6090705@list-post.mks-mail.de> References: <550ACA02.5080704@xdrv.co.uk> <550AD088.6090705@list-post.mks-mail.de> Message-ID: <550AE2EB.2050403@xdrv.co.uk> On 19/03/2015 13:35, Markus Sch?nhaber wrote: >> dumped". If I put "abort()" in the code at the point it is reporting >> then a core file is written, so there is no OS reason why a core can not >> be written. It appears to be dovecot's choice to control the child. In >> an attempt to obtain a core file I have: >> >> service imap-login { >> chroot = >> drop_priv_before_exec = yes >> user = dovecot >> } >> >> How can I make imap-login to produce a core file? > > Maybe this helps: > http://dovecot.org/bugreport.html#coredumps I can create core dumps but not from imap-login, so it's not a general system restriction. Adding abort() to the master gives a core - but but not a relevant one. Adding an arbitrary abort() to imap-login gives a "core not dumped" message. James. From user+dovecot at localhost.localdomain.org Thu Mar 19 20:59:49 2015 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Thu, 19 Mar 2015 20:59:49 +0000 Subject: core not dumped In-Reply-To: <550AE2EB.2050403@xdrv.co.uk> References: <550ACA02.5080704@xdrv.co.uk> <550AD088.6090705@list-post.mks-mail.de> <550AE2EB.2050403@xdrv.co.uk> Message-ID: <550B38C5.4080004@localhost.localdomain.org> On 03/19/2015 02:53 PM, James wrote: >>> ... >>> service imap-login { >>> chroot = >>> drop_priv_before_exec = yes >>> user = dovecot >>> } >>> >>> How can I make imap-login to produce a core file? > ... > I can create core dumps but not from imap-login, so it's not a general > system restriction. Adding abort() to the master gives a core - but but > not a relevant one. Adding an arbitrary abort() to imap-login gives a > "core not dumped" message. Try this configuration: service imap-login { executable = imap-login -D } Your configuration (doveconf -n output) may be also helpfull Regards, Pascal -- The trapper recommends today: defaced.1507821 at localdomain.org From leolistas at solutti.com.br Thu Mar 19 21:32:23 2015 From: leolistas at solutti.com.br (Leonardo Rodrigues) Date: Thu, 19 Mar 2015 18:32:23 -0300 Subject: check if anvil is working Message-ID: <550B4067.2060204@solutti.com.br> Hi, Is there any way of making sure the anvil service, used to penalty login fails, is working ? I dont see anything on the logs regarding it neither have it configured. It's not on my configuration files, but it can be seen on a 'dovecot -a' dump. Should it be logging something ? Should i have enabled it somehow ? Thanks for the answers. [root at correio log]# dovecot --version 2.2.13 (from dovecot -a dump) service anvil { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = anvil extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 1 protocol = service_count = 0 type = anvil unix_listener anvil-auth-penalty { group = mode = 0600 user = } unix_listener anvil { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } -- Atenciosamente / Sincerily, Leonardo Rodrigues Solutti Tecnologia http://www.solutti.com.br Minha armadilha de SPAM, N?O mandem email gertrudes at solutti.com.br My SPAMTRAP, do not email it From stephan at rename-it.nl Thu Mar 19 22:42:52 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 19 Mar 2015 23:42:52 +0100 Subject: Released Pigeonhole v0.4.7 for Dovecot v2.2.16 Message-ID: <550B50EC.4040506@rename-it.nl> Hello Dovecot users, Here is the final 0.4.7 release. No changes were committed since the last release candidate. Changelog v0.4.7: * editheader extension: Made protection against addition and deletion of headers configurable separately. Also, the `Received' and `Auto-Submitted' headers are no longer protected against addition by default. * Turned message envelope address parse errors into warnings. * The interpreter now accepts non-standard domain names, e.g. containing '_'. + Implemented the Sieve index extension (RFC 5260). + Implemented support for the mboxmetadata and servermetadata extensions (RFC 5490). + Implemented new sieve commands for the doveadm command line utility. These commands are currently limited to ManageSieve operations, but the other current sieve tools will be migrated to doveadm in the near future as well. + Added more debug output about binary up-to-date checking. + Added script metadata to binary dump output. - Fixed Sieve script binary up-to-date checking by normalizing the script location. - The Sieve interpreter now flushes the duplicate database during start phase of result execution rather than commit phase. This makes sure locks on the duplicate database are released as soon as possible, preventing contention. - Performed a few optimizations in the lexical scanner of the language. - Fixed bug in `:matches' match-type that made a pattern without wildcards match as if there were a '*' at the beginning. - Fixed crash in validation of the string parameter of the comparator tag. - extprograms extension: Made sure supplemental group privileges are also dropped. This was a problem reported by Debian lintian. - Fixed bug in handling of binary errors for action side-effects and message overrides. - file script storage: Restructured storage initialization to address backwards compatibility issues. - dict script storage: Fixed small memory allocation bug. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.7.tar.gz http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.7.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From jtam.home at gmail.com Fri Mar 20 00:53:34 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Thu, 19 Mar 2015 17:53:34 -0700 (PDT) Subject: fast doveadm search results In-Reply-To: References: Message-ID: Vu Ngoc VU writes: > My goal is to be able to find mails quickly with "doveadm search". > We are using dovecot in a corporate. And some managers, sometimes ask to delete some messages. > These messages are "unwanted" ones: mistakenly sent, defamatory... > > So, that situations appear rarely, but when asked, they expected me to > deleted quickly these messages from few to thousands of users > mailboxes. > > Is that possible to change behaviour of Dovecot, some headers like > sender, message-id, date are stored in cache for at least 48h? (7 days > would be better). So is this a fair summary of what you are asking: you want to load dovecot's caches with searchable items (i.e. specified headers) *and* have them expire and removed from caches after a preset amount of time? Caches are persistent: once an item is cached, it stays in the cache. I can't think of a good way to purge them without a lot of bother, but maybe you don't really need this. If you want to preload the cache with searchable items, you can periodically run a search/fetch command, which will load the cache with headers you want. http://www.dovecot.org/list/dovecot/2010-October/053521.html Maybe you can even add these items into the cache upon delivery, but someone more knowlegable than I would have to help you with that. The FTS/Solr plugin might also help. You seem to be optimizing the wrong problem. Reading between the lines of your message, your management wants a technical solution to a human problem (someone did an oops). Joseph Tam From jtam.home at gmail.com Fri Mar 20 01:05:48 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Thu, 19 Mar 2015 18:05:48 -0700 (PDT) Subject: fast doveadm search results In-Reply-To: References: Message-ID: Steffen Kaiser writes: > e.g. put message-id into mail_always_cache_fields > > - From http://dovecot.org/pipermail/dovecot-cvs/2013-May/023261.html > > lib-storage: Allow mail_*cache_fields settings to specify any hdr.* fields ... but I belatedly wrote ... > Maybe you can even add these items into the cache upon delivery, > but someone more knowlegable than I would have to help you with that. The drawback of getting thing in digest form -- you look like a boob when answering late. I defer to Steffen's superior knowledge. Joseph Tam From vvu at mcra.fr Fri Mar 20 08:37:34 2015 From: vvu at mcra.fr (Vu Ngoc VU) Date: Fri, 20 Mar 2015 09:37:34 +0100 (CET) Subject: fast doveadm search results In-Reply-To: References: Message-ID: > Date: Fri, 20 Mar 2015 02:05:48 > From: Joseph Tam > To: dovecot at dovecot.org > Subject: Re: fast doveadm search results > > Steffen Kaiser writes: > >> e.g. put message-id into mail_always_cache_fields >> >> - From http://dovecot.org/pipermail/dovecot-cvs/2013-May/023261.html >> >> lib-storage: Allow mail_*cache_fields settings to specify any hdr.* fields > > ... but I belatedly wrote ... ahah, in France, we says "that happens" >> Maybe you can even add these items into the cache upon delivery, >> but someone more knowlegable than I would have to help you with that. > > The drawback of getting thing in digest form -- you look like a boob > when answering late. I defer to Steffen's superior knowledge. Don't blame yourself. It's not a problem to me receiving twice same answer. I prefer more to less :) (not about unix tools) But to answer your questions, I'm not really interested in purging the cache data. I just think that cache has expiration delay. The only point is to get this "doveadm search" answering me in minutes instead of hours. Writing my original post, I didn't get if it was slow because of: - data I'm searching are not cached at all? (headers like From, Date, Message-ID...) => I wanted to know if dovecot allows to add some headers. Stephen answered to that question. - these data a cache, but for an extremely short time, like the user session => that's why I asked if it is possible to extend cache validity to at least 48h. But for sure, if these data remain forever, it'll be better ! :) - NFS limitations => do I have to re-install dovecot on my NFS servers? I prefer not. From gedalya at gedalya.net Fri Mar 20 10:40:41 2015 From: gedalya at gedalya.net (Gedalya) Date: Fri, 20 Mar 2015 06:40:41 -0400 Subject: [Dovecot] imapc - different settings for different namespaces In-Reply-To: <75314229-D383-4A2C-BCA7-C39F73DE8FE5@iki.fi> References: <75314229-D383-4A2C-BCA7-C39F73DE8FE5@iki.fi> Message-ID: <550BF929.7070306@gedalya.net> On 05/31/2013 04:21 PM, Timo Sirainen wrote: > Originally I was planning on allowing all kinds of mail settings inside namespace {}, including imapc_* settings. But that's a bit difficult to implement (although I think it will happen some day). So for now I was thinking: > > imapc foo { > host = imap.foo.com > master_user = foomaster > password = foopass > } > > imapc bar { > host = imap.bar.com > ... > } > > namespace { > prefix = foo/ > location = imapc:foo > } > namespace { > prefix = bar/ > location = imapc:bar > } > > So basically if the path after imapc isn't absolute (/path or ~/path), then treat the path as the imapc section name and use its settings. > > Anyone have any improvement suggestions? I was planning on adding this to v2.2.3. I'm not sure if the section names should be "imapc" or something else (e.g. imap_remote {}). > What ever happened with this? Doesn't look like it was committed. This looks like a cool idea. I was just thinking about pulling multiple external IMAP accounts under a single dovecot account and I ran into this. From lista at xdrv.co.uk Fri Mar 20 11:10:25 2015 From: lista at xdrv.co.uk (James) Date: Fri, 20 Mar 2015 11:10:25 +0000 Subject: core not dumped In-Reply-To: <550B38C5.4080004@localhost.localdomain.org> References: <550ACA02.5080704@xdrv.co.uk> <550AD088.6090705@list-post.mks-mail.de> <550AE2EB.2050403@xdrv.co.uk> <550B38C5.4080004@localhost.localdomain.org> Message-ID: <550C0021.8040303@xdrv.co.uk> On 19/03/2015 20:59, Pascal Volk wrote: > Try this configuration: > > service imap-login { > executable = imap-login -D > } Many thanks, Pascal. That did it. Now I have a core I can investigate the real problem - it's openssl that is failing, ha, maybe the problem will just go away with openssl 1.0.2a. James. From lista at xdrv.co.uk Fri Mar 20 11:59:09 2015 From: lista at xdrv.co.uk (James) Date: Fri, 20 Mar 2015 11:59:09 +0000 Subject: imap-login SSLv3 causes signal 11, core dump and DoS. ssl_protocols = ?? Message-ID: <550C0B8D.9010004@xdrv.co.uk> Connecting to dovecot with ssl3 causes imap-login to die: $ openssl s_client -connect localhost:993 -ssl3 CONNECTED(00000003) 4277630796:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:s3_pkt.c:1461:SSL alert number 40 4277630796:error:1409E0E5:SSL routines:ssl3_write_bytes:ssl handshake failure:s3_pkt.c:645: --- no peer certificate available --- No client certificate CA names sent --- SSL handshake has read 7 bytes and written 0 bytes --- New, (NONE), Cipher is (NONE) Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE No ALPN negotiated SSL-Session: Protocol : SSLv3 Cipher : 0000 Session-ID: Session-ID-ctx: Master-Key: Key-Arg : None PSK identity: None PSK identity hint: None SRP username: None Start Time: 1426851034 Timeout : 7200 (sec) Verify return code: 0 (ok) --- syslog: Mar 20 11:30:35 MAILHOST dovecot: [ID 583609 mail.crit] imap-login: Fatal: master: service(imap-login): child 21918 killed with signal 11 (core dumped) [last ip=127.0.0.1] dovecot.conf had: ssl_protocols = !SSLv2 !SSLv3 removing that line stops the core dump and syslog then shows: Mar 20 11:36:25 MAILHOST dovecot: [ID 583609 mail.info] imap-login: Disconnected (disconnected before auth was ready, waited 0 secs): user=<>, rip=127.0.0.1, lip=127.0.0.1, TLS handshaking: SSL_accept() failed: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol, session= the "SSL23_GET_CLIENT_HELLO:unsupported protocol" seems to do what I thought the ssl_protocols setting did. Do I still need, if I ever needed, the "ssl_protocols = " setting? James. # dovecot -n # 2.2.16: /etc/opt/XXXX/dovecot/dovecot.conf # Pigeonhole version 0.4.7 # OS: SunOS 5.10 i86pc auth_mechanisms = plain login digest-md5 cram-md5 base_dir = /var/opt/XXXX/dovecot/ lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_trusted_networks = 111.222.333.444/24 mail_gid = vmail mail_home = /XXXXXX/XXXX/%d/%n mail_location = maildir:/XXXXX/XXXX/%d/%n/Maildir mail_max_userip_connections = 20 mail_plugins = quota mail_uid = vmail mailbox_idle_check_interval = 10 secs managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate passdb { args = /etc/opt/XXXX/dovecot/dovecot-sql.conf driver = sql } plugin { fts_autoindex = yes quota = maildir:User quota quota_rule = *:storage=1G quota_rule2 = Trash:storage=+10% quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=95%% quota-warning 95 %u quota_warning3 = storage=99%% quota-warning 99 %u sieve = /XXXXX/XXXX/%d/%n/dovecot.sieve sieve_dir = /XXXXX/XXXX/%d/%n/sieve } protocols = imap lmtp sieve service auth { drop_priv_before_exec = yes unix_listener auth-client { mode = 0660 } unix_listener auth-master { mode = 0600 } user = root } service imap-login { chroot = drop_priv_before_exec = yes executable = imap-login -D service_count = 1 user = dovecot } service lmtp { group = vmail unix_listener lmtp { mode = 0666 } user = vmail } service quota-warning { executable = script /etc/opt/XXXX/dovecot/quota-warning user = vmail } ssl_cert = References: Message-ID: <626BAD45-6551-456A-B7A6-3D7C9B40F4AE@iki.fi> On 20 Mar 2015, at 10:37, Vu Ngoc VU wrote: > > But to answer your questions, I'm not really interested in purging the cache data. > I just think that cache has expiration delay. > The only point is to get this "doveadm search" answering me in minutes instead of hours. > Writing my original post, I didn't get if it was slow because of: > - data I'm searching are not cached at all? (headers like From, Date, Message-ID...) > => I wanted to know if dovecot allows to add some headers. > Stephen answered to that question. By default all headers are added to cache the first time they're accessed in the folder (e.g. via FETCH or SEARCH). Also mails that are newly delivered by Dovecot will add those headers to cache immediately. > - these data a cache, but for an extremely short time, like the user session > => that's why I asked if it is possible to extend cache validity to at least 48h. > But for sure, if these data remain forever, it'll be better ! :) Dovecot automatically figures out if the data should stay in cache for 1 week or forever. It sounds like something's wrong in your system if it's not already automatically performing fast searches. The first time a search on a header is done it might be slow if the data isn't in cache, but all subsequent times should be very fast. Not hours or minutes but seconds. No need to modify the mail_cache_* settings. It might be helpful if you posted your whole doveconf -n output. > - NFS limitations > => do I have to re-install dovecot on my NFS servers? I prefer not. What do you mean by this? You're using NFS now to store emails but with one Dovecot server? That should work fine, although NFS of course always adds some extra overhead. From tss at iki.fi Fri Mar 20 18:24:27 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 20 Mar 2015 20:24:27 +0200 Subject: imap-login SSLv3 causes signal 11, core dump and DoS. ssl_protocols = ?? In-Reply-To: <550C0B8D.9010004@xdrv.co.uk> References: <550C0B8D.9010004@xdrv.co.uk> Message-ID: On 20 Mar 2015, at 13:59, James wrote: > > Connecting to dovecot with ssl3 causes imap-login to die: > > Mar 20 11:30:35 MAILHOST dovecot: [ID 583609 mail.crit] imap-login: Fatal: master: service(imap-login): child 21918 killed with signal 11 (core dumped) [last ip=127.0.0.1] I can't reproduce it. I tried it with the same ssl_* settings you had. Can you get a gdb backtrace from the crash? It says "core dumped", so I guess there should be a core file somewhere. http://dovecot.org/bugreport.html has some more info on how to get it. > dovecot.conf had: > ssl_protocols = !SSLv2 !SSLv3 > > removing that line stops the core dump and syslog then shows: > > Mar 20 11:36:25 MAILHOST dovecot: [ID 583609 mail.info] imap-login: Disconnected (disconnected before auth was ready, waited 0 secs): user=<>, rip=127.0.0.1, lip=127.0.0.1, TLS handshaking: SSL_accept() failed: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol, session= > > > > the "SSL23_GET_CLIENT_HELLO:unsupported protocol" seems to do what I thought the ssl_protocols setting did. > Do I still need, if I ever needed, the "ssl_protocols = " setting? All these ssl_* settings just go to OpenSSL without Dovecot (or I) knowing all that much about them. I think you still need it, but maybe it's because your ssl_cipher_list is so limited that it fails the session anyway (just my guess). From tss at iki.fi Fri Mar 20 18:30:19 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 20 Mar 2015 20:30:19 +0200 Subject: [Dovecot] imapc - different settings for different namespaces In-Reply-To: <550BF929.7070306@gedalya.net> References: <75314229-D383-4A2C-BCA7-C39F73DE8FE5@iki.fi> <550BF929.7070306@gedalya.net> Message-ID: <8F75D9D6-A92C-475E-8BC1-0478D2A9B533@iki.fi> On 20 Mar 2015, at 12:40, Gedalya wrote: > > On 05/31/2013 04:21 PM, Timo Sirainen wrote: >> Originally I was planning on allowing all kinds of mail settings inside namespace {}, including imapc_* settings. But that's a bit difficult to implement (although I think it will happen some day). So for now I was thinking: >> >> imapc foo { >> host = imap.foo.com >> master_user = foomaster >> password = foopass >> } >> >> imapc bar { >> host = imap.bar.com >> ... >> } >> >> namespace { >> prefix = foo/ >> location = imapc:foo >> } >> namespace { >> prefix = bar/ >> location = imapc:bar >> } >> >> So basically if the path after imapc isn't absolute (/path or ~/path), then treat the path as the imapc section name and use its settings. >> >> Anyone have any improvement suggestions? I was planning on adding this to v2.2.3. I'm not sure if the section names should be "imapc" or something else (e.g. imap_remote {}). >> > > What ever happened with this? Doesn't look like it was committed. This looks like a cool idea. I was just thinking about pulling multiple external IMAP accounts under a single dovecot account and I ran into this. I didn't need it after all for the project where it first seemed to be necessary. Attached the code that I wrote so far, which I think allows you to configure such imapc {} blocks, but there's no code to actually use them. -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 12948 bytes Desc: not available URL: -------------- next part -------------- From tss at iki.fi Fri Mar 20 18:47:51 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 20 Mar 2015 20:47:51 +0200 Subject: How about an option to disbale headers? (was Re: Patch for "doveadm -f table" nit) In-Reply-To: References: <78D069B1-3D22-456B-B13E-1FD9AC871157@iki.fi> <550A891D.8040108@gedalya.net> Message-ID: <43DCC0A1-BD9B-4A1E-88EE-51517B3EBBB0@iki.fi> On 19 Mar 2015, at 12:25, Steffen Kaiser wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Thu, 19 Mar 2015, Gedalya wrote: >> On 03/18/2015 08:49 PM, Timo Sirainen wrote: >>> There's no reason why flow and pager should write headers to stderr because it would always result only in a mess. But instead of changing table headers to write to stdout, I think a better fix would be to make tab formatter write headers to stderr. Including headers in stdout makes it more difficult to write scripts that access the actual data. For example now you can do "doveadm who -1 | sort" and the output will work. If headers were written to stdout you'd have to make it more complicated. Also you can now easily specify what you want to do with the headers, 2>/dev/null if you don't care about them or 2>&1 if you want to include them in stdout (which works even after |sort). > > doveadm .... 2> /dev/null > will suppress headers _and_ error messages Hm. Yeah, that's not good. >>> So, I'll add in my v2.3 TODO that tab formatter should write to stderr.. >> >> I've been using "-f flow fetch text | sed s/^text=//" when training spamassasin. Couldn't find a straightforward 'fetch raw message'. Seems unnecessarily awkward. Moving headers to stderr would help this, though. >> > > How about a formatter option to enable/disable headers at all? Like, > > who -H > ps h > ps --headers > > Most probably it useful for scripts only, so add an option to suppress all > headers will fit most case. I guess that would be best. Added -h parameter now to hg. I didn't change the stdout/stderr behavior for now - have to wait until v2.3 for that. I guess that could also be changed to stdout then. From tss at iki.fi Fri Mar 20 18:51:58 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 20 Mar 2015 20:51:58 +0200 Subject: indexer-worker panics with latest mercurial In-Reply-To: <5501FD50.8060702@laeeth.com> References: <5501FD50.8060702@laeeth.com> Message-ID: On 12 Mar 2015, at 22:55, Laeeth Isharc wrote: > > Hi. > > I am seeing the following in my logs. Happy to downgrade to an earlier version if you think this might be the problem, but dovecot.org is extremely slow (and has been for months) so I found it easier just to pull the latest from mercurial. dovecot.org isn't slow as far as I know, although I've noticed some network connectivity problems to some places .. It's in Azure, so I guess somehow related to that.. > Mar 12 20:48:39 indexer: Error: Indexer worker disconnected, discarding 1 reques > ts for laeeth at laeeth.com > Mar 12 20:48:39 indexer-worker(laeeth at laeeth.com): Fatal: master: service(indexe > r-worker): child 24003 killed with signal 6 (core dumps disabled) > Mar 12 20:49:01 dsync-local(laeeth at laeeth.com): Error: Couldn't lock /home/mail/ > laeeth_laeeth_com/.dovecot-sync.lock: Timed out after 30 seconds > Mar 12 20:49:16 indexer-worker(laeeth at laeeth.com): Error: fts_tika: PUT http://l > ocalhost:9997/tika/ failed: 500 Server Error > Mar 12 20:49:17 indexer-worker(rosie at kaleidicassociates.com): Warning: I/O leak: > 0x7fc47d60fcf0 (line 127, fd 25) > Mar 12 20:49:17 indexer-worker(rosie at kaleidicassociates.com): Panic: file ioloop > .c: line 39 (io_add_file): assertion failed: (callback != NULL) I tried this a few times, but I wasn't able to cause the crash. Since you're already pulling from mercurial, can you find which mercurial commit caused it to start breaking? From tss at iki.fi Fri Mar 20 18:53:39 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 20 Mar 2015 20:53:39 +0200 Subject: fast sync vs. full sync In-Reply-To: <54F833EE.1010708@wk-serv.de> References: <54F833EE.1010708@wk-serv.de> Message-ID: On 05 Mar 2015, at 12:46, Patrick Westenberg wrote: > > Hi everyone, > > can anybody explain the difference between Dovecots fast sync and full sync in replication mode? There are actually 3 dsync modes, all explained in doveadm-sync man page: o Full synchronization (-f parameter) scans through all the messages in all the mailboxes. This guarantees that every- thing will be synchronized, but it's unnecessarily slow for incremental synchronization. o Fast synchronization (default) first attempts to find mail- boxes that have changed, and synchronize only those. This is done by checking the mailboxes' metadata (NEXTUID and HIGHESTMODSEQ). Usually this works fine, especially with one-way synchronization, but if both sides do exactly the same number of changes, the metadata may end up containing the same values even if the changes were different. o Stateful synchronization (-s parameter) is the most effi- cient way to synchronize mailboxes. It relies on having the earlier dsync run's state saved somewhere and being passed to the next dsync run. Based on this state dsync can send only the changes that happened after the previous dsync run. As long as the state or the mailboxes aren't corrupted this algorithm should work perfectly. The replicator process uses this internally to perform most of the synchronization. From tss at iki.fi Fri Mar 20 19:06:12 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 20 Mar 2015 21:06:12 +0200 Subject: [PATCH] increase fd_limit to max_client_limit automatically In-Reply-To: <20150122120123.GK16405@kekkonen.niksula.hut.fi> References: <20150122120123.GK16405@kekkonen.niksula.hut.fi> Message-ID: On 22 Jan 2015, at 14:01, Lauri Tirkkonen wrote: > > Hi, with a low soft limit on file descriptors, dovecot 2.2.15 warns on > startup: > > Warning: fd limit (ulimit -n) is lower than required under max. load > (256 < 1000), because of default_client_limit > > It could try increasing the limit first, and only report the warning if that > fails. I'm attaching a patch that does just this. It could .. But somehow it doesn't seem like a good idea to me to do automatically. Maybe the limits are there intentionally. Maybe it's the Dovecot settings that are wrong and not the ulimits. From space.ship.traveller at gmail.com Fri Mar 20 20:02:32 2015 From: space.ship.traveller at gmail.com (Samuel Williams) Date: Sat, 21 Mar 2015 09:02:32 +1300 Subject: Strange empty folders 1 1 1 1 1 Message-ID: I just updated to the latest dovecot from 1.2 and it has been working great for most of my clients. But one client,who is using Outlook, has found that hundreds of folders have been created, e.g. admin at domain Deleted Items 1 admin at domain Deleted Items 1 1 admin at domain Deleted Items 1 1 1 .... admin at domain Deleted Items 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 This appears to be affecting all folders in his account, across multiple accounts for his domain. It isn't affecting any other user, even other users using outlook. All the folders are empty. Just wondering if anyone has any suggestions or ideas why this is happening. Thanks Samuel From space.ship.traveller at gmail.com Fri Mar 20 20:18:56 2015 From: space.ship.traveller at gmail.com (Samuel Williams) Date: Sat, 21 Mar 2015 09:18:56 +1300 Subject: Deleting empty folders Message-ID: So, along with the problem of lots of folders ending in 1 1 1 1, I'd like to have a strategy to delete these. I was trying to understand if it is possible to use http://linux.die.net/man/1/doveadm-expunge to delete empty folders, but without also deleting messages? Is this possible and if so what is the syntax? Kind regards, Samuel From HFlor at gmx.de Sat Mar 21 08:22:48 2015 From: HFlor at gmx.de (Hardy Flor) Date: Sat, 21 Mar 2015 09:22:48 +0100 Subject: Deleting empty folders In-Reply-To: References: Message-ID: <550D2A58.3040504@gmx.de> With doveadm mailbox status -u ... messages"*" There is a list of folders and the number of messages and then with doveadm mailbox delete -u ... "" to delete. Am 20.03.2015 um 21:18 schrieb Samuel Williams: > So, along with the problem of lots of folders ending in 1 1 1 1, I'd like > to have a strategy to delete these. > > I was trying to understand if it is possible to use > http://linux.die.net/man/1/doveadm-expunge to delete empty folders, but > without also deleting messages? Is this possible and if so what is the > syntax? > > Kind regards, > Samuel From sager at agitos.de Sat Mar 21 08:14:22 2015 From: sager at agitos.de (Florian Sager) Date: Sat, 21 Mar 2015 09:14:22 +0100 Subject: IMAP ANNOTATE Extension RFC5257: priority on roadmap Message-ID: <550D285E.50801@agitos.de> Hi Timo, congrats to the merger with OX. Currently the implementation of RFC 5257, ANNOTATE-EXPERIMENT-1, has only low priority on http://wiki2.dovecot.org/Roadmap I want to explain a scenario that would benefit from annotation support to - maybe - increase the priority in your roadmap: I'm currently working on a project to publish bank customer related documents inside a banking webinterface (they call it 'postbox'). There are different requirements that would be satisfied very well by an IMAP server as data storage/provider for a long term email/document archive inside the bank. Annotations would be necessary to tag emails with customer numbers and document IDs to allow a delayed grouping of several emails by such identifiers (message IDs may not be available for references so annotations have to be used to compute references indirectly). Same for sorting and filtering by customer number or account number. I would highly appreciate if you at Dovecot could increase the priority of ANNOTATE in your roadmap. Best regards, Florian -- Agitos GmbH Emil-Geis-Stra?e 40 D-81379 M?nchen Telefon: +49-89-381 564 46-0 Telefax: +49-89-381 564 46-9 E-Mail: support at agitos.de Web: http://www.agitos.de Registergericht: Amtsgericht M?nchen, HRB 213590 Gesch?ftsf?hrer: Dipl.-Inf. (Univ.) Florian Sager Ust-Identifikationsnummer: DE296114074 From lista at xdrv.co.uk Sat Mar 21 10:00:58 2015 From: lista at xdrv.co.uk (James) Date: Sat, 21 Mar 2015 10:00:58 +0000 Subject: imap-login SSLv3 causes signal 11, core dump and DoS. ssl_protocols = ?? In-Reply-To: References: <550C0B8D.9010004@xdrv.co.uk> Message-ID: <550D415A.4030900@xdrv.co.uk> On 20/03/2015 18:24, Timo Sirainen wrote: >> Connecting to dovecot with ssl3 causes imap-login to die: >> >> Mar 20 11:30:35 MAILHOST dovecot: [ID 583609 mail.crit] imap-login: Fatal: master: service(imap-login): child 21918 killed with signal 11 (core dumped) [last ip=127.0.0.1] > > I can't reproduce it. I tried it with the same ssl_* settings you had. Can you get a gdb backtrace from the crash? It says "core dumped", so I guess there should be a core file somewhere. http://dovecot.org/bugreport.html has some more info on how to get it. Thank you for your interest, here is a dbx trace. This was with OpenSSL 1.0.2a. (dbx) where =>[1] ssl3_get_client_hello(s = 0x809b2a0) (optimized), at 0xfe9db0d5 (line ~1362) in "s3_srvr.c" [2] ssl3_accept(s = 0x809b2a0) (optimized), at 0xfe9d9892 (line ~357) in "s3_srvr.c" [3] SSL_accept(s = 0x809b2a0) (optimized), at 0xfea09f07 (line ~990) in "ssl_lib.c" [4] ssl_handshake(proxy = 0x809ba38) (optimized), at 0xfee35c18 (line ~481) in "ssl-proxy-openssl.c" [5] ssl_step(proxy = 0x809ba38) (optimized), at 0xfee35ee0 (line ~545) in "ssl-proxy-openssl.c" [6] ssl_proxy_flush(proxy = 0x809ba38) (optimized), at 0xfee3680c (line ~817) in "ssl-proxy-openssl.c" [7] ssl_proxy_destroy(proxy = 0x809ba38) (optimized), at 0xfee3686b (line ~825) in "ssl-proxy-openssl.c" [8] ssl_handle_error(proxy = 0x809ba38, ret = -1, func_name = 0xfee3b2d8 "SSL_accept()") (optimized), at 0xfee35bc0 (line ~465) in "ssl-proxy-openssl.c" [9] ssl_handshake(proxy = 0x809ba38) (optimized), at 0xfee35cc9 (line ~483) in "ssl-proxy-openssl.c" [10] ssl_step(proxy = 0x809ba38) (optimized), at 0xfee35ee0 (line ~545) in "ssl-proxy-openssl.c" [11] ssl_proxy_start(proxy = 0x809ba38) (optimized), at 0xfee36341 (line ~685) in "ssl-proxy-openssl.c" [12] client_connected_finish(conn = 0x8047ae0) (optimized), at 0xfee31d62 (line ~151) in "main.c" [13] client_connected(conn = 0x8047ae0) (optimized), at 0xfee32148 (line ~246) in "main.c" [14] master_service_listen(l = 0x8096b30) (optimized), at 0xfecfac7e (line ~837) in "master-service.c" [15] io_loop_call_io(io = 0x8096bd0) (optimized), at 0xfeda764b (line ~501) in "ioloop.c" [16] io_loop_handler_run_internal(ioloop = 0x8071d70) (optimized), at 0xfedaa419 (line ~211) in "ioloop-poll.c" [17] io_loop_handler_run(ioloop = 0x8071d70) (optimized), at 0xfeda77be (line ~548) in "ioloop.c" [18] io_loop_run(ioloop = 0x8071d70) (optimized), at 0xfeda7711 (line ~525) in "ioloop.c" [19] master_service_run(service = 0x8071cb8, callback = 0xfee32040 = &`libdovecot-login.so.0.0.0`main.c`client_connected(struct master_service_connection *conn)) (optimized), at 0xfecfa3d7 (line ~569) in "master-service.c" [20] login_binary_run(binary = 0x8068c50, argc = 2, argv = 0x8047d4c) (optimized), at 0xfee3294a (line ~470) in "main.c" [21] main(argc = 2, argv = 0x8047d4c) (optimized), at 0x8054de7 (line ~706) in "client.c" >> dovecot.conf had: >> ssl_protocols = !SSLv2 !SSLv3 >> >> removing that line stops the core dump and syslog then shows: >> >> Mar 20 11:36:25 MAILHOST dovecot: [ID 583609 mail.info] imap-login: Disconnected (disconnected before auth was ready, waited 0 secs): user=<>, rip=127.0.0.1, lip=127.0.0.1, TLS handshaking: SSL_accept() failed: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol, session= >> >> >> >> the "SSL23_GET_CLIENT_HELLO:unsupported protocol" seems to do what I thought the ssl_protocols setting did. >> Do I still need, if I ever needed, the "ssl_protocols = " setting? > > All these ssl_* settings just go to OpenSSL without Dovecot (or I) knowing all that much about them. I think you still need it, but maybe it's because your ssl_cipher_list is so limited that it fails the session anyway (just my guess). I admit I just copied from somewhere else without full understanding. Please if someone can advise me on settings for ssl_protocols and ssl_cipher_list then I'll use. Removing "ssl_cipher_list = ", so using the default, does not cure the problem. James. From space.ship.traveller at gmail.com Sat Mar 21 10:48:24 2015 From: space.ship.traveller at gmail.com (Samuel Williams) Date: Sat, 21 Mar 2015 23:48:24 +1300 Subject: Deleting empty folders In-Reply-To: <550D2A58.3040504@gmx.de> References: <550D2A58.3040504@gmx.de> Message-ID: Thanks Hardy. I have 1600 folders to delete.. in the end I did find -type d -name "* 1" -exec rm -r {} \; On 21 March 2015 at 21:22, Hardy Flor wrote: > With > doveadm mailbox status -u ... messages"*" > There is a list of folders and the number of messages and then with > doveadm mailbox delete -u ... "" > to delete. > > Am 20.03.2015 um 21:18 schrieb Samuel Williams: > > So, along with the problem of lots of folders ending in 1 1 1 1, I'd like >> to have a strategy to delete these. >> >> I was trying to understand if it is possible to use >> http://linux.die.net/man/1/doveadm-expunge to delete empty folders, but >> without also deleting messages? Is this possible and if so what is the >> syntax? >> >> Kind regards, >> Samuel >> > From lista at xdrv.co.uk Sat Mar 21 10:51:39 2015 From: lista at xdrv.co.uk (James) Date: Sat, 21 Mar 2015 10:51:39 +0000 Subject: imap-login SSLv3 causes signal 11, core dump and DoS. ssl_protocols = ?? In-Reply-To: <550D415A.4030900@xdrv.co.uk> References: <550C0B8D.9010004@xdrv.co.uk> <550D415A.4030900@xdrv.co.uk> Message-ID: <550D4D3B.2000607@xdrv.co.uk> On 21/03/2015 10:00, James wrote: >>> the "SSL23_GET_CLIENT_HELLO:unsupported protocol" seems to do what I >>> thought the ssl_protocols setting did. >>> Do I still need, if I ever needed, the "ssl_protocols = " setting? >> >> All these ssl_* settings just go to OpenSSL without Dovecot (or I) >> knowing all that much about them. I think you still need it, but maybe >> it's because your ssl_cipher_list is so limited that it fails the >> session anyway (just my guess). I'd better add this PS, my openssl is compiled with "no-ssl3" which is where the the SSL23 unsupported is coming from. I've remove the "no-ssl3" from openssl indeed it accepts the connection, however, with "ssl_protocols = !SSLv2 !SSLv3" in dovecot.conf imap-login still sig 11s. James. From h.reindl at thelounge.net Sat Mar 21 10:55:31 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 21 Mar 2015 11:55:31 +0100 Subject: imap-login SSLv3 causes signal 11, core dump and DoS. ssl_protocols = ?? In-Reply-To: <550D4D3B.2000607@xdrv.co.uk> References: <550C0B8D.9010004@xdrv.co.uk> <550D415A.4030900@xdrv.co.uk> <550D4D3B.2000607@xdrv.co.uk> Message-ID: <550D4E23.5020508@thelounge.net> Am 21.03.2015 um 11:51 schrieb James: > On 21/03/2015 10:00, James wrote: > >>>> the "SSL23_GET_CLIENT_HELLO:unsupported protocol" seems to do what I >>>> thought the ssl_protocols setting did. >>>> Do I still need, if I ever needed, the "ssl_protocols = " setting? >>> >>> All these ssl_* settings just go to OpenSSL without Dovecot (or I) >>> knowing all that much about them. I think you still need it, but maybe >>> it's because your ssl_cipher_list is so limited that it fails the >>> session anyway (just my guess). > > I'd better add this PS, my openssl is compiled with "no-ssl3" which is > where the the SSL23 unsupported is coming from. I've remove the > "no-ssl3" from openssl indeed it accepts the connection, however, with > "ssl_protocols = !SSLv2 !SSLv3" in dovecot.conf imap-login still sig 11s well, remove that brickage of "special compile" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From lista at xdrv.co.uk Sat Mar 21 11:02:41 2015 From: lista at xdrv.co.uk (James) Date: Sat, 21 Mar 2015 11:02:41 +0000 Subject: imap-login SSLv3 causes signal 11, core dump and DoS. ssl_protocols = ?? In-Reply-To: <550D4E23.5020508@thelounge.net> References: <550C0B8D.9010004@xdrv.co.uk> <550D415A.4030900@xdrv.co.uk> <550D4D3B.2000607@xdrv.co.uk> <550D4E23.5020508@thelounge.net> Message-ID: <550D4FD1.7020300@xdrv.co.uk> On 21/03/2015 10:55, Reindl Harald wrote: > > well, remove that brickage of "special compile" I'm sorry but I did not understand your comment. From h.reindl at thelounge.net Sat Mar 21 11:07:08 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 21 Mar 2015 12:07:08 +0100 Subject: imap-login SSLv3 causes signal 11, core dump and DoS. ssl_protocols = ?? In-Reply-To: <550D4FD1.7020300@xdrv.co.uk> References: <550C0B8D.9010004@xdrv.co.uk> <550D415A.4030900@xdrv.co.uk> <550D4D3B.2000607@xdrv.co.uk> <550D4E23.5020508@thelounge.net> <550D4FD1.7020300@xdrv.co.uk> Message-ID: <550D50DC.3060008@thelounge.net> Am 21.03.2015 um 12:02 schrieb James: > On 21/03/2015 10:55, Reindl Harald wrote: >> >> well, remove that brickage of "special compile" > > I'm sorry but I did not understand your comment why do you compile openssl that way? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From lista at xdrv.co.uk Sat Mar 21 11:12:24 2015 From: lista at xdrv.co.uk (James) Date: Sat, 21 Mar 2015 11:12:24 +0000 Subject: imap-login SSLv3 causes signal 11, core dump and DoS. ssl_protocols = ?? In-Reply-To: <550D50DC.3060008@thelounge.net> References: <550C0B8D.9010004@xdrv.co.uk> <550D415A.4030900@xdrv.co.uk> <550D4D3B.2000607@xdrv.co.uk> <550D4E23.5020508@thelounge.net> <550D4FD1.7020300@xdrv.co.uk> <550D50DC.3060008@thelounge.net> Message-ID: <550D5218.6040508@xdrv.co.uk> On 21/03/2015 11:07, Reindl Harald wrote: >>> well, remove that brickage of "special compile" >> >> I'm sorry but I did not understand your comment > > why do you compile openssl that way? What way? With or without ssl3? I've now done it both ways. Reading: https://wiki.openssl.org/index.php/Compilation_and_Installation no-ssl3 seems to be a popular and legitimate option. From h.reindl at thelounge.net Sat Mar 21 11:15:10 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 21 Mar 2015 12:15:10 +0100 Subject: imap-login SSLv3 causes signal 11, core dump and DoS. ssl_protocols = ?? In-Reply-To: <550D5218.6040508@xdrv.co.uk> References: <550C0B8D.9010004@xdrv.co.uk> <550D415A.4030900@xdrv.co.uk> <550D4D3B.2000607@xdrv.co.uk> <550D4E23.5020508@thelounge.net> <550D4FD1.7020300@xdrv.co.uk> <550D50DC.3060008@thelounge.net> <550D5218.6040508@xdrv.co.uk> Message-ID: <550D52BE.6050408@thelounge.net> Am 21.03.2015 um 12:12 schrieb James: > On 21/03/2015 11:07, Reindl Harald wrote: > >>>> well, remove that brickage of "special compile" >>> >>> I'm sorry but I did not understand your comment >> >> why do you compile openssl that way? > > What way? With or without ssl3? I've now done it both ways. > > Reading: > https://wiki.openssl.org/index.php/Compilation_and_Installation > no-ssl3 seems to be a popular and legitimate option that maybe all fine and true, but since others can't reproduce your problem it's likely your openssl build and not dovecot itself -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From lista at xdrv.co.uk Sat Mar 21 11:21:50 2015 From: lista at xdrv.co.uk (James) Date: Sat, 21 Mar 2015 11:21:50 +0000 Subject: imap-login SSLv3 causes signal 11, core dump and DoS. ssl_protocols = ?? In-Reply-To: <550D52BE.6050408@thelounge.net> References: <550C0B8D.9010004@xdrv.co.uk> <550D415A.4030900@xdrv.co.uk> <550D4D3B.2000607@xdrv.co.uk> <550D4E23.5020508@thelounge.net> <550D4FD1.7020300@xdrv.co.uk> <550D50DC.3060008@thelounge.net> <550D5218.6040508@xdrv.co.uk> <550D52BE.6050408@thelounge.net> Message-ID: <550D544E.8000806@xdrv.co.uk> On 21/03/2015 11:15, Reindl Harald wrote: > that maybe all fine and true, but since others can't reproduce your > problem it's likely your openssl build and not dovecot itself http://www.dovecot.org/bugreport.html "Whenever Dovecot crashes, ..." "No matter how that happened, it's a bug and will be fixed ..." From roland at micite.net Sat Mar 21 11:25:53 2015 From: roland at micite.net (Roland van Laar) Date: Sat, 21 Mar 2015 12:25:53 +0100 Subject: FreeBSD ZFS maildir to mdbox In-Reply-To: <550928AD.10807@solu.fi> References: <5508A615.3060501@micite.net> <550928AD.10807@solu.fi> Message-ID: <550D5541.4000501@micite.net> On 18-03-15 08:26, Toni Mattila wrote: > Hi, > > On 18-Mar-15 00:09, Roland van Laar wrote: >> I'm converting my mailbox from Maildir to mdbox.. >> The Maildir is from an 1.2 server. >> The same conversion on a virtual ubuntu 14.04 box works with mentions of >> "filename has the wrong S value" and Corrupted index errors. > > You should fix the Maildir files first to have correct S= (size) on > them. Older maildrops and qmail likes to create wrong sizes and newer > dovecots rely on that S= to be correct. > > You can use http://www.dovecot.org/tools/maildir-size-fix.pl or > similar script to fix your existing maildirs. That fixed it for me, the migration from maildir to mdbox is working now. I had trouble running the script at first, I added some extra information about the different kind of commandline options. Feel free to include them in the script on dovecot.org. LINE: 233 if (scalar @ARGV == 0) { print STDERR "Usage: maildir-size-fix.pl /path/to/Maildir\n"; print STDERR "-c :Check if the files are compressed. Use the uncompressed size for S=size.\n"; print STDERR "-f :If S=size already exists, verify that it is correct.\n"; print STDERR "-n :If filename doesn't already have a S=size, add it.\n"; print STDERR "-p :If UIDLs are based on filename and no P entry already \t exist for a message, write a P entry so it doesn't \t change when renaming a file.\n"; print STDERR "-r :Recursively scan the maildir for subdirectories.\n"; print STDERR "-v :Verbose logging.\n"; exit 1 } > > Best Regards, > Toni > From number6 at noisynotes.com Sat Mar 21 18:43:48 2015 From: number6 at noisynotes.com (Steve Matzura) Date: Sat, 21 Mar 2015 14:43:48 -0400 Subject: Transitioning from version 1 to version 2 Message-ID: I'm wrestling with transitioning from a 1.0.15 system to a 2.2.25 system. The old dovecot.conf cannot be dropped into position on the new system because so many config options and symbols have changed. It seems every time I change something to match the new formats, I break something else! Anyone out there have maybe half an hour to spare to look at my old and new files to help me whip them into shape? From HFlor at gmx.de Sat Mar 21 18:50:05 2015 From: HFlor at gmx.de (Hardy Flor) Date: Sat, 21 Mar 2015 19:50:05 +0100 Subject: Deleting empty folders In-Reply-To: References: <550D2A58.3040504@gmx.de> Message-ID: <550DBD5D.60104@gmx.de> Hello Samuel, with mdbox as Maildir no direct file operations should beperforms. For empty mailboxes that will ever happen, the "doveadm mailbox delete" deletes the emails, if required in the mailbox. Hardy Am 21.03.2015 um 11:48 schrieb Samuel Williams: > Thanks Hardy. I have 1600 folders to delete.. in the end I did find -type d > -name "* 1" -exec rm -r {} \; > From janm-dovecot at transactionware.com Sun Mar 22 02:25:03 2015 From: janm-dovecot at transactionware.com (Jan Mikkelsen) Date: 22 Mar 2015 02:25:03 -0000 Subject: 2.2.16 link failure on FreeBSD 10.1, with patch Message-ID: <20150322022503.49736.qmail@vanaheim.transactionware.com> Hi, Compiling on FreeBSD 10.1 gives linker errors when linking test-message-snippet. The underlying problem is that libiconv appears on the actual linker line after libcharset.a, which leads to unresolved libiconv symbols. This build process worked fine with 2.2.15. The patch below to src/lib-charset/Makefile.in resolves the problem for me and seems broadly correct. libcharset.a does depend on libiconv so it should probably be declared that way. There is probably a more correct way to make the patch to Makefile.am, but I don't really use automake. Hopefully helpful to someone. Jan Mikkelsen. Patch: --- dovecot-2.2.16/src/lib-charset/Makefile.in 2015-03-13 02:41:16.000000000 +1100 +++ dovecot-2.2.16.new/src/lib-charset/Makefile.in 2015-03-21 13:58:21.951293274 +1100 @@ -92,7 +92,7 @@ CONFIG_CLEAN_FILES = CONFIG_CLEAN_VPATH_FILES = LTLIBRARIES = $(noinst_LTLIBRARIES) -libcharset_la_LIBADD = +libcharset_la_LIBADD = $(LTLIBICONV) am_libcharset_la_OBJECTS = charset-iconv.lo charset-utf8.lo libcharset_la_OBJECTS = $(am_libcharset_la_OBJECTS) AM_V_lt = $(am__v_lt_ at AM_V@) Error messages: libtool: link: cc -std=gnu99 -I/usr/local/include -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -Wl,--as-needed -o test-message-snippet test-message-snippet.o .libs/message-snippet.o .libs/mail-html2text.o .libs/message-decoder.o .libs/quoted-printable.o .libs/rfc822-parser.o .libs/rfc2231-parser.o /usr/local/lib/libiconv.so -Wl,-rpath -Wl,/usr/local/lib .libs/message-parser.o .libs/message-header-parser.o .libs/message-header-decode.o .libs/message-size.o -L/usr/local/lib ../lib-charset/.libs/libcharset.a ../lib-test/.libs/libtest.a ../lib/.libs/liblib.a ../lib-charset/.libs/libcharset.a(charset-iconv.o): In function `charset_to_utf8_begin': charset-iconv.c:(.text+0x49): undefined reference to `libiconv_open' ../lib-charset/.libs/libcharset.a(charset-iconv.o): In function `charset_to_utf8_end': charset-iconv.c:(.text+0x151): undefined reference to `libiconv_close' ../lib-charset/.libs/libcharset.a(charset-iconv.o): In function `charset_to_utf8_reset': charset-iconv.c:(.text+0x211): undefined reference to `libiconv' ../lib-charset/.libs/libcharset.a(charset-iconv.o): In function `charset_to_utf8_try': charset-iconv.c:(.text+0x425): undefined reference to `libiconv' cc: error: linker command failed with exit code 1 (use -v to see invocation) ../lib-charset/.libs/libcharset.a(charset-iconv.o): In function `charset_to_utf8_begin': charset-iconv.c:(.text+0x49): undefined reference to `libiconv_open' ../lib-charset/.libs/libcharset.a(charset-iconv.o): In function `charset_to_utf8_end': charset-iconv.c:(.text+0x151): undefined reference to `libiconv_close' ../lib-charset/.libs/libcharset.a(charset-iconv.o): In function `charset_to_utf8_reset': charset-iconv.c:(.text+0x211): undefined reference to `libiconv' ../lib-charset/.libs/libcharset.a(charset-iconv.o): In function `charset_to_utf8_try': charset-iconv.c:(.text+0x425): undefined reference to `libiconv' From tss at iki.fi Sun Mar 22 17:06:08 2015 From: tss at iki.fi (Timo Sirainen) Date: Sun, 22 Mar 2015 19:06:08 +0200 Subject: IMAP ANNOTATE Extension RFC5257: priority on roadmap In-Reply-To: <550D285E.50801@agitos.de> References: <550D285E.50801@agitos.de> Message-ID: <3835EC85-05B5-4998-9B47-6210A381A3A1@iki.fi> On 21 Mar 2015, at 10:14, Florian Sager wrote: > > Hi Timo, > > congrats to the merger with OX. > > Currently the implementation of RFC 5257, ANNOTATE-EXPERIMENT-1, has > only low priority on http://wiki2.dovecot.org/Roadmap > I want to explain a scenario that would benefit from annotation support > to - maybe - increase the priority in your roadmap: > > I'm currently working on a project to publish bank customer related > documents inside a banking webinterface (they call it 'postbox'). > > There are different requirements that would be satisfied very well by an > IMAP server as data storage/provider for a long term email/document > archive inside the bank. > > Annotations would be necessary to tag emails with customer numbers and > document IDs to allow a delayed grouping of several emails by such > identifiers (message IDs may not be available for references so > annotations have to be used to compute references indirectly). > Same for sorting and filtering by customer number or account number. > > I would highly appreciate if you at Dovecot could increase the priority > of ANNOTATE in your roadmap. The Roadmap page hasn't been updated for a while now. ANNOTATE is something that I'd like to implement at some point, but it's also quite a large change and likely will have to wait until Dovecot v2.3. You could already implement similar functionality with METADATA, although that would be kind of kludgy. Basically use SETMETADATA folder (/private/vendor/vendor.abcdefg/key/123 "value") where 123 would be the UID. You'd of course somehow need to explicitly delete the metadata entries when messages are expunged. From zucca at systemschmiede.com Sun Mar 22 18:03:41 2015 From: zucca at systemschmiede.com (zucca at systemschmiede.com) Date: Sun, 22 Mar 2015 19:03:41 +0100 Subject: Error after setting up fts /solr for Open-Xchange Message-ID: <550F03FD.9080104@systemschmiede.com> Hi List, i have tried to get fts / solr running with virtual all folder with Open-Xchange. I followed a pretty well written howto from Open-Xchanges Intranet, and i first thought everything was running smooth, as the first one or two searches were successful. However now whenever i try to search again for something in the virtual "all folders" Open-Xchange tells me: *Error* *Folder "virtual.all" has been closed on mail server xxx.xxx.xxx.xxx Probably your request took too long.** * /var/log/mail.err says: Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Panic: file fts-search-serialize.c: line 63 (fts_search_deserialize_add_idx): assertion failed: (*idx < buf->used) Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x6c15f) [0x7f5110f5e15f] -> /usr/lib/dovecot/libdovecot.so.0(+0x6c1be) [0x7f5110f5e1be] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f5110f1740e] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xc847) [0x7f510fb07847] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_search_deserialize_add_nonmatches+0x1c) [0x7f510fb0799c] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd263) [0x7f510fb08263] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd3e6) [0x7f510fb083e6] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa812e) [0x7f511127112e] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblock+0x4b) [0x7f51112717bb] -> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_search_next_nonblock+0x13e) [0x7f510f8f2a7e] -> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_search_next_nonblock+0x7e) [0x7f510f8f29be] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x27) [0x7f511124b277] -> dovecot/imap(+0x1f27f) [0x7f511172527f] -> dovecot/imap(imap_search_start+0xfd) [0x7f51117256fd] -> dovecot/imap(cmd_sort+0x205) [0x7f511171b6c5] -> dovecot/imap(command_exec+0x3c) [0x7f511171fc4c] -> dovecot/imap(+0x18c30) [0x7f511171ec30] -> dovecot/imap(+0x18cea) [0x7f511171ecea] -> dovecot/imap(client_handle_input+0x115) [0x7f511171efb5] -> dovecot/imap(client_input+0x75) [0x7f511171f385] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4e) [0x7f5110f6efbe] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) [0x7f5110f6ffb7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f5110f6f049] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f5110f6f0c8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f5110f1c7b3] -> dovecot/imap(main+0x2ae) [0x7f511171352e] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f5110b85ead] -> dovecot/imap(+0xd69d) [0x7f511171369d] Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Fatal: master: service(imap): child 6594 killed with signal 6 (core dumps disabled) Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Panic: file fts-search-serialize.c: line 63 (fts_search_deserialize_add_idx): assertion failed: (*idx < buf->used) Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x6c15f) [0x7fae1c14c15f] -> /usr/lib/dovecot/libdovecot.so.0(+0x6c1be) [0x7fae1c14c1be] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fae1c10540e] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xc847) [0x7fae1acf5847] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_search_deserialize_add_nonmatches+0x1c) [0x7fae1acf599c] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd263) [0x7fae1acf6263] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd3e6) [0x7fae1acf63e6] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa812e) [0x7fae1c45f12e] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblock+0x4b) [0x7fae1c45f7bb] -> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_search_next_nonblock+0x13e) [0x7fae1aae0a7e] -> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_search_next_nonblock+0x7e) [0x7fae1aae09be] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x27) [0x7fae1c439277] -> dovecot/imap(+0x1f27f) [0x7fae1c91327f] -> dovecot/imap(imap_search_start+0xfd) [0x7fae1c9136fd] -> dovecot/imap(cmd_sort+0x205) [0x7fae1c9096c5] -> dovecot/imap(command_exec+0x3c) [0x7fae1c90dc4c] -> dovecot/imap(+0x18c30) [0x7fae1c90cc30] -> dovecot/imap(+0x18cea) [0x7fae1c90ccea] -> dovecot/imap(client_handle_input+0x115) [0x7fae1c90cfb5] -> dovecot/imap(client_input+0x75) [0x7fae1c90d385] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4e) [0x7fae1c15cfbe] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) [0x7fae1c15dfb7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7fae1c15d049] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fae1c15d0c8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fae1c10a7b3] -> dovecot/imap(main+0x2ae) [0x7fae1c90152e] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7fae1bd73ead] -> dovecot/imap(+0xd69d) [0x7fae1c90169d] Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Fatal: master: service(imap): child 6598 killed with signal 6 (core dumps disabled) I don't even have any fts-search-serialize.c on the system... Maybe someone is able to help me? I know we're nearly there, but i can't find anything on this anywhere. If there's anything else you need to know in order to help, please let me know. Thank you very very much! Sascha _this is set under /opt/open-xchange/etc/findbasic.properties for Open-Xchange__:_ com.openexchange.find.basic.mail.allMessagesFolder = virtual.all _grep -v '^ *\(#.*\)\?$' /etc/dovecot/dovecot-sql.conf:_ driver = mysql connect = host=localhost dbname=dbispconfig user=ispconfig password=e04a9349a47706e4f18bee2782349355 default_pass_scheme = CRYPT password_query = SELECT password FROM mail_user WHERE (login = '%u' OR email = '%u') AND disable%Ls = 'n' user_query = SELECT email as user, maildir as home, CONCAT('maildir:', maildir, '/Maildir') as mail, uid, gid, CONCAT('*:storage=', quota, 'B') AS quota_rule, CONCAT(maildir, '/.sieve') as sieve FROM mail_user WHERE (login = '%u' OR email = '%u') AND `disable%Ls` = 'n' _||dovecot --version: _2.2.13 _dovecot -n: _# 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-37-pve x86_64 Debian 7.8 simfs auth_mechanisms = plain login disable_plaintext_auth = no listen = *,[::] log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/var/vmail/%d/%n/Maildir mail_max_userip_connections = 150 mail_plugins = " acl fts fts_solr virtual" mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = prefix = separator = . type = private } namespace { list = children location = maildir:/var/vmail/%%d/%%n/Maildir:INDEX=/var/vmail/%d/%n/Maildir/shared/%%u prefix = Shared/.%%n/. separator = . subscriptions = no type = shared } namespace virtual { location = virtual:/etc/dovecot/virtual:INDEX=~/virtual mailbox all { special_use = \All } prefix = virtual. separator = . } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { acl = vfile acl_shared_dict = file:/var/vmail/shared-mailboxes.db fts = solr fts_autoindex = yes fts_solr = url=http://localhost:8090/solr/ quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve = /var/vmail/%d/%n/.sieve } protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service imap-login { client_limit = 1000 process_limit = 500 } ssl_ca = References: <0DC5FD73-B4A7-4811-BD7F-7B053BAF47E8@iki.fi> Message-ID: <550F9247.3060605@gedalya.net> This happened after upgrading to 2.2.16. Running on Debian wheezy amd64. We never had such a crash on this server before. However, this too was a single incident. All accounts use Maildir. Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Error: Log synchronization error at seq=0,offset=0 for /stor/mail/domains/----/----/Maildir/dovecot.index: Append with UID 6684, but next_uid = 6685 Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Error: /stor/mail/domains/----/----/Maildir/dovecot.index view syncing failed to apply changes Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Panic: file index-sync.c: line 265 (index_mailbox_sync_next_expunge): assertion failed: (range->seq2 <= ctx->messages_count) Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7b57f) [0x7f34cd27c57f] -> /usr/lib/dovecot/libdovecot.so.0(+0x7b5de) [0x7f34cd27c5de] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f34cd22b0d5] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb5391) [0x7f34cd5a6391] -> dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting](imap_sync_more+0xce) [0x7f34cda5ad2e] -> dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting](+0x22344) [0x7f34cda5b344] -> dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting](cmd_sync_delayed+0x1bf) [0x7f34cda5b67f] -> dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting](client_output+0xe0) [0x7f34cda52910] -> /usr/lib/dovecot/libdovecot.so.0(+0x98a85) [0x7f34cd299a85] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7f34cd28e80b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb) [0x7f34cd28f88b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f34cd28e899] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f34cd28e918] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f34cd2307d3] -> dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting](main+0x2b7) [0x7f34cda46997] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f34cce94ead] -> dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting](+0xdb09) [0x7f34cda46b09] Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Fatal: master: service(imap): child 30144 killed with signal 6 (core dumps disabled) # doveconf -n # 2.2.16: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.7 # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8 auth_master_user_separator = * auth_mechanisms = plain login cram-md5 auth_verbose = yes auth_verbose_passwords = plain dict { expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no login_greeting = Dovecot ready login_log_format_elements = user=<%u> method=%m rip=%r lip=%l pip=%{real_rip} mpid=%e %c session=<%{session}> mail_gid = vmail mail_location = /nowhere mail_plugins = quota expire listescape mail_uid = vmail managesieve_sieve_capability = fileinto envelope encoded-character subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables mailbox date index ihave duplicate namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { antispam_backend = pipe antispam_pipe_program = /usr/bin/ssh antispam_pipe_program_args = -l;spamd;-i;/etc/dovecot/sareport.key;mx1.xxxxx antispam_pipe_program_notspam_arg = revoke antispam_pipe_program_spam_arg = report antispam_pipe_tmpdir = /tmp antispam_spam = Junk antispam_trash_pattern_ignorecase = trash;Deleted * expire = Trash expire2 = Trash/* expire3 = Junk expire_dict = proxy::expire quota = dict:user::proxy::quota quota_rule = *:storage=2G quota_rule2 = Trash:storage=+250M quota_rule3 = Junk:ignore quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=85%% quota-warning 85 %u sieve = ~/.dovecot.sieve sieve_before = /etc/dovecot/sieve-global/fileinto-spam.sieve sieve_dir = ~/sieve sieve_extensions = -vacation -body -reject -enotify -environment -virustest -spamtest sieve_max_actions = 32 sieve_max_redirects = 4 sieve_max_script_size = 10K sieve_quota_max_scripts = 100 sieve_quota_max_storage = 4M } pop3_no_flag_updates = yes protocols = imap pop3 lmtp sieve service auth-worker { user = $default_internal_user } service auth { client_limit = 2448 unix_listener auth-userdb { group = root mode = 0600 user = vmail } } service dict { unix_listener dict { group = root mode = 0600 user = vmail } } service imap-login { process_min_avail = 4 service_count = 0 vsz_limit = 192 M } service imap-postlogin { executable = script-login /usr/local/bin/postlogin unix_listener imap-postlogin { group = root mode = 0600 user = vmail } user = root } service imap { executable = imap imap-postlogin process_limit = 1024 vsz_limit = 128 M } service lmtp { inet_listener lmtp { address = 10.44.23.1 port = 7025 } } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 0 vsz_limit = 64 M } service managesieve { process_limit = 20 } service pop3-login { process_min_avail = 4 service_count = 0 vsz_limit = 192 M } service pop3 { executable = pop3 imap-postlogin process_limit = 1024 } service quota-warning { executable = script /usr/local/bin/quota-warning unix_listener quota-warning { user = vmail } user = dovecot } ssl_cert = Hello list, I have been struggling with establishing a working installation of dovecot with mySQL quota support. I have not been able to find a guide or tutorial that works for the current revisions of software I have at my disposal. Most notably, the error I see in the mail log is as follows : Mar 23 00:55:31 host dovecot: lmtp(328): Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Mar 23 00:55:31 host dovecot: lmtp(328): Error: dlopen(/usr/lib/dovecot/modules/lib11_imap_quota_plugin.so) failed: /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so: undefined symbol: command_unregister Mar 23 00:55:31 host dovecot: lmtp(328): Fatal: Couldn't load required plugins I have attached my dovecot -n output. The installed revision of dovecot is 2.2.9 on ubuntu 14.04.02 LTS. Thank you in advance. -- -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: dovecot-config.txt URL: From vvu at mcra.fr Mon Mar 23 08:36:08 2015 From: vvu at mcra.fr (Vu Ngoc VU) Date: Mon, 23 Mar 2015 09:36:08 +0100 (CET) Subject: fast doveadm search results In-Reply-To: <626BAD45-6551-456A-B7A6-3D7C9B40F4AE@iki.fi> References: <626BAD45-6551-456A-B7A6-3D7C9B40F4AE@iki.fi> Message-ID: Hello Timo, thanks you so much for answering. > Date: Fri, 20 Mar 2015 19:16:55 > From: Timo Sirainen > > On 20 Mar 2015, at 10:37, Vu Ngoc VU wrote: >> >> But to answer your questions, I'm not really interested in purging the cache data. >> I just think that cache has expiration delay. >> The only point is to get this "doveadm search" answering me in minutes instead of hours. >> Writing my original post, I didn't get if it was slow because of: >> - data I'm searching are not cached at all? (headers like From, Date, Message-ID...) >> => I wanted to know if dovecot allows to add some headers. >> Stephen answered to that question. > > By default all headers are added to cache the first time they're accessed in the folder (e.g. via FETCH or SEARCH). Also mails that are newly delivered by Dovecot will add those headers to cache immediately. Wow, I'm wondering how Dovecot manages to know if it is 1st time access. So, no matter if it is from a MUA or with dovecot's administrative commands like doveadm search/fetch ? Yes, delivery is done by dovecot LMTP on a separated server, I plan to move this service into others servers that do imap/pop3. >> - these data a cache, but for an extremely short time, like the user session >> => that's why I asked if it is possible to extend cache validity to at least 48h. >> But for sure, if these data remain forever, it'll be better ! :) > > Dovecot automatically figures out if the data should stay in cache for 1 week or forever. It sounds like something's wrong in your system if it's not already automatically performing fast searches. The first time a search on a header is done it might be slow if the data isn't in cache, but all subsequent times should be very fast. Not hours or minutes but seconds. No need to modify the mail_cache_* settings. It might be helpful if you posted your whole doveconf -n output. I've read on the website and you confirmed here that dovecot tries to be smart about how to manage the cache. But is there settings so I can ask dovecot to never remove cache entries? I don't want dovecot to try to evaluate if the MUA needs/asks these data often. Like I wrote before, I host mails for nearly 10 companies inside a same "group" (sorry, english is not my native language). And mails is a tool overused here, so even if it is rare cases, sometimes, they ask me to delete some mails quickly. Then, I prefer to waste some I/O and disk usage for the cache (or indexes, whatever it is called) to have doveadm search answer fast. If there is no "never_purge_cache" or "dont_try_to_be_smart" setting, would it be some command to run every night with CRON to update/refill the cache? >> - NFS limitations >> => do I have to re-install dovecot on my NFS servers? I prefer not. > > What do you mean by this? You're using NFS now to store emails but with one Dovecot server? That should work fine, although NFS of course always adds some extra overhead. I can paste configuration somewhere, but since I have several servers, maybe we should decide which ones to run `doveconf -n` from. My setup looks like this, I know there are some design errors: - storage bays from DELL (MD1220) with 24 hdd, directly SAS attached to some physical servers - these servers attached to the storage bays are NFS servers - 2 openvz containers running dovecot as director proxy + postfix submission (I called these mailhubs) - 2 openvz containers running dovecot as IMAP/POP3 backends, they are nfs clients to access mailboxes - 1 openvz container acting as MX with postfix and the only one that has public+private ip addresses this server only receives mails for our domains and then "transports" to lmtp servers. It is not part of Director setup. - 1 openvz container called lmtp, this one is nfs client too, and only delivers mails This one only do delivery and is not part of Director setup, I plan to remove it from the "archi". All systems are Ubuntu 14.04 (so it's dovecot 2.2.9 via packages) and filesystem on storage servers is ext4 on hardware RAID-6. I need to migrate others domains hosted on older setup (with Courier). But before that, I prefer having my "new setup" working as expected. And I also want to understand important parts (for me) like caching of messages. To enhance searching performance, I can consider many solutions like integrating SSD, storing indexes/caches in some memcached (we can put 1 TB of RAM on our servers). But, for now, I need to know if I can configure dovecot so that it add headers I need to the cache and never remove these. I'll do further tests. But for the moment, I've tried this: - delete dovecot.index.cache file for a mailbox - running doveadm search on it and check contents of the new cache file with strings command => only headers I ask in search command are stored - running another search with another header as criteria => the cache file now has stored new asked headers. As you wrote that on delivery "all" headers should be stored in cache, I'll do some tests on this today. But for now, running doveadm on same mailboxes as friday is fast. So I still don't know how this caching works, it is "too smart" for me :) Have a nice day/week, and sorry for this long message, I think it's like a mess :/ From gedalya at gedalya.net Mon Mar 23 08:43:15 2015 From: gedalya at gedalya.net (Gedalya) Date: Mon, 23 Mar 2015 04:43:15 -0400 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> Message-ID: <550FD223.4020309@gedalya.net> On 03/23/2015 04:34 AM, rooster wrote: > Hello list, > > I have been struggling with establishing a working installation of dovecot with mySQL quota support. I have not been able to find a guide or tutorial that works for the current revisions of software I have at my disposal. > > Most notably, the error I see in the mail log is as follows : > > Mar 23 00:55:31 host dovecot: lmtp(328): Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so > Mar 23 00:55:31 host dovecot: lmtp(328): Error: dlopen(/usr/lib/dovecot/modules/lib11_imap_quota_plugin.so) failed: /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so: undefined symbol: command_unregister > Mar 23 00:55:31 host dovecot: lmtp(328): Fatal: Couldn't load required plugins > > I have attached my dovecot -n output. The installed revision of dovecot is 2.2.9 on ubuntu 14.04.02 LTS. > > Thank you in advance. > > -- > The imap_quota plugin is relevant only under protocol imap {}. It reports quota usage via the IMAP protocol. Under lmtp and any other protocol that can change quota usage, you need to load only the quota plugin, which does the actual work of accounting and enforcing the quota. From ltirkkon at niksula.hut.fi Mon Mar 23 08:51:35 2015 From: ltirkkon at niksula.hut.fi (Lauri Tirkkonen) Date: Mon, 23 Mar 2015 10:51:35 +0200 Subject: [PATCH] increase fd_limit to max_client_limit automatically In-Reply-To: References: <20150122120123.GK16405@kekkonen.niksula.hut.fi> Message-ID: <20150323085134.GO13088@kekkonen.niksula.hut.fi> On Fri, Mar 20 2015 21:06:12 +0200, Timo Sirainen wrote: > On 22 Jan 2015, at 14:01, Lauri Tirkkonen wrote: > > > > Hi, with a low soft limit on file descriptors, dovecot 2.2.15 warns on > > startup: > > > > Warning: fd limit (ulimit -n) is lower than required under max. load > > (256 < 1000), because of default_client_limit > > > > It could try increasing the limit first, and only report the warning if that > > fails. I'm attaching a patch that does just this. > > It could .. But somehow it doesn't seem like a good idea to me to do > automatically. Maybe the limits are there intentionally. I would make the argument that that's why there are both soft and hard limits. Any program can raise their soft ulimits to the current hard limit - I see the soft limit as more of a guard against misconfiguration than a resource control, so I don't think there is anything wrong with raising it when you have a legitimate reason to do so. On systems with low default fd limits (such as some illumos distributions; see https://www.illumos.org/issues/3772), the alternative is requiring administrators to create startup wrappers for dovecot that raise the limit, but since it's not a privileged operation that just seems silly. -- Lauri Tirkkonen Niksula systems specialist From felix at zandanel.me Mon Mar 23 11:51:42 2015 From: felix at zandanel.me (Felix Zandanel) Date: Mon, 23 Mar 2015 12:51:42 +0100 Subject: Fixing broken UTF-8 handling with MySQL driver In-Reply-To: References: <71AB8C5D-70A5-44B3-81B9-50D2429F9155@zandanel.me> Message-ID: > Am 12.03.2015 um 14:55 schrieb Timo Sirainen : > > On 11 Mar 2015, at 21:31, Felix Zandanel wrote: >> >> Although my MySQL installation is set up to use UTF-8 as the default charset on every level (config, database, table and field), and the character_set_* runtime variables all yield the value "utf8", it still seems that the mysql client library must be instructed to actually use UTF-8 explicitly. Adding the following statement to driver_mysql_connect() fixes the issue for me: >> >> mysql_options(db->mysql, MYSQL_SET_CHARSET_NAME, "utf8"); > > I think you can also add to /etc/my.cnf : > > [client] > default-character-set = utf8 Sorry for the late reply. You were so right, that simple line did the trick. My fault, I didn't read the whole charset documentation of MySQL. It's a shame that UTF-8 isn't the default setting. Anyway, as dovecot's internals expect all input strings to be UTF-8, wouldn't it be useful to enforce UTF-8 in the database drivers? Using anything else than ASCII / UTF-8 for dovecot's MySQL connections doesn't really make sense, I think. Also, a "default-character-set = utf8" line in my.cnf is a system wide configuration, which might break other software interacting with MySQL?in theory. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: Message signed with OpenPGP using GPGMail URL: From andreas.kasenides at cs.ucy.ac.cy Mon Mar 23 14:08:37 2015 From: andreas.kasenides at cs.ucy.ac.cy (Andreas Kasenides) Date: Mon, 23 Mar 2015 16:08:37 +0200 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <8250CA2C-936F-468D-B5C6-88954DC6FB2F@iki.fi> References: <8250CA2C-936F-468D-B5C6-88954DC6FB2F@iki.fi> Message-ID: <55101E65.6080508@cs.ucy.ac.cy> I find it extremely interesting that no one has commented on the merger of Dovecot Oy and Open-Xchange AG as announced by Timo on the 19th. Is this something that was known a long time ago and I missed? OK checked the on-line archive of the mailing list, no comments there - its not my email set-up - LOL. I am usually emotionally (at least) against of open-source projects loosing their independence to large corporations. Possibly due to bad experiences in the past when OSS were driven from Open to Obscure in the process of trying to make money out of them. I have several examples in mind but I will not give names. At least that is the impression I have which might be entirely wrong since when big companies begin to ask for large sums of money we just have to move away due to the small budget. Anyway this is not to about judging the move. Which I cannot do since I have no knowledge whatsover of the Dovecot enterprise internals and the difficulties that come with managing a leading software product. And, secondly, since I am (my employer ie) a non paying customer!! I was just struck by the fact that no one has commented on it. I wish Dovecot the best in the new environment. Andreas On 19/03/15 12:26, Timo Sirainen wrote: > Hi all, > > Today I can finally announce that Dovecot Oy company has merged with Open-Xchange AG. This helps us to get more Dovecot developers, support people and so on. Most importantly, eventually it should allow me to get back to doing what I like the most: Designing new and interesting stuff for Dovecot and perfecting the old stuff :) OX is a great match to Dovecot going forward. They also really like open source and share our plans for the future. Nothing big will change as a result of this merger: Dovecot will stay Dovecot with its own name and release schedules. We're not going to force OX and Dovecot to be the same product, other than having a somewhat deeper integration between them. > > Here are the press release links about it: > http://www.dovecot.fi/open-xchange-and-dovecot-announce-merger-to-create-worlds-leading-open-source-messaging-software-provider/ > http://www.open-xchange.com/dovecot > http://www.open-xchange.com/announcements/18 -- Andreas Kasenides Senior IT Officer Dept. of Computer Science, University of Cyprus Tel: 22892714, Fax: 22892701 (5B4ANK) From adi at ente.limmat.ch Mon Mar 23 15:17:10 2015 From: adi at ente.limmat.ch (Adrian Zaugg) Date: Mon, 23 Mar 2015 16:17:10 +0100 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <55101E65.6080508@cs.ucy.ac.cy> References: <8250CA2C-936F-468D-B5C6-88954DC6FB2F@iki.fi> <55101E65.6080508@cs.ucy.ac.cy> Message-ID: <55102E76.1040607@ente.limmat.ch> I think everyone shares your concerns. But there are no rules that the outcome of this merger must get something bad, so let's see what happens. I hope that it's true what Timo said and that dovecot can evolve and get even better as it is today. Good luck guys! Regards, Adrian. On 23.03.15 15:08, Andreas Kasenides wrote: > I find it extremely interesting that no one has commented on the merger > of Dovecot Oy and Open-Xchange AG as announced by Timo on the 19th. Is > this something that was known a long time ago and I missed? OK checked > the on-line archive of the mailing list, no comments there - its not my > email set-up - LOL. > I am usually emotionally (at least) against of open-source projects > loosing their independence to large corporations. Possibly due to bad > experiences in the past when OSS were driven from Open to Obscure in the > process of trying to make money out of them. I have several examples in > mind but I will not give names. At least that is the impression I have > which might be entirely wrong since when big companies begin to ask for > large sums of money we just have to move away due to the small budget. > Anyway this is not to about judging the move. Which I cannot do since I > have no knowledge whatsover of the Dovecot enterprise internals and the > difficulties that come with managing a leading software product. And, > secondly, since I am (my employer ie) a non paying customer!! > I was just struck by the fact that no one has commented on it. > > I wish Dovecot the best in the new environment. > > Andreas > > On 19/03/15 12:26, Timo Sirainen wrote: >> Hi all, >> >> Today I can finally announce that Dovecot Oy company has merged with >> Open-Xchange AG. This helps us to get more Dovecot developers, support >> people and so on. Most importantly, eventually it should allow me to >> get back to doing what I like the most: Designing new and interesting >> stuff for Dovecot and perfecting the old stuff :) OX is a great match >> to Dovecot going forward. They also really like open source and share >> our plans for the future. Nothing big will change as a result of this >> merger: Dovecot will stay Dovecot with its own name and release >> schedules. We're not going to force OX and Dovecot to be the same >> product, other than having a somewhat deeper integration between them. >> >> Here are the press release links about it: >> http://www.dovecot.fi/open-xchange-and-dovecot-announce-merger-to-create-worlds-leading-open-source-messaging-software-provider/ >> >> http://www.open-xchange.com/dovecot >> http://www.open-xchange.com/announcements/18 > > From e1c1bac6253dc54a1e89ddc046585792 at posteo.net Mon Mar 23 17:34:21 2015 From: e1c1bac6253dc54a1e89ddc046585792 at posteo.net (Philipp) Date: Mon, 23 Mar 2015 18:34:21 +0100 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <55101E65.6080508@cs.ucy.ac.cy> References: <8250CA2C-936F-468D-B5C6-88954DC6FB2F@iki.fi> <55101E65.6080508@cs.ucy.ac.cy> Message-ID: <6f76c99be9bd071eb0402df9b96a1291@posteo.de> Am 23.03.2015 15:08 schrieb Andreas Kasenides: > I am usually emotionally (at least) against of open-source projects > loosing their independence to large corporations. Possibly due to bad > experiences in the past when OSS were driven from Open to Obscure in > the process of trying to make money out of them. 2ct from me-- I put it this way: if dovecot would have been a "pure" OSS before getting bought OAX, then response might would've been different. But Dovecot OY is "making money out of it" for how long now? Five years - and that's not only "selling support". Nginx went the same path - I'd say even more aggressive, given feature set differences. Timo and his team has shown that they care about their OSS tree, e.g. do you really think all that replication and director "stuff" came from pure boredom? Show me OSS for cyrus, courier, .. on that level of "enterprisey". Timo clearly wrote that this company merge is about putting together the PAID services efforts (think hotlines and such) and not mangling OX and dovecot into some "blob" (well, might happen one day in the future, but you would never know about *any* OSS unless you find a crystal globe telling you). If my customers are reluctant to move IMAP servers to dovecot, because for now they have/had OX+whatever.. well, what better argument could one have then: Did you know that they are one company and have shared support now? .. Just thinkin' and sayin'. From patrick at coffininc.com Mon Mar 23 18:41:36 2015 From: patrick at coffininc.com (Patrick Coffin) Date: Mon, 23 Mar 2015 11:41:36 -0700 Subject: Dovecot Oy merger with Open-Xchange AG Message-ID: To Timo and the Dovecot guys - congratulations! I'm sure this merger with OpenXchange is going to provide you with a lot of resources and opportunities. As a longtime user of dovecot, I do have a few concerns. I wonder if you can answer some questions for me. You say that OpenXchange really likes open source and shares your plans for the future. Is this a commitment that future versions core dovecot product will remain free and truly open source? According to Wikipedia (http://en.wikipedia.org/wiki/Open-Xchange#Licensing ) , OpenXchange's backend is GPL'd, but the front-end is not - it's released under "Creative Common's Share Alike, Non Commercial, Attribution". The article points out: "The restriction to Non Commercial in the Creative Commons license for the Frontend restricts re-distribution to third parties, i.e. hosted deployments for third parties. However, since the front-end license prohibits commercial re-distribution, the software is neither free software nor open source software since the definitions of both require such re-distribution to be permitted" Do you expect that dovecot is moving in that direction? Here's why I'm asking: As a hosted email provider, I've long used dovecot, and been quite happy with it. But I have some concerns, as is common when any popular open source project gets acquired by a commercial entitiy. The current dovecot license is a mixture of the MIT and LGPL licenses. Will this remain? Or is dovecot going to go the way of OpenXchange licensing? What about other pieces of the dovecot ecosystem, such as the Object Storage plugin - will that remain closed source and proprietary? Or will you follow the lead of companies like RedHat and be truly open source? Is there a possibility that future versions of dovecot that contain what we might consider core features will be available only in the commercial version of the product? If I base a part of my business on a piece of software I've been running for the last 10 years, am I going to find myself in trouble in a year or two, when some new version of dovecot comes out with changes that I need, and I have to move to a commercial product, which I may or may not be able to afford? I'd love to hear that you're going to be following a model like RedHat did when they acquired GlusterFS and created the RedHat Storage Server. Gluster development is still going strong, and still completely open source. But they make money from people like me who know that by buying a contract, we can get the kind of support we need for such a critical part of our infrastructure. Again, congratulations and, as always, thanks for all the hard work creating dovecot in the first place. Patrick From michal at mailmix.pl Mon Mar 23 20:32:30 2015 From: michal at mailmix.pl (=?UTF-8?B?TWljaGHFgiBHacW8ecWEc2tp?=) Date: Mon, 23 Mar 2015 21:32:30 +0100 Subject: [PATCH] [dovecot 2.2.9] Quota warnings ignored with FS quotas In-Reply-To: <54E3907F.8060009@mailmix.pl> References: <546F8CCE.3070400@localdomain.pl> <54E3907F.8060009@mailmix.pl> Message-ID: <5510785E.80702@mailmix.pl> On 17.02.2015 20:03, Micha? Gi?y?ski wrote: > > On 21.11.2014 20:04, Grzegorz Nosek wrote: >> Hi all, >> >> I noticed that in some circumstances quota warnings are ignored. The >> bug arises when both of the following are used: >> >> 1. percentage-based quota warnings, i.e.: >> >> quota_warning = storage=1%% quota-warning 1 %u >> >> 2. filesystem quota backend (and probably others, except for quotas >> configured directly in dovecot config) >> >> Percentage-based quota warnings have rule.bytes_limit recalculated >> based on root_set->default_rule.bytes_limit, however this value is >> zero when FS quotas are in use. Real quota values (from quotactl) are >> fetched very late, in quota_warnings_execute() but at that point no >> recalculation happens. As the warning rules have bytes_limit==0, >> they're effectively ignored. >> >> The patch below enables quota warnings to be sent when using >> filesystem (and possibly maildirsize-based) quotas. >> >> Based and tested on Ubuntu 14.04's dovecot 2.2.9. >> >> Best regards, >> Grzegorz Nosek >> >> >> diff --git a/src/plugins/quota/quota.c b/src/plugins/quota/quota.c >> index adbd70d..8e4d7e0 100644 >> --- a/src/plugins/quota/quota.c >> +++ b/src/plugins/quota/quota.c >> @@ -1163,6 +1163,8 @@ static void quota_warnings_execute(struct >> quota_transaction_context *ctx, >> &count_current, &count_limit) < 0) >> return; >> >> + quota_root_recalculate_relative_rules(root->set, bytes_limit, >> count_limit); >> + >> bytes_before = bytes_current - ctx->bytes_used; >> count_before = count_current - ctx->count_used; >> for (i = 0; i < count; i++) { > Hi, > this patch realy fix the bug in quota_warning. Is it possible to add > this patch to the next release ? > Hi, I saw that new dovecot was released. Is the quota_warning has been improved ? -- Pozdrawiam / Best Regards Micha? Gi?y?ski From yawowb+dovecot at nuclei.ca Mon Mar 23 22:24:41 2015 From: yawowb+dovecot at nuclei.ca (rooster) Date: Mon, 23 Mar 2015 15:24:41 -0700 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: <550FD223.4020309@gedalya.net> References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> <550FD223.4020309@gedalya.net> Message-ID: > On 2015-03-23, at 1:43 AM, Gedalya wrote: > > The imap_quota plugin is relevant only under protocol imap {}. It reports quota usage via the IMAP protocol. Under lmtp and any other protocol that can change quota usage, you need to load only the quota plugin, which does the actual work of accounting and enforcing the quota. Thank you for the hint. So what I need to do is remove $mail_plugins from each "mail_plugins =? (I think the information I found was wrong in using $mail_plugins on each mail_plugins =, based on what you said above) , and only set the correct corresponding mail plugins? Is my example below, correct? e.g. : protocol imap { mail_plugins = zlib quota imap_quota imap_zlib antispam } protocol lmtp { mail_plugins = zlib quota } protocol lda { mail_plugins = } -- From gedalya at gedalya.net Mon Mar 23 22:38:18 2015 From: gedalya at gedalya.net (Gedalya) Date: Mon, 23 Mar 2015 18:38:18 -0400 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> <550FD223.4020309@gedalya.net> Message-ID: <551095DA.4020201@gedalya.net> On 03/23/2015 06:24 PM, rooster wrote: >> On 2015-03-23, at 1:43 AM, Gedalya wrote: >> >> The imap_quota plugin is relevant only under protocol imap {}. It reports quota usage via the IMAP protocol. Under lmtp and any other protocol that can change quota usage, you need to load only the quota plugin, which does the actual work of accounting and enforcing the quota. > > Thank you for the hint. So what I need to do is remove $mail_plugins from each "mail_plugins =? (I think the information I found was wrong in using $mail_plugins on each mail_plugins =, based on what you said above) That idea is to set the global mail_plugins to contain the plugins you would want enabled in *all* places. If there is not a single such plugin, then indeed you can set the global mail_plugins to empty, but then you should still use $mail_plugins everywhere because then it would just work, should you ever add a plugin to the global list. > , and only set the correct corresponding mail plugins? Is my example below, correct? > > e.g. : > > protocol imap { > mail_plugins = zlib quota imap_quota imap_zlib antispam > } > > protocol lmtp { > mail_plugins = zlib quota > } > > protocol lda { > mail_plugins = I don't see why you shouldn't add quota here, and zlib too in your case. If you don't use lda at all, it still won't hurt. > } > > -- Finally, doveadm is sort of a "protocol" too and any protocol for which you did not specify its own "mail_plugins = ..." will use the global value itself. So for example here you explicitly turned off all plugins for lda, overriding the global value. And doveadm will use the global value. So just take it all into consideration. From yawowb+dovecot at nuclei.ca Tue Mar 24 04:50:11 2015 From: yawowb+dovecot at nuclei.ca (rooster) Date: Mon, 23 Mar 2015 21:50:11 -0700 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: <551095DA.4020201@gedalya.net> References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> <550FD223.4020309@gedalya.net> <551095DA.4020201@gedalya.net> Message-ID: <9EEAC044-2C2B-40F7-8077-B9734FB061DF@nuclei.ca> > On 2015-03-23, at 3:38 PM, Gedalya wrote: > > That idea is to set the global mail_plugins to contain the plugins you would want enabled in *all* places. If there is not a single such plugin, then indeed you can set the global mail_plugins to empty, but then you should still use $mail_plugins everywhere because then it would just work, should you ever add a plugin to the global list. That makes sense. :) >> protocol lda { >> mail_plugins = > I don't see why you shouldn't add quota here, and zlib too in your case. If you don't use lda at all, it still won't hurt. I?m certain it is set. I will confirm my current configuration in each file and post the results. > Finally, doveadm is sort of a "protocol" too and any protocol for which you did not specify its own "mail_plugins = ..." will use the global value itself. > So for example here you explicitly turned off all plugins for lda, overriding the global value. And doveadm will use the global value. So just take it all into consideration. Thank you for the tidbit. That is good to know. Here are the currently set parameters for mail_plugins in each config file : 10-mail.conf : mail_plugins = $mail_plugins zlib quota 15-lda.conf : mail_plugins = $mail_plugins sieve 20-imap.conf : mail_plugins = $mail_plugins imap_zlib imap_quota antispam 20-lmtp.conf : mail_plugins = $mail_plugins sieve If the above are the correct settings, why do I get the following errors? Mar 23 00:55:31 host dovecot: lmtp(328): Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Mar 23 00:55:31 host dovecot: lmtp(328): Error: dlopen(/usr/lib/dovecot/modules/lib11_imap_quota_plugin.so) failed: /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so: undefined symbol: command_unregister Mar 23 00:55:31 host dovecot: lmtp(328): Fatal: Couldn't load required plugins -- From gedalya at gedalya.net Tue Mar 24 04:59:58 2015 From: gedalya at gedalya.net (Gedalya) Date: Tue, 24 Mar 2015 00:59:58 -0400 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: <9EEAC044-2C2B-40F7-8077-B9734FB061DF@nuclei.ca> References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> <550FD223.4020309@gedalya.net> <551095DA.4020201@gedalya.net> <9EEAC044-2C2B-40F7-8077-B9734FB061DF@nuclei.ca> Message-ID: <5510EF4E.9080504@gedalya.net> On 03/24/2015 12:50 AM, rooster wrote: > 10-mail.conf : mail_plugins = $mail_plugins zlib quota > > 15-lda.conf : mail_plugins = $mail_plugins sieve > > 20-imap.conf : mail_plugins = $mail_plugins imap_zlib imap_quota antispam > > 20-lmtp.conf : mail_plugins = $mail_plugins sieve > > > If the above are the correct settings, why do I get the following errors? > > Mar 23 00:55:31 host dovecot: lmtp(328): Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so > Mar 23 00:55:31 host dovecot: lmtp(328): Error: dlopen(/usr/lib/dovecot/modules/lib11_imap_quota_plugin.so) failed: /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so: undefined symbol: command_unregister > Mar 23 00:55:31 host dovecot: lmtp(328): Fatal: Couldn't load required plugins Doesn't make much sense. If that's the current config (and you restarted dovecot) then that shouldn't be happening. A good way to understand dovecot's config as it actually got applied is to run 'doveconf -n', try to review the output, and post here after masking out private info if you need further help. From nick.z.edwards at gmail.com Tue Mar 24 05:15:39 2015 From: nick.z.edwards at gmail.com (Nick Edwards) Date: Tue, 24 Mar 2015 15:15:39 +1000 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <55102E76.1040607@ente.limmat.ch> References: <8250CA2C-936F-468D-B5C6-88954DC6FB2F@iki.fi> <55101E65.6080508@cs.ucy.ac.cy> <55102E76.1040607@ente.limmat.ch> Message-ID: On 3/24/15, Adrian Zaugg wrote: > I think everyone shares your concerns. But there are no rules that the > outcome of this merger must get something bad, so let's see what > happens. I hope that it's true what Timo said and that dovecot can > evolve and get even better as it is today. Good luck guys! > Indeed, and quite the opposite to MySQL when we all knew that oracle would destroy it or constrain it, before Sun sold out to them, and what we all forecast of course came true, oracle has done so badly with it, its original author Monty came back with a fork, one that every distro just about has changed to! Infact I dont know anyone using mysql today, most sane admins moved to mariadb where it is in safe hands. hopefuly Timo has a legal back door open to do the same should things change and go hte way mysql did. From skdovecot at smail.inf.fh-brs.de Tue Mar 24 08:42:43 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen) Date: Tue, 24 Mar 2015 09:42:43 +0100 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> Message-ID: <55112383.8080909@smail.inf.fh-brs.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 rooster wrote: > Most notably, the error I see in the mail log is as follows : > > Mar 23 00:55:31 host dovecot: lmtp(328): Debug: Module loaded: > /usr/lib/dovecot/modules/lib10_quota_plugin.so Mar 23 00:55:31 host > dovecot: lmtp(328): Error: > dlopen(/usr/lib/dovecot/modules/lib11_imap_quota_plugin.so) failed: > /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so: undefined > symbol: command_unregister it's a linker problem. Are you sure, /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so exists at all, is readable, is from the very same Dovecot version? - -- Steffen -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQEVAwUBVREjg3z1H7kL/d9rAQJ8xQgAw5eeh/Fqx5dDzjf+CHTAVV8NFHRG9pC3 DUIXW11YhJTMk7xEyyCDo0O3BLZHNv4UL1ekrKe4OwOyd2qiNkXnuGfX5kOhEscx wUBVgeIMj5/TUfY1B3k4+h8nCQD9fsMDV+D5cbrLjOZWrQJfJQl0VBXJ0/GXpefX BfAnWZ3HBoBbXJ64po5YVUhickEdXb9k2Li7XqUzeYOzdWUbXx/ZQMJC2BX855iV x3KDRKCBHinRJysA761xXJo0vbOmmgEBJrdE3PWXCODu2k+B+SxCEwboaqhB5Ak6 CV3DGVDJ6h5wLca6Ce2EOEI00iOOHfBYP3OLTIgvAp7MuoeLu223cw== =W8lh -----END PGP SIGNATURE----- From yawowb+dovecot at nuclei.ca Tue Mar 24 09:05:06 2015 From: yawowb+dovecot at nuclei.ca (rooster) Date: Tue, 24 Mar 2015 02:05:06 -0700 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: <55112383.8080909@smail.inf.fh-brs.de> References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> <55112383.8080909@smail.inf.fh-brs.de> Message-ID: > On 2015-03-24, at 1:42 AM, Steffen wrote: > > it's a linker problem. Are you sure, > /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so exists at all, is > readable, is from the very same Dovecot version? > > - -- > Steffen It might be a operating permissions issue then. The file(s) exist but the permissions are root:root on /usr/lib/dovecot and everything below. Does this mean I need to change the auth workers "user = " parameter? Some are set to mail and some to the virtual mail user (vmail). Here are the dovecot packages I have installed : ii dovecot-antispam 2.0+20130822-2build1 ii dovecot-core 1:2.2.9-1ubuntu2.1 ii dovecot-imapd 1:2.2.9-1ubuntu2.1 ii dovecot-lmtpd 1:2.2.9-1ubuntu2.1 ii dovecot-managesieved 1:2.2.9-1ubuntu2.1 ii dovecot-mysql 1:2.2.9-1ubuntu2.1 ii dovecot-pop3d 1:2.2.9-1ubuntu2.1 ii dovecot-sieve 1:2.2.9-1ubuntu2.1 -- From skdovecot at smail.inf.fh-brs.de Tue Mar 24 09:33:10 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen) Date: Tue, 24 Mar 2015 10:33:10 +0100 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> <55112383.8080909@smail.inf.fh-brs.de> Message-ID: <55112F56.5010100@smail.inf.fh-brs.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 rooster wrote: > >> On 2015-03-24, at 1:42 AM, Steffen >> wrote: >> >> it's a linker problem. Are you sure, >> /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so exists at >> all, is readable, is from the very same Dovecot version? >> > It might be a operating permissions issue then. The file(s) exist > but the permissions are root:root on /usr/lib/dovecot and > everything below. What are the permissions of the requested file itself? You say who owns the directory entries, but do not mention the permissions, please run: ls -alR /usr/lib/dovecot/modules/ > Does this mean I need to change the auth workers "user = " > parameter? Some are set to mail and some to the virtual mail user > (vmail). No. - -- Steffen -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQEVAwUBVREvVnz1H7kL/d9rAQK6lAgAh5K8YAf+jPmfCpmmTv2etx7rpIN1yvOC NcLwbl4G1BqL7t9PMkCw64Nk6f3dOaqm6XhE4a1ow4eVR0NxzU5f24yRWAiro4bd M70ezo8fRrESy+fennRsw1++NXj6YfeWv/Pab+vpLuaxDAyGng9ST+UeOJQ1y84f Rko0Osg1+7bAJPu3L1FaK+uwBh6+8W4YZKfaas6BVq/QCcehFqyMnSmvZLWMjF2b eby9npwXQlx4zNC9Hya5Wo6uX5CiNSv6hj6O5iUwD4Lox+a7W2Veemb1SQwvoCMF cz8ap4QDcwqzgw0UnjU7I1j0bqVQBXcbaYSBcxZ9voPPtd5B/shUnA== =RjrD -----END PGP SIGNATURE----- From yawowb+dovecot at nuclei.ca Tue Mar 24 11:02:32 2015 From: yawowb+dovecot at nuclei.ca (rooster) Date: Tue, 24 Mar 2015 04:02:32 -0700 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: <55112F56.5010100@smail.inf.fh-brs.de> References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> <55112383.8080909@smail.inf.fh-brs.de> <55112F56.5010100@smail.inf.fh-brs.de> Message-ID: <6BD41A7B-3DCD-4F71-B7B9-79EC9D7D9F96@nuclei.ca> > On 2015-03-24, at 2:33 AM, Steffen wrote: > > What are the permissions of the requested file itself? > You say who owns the directory entries, but do not mention the > permissions, please run: > > ls -alR /usr/lib/dovecot/modules/ all directories are drwx r-x r-x all files are -rw -r- - r- - all links are lrwx rwx rwx >> Does this mean I need to change the auth workers "user = " >> parameter? Some are set to mail and some to the virtual mail user >> (vmail). > > No. Okay. -- From florent at coppint.com Tue Mar 24 11:27:46 2015 From: florent at coppint.com (Florent B) Date: Tue, 24 Mar 2015 12:27:46 +0100 Subject: Option to not add "Received" header ? Message-ID: <55114A32.4090201@coppint.com> Hi everyone, I use Dovecot in lmtp mode to receive mails. I would like an option to tell Dovecot to not add a "Reveived" header on each server (I use a director, so Director also adds this header). Is it possible to do this ? Or could it be a future feature ? Thank you. Florent From florent at coppint.com Tue Mar 24 11:33:46 2015 From: florent at coppint.com (Florent B) Date: Tue, 24 Mar 2015 12:33:46 +0100 Subject: Option to not add "Received" header ? In-Reply-To: <55114A32.4090201@coppint.com> References: <55114A32.4090201@coppint.com> Message-ID: <55114B9A.9090209@coppint.com> I know about RFC's, but that could be an option, not enabled by default. From skdovecot at smail.inf.fh-brs.de Tue Mar 24 13:07:54 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Tue, 24 Mar 2015 14:07:54 +0100 (CET) Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: <6BD41A7B-3DCD-4F71-B7B9-79EC9D7D9F96@nuclei.ca> References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> <55112383.8080909@smail.inf.fh-brs.de> <55112F56.5010100@smail.inf.fh-brs.de> <6BD41A7B-3DCD-4F71-B7B9-79EC9D7D9F96@nuclei.ca> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 24 Mar 2015, rooster wrote: >> On 2015-03-24, at 2:33 AM, Steffen wrote: >> >> What are the permissions of the requested file itself? >> You say who owns the directory entries, but do not mention the >> permissions, please run: >> >> ls -alR /usr/lib/dovecot/modules/ > > all directories are drwx r-x r-x > all files are -rw -r- - r- - > all links are lrwx rwx rwx :-) OK, this reply doesn't necessarily answers my question, please run & post output of: file /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so ldd /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so ls -al /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVRFhqnz1H7kL/d9rAQKIqggAlYlwSQjX07bKpb11IPgsM3xl9/Xpx8Lf 2etmiJeBei8y1vyG5xwwVe4gtBTzCowIp5U8UwFVYo/Gv0RoLFXaLXGebfnAIkxD RZMzVqGBNK7qDb0K76OrKiaaLiX8xFAniQDu3ZZof9XadMCvBckvDkLcG+MdW7Tz l9xNQKvBkyeNMWW7Gn1TbRVLEE6URi7gjYheFFWM5cWAS6aMNPCa1+OE86ZNp3f8 xBKys7xpjHpoCNbc84OvwB0cjsbFJkY7eR9BLCtC3+tisEdVCDkYpwCxaqA8vH3m 2UsltJ8WmQ00NIwIXUhJ+mm9XBLQGXRkibI5R4I1OGeU+XtttlapKg== =eurE -----END PGP SIGNATURE----- From tss at iki.fi Tue Mar 24 17:49:31 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 24 Mar 2015 18:49:31 +0100 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: References: Message-ID: <3CB82A3E-E3FB-44A0-879E-C08A3E2DA0E7@iki.fi> On 23 Mar 2015, at 19:41, Patrick Coffin wrote: > > > To Timo and the Dovecot guys - congratulations! I'm sure this merger with OpenXchange is going to provide you with a lot of resources and opportunities. > > As a longtime user of dovecot, I do have a few concerns. I wonder if you can answer some questions for me. I could probably give better answers after talking to other people, but for now: > You say that OpenXchange really likes open source and shares your plans for the future. Is this a commitment that future versions core dovecot product will remain free and truly open source? I haven't heard anyone at OX asking us to close down anything or change any licenses. > The current dovecot license is a mixture of the MIT and LGPL licenses. Will this remain? Or is dovecot going to go the way of OpenXchange licensing? It would be difficult to change Dovecot license at this point since there are so many outside contributions owning copyrights. > What about other pieces of the dovecot ecosystem, such as the Object Storage plugin - will that remain closed source and proprietary? Or will you follow the lead of companies like RedHat and be truly open source? > > Is there a possibility that future versions of dovecot that contain what we might consider core features will be available only in the commercial version of the product? > > If I base a part of my business on a piece of software I've been running for the last 10 years, am I going to find myself in trouble in a year or two, when some new version of dovecot comes out with changes that I need, and I have to move to a commercial product, which I may or may not be able to afford? What you have now won't be taken away. IMAP hasn't changed much for a long time, so I think it's unlikely that a new version would have something that you really can't live without. BTW. PowerDNS also announced their merger with Open-Xchange today. I think that should also be reassuring that there is now another open source project that is happy about their OX-merger. http://blog.powerdns.com/2015/03/24/powerdns-and-open-xchange-agree-to-merge/ From yawowb+dovecot at nuclei.ca Tue Mar 24 21:18:07 2015 From: yawowb+dovecot at nuclei.ca (rooster) Date: Tue, 24 Mar 2015 14:18:07 -0700 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> <55112383.8080909@smail.inf.fh-brs.de> <55112F56.5010100@smail.inf.fh-brs.de> <6BD41A7B-3DCD-4F71-B7B9-79EC9D7D9F96@nuclei.ca> Message-ID: <1E597353-D4A3-43D7-A510-6A9552FC7890@nuclei.ca> > On 2015-03-24, at 6:07 AM, Steffen Kaiser wrote: > >> all directories are drwx r-x r-x >> all files are -rw -r- - r- - >> all links are lrwx rwx rwx > > :-) OK, this reply doesn't necessarily answers my question, please run & post output of: > > file /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so > ldd /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so > ls -al /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so I did what I thought you requested but I see now, not so. :) Here is the output as requested (thank you for the continued assistance). user at host: # file /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so: ELF 32-bit MSB shared object, PowerPC or cisco 4500, version 1 (SYSV), dynamically linked, BuildID[sha1]=f7e458d8845c6f4131d16cb69c30c22446f60550, stripped user at host: # ldd /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so linux-vdso32.so.1 => (0x00100000) lib10_quota_plugin.so => /usr/lib/dovecot/modules/lib10_quota_plugin.so (0x6ffaa000) libc.so.6 => /lib/powerpc-linux-gnu/libc.so.6 (0x6fe0b000) /lib/ld.so.1 (0x206e6000) user at host: # ls -al /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so -rw-r--r-- 1 root root 9512 May 14 2014 /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so -- From brong at fastmail.fm Wed Mar 25 01:04:00 2015 From: brong at fastmail.fm (Bron Gondwana) Date: Wed, 25 Mar 2015 12:04:00 +1100 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <6f76c99be9bd071eb0402df9b96a1291@posteo.de> References: <8250CA2C-936F-468D-B5C6-88954DC6FB2F@iki.fi> <55101E65.6080508@cs.ucy.ac.cy> <6f76c99be9bd071eb0402df9b96a1291@posteo.de> Message-ID: <1427245440.990524.244834465.7725C389@webmail.messagingengine.com> On Tue, Mar 24, 2015, at 04:34 AM, Philipp wrote: > Show me OSS for cyrus, courier, .. on that level of "enterprisey". Hey, FastMail has been contributing pretty advanced stuff to Cyrus for years, and it's all open. Back on topic - contrats Timo. Hope the merger works well for you. It's good to have two strong open-source mail servers out there. Bron. -- Bron Gondwana brong at fastmail.fm From nick.z.edwards at gmail.com Wed Mar 25 12:23:39 2015 From: nick.z.edwards at gmail.com (Nick Edwards) Date: Wed, 25 Mar 2015 22:23:39 +1000 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <3CB82A3E-E3FB-44A0-879E-C08A3E2DA0E7@iki.fi> References: <3CB82A3E-E3FB-44A0-879E-C08A3E2DA0E7@iki.fi> Message-ID: On 3/25/15, Timo Sirainen wrote: > On 23 Mar 2015, at 19:41, Patrick Coffin wrote: >> >> >> To Timo and the Dovecot guys - congratulations! I'm sure this merger with >> OpenXchange is going to provide you with a lot of resources and >> opportunities. >> >> As a longtime user of dovecot, I do have a few concerns. I wonder if you >> can answer some questions for me. > > I could probably give better answers after talking to other people, but for > now: So you no longer have the final say with dovecot. > > What you have now won't be taken away. IMAP hasn't changed much for a long > time, so I think it's unlikely that a new version would have something that > you really can't live without. > So there *is* a chance it will be commercialised From skdovecot at smail.inf.fh-brs.de Wed Mar 25 12:29:45 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 25 Mar 2015 13:29:45 +0100 (CET) Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: <1E597353-D4A3-43D7-A510-6A9552FC7890@nuclei.ca> References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> <55112383.8080909@smail.inf.fh-brs.de> <55112F56.5010100@smail.inf.fh-brs.de> <6BD41A7B-3DCD-4F71-B7B9-79EC9D7D9F96@nuclei.ca> <1E597353-D4A3-43D7-A510-6A9552FC7890@nuclei.ca> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 24 Mar 2015, rooster wrote: >> On 2015-03-24, at 6:07 AM, Steffen Kaiser wrote: >> >>> all directories are drwx r-x r-x >>> all files are -rw -r- - r- - >>> all links are lrwx rwx rwx >> >> :-) OK, this reply doesn't necessarily answers my question, please run & post output of: >> >> file /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so >> ldd /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so >> ls -al /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so > > > I did what I thought you requested but I see now, not so. :) > > Here is the output as requested (thank you for the continued assistance). > > user at host: # file /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so > /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so: ELF 32-bit MSB shared object, PowerPC or cisco 4500, version 1 (SYSV), dynamically linked, BuildID[sha1]=f7e458d8845c6f4131d16cb69c30c22446f60550, stripped > > user at host: # ldd /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so > linux-vdso32.so.1 => (0x00100000) > lib10_quota_plugin.so => /usr/lib/dovecot/modules/lib10_quota_plugin.so (0x6ffaa000) > libc.so.6 => /lib/powerpc-linux-gnu/libc.so.6 (0x6fe0b000) > /lib/ld.so.1 (0x206e6000) > > user at host: # ls -al /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so > -rw-r--r-- 1 root root 9512 May 14 2014 /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so it's a pity, I hoped there is a symlink problem. Could you please try the three commands with this file: /usr/lib/dovecot/modules/lib10_quota_plugin.so as well? - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVRKqOXz1H7kL/d9rAQJmeAf/SnxAj/OCXd2/Ud5LfwNJhwli1Mv+iGAM PmqMPhZgvw8W9NbDotz8CPuZvj7P/Mm1Bbk6sL+5WFRSAt+WJtdPzjyaxkFBhuTN YqD2knOUgjPH1UTrKR0jyA/OJCwvg72tmVe8uFQ0KEIGY9AXkQzf0bJSnW3+E/yh lD4NT6CaoGUwNNQusNyVjEdKaqZ60BCVBA8/HF66hLRaRb96trnJp9JihmbRt9DB Trg2gzm2bokAD9foi1ba8UuHSQbzdUxqAuDDU/WQrRDYdV4hHmrz3hX/c/M8NPfI o10FDhjgfnWn3+zHz5x9DIUhTPIbbRZ+NqgHM8TTLyVXhycuc7V+rw== =2SmU -----END PGP SIGNATURE----- From pch at myzel.net Wed Mar 25 12:46:41 2015 From: pch at myzel.net (Peter Chiochetti) Date: Wed, 25 Mar 2015 13:46:41 +0100 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: References: <3CB82A3E-E3FB-44A0-879E-C08A3E2DA0E7@iki.fi> Message-ID: <5512AE31.9080207@myzel.net> Am 25.03.2015 um 13:23 schrieb Nick Edwards: > > So there *is* a chance it will be commercialised > Hasn't it been commercial for a long time? -- Peter From w.rouesnel at gmail.com Wed Mar 25 12:58:19 2015 From: w.rouesnel at gmail.com (Will Rouesnel) Date: Wed, 25 Mar 2015 23:58:19 +1100 Subject: Upgrading a dovecot director configuration? Message-ID: <5512B0EB.7030901@gmail.com> Is Dovecot Director compatible across Dovecot versions? i.e. 2.0.x to 2.1.x or 2.2.x? I have a 6 host cluster in a director configuration which is getting somewhat aged, but it would be very beneficial if the director hosts could be upgraded seamlessly. Is such a thing possible, or will their be compatibility issues? Would a workaround be possible (i.e. migrate to a user system which enforces proxy destinations by another means, then do an upgrade?) From dh at powerhosting.dk Wed Mar 25 15:12:09 2015 From: dh at powerhosting.dk (=?UTF-8?B?RGVubmlzIEjDuGpnYWFyZA==?=) Date: Wed, 25 Mar 2015 16:12:09 +0100 Subject: Courier-dovecot migration issue: Forward and Junk flags Message-ID: <5512D049.2080405@powerhosting.dk> Hello.. Another courier -> dovecot migrater here.. I ham having a bit of trouble converting the courier maildir correctly, or actually i'm almost there. The only thing i experience is that forwarded messages and messages marked as junk (in thunderbird) are downloaded again (i am using thunderbird and caching) after running the courier-dovecot-migrate.pl script and connecting to the newly configured dovecot imap server.. So i think that the Forward and Junk flag is not being converted.. From brad at comstyle.com Wed Mar 25 15:58:47 2015 From: brad at comstyle.com (Brad Smith) Date: Wed, 25 Mar 2015 11:58:47 -0400 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <5512AE31.9080207@myzel.net> References: <3CB82A3E-E3FB-44A0-879E-C08A3E2DA0E7@iki.fi> <5512AE31.9080207@myzel.net> Message-ID: <5512DB37.8080301@comstyle.com> On 03/25/15 08:46, Peter Chiochetti wrote: > Am 25.03.2015 um 13:23 schrieb Nick Edwards: >> >> So there *is* a chance it will be commercialised >> > > Hasn't it been commercial for a long time? When was the last time you paid for Dovecot? The base product is open source and free for anyone to use. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From me at junc.eu Wed Mar 25 17:03:06 2015 From: me at junc.eu (Benny Pedersen) Date: Wed, 25 Mar 2015 18:03:06 +0100 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <5512DB37.8080301@comstyle.com> References: "\" " "<3CB82A3E-E3FB-44A0-879E-C08A3E2DA0E7@iki.fi>\" <5512AE31.9080207@myzel.net>" <5512DB37.8080301@comstyle.com> Message-ID: <36cc6c37438237ac256e35968940bc95@junc.eu> Brad Smith skrev den 2015-03-25 16:58: > On 03/25/15 08:46, Peter Chiochetti wrote: >> Am 25.03.2015 um 13:23 schrieb Nick Edwards: >>> So there *is* a chance it will be commercialised >> Hasn't it been commercial for a long time? > When was the last time you paid for Dovecot? The base product is > open source and free for anyone to use. only paid here by compileing time, still have dovecot v1 working, so open source it not complete free, as long it compiles fine i am happy From h.reindl at thelounge.net Wed Mar 25 17:05:42 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 25 Mar 2015 18:05:42 +0100 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <5512DB37.8080301@comstyle.com> References: <3CB82A3E-E3FB-44A0-879E-C08A3E2DA0E7@iki.fi> <5512AE31.9080207@myzel.net> <5512DB37.8080301@comstyle.com> Message-ID: <5512EAE6.6000405@thelounge.net> Am 25.03.2015 um 16:58 schrieb Brad Smith: > On 03/25/15 08:46, Peter Chiochetti wrote: >> Am 25.03.2015 um 13:23 schrieb Nick Edwards: >>> >>> So there *is* a chance it will be commercialised >> >> Hasn't it been commercial for a long time? > > When was the last time you paid for Dovecot? The base product is > open source and free for anyone to use and why people don't shut up until a single sign that this would ever change happened? is redhat a commercial company - yes it is - is the software available as open source and for free - yes it is a lot of responses in that thread are just whining for fun -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From h.reindl at thelounge.net Wed Mar 25 17:08:27 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 25 Mar 2015 18:08:27 +0100 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <36cc6c37438237ac256e35968940bc95@junc.eu> References: <5512DB37.8080301@comstyle.com> <36cc6c37438237ac256e35968940bc95@junc.eu> Message-ID: <5512EB8B.3010705@thelounge.net> Am 25.03.2015 um 18:03 schrieb Benny Pedersen: > Brad Smith skrev den 2015-03-25 16:58: >> On 03/25/15 08:46, Peter Chiochetti wrote: >>> Am 25.03.2015 um 13:23 schrieb Nick Edwards: >>>> So there *is* a chance it will be commercialised >>> Hasn't it been commercial for a long time? >> When was the last time you paid for Dovecot? The base product is >> open source and free for anyone to use. > > only paid here by compileing time, still have dovecot v1 working, so > open source it not complete free, as long it compiles fine i am happy that you compile at your own and that you still use dovecot 1.x is *your own* decision and si opensource *is complete free* with your argumentation making a shit would also not be completly free because you need to pinch ass bakes..... -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From me at junc.eu Wed Mar 25 17:28:18 2015 From: me at junc.eu (Benny Pedersen) Date: Wed, 25 Mar 2015 18:28:18 +0100 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <5512EB8B.3010705@thelounge.net> References: <5512DB37.8080301@comstyle.com> <36cc6c37438237ac256e35968940bc95@junc.eu> <5512EB8B.3010705@thelounge.net> Message-ID: <75d93937dffa88a055e02054a53bd1c2@junc.eu> Reindl Harald skrev den 2015-03-25 18:08: > with your argumentation making a shit would also not be completly free > because you need to pinch ass bakes..... and you write this on public walls ? From h.reindl at thelounge.net Wed Mar 25 17:31:17 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 25 Mar 2015 18:31:17 +0100 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <75d93937dffa88a055e02054a53bd1c2@junc.eu> References: <5512DB37.8080301@comstyle.com> <36cc6c37438237ac256e35968940bc95@junc.eu> <5512EB8B.3010705@thelounge.net> <75d93937dffa88a055e02054a53bd1c2@junc.eu> Message-ID: <5512F0E5.1020603@thelounge.net> Am 25.03.2015 um 18:28 schrieb Benny Pedersen: > Reindl Harald skrev den 2015-03-25 18:08: > >> with your argumentation making a shit would also not be completly free >> because you need to pinch ass bakes..... > > and you write this on public walls? DON'T QUOTE OUT OF CONTEXT BOY, YOU HAVE MISSED YOUR "only paid here by compileing time" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From brad at comstyle.com Wed Mar 25 19:20:39 2015 From: brad at comstyle.com (Brad Smith) Date: Wed, 25 Mar 2015 15:20:39 -0400 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <36cc6c37438237ac256e35968940bc95@junc.eu> References: "\" " "<3CB82A3E-E3FB-44A0-879E-C08A3E2DA0E7@iki.fi>\" <5512AE31.9080207@myzel.net>" <5512DB37.8080301@comstyle.com> <36cc6c37438237ac256e35968940bc95@junc.eu> Message-ID: <55130A87.4040408@comstyle.com> On 03/25/15 13:03, Benny Pedersen wrote: > Brad Smith skrev den 2015-03-25 16:58: >> On 03/25/15 08:46, Peter Chiochetti wrote: >>> Am 25.03.2015 um 13:23 schrieb Nick Edwards: >>>> So there *is* a chance it will be commercialised >>> Hasn't it been commercial for a long time? >> When was the last time you paid for Dovecot? The base product is >> open source and free for anyone to use. > > only paid here by compileing time, still have dovecot v1 working, so > open source it not complete free, as long it compiles fine i am happy Not making any sense. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From me at junc.eu Wed Mar 25 19:34:40 2015 From: me at junc.eu (Benny Pedersen) Date: Wed, 25 Mar 2015 20:34:40 +0100 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <55130A87.4040408@comstyle.com> References: "\"\\\" \" \"<3CB82A3E-E3FB-44A0-879E-C08A3E2DA0E7@iki.fi>\\\" " "\"<5512AE31.9080207@myzel.net>\\\" <5512DB37.8080301@comstyle.com>" <36cc6c37438237ac256e35968940bc95@junc.eu>" <55130A87.4040408@comstyle.com> Message-ID: <429e3ef2139f4aab01df0fdb8e2ca9bf@junc.eu> Brad Smith skrev den 2015-03-25 20:20: >> only paid here by compileing time, still have dovecot v1 working, so >> open source it not complete free, as long it compiles fine i am happy > > Not making any sense. punktum ? if i really need to install precompiled problems i could aswell install windows 10, and be happy, its just not opensource when the sources is not shown, where is the source codes for android ?, its based on linux with is opensource, but where is the source for android ? got my point ? i just say that opensource brands is not really opensource if one install it precompiled, punktum as you write it From h.reindl at thelounge.net Wed Mar 25 19:46:20 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 25 Mar 2015 20:46:20 +0100 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <429e3ef2139f4aab01df0fdb8e2ca9bf@junc.eu> References: <36cc6c37438237ac256e35968940bc95@junc.eu> <429e3ef2139f4aab01df0fdb8e2ca9bf@junc.eu> Message-ID: <5513108C.6020506@thelounge.net> Am 25.03.2015 um 20:34 schrieb Benny Pedersen: > Brad Smith skrev den 2015-03-25 20:20: > >>> only paid here by compileing time, still have dovecot v1 working, so >>> open source it not complete free, as long it compiles fine i am happy >> >> Not making any sense. > > punktum ? "only paid here by compileing time" is nonsense > if i really need to install precompiled problems i could aswell install > windows 10, and be happy, its just not opensource when the sources is > not shown foolish trolling - the source needs to be available or does your self compiled binary show you his source at startup? you can download the source from Redhat, Fedora, OpenSUSE, Debian and so it is shown - you just need to look at it - well, but you don't understand it anyways, no difference to your way of download, unpack and compile a source you don#t understand > where is the source codes for android ? available or where do alternate ROM providers take it > its based on linux with is opensource, but where is > the source for android? https://source.android.com/source/downloading.html > got my point ? as in 98% of your posts you have no point > i just say that opensource brands is not really opensource if one > install it precompiled, punktum as you write it bullshit - there is no difference between install the binary a distribution build from the source tarball than download the tarball and call make scripts until you want change some default flags -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From brad at comstyle.com Wed Mar 25 19:54:35 2015 From: brad at comstyle.com (Brad Smith) Date: Wed, 25 Mar 2015 15:54:35 -0400 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <429e3ef2139f4aab01df0fdb8e2ca9bf@junc.eu> References: "\"\\\" \" \"<3CB82A3E-E3FB-44A0-879E-C08A3E2DA0E7@iki.fi>\\\" " "\"<5512AE31.9080207@myzel.net>\\\" <5512DB37.8080301@comstyle.com>" <36cc6c37438237ac256e35968940bc95@junc.eu>" <55130A87.4040408@comstyle.com> <429e3ef2139f4aab01df0fdb8e2ca9bf@junc.eu> Message-ID: <5513127B.4060202@comstyle.com> On 03/25/15 15:34, Benny Pedersen wrote: > Brad Smith skrev den 2015-03-25 20:20: > >>> only paid here by compileing time, still have dovecot v1 working, so >>> open source it not complete free, as long it compiles fine i am happy >> >> Not making any sense. > > punktum ? > > if i really need to install precompiled problems i could aswell install > windows 10, and be happy, its just not opensource when the sources is > not shown, where is the source codes for android ?, its based on linux > with is opensource, but where is the source for android ? > > got my point ? > > i just say that opensource brands is not really opensource if one > install it precompiled, punktum as you write it *shakes head* So many flaws with that logic. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From dmiller at amfes.com Wed Mar 25 20:46:11 2015 From: dmiller at amfes.com (Daniel Miller) Date: Wed, 25 Mar 2015 13:46:11 -0700 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <8250CA2C-936F-468D-B5C6-88954DC6FB2F@iki.fi> References: <8250CA2C-936F-468D-B5C6-88954DC6FB2F@iki.fi> Message-ID: On 3/19/2015 3:26 AM, Timo Sirainen wrote: > Hi all, > > Today I can finally announce that Dovecot Oy company has merged with Open-Xchange AG. This helps us to get more Dovecot developers, support people and so on. Most importantly, eventually it should allow me to get back to doing what I like the most: Designing new and interesting stuff for Dovecot and perfecting the old stuff :) OX is a great match to Dovecot going forward. They also really like open source and share our plans for the future. Nothing big will change as a result of this merger: Dovecot will stay Dovecot with its own name and release schedules. We're not going to force OX and Dovecot to be the same product, other than having a somewhat deeper integration between them. > My initial impression is...sounds great! Then, after further thought, and watching the flame war, I've changed my mind to...sounds great! I'm operating under the assumption that you are continuing to be in charge of Dovecot and will choose what and how to implement changes and fixes. I'm further operating under the assumption that you may choose to have certain features, appropriate for larger installations, that you will want to receive compensation for from your users. And I'm assuming that by having OX behind you, those initial assumptions remain - Dovecot remains your baby, you will grow it as you see fit - but now you've got some financial backing that allows you more freedom to continue to develop Dovecot for general-purpose use while reasonably having certain features developed to support the paid model. If I'm mistaken then please correct me - but I'm seeing nothing but upside. Certainly for you, and if you were to abandon open source Dovecot today (which I've seen absolutely no indication) you've already provided a tool that has a significant user base and you deserve to be rewarded for it. But based on your previous actions and your original post, and I have no reason not to take you at your word, this sounds like a win/win for Dovecot developers and users. Congratulations! -- Daniel From yawowb+dovecot at nuclei.ca Wed Mar 25 21:27:09 2015 From: yawowb+dovecot at nuclei.ca (rooster) Date: Wed, 25 Mar 2015 14:27:09 -0700 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> <55112383.8080909@smail.inf.fh-brs.de> <55112F56.5010100@smail.inf.fh-brs.de> <6BD41A7B-3DCD-4F71-B7B9-79EC9D7D9F96@nuclei.ca> <1E597353-D4A3-43D7-A510-6A9552FC7890@nuclei.ca> Message-ID: > On 2015-03-25, at 5:29 AM, Steffen Kaiser wrote: > > it's a pity, I hoped there is a symlink problem. > > Could you please try the three commands with this file: > > /usr/lib/dovecot/modules/lib10_quota_plugin.so > > as well? > > - -- Steffen Kaiser I too wish it were something simple and it very well maybe and it?s just the errors are not pointing us in the right direction. :) Here is the output as requested. user at host: # file /usr/lib/dovecot/modules/lib10_quota_plugin.so /usr/lib/dovecot/modules/lib10_quota_plugin.so: ELF 32-bit MSB shared object, PowerPC or cisco 4500, version 1 (SYSV), dynamically linked, BuildID[sha1]=75eee844003c3da4dfeeafc470095db07bf121ea, stripped user at host: # ldd /usr/lib/dovecot/modules/lib10_quota_plugin.so linux-vdso32.so.1 => (0x00100000) libc.so.6 => /lib/powerpc-linux-gnu/libc.so.6 (0x6fe2e000) /lib/ld.so.1 (0x2056c000) user at host: # ls -la /usr/lib/dovecot/modules/lib10_quota_plugin.so -rw-r--r-- 1 root root 75552 May 14 2014 /usr/lib/dovecot/modules/lib10_quota_plugin.so -- From andreas.kasenides at cs.ucy.ac.cy Wed Mar 25 21:46:26 2015 From: andreas.kasenides at cs.ucy.ac.cy (Andreas Kasenides) Date: Wed, 25 Mar 2015 23:46:26 +0200 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: References: <8250CA2C-936F-468D-B5C6-88954DC6FB2F@iki.fi> Message-ID: <55132CB2.7030203@cs.ucy.ac.cy> I hate to have started this, especially the "sister" thread that has dissented into a flame war of what is OSS. Let me say that I believe there is nothing wrong trying to make money on ones efforts. Actually it is a must. How can anyone continue to put efforts into a project when there is no reward? Especially when most of the effort is by a single individual. Secondly there comes a point in time when any project needs help to advance. Any one individual will be unable to manage all the things that need to be done. It will either become a team effort of individuals employed elsewhere or somehow enter the commercial sector in some form. Both of these models have many examples out there and in the mean time maintaining their OSS root and community. What I was mostly worried about was a sudden and rapid commercialization of the project in such a way that it completely disappears from the OSS community. I will give you an extreme example that we had the pleasure to be involved as payed customers and debugging contributors: KnowledgeTree DMS. If you do not know the story you will simply not find it. After years of the community contributing to the project a sudden shift to complete commercialization destroyed the project entirely: ie sourceforge project closed, source code disappeared, mailing lists vanished even the domain name name closed down. If it wasn't for third party storage/downloading sites the project source code would have been practically non-existent. I consider such behavior firstly immoral since a project's success is not only its design but largely also its debugging, mostly done by thousands of unknown helpers writing their experiences and problems in mailing lists. I hope Timo manages well, keeps the community going but also makes a living (or a ton of money ) out of Dovecot. He deserves it. It is not impossible, others have done so successfully. On 25/03/15 22:46, Daniel Miller wrote: > On 3/19/2015 3:26 AM, Timo Sirainen wrote: >> Hi all, >> >> Today I can finally announce that Dovecot Oy company has merged with >> Open-Xchange AG. This helps us to get more Dovecot developers, >> support people and so on. Most importantly, eventually it should >> allow me to get back to doing what I like the most: Designing new and >> interesting stuff for Dovecot and perfecting the old stuff :) OX is a >> great match to Dovecot going forward. They also really like open >> source and share our plans for the future. Nothing big will change as >> a result of this merger: Dovecot will stay Dovecot with its own name >> and release schedules. We're not going to force OX and Dovecot to be >> the same product, other than having a somewhat deeper integration >> between them. >> > > My initial impression is...sounds great! Then, after further thought, > and watching the flame war, I've changed my mind to...sounds great! > > I'm operating under the assumption that you are continuing to be in > charge of Dovecot and will choose what and how to implement changes > and fixes. I'm further operating under the assumption that you may > choose to have certain features, appropriate for larger installations, > that you will want to receive compensation for from your users. And > I'm assuming that by having OX behind you, those initial assumptions > remain - Dovecot remains your baby, you will grow it as you see fit - > but now you've got some financial backing that allows you more freedom > to continue to develop Dovecot for general-purpose use while > reasonably having certain features developed to support the paid model. > > If I'm mistaken then please correct me - but I'm seeing nothing but > upside. Certainly for you, and if you were to abandon open source > Dovecot today (which I've seen absolutely no indication) you've > already provided a tool that has a significant user base and you > deserve to be rewarded for it. But based on your previous actions and > your original post, and I have no reason not to take you at your word, > this sounds like a win/win for Dovecot developers and users. > Congratulations! > From asai at globalchangemusic.org Wed Mar 25 22:39:52 2015 From: asai at globalchangemusic.org (Asai) Date: Wed, 25 Mar 2015 15:39:52 -0700 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <55132CB2.7030203@cs.ucy.ac.cy> References: <8250CA2C-936F-468D-B5C6-88954DC6FB2F@iki.fi> <55132CB2.7030203@cs.ucy.ac.cy> Message-ID: <55133938.3000207@globalchangemusic.org> Agreed. I think this is a positive move for Dovecot and Timo, Mikko, et al. I think only good will come of this for open source communications. On 3/25/15 2:46 PM, Andreas Kasenides wrote: > I hate to have started this, especially the "sister" thread that has > dissented into a flame war of what is OSS. > > Let me say that I believe there is nothing wrong trying to make money > on ones efforts. Actually it is a must. How can anyone continue to put > efforts into a project when there is no reward? Especially when most > of the effort is by a single individual. Secondly there comes a point > in time when any project needs help to advance. Any one individual > will be unable to manage all the things that need to be done. It will > either become a team effort of individuals employed elsewhere or > somehow enter the commercial sector in some form. Both of these models > have many examples out there and in the mean time maintaining their > OSS root and community. > > What I was mostly worried about was a sudden and rapid > commercialization of the project in such a way that it completely > disappears from the OSS community. I will give you an extreme example > that we had the pleasure to be involved as payed customers and > debugging contributors: KnowledgeTree DMS. If you do not know the > story you will simply not find it. After years of the community > contributing to the project a sudden shift to complete > commercialization destroyed the project entirely: ie sourceforge > project closed, source code disappeared, mailing lists vanished even > the domain name name closed down. If it wasn't for third party > storage/downloading sites the project source code would have been > practically non-existent. I consider such behavior firstly immoral > since a project's success is not only its design but largely also its > debugging, mostly done by thousands of unknown helpers writing their > experiences and problems in mailing lists. > > I hope Timo manages well, keeps the community going but also makes a > living (or a ton of money ) out of Dovecot. He deserves it. It is not > impossible, others have done so successfully. > > > On 25/03/15 22:46, Daniel Miller wrote: >> On 3/19/2015 3:26 AM, Timo Sirainen wrote: >>> Hi all, >>> >>> Today I can finally announce that Dovecot Oy company has merged with >>> Open-Xchange AG. This helps us to get more Dovecot developers, >>> support people and so on. Most importantly, eventually it should >>> allow me to get back to doing what I like the most: Designing new >>> and interesting stuff for Dovecot and perfecting the old stuff :) OX >>> is a great match to Dovecot going forward. They also really like >>> open source and share our plans for the future. Nothing big will >>> change as a result of this merger: Dovecot will stay Dovecot with >>> its own name and release schedules. We're not going to force OX and >>> Dovecot to be the same product, other than having a somewhat deeper >>> integration between them. >>> >> >> My initial impression is...sounds great! Then, after further >> thought, and watching the flame war, I've changed my mind to...sounds >> great! >> >> I'm operating under the assumption that you are continuing to be in >> charge of Dovecot and will choose what and how to implement changes >> and fixes. I'm further operating under the assumption that you may >> choose to have certain features, appropriate for larger >> installations, that you will want to receive compensation for from >> your users. And I'm assuming that by having OX behind you, those >> initial assumptions remain - Dovecot remains your baby, you will grow >> it as you see fit - but now you've got some financial backing that >> allows you more freedom to continue to develop Dovecot for >> general-purpose use while reasonably having certain features >> developed to support the paid model. >> >> If I'm mistaken then please correct me - but I'm seeing nothing but >> upside. Certainly for you, and if you were to abandon open source >> Dovecot today (which I've seen absolutely no indication) you've >> already provided a tool that has a significant user base and you >> deserve to be rewarded for it. But based on your previous actions >> and your original post, and I have no reason not to take you at your >> word, this sounds like a win/win for Dovecot developers and users. >> Congratulations! >> -- --asai From skdovecot at smail.inf.fh-brs.de Thu Mar 26 07:12:12 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 26 Mar 2015 08:12:12 +0100 (CET) Subject: Courier-dovecot migration issue: Forward and Junk flags In-Reply-To: <5512D049.2080405@powerhosting.dk> References: <5512D049.2080405@powerhosting.dk> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 25 Mar 2015, Dennis H?jgaard wrote: > Hello.. Another courier -> dovecot migrater here.. I ham having a bit of > trouble converting the courier maildir correctly, or actually i'm almost > there. The only thing i experience is that forwarded messages and messages > marked as junk (in thunderbird) are downloaded again (i am using thunderbird > and caching) after running the courier-dovecot-migrate.pl script and > connecting to the newly configured dovecot imap server.. So i think that the > Forward and Junk flag is not being converted.. Could you check if the converted Dovecot Maildir contains a file named dovecot-keywords with contents like this: 0 Junk 1 nonjunk 2 $Forwarded 3 $label1 4 Old 5 $NotJunk 6 $MDNSent 7 $Junk Could you also check if your Courier Maildir contains a file named courierimapkeywords/:list and if this file contains the keywords, too? - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVROxTHz1H7kL/d9rAQJJAAf/UTDcy5cBaA83dd0xiqQ+WHnDK0gkIqKQ privBnUqzxwDvfH6aBdEsl64VFU2hN1ItQR42p2/4YT6JV+Kb+ZuJkSUQ/jhtpzS Fu8+r4RGxBpPrP7rr3ObXv89YeKDt5+znDL0UH+lz1AQefAP8IGaydMHHZd7CIn2 Dy+hM0dmBg54YAxzRlnjSEcaDEH94Le/D0twrYS1Rqu/u1uEdC7gyQ9pwbpAUAQl J1hxObo18ElyxtZdGYJ53n1Cd9oV+mIzp0P72thcsB2Xu7tgNOBMpvp0O8pi7q+5 FKBzhJgv1a7V6s/+n88LilNua+CjHHiFaON29oYu8dybvNdUpUCYiQ== =6hFP -----END PGP SIGNATURE----- From dh at powerhosting.dk Thu Mar 26 07:45:32 2015 From: dh at powerhosting.dk (=?UTF-8?B?RGVubmlzIEjDuGpnYWFyZA==?=) Date: Thu, 26 Mar 2015 08:45:32 +0100 Subject: Courier-dovecot migration issue: Forward and Junk flags In-Reply-To: References: <5512D049.2080405@powerhosting.dk> Message-ID: <5513B91C.5030602@powerhosting.dk> Hey Steffen Thank you for your feedback. I have the following in my Maildir: cat courierimapkeywords/\:list JUNK $Forwarded $label4 $label1 cat dovecot-keywords 0 JUNK 1 $Forwarded 2 $label4 3 $label1 On 2015-03-26 08:12, Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Wed, 25 Mar 2015, Dennis H?jgaard wrote: > >> Hello.. Another courier -> dovecot migrater here.. I ham having a bit >> of trouble converting the courier maildir correctly, or actually i'm >> almost there. The only thing i experience is that forwarded messages >> and messages marked as junk (in thunderbird) are downloaded again (i >> am using thunderbird and caching) after running the >> courier-dovecot-migrate.pl script and > >> connecting to the newly configured dovecot imap server.. So i think >> that the Forward and Junk flag is not being converted.. > > Could you check if the converted Dovecot Maildir contains a file named > dovecot-keywords with contents like this: > > 0 Junk > 1 nonjunk > 2 $Forwarded > 3 $label1 > 4 Old > 5 $NotJunk > 6 $MDNSent > 7 $Junk > > Could you also check if your Courier Maildir contains a file named > courierimapkeywords/:list and if this file contains the keywords, too? > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVROxTHz1H7kL/d9rAQJJAAf/UTDcy5cBaA83dd0xiqQ+WHnDK0gkIqKQ > privBnUqzxwDvfH6aBdEsl64VFU2hN1ItQR42p2/4YT6JV+Kb+ZuJkSUQ/jhtpzS > Fu8+r4RGxBpPrP7rr3ObXv89YeKDt5+znDL0UH+lz1AQefAP8IGaydMHHZd7CIn2 > Dy+hM0dmBg54YAxzRlnjSEcaDEH94Le/D0twrYS1Rqu/u1uEdC7gyQ9pwbpAUAQl > J1hxObo18ElyxtZdGYJ53n1Cd9oV+mIzp0P72thcsB2Xu7tgNOBMpvp0O8pi7q+5 > FKBzhJgv1a7V6s/+n88LilNua+CjHHiFaON29oYu8dybvNdUpUCYiQ== > =6hFP > -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Thu Mar 26 08:21:31 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 26 Mar 2015 09:21:31 +0100 (CET) Subject: Courier-dovecot migration issue: Forward and Junk flags In-Reply-To: <5513B91C.5030602@powerhosting.dk> References: <5512D049.2080405@powerhosting.dk> <5513B91C.5030602@powerhosting.dk> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 26 Mar 2015, Dennis H?jgaard wrote: > Thank you for your feedback. I have the following in my Maildir: > > cat courierimapkeywords/\:list > JUNK > $Forwarded > $label4 > $label1 > > cat dovecot-keywords > 0 JUNK > 1 $Forwarded > 2 $label4 > 3 $label1 Could you check if a message that is marked as JUNK in Courier has an "a" (lower letter A) appended to the filename in Dovecot's Maildir? and a forwarded one should have a "b". > > > On 2015-03-26 08:12, Steffen Kaiser wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Wed, 25 Mar 2015, Dennis H?jgaard wrote: >> >>> Hello.. Another courier -> dovecot migrater here.. I ham having a bit of >>> trouble converting the courier maildir correctly, or actually i'm almost >>> there. The only thing i experience is that forwarded messages and messages >>> marked as junk (in thunderbird) are downloaded again (i am using >>> thunderbird and caching) after running the courier-dovecot-migrate.pl >>> script and >> >>> connecting to the newly configured dovecot imap server.. So i think that >>> the Forward and Junk flag is not being converted.. >> >> Could you check if the converted Dovecot Maildir contains a file named >> dovecot-keywords with contents like this: >> >> 0 Junk >> 1 nonjunk >> 2 $Forwarded >> 3 $label1 >> 4 Old >> 5 $NotJunk >> 6 $MDNSent >> 7 $Junk >> >> Could you also check if your Courier Maildir contains a file named >> courierimapkeywords/:list and if this file contains the keywords, too? >> >> - -- Steffen Kaiser >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1 >> >> iQEVAwUBVROxTHz1H7kL/d9rAQJJAAf/UTDcy5cBaA83dd0xiqQ+WHnDK0gkIqKQ >> privBnUqzxwDvfH6aBdEsl64VFU2hN1ItQR42p2/4YT6JV+Kb+ZuJkSUQ/jhtpzS >> Fu8+r4RGxBpPrP7rr3ObXv89YeKDt5+znDL0UH+lz1AQefAP8IGaydMHHZd7CIn2 >> Dy+hM0dmBg54YAxzRlnjSEcaDEH94Le/D0twrYS1Rqu/u1uEdC7gyQ9pwbpAUAQl >> J1hxObo18ElyxtZdGYJ53n1Cd9oV+mIzp0P72thcsB2Xu7tgNOBMpvp0O8pi7q+5 >> FKBzhJgv1a7V6s/+n88LilNua+CjHHiFaON29oYu8dybvNdUpUCYiQ== >> =6hFP >> -----END PGP SIGNATURE----- > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVRPBi3z1H7kL/d9rAQIGIggAw3QSGZzvVGHfE7sMsYwmvcG7KxKfVUvc nMnEqQzfugjEEp6orul8JAoJxV11+8pHAZ3ftJDQZitDS0Pa+hzrwMLoUQEGxlS/ XgmMhUc+pmRXGCjZAd1iV/sTTRMY/uxjFRGLDsxwVPMZ8lZeYNMWd/nsEoxEAXGQ Q+Og9dhRdUcxLxMBVrJBhCCeuXZW2TXdmOF7dfZE/nv32ETTFveVkA1ycfiij/4J UzkCNNxLeGPPu+PQFdkf/c3xgcr6MDprWYMv1dILYC030wiEVLYI2ha2tyKSH+OH adB5+5F9qvDsFFGII0raGqB32pFqPjbhsXVFZii9Dwe+ZBBinDW/RQ== =7j1W -----END PGP SIGNATURE----- From giorgio.paolucci at unipd.it Thu Mar 26 09:31:00 2015 From: giorgio.paolucci at unipd.it (giorgio paolucci) Date: Thu, 26 Mar 2015 10:31:00 +0100 Subject: Lazy_expunge issue... Message-ID: <5513D1D4.806@unipd.it> Hello everybody, I have a question about lazy_expunge plugin. I set up Dovecot with lazy_expunge plugin on a Debian Wheezy machine; I setup also a global acl to keep .expunged namespace readonly for all users. Imap client is Horde Webmail Edition 5.2.3 When I try to delete folders containing subfolders, lazy_expunge copies into expunged private namespace only the upper level folder. The subfolder and all messages inside it are lost. Quota is also corrupted (subfolder messages are still accounted in the Message counter and disk quota) and a recalc is needed to fix. Problem can be reproduced this way: 1)Create with horde a folder "extern" 2)Create a subfolder "extern/subfolder" 3) copy some messages into extern and some into subfolder (in my test 5 messages each) 4) verify quota usage: root at gold:# doveadm quota get -u testimap2.csia at unipd.it Quota name Type Value Limit % User quota STORAGE 4 1126400 0 User quota MESSAGE 10 - 0 3)Delete extern with option to delete all subfolders. 4)verify that all folders and messages have been deleted 5)verify that only extern folder has been copied into .expunged namespace and verify that quota still reports the 5 messages that were in subfolder: root at gold:# doveadm quota get -u testimap2.csia at unipd.it Quota name Type Value Limit % User quota STORAGE 2 1126400 0 User quota MESSAGE 5 - 0 6) recalc quota and check again that quota has benne corrected: # doveadm quota recalc -u testimap2.csia at unipd.it root at gold:# doveadm quota get -u testimap2.csia at unipd.it Quota name Type Value Limit % User quota STORAGE 0 1126400 0 User quota MESSAGE 0 - 0 Is there anything I am missing? Thanks in advance. Best regards. Giorgio Paolucci ================================================ dovecot version 2.2.13 OS :Linux bronze 3.2.0-4-amd64 #1 SMP Debian 3.2.65-1+deb7u1 x86_64 GNU/Linux ================================================= root at bronze:/etc/dovecot/conf.d# more ../dovecot-acl _PostaCancellataPerErrore* owner lrik ================================================= root at bronze:/etc/dovecot/conf.d# dovecot -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8 auth_default_realm = unipd.it disable_plaintext_auth = no log_path = /var/log/dovecot/dovecot.log mail_debug = yes mail_fsync = always mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota listescape lazy_expunge acl mail_log notify mdbox_rotate_interval = 1 days mdbox_rotate_size = 10 M mmap_disable = yes namespace { hidden = no inbox = no list = yes location = maildir:/SYSTEM/USERS/mailbox/%d/%1n/%u/expunged mailbox .LEGGIMI { auto = subscribe } mailbox INBOX.Trash { auto = subscribe special_use = \Trash } prefix = _PostaCancellataPerErrore/ separator = / subscriptions = yes type = private } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } passdb { args = /etc/dovecot/conf.d/dovecot-sqlSSO.conf.ext driver = sql } plugin { acl = vfile:/etc/dovecot/dovecot-acl lazy_expunge = _PostaCancellataPerErrore/ lazy_expunge_only_last_instance = yes mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size pop3_migration_mailbox = INBOX quota = dict:User quota::file:/SYSTEM/USERS/homes/%d/%1u/%u/%u.quota quota_rule = *:storage=1100M quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } pop3c_host = 147.162.10.68 protocols = imap pop3 lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service imap-login { inet_listener imaps { ssl = no } process_min_avail = 20 service_count = 1 } service imap { executable = imap postlogin } service lmtp { executable = lmtp inet_listener lmtp { port = 2223 } process_min_avail = 15 } service pop3-login { inet_listener pop3s { ssl = no } } service postlogin { executable = script-login -d rawlog user = vmail } service quota-warning { executable = script /SYSTEM/DOVECOT/scripts/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = no userdb { driver = prefetch } userdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } userdb { driver = prefetch } userdb { args = /etc/dovecot/conf.d/dovecot-sqlSSO.conf.ext driver = sql } protocol lmtp { info_log_path = /tmp/dovecot-lmtp-info.log log_path = /tmp/dovecot-lmtp.log mail_plugins = quota listescape lazy_expunge acl mail_log notify } protocol imap { imap_max_line_length = 128 k mail_plugins = quota listescape lazy_expunge acl mail_log notify imap_quota acl imap_acl } protocol doveadm { mail_plugins = quota listescape lazy_expunge acl mail_log notify pop3_migration } protocol pop3 { pop3_no_flag_updates = yes pop3_uidl_format = %10v.%09u } =================================== root at gold:# doveadm user -u testimap2.csia at unipd.it userdb: testimap2.csia at unipd.it uid : 1003 gid : 1003 home : /SYSTEM/USERS/homes/unipd.it/t/testimap2.csia at unipd.it mail : sdbox:/NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox:INDEX=/INDEX1/unipd.it/t/testimap2.csia at unipd.it:ALT=/ALTSTORAGE1/unipd.it/t/testimap2.csia at unipd.it ------------------------------- Here follows IMAP transaction in and out from rawlog IN: 2 ENABLE QRESYNC 3 NAMESPACE 4 LIST () "" (extern/*) 5 DELETE extern 6 UNSUBSCRIBE extern 7 DELETE extern/subfolder 8 UNSUBSCRIBE extern/subfolder 9 LOGOUT OUT: root at gold:/SYSTEM/USERS/homes/unipd.it/t/testimap2.csia at unipd.it/dovecot.rawlog# more 20150326-095943-1773.out 1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=OR DEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARC H ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE QUOTA ACL RIGHTS=texk] Logged in * ENABLED QRESYNC 2 OK Enabled. * NAMESPACE (("" "/")("_PostaCancellataPerErrore/" "/")) NIL NIL 3 OK Namespace completed. * LIST () "/" extern/subfolder 4 OK List completed. 5 OK Delete completed. 6 OK Unsubscribe completed. 7 OK Delete completed. 8 OK Unsubscribe completed. * BYE Logging out 9 OK Logout completed. ================================== Here follows debug log from dovecot. Mar 26 09:59:43 imap-login: Info: Login: user=, method=PLAIN, rip=147.162.10.141, lip=147.162.10.84, mpid=1771, session= Mar 26 09:59:43 script-login: Debug: Added userdb setting: mail_location=sdbox:/NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox:INDEX=/INDEX1/unipd.it/t/testimap2.csia at unipd.it:ALT=/ALTSTORAGE1/unipd.it/t/testimap2.csia at unipd.it Mar 26 09:59:43 imap: Debug: Loading modules from directory: /usr/lib/dovecot/modules Mar 26 09:59:43 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib01_acl_plugin.so Mar 26 09:59:43 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib02_imap_acl_plugin.so Mar 26 09:59:43 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib02_lazy_expunge_plugin.so Mar 26 09:59:43 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Mar 26 09:59:43 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so Mar 26 09:59:43 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib15_notify_plugin.so Mar 26 09:59:43 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib20_listescape_plugin.so Mar 26 09:59:43 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib20_mail_log_plugin.so Mar 26 09:59:43 imap: Debug: Added userdb setting: mail_location=sdbox:/NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox:INDEX=/INDEX1/unipd.it/t/testimap2.csia at unipd.it:ALT=/ALTSTORAGE1/unipd.it/t/testimap2.csia at unipd.it Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: Effective uid=1003, gid=1003, home=/SYSTEM/USERS/homes/unipd.it/t/testimap2.csia at unipd.it Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: acl: No acl_shared_dict setting - shared mailbox listing is disabled Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: Quota root: name=User quota backend=dict args=:file:/SYSTEM/USERS/homes/unipd.it/t/testimap2.csia at unipd.it/testimap2.csia at unipd.it.quota Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: Quota rule: root=User quota mailbox=* bytes=1153433600 messages=0 Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: Quota rule: root=User quota mailbox=Trash bytes=+104857600 messages=0 Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: Quota warning: bytes=1095761920 (95%) messages=0 reverse=no command=quota-warning 95 testimap2.csia at unipd.it Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: Quota warning: bytes=922746880 (80%) messages=0 reverse=no command=quota-warning 80 testimap2.csia at unipd.it Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: Quota grace: root=User quota bytes=115343360 (10%) Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: dict quota: user=testimap2.csia at unipd.it, uri=file:/SYSTEM/USERS/homes/unipd.it/t/testimap2.csia at unipd.it/testimap2.csia at unipd.it.quota, noenforcing=0 Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=sdbox:/NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox:INDEX=/INDEX1/unipd.it/t/testimap2.csia at unipd.it:ALT=/ALTSTORAGE1/unipd.it/t/testimap2.csia at unipd.it Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: fs: root=/NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox, index=/INDEX1/unipd.it/t/testimap2.csia at unipd.it, indexpvt=, control=, inbox=, alt=/ALTSTORAGE1/unipd.it/t/testimap2.csia at unipd.it Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: acl: initializing backend with data: vfile:/etc/dovecot/dovecot-acl Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: acl: acl username = testimap2.csia at unipd.it Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: acl: owner = 1 Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: acl vfile: Global ACL file: /etc/dovecot/dovecot-acl Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: Namespace : type=private, prefix=_PostaCancellataPerErrore/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=yes location=maildir:/SYSTEM/USERS/mailbox/unipd.it/t/testimap2.csia at unipd.it/expunged Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: maildir++: root=/SYSTEM/USERS/mailbox/unipd.it/t/testimap2.csia at unipd.it/expunged, index=, indexpvt=, control=, inbox=, alt= Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: acl: initializing backend with data: vfile:/etc/dovecot/dovecot-acl Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: acl: acl username = testimap2.csia at unipd.it Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: acl: owner = 1 Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: acl vfile: Global ACL file: /etc/dovecot/dovecot-acl Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: acl vfile: file /NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox/mailboxes/dovecot-acl not found Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: acl vfile: file /NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox/mailboxes/extern/subfolder/dbox-Mails/dovecot-acl not found Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: acl vfile: file /NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox/mailboxes/extern/dbox-Mails/dovecot-acl not found Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: acl vfile: file /SYSTEM/USERS/mailbox/unipd.it/t/testimap2.csia at unipd.it/expunged/.extern/dovecot-acl not found Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: acl vfile: file /SYSTEM/USERS/mailbox/unipd.it/t/testimap2.csia at unipd.it/expunged/dovecot-acl not found Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: Namespace _PostaCancellataPerErrore/: /SYSTEM/USERS/mailbox/unipd.it/t/testimap2.csia at unipd.it/expunged/.extern doesn't exist yet, using default permissions Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: Namespace _PostaCancellataPerErrore/: Using permissions from /SYSTEM/USERS/mailbox/unipd.it/t/testimap2.csia at unipd.it/expunged: mode=0700 gid=default Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: acl vfile: file /SYSTEM/USERS/mailbox/unipd.it/t/testimap2.csia at unipd.it/expunged/dovecot-acl not found Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: acl vfile: file /SYSTEM/USERS/mailbox/unipd.it/t/testimap2.csia at unipd.it/expunged/.extern/dovecot-acl not found Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Info: copy from extern: box=_PostaCancellataPerErrore/extern, uid=1, msgid=, size=523 Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Info: copy from extern: box=_PostaCancellataPerErrore/extern, uid=2, msgid=, size=523 Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Info: copy from extern: box=_PostaCancellataPerErrore/extern, uid=3, msgid=, size=523 Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Info: copy from extern: box=_PostaCancellataPerErrore/extern, uid=4, msgid=, size=523 Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Info: copy from extern: box=_PostaCancellataPerErrore/extern, uid=5, msgid=, size=524 Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Info: expunge: box=extern, uid=1, msgid=, size=523 Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Info: expunge: box=extern, uid=2, msgid=, size=523 Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Info: expunge: box=extern, uid=3, msgid=, size=523 Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Info: expunge: box=extern, uid=4, msgid=, size=523 Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Info: expunge: box=extern, uid=5, msgid=, size=524 Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Debug: Namespace : Using permissions from /NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox: mode=0755 gid=default Mar 26 09:59:43 imap(testimap2.csia at unipd.it): Info: Mailbox deleted: extern Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Info: Mailbox deleted: extern/subfolder Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Info: Disconnected: Logged out in=162 out=704 Mar 26 09:59:44 imap-login: Info: Login: user=, method=PLAIN, rip=147.162.10.141, lip=147.162.10.84, mpid=1778, session= Mar 26 09:59:44 script-login: Debug: Added userdb setting: mail_location=sdbox:/NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox:INDEX=/INDEX1/unipd.it/t/testimap2.csia at unipd.it:ALT=/ALTSTORAGE1/unipd.it/t/testimap2.csia at unipd.it Mar 26 09:59:44 imap: Debug: Loading modules from directory: /usr/lib/dovecot/modules Mar 26 09:59:44 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib01_acl_plugin.so Mar 26 09:59:44 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib02_imap_acl_plugin.so Mar 26 09:59:44 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib02_lazy_expunge_plugin.so Mar 26 09:59:44 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Mar 26 09:59:44 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so Mar 26 09:59:44 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib15_notify_plugin.so Mar 26 09:59:44 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib20_listescape_plugin.so Mar 26 09:59:44 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib20_mail_log_plugin.so Mar 26 09:59:44 imap: Debug: Added userdb setting: mail_location=sdbox:/NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox:INDEX=/INDEX1/unipd.it/t/testimap2.csia at unipd.it:ALT=/ALTSTORAGE1/unipd.it/t/testimap2.csia at unipd.it Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: Effective uid=1003, gid=1003, home=/SYSTEM/USERS/homes/unipd.it/t/testimap2.csia at unipd.it Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: acl: No acl_shared_dict setting - shared mailbox listing is disabled Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: Quota root: name=User quota backend=dict args=:file:/SYSTEM/USERS/homes/unipd.it/t/testimap2.csia at unipd.it/testimap2.csia at unipd.it.quota Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: Quota rule: root=User quota mailbox=* bytes=1153433600 messages=0 Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: Quota rule: root=User quota mailbox=Trash bytes=+104857600 messages=0 Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: Quota warning: bytes=1095761920 (95%) messages=0 reverse=no command=quota-warning 95 testimap2.csia at unipd.it Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: Quota warning: bytes=922746880 (80%) messages=0 reverse=no command=quota-warning 80 testimap2.csia at unipd.it Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: Quota grace: root=User quota bytes=115343360 (10%) Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: dict quota: user=testimap2.csia at unipd.it, uri=file:/SYSTEM/USERS/homes/unipd.it/t/testimap2.csia at unipd.it/testimap2.csia at unipd.it.quota, noenforcing=0 Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=sdbox:/NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox:INDEX=/INDEX1/unipd.it/t/testimap2.csia at unipd.it:ALT=/ALTSTORAGE1/unipd.it/t/testimap2.csia at unipd.it Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: fs: root=/NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox, index=/INDEX1/unipd.it/t/testimap2.csia at unipd.it, indexpvt=, control=, inbox=, alt=/ALTSTORAGE1/unipd.it/t/testimap2.csia at unipd.it Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: acl: initializing backend with data: vfile:/etc/dovecot/dovecot-acl Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: acl: acl username = testimap2.csia at unipd.it Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: acl: owner = 1 Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: acl vfile: Global ACL file: /etc/dovecot/dovecot-acl Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: Namespace : type=private, prefix=_PostaCancellataPerErrore/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=yes location=maildir:/SYSTEM/USERS/mailbox/unipd.it/t/testimap2.csia at unipd.it/expunged Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: maildir++: root=/SYSTEM/USERS/mailbox/unipd.it/t/testimap2.csia at unipd.it/expunged, index=, indexpvt=, control=, inbox=, alt= Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: acl: initializing backend with data: vfile:/etc/dovecot/dovecot-acl Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: acl: acl username = testimap2.csia at unipd.it Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: acl: owner = 1 Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: acl vfile: Global ACL file: /etc/dovecot/dovecot-acl Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Debug: acl vfile: file /NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox/mailboxes/INBOX/dbox-Mails/dovecot-acl not found Mar 26 09:59:44 imap(testimap2.csia at unipd.it): Info: Disconnected: Logged out in=77 out=1009 Mar 26 10:01:50 imap(testimap2.csia at unipd.it): Debug: acl vfile: file /SYSTEM/USERS/mailbox/unipd.it/t/testimap2.csia at unipd.it/expunged/.INBOX\2eTrash/dovecot-acl not found Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: acl vfile: file /SYSTEM/USERS/mailbox/unipd.it/t/testimap2.csia at unipd.it/expunged/.\2eLEGGIMI/dovecot-acl not found Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: acl vfile: file /NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox/mailboxes/INBOX.Trash/dbox-Mails/dovecot-acl not found Mar 26 10:01:51 imap-login: Info: Login: user=, method=PLAIN, rip=147.162.200.67, lip=147.162.10.86, mpid=2474, session= Mar 26 10:01:51 script-login: Debug: Added userdb setting: mail_location=sdbox:/NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox:INDEX=/INDEX1/unipd.it/t/testimap2.csia at unipd.it:ALT=/ALTSTORAGE1/unipd.it/t/testimap2.csia at unipd.it Mar 26 10:01:51 imap: Debug: Loading modules from directory: /usr/lib/dovecot/modules Mar 26 10:01:51 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib01_acl_plugin.so Mar 26 10:01:51 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib02_imap_acl_plugin.so Mar 26 10:01:51 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib02_lazy_expunge_plugin.so Mar 26 10:01:51 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Mar 26 10:01:51 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so Mar 26 10:01:51 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib15_notify_plugin.so Mar 26 10:01:51 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib20_listescape_plugin.so Mar 26 10:01:51 imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib20_mail_log_plugin.so Mar 26 10:01:51 imap: Debug: Added userdb setting: mail_location=sdbox:/NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox:INDEX=/INDEX1/unipd.it/t/testimap2.csia at unipd.it:ALT=/ALTSTORAGE1/unipd.it/t/testimap2.csia at unipd.it Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: Effective uid=1003, gid=1003, home=/SYSTEM/USERS/homes/unipd.it/t/testimap2.csia at unipd.it Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: acl: No acl_shared_dict setting - shared mailbox listing is disabled Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: Quota root: name=User quota backend=dict args=:file:/SYSTEM/USERS/homes/unipd.it/t/testimap2.csia at unipd.it/testimap2.csia at unipd.it.quota Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: Quota rule: root=User quota mailbox=* bytes=1153433600 messages=0 Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: Quota rule: root=User quota mailbox=Trash bytes=+104857600 messages=0 Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: Quota warning: bytes=1095761920 (95%) messages=0 reverse=no command=quota-warning 95 testimap2.csia at unipd.it Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: Quota warning: bytes=922746880 (80%) messages=0 reverse=no command=quota-warning 80 testimap2.csia at unipd.it Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: Quota grace: root=User quota bytes=115343360 (10%) Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: dict quota: user=testimap2.csia at unipd.it, uri=file:/SYSTEM/USERS/homes/unipd.it/t/testimap2.csia at unipd.it/testimap2.csia at unipd.it.quota, noenforcing=0 Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=sdbox:/NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox:INDEX=/INDEX1/unipd.it/t/testimap2.csia at unipd.it:ALT=/ALTSTORAGE1/unipd.it/t/testimap2.csia at unipd.it Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: fs: root=/NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox, index=/INDEX1/unipd.it/t/testimap2.csia at unipd.it, indexpvt=, control=, inbox=, alt=/ALTSTORAGE1/unipd.it/t/testimap2.csia at unipd.it Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: acl: initializing backend with data: vfile:/etc/dovecot/dovecot-acl Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: acl: acl username = testimap2.csia at unipd.it Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: acl: owner = 1 Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: acl vfile: Global ACL file: /etc/dovecot/dovecot-acl Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: Namespace : type=private, prefix=_PostaCancellataPerErrore/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=yes location=maildir:/SYSTEM/USERS/mailbox/unipd.it/t/testimap2.csia at unipd.it/expunged Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: maildir++: root=/SYSTEM/USERS/mailbox/unipd.it/t/testimap2.csia at unipd.it/expunged, index=, indexpvt=, control=, inbox=, alt= Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: acl: initializing backend with data: vfile:/etc/dovecot/dovecot-acl Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: acl: acl username = testimap2.csia at unipd.it Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: acl: owner = 1 Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: acl vfile: Global ACL file: /etc/dovecot/dovecot-acl Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: acl vfile: file /NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox/mailboxes/INBOX.Trash/dbox-Mails/dovecot-acl not found Mar 26 10:01:51 imap(testimap2.csia at unipd.it): Debug: acl vfile: file /NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox/mailboxes/Drafts/dbox-Mails/dovecot-acl not found Mar 26 10:01:52 imap(testimap2.csia at unipd.it): Debug: acl vfile: file /NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox/mailboxes/Spam/dbox-Mails/dovecot-acl not found Mar 26 10:01:52 imap(testimap2.csia at unipd.it): Debug: acl vfile: file /NAS1/unipd.it/mailbox/t/testimap2.csia at unipd.it/sdbox/mailboxes/Templates/dbox-Mails/dovecot-acl not found From bernd at petrovitsch.priv.at Thu Mar 26 11:05:46 2015 From: bernd at petrovitsch.priv.at (Bernd Petrovitsch) Date: Thu, 26 Mar 2015 12:05:46 +0100 Subject: Dovecot Oy merger with Open-Xchange AG In-Reply-To: <55101E65.6080508@cs.ucy.ac.cy> References: <8250CA2C-936F-468D-B5C6-88954DC6FB2F@iki.fi> <55101E65.6080508@cs.ucy.ac.cy> Message-ID: <1427367947.3163.10.camel@thorin> On Mon, 2015-03-23 at 16:08 +0200, Andreas Kasenides wrote: > I am usually emotionally (at least) against of open-source projects > loosing their independence to large corporations. Possibly due to bad OX-AG is a "large corporation"? Did I miss something? Kind regards, Bernd -- "I dislike type abstraction if it has no real reason. And saving on typing is not a good reason - if your typing speed is the main issue when you're coding, you're doing something seriously wrong." - Linus Torvalds From EDV2 at BehrensM.de Thu Mar 26 12:25:39 2015 From: EDV2 at BehrensM.de (BehrensM) Date: Thu, 26 Mar 2015 13:25:39 +0100 Subject: Error "Next message unexpectedly lost from mbox file" Message-ID: <5513FAC3.8030509@BehrensM.de> Hi, i am the new one;) - and my first post is about a problem i ran into. I installed a mailserver on a RaspberryPi. It works mostly well, using dovecot 2.1.7. (exim4 4.8 (not in use, smtp direct to Provider), fetchmail 6.3.21, RasPi 3.12.28+, Roundcube (no version known) - the whole system went through an apg-get update/uppgrade this night to look if a known problem already been fixed. When getting bigger Attachments (not sure if it occurs by smaller too, but bigger attachments are normally "to open" and often fail) Not all of them, but much. Unpredictable when it occurs. These Attachments are shown with less size in the Mailclient (Thunderbird actual version). In Roundcube they are shown to small, too. But it looks as the mail IS "fullsized" on the lokal Mailserver: When i use Roundcube and try to open an x00kB .zip, Roundcube starts to get the whole xyMB (real size) .zip-file, even when disconnected from internet. Thunderbird behaves different, doesnt load the full file, so "ignore" is no option;) And yes, i always dont like that option:) Beside the lokal mailserver-account(s) there is/are pulled Mails via imap direct from the mailprovider, the "old" solution which shall be replaced by the new mailserver. imap direct from provider shows the mails/Attachments in corrects size an opening works. Only using the local mailserver fails. In the log-files i found (i quote a part in which - cronjob matching - one of these defect mails was pulled): In Syslog: Mar 26 09:40:01 raspberrypi /USR/SBIN/CRON[26696]: (info) CMD (/usr/bin/fetchmail -s --fetchsizelimit 0 --fetchlimit 0 --limit 0 --timeout 300) Mar 26 09:40:05 raspberrypi dovecot: imap(info): Error: Next message unexpectedly lost from mbox file /home/info/mail/.INBOX at 1066141935 (cached) Mar 26 09:40:05 raspberrypi dovecot: imap(info): Error: read(/home/info/mail/.INBOX) failed: Invalid argument (FETCH for mailbox INBOX UID 1247) Mar 26 09:40:05 raspberrypi dovecot: imap(info): Disconnected: Internal error occurred. Refer to server log for more information. [2015-03-26 09:40:05] in= 2693 out=172626 Mar 26 09:40:07 raspberrypi dovecot: imap(info): Panic: file mbox-sync.c: line 1286 (mbox_sync_handle_eof_updates): assertion failed: (trailer_size <= 2) Mar 26 09:40:07 raspberrypi dovecot: imap(info): Fatal: master: service(imap): child 26341 killed with signal 6 (core dumps disabled) Mar 26 09:40:51 raspberrypi dovecot: imap-login: Login: user=, method=PLAIN, rip=192.168.100.27, lip=192.168.100.199, mpid=26723, TLS, session= Mar 26 09:42:16 raspberrypi dovecot: imap(info): Connection closed in=752 out=21810 And in mail.err: Mar 26 09:40:05 raspberrypi dovecot: imap(info): Error: Next message unexpectedly lost from mbox file /home/info/mail/.INBOX at 1066141935 (cached) Mar 26 09:40:05 raspberrypi dovecot: imap(info): Error: read(/home/info/mail/.INBOX) failed: Invalid argument (FETCH for mailbox INBOX UID 1247) Mar 26 09:40:07 raspberrypi dovecot: imap(info): Panic: file mbox-sync.c: line 1286 (mbox_sync_handle_eof_updates): assertion failed: (trailer_size <= 2) Mar 26 09:40:07 raspberrypi dovecot: imap(info): Fatal: master: service(imap): child 26341 killed with signal 6 (core dumps disabled) (To tell the whole story: /home/$user/mail is link by NFS to a NAS. The limit 0 and timeout entries are just to test after running into the error and before i discovered that not fetchmail but dovecot looks to cause the problem) In the archive i only found an issue in 2010 with a CR/LF-problem, but it is a) a long time ago an b) i am not sure that my specific problem comes from the same direction. http://www.dovecot.org/list/dovecot/2010-November/054938.html Any suggestions how to solve that problem? Thanks. -------------- next part -------------- root at raspberrypi:~# doveconf -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.12.28+ armv6l Debian 7.8 mail_location = mbox:~/mail:LAYOUT=maildir++:INBOX=~/mail/.INBOX:CONTROL=~/mail/control:INDEX=~/mail/indexes mail_privileged_group = 1 namespace { inbox = yes location = mailbox { special_use = \Drafts name = Drafts } mailbox { special_use = \Junk name = Junk } mailbox { special_use = \Sent name = Sent } mailbox { special_use = \Sent name = Sent Messages } mailbox { special_use = \Trash name = Trash } prefix = name = inbox } passdb { driver = pam } protocols = " imap" service replication-notify-fifo { name = aggregator } service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap { name = imap-login } service login/imap { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service replicator { name = replicator } service login/ssl-params { name = ssl-params } service stats-mail { name = stats } ssl_cert = References: <8250CA2C-936F-468D-B5C6-88954DC6FB2F@iki.fi> <55101E65.6080508@cs.ucy.ac.cy> <1427367947.3163.10.camel@thorin> Message-ID: <55141F56.5070708@cs.ucy.ac.cy> On 26/03/15 13:05, Bernd Petrovitsch wrote: > On Mon, 2015-03-23 at 16:08 +0200, Andreas Kasenides wrote: >> I am usually emotionally (at least) against of open-source projects >> loosing their independence to large corporations. Possibly due to bad > OX-AG is a "large corporation"? > Did I miss something? > > Kind regards, > Bernd I have no idea how large is OX-AG. That is not what I said anyway. At least not what I meant. I was trying to emphasize large or larger commercial entities trying to take advantage of the OSS community. Which happened many times in the past. Andreas -- Andreas Kasenides Senior IT Officer Dept. of Computer Science, University of Cyprus Tel: 22892714, Fax: 22892701 (5B4ANK) From uothrawn at yahoo.com Fri Mar 27 02:58:03 2015 From: uothrawn at yahoo.com (G H) Date: Thu, 26 Mar 2015 22:58:03 -0400 Subject: Option to not add "Received" header ? Message-ID: <271445.10397.bm@smtp226.mail.bf1.yahoo.com> You could remove them with sieve in the latest version of pigeonhole. On Mar 24, 2015 7:33 AM, Florent B wrote: > > I know about RFC's, but that could be an option, not enabled by default. From miloslav.hula at gmail.com Fri Mar 27 08:19:10 2015 From: miloslav.hula at gmail.com (=?ISO-8859-2?Q?Miloslav_H=F9la?=) Date: Fri, 27 Mar 2015 09:19:10 +0100 Subject: Migrating from Cyrus to Dovecot Message-ID: <5515127E.2050205@gmail.com> Hi, we are migrating from Cyrus 2.3.7 to Dovecot 2.2.13. We have ~7000 maildirs with ~500GB. Our goal is to do the migration without users have notice and with the shortest service downtime. The users use IMAP (with shared folders and ACL), POP3 and sieve filters. As a first choice, we tried the Dovecot's dsync tool. First tests were great, but we are not able to change the Cyrus auth backend for migration. Moreover, this migration seems too slow for us. As a second try, we tried the cyrus2dovecot migrating Perl scripts (and their derivates) from Wiki2. More or less they works but we found we need more control during the migration. So, as a third try, we wrote own migrating scripts. And thanks to the cyrus2dovecot it wasn't too much complicated. And there are my questions: A) Files and dirs timestamps The mtime of email file is important as an internal date as I found on Wiki2. But what about timestamps of cur/new/tmp directories or Dovecot's internal files line dovecot-uidlist? Do they play some role here? B) The 128 bit mailbox UID The Wiki2 speaks about 128 bit mailbox UID at first line of dovecot-uidlist. Cyrus preserves only 64 bit UID. Is this mailbox UID required by Dovecot? If so, can we use 0000000000000000501100008c4a11c1 (Cyrus UID padded by zeros)? C) Format of dovecot-uidlist records Wiki2 shows two examples: 25006 :1276528487.M364837P9451.kurkku,S=1355,W=1394:2, 25017 W2481 :1276533073.M242911P3632.kurkku:2,F Which format is preferred? Or what the benefits are? D) Converting between CRLF and LF If I understand correctly, Dovecot stores emails with LF only. We have all emails with CRLF now on Cyrus and converting them to LF only is a little more time consuming. Is there any benefit to do that? Or can we live with 'mail_save_crlf' without problems? E) POP3 backend I found many informations about IMAP internals but few on POP3 internals. What do I need to do POP3 migration transparent for user? Many thanks for any answers. Regards, Milo From florent at coppint.com Fri Mar 27 08:58:20 2015 From: florent at coppint.com (Florent B) Date: Fri, 27 Mar 2015 09:58:20 +0100 Subject: Option to not add "Received" header ? In-Reply-To: <271445.10397.bm@smtp226.mail.bf1.yahoo.com> References: <271445.10397.bm@smtp226.mail.bf1.yahoo.com> Message-ID: <55151BAC.30407@coppint.com> Nice thank you :) On 03/27/2015 03:58 AM, G H wrote: > You could remove them with sieve in the latest version of pigeonhole. > > On Mar 24, 2015 7:33 AM, Florent B wrote: >> I know about RFC's, but that could be an option, not enabled by default. From pch at myzel.net Fri Mar 27 10:36:13 2015 From: pch at myzel.net (Peter Chiochetti) Date: Fri, 27 Mar 2015 11:36:13 +0100 Subject: IMAP ANNOTATE Extension RFC5257: priority on roadmap In-Reply-To: <3835EC85-05B5-4998-9B47-6210A381A3A1@iki.fi> References: <550D285E.50801@agitos.de> <3835EC85-05B5-4998-9B47-6210A381A3A1@iki.fi> Message-ID: <5515329D.5040702@myzel.net> Am 2015-03-22 um 18:06 schrieb Timo Sirainen: > On 21 Mar 2015, at 10:14, Florian Sager wrote: >> >> I would highly appreciate if you at Dovecot could increase the >> priority of ANNOTATE in your roadmap. > > ANNOTATE is > something that I'd like to implement at some point, but it's also > quite a large change and likely will have to wait until Dovecot v2.3. > Ideally, and of course, I speak from my personal point of view, annotations should be something, that SOLR would index according to its own mapping, which I am in control of. Use case: incoming mails here are not directly answered in the MUA, but some third party software creates outgoing emails, with a formal yet unique subject; inbound messages could then be /tagged/ with the same identifier and a single search would yield the complete correspondence? Basically, adding a header to a message would do my job as well, but seems to contradict with what IMAP seems to be about: preserving messages unadulterated, which is a noble goal indeed. OTOH, no hurry, client support of ANNOTATE is not there yet either (at least in Thunderbird). -- peter From edgaras.lukosevicius at gmail.com Fri Mar 27 11:21:15 2015 From: edgaras.lukosevicius at gmail.com (=?utf-8?Q?Edgaras_Luko=C5=A1evi=C4=8Dius?=) Date: Fri, 27 Mar 2015 13:21:15 +0200 Subject: postfix sasl -> haproxy -> dovecot auth Message-ID: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> Hello, is it possible to configure configure haproxy to work with postfix sasl and dovecot auth like this: clients -> 25:postfix -> 20025:haproxy -> 20025:auth-backend-1, 20025:auth-backend-2 The configuration I have now gives me this error randomly: 535 5.7.8 Error: authentication failed: Connection lost to authentication server This is probably because haproxy change servers while session is still active (postfix sasl don?t establish new connection to auth service every time new auth request arrives) Note that haproxy is between postfix and dovecot and is not facing clients directly, so there is no way to keep persistent connections by client ip. # POSTFIX smtpd_sasl_auth_enable = yes smtpd_sasl_authenticated_header = yes smtpd_sasl_exceptions_networks = smtpd_sasl_local_domain = smtpd_sasl_path = inet:127.0.0.1:20025 smtpd_sasl_security_options = noanonymous smtpd_sasl_tls_security_options = $smtpd_sasl_security_options smtpd_sasl_type = dovecot # HAPROX frontend postfix-sasl bind 127.0.0.1:20025 default_backend dovecot-auth backend dovecot-auth mode tcp option tcplog option srvtcpka hash-type consistent balance roundrobin server mail-backend-1 31.220.19.52:20025 check server mail-backend-2 31.220.19.53:20025 check From me at junc.eu Fri Mar 27 11:29:54 2015 From: me at junc.eu (Benny Pedersen) Date: Fri, 27 Mar 2015 12:29:54 +0100 Subject: postfix sasl -> haproxy -> dovecot auth In-Reply-To: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> References: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> Message-ID: <1225d5d94e525d8508c8147e3ce117eb@junc.eu> Edgaras Luko?evi?ius skrev den 2015-03-27 12:21: > is it possible to configure configure haproxy to work with postfix > sasl and dovecot auth like this: > > clients -> 25:postfix -> 20025:haproxy -> 20025:auth-backend-1, > 20025:auth-backend-2 configure cyrus-sasl as a remote imap client is more simple if imap hostname is dns round robin it would be ha-avail already keep postfix simple From edgaras.lukosevicius at gmail.com Fri Mar 27 13:34:04 2015 From: edgaras.lukosevicius at gmail.com (=?utf-8?Q?Edgaras_Luko=C5=A1evi=C4=8Dius?=) Date: Fri, 27 Mar 2015 15:34:04 +0200 Subject: postfix sasl -> haproxy -> dovecot auth In-Reply-To: <1225d5d94e525d8508c8147e3ce117eb@junc.eu> References: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> <1225d5d94e525d8508c8147e3ce117eb@junc.eu> Message-ID: Can?t dovecot authenticate against imap? What I need is to make smtp authentication balanced and keep everything in backend (private network) On 27 Mar 2015, at 13:29, Benny Pedersen wrote: > Edgaras Luko?evi?ius skrev den 2015-03-27 12:21: > >> is it possible to configure configure haproxy to work with postfix >> sasl and dovecot auth like this: >> clients -> 25:postfix -> 20025:haproxy -> 20025:auth-backend-1, >> 20025:auth-backend-2 > > configure cyrus-sasl as a remote imap client is more simple > > if imap hostname is dns round robin it would be ha-avail already > > keep postfix simple From gedalya at gedalya.net Fri Mar 27 13:48:17 2015 From: gedalya at gedalya.net (Gedalya) Date: Fri, 27 Mar 2015 09:48:17 -0400 Subject: postfix sasl -> haproxy -> dovecot auth In-Reply-To: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> References: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> Message-ID: <55155FA1.6060601@gedalya.net> On 03/27/2015 07:21 AM, Edgaras Luko?evi?ius wrote: > Hello, > > is it possible to configure configure haproxy to work with postfix sasl and dovecot auth like this: > > clients -> 25:postfix -> 20025:haproxy -> 20025:auth-backend-1, 20025:auth-backend-2 Why don't you set up a dovecot locally (with only auth service) on each postfix box? From me at junc.eu Fri Mar 27 13:49:35 2015 From: me at junc.eu (Benny Pedersen) Date: Fri, 27 Mar 2015 14:49:35 +0100 Subject: postfix sasl -> haproxy -> dovecot auth In-Reply-To: References: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> <1225d5d94e525d8508c8147e3ce117eb@junc.eu> Message-ID: <87c6c2e78aeb783497fbc2d020fc0e76@junc.eu> Edgaras Luko?evi?ius skrev den 2015-03-27 14:34: > Can?t dovecot authenticate against imap? will it be trusted ? > What I need is to make smtp authentication balanced and keep > everything in backend (private network) dovecot is not a smtp server, thats why i say cyrus-sasl yes cyrus-sasl is ha-awail with rimap, but there is a minor problem with it, haproxy and rimap have both the same problem to connect to one ip that times out before the next ip is used, haproxy does imho not solve this better then rimap From cma at cmadams.net Fri Mar 27 13:52:25 2015 From: cma at cmadams.net (Chris Adams) Date: Fri, 27 Mar 2015 08:52:25 -0500 Subject: postfix sasl -> haproxy -> dovecot auth In-Reply-To: References: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> <1225d5d94e525d8508c8147e3ce117eb@junc.eu> Message-ID: <20150327135225.GC4727@cmadams.net> Once upon a time, Edgaras Luko?evi?ius said: > What I need is to make smtp authentication balanced and keep everything in backend (private network) If you have more than one Postfix server, each one must talk to its own private Dovecot server for auth. The Dovecot auth protocol includes a client (Postfix) assigned ID, and Postfix uses the process ID. If you have multiple Postfix servers talking to one Dovecot server, you'll get ID conflicts and dropped auths. I ended up putting a local instance of Dovecot on each Postfix server, with no protcols configured except for auth. Not quite as HA, but I have my monitoring system doing SMTP AUTH (never have had a problem with the setup); you could probably have HAProxy do it as well (IIRC it can do some basic expect-style send/receive). -- Chris Adams From edgaras.lukosevicius at gmail.com Fri Mar 27 13:58:59 2015 From: edgaras.lukosevicius at gmail.com (=?utf-8?Q?Edgaras_Luko=C5=A1evi=C4=8Dius?=) Date: Fri, 27 Mar 2015 15:58:59 +0200 Subject: postfix sasl -> haproxy -> dovecot auth In-Reply-To: <87c6c2e78aeb783497fbc2d020fc0e76@junc.eu> References: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> <1225d5d94e525d8508c8147e3ce117eb@junc.eu> <87c6c2e78aeb783497fbc2d020fc0e76@junc.eu> Message-ID: <719BCBAA-A72F-4C70-8162-9ECB92A00869@gmail.com> I don?t want to allow public network facing servers to be able to reach passwords database. And I want to segregate roles of the servers. If I will setup dovecot locally I will still have to provide it access to database (eg. /etc/dovecot/dovecot-sql.conf.ext). On 27 Mar 2015, at 15:49, Benny Pedersen wrote: > Edgaras Luko?evi?ius skrev den 2015-03-27 14:34: >> Can?t dovecot authenticate against imap? > > will it be trusted ? > >> What I need is to make smtp authentication balanced and keep >> everything in backend (private network) > > dovecot is not a smtp server, thats why i say cyrus-sasl > > yes cyrus-sasl is ha-awail with rimap, but there is a minor problem with it, haproxy and rimap have both the same problem to connect to one ip that times out before the next ip is used, haproxy does imho not solve this better then rimap From me at junc.eu Fri Mar 27 14:04:33 2015 From: me at junc.eu (Benny Pedersen) Date: Fri, 27 Mar 2015 15:04:33 +0100 Subject: postfix sasl -> haproxy -> dovecot auth In-Reply-To: <55155FA1.6060601@gedalya.net> References: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> <55155FA1.6060601@gedalya.net> Message-ID: <05f0d82d03400b9762a181b560580c87@junc.eu> Gedalya skrev den 2015-03-27 14:48: >> is it possible to configure configure haproxy to work with postfix >> sasl and dovecot auth like this: >> clients -> 25:postfix -> 20025:haproxy -> 20025:auth-backend-1, >> 20025:auth-backend-2 > Why don't you set up a dovecot locally (with only auth service) on > each postfix box? cyrus-sasl is still needed, and dovecot will be overkill just for auth client From me at junc.eu Fri Mar 27 14:27:36 2015 From: me at junc.eu (Benny Pedersen) Date: Fri, 27 Mar 2015 15:27:36 +0100 Subject: postfix sasl -> haproxy -> dovecot auth In-Reply-To: <719BCBAA-A72F-4C70-8162-9ECB92A00869@gmail.com> References: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> <1225d5d94e525d8508c8147e3ce117eb@junc.eu> <87c6c2e78aeb783497fbc2d020fc0e76@junc.eu> <719BCBAA-A72F-4C70-8162-9ECB92A00869@gmail.com> Message-ID: <4fbc8e4c114536218f2686b56b329b76@junc.eu> Edgaras Luko?evi?ius skrev den 2015-03-27 14:58: > I don?t want to allow public network facing servers to be able to > reach passwords database. And I want to segregate roles of the > servers. > If I will setup dovecot locally I will still have to provide it access > to database (eg. /etc/dovecot/dovecot-sql.conf.ext). did you read cyrus-sasl docs ? it can read auth from sql, so no need for dovecot there From h.reindl at thelounge.net Fri Mar 27 14:37:54 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 27 Mar 2015 15:37:54 +0100 Subject: postfix sasl -> haproxy -> dovecot auth In-Reply-To: <87c6c2e78aeb783497fbc2d020fc0e76@junc.eu> References: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> <1225d5d94e525d8508c8147e3ce117eb@junc.eu> <87c6c2e78aeb783497fbc2d020fc0e76@junc.eu> Message-ID: <55156B42.1050804@thelounge.net> Am 27.03.2015 um 14:49 schrieb Benny Pedersen: >> What I need is to make smtp authentication balanced and keep >> everything in backend (private network) > > dovecot is not a smtp server, thats why i say cyrus-sasl jesus christ keep your smart-ass responses for yourself http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From h.reindl at thelounge.net Fri Mar 27 14:41:30 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 27 Mar 2015 15:41:30 +0100 Subject: postfix sasl -> haproxy -> dovecot auth In-Reply-To: <05f0d82d03400b9762a181b560580c87@junc.eu> References: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> <55155FA1.6060601@gedalya.net> <05f0d82d03400b9762a181b560580c87@junc.eu> Message-ID: <55156C1A.3010207@thelounge.net> Am 27.03.2015 um 15:04 schrieb Benny Pedersen: > Gedalya skrev den 2015-03-27 14:48: > >>> is it possible to configure configure haproxy to work with postfix >>> sasl and dovecot auth like this: >>> clients -> 25:postfix -> 20025:haproxy -> 20025:auth-backend-1, >>> 20025:auth-backend-2 >> Why don't you set up a dovecot locally (with only auth service) on >> each postfix box? > > cyrus-sasl is still needed bullshit and to be honest nobody right in his mind aware of the capabilities configures cyrus-sasl on a server where postfix and dovecot are running already instead just use one common auth layer for incoming and outgoing mail supporting the same mechs and configuration http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL # configure backend for postfix sasl-auth service auth { unix_listener /var/spool/postfix/private/auth { mode = 0660 user = postfix group = postfix } } smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From silvia.tormo at addapta.com Thu Mar 26 14:52:54 2015 From: silvia.tormo at addapta.com (Silvia Tormo) Date: Thu, 26 Mar 2015 15:52:54 +0100 Subject: Error: open() failed with file /var/vmail/... Too many open files Message-ID: Hello guys we have Dovecot 2.0.19 and we are getting those errors opening files of dovecot Too many open files, we?ve tried to increase limits in /etc/security/limits Wih ulimits ?a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 63735 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 102400 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 63735 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited But if i make cat/proc/dovecotpid/limits Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size 0 unlimited bytes Max resident set unlimited unlimited bytes Max processes 63735 63735 processes Max open files 1024 4096 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 63735 63735 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us I am still getting 1024 limit The only warning that we get is doveconf: Warning: service auth { client_limit=4096 } is lower than required under max. load (9516) doveconf: Warning: service anvil { client_limit=1000 } is lower than required under max. load (4299) I?ve tried all i have found in internet, but there is no more info, and is like i?ve increased system limits but not dovecot limits. Thankyou This is our CONF # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.8.0-44-generic x86_64 Ubuntu 12.04.5 LTS ext4 auth_anonymous_username = anonymous auth_cache_negative_ttl = 1 hours auth_cache_size = 0 auth_cache_ttl = 1 hours auth_debug = no auth_debug_passwords = no auth_default_realm = auth_failure_delay = 2 secs auth_first_valid_uid = 500 auth_gssapi_hostname = VM8-ATISA.atisa.es auth_krb5_keytab = /etc/dovecot/dovecot.keytab auth_last_valid_uid = 0 auth_master_user_separator = auth_mechanisms = plain login gssapi auth_realms = auth_socket_path = auth-userdb auth_ssl_require_client_cert = no auth_ssl_username_from_cert = no auth_use_winbind = no auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ auth_username_format = auth_username_translation = auth_verbose = no auth_verbose_passwords = no auth_winbind_helper_path = /usr/bin/ntlm_auth auth_worker_max_count = 30 base_dir = /var/run/dovecot config_cache_size = 1 M debug_log_path = default_client_limit = 1000 default_idle_kill = 60 default_internal_user = dovecot default_login_user = dovenull default_process_limit = 100 default_vsz_limit = 256 M deliver_log_format = msgid=%m: %$ dict_db_config = director_doveadm_port = 0 director_mail_servers = director_servers = director_user_expire = 15 mins disable_plaintext_auth = no dotlock_use_excl = yes doveadm_allowed_commands = doveadm_password = doveadm_proxy_port = 0 doveadm_socket_path = doveadm-server doveadm_worker_count = 0 first_valid_gid = 112 first_valid_uid = 107 hostname = imap_capability = imap_client_workarounds = imap_id_log = imap_id_send = imap_idle_notify_interval = 2 mins imap_logout_format = bytes=%i/%o imap_max_line_length = 64 k import_environment = TZ info_log_path = instance_name = dovecot last_valid_gid = 0 last_valid_uid = 0 lda_mailbox_autocreate = no lda_mailbox_autosubscribe = no lda_original_recipient_header = libexec_dir = /usr/lib/dovecot listen = *, :: lmtp_proxy = no lmtp_save_to_detail_mailbox = no lock_method = fcntl log_path = syslog log_timestamp = "%Y-%m-%d %H:%M:%S " login_access_sockets = login_greeting = Dovecot ready. login_log_format = %$: %s login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c login_trusted_networks = mail_access_groups = mail_attachment_dir = mail_attachment_fs = sis posix mail_attachment_hash = %{sha1} mail_attachment_min_size = 128 k mail_cache_fields = flags mail_cache_min_mail_count = 0 mail_chroot = mail_debug = no mail_fsync = optimized mail_full_filesystem_access = no mail_gid = 112 mail_home = /var/vmail/sieve/%d/%u mail_location = maildir:/var/vmail/%$ mail_log_prefix = "%s(%u): " mail_max_keyword_length = 50 mail_max_lock_timeout = 0 mail_max_userip_connections = 10 mail_never_cache_fields = imap.envelope mail_nfs_index = no mail_nfs_storage = no mail_plugin_dir = /usr/lib/dovecot/modules mail_plugins = mail_privileged_group = mail_save_crlf = no mail_temp_dir = /tmp mail_uid = 107 mailbox_idle_check_interval = 30 secs mailbox_list_index_disable = no maildir_copy_with_hardlinks = yes maildir_stat_dirs = no maildir_very_dirty_syncs = no managesieve_client_workarounds = managesieve_implementation_string = Dovecot Pigeonhole managesieve_logout_format = bytes=%i/%o managesieve_max_compile_errors = 5 managesieve_max_line_length = 65536 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body e notify environment mailbox date ihave master_user_separator = mbox_dirty_syncs = yes mbox_dotlock_change_timeout = 2 mins mbox_lazy_writes = yes mbox_lock_timeout = 5 mins mbox_min_index_size = 0 mbox_read_locks = fcntl mbox_very_dirty_syncs = no mbox_write_locks = dotlock fcntl mdbox_preallocate_space = no mdbox_rotate_interval = 0 mdbox_rotate_size = 2 M mmap_disable = no passdb { args = /etc/dovecot/dovecot-ldap.conf deny = no driver = ldap master = no pass = no } plugin { autocreate = Trash autocreate2 = Spam autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = Spam autosubscribe3 = Sent autosubscribe4 = Drafts quota = maildir:User quota quota_rule = *:storage=0 sieve = /var/vmail/sieve/%d/%u/sieve-script sieve_dir = /var/vmail/sieve/%d/%u sieve_global_path = /var/vmail/sieve/default.sieve sieve_storage = /var/vmail/sieve/%d/%u } pop3_client_workarounds = pop3_enable_last = no pop3_fast_size_lookups = no pop3_lock_session = no pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_no_flag_updates = no pop3_reuse_xuidl = no pop3_save_uidl = no pop3_uidl_format = %08Xu%08Xv postmaster_address = protocols = pop3 imap sieve quota_full_tempfail = no recipient_delimiter = + rejection_reason = Your message to <%t> was automatically rejected:%n%r rejection_subject = Rejected: %s sendmail_path = /usr/sbin/sendmail service anvil { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = anvil extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 1 protocol = service_count = 0 type = anvil unix_listener anvil-auth-penalty { group = mode = 0600 user = } unix_listener anvil { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service auth-worker { chroot = client_limit = 1 drop_priv_before_exec = no executable = auth -w extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener auth-worker { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service auth { chroot = client_limit = 4096 drop_priv_before_exec = no executable = /usr/lib/dovecot/auth extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-client { group = mode = 0600 user = } unix_listener auth-login { group = mode = 0600 user = $default_internal_user } unix_listener auth-master { group = ebox mode = 0600 user = ebox } unix_listener auth-userdb { group = mode = 0600 user = } unix_listener login/login { group = mode = 0666 user = } user = dovecot vsz_limit = 18446744073709551615 B } service config { chroot = client_limit = 0 drop_priv_before_exec = no executable = config extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = config unix_listener config { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service dict { chroot = client_limit = 1 drop_priv_before_exec = no executable = dict extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dict { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service director { chroot = client_limit = 0 drop_priv_before_exec = no executable = director extra_groups = fifo_listener login/proxy-notify { group = mode = 00 user = } group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener director-admin { group = mode = 0600 user = } unix_listener login/director { group = mode = 00 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service dns_client { chroot = client_limit = 1 drop_priv_before_exec = no executable = dns-client extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dns-client { group = mode = 0666 user = } unix_listener login/dns-client { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service doveadm { chroot = client_limit = 1 drop_priv_before_exec = no executable = doveadm-server extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener doveadm-server { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service imap-login { chroot = login client_limit = 1024 drop_priv_before_exec = no executable = imap-login extra_groups = group = idle_kill = 0 inet_listener imap { address = * port = 143 ssl = no } inet_listener imaps { address = * port = 993 ssl = yes } privileged_group = process_limit = 4096 process_min_avail = 0 protocol = imap service_count = 0 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service imap { chroot = client_limit = 1024 drop_priv_before_exec = no executable = imap extra_groups = group = idle_kill = 0 privileged_group = process_limit = 4096 process_min_avail = 0 protocol = imap service_count = 0 type = unix_listener login/imap { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service ipc { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = ipc extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener ipc { group = mode = 0600 user = } unix_listener login/ipc-proxy { group = mode = 0600 user = $default_login_user } user = $default_internal_user vsz_limit = 18446744073709551615 B } service lmtp { chroot = client_limit = 1 drop_priv_before_exec = no executable = lmtp extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = lmtp service_count = 0 type = unix_listener lmtp { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B service log { chroot = client_limit = 0 drop_priv_before_exec = no executable = log extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = log user = vsz_limit = 18446744073709551615 B } service managesieve-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = managesieve-login extra_groups = group = idle_kill = 0 inet_listener sieve { address = port = 4190 ssl = no } privileged_group = process_limit = 0 process_min_avail = 0 protocol = sieve service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service managesieve { chroot = client_limit = 1 drop_priv_before_exec = no executable = managesieve extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = sieve service_count = 1 type = unix_listener login/sieve { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service pop3-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = pop3-login extra_groups = group = idle_kill = 0 inet_listener pop3 { address = port = 110 ssl = no } inet_listener pop3s { address = port = 995 ssl = yes } privileged_group = process_limit = 0 process_min_avail = 0 protocol = pop3 service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service pop3 { chroot = client_limit = 1 drop_priv_before_exec = no executable = pop3 extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = pop3 service_count = 1 type = unix_listener login/pop3 { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service ssl-params { chroot = client_limit = 0 drop_priv_before_exec = no executable = ssl-params extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = startup unix_listener login/ssl-params { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } shutdown_clients = yes ssl = yes ssl_ca = ssl_cert = From edgaras.lukosevicius at gmail.com Fri Mar 27 15:00:40 2015 From: edgaras.lukosevicius at gmail.com (=?utf-8?Q?Edgaras_Luko=C5=A1evi=C4=8Dius?=) Date: Fri, 27 Mar 2015 17:00:40 +0200 Subject: postfix sasl -> haproxy -> dovecot auth In-Reply-To: <4fbc8e4c114536218f2686b56b329b76@junc.eu> References: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> <1225d5d94e525d8508c8147e3ce117eb@junc.eu> <87c6c2e78aeb783497fbc2d020fc0e76@junc.eu> <719BCBAA-A72F-4C70-8162-9ECB92A00869@gmail.com> <4fbc8e4c114536218f2686b56b329b76@junc.eu> Message-ID: <3FCB3879-1ECF-4209-829E-3DF2F4DC9033@gmail.com> I will install cyrus-sasl and see how it goes. Anyway, it would be nice to have same features (authentication agains imap) in dovecot. On 27 Mar 2015, at 16:27, Benny Pedersen wrote: > Edgaras Luko?evi?ius skrev den 2015-03-27 14:58: > >> I don?t want to allow public network facing servers to be able to >> reach passwords database. And I want to segregate roles of the >> servers. > >> If I will setup dovecot locally I will still have to provide it access >> to database (eg. /etc/dovecot/dovecot-sql.conf.ext). > > did you read cyrus-sasl docs ? > > it can read auth from sql, so no need for dovecot there From gedalya at gedalya.net Fri Mar 27 15:03:41 2015 From: gedalya at gedalya.net (Gedalya) Date: Fri, 27 Mar 2015 11:03:41 -0400 Subject: postfix sasl -> haproxy -> dovecot auth In-Reply-To: <3FCB3879-1ECF-4209-829E-3DF2F4DC9033@gmail.com> References: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> <1225d5d94e525d8508c8147e3ce117eb@junc.eu> <87c6c2e78aeb783497fbc2d020fc0e76@junc.eu> <719BCBAA-A72F-4C70-8162-9ECB92A00869@gmail.com> <4fbc8e4c114536218f2686b56b329b76@junc.eu> <3FCB3879-1ECF-4209-829E-3DF2F4DC9033@gmail.com> Message-ID: <5515714D.4050207@gedalya.net> On 03/27/2015 11:00 AM, Edgaras Luko?evi?ius wrote: > Anyway, it would be nice to have same features (authentication agains imap) in dovecot. Check this out http://wiki2.dovecot.org/PasswordDatabase/IMAP From me at junc.eu Fri Mar 27 15:10:23 2015 From: me at junc.eu (Benny Pedersen) Date: Fri, 27 Mar 2015 16:10:23 +0100 Subject: postfix sasl -> haproxy -> dovecot auth In-Reply-To: <3FCB3879-1ECF-4209-829E-3DF2F4DC9033@gmail.com> References: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> <1225d5d94e525d8508c8147e3ce117eb@junc.eu> <87c6c2e78aeb783497fbc2d020fc0e76@junc.eu> <719BCBAA-A72F-4C70-8162-9ECB92A00869@gmail.com> <4fbc8e4c114536218f2686b56b329b76@junc.eu> <3FCB3879-1ECF-4209-829E-3DF2F4DC9033@gmail.com> Message-ID: <15c8b6cfde18d42fc69683fa08cd09ac@junc.eu> Edgaras Luko?evi?ius skrev den 2015-03-27 16:00: > I will install cyrus-sasl and see how it goes. +1 > Anyway, it would be nice to have same features (authentication agains > imap) in dovecot. read more docs in cyrus-sasl, did i say rimap ? :=) From me at junc.eu Fri Mar 27 15:14:31 2015 From: me at junc.eu (Benny Pedersen) Date: Fri, 27 Mar 2015 16:14:31 +0100 Subject: postfix sasl -> haproxy -> dovecot auth In-Reply-To: <5515714D.4050207@gedalya.net> References: "\"\\\"<00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> <1225d5d94e525d8508c8147e3ce117eb@junc.eu> " <87c6c2e78aeb783497fbc2d020fc0e76@junc.eu>" "<719BCBAA-A72F-4C70-8162-9ECB92A00869@gmail.com>\" <4fbc8e4c114536218f2686b56b329b76@junc.eu> <3FCB3879-1ECF-4209-829E-3DF2F4DC9033@gmail.com>" <5515714D.4050207@gedalya.net> Message-ID: <3c7359b1c0459199cf4432aad510ef23@junc.eu> Gedalya skrev den 2015-03-27 16:03: > On 03/27/2015 11:00 AM, Edgaras Luko?evi?ius wrote: >> Anyway, it would be nice to have same features (authentication agains >> imap) in dovecot. > Check this out > http://wiki2.dovecot.org/PasswordDatabase/IMAP this is when one dovecot need to have auth from another dovecot auth master backend, not when postfix need to have authed users on remote imho there says postfix in subject ? From gedalya at gedalya.net Fri Mar 27 15:18:05 2015 From: gedalya at gedalya.net (Gedalya) Date: Fri, 27 Mar 2015 11:18:05 -0400 Subject: postfix sasl -> haproxy -> dovecot auth In-Reply-To: <3c7359b1c0459199cf4432aad510ef23@junc.eu> References: "\"\\\"<00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> <1225d5d94e525d8508c8147e3ce117eb@junc.eu> " <87c6c2e78aeb783497fbc2d020fc0e76@junc.eu>" "<719BCBAA-A72F-4C70-8162-9ECB92A00869@gmail.com>\" <4fbc8e4c114536218f2686b56b329b76@junc.eu> <3FCB3879-1ECF-4209-829E-3DF2F4DC9033@gmail.com>" <5515714D.4050207@gedalya.net> <3c7359b1c0459199cf4432aad510ef23@junc.eu> Message-ID: <551574AD.6030801@gedalya.net> On 03/27/2015 11:14 AM, Benny Pedersen wrote: > Gedalya skrev den 2015-03-27 16:03: >> On 03/27/2015 11:00 AM, Edgaras Luko?evi?ius wrote: >>> Anyway, it would be nice to have same features (authentication >>> agains imap) in dovecot. >> Check this out >> http://wiki2.dovecot.org/PasswordDatabase/IMAP > > this is when one dovecot need to have auth from another dovecot auth > master backend, not when postfix need to have authed users on remote > In that case you should kindly update the Wiki page to say that. From edgaras.lukosevicius at gmail.com Fri Mar 27 19:34:25 2015 From: edgaras.lukosevicius at gmail.com (=?UTF-8?Q?Edgaras_Luko=C5=A1evi=C4=8Dius?=) Date: Fri, 27 Mar 2015 21:34:25 +0200 Subject: postfix sasl -> haproxy -> dovecot auth In-Reply-To: <15c8b6cfde18d42fc69683fa08cd09ac@junc.eu> References: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> <1225d5d94e525d8508c8147e3ce117eb@junc.eu> <87c6c2e78aeb783497fbc2d020fc0e76@junc.eu> <719BCBAA-A72F-4C70-8162-9ECB92A00869@gmail.com> <4fbc8e4c114536218f2686b56b329b76@junc.eu> <3FCB3879-1ECF-4209-829E-3DF2F4DC9033@gmail.com> <15c8b6cfde18d42fc69683fa08cd09ac@junc.eu> Message-ID: Are you a cyrus developer ir what? :D 2015 kov. 27 17:10 "Benny Pedersen" ra??: > Edgaras Luko?evi?ius skrev den 2015-03-27 16:00: > >> I will install cyrus-sasl and see how it goes. >> > > +1 > > Anyway, it would be nice to have same features (authentication agains >> imap) in dovecot. >> > > read more docs in cyrus-sasl, did i say rimap ? :=) > From tss at iki.fi Fri Mar 27 20:56:23 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 27 Mar 2015 22:56:23 +0200 Subject: postfix sasl -> haproxy -> dovecot auth In-Reply-To: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> References: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> Message-ID: On 27 Mar 2015, at 13:21, Edgaras Luko?evi?ius wrote: > > Hello, > > is it possible to configure configure haproxy to work with postfix sasl and dovecot auth like this: > > clients -> 25:postfix -> 20025:haproxy -> 20025:auth-backend-1, 20025:auth-backend-2 > > The configuration I have now gives me this error randomly: > 535 5.7.8 Error: authentication failed: Connection lost to authentication server > > This is probably because haproxy change servers while session is still active (postfix sasl don?t establish new connection to auth service every time new auth request arrives) > > Note that haproxy is between postfix and dovecot and is not facing clients directly, so there is no way to keep persistent connections by client ip. There's nothing Dovecot can do about it, because the error handling is in Postfix code. Although I suppose Dovecot-auth could drop the connection itself, but that would still cause random problems if Postfix was just about to authenticate using that connection. Postfix could in theory handle a dropped auth connection by reconnecting and retrying, although maybe still logging a warning. That doesn't work perfectly for all SASL mechanisms though. Does haproxy disconnect completely randomly or only after the connection has existed for n minutes? Maybe having a Postfix setting for "max time for auth connection existence before reconnect" would work too if it was set lower than haproxy's connection timeout. But yeah, in any case it would need Postfix code changes. As for recommending IMAP authentication (whether via Cyrus or via Dovecot imapc): It would work of course, but I wouldn't want to use it myself. It causes extra confusion in logs because you now need to separate out the SMTP-auth-IMAP-logins from regular IMAP-logins (assuming you care about that at all). IMAP login is also much more expensive than a simple authentication check. From tss at iki.fi Fri Mar 27 21:03:48 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 27 Mar 2015 23:03:48 +0200 Subject: Error: open() failed with file /var/vmail/... Too many open files In-Reply-To: References: Message-ID: <14A9C026-1097-4959-A3B8-23E5B0D7ECEF@iki.fi> On 26 Mar 2015, at 16:52, Silvia Tormo wrote: > > Hello guys we have Dovecot 2.0.19 and we are getting those errors opening > files of dovecot Too many open files, we?ve tried to increase limits in > /etc/security/limits /etc/security/ settings don't affect Dovecot (or any other server program). You need to put the ulimit commands somewhere in the init script. Or perhaps adding the ulimit command to /etc/default/dovecot would work. I don't know what's the correct Ubuntu way to do it. From tss at iki.fi Fri Mar 27 21:12:12 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 27 Mar 2015 23:12:12 +0200 Subject: Migrating from Cyrus to Dovecot In-Reply-To: <5515127E.2050205@gmail.com> References: <5515127E.2050205@gmail.com> Message-ID: <715D864F-3DE2-4458-A293-B4F9BEF88C16@iki.fi> > On 27 Mar 2015, at 10:19, Miloslav H?la wrote: > > Hi, > > we are migrating from Cyrus 2.3.7 to Dovecot 2.2.13. We have ~7000 maildirs with ~500GB. Our goal is to do the migration without users have notice and with the shortest service downtime. The users use IMAP (with shared folders and ACL), POP3 and sieve filters. > > As a first choice, we tried the Dovecot's dsync tool. First tests were great, but we are not able to change the Cyrus auth backend for migration. Moreover, this migration seems too slow for us. > > As a second try, we tried the cyrus2dovecot migrating Perl scripts (and their derivates) from Wiki2. More or less they works but we found we need more control during the migration. > > So, as a third try, we wrote own migrating scripts. And thanks to the cyrus2dovecot it wasn't too much complicated. And there are my questions: > > A) Files and dirs timestamps > The mtime of email file is important as an internal date as I found on Wiki2. But what about timestamps of cur/new/tmp directories or Dovecot's internal files line dovecot-uidlist? Do they play some role here? No. > B) The 128 bit mailbox UID > The Wiki2 speaks about 128 bit mailbox UID at first line of dovecot-uidlist. Cyrus preserves only 64 bit UID. Is this mailbox UID required by Dovecot? If so, can we use 0000000000000000501100008c4a11c1 (Cyrus UID padded by zeros)? The mailbox GUID is internal to Dovecot. There's no standard IMAP way to see it, so there's no need to migrate it. Better not to set it and let Dovecot generate it automatically. > C) Format of dovecot-uidlist records > Wiki2 shows two examples: > 25006 :1276528487.M364837P9451.kurkku,S=1355,W=1394:2, > 25017 W2481 :1276533073.M242911P3632.kurkku:2,F > > Which format is preferred? Or what the benefits are? If W=size is in the filename, it never needs to be recalculated if dovecot-uidlist is lost. Of course, dovecot-uidlist should never be lost. So I don't think it makes a huge difference. If you care about performance, sdbox/mdbox mailbox format would behave much better. sdbox is a close match to Cyrus - so with Maildir you're actually likely making the disk I/O performance somewhat slower in Dovecot than in Cyrus, although that also depends on other things. > D) Converting between CRLF and LF > If I understand correctly, Dovecot stores emails with LF only. We have all emails with CRLF now on Cyrus and converting them to LF only is a little more time consuming. Is there any benefit to do that? Or can we live with 'mail_save_crlf' without problems? Dovecot can automatically handle both mixed CRLF and LF mails, you can keep old mails as CRLF and new mails as LF. mail_save_crlf setting only controls what is used for new emails. If you want to save more disk space you can enable compression. > E) POP3 backend > I found many informations about IMAP internals but few on POP3 internals. What do I need to do POP3 migration transparent for user? Just preserve the UIDL. See the pop3_uidl_format setting in http://wiki2.dovecot.org/Migration/Cyrus From tss at iki.fi Fri Mar 27 21:15:33 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 27 Mar 2015 23:15:33 +0200 Subject: Courier-dovecot migration issue: Forward and Junk flags In-Reply-To: <5512D049.2080405@powerhosting.dk> References: <5512D049.2080405@powerhosting.dk> Message-ID: <8A2073C2-8DF7-4B5B-9903-FA4C2F8E23B4@iki.fi> On 25 Mar 2015, at 17:12, Dennis H?jgaard wrote: > > Hello.. Another courier -> dovecot migrater here.. I ham having a bit of trouble converting the courier maildir correctly, or actually i'm almost there. The only thing i experience is that forwarded messages and messages marked as junk (in thunderbird) are downloaded again (i am using thunderbird and caching) after running the courier-dovecot-migrate.pl script and connecting to the newly configured dovecot imap server.. So i think that the Forward and Junk flag is not being converted.. What do you mean "downloaded again" - how do you see it? Are you talking about IMAP or POP3 protocol? A missing IMAP flag doesn't cause redownloading, it would at most cause the mail to not be displayed as forwarded/junk. It sounds like something else is going wrong (the mail getting a new UID for some reason?) From silvia.tormo at addapta.com Fri Mar 27 21:22:45 2015 From: silvia.tormo at addapta.com (Silvia Tormo) Date: Fri, 27 Mar 2015 22:22:45 +0100 Subject: Error: open() failed with file /var/vmail/... Too many open files Message-ID: <20150327212248.5A2F6802382@VM8-ADDAPTA.addapta.com> I've already tried creating etc/default/dovecot chmod +x it and inside put ulimit but didn't work. Also i've tried to put the ulimit into the /etc/init.d/dovecot linked file but i found still 1024 the limit kn dovecot. El 27/03/2015 22:03, Timo Sirainen escribi?: > > On 26 Mar 2015, at 16:52, Silvia Tormo wrote: > > > > Hello guys we have Dovecot 2.0.19 and we are getting those errors opening > > files of dovecot Too many open files, we?ve tried to increase limits in > > /etc/security/limits > > /etc/security/ settings don't affect Dovecot (or any other server program). You need to put the ulimit commands somewhere in the init script. Or perhaps adding the ulimit command to /etc/default/dovecot would work. I don't know what's the correct Ubuntu way to do it. From jtam.home at gmail.com Fri Mar 27 22:32:49 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Fri, 27 Mar 2015 15:32:49 -0700 (PDT) Subject: Error "Next message unexpectedly lost from mbox file" In-Reply-To: References: Message-ID: BehrensM writes: > Beside the lokal mailserver-account(s) there is/are pulled Mails via > imap direct from the mailprovider, the "old" solution which shall be > replaced by the new mailserver. imap direct from provider shows the > mails/Attachments in corrects size an opening works. > Only using the local mailserver fails. > ... > Mar 26 09:40:05 raspberrypi dovecot: imap(info): Error: Next message > unexpectedly lost from mbox file /home/info/mail/.INBOX at 1066141935 > (cached) I couldn't follow all that you wrote, but these errors (actually, more like warnings) usually occur if you have a mail reader or any utility that modifies a mailbox outside of dovecot's sight. For example, a mail reader that does local manipulation of mbox files rather than via IMAP. This log message is more or less saying that the indices are out of sync with respect to the mailbox, and the message that it expected to find at a file offset is not there anymore. > Mar 26 09:40:07 raspberrypi dovecot: imap(info): Panic: file > mbox-sync.c: line 1286 (mbox_sync_handle_eof_updates): assertion failed: > (trailer_size <= 2) This, though, should not happen. Your version of dovecot is rather old, so maybe an update will solve this. Joseph Tam From me at junc.eu Sat Mar 28 04:07:55 2015 From: me at junc.eu (Benny Pedersen) Date: Sat, 28 Mar 2015 05:07:55 +0100 Subject: postfix sasl -> haproxy -> dovecot auth In-Reply-To: References: <00EAEBA7-D2AC-4D0E-AAD1-7A7182E6414F@gmail.com> <1225d5d94e525d8508c8147e3ce117eb@junc.eu> <87c6c2e78aeb783497fbc2d020fc0e76@junc.eu> <719BCBAA-A72F-4C70-8162-9ECB92A00869@gmail.com> <4fbc8e4c114536218f2686b56b329b76@junc.eu> <3FCB3879-1ECF-4209-829E-3DF2F4DC9033@gmail.com> <15c8b6cfde18d42fc69683fa08cd09ac@junc.eu> Message-ID: Edgaras Luko?evi?ius skrev den 2015-03-27 20:34: > Are you a cyrus developer ir what? :D thanks for asking :) just used cyrus-sasl with sql long time ago, when i runned courier-imap and postfix lda with openwisp admin, fork of postfixadmin, while coded a bit of policyd v1, sadly no one use policyd v1 anymore :( if it works now i am happy that you now finaly got it From yawowb+dovecot at nuclei.ca Sat Mar 28 09:24:37 2015 From: yawowb+dovecot at nuclei.ca (rooster) Date: Sat, 28 Mar 2015 02:24:37 -0700 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> Message-ID: <86653E56-6216-4F78-BBC7-E9E4F950F374@nuclei.ca> > On 2015-03-23, at 1:34 AM, rooster wrote: > > Hello list, > > I have been struggling with establishing a working installation of dovecot with mySQL quota support. I have not been able to find a guide or tutorial that works for the current revisions of software I have at my disposal. > > Most notably, the error I see in the mail log is as follows : > > Mar 23 00:55:31 host dovecot: lmtp(328): Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so > Mar 23 00:55:31 host dovecot: lmtp(328): Error: dlopen(/usr/lib/dovecot/modules/lib11_imap_quota_plugin.so) failed: /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so: undefined symbol: command_unregister > Mar 23 00:55:31 host dovecot: lmtp(328): Fatal: Couldn't load required plugins > > I have attached my dovecot -n output. The installed revision of dovecot is 2.2.9 on ubuntu 14.04.02 LTS. > > Thank you in advance. > > -- > > It should be noted, that if I remove quota and imap_quota from mail_plugins , the error message goes away. Of course though, this is not the desired configuration (no quota support). -- From zucca at systemschmiede.com Sat Mar 28 09:42:23 2015 From: zucca at systemschmiede.com (zucca at systemschmiede.com) Date: Sat, 28 Mar 2015 10:42:23 +0100 Subject: Error after setting up fts /solr for Open-Xchange In-Reply-To: <550F03FD.9080104@systemschmiede.com> References: <550F03FD.9080104@systemschmiede.com> Message-ID: <5516777F.3020606@systemschmiede.com> Hi everyone, may i jolt this again please...still found no solution at all to this really. Thank you very much, i appreciate your help! Cheers Sascha Am 22.03.2015 um 19:03 schrieb zucca at systemschmiede.com: > Hi List, > > i have tried to get fts / solr running with virtual all folder with > Open-Xchange. > I followed a pretty well written howto from Open-Xchanges Intranet, > and i first thought everything was running smooth, as the first one or > two searches were successful. However now whenever i try to search > again for something in the virtual "all folders" Open-Xchange tells me: > > *Error* > *Folder "virtual.all" has been closed on mail server xxx.xxx.xxx.xxx > Probably your request took too long.** > * > > /var/log/mail.err says: > Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Panic: file > fts-search-serialize.c: line 63 (fts_search_deserialize_add_idx): > assertion failed: (*idx < buf->used) > Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Error: Raw > backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x6c15f) [0x7f5110f5e15f] > -> /usr/lib/dovecot/libdovecot.so.0(+0x6c1be) [0x7f5110f5e1be] -> > /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f5110f1740e] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xc847) [0x7f510fb07847] > -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_search_deserialize_add_nonmatches+0x1c) > [0x7f510fb0799c] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd263) [0x7f510fb08263] > -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd3e6) > [0x7f510fb083e6] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa812e) > [0x7f511127112e] -> > /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblock+0x4b) > [0x7f51112717bb] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_search_next_nonblock+0x13e) > [0x7f510f8f2a7e] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_search_next_nonblock+0x7e) > [0x7f510f8f29be] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x27) > [0x7f511124b277] -> dovecot/imap(+0x1f27f) [0x7f511172527f] -> > dovecot/imap(imap_search_start+0xfd) [0x7f51117256fd] -> > dovecot/imap(cmd_sort+0x205) [0x7f511171b6c5] -> > dovecot/imap(command_exec+0x3c) [0x7f511171fc4c] -> > dovecot/imap(+0x18c30) [0x7f511171ec30] -> dovecot/imap(+0x18cea) > [0x7f511171ecea] -> dovecot/imap(client_handle_input+0x115) > [0x7f511171efb5] -> dovecot/imap(client_input+0x75) [0x7f511171f385] > -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4e) > [0x7f5110f6efbe] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) > [0x7f5110f6ffb7] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) > [0x7f5110f6f049] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) > [0x7f5110f6f0c8] -> > /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) > [0x7f5110f1c7b3] -> dovecot/imap(main+0x2ae) [0x7f511171352e] -> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) > [0x7f5110b85ead] -> dovecot/imap(+0xd69d) [0x7f511171369d] > Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Fatal: > master: service(imap): child 6594 killed with signal 6 (core dumps > disabled) > Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Panic: file > fts-search-serialize.c: line 63 (fts_search_deserialize_add_idx): > assertion failed: (*idx < buf->used) > Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Error: Raw > backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x6c15f) [0x7fae1c14c15f] > -> /usr/lib/dovecot/libdovecot.so.0(+0x6c1be) [0x7fae1c14c1be] -> > /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fae1c10540e] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xc847) [0x7fae1acf5847] > -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_search_deserialize_add_nonmatches+0x1c) > [0x7fae1acf599c] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd263) [0x7fae1acf6263] > -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd3e6) > [0x7fae1acf63e6] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa812e) > [0x7fae1c45f12e] -> > /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblock+0x4b) > [0x7fae1c45f7bb] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_search_next_nonblock+0x13e) > [0x7fae1aae0a7e] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_search_next_nonblock+0x7e) > [0x7fae1aae09be] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x27) > [0x7fae1c439277] -> dovecot/imap(+0x1f27f) [0x7fae1c91327f] -> > dovecot/imap(imap_search_start+0xfd) [0x7fae1c9136fd] -> > dovecot/imap(cmd_sort+0x205) [0x7fae1c9096c5] -> > dovecot/imap(command_exec+0x3c) [0x7fae1c90dc4c] -> > dovecot/imap(+0x18c30) [0x7fae1c90cc30] -> dovecot/imap(+0x18cea) > [0x7fae1c90ccea] -> dovecot/imap(client_handle_input+0x115) > [0x7fae1c90cfb5] -> dovecot/imap(client_input+0x75) [0x7fae1c90d385] > -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4e) > [0x7fae1c15cfbe] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) > [0x7fae1c15dfb7] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) > [0x7fae1c15d049] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) > [0x7fae1c15d0c8] -> > /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) > [0x7fae1c10a7b3] -> dovecot/imap(main+0x2ae) [0x7fae1c90152e] -> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) > [0x7fae1bd73ead] -> dovecot/imap(+0xd69d) [0x7fae1c90169d] > Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Fatal: > master: service(imap): child 6598 killed with signal 6 (core dumps > disabled) > > I don't even have any fts-search-serialize.c on the system... > > Maybe someone is able to help me? > I know we're nearly there, but i can't find anything on this anywhere. > If there's anything else you need to know in order to help, please let > me know. > Thank you very very much! > Sascha > > > _this is set under /opt/open-xchange/etc/findbasic.properties for > Open-Xchange__:_ > > com.openexchange.find.basic.mail.allMessagesFolder = virtual.all > > > > _grep -v '^ *\(#.*\)\?$' /etc/dovecot/dovecot-sql.conf:_ > > driver = mysql > connect = host=localhost dbname=dbispconfig user=ispconfig > password=123456789123456789123456789 > default_pass_scheme = CRYPT > password_query = SELECT password FROM mail_user WHERE (login = '%u' OR > email = '%u') AND disable%Ls = 'n' > user_query = SELECT email as user, maildir as home, CONCAT('maildir:', > maildir, '/Maildir') as mail, uid, gid, CONCAT('*:storage=', quota, > 'B') AS quota_rule, CONCAT(maildir, '/.sieve') as sieve FROM mail_user > WHERE (login = '%u' OR email = '%u') AND `disable%Ls` = 'n' > > > _||dovecot --version: > > _2.2.13 > > > _dovecot -n: > > _# 2.2.13: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-37-pve x86_64 Debian 7.8 simfs > auth_mechanisms = plain login > disable_plaintext_auth = no > listen = *,[::] > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_location = maildir:/var/vmail/%d/%n/Maildir > mail_max_userip_connections = 150 > mail_plugins = " acl fts fts_solr virtual" > mail_privileged_group = vmail > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date ihave > namespace { > inbox = yes > location = > prefix = > separator = . > type = private > } > namespace { > list = children > location = > maildir:/var/vmail/%%d/%%n/Maildir:INDEX=/var/vmail/%d/%n/Maildir/shared/%%u > prefix = Shared/.%%n/. > separator = . > subscriptions = no > type = shared > } > namespace virtual { > location = virtual:/etc/dovecot/virtual:INDEX=~/virtual > mailbox all { > special_use = \All > } > prefix = virtual. > separator = . > } > passdb { > args = /etc/dovecot/dovecot-sql.conf > driver = sql > } > plugin { > acl = vfile > acl_shared_dict = file:/var/vmail/shared-mailboxes.db > fts = solr > fts_autoindex = yes > fts_solr = url=http://localhost:8090/solr/ > quota = dict:user::file:/var/vmail/%d/%n/.quotausage > sieve = /var/vmail/%d/%n/.sieve > } > protocols = imap pop3 sieve > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > unix_listener auth-userdb { > group = vmail > mode = 0600 > user = vmail > } > user = root > } > service imap-login { > client_limit = 1000 > process_limit = 500 > } > ssl_ca = ssl_cert = ssl_key = ssl_protocols = !SSLv2 !SSLv3 > userdb { > args = /etc/dovecot/dovecot-sql.conf > driver = sql > } > protocol imap { > mail_plugins = " acl fts fts_solr virtual imap_acl quota imap_quota" > } > protocol pop3 { > mail_plugins = quota virtual > pop3_uidl_format = %08Xu%08Xv > } > protocol lda { > mail_plugins = sieve quota virtual > } From rs at sys4.de Sat Mar 28 10:21:09 2015 From: rs at sys4.de (Robert Schetterer) Date: Sat, 28 Mar 2015 11:21:09 +0100 Subject: Error after setting up fts /solr for Open-Xchange In-Reply-To: <5516777F.3020606@systemschmiede.com> References: <550F03FD.9080104@systemschmiede.com> <5516777F.3020606@systemschmiede.com> Message-ID: <55168095.4010807@sys4.de> Am 28.03.2015 um 10:42 schrieb zucca at systemschmiede.com: > Hi everyone, > > may i jolt this again please...still found no solution at all to this > really. > Thank you very much, i appreciate your help! > Cheers > Sascha a good idea might be upgrade to recent, before do more debug Timo Sirainen [Thu, 12 Mar 2015 17:41:05 +0200] rev 18340 Released v2.2.16 lots of fixes with i.e fts http://hg.dovecot.org/dovecot-2.2/log?rev=fts your version Timo Sirainen [Sun, 11 May 2014 22:33:08 +0300] rev 17372 Released v2.2.13. > > Am 22.03.2015 um 19:03 schrieb zucca at systemschmiede.com: >> Hi List, >> >> i have tried to get fts / solr running with virtual all folder with >> Open-Xchange. >> I followed a pretty well written howto from Open-Xchanges Intranet, >> and i first thought everything was running smooth, as the first one or >> two searches were successful. However now whenever i try to search >> again for something in the virtual "all folders" Open-Xchange tells me: >> >> *Error* >> *Folder "virtual.all" has been closed on mail server xxx.xxx.xxx.xxx >> Probably your request took too long.** >> * >> >> /var/log/mail.err says: >> Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Panic: file >> fts-search-serialize.c: line 63 (fts_search_deserialize_add_idx): >> assertion failed: (*idx < buf->used) >> Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Error: Raw >> backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x6c15f) [0x7f5110f5e15f] >> -> /usr/lib/dovecot/libdovecot.so.0(+0x6c1be) [0x7f5110f5e1be] -> >> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f5110f1740e] -> >> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xc847) [0x7f510fb07847] >> -> >> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_search_deserialize_add_nonmatches+0x1c) >> [0x7f510fb0799c] -> >> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd263) [0x7f510fb08263] >> -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd3e6) >> [0x7f510fb083e6] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa812e) >> [0x7f511127112e] -> >> /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblock+0x4b) >> [0x7f51112717bb] -> >> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_search_next_nonblock+0x13e) >> [0x7f510f8f2a7e] -> >> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_search_next_nonblock+0x7e) >> [0x7f510f8f29be] -> >> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x27) >> [0x7f511124b277] -> dovecot/imap(+0x1f27f) [0x7f511172527f] -> >> dovecot/imap(imap_search_start+0xfd) [0x7f51117256fd] -> >> dovecot/imap(cmd_sort+0x205) [0x7f511171b6c5] -> >> dovecot/imap(command_exec+0x3c) [0x7f511171fc4c] -> >> dovecot/imap(+0x18c30) [0x7f511171ec30] -> dovecot/imap(+0x18cea) >> [0x7f511171ecea] -> dovecot/imap(client_handle_input+0x115) >> [0x7f511171efb5] -> dovecot/imap(client_input+0x75) [0x7f511171f385] >> -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4e) >> [0x7f5110f6efbe] -> >> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) >> [0x7f5110f6ffb7] -> >> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) >> [0x7f5110f6f049] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) >> [0x7f5110f6f0c8] -> >> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) >> [0x7f5110f1c7b3] -> dovecot/imap(main+0x2ae) [0x7f511171352e] -> >> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) >> [0x7f5110b85ead] -> dovecot/imap(+0xd69d) [0x7f511171369d] >> Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Fatal: >> master: service(imap): child 6594 killed with signal 6 (core dumps >> disabled) >> Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Panic: file >> fts-search-serialize.c: line 63 (fts_search_deserialize_add_idx): >> assertion failed: (*idx < buf->used) >> Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Error: Raw >> backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x6c15f) [0x7fae1c14c15f] >> -> /usr/lib/dovecot/libdovecot.so.0(+0x6c1be) [0x7fae1c14c1be] -> >> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fae1c10540e] -> >> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xc847) [0x7fae1acf5847] >> -> >> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_search_deserialize_add_nonmatches+0x1c) >> [0x7fae1acf599c] -> >> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd263) [0x7fae1acf6263] >> -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd3e6) >> [0x7fae1acf63e6] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa812e) >> [0x7fae1c45f12e] -> >> /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblock+0x4b) >> [0x7fae1c45f7bb] -> >> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_search_next_nonblock+0x13e) >> [0x7fae1aae0a7e] -> >> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_search_next_nonblock+0x7e) >> [0x7fae1aae09be] -> >> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x27) >> [0x7fae1c439277] -> dovecot/imap(+0x1f27f) [0x7fae1c91327f] -> >> dovecot/imap(imap_search_start+0xfd) [0x7fae1c9136fd] -> >> dovecot/imap(cmd_sort+0x205) [0x7fae1c9096c5] -> >> dovecot/imap(command_exec+0x3c) [0x7fae1c90dc4c] -> >> dovecot/imap(+0x18c30) [0x7fae1c90cc30] -> dovecot/imap(+0x18cea) >> [0x7fae1c90ccea] -> dovecot/imap(client_handle_input+0x115) >> [0x7fae1c90cfb5] -> dovecot/imap(client_input+0x75) [0x7fae1c90d385] >> -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4e) >> [0x7fae1c15cfbe] -> >> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) >> [0x7fae1c15dfb7] -> >> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) >> [0x7fae1c15d049] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) >> [0x7fae1c15d0c8] -> >> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) >> [0x7fae1c10a7b3] -> dovecot/imap(main+0x2ae) [0x7fae1c90152e] -> >> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) >> [0x7fae1bd73ead] -> dovecot/imap(+0xd69d) [0x7fae1c90169d] >> Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Fatal: >> master: service(imap): child 6598 killed with signal 6 (core dumps >> disabled) >> >> I don't even have any fts-search-serialize.c on the system... >> >> Maybe someone is able to help me? >> I know we're nearly there, but i can't find anything on this anywhere. >> If there's anything else you need to know in order to help, please let >> me know. >> Thank you very very much! >> Sascha >> >> >> _this is set under /opt/open-xchange/etc/findbasic.properties for >> Open-Xchange__:_ >> >> com.openexchange.find.basic.mail.allMessagesFolder = virtual.all >> >> >> >> _grep -v '^ *\(#.*\)\?$' /etc/dovecot/dovecot-sql.conf:_ >> >> driver = mysql >> connect = host=localhost dbname=dbispconfig user=ispconfig >> password=123456789123456789123456789 >> default_pass_scheme = CRYPT >> password_query = SELECT password FROM mail_user WHERE (login = '%u' OR >> email = '%u') AND disable%Ls = 'n' >> user_query = SELECT email as user, maildir as home, CONCAT('maildir:', >> maildir, '/Maildir') as mail, uid, gid, CONCAT('*:storage=', quota, >> 'B') AS quota_rule, CONCAT(maildir, '/.sieve') as sieve FROM mail_user >> WHERE (login = '%u' OR email = '%u') AND `disable%Ls` = 'n' >> >> >> _||dovecot --version: >> >> _2.2.13 >> >> >> _dovecot -n: >> >> _# 2.2.13: /etc/dovecot/dovecot.conf >> # OS: Linux 2.6.32-37-pve x86_64 Debian 7.8 simfs >> auth_mechanisms = plain login >> disable_plaintext_auth = no >> listen = *,[::] >> log_timestamp = "%Y-%m-%d %H:%M:%S " >> mail_location = maildir:/var/vmail/%d/%n/Maildir >> mail_max_userip_connections = 150 >> mail_plugins = " acl fts fts_solr virtual" >> mail_privileged_group = vmail >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date ihave >> namespace { >> inbox = yes >> location = >> prefix = >> separator = . >> type = private >> } >> namespace { >> list = children >> location = >> maildir:/var/vmail/%%d/%%n/Maildir:INDEX=/var/vmail/%d/%n/Maildir/shared/%%u >> >> prefix = Shared/.%%n/. >> separator = . >> subscriptions = no >> type = shared >> } >> namespace virtual { >> location = virtual:/etc/dovecot/virtual:INDEX=~/virtual >> mailbox all { >> special_use = \All >> } >> prefix = virtual. >> separator = . >> } >> passdb { >> args = /etc/dovecot/dovecot-sql.conf >> driver = sql >> } >> plugin { >> acl = vfile >> acl_shared_dict = file:/var/vmail/shared-mailboxes.db >> fts = solr >> fts_autoindex = yes >> fts_solr = url=http://localhost:8090/solr/ >> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >> sieve = /var/vmail/%d/%n/.sieve >> } >> protocols = imap pop3 sieve >> service auth { >> unix_listener /var/spool/postfix/private/auth { >> group = postfix >> mode = 0660 >> user = postfix >> } >> unix_listener auth-userdb { >> group = vmail >> mode = 0600 >> user = vmail >> } >> user = root >> } >> service imap-login { >> client_limit = 1000 >> process_limit = 500 >> } >> ssl_ca = > ssl_cert = > ssl_key = > ssl_protocols = !SSLv2 !SSLv3 >> userdb { >> args = /etc/dovecot/dovecot-sql.conf >> driver = sql >> } >> protocol imap { >> mail_plugins = " acl fts fts_solr virtual imap_acl quota imap_quota" >> } >> protocol pop3 { >> mail_plugins = quota virtual >> pop3_uidl_format = %08Xu%08Xv >> } >> protocol lda { >> mail_plugins = sieve quota virtual >> } Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From zucca at systemschmiede.com Sat Mar 28 11:22:15 2015 From: zucca at systemschmiede.com (Sascha Zucca) Date: Sat, 28 Mar 2015 12:22:15 +0100 Subject: Error after setting up fts /solr for Open-Xchange In-Reply-To: <55168095.4010807@sys4.de> References: <550F03FD.9080104@systemschmiede.com> <5516777F.3020606@systemschmiede.com> <55168095.4010807@sys4.de> Message-ID: <93872201-0FC3-4BE4-9F25-918BBBF6500D@systemschmiede.com> Allright... will try. Thanks Am 28. M?rz 2015 11:21:09 MEZ, schrieb Robert Schetterer : >Am 28.03.2015 um 10:42 schrieb zucca at systemschmiede.com: >> Hi everyone, >> >> may i jolt this again please...still found no solution at all to this >> really. >> Thank you very much, i appreciate your help! >> Cheers >> Sascha > > >a good idea might be upgrade to recent, before do more debug > > >Timo Sirainen [Thu, 12 Mar 2015 17:41:05 +0200] rev 18340 >Released v2.2.16 > >lots of fixes with i.e fts > >http://hg.dovecot.org/dovecot-2.2/log?rev=fts > >your version >Timo Sirainen [Sun, 11 May 2014 22:33:08 +0300] rev 17372 >Released v2.2.13. > >> >> Am 22.03.2015 um 19:03 schrieb zucca at systemschmiede.com: >>> Hi List, >>> >>> i have tried to get fts / solr running with virtual all folder with >>> Open-Xchange. >>> I followed a pretty well written howto from Open-Xchanges Intranet, >>> and i first thought everything was running smooth, as the first one >or >>> two searches were successful. However now whenever i try to search >>> again for something in the virtual "all folders" Open-Xchange tells >me: >>> >>> *Error* >>> *Folder "virtual.all" has been closed on mail server xxx.xxx.xxx.xxx >>> Probably your request took too long.** >>> * >>> >>> /var/log/mail.err says: >>> Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Panic: file >>> fts-search-serialize.c: line 63 (fts_search_deserialize_add_idx): >>> assertion failed: (*idx < buf->used) >>> Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Error: Raw >>> backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x6c15f) >[0x7f5110f5e15f] >>> -> /usr/lib/dovecot/libdovecot.so.0(+0x6c1be) [0x7f5110f5e1be] -> >>> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f5110f1740e] -> >>> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xc847) >[0x7f510fb07847] >>> -> >>> >/usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_search_deserialize_add_nonmatches+0x1c) >>> [0x7f510fb0799c] -> >>> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd263) >[0x7f510fb08263] >>> -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd3e6) >>> [0x7f510fb083e6] -> >/usr/lib/dovecot/libdovecot-storage.so.0(+0xa812e) >>> [0x7f511127112e] -> >>> >/usr/lib/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblock+0x4b) >>> [0x7f51112717bb] -> >>> >/usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_search_next_nonblock+0x13e) >>> [0x7f510f8f2a7e] -> >>> >/usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_search_next_nonblock+0x7e) >>> [0x7f510f8f29be] -> >>> >/usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x27) >>> [0x7f511124b277] -> dovecot/imap(+0x1f27f) [0x7f511172527f] -> >>> dovecot/imap(imap_search_start+0xfd) [0x7f51117256fd] -> >>> dovecot/imap(cmd_sort+0x205) [0x7f511171b6c5] -> >>> dovecot/imap(command_exec+0x3c) [0x7f511171fc4c] -> >>> dovecot/imap(+0x18c30) [0x7f511171ec30] -> dovecot/imap(+0x18cea) >>> [0x7f511171ecea] -> dovecot/imap(client_handle_input+0x115) >>> [0x7f511171efb5] -> dovecot/imap(client_input+0x75) [0x7f511171f385] >>> -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4e) >>> [0x7f5110f6efbe] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) >>> [0x7f5110f6ffb7] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) >>> [0x7f5110f6f049] -> >/usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) >>> [0x7f5110f6f0c8] -> >>> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) >>> [0x7f5110f1c7b3] -> dovecot/imap(main+0x2ae) [0x7f511171352e] -> >>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) >>> [0x7f5110b85ead] -> dovecot/imap(+0xd69d) [0x7f511171369d] >>> Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Fatal: >>> master: service(imap): child 6594 killed with signal 6 (core dumps >>> disabled) >>> Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Panic: file >>> fts-search-serialize.c: line 63 (fts_search_deserialize_add_idx): >>> assertion failed: (*idx < buf->used) >>> Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Error: Raw >>> backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x6c15f) >[0x7fae1c14c15f] >>> -> /usr/lib/dovecot/libdovecot.so.0(+0x6c1be) [0x7fae1c14c1be] -> >>> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fae1c10540e] -> >>> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xc847) >[0x7fae1acf5847] >>> -> >>> >/usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_search_deserialize_add_nonmatches+0x1c) >>> [0x7fae1acf599c] -> >>> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd263) >[0x7fae1acf6263] >>> -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd3e6) >>> [0x7fae1acf63e6] -> >/usr/lib/dovecot/libdovecot-storage.so.0(+0xa812e) >>> [0x7fae1c45f12e] -> >>> >/usr/lib/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblock+0x4b) >>> [0x7fae1c45f7bb] -> >>> >/usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_search_next_nonblock+0x13e) >>> [0x7fae1aae0a7e] -> >>> >/usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_search_next_nonblock+0x7e) >>> [0x7fae1aae09be] -> >>> >/usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x27) >>> [0x7fae1c439277] -> dovecot/imap(+0x1f27f) [0x7fae1c91327f] -> >>> dovecot/imap(imap_search_start+0xfd) [0x7fae1c9136fd] -> >>> dovecot/imap(cmd_sort+0x205) [0x7fae1c9096c5] -> >>> dovecot/imap(command_exec+0x3c) [0x7fae1c90dc4c] -> >>> dovecot/imap(+0x18c30) [0x7fae1c90cc30] -> dovecot/imap(+0x18cea) >>> [0x7fae1c90ccea] -> dovecot/imap(client_handle_input+0x115) >>> [0x7fae1c90cfb5] -> dovecot/imap(client_input+0x75) [0x7fae1c90d385] >>> -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4e) >>> [0x7fae1c15cfbe] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) >>> [0x7fae1c15dfb7] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) >>> [0x7fae1c15d049] -> >/usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) >>> [0x7fae1c15d0c8] -> >>> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) >>> [0x7fae1c10a7b3] -> dovecot/imap(main+0x2ae) [0x7fae1c90152e] -> >>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) >>> [0x7fae1bd73ead] -> dovecot/imap(+0xd69d) [0x7fae1c90169d] >>> Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): Fatal: >>> master: service(imap): child 6598 killed with signal 6 (core dumps >>> disabled) >>> >>> I don't even have any fts-search-serialize.c on the system... >>> >>> Maybe someone is able to help me? >>> I know we're nearly there, but i can't find anything on this >anywhere. >>> If there's anything else you need to know in order to help, please >let >>> me know. >>> Thank you very very much! >>> Sascha >>> >>> >>> _this is set under /opt/open-xchange/etc/findbasic.properties for >>> Open-Xchange__:_ >>> >>> com.openexchange.find.basic.mail.allMessagesFolder = virtual.all >>> >>> >>> >>> _grep -v '^ *\(#.*\)\?$' /etc/dovecot/dovecot-sql.conf:_ >>> >>> driver = mysql >>> connect = host=localhost dbname=dbispconfig user=ispconfig >>> password=123456789123456789123456789 >>> default_pass_scheme = CRYPT >>> password_query = SELECT password FROM mail_user WHERE (login = '%u' >OR >>> email = '%u') AND disable%Ls = 'n' >>> user_query = SELECT email as user, maildir as home, >CONCAT('maildir:', >>> maildir, '/Maildir') as mail, uid, gid, CONCAT('*:storage=', quota, >>> 'B') AS quota_rule, CONCAT(maildir, '/.sieve') as sieve FROM >mail_user >>> WHERE (login = '%u' OR email = '%u') AND `disable%Ls` = 'n' >>> >>> >>> _||dovecot --version: >>> >>> _2.2.13 >>> >>> >>> _dovecot -n: >>> >>> _# 2.2.13: /etc/dovecot/dovecot.conf >>> # OS: Linux 2.6.32-37-pve x86_64 Debian 7.8 simfs >>> auth_mechanisms = plain login >>> disable_plaintext_auth = no >>> listen = *,[::] >>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>> mail_location = maildir:/var/vmail/%d/%n/Maildir >>> mail_max_userip_connections = 150 >>> mail_plugins = " acl fts fts_solr virtual" >>> mail_privileged_group = vmail >>> managesieve_notify_capability = mailto >>> managesieve_sieve_capability = fileinto reject envelope >>> encoded-character vacation subaddress comparator-i;ascii-numeric >>> relational regex imap4flags copy include variables body enotify >>> environment mailbox date ihave >>> namespace { >>> inbox = yes >>> location = >>> prefix = >>> separator = . >>> type = private >>> } >>> namespace { >>> list = children >>> location = >>> >maildir:/var/vmail/%%d/%%n/Maildir:INDEX=/var/vmail/%d/%n/Maildir/shared/%%u >>> >>> prefix = Shared/.%%n/. >>> separator = . >>> subscriptions = no >>> type = shared >>> } >>> namespace virtual { >>> location = virtual:/etc/dovecot/virtual:INDEX=~/virtual >>> mailbox all { >>> special_use = \All >>> } >>> prefix = virtual. >>> separator = . >>> } >>> passdb { >>> args = /etc/dovecot/dovecot-sql.conf >>> driver = sql >>> } >>> plugin { >>> acl = vfile >>> acl_shared_dict = file:/var/vmail/shared-mailboxes.db >>> fts = solr >>> fts_autoindex = yes >>> fts_solr = url=http://localhost:8090/solr/ >>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >>> sieve = /var/vmail/%d/%n/.sieve >>> } >>> protocols = imap pop3 sieve >>> service auth { >>> unix_listener /var/spool/postfix/private/auth { >>> group = postfix >>> mode = 0660 >>> user = postfix >>> } >>> unix_listener auth-userdb { >>> group = vmail >>> mode = 0600 >>> user = vmail >>> } >>> user = root >>> } >>> service imap-login { >>> client_limit = 1000 >>> process_limit = 500 >>> } >>> ssl_ca = >>> ssl_cert = >> ssl_key = >> ssl_protocols = !SSLv2 !SSLv3 >>> userdb { >>> args = /etc/dovecot/dovecot-sql.conf >>> driver = sql >>> } >>> protocol imap { >>> mail_plugins = " acl fts fts_solr virtual imap_acl quota >imap_quota" >>> } >>> protocol pop3 { >>> mail_plugins = quota virtual >>> pop3_uidl_format = %08Xu%08Xv >>> } >>> protocol lda { >>> mail_plugins = sieve quota virtual >>> } > > > >Best Regards >MfG Robert Schetterer > >-- >[*] sys4 AG > >http://sys4.de, +49 (89) 30 90 46 64 >Franziskanerstra?e 15, 81669 M?nchen > >Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 >Vorstand: Patrick Ben Koetter, Marc Schiffbauer >Aufsichtsratsvorsitzender: Florian Kirstein -- Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet. From zucca at systemschmiede.com Sat Mar 28 13:24:11 2015 From: zucca at systemschmiede.com (zucca at systemschmiede.com) Date: Sat, 28 Mar 2015 14:24:11 +0100 Subject: Error after setting up fts /solr for Open-Xchange In-Reply-To: <93872201-0FC3-4BE4-9F25-918BBBF6500D@systemschmiede.com> References: <550F03FD.9080104@systemschmiede.com> <5516777F.3020606@systemschmiede.com> <55168095.4010807@sys4.de> <93872201-0FC3-4BE4-9F25-918BBBF6500D@systemschmiede.com> Message-ID: <5516AB7B.4020605@systemschmiede.com> Hi, just saw, that v2.2.13 is still the latest version in wheezy backports. I don't really like the idea of messing around with self compiled versions, at least not, if my original problem might as well have a good possibility not to be resolved by the update. I can't find anything regarding the " file fts-search-serialize.c: line 63 (fts_search_deserialize_add_idx): assertion failed: (*idx < buf->used)" Error in the release notes for v2.2.16, so presumably won't have any effect on it, what do you think? Still worth it? *Janette Kelm* jkelm at inovex.de /Wed Aug 20 15:44:34 UTC 2014/ seems to have posted exactly the same problem to this list, and also stated: At google I found the patchhttp://hg.dovecot.org/dovecot-2.2/rev/d63b209737be, but this is already included in dovecot2.2.13, so it is not the reason. But there is also still no solution to her quite old request. Maybe i am just doing something wrong? Did you manage to solve this in the meantime, Janette? looking forward hearing from you thanks Sascha Am 28.03.2015 um 12:22 schrieb Sascha Zucca: > Allright... will try. > Thanks > > Am 28. M?rz 2015 11:21:09 MEZ, schrieb Robert Schetterer : > > Am 28.03.2015 um 10:42 schrieb zucca at systemschmiede.com: > > Hi everyone, may i jolt this again please...still found no > solution at all to this really. Thank you very much, i > appreciate your help! Cheers Sascha > > > > a good idea might be upgrade to recent, before do more debug > > > Timo Sirainen [Thu, 12 Mar 2015 17:41:05 +0200] rev 18340 > Released v2.2.16 > > lots of fixes with i.e fts > > http://hg.dovecot.org/dovecot-2.2/log?rev=fts > > your version > Timo Sirainen [Sun, 11 May 2014 22:33:08 +0300] rev 17372 > Released v2.2.13. > > Am 22.03.2015 um 19:03 schrieb zucca at systemschmiede.com: > > Hi List, i have tried to get fts / solr running with > virtual all folder with Open-Xchange. I followed a pretty > well written howto from Open-Xchanges Intranet, and i > first thought everything was running smooth, as the first > one or two searches were successful. However now whenever > i try to search again for something in the virtual "all > folders" Open-Xchange tells me: *Error* *Folder > "virtual.all" has been closed on mail server > xxx.xxx.xxx.xxx Probably your request took too long.** * > /var/log/mail.err says: Mar 22 18:54:24 mailserver > dovecot: imap(zucca at foo.bar): Panic: file > fts-search-serialize.c: line 63 > (fts_search_deserialize_add_idx): assertion failed: (*idx > < buf->used) Mar 22 18:54:24 mailserver dovecot: > imap(zucca at foo.bar): Error: Raw backtrace: > /usr/lib/dovecot/libdovecot.so > .0(+0x6c15f) [0x7f5110f5e15f] -> > /usr/lib/dovecot/libdovecot.so > .0(+0x6c1be) [0x7f5110f5e1be] -> > /usr/lib/dovecot/libdovecot.so > .0(i_fatal+0) [0x7f5110f1740e] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so > (+0xc847) [0x7f510fb07847] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so > (fts_search_deserialize_add_nonmatches+0x1c) > [0x7f510fb0799c] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so > (+0xd263) [0x7f510fb08263] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so > (+0xd3e6) [0x7f510fb083e6] -> > /usr/lib/dovecot/libdovecot-storage.so > .0(+0xa812e) > [0x7f511127112e] -> /usr/lib/dovecot/libdovecot-storage.so > .0(index_storage_search_next_nonblock+0x4b) > [0x7f51112717bb] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so > (virtual_search_next_nonblock+0x13e) > [0x7f510f8f2a7e] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so > (virtual_search_next_nonblock+0x7e) > [0x7f510f8f29be] -> /usr/lib/dovecot/libdovecot-storage.so > .0(mailbox_search_next_nonblock+0x27) > [0x7f511124b277] -> dovecot/imap(+0x1f27f) > [0x7f511172527f] -> dovecot/imap(imap_search_start+0xfd) > [0x7f51117256fd] -> dovecot/imap(cmd_sort+0x205) > [0x7f511171b6c5] -> dovecot/imap(command_exec+0x3c) > [0x7f511171fc4c] -> dovecot/imap(+0x18c30) > [0x7f511171ec30] -> dovecot/imap(+0x18cea) > [0x7f511171ecea] -> > dovecot/imap(client_handle_input+0x115) [0x7f511171efb5] > -> dovecot/imap(client_input+0x75) [0x7f511171f385] -> > /usr/lib/dovecot/libdovecot.so > .0(io_loop_call_io+0x4e) > [0x7f5110f6efbe] -> /usr/lib/dovecot/libdovecot.so > .0(io_loop_handler_run_internal+0xd7) [0x7f5110f6ffb7] > -> /usr/lib/dovecot/libdovecot.so > .0(io_loop_handler_run+0x9) > [0x7f5110f6f049] -> /usr/lib/dovecot/libdovecot.so > .0(io_loop_run+0x38) > [0x7f5110f6f0c8] -> /usr/lib/dovecot/libdovecot.so > .0(master_service_run+0x13) > [0x7f5110f1c7b3] -> dovecot/imap(main+0x2ae) > [0x7f511171352e] -> /lib/x86_64-linux-gnu/libc.so > .6(__libc_start_main+0xfd) > [0x7f5110b85ead] -> dovecot/imap(+0xd69d) [0x7f511171369d] > Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): > Fatal: master: service(imap): child 6594 killed with > signal 6 (core dumps disabled) Mar 22 18:54:24 mailserver > dovecot: imap(zucca at foo.bar): Panic: file > fts-search-serialize.c: line 63 > (fts_search_deserialize_add_idx): assertion failed: (*idx > < buf->used) Mar 22 18:54:24 mailserver dovecot: > imap(zucca at foo.bar): Error: Raw backtrace: > /usr/lib/dovecot/libdovecot.so > .0(+0x6c15f) [0x7fae1c14c15f] -> > /usr/lib/dovecot/libdovecot.so > .0(+0x6c1be) [0x7fae1c14c1be] -> > /usr/lib/dovecot/libdovecot.so > .0(i_fatal+0) [0x7fae1c10540e] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so > (+0xc847) [0x7fae1acf5847] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so > (fts_search_deserialize_add_nonmatches+0x1c) > [0x7fae1acf599c] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so > (+0xd263) [0x7fae1acf6263] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so > (+0xd3e6) [0x7fae1acf63e6] -> > /usr/lib/dovecot/libdovecot-storage.so > .0(+0xa812e) > [0x7fae1c45f12e] -> /usr/lib/dovecot/libdovecot-storage.so > .0(index_storage_search_next_nonblock+0x4b) > [0x7fae1c45f7bb] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so > (virtual_search_next_nonblock+0x13e) > [0x7fae1aae0a7e] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so > (virtual_search_next_nonblock+0x7e) > [0x7fae1aae09be] -> /usr/lib/dovecot/libdovecot-storage.so > .0(mailbox_search_next_nonblock+0x27) > [0x7fae1c439277] -> dovecot/imap(+0x1f27f) > [0x7fae1c91327f] -> dovecot/imap(imap_search_start+0xfd) > [0x7fae1c9136fd] -> dovecot/imap(cmd_sort+0x205) > [0x7fae1c9096c5] -> dovecot/imap(command_exec+0x3c) > [0x7fae1c90dc4c] -> dovecot/imap(+0x18c30) > [0x7fae1c90cc30] -> dovecot/imap(+0x18cea) > [0x7fae1c90ccea] -> > dovecot/imap(client_handle_input+0x115) [0x7fae1c90cfb5] > -> dovecot/imap(client_input+0x75) [0x7fae1c90d385] -> > /usr/lib/dovecot/libdovecot.so > .0(io_loop_call_io+0x4e) > [0x7fae1c15cfbe] -> /usr/lib/dovecot/libdovecot.so > .0(io_loop_handler_run_internal+0xd7) [0x7fae1c15dfb7] > -> /usr/lib/dovecot/libdovecot.so > .0(io_loop_handler_run+0x9) > [0x7fae1c15d049] -> /usr/lib/dovecot/libdovecot.so > .0(io_loop_run+0x38) > [0x7fae1c15d0c8] -> /usr/lib/dovecot/libdovecot.so > .0(master_service_run+0x13) > [0x7fae1c10a7b3] -> dovecot/imap(main+0x2ae) > [0x7fae1c90152e] -> /lib/x86_64-linux-gnu/libc.so > .6(__libc_start_main+0xfd) > [0x7fae1bd73ead] -> dovecot/imap(+0xd69d) [0x7fae1c90169d] > Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): > Fatal: master: service(imap): child 6598 killed with > signal 6 (core dumps disabled) I don't even have any > fts-search-serialize.c on the system... Maybe someone is > able to help me? I know we're nearly there, but i can't > find anything on this anywhere. If there's anything else > you need to know in order to help, please let me know. > Thank you very very much! Sascha _this is set under > /opt/open-xchange/etc/findbasic.properties for > Open-Xchange__:_ > com.openexchange.find.basic.mail.allMessagesFolder = > virtual.all _grep -v '^ *\(#.*\)\?$' > /etc/dovecot/dovecot-sql.conf:_ driver = mysql connect = > host=localhost dbname=dbispconfig user=ispconfig > password=123456789123456789123456789 default_pass_scheme = > CRYPT password_query = SELECT password FROM mail_user > WHERE (login = '%u' OR email = '%u') AND disable%Ls = 'n' > user_query = SELECT email as user, maildir as home, > CONCAT('maildir:', maildir, '/Maildir') as mail, uid, gid, > CONCAT('*:storage=', quota, 'B') AS quota_rule, > CONCAT(maildir, '/.sieve') as sieve FROM mail_user WHERE > (login = '%u' OR email = '%u') AND `disable%Ls` = 'n' > _||dovecot --version: _2.2.13 _dovecot -n: _# 2.2.13: > /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-37-pve x86_64 > Debian 7.8 simfs auth_mechanisms = plain login > disable_plaintext_auth = no listen = *,[::] log_timestamp > = "%Y-%m-%d %H:%M:%S " mail_location = > maildir:/var/vmail/%d/%n/Maildir > mail_max_userip_connections = 150 mail_plugins = " acl fts > fts_solr virtual" mail_privileged_group = vmail > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress > comparator-i;ascii-numeric relational regex imap4flags > copy include variables body enotify environment mailbox > date ihave namespace { inbox = yes location = prefix = > separator = . type = private } namespace { list = children > location = > maildir:/var/vmail/%%d/%%n/Maildir:INDEX=/var/vmail/%d/%n/Maildir/shared/%%u > prefix = Shared/.%%n/. separator = . subscriptions = no > type = shared } namespace virtual { location = > virtual:/etc/dovecot/virtual:INDEX=~/virtual mailbox all { > special_use = \All } prefix = virtual. separator = . } > passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql > } plugin { acl = vfile acl_shared_dict = > file:/var/vmail/shared-mailboxes.db fts = solr > fts_autoindex = yes fts_solr = > url=http://localhost:8090/solr/ quota = > dict:user::file:/var/vmail/%d/%n/.quotausage sieve = > /var/vmail/%d/%n/.sieve } protocols = imap pop3 sieve > service auth { unix_listener > /var/spool/postfix/private/auth { group = postfix mode = > 0660 user = postfix } unix_listener auth-userdb { group = > vmail mode = 0600 user = vmail } user = root } service > imap-login { client_limit = 1000 process_limit = 500 } > ssl_ca = > -bundle ssl_cert = ssl_key = !SSLv3 userdb { args = /etc/dovecot/dovecot-sql.conf > driver = sql } protocol imap { mail_plugins = " acl fts > fts_solr virtual imap_acl quota imap_quota" } protocol > pop3 { mail_plugins = quota virtual pop3_uidl_format = > %08Xu%08Xv } protocol lda { mail_plugins = sieve quota > virtual } > > > > > Best Regards > MfG Robert Schetterer > > > -- > Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail > gesendet. From rs at sys4.de Sat Mar 28 14:39:45 2015 From: rs at sys4.de (Robert Schetterer) Date: Sat, 28 Mar 2015 15:39:45 +0100 Subject: Error after setting up fts /solr for Open-Xchange In-Reply-To: <5516AB7B.4020605@systemschmiede.com> References: <550F03FD.9080104@systemschmiede.com> <5516777F.3020606@systemschmiede.com> <55168095.4010807@sys4.de> <93872201-0FC3-4BE4-9F25-918BBBF6500D@systemschmiede.com> <5516AB7B.4020605@systemschmiede.com> Message-ID: <5516BD31.7010606@sys4.de> Am 28.03.2015 um 14:24 schrieb zucca at systemschmiede.com: > Hi, > just saw, that v2.2.13 is still the latest version in wheezy backports. use http://wiki.dovecot.org/PrebuiltBinaries#Automatically_Built_Packages > I don't really like the idea of messing around with self compiled > versions, at least not, if my original problem might as well have a > good possibility not to be resolved by the update. > I can't find anything regarding the " file > fts-search-serialize.c: line 63 (fts_search_deserialize_add_idx): > assertion failed: (*idx < buf->used)" > Error in the release notes for v2.2.16, so presumably won't have any > effect on it, what do you think? Still worth it? > > *Janette Kelm* jkelm at inovex.de > > > /Wed Aug 20 15:44:34 UTC 2014/ > seems to have posted exactly the same problem to this list, and also > stated: > > At google > I found the patchhttp://hg.dovecot.org/dovecot-2.2/rev/d63b209737be, > but this is already included in dovecot2.2.13, so it is not the reason. > > But there is also still no solution to her quite old request. > Maybe i am just doing something wrong? > Did you manage to solve this in the meantime, Janette? > > looking forward hearing from you > thanks > Sascha > > > > > Am 28.03.2015 um 12:22 schrieb Sascha Zucca: >> Allright... will try. >> Thanks >> >> Am 28. M?rz 2015 11:21:09 MEZ, schrieb Robert Schetterer : >> >> Am 28.03.2015 um 10:42 schrieb zucca at systemschmiede.com: >> >> Hi everyone, may i jolt this again please...still found no >> solution at all to this really. Thank you very much, i >> appreciate your help! Cheers Sascha >> >> >> a good idea might be upgrade to recent, before do more debug >> >> >> Timo Sirainen [Thu, 12 Mar 2015 17:41:05 +0200] rev >> 18340 >> Released v2.2.16 >> >> lots of fixes with i.e fts >> >> http://hg.dovecot.org/dovecot-2.2/log?rev=fts >> >> your version >> Timo Sirainen [Sun, 11 May 2014 22:33:08 +0300] rev >> 17372 >> Released v2.2.13. >> >> Am 22.03.2015 um 19:03 schrieb zucca at systemschmiede.com: >> >> Hi List, i have tried to get fts / solr running with >> virtual all folder with Open-Xchange. I followed a pretty >> well written howto from Open-Xchanges Intranet, and i >> first thought everything was running smooth, as the first >> one or two searches were successful. However now whenever >> i try to search again for something in the virtual "all >> folders" Open-Xchange tells me: *Error* *Folder >> "virtual.all" has been closed on mail server >> xxx.xxx.xxx.xxx Probably your request took too long.** * >> /var/log/mail.err says: Mar 22 18:54:24 mailserver >> dovecot: imap(zucca at foo.bar): Panic: file >> fts-search-serialize.c: line 63 >> (fts_search_deserialize_add_idx): assertion failed: (*idx >> < buf->used) Mar 22 18:54:24 mailserver dovecot: >> imap(zucca at foo.bar): Error: Raw backtrace: >> /usr/lib/dovecot/libdovecot.so >> .0(+0x6c15f) [0x7f5110f5e15f] -> >> /usr/lib/dovecot/libdovecot.so >> .0(+0x6c1be) [0x7f5110f5e1be] -> >> /usr/lib/dovecot/libdovecot.so >> .0(i_fatal+0) [0x7f5110f1740e] -> >> /usr/lib/dovecot/modules/lib20_fts_plugin.so >> (+0xc847) [0x7f510fb07847] -> >> /usr/lib/dovecot/modules/lib20_fts_plugin.so >> >> (fts_search_deserialize_add_nonmatches+0x1c) >> [0x7f510fb0799c] -> >> /usr/lib/dovecot/modules/lib20_fts_plugin.so >> (+0xd263) [0x7f510fb08263] -> >> /usr/lib/dovecot/modules/lib20_fts_plugin.so >> (+0xd3e6) [0x7f510fb083e6] -> >> /usr/lib/dovecot/libdovecot-storage.so >> .0(+0xa812e) >> [0x7f511127112e] -> /usr/lib/dovecot/libdovecot-storage.so >> >> .0(index_storage_search_next_nonblock+0x4b) >> [0x7f51112717bb] -> >> /usr/lib/dovecot/modules/lib20_virtual_plugin.so >> (virtual_search_next_nonblock+0x13e) >> [0x7f510f8f2a7e] -> >> /usr/lib/dovecot/modules/lib20_virtual_plugin.so >> (virtual_search_next_nonblock+0x7e) >> [0x7f510f8f29be] -> /usr/lib/dovecot/libdovecot-storage.so >> >> .0(mailbox_search_next_nonblock+0x27) >> [0x7f511124b277] -> dovecot/imap(+0x1f27f) >> [0x7f511172527f] -> dovecot/imap(imap_search_start+0xfd) >> [0x7f51117256fd] -> dovecot/imap(cmd_sort+0x205) >> [0x7f511171b6c5] -> dovecot/imap(command_exec+0x3c) >> [0x7f511171fc4c] -> dovecot/imap(+0x18c30) >> [0x7f511171ec30] -> dovecot/imap(+0x18cea) >> [0x7f511171ecea] -> >> dovecot/imap(client_handle_input+0x115) [0x7f511171efb5] >> -> dovecot/imap(client_input+0x75) [0x7f511171f385] -> >> /usr/lib/dovecot/libdovecot.so >> .0(io_loop_call_io+0x4e) >> [0x7f5110f6efbe] -> /usr/lib/dovecot/libdovecot.so >> >> .0(io_loop_handler_run_internal+0xd7) >> [0x7f5110f6ffb7] >> -> /usr/lib/dovecot/libdovecot.so >> .0(io_loop_handler_run+0x9) >> [0x7f5110f6f049] -> /usr/lib/dovecot/libdovecot.so >> .0(io_loop_run+0x38) >> [0x7f5110f6f0c8] -> /usr/lib/dovecot/libdovecot.so >> .0(master_service_run+0x13) >> [0x7f5110f1c7b3] -> dovecot/imap(main+0x2ae) >> [0x7f511171352e] -> /lib/x86_64-linux-gnu/libc.so >> .6(__libc_start_main+0xfd) >> [0x7f5110b85ead] -> dovecot/imap(+0xd69d) [0x7f511171369d] >> Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): >> Fatal: master: service(imap): child 6594 killed with >> signal 6 (core dumps disabled) Mar 22 18:54:24 mailserver >> dovecot: imap(zucca at foo.bar): Panic: file >> fts-search-serialize.c: line 63 >> (fts_search_deserialize_add_idx): assertion failed: (*idx >> < buf->used) Mar 22 18:54:24 mailserver dovecot: >> imap(zucca at foo.bar): Error: Raw backtrace: >> /usr/lib/dovecot/libdovecot.so >> .0(+0x6c15f) [0x7fae1c14c15f] -> >> /usr/lib/dovecot/libdovecot.so >> .0(+0x6c1be) [0x7fae1c14c1be] -> >> /usr/lib/dovecot/libdovecot.so >> .0(i_fatal+0) [0x7fae1c10540e] -> >> /usr/lib/dovecot/modules/lib20_fts_plugin.so >> (+0xc847) [0x7fae1acf5847] -> >> /usr/lib/dovecot/modules/lib20_fts_plugin.so >> >> (fts_search_deserialize_add_nonmatches+0x1c) >> [0x7fae1acf599c] -> >> /usr/lib/dovecot/modules/lib20_fts_plugin.so >> (+0xd263) [0x7fae1acf6263] -> >> /usr/lib/dovecot/modules/lib20_fts_plugin.so >> (+0xd3e6) [0x7fae1acf63e6] -> >> /usr/lib/dovecot/libdovecot-storage.so >> .0(+0xa812e) >> [0x7fae1c45f12e] -> /usr/lib/dovecot/libdovecot-storage.so >> >> .0(index_storage_search_next_nonblock+0x4b) >> [0x7fae1c45f7bb] -> >> /usr/lib/dovecot/modules/lib20_virtual_plugin.so >> (virtual_search_next_nonblock+0x13e) >> [0x7fae1aae0a7e] -> >> /usr/lib/dovecot/modules/lib20_virtual_plugin.so >> (virtual_search_next_nonblock+0x7e) >> [0x7fae1aae09be] -> /usr/lib/dovecot/libdovecot-storage.so >> >> .0(mailbox_search_next_nonblock+0x27) >> [0x7fae1c439277] -> dovecot/imap(+0x1f27f) >> [0x7fae1c91327f] -> dovecot/imap(imap_search_start+0xfd) >> [0x7fae1c9136fd] -> dovecot/imap(cmd_sort+0x205) >> [0x7fae1c9096c5] -> dovecot/imap(command_exec+0x3c) >> [0x7fae1c90dc4c] -> dovecot/imap(+0x18c30) >> [0x7fae1c90cc30] -> dovecot/imap(+0x18cea) >> [0x7fae1c90ccea] -> >> dovecot/imap(client_handle_input+0x115) [0x7fae1c90cfb5] >> -> dovecot/imap(client_input+0x75) [0x7fae1c90d385] -> >> /usr/lib/dovecot/libdovecot.so >> .0(io_loop_call_io+0x4e) >> [0x7fae1c15cfbe] -> /usr/lib/dovecot/libdovecot.so >> >> .0(io_loop_handler_run_internal+0xd7) >> [0x7fae1c15dfb7] >> -> /usr/lib/dovecot/libdovecot.so >> .0(io_loop_handler_run+0x9) >> [0x7fae1c15d049] -> /usr/lib/dovecot/libdovecot.so >> .0(io_loop_run+0x38) >> [0x7fae1c15d0c8] -> /usr/lib/dovecot/libdovecot.so >> .0(master_service_run+0x13) >> [0x7fae1c10a7b3] -> dovecot/imap(main+0x2ae) >> [0x7fae1c90152e] -> /lib/x86_64-linux-gnu/libc.so >> .6(__libc_start_main+0xfd) >> [0x7fae1bd73ead] -> dovecot/imap(+0xd69d) [0x7fae1c90169d] >> Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): >> Fatal: master: service(imap): child 6598 killed with >> signal 6 (core dumps disabled) I don't even have any >> fts-search-serialize.c on the system... Maybe someone is >> able to help me? I know we're nearly there, but i can't >> find anything on this anywhere. If there's anything else >> you need to know in order to help, please let me know. >> Thank you very very much! Sascha _this is set under >> /opt/open-xchange/etc/findbasic.properties for >> Open-Xchange__:_ >> com.openexchange.find.basic.mail.allMessagesFolder = >> virtual.all _grep -v '^ *\(#.*\)\?$' >> /etc/dovecot/dovecot-sql.conf:_ driver = mysql connect = >> host=localhost dbname=dbispconfig user=ispconfig >> password=123456789123456789123456789 default_pass_scheme = >> CRYPT password_query = SELECT password FROM mail_user >> WHERE (login = '%u' OR email = '%u') AND disable%Ls = 'n' >> user_query = SELECT email as user, maildir as home, >> CONCAT('maildir:', maildir, '/Maildir') as mail, uid, gid, >> CONCAT('*:storage=', quota, 'B') AS quota_rule, >> CONCAT(maildir, '/.sieve') as sieve FROM mail_user WHERE >> (login = '%u' OR email = '%u') AND `disable%Ls` = 'n' >> _||dovecot --version: _2.2.13 _dovecot -n: _# 2.2.13: >> /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-37-pve x86_64 >> Debian 7.8 simfs auth_mechanisms = plain login >> disable_plaintext_auth = no listen = *,[::] log_timestamp >> = "%Y-%m-%d %H:%M:%S " mail_location = >> maildir:/var/vmail/%d/%n/Maildir >> mail_max_userip_connections = 150 mail_plugins = " acl fts >> fts_solr virtual" mail_privileged_group = vmail >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress >> comparator-i;ascii-numeric relational regex imap4flags >> copy include variables body enotify environment mailbox >> date ihave namespace { inbox = yes location = prefix = >> separator = . type = private } namespace { list = children >> location = >> >> maildir:/var/vmail/%%d/%%n/Maildir:INDEX=/var/vmail/%d/%n/Maildir/shared/%%u >> >> prefix = Shared/.%%n/. separator = . subscriptions = no >> type = shared } namespace virtual { location = >> virtual:/etc/dovecot/virtual:INDEX=~/virtual mailbox all { >> special_use = \All } prefix = virtual. separator = . } >> passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql >> } plugin { acl = vfile acl_shared_dict = >> file:/var/vmail/shared-mailboxes.db fts = solr >> fts_autoindex = yes fts_solr = >> url=http://localhost:8090/solr/ quota = >> dict:user::file:/var/vmail/%d/%n/.quotausage sieve = >> /var/vmail/%d/%n/.sieve } protocols = imap pop3 sieve >> service auth { unix_listener >> /var/spool/postfix/private/auth { group = postfix mode = >> 0660 user = postfix } unix_listener auth-userdb { group = >> vmail mode = 0600 user = vmail } user = root } service >> imap-login { client_limit = 1000 process_limit = 500 } >> ssl_ca = >> > -bundle ssl_cert = > ssl_key = > !SSLv3 userdb { args = /etc/dovecot/dovecot-sql.conf >> driver = sql } protocol imap { mail_plugins = " acl fts >> fts_solr virtual imap_acl quota imap_quota" } protocol >> pop3 { mail_plugins = quota virtual pop3_uidl_format = >> %08Xu%08Xv } protocol lda { mail_plugins = sieve quota >> virtual } >> >> >> >> Best Regards >> MfG Robert Schetterer >> >> >> -- >> Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail >> gesendet. Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From laurent.rathle at free.fr Sat Mar 28 14:39:53 2015 From: laurent.rathle at free.fr (Laurent Rathle) Date: Sat, 28 Mar 2015 15:39:53 +0100 Subject: Problem connecting to an imap account with Dovecot Message-ID: Hello, I have several accounts on my server. I can?t connect to one of them using Apple mail on Mac or IOS to create my account or read my mail in these applications. I can send mails to this address and read them with my Horde webmail. I have these messages in the log : Mar 28 12:21:49 sd-41648 dovecot: imap-login: Login: user=, method=PLAIN, rip=::1, lip=::1, mpid=13705, TLS, session= Mar 28 12:21:50 sd-41648 dovecot: service=imap, user=laurent at xxxxxx.fr, ip=[::1]. Disconnected: Logged out rcvd=26, sent=435 Mar 28 12:21:50 sd-41648 dovecot: imap-login: Login: user=, method=PLAIN, rip=::1, lip=::1, mpid=13707, TLS, session= Mar 28 12:21:51 sd-41648 dovecot: service=imap, user=laurent at xxxxxx.fr, ip=[::1]. Disconnected: Logged out rcvd=89, sent=1447 Mar 28 12:21:21 sd-41648 postfix/master[31571]: message repeated 4 times: [ warning: master_wakeup_timer_event: service pickup(public/pickup): Connection refused] Mar 28 12:21:52 sd-41648 postfix/master[31571]: warning: master_wakeup_timer_event: service qmgr(public/qmgr): Connection refused Mar 28 12:21:53 sd-41648 dovecot: imap-login: Login: user=, method=PLAIN, rip=::1, lip=::1, mpid=13709, TLS, session= Mar 28 12:21:54 sd-41648 dovecot: service=imap, user=laurent at xxxxxx.fr, ip=[::1]. Disconnected: Logged out rcvd=44, sent=457 Mar 28 12:21:54 sd-41648 dovecot: imap-login: Login: user=, method=PLAIN, rip=::1, lip=::1, mpid=13711, TLS, session=<9vDSdFcSnQAAAAAAAAAAAAAAAAAAAAAB> Mar 28 12:21:56 sd-41648 dovecot: service=imap, user=laurent at xxxxxx.fr, ip=[::1]. Disconnected: Logged out rcvd=860, sent=132669 Mar 28 12:21:57 sd-41648 dovecot: imap-login: Login: user=, method=PLAIN, rip=::1, lip=::1, mpid=13713, TLS, session=<1pvxdFcSpwAAAAAAAAAAAAAAAAAAAAAB> Mar 28 12:21:58 sd-41648 dovecot: service=imap, user=laurent at xxxxxx.fr, ip=[::1]. Disconnected: Logged out rcvd=1266, sent=108893 Mar 28 12:22:12 sd-41648 dovecot: imap-login: Login: user=, method=PLAIN, rip=::1, lip=::1, mpid=13720, TLS, session= Mar 28 12:22:20 sd-41648 dovecot: service=imap, user=laurent at xxxxxx.fr, ip=[::1]. Disconnected: Logged out rcvd=309, sent=11066 Mar 28 12:22:21 sd-41648 postfix/master[31571]: warning: master_wakeup_timer_event: service pickup(public/pickup): Connection refused Mar 28 12:22:23 sd-41648 dovecot: imap-login: Login: user=, method=PLAIN, rip=::1, lip=::1, mpid=13724, TLS, session= Mar 28 12:22:23 sd-41648 dovecot: service=imap, user=laurent at xxxxxx.fr, ip=[::1]. Disconnected: Logged out rcvd=214, sent=3159 I have : Plesk 12 Ubuntu 14.04.2 Dovecot 2.2.12 You can see my configuration file here : http://pastie.org/10059309 What should I do ? Thank you From tom at whyscream.net Sat Mar 28 15:09:32 2015 From: tom at whyscream.net (Tom Hendrikx) Date: Sat, 28 Mar 2015 16:09:32 +0100 Subject: Problem connecting to an imap account with Dovecot In-Reply-To: References: Message-ID: <5516C42C.6010605@whyscream.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 28-03-15 15:39, Laurent Rathle wrote: > Hello, > > I have several accounts on my server. I can?t connect to one of > them using Apple mail on Mac or IOS to create my account or read my > mail in these applications. I can send mails to this address and > read them with my Horde webmail. I have these messages in the log > : > > Mar 28 12:21:49 sd-41648 dovecot: imap-login: Login: > user=, method=PLAIN, rip=::1, lip=::1, > mpid=13705, TLS, session= Login from localhost (rip=::1), probably using horde webmail > Mar 28 12:21:50 sd-41648 dovecot: service=imap, > user=laurent at xxxxxx.fr, ip=[::1]. Disconnected: Logged out rcvd=26, > sent=435 Logout from localhost, same same. rcvd and sent data are larger than zero, seems to work fine. > Mar 28 12:21:21 sd-41648 postfix/master[31571]: message repeated 4 > times: [ warning: master_wakeup_timer_event: service > pickup(public/pickup): Connection refused] Mar 28 12:21:52 sd-41648 > postfix/master[31571]: warning: master_wakeup_timer_event: service > qmgr(public/qmgr): Connection refused Unrelated to dovecot, but your server seems to have other issues too. There are no log lines from dovecot that tell us about connections from remote (i.e. non localhost) hosts. If you can't connect using an external client, maybe your firewall is still closed, or dovecot is only listening on localhost? You should start here: http://wiki.dovecot.org/TestInstallation and tell where you got stuck (and include the details of the steps you managed to complete). > > I have : > > Plesk 12 Ubuntu 14.04.2 Dovecot 2.2.12 > > You can see my configuration file here : > http://pastie.org/10059309 No obvious stuff in there other than the qmail references (despite having postfix running too), but please post `dovecot -n` output directly in your mail, next time. Thank you ;) > > What should I do ? > > Thank you > -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJVFsQsAAoJEJPfMZ19VO/1iPMQAJNUjT3tw8bPZzJ4fBGZprcE gWSP5QeE5ysDoHy7v0Sh01IeZRezfQDAz9bXlowdvVJacjpqpNzTasCuZz0hGL/F PZLS2ESsIAlbQkDdWt7+O2k85jMNtWOEe/iYbnVgRczJr+3LaEvCBia8m/s9iAQg d+gc6OuWKe0CmExXHS8u8L0CoW46MxM509hwD51ToL4KDNByI0EgSymH84hopggr ajk/KEeR79q3VmsYox5xCZ7vvsJeWJ7s1iQjn3I24bBHafWV6jfGlDJ8jGSXQQL8 T8JUElRHGqYhm3fyfMmQhPi1evvn8UASxh2U35Tek4Ea7udL83ZL4KNaXb2VeUsH uV7m7QgoVbE0wUIfmJdTIhgLJi4KXrThkGFgKoKNqBQRYGW6nX/61K1rqNSd34aQ XjcHWZ3mdVXse87HlWpOgUWG0b4kPLWUHenGo2loJP1jweGGTj2GAf9oxk39l1z/ Swp88GX0p6+Hl80btA7QAs/mHfAWdZgSct67PwxEW45hSlX6pI9+RCGt8C3F3drY bEQ/Pl7Aj5I7OD0htNb1xiS+BjlqGPiDYs0Prjo/n3n5vg2LJDCFi+1ThtJ+4QJw gzuM8Or3P1J8iPy1HhqNF/EzMyyhlFgDXlKmLTc9HzWVF59g8w+7jZgsIBFJBSVd 3FhDtoKm5b72D7efcfLU =rRD5 -----END PGP SIGNATURE----- From gedalya at gedalya.net Sat Mar 28 15:16:00 2015 From: gedalya at gedalya.net (Gedalya) Date: Sat, 28 Mar 2015 11:16:00 -0400 Subject: v2.2.16 released In-Reply-To: <550F9247.3060605@gedalya.net> References: <0DC5FD73-B4A7-4811-BD7F-7B053BAF47E8@iki.fi> <550F9247.3060605@gedalya.net> Message-ID: <5516C5B0.7010606@gedalya.net> On 03/23/2015 12:10 AM, Gedalya wrote: > This happened after upgrading to 2.2.16. Running on Debian wheezy amd64. > We never had such a crash on this server before. However, this too was > a single incident. > All accounts use Maildir. > > > Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Error: Log > synchronization error at seq=0,offset=0 for > /stor/mail/domains/----/----/Maildir/dovecot.index: Append with UID > 6684, but next_uid = 6685 > Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Error: > /stor/mail/domains/----/----/Maildir/dovecot.index view syncing failed > to apply changes > Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Panic: file > index-sync.c: line 265 (index_mailbox_sync_next_expunge): assertion > failed: (range->seq2 <= ctx->messages_count) > Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Error: Raw > backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7b57f) [0x7f34cd27c57f] > -> /usr/lib/dovecot/libdovecot.so.0(+0x7b5de) [0x7f34cd27c5de] -> > /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f34cd22b0d5] -> > /usr/lib/dovecot/libdovecot-storage.so.0(+0xb5391) [0x7f34cd5a6391] -> > dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - 2373 bytes > waiting](imap_sync_more+0xce) [0x7f34cda5ad2e] -> dovecot/imap > [---- at ---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting](+0x22344) > [0x7f34cda5b344] -> dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - > 2373 bytes waiting](cmd_sync_delayed+0x1bf) [0x7f34cda5b67f] -> > dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - 2373 bytes > waiting](client_output+0xe0) [0x7f34cda52910] -> > /usr/lib/dovecot/libdovecot.so.0(+0x98a85) [0x7f34cd299a85] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) > [0x7f34cd28e80b] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb) > [0x7f34cd28f88b] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) > [0x7f34cd28e899] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) > [0x7f34cd28e918] -> > /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) > [0x7f34cd2307d3] -> dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - > 2373 bytes waiting](main+0x2b7) [0x7f34cda46997] -> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) > [0x7f34cce94ead] -> dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - > 2373 bytes waiting](+0xdb09) [0x7f34cda46b09] > Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Fatal: master: > service(imap): child 30144 killed with signal 6 (core dumps disabled) > And now it just happened, again, same user! The previous dovecot version before the upgrade was 2.2.12 Mar 28 07:51:17 imap1 dovecot: imap(---- at ----.com): Error: Log synchronization error at seq=0,offset=0 for /stor/mail/domains/----.com/----/Maildir/dovecot.index: Append with UID 12794, but next_uid = 12795 Mar 28 07:51:17 imap1 dovecot: imap(---- at ----.com): Error: /stor/mail/domains/----.com/----/Maildir/dovecot.index view syncing failed to apply changes Mar 28 07:51:17 imap1 dovecot: imap(---- at ----.com): Panic: file index-sync.c: line 265 (index_mailbox_sync_next_expunge): assertion failed: (range->seq2 <= ctx->messages_count) Mar 28 07:51:17 imap1 dovecot: imap(---- at ----.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7b57f) [0x7f136acdc57f] -> /usr/lib/dovecot/libdovecot.so.0(+0x7b5de) [0x7f136acdc5de] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f136ac8b0d5] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb5391) [0x7f136b006391] -> dovecot/imap [---- at ----.com xx.xx.xx.xx UID FETCH](imap_sync_more+0xce) [0x7f136b4bad2e] -> dovecot/imap [---- at ----.com xx.xx.xx.xx UID FETCH](+0x22344) [0x7f136b4bb344] -> dovecot/imap [---- at ----.com xx.xx.xx.xx UID FETCH](cmd_sync_delayed+0x1bf) [0x7f136b4bb67f] -> dovecot/imap [---- at ----.com xx.xx.xx.xx UID FETCH](client_handle_input+0x1d5) [0x7f136b4b2725] -> dovecot/imap [---- at ----.com xx.xx.xx.xx UID FETCH](client_input+0x75) [0x7f136b4b2a35] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7f136acee80b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb) [0x7f136acef88b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f136acee899] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f136acee918] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f136ac907d3] -> dovecot/imap [---- at ----.com xx.xx.xx.xx UID FETCH](main+0x2b7) [0x7f136b4a6997] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f136a8f4ead] -> dovecot/imap [---- at ----.com xx.xx.xx.xx UID FETCH](+0xdb09) [0x7f136b4a6b09] Mar 28 07:51:17 imap1 dovecot: imap(---- at ----.com): Fatal: master: service(imap): child 12444 killed with signal 6 (core dumps disabled) From zucca at systemschmiede.com Sat Mar 28 17:02:38 2015 From: zucca at systemschmiede.com (zucca at systemschmiede.com) Date: Sat, 28 Mar 2015 18:02:38 +0100 Subject: Error after setting up fts /solr for Open-Xchange In-Reply-To: <5516BD31.7010606@sys4.de> References: <550F03FD.9080104@systemschmiede.com> <5516777F.3020606@systemschmiede.com> <55168095.4010807@sys4.de> <93872201-0FC3-4BE4-9F25-918BBBF6500D@systemschmiede.com> <5516AB7B.4020605@systemschmiede.com> <5516BD31.7010606@sys4.de> Message-ID: <5516DEAE.8050906@systemschmiede.com> Hi! Well...That seemed to have worked in fact. Updated to 2:2.2.16-1~auto+36. "All folders"-search works, and no errors are being shown. Besides, the all folder search in Open-Xchange looks great and is lightning fast! Thank you very much for your help Sascha Am 28.03.2015 um 15:39 schrieb Robert Schetterer: > Am 28.03.2015 um 14:24 schrieb zucca at systemschmiede.com: >> Hi, >> just saw, that v2.2.13 is still the latest version in wheezy backports. > use > > http://wiki.dovecot.org/PrebuiltBinaries#Automatically_Built_Packages > > >> I don't really like the idea of messing around with self compiled >> versions, at least not, if my original problem might as well have a >> good possibility not to be resolved by the update. >> I can't find anything regarding the " file >> fts-search-serialize.c: line 63 (fts_search_deserialize_add_idx): >> assertion failed: (*idx < buf->used)" >> Error in the release notes for v2.2.16, so presumably won't have any >> effect on it, what do you think? Still worth it? >> >> *Janette Kelm* jkelm at inovex.de >> >> >> /Wed Aug 20 15:44:34 UTC 2014/ >> seems to have posted exactly the same problem to this list, and also >> stated: >> >> At google >> I found the patchhttp://hg.dovecot.org/dovecot-2.2/rev/d63b209737be, >> but this is already included in dovecot2.2.13, so it is not the reason. >> >> But there is also still no solution to her quite old request. >> Maybe i am just doing something wrong? >> Did you manage to solve this in the meantime, Janette? >> >> looking forward hearing from you >> thanks >> Sascha >> >> >> >> >> Am 28.03.2015 um 12:22 schrieb Sascha Zucca: >>> Allright... will try. >>> Thanks >>> >>> Am 28. M?rz 2015 11:21:09 MEZ, schrieb Robert Schetterer : >>> >>> Am 28.03.2015 um 10:42 schrieb zucca at systemschmiede.com: >>> >>> Hi everyone, may i jolt this again please...still found no >>> solution at all to this really. Thank you very much, i >>> appreciate your help! Cheers Sascha >>> >>> >>> a good idea might be upgrade to recent, before do more debug >>> >>> >>> Timo Sirainen [Thu, 12 Mar 2015 17:41:05 +0200] rev >>> 18340 >>> Released v2.2.16 >>> >>> lots of fixes with i.e fts >>> >>> http://hg.dovecot.org/dovecot-2.2/log?rev=fts >>> >>> your version >>> Timo Sirainen [Sun, 11 May 2014 22:33:08 +0300] rev >>> 17372 >>> Released v2.2.13. >>> >>> Am 22.03.2015 um 19:03 schrieb zucca at systemschmiede.com: >>> >>> Hi List, i have tried to get fts / solr running with >>> virtual all folder with Open-Xchange. I followed a pretty >>> well written howto from Open-Xchanges Intranet, and i >>> first thought everything was running smooth, as the first >>> one or two searches were successful. However now whenever >>> i try to search again for something in the virtual "all >>> folders" Open-Xchange tells me: *Error* *Folder >>> "virtual.all" has been closed on mail server >>> xxx.xxx.xxx.xxx Probably your request took too long.** * >>> /var/log/mail.err says: Mar 22 18:54:24 mailserver >>> dovecot: imap(zucca at foo.bar): Panic: file >>> fts-search-serialize.c: line 63 >>> (fts_search_deserialize_add_idx): assertion failed: (*idx >>> < buf->used) Mar 22 18:54:24 mailserver dovecot: >>> imap(zucca at foo.bar): Error: Raw backtrace: >>> /usr/lib/dovecot/libdovecot.so >>> .0(+0x6c15f) [0x7f5110f5e15f] -> >>> /usr/lib/dovecot/libdovecot.so >>> .0(+0x6c1be) [0x7f5110f5e1be] -> >>> /usr/lib/dovecot/libdovecot.so >>> .0(i_fatal+0) [0x7f5110f1740e] -> >>> /usr/lib/dovecot/modules/lib20_fts_plugin.so >>> (+0xc847) [0x7f510fb07847] -> >>> /usr/lib/dovecot/modules/lib20_fts_plugin.so >>> >>> (fts_search_deserialize_add_nonmatches+0x1c) >>> [0x7f510fb0799c] -> >>> /usr/lib/dovecot/modules/lib20_fts_plugin.so >>> (+0xd263) [0x7f510fb08263] -> >>> /usr/lib/dovecot/modules/lib20_fts_plugin.so >>> (+0xd3e6) [0x7f510fb083e6] -> >>> /usr/lib/dovecot/libdovecot-storage.so >>> .0(+0xa812e) >>> [0x7f511127112e] -> /usr/lib/dovecot/libdovecot-storage.so >>> >>> .0(index_storage_search_next_nonblock+0x4b) >>> [0x7f51112717bb] -> >>> /usr/lib/dovecot/modules/lib20_virtual_plugin.so >>> (virtual_search_next_nonblock+0x13e) >>> [0x7f510f8f2a7e] -> >>> /usr/lib/dovecot/modules/lib20_virtual_plugin.so >>> (virtual_search_next_nonblock+0x7e) >>> [0x7f510f8f29be] -> /usr/lib/dovecot/libdovecot-storage.so >>> >>> .0(mailbox_search_next_nonblock+0x27) >>> [0x7f511124b277] -> dovecot/imap(+0x1f27f) >>> [0x7f511172527f] -> dovecot/imap(imap_search_start+0xfd) >>> [0x7f51117256fd] -> dovecot/imap(cmd_sort+0x205) >>> [0x7f511171b6c5] -> dovecot/imap(command_exec+0x3c) >>> [0x7f511171fc4c] -> dovecot/imap(+0x18c30) >>> [0x7f511171ec30] -> dovecot/imap(+0x18cea) >>> [0x7f511171ecea] -> >>> dovecot/imap(client_handle_input+0x115) [0x7f511171efb5] >>> -> dovecot/imap(client_input+0x75) [0x7f511171f385] -> >>> /usr/lib/dovecot/libdovecot.so >>> .0(io_loop_call_io+0x4e) >>> [0x7f5110f6efbe] -> /usr/lib/dovecot/libdovecot.so >>> >>> .0(io_loop_handler_run_internal+0xd7) >>> [0x7f5110f6ffb7] >>> -> /usr/lib/dovecot/libdovecot.so >>> .0(io_loop_handler_run+0x9) >>> [0x7f5110f6f049] -> /usr/lib/dovecot/libdovecot.so >>> .0(io_loop_run+0x38) >>> [0x7f5110f6f0c8] -> /usr/lib/dovecot/libdovecot.so >>> .0(master_service_run+0x13) >>> [0x7f5110f1c7b3] -> dovecot/imap(main+0x2ae) >>> [0x7f511171352e] -> /lib/x86_64-linux-gnu/libc.so >>> .6(__libc_start_main+0xfd) >>> [0x7f5110b85ead] -> dovecot/imap(+0xd69d) [0x7f511171369d] >>> Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): >>> Fatal: master: service(imap): child 6594 killed with >>> signal 6 (core dumps disabled) Mar 22 18:54:24 mailserver >>> dovecot: imap(zucca at foo.bar): Panic: file >>> fts-search-serialize.c: line 63 >>> (fts_search_deserialize_add_idx): assertion failed: (*idx >>> < buf->used) Mar 22 18:54:24 mailserver dovecot: >>> imap(zucca at foo.bar): Error: Raw backtrace: >>> /usr/lib/dovecot/libdovecot.so >>> .0(+0x6c15f) [0x7fae1c14c15f] -> >>> /usr/lib/dovecot/libdovecot.so >>> .0(+0x6c1be) [0x7fae1c14c1be] -> >>> /usr/lib/dovecot/libdovecot.so >>> .0(i_fatal+0) [0x7fae1c10540e] -> >>> /usr/lib/dovecot/modules/lib20_fts_plugin.so >>> (+0xc847) [0x7fae1acf5847] -> >>> /usr/lib/dovecot/modules/lib20_fts_plugin.so >>> >>> (fts_search_deserialize_add_nonmatches+0x1c) >>> [0x7fae1acf599c] -> >>> /usr/lib/dovecot/modules/lib20_fts_plugin.so >>> (+0xd263) [0x7fae1acf6263] -> >>> /usr/lib/dovecot/modules/lib20_fts_plugin.so >>> (+0xd3e6) [0x7fae1acf63e6] -> >>> /usr/lib/dovecot/libdovecot-storage.so >>> .0(+0xa812e) >>> [0x7fae1c45f12e] -> /usr/lib/dovecot/libdovecot-storage.so >>> >>> .0(index_storage_search_next_nonblock+0x4b) >>> [0x7fae1c45f7bb] -> >>> /usr/lib/dovecot/modules/lib20_virtual_plugin.so >>> (virtual_search_next_nonblock+0x13e) >>> [0x7fae1aae0a7e] -> >>> /usr/lib/dovecot/modules/lib20_virtual_plugin.so >>> (virtual_search_next_nonblock+0x7e) >>> [0x7fae1aae09be] -> /usr/lib/dovecot/libdovecot-storage.so >>> >>> .0(mailbox_search_next_nonblock+0x27) >>> [0x7fae1c439277] -> dovecot/imap(+0x1f27f) >>> [0x7fae1c91327f] -> dovecot/imap(imap_search_start+0xfd) >>> [0x7fae1c9136fd] -> dovecot/imap(cmd_sort+0x205) >>> [0x7fae1c9096c5] -> dovecot/imap(command_exec+0x3c) >>> [0x7fae1c90dc4c] -> dovecot/imap(+0x18c30) >>> [0x7fae1c90cc30] -> dovecot/imap(+0x18cea) >>> [0x7fae1c90ccea] -> >>> dovecot/imap(client_handle_input+0x115) [0x7fae1c90cfb5] >>> -> dovecot/imap(client_input+0x75) [0x7fae1c90d385] -> >>> /usr/lib/dovecot/libdovecot.so >>> .0(io_loop_call_io+0x4e) >>> [0x7fae1c15cfbe] -> /usr/lib/dovecot/libdovecot.so >>> >>> .0(io_loop_handler_run_internal+0xd7) >>> [0x7fae1c15dfb7] >>> -> /usr/lib/dovecot/libdovecot.so >>> .0(io_loop_handler_run+0x9) >>> [0x7fae1c15d049] -> /usr/lib/dovecot/libdovecot.so >>> .0(io_loop_run+0x38) >>> [0x7fae1c15d0c8] -> /usr/lib/dovecot/libdovecot.so >>> .0(master_service_run+0x13) >>> [0x7fae1c10a7b3] -> dovecot/imap(main+0x2ae) >>> [0x7fae1c90152e] -> /lib/x86_64-linux-gnu/libc.so >>> .6(__libc_start_main+0xfd) >>> [0x7fae1bd73ead] -> dovecot/imap(+0xd69d) [0x7fae1c90169d] >>> Mar 22 18:54:24 mailserver dovecot: imap(zucca at foo.bar): >>> Fatal: master: service(imap): child 6598 killed with >>> signal 6 (core dumps disabled) I don't even have any >>> fts-search-serialize.c on the system... Maybe someone is >>> able to help me? I know we're nearly there, but i can't >>> find anything on this anywhere. If there's anything else >>> you need to know in order to help, please let me know. >>> Thank you very very much! Sascha _this is set under >>> /opt/open-xchange/etc/findbasic.properties for >>> Open-Xchange__:_ >>> com.openexchange.find.basic.mail.allMessagesFolder = >>> virtual.all _grep -v '^ *\(#.*\)\?$' >>> /etc/dovecot/dovecot-sql.conf:_ driver = mysql connect = >>> host=localhost dbname=dbispconfig user=ispconfig >>> password=123456789123456789123456789 default_pass_scheme = >>> CRYPT password_query = SELECT password FROM mail_user >>> WHERE (login = '%u' OR email = '%u') AND disable%Ls = 'n' >>> user_query = SELECT email as user, maildir as home, >>> CONCAT('maildir:', maildir, '/Maildir') as mail, uid, gid, >>> CONCAT('*:storage=', quota, 'B') AS quota_rule, >>> CONCAT(maildir, '/.sieve') as sieve FROM mail_user WHERE >>> (login = '%u' OR email = '%u') AND `disable%Ls` = 'n' >>> _||dovecot --version: _2.2.13 _dovecot -n: _# 2.2.13: >>> /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-37-pve x86_64 >>> Debian 7.8 simfs auth_mechanisms = plain login >>> disable_plaintext_auth = no listen = *,[::] log_timestamp >>> = "%Y-%m-%d %H:%M:%S " mail_location = >>> maildir:/var/vmail/%d/%n/Maildir >>> mail_max_userip_connections = 150 mail_plugins = " acl fts >>> fts_solr virtual" mail_privileged_group = vmail >>> managesieve_notify_capability = mailto >>> managesieve_sieve_capability = fileinto reject envelope >>> encoded-character vacation subaddress >>> comparator-i;ascii-numeric relational regex imap4flags >>> copy include variables body enotify environment mailbox >>> date ihave namespace { inbox = yes location = prefix = >>> separator = . type = private } namespace { list = children >>> location = >>> >>> maildir:/var/vmail/%%d/%%n/Maildir:INDEX=/var/vmail/%d/%n/Maildir/shared/%%u >>> >>> prefix = Shared/.%%n/. separator = . subscriptions = no >>> type = shared } namespace virtual { location = >>> virtual:/etc/dovecot/virtual:INDEX=~/virtual mailbox all { >>> special_use = \All } prefix = virtual. separator = . } >>> passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql >>> } plugin { acl = vfile acl_shared_dict = >>> file:/var/vmail/shared-mailboxes.db fts = solr >>> fts_autoindex = yes fts_solr = >>> url=http://localhost:8090/solr/ quota = >>> dict:user::file:/var/vmail/%d/%n/.quotausage sieve = >>> /var/vmail/%d/%n/.sieve } protocols = imap pop3 sieve >>> service auth { unix_listener >>> /var/spool/postfix/private/auth { group = postfix mode = >>> 0660 user = postfix } unix_listener auth-userdb { group = >>> vmail mode = 0600 user = vmail } user = root } service >>> imap-login { client_limit = 1000 process_limit = 500 } >>> ssl_ca = >>> >> -bundle ssl_cert = >> ssl_key = >> !SSLv3 userdb { args = /etc/dovecot/dovecot-sql.conf >>> driver = sql } protocol imap { mail_plugins = " acl fts >>> fts_solr virtual imap_acl quota imap_quota" } protocol >>> pop3 { mail_plugins = quota virtual pop3_uidl_format = >>> %08Xu%08Xv } protocol lda { mail_plugins = sieve quota >>> virtual } >>> >>> >>> >>> Best Regards >>> MfG Robert Schetterer >>> >>> >>> -- >>> Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail >>> gesendet. > > > Best Regards > MfG Robert Schetterer > From h.reindl at thelounge.net Sat Mar 28 18:31:18 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 28 Mar 2015 19:31:18 +0100 Subject: Error after setting up fts /solr for Open-Xchange In-Reply-To: <5516DEAE.8050906@systemschmiede.com> References: <550F03FD.9080104@systemschmiede.com> <5516777F.3020606@systemschmiede.com> <55168095.4010807@sys4.de> <93872201-0FC3-4BE4-9F25-918BBBF6500D@systemschmiede.com> <5516AB7B.4020605@systemschmiede.com> <5516BD31.7010606@sys4.de> <5516DEAE.8050906@systemschmiede.com> Message-ID: <5516F376.7060905@thelounge.net> Am 28.03.2015 um 18:02 schrieb zucca at systemschmiede.com: > Well...That seemed to have worked in fact. > Updated to 2:2.2.16-1~auto+36. > "All folders"-search works, and no errors are being shown. Besides, the > all folder search in Open-Xchange looks great and is lightning fast! well, the first step in case of troubles should always be update to the last recent version (every software not only postfix) - time and effort to make new releases is spent for good reasons :-) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From zucca at systemschmiede.com Sat Mar 28 18:59:41 2015 From: zucca at systemschmiede.com (zucca at systemschmiede.com) Date: Sat, 28 Mar 2015 19:59:41 +0100 Subject: Error after setting up fts /solr for Open-Xchange In-Reply-To: <5516F376.7060905@thelounge.net> References: <550F03FD.9080104@systemschmiede.com> <5516777F.3020606@systemschmiede.com> <55168095.4010807@sys4.de> <93872201-0FC3-4BE4-9F25-918BBBF6500D@systemschmiede.com> <5516AB7B.4020605@systemschmiede.com> <5516BD31.7010606@sys4.de> <5516DEAE.8050906@systemschmiede.com> <5516F376.7060905@thelounge.net> Message-ID: <5516FA1D.6090501@systemschmiede.com> guess you're right! :-) Systemschmiede IT-L?sungen Zucca & Teves GbR August-Thyssen-Stra?e 2-4 52511 Geilenkirchen www.systemschmiede.com Tel.: +49 (0)2451 - 9088390 Fax: +49 (0)2451 - 9124070 Mobil: +49 (0)178 - 4073254 Ust.ID: DE248030393 Am 28.03.2015 um 19:31 schrieb Reindl Harald: > > > Am 28.03.2015 um 18:02 schrieb zucca at systemschmiede.com: >> Well...That seemed to have worked in fact. >> Updated to 2:2.2.16-1~auto+36. >> "All folders"-search works, and no errors are being shown. Besides, the >> all folder search in Open-Xchange looks great and is lightning fast! > > well, the first step in case of troubles should always be update to > the last recent version (every software not only postfix) - time and > effort to make new releases is spent for good reasons :-) > From laurent.rathle at free.fr Sat Mar 28 23:30:02 2015 From: laurent.rathle at free.fr (Laurent Rathle) Date: Sun, 29 Mar 2015 00:30:02 +0100 Subject: Problem connecting to an imap account with Dovecot In-Reply-To: <5516C42C.6010605@whyscream.net> References: <5516C42C.6010605@whyscream.net> Message-ID: <42741E8E-1D5C-4B59-9A3B-684F1A3A0A3C@free.fr> > Le 28 mars 2015 ? 16:09, Tom Hendrikx a ?crit : > > You should start here: http://wiki.dovecot.org/TestInstallation and > tell where you got stuck (and include the details of the steps you > managed to complete). Thank you. Doing the tests explained in the wiki, I?ve finally discovered that there was a bad configuration in my DNS, and that the mail server was pointing at a wrong IP. Now everything is fine. Sorry for the external link to the config file. I?m used to forums and I was thinking that adding the config file content in the mail would be too long :) Best regards From yawowb+dovecot at nuclei.ca Sun Mar 29 07:26:15 2015 From: yawowb+dovecot at nuclei.ca (rooster) Date: Sun, 29 Mar 2015 00:26:15 -0700 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> Message-ID: <467C85A8-46A4-4299-8C82-0EA6734E7081@nuclei.ca> > On 2015-03-23, at 1:34 AM, rooster wrote: > > Hello list, > > I have been struggling with establishing a working installation of dovecot with mySQL quota support. I have not been able to find a guide or tutorial that works for the current revisions of software I have at my disposal. > > Most notably, the error I see in the mail log is as follows : > > Mar 23 00:55:31 host dovecot: lmtp(328): Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so > Mar 23 00:55:31 host dovecot: lmtp(328): Error: dlopen(/usr/lib/dovecot/modules/lib11_imap_quota_plugin.so) failed: /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so: undefined symbol: command_unregister > Mar 23 00:55:31 host dovecot: lmtp(328): Fatal: Couldn't load required plugins > > I have attached my dovecot -n output. The installed revision of dovecot is 2.2.9 on ubuntu 14.04.02 LTS. > > Thank you in advance. > > -- > > Since I hadn?t tested access to dovecot yet, I ran through the test steps posted on the wiki (http://wiki.dovecot.org/TestInstallation). I was able to log in and list the mailboxes successfully, but the error above still persists on message delivery. Should I be using lmtp with postfix + mysql and dovecot? Or should I be using lda? The directions I read were for lmtp. -- From skdovecot at smail.inf.fh-brs.de Sun Mar 29 17:02:09 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen) Date: Sun, 29 Mar 2015 19:02:09 +0200 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: <467C85A8-46A4-4299-8C82-0EA6734E7081@nuclei.ca> References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> <467C85A8-46A4-4299-8C82-0EA6734E7081@nuclei.ca> Message-ID: <55183011.30808@smail.inf.fh-brs.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 rooster wrote: > >> On 2015-03-23, at 1:34 AM, rooster >> wrote: >> >> Hello list, >> >> I have been struggling with establishing a working installation >> of dovecot with mySQL quota support. I have not been able to find >> a guide or tutorial that works for the current revisions of >> software I have at my disposal. >> >> Most notably, the error I see in the mail log is as follows : >> >> Mar 23 00:55:31 host dovecot: lmtp(328): Debug: Module loaded: >> /usr/lib/dovecot/modules/lib10_quota_plugin.so Mar 23 00:55:31 >> host dovecot: lmtp(328): Error: >> dlopen(/usr/lib/dovecot/modules/lib11_imap_quota_plugin.so) >> failed: /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so: >> undefined symbol: command_unregister Mar 23 00:55:31 host >> dovecot: lmtp(328): Fatal: Couldn't load required plugins >> >> I have attached my dovecot -n output. The installed revision of >> dovecot is 2.2.9 on ubuntu 14.04.02 LTS. >> >> Thank you in advance. >> >> -- >> >> > > > Since I hadn?t tested access to dovecot yet, I ran through the test > steps posted on the wiki > (http://wiki.dovecot.org/TestInstallation). I was able to log in > and list the mailboxes successfully, but the error above still > persists on message delivery. > > Should I be using lmtp with postfix + mysql and dovecot? Or should > I be using lda? The directions I read were for lmtp. The actual question is, why your system cannot load the shared libray lib11_imap_quota_plugin.so dynamically, although lib10_quota_plugin.so works. Maybe you should remove the failed file and force a reinstall of the package. I don't suppose, the package is defective, someone other should have reported this problem by now. - -- Steffen -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQEVAwUBVRgwEHz1H7kL/d9rAQKJZgf9GDoJL2sZ4Aik7Df5Ld290tG4LeHeAfcB Wo5GmCsYQJwDbCUKflzYYsGk37/MwVEh0xP7gdbclgT+VTVv7f6n/VgnM/V1w+R7 cLtv56LD6fyfHOylLhykDUiWsQiH41gLb+SUdy7LV11oGPM8BgpY1pQreXmUq2QR CY8nDrVIEZ774VG7TyJBacLQ2/bsavW0s0eLxkLKVykptDH8ZVtas1GsRqlWlEC6 HdzFE5eb2TfilorwMH+l2DjiRwNsaBzO1BkeGnnCnTMz2OK1KOZxkfOWtgTSUQw+ G15V88MEe5PaxDJez/Yj8qtnJsv8QLD+ekRjc407rU3zzM6a/67zlA== =xxG5 -----END PGP SIGNATURE----- From gedalya at gedalya.net Sun Mar 29 17:16:16 2015 From: gedalya at gedalya.net (Gedalya) Date: Sun, 29 Mar 2015 13:16:16 -0400 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: <55183011.30808@smail.inf.fh-brs.de> References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> <467C85A8-46A4-4299-8C82-0EA6734E7081@nuclei.ca> <55183011.30808@smail.inf.fh-brs.de> Message-ID: <55183360.1000603@gedalya.net> On 03/29/2015 01:02 PM, Steffen wrote: > The actual question is, why your system cannot load the shared libray > lib11_imap_quota_plugin.so dynamically, although lib10_quota_plugin.so > works. > > Maybe you should remove the failed file and force a reinstall of the > package. I don't suppose, the package is defective, someone other > should have reported this problem by now. This problem can be reproduced in one second. Here you go: protocol lmtp { mail_plugins = $mail_plugins sieve imap_quota } Mar 29 13:13:33 mail dovecot: lmtp(25315): Connect from local Mar 29 13:13:33 mail dovecot: lmtp(gedalya at gedalya.net): Error: Couldn't load required plugin /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so: dlopen() failed: /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so: undefined symbol: command_unregister Simple: imap_quota is not a plugin for lmtp, it is a plugin for imap. Notice the last bit of the error message. It won't work just like you couldn't dynamically load it into LibreOffice or whatever. From h.reindl at thelounge.net Mon Mar 30 09:25:08 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 30 Mar 2015 11:25:08 +0200 Subject: sieve rule for "header don't exist" Message-ID: <55191674.4020306@thelounge.net> is there a way to expresse when the header "X-Spam-Status" *do not* exist move the message to a different folder? :contains, :matches and :is are not helpful here background: the spamass-milter option -B is lacking the spamassassin headers in case of milter-rejects and via sendmail generated BCC while flagged messages contain the headers - so it would be nice to move the rejected ones to a subfolder "REJECTED" instead into the inbox -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From ml+dovecot at valo.at Mon Mar 30 09:41:13 2015 From: ml+dovecot at valo.at (Christian Kivalo) Date: Mon, 30 Mar 2015 11:41:13 +0200 Subject: sieve rule for "header don't exist" In-Reply-To: <55191674.4020306@thelounge.net> References: <55191674.4020306@thelounge.net> Message-ID: <8b157d7cf36013ac208d932eec41bb58@valo.at> On 2015-03-30 11:25, Reindl Harald wrote: > is there a way to expresse when the header "X-Spam-Status" *do not* > exist move the message to a different folder? > > :contains, :matches and :is are not helpful here > Have you tried using the "exist" test from the sieve rfc 5.5. Test exists Usage: exists The "exists" test is true if the headers listed in the header-names argument exist within the message. All of the headers must exist or the test is false. The following example throws out mail that doesn't have a From header and a Date header. Example: if not exists ["From","Date"] { discard; } https://tools.ietf.org/html/rfc5228#page-28 i have not tried it myself but the core of rfc 5228 is reported to be fully support by pigeonhole regards - christian From h.reindl at thelounge.net Mon Mar 30 10:49:43 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 30 Mar 2015 12:49:43 +0200 Subject: sieve rule for "header don't exist" In-Reply-To: <8b157d7cf36013ac208d932eec41bb58@valo.at> References: <55191674.4020306@thelounge.net> <8b157d7cf36013ac208d932eec41bb58@valo.at> Message-ID: <55192A47.9030607@thelounge.net> Am 30.03.2015 um 11:41 schrieb Christian Kivalo: > On 2015-03-30 11:25, Reindl Harald wrote: >> is there a way to expresse when the header "X-Spam-Status" *do not* >> exist move the message to a different folder? >> >> :contains, :matches and :is are not helpful here >> > > Have you tried using the "exist" test from the sieve rfc indeed - that works - thanks! require ["fileinto"]; if not exists ["X-Spam-Status"] { fileinto "REJECTED"; } else { keep; } > 5.5. Test exists > > > Usage: exists > > The "exists" test is true if the headers listed in the header-names > argument exist within the message. All of the headers must exist or > the test is false. > > The following example throws out mail that doesn't have a From header > and a Date header. > > Example: if not exists ["From","Date"] { > discard; > } > > https://tools.ietf.org/html/rfc5228#page-28 > > i have not tried it myself but the core of rfc 5228 is reported to be > fully support by pigeonhole -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From yawowb+dovecot at nuclei.ca Mon Mar 30 11:05:12 2015 From: yawowb+dovecot at nuclei.ca (rooster) Date: Mon, 30 Mar 2015 04:05:12 -0700 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: <55183360.1000603@gedalya.net> References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> <467C85A8-46A4-4299-8C82-0EA6734E7081@nuclei.ca> <55183011.30808@smail.inf.fh-brs.de> <55183360.1000603@gedalya.net> Message-ID: > On 2015-03-29, at 10:16 AM, Gedalya wrote: > > This problem can be reproduced in one second. Here you go: > > protocol lmtp { > mail_plugins = $mail_plugins sieve imap_quota > } > > Mar 29 13:13:33 mail dovecot: lmtp(25315): Connect from local > Mar 29 13:13:33 mail dovecot: lmtp(gedalya at gedalya.net): Error: Couldn't load required plugin /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so: dlopen() failed: /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so: undefined symbol: command_unregister > > Simple: imap_quota is not a plugin for lmtp, it is a plugin for imap. Notice the last bit of the error message. It won't work just like you couldn't dynamically load it into LibreOffice or whatever. Thank you for pinpointing the issue. Unfortunately, I am at a lost then as to why it fails for me. My settings are as follows : 10-mail.conf : mail_plugins = $mail_plugins zlib quota 15-lda.conf : mail_plugins = $mail_plugins sieve 20-imap.conf : mail_plugins = $mail_plugins imap_zlib imap_quota antispam 20-lmtp.conf : mail_plugins = $mail_plugins sieve Am I to remove each $mail_plugins and then explicitly set each mail_plugins = ""? Looking at my doveconf -n file I posted originally, I do see the following : protocol lmtp { mail_plugins = " zlib quota imap_zlib imap_quota antispam sieve" } And checking the doveconf - n output right now on the server I see the same as above (I was checking in case I had attempted changes since my original post, but nothing has changed). So, either due to a bug or working as intended (or possibly, both), my lmtp protocol section is loading all $mail_plugins. Is it correct to theorize that if the file 20-lmtp.conf were to be renamed so that it was numbered between 16 and 19, that this $mail_plugins problem would be resolved? Or should I rename 20-imap.conf and 20-pop3.conf (I have not enabled POP3 as of yet), to higher digits (e.g. : 21-imap.conf and 21-pop3.conf)? If renaming the conf files to higher digits will fix this for me, I am inclined to do so to the imap and pop3 conf files. :) If this is a bug, is this the ubuntu package at fault, or something further up the chain? -- From gedalya at gedalya.net Mon Mar 30 11:24:00 2015 From: gedalya at gedalya.net (Gedalya) Date: Mon, 30 Mar 2015 07:24:00 -0400 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> <467C85A8-46A4-4299-8C82-0EA6734E7081@nuclei.ca> <55183011.30808@smail.inf.fh-brs.de> <55183360.1000603@gedalya.net> Message-ID: <55193250.8040400@gedalya.net> On 03/30/2015 07:05 AM, rooster wrote: > 10-mail.conf : mail_plugins = $mail_plugins zlib quota > > 15-lda.conf : mail_plugins = $mail_plugins sieve > > 20-imap.conf : mail_plugins = $mail_plugins imap_zlib imap_quota antispam > > 20-lmtp.conf : mail_plugins = $mail_plugins sieve > > Am I to remove each $mail_plugins and then explicitly set each mail_plugins = ""? Looking at my doveconf -n file I posted originally, I do see the following : > > protocol lmtp { > mail_plugins = " zlib quota imap_zlib imap_quota antispam sieve" > } > > And checking the doveconf - n output right now on the server I see the same as above (I was checking in case I had attempted changes since my original post, but nothing has changed). > > So, either due to a bug or working as intended (or possibly, both), my lmtp protocol section is loading all $mail_plugins. > Is it correct to theorize that if the file 20-lmtp.conf were to be renamed so that it was numbered between 16 and 19, that this $mail_plugins problem would be resolved? Or should I rename 20-imap.conf and 20-pop3.conf (I have not enabled POP3 as of yet), to higher digits (e.g. : 21-imap.conf and 21-pop3.conf)? If renaming the conf files to higher digits will fix this for me, I am inclined to do so to the imap and pop3 conf files.:) > > If this is a bug, is this the ubuntu package at fault, or something further up the chain? The order in which the config files are included and parsed ought not to matter. Dovecot parses the entire config, resulting in what you see in doveconf -n output, before it does anything. Only the final result applies. If your config file says: key = value1 key = value1 value2 (or, key = $key value2 - same thing) Then key = value1 value2, and that and only that ever matters. I have a suspicion as to what your issue is. Here is a clarification. Dovecot has only one single config file - the result of concatenating all files as included out of the dovecot.conf file. The names of the individual files in conf.d are technically meaningless. The config is organized into topics only to make it easier to humans to find their way around. The way to apply a setting to a specific component is to put it under a section such as: protocol imap {}. You've mentioned that you have different mail_plugins settings in different _files_, but are you sure they are always inside the protocl xxx {} section? From yawowb+dovecot at nuclei.ca Mon Mar 30 23:11:01 2015 From: yawowb+dovecot at nuclei.ca (rooster) Date: Mon, 30 Mar 2015 16:11:01 -0700 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: <55193250.8040400@gedalya.net> References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> <467C85A8-46A4-4299-8C82-0EA6734E7081@nuclei.ca> <55183011.30808@smail.inf.fh-brs.de> <55183360.1000603@gedalya.net> <55193250.8040400@gedalya.net> Message-ID: <33ED421C-7032-4E8E-9629-C2F3FA83A844@nuclei.ca> > On 2015-03-30, at 4:24 AM, Gedalya wrote: > > The order in which the config files are included and parsed ought not to matter. Dovecot parses the entire config, resulting in what you see in doveconf -n output, before it does anything. Only the final result applies. > > If your config file says: > > key = value1 > key = value1 value2 (or, key = $key value2 - same thing) > > Then key = value1 value2, and that and only that ever matters. > > I have a suspicion as to what your issue is. > > Here is a clarification. Dovecot has only one single config file - the result of concatenating all files as included out of the dovecot.conf file. The names of the individual files in conf.d are technically meaningless. The config is organized into topics only to make it easier to humans to find their way around. > > The way to apply a setting to a specific component is to put it under a section such as: protocol imap {}. You've mentioned that you have different mail_plugins settings in different _files_, but are you sure they are always inside the protocl xxx {} section? Thank you for this detail. While I suspected such operation, I now know for sure. While not all protocol sections are defined in each file, here are the contents of the other files that contain mail_plugins parameters : 10-mail.conf: mail_plugins = $mail_plugins zlib quota 20-imap.conf: mail_plugins = $mail_plugins imap_zlib imap_quota antispam And here are the contents of each defined protocol section : 15-lda.conf: protocol lda { 15-lda.conf: mail_plugins = $mail_plugins sieve 15-lda.conf: } 20-lmtp.conf: protocol lmtp { 20-lmtp.conf: mail_plugins = $mail_plugins sieve 20-lmtp.conf: } Given that doveconf -n shows : protocol lmtp { mail_plugins = " zlib quota imap_zlib imap_quota antispam sieve" } I need to change something, somewhere. I am not opposed to removing $mail_plugins from any protocol section. Is it syntactically correct to use $mail_plugins in the 10-mail.conf file? Perhaps this is my source of the issues for lmtp? Here are the results of my tests when making changes : test 1 ) - removed $mail_plugins 10-mail.conf: mail_plugins = zlib quota - doveconf - n output shows : mail_plugins = zlib quota imap_zlib imap_quota antispam protocol lmtp { mail_plugins = zlib quota imap_zlib imap_quota antispam sieve } test 2 ) - removed mail_plugins from 10-mail.conf - doveconf - n output shows : mail_plugins = " imap_zlib imap_quota antispam" protocol lmtp { mail_plugins = " imap_zlib imap_quota antispam sieve" } test 3 ) - removed mail_plugins from 10-mail.conf - removed $mail_plugins from 15-lda.conf, 20-imap.conf, and 20-lmtp.conf - explicitly set mail_plugins in 15-lda.conf, 20-imap.conf, and 20-lmtp.conf - doveconf - n output shows : mail_plugins = zlib quota imap_zlib imap_quota antispam protocol lmtp { mail_plugins = zlib quota sieve } I?m curious as to why on the 2nd test, there are quote marks in the mail_plugins parameters but not in the 1st or 3rd test? I think the 3rd test is the desired operating configuration. If so, why is using $mail_plugins causing an issue? Is this a bug in the ubuntu 14.04 packages? Or was I wrong to use $mail_plugins in the lmtp protocol section (if so, the same is likely true for the lda protocol section)? -- From gedalya at gedalya.net Mon Mar 30 23:57:42 2015 From: gedalya at gedalya.net (Gedalya) Date: Mon, 30 Mar 2015 19:57:42 -0400 Subject: quota setup assistance (postfix+dovecot+mysql) In-Reply-To: <33ED421C-7032-4E8E-9629-C2F3FA83A844@nuclei.ca> References: <13249365-841B-4CD4-AD97-A6DECB0D5B04@nuclei.ca> <467C85A8-46A4-4299-8C82-0EA6734E7081@nuclei.ca> <55183011.30808@smail.inf.fh-brs.de> <55183360.1000603@gedalya.net> <55193250.8040400@gedalya.net> <33ED421C-7032-4E8E-9629-C2F3FA83A844@nuclei.ca> Message-ID: <5519E2F6.7090106@gedalya.net> On 03/30/2015 07:11 PM, rooster wrote: > I?m curious as to why on the 2nd test, there are quote marks in the mail_plugins parameters but not in the 1st or 3rd test? When you use $mail_plugins for the first time, it is equal to the empty string, because it wasn't set yet. When you set a config setting to $mail_plugins followed by a space, and then some more stuff, you are effectively putting a leading space into the value. doveconf -n represents this clearly by quoting the string. This is not important. > I think the 3rd test is the desired operating configuration. No. You are putting imap - specific plugins in the global mail_plugins. > If so, why is using $mail_plugins causing an issue? Is this a bug in the ubuntu 14.04 packages? No. > Or was I wrong to use $mail_plugins in the lmtp protocol section No. You need to revise your general understanding of how dovecot configuration works. I suggest you read slowly and carefully my previous messages in this thread, as I have already explained a few facts that are relevant to your trouble, plus whatever other sources, such as the dovecot wiki. From miloslav.hula at gmail.com Tue Mar 31 12:16:14 2015 From: miloslav.hula at gmail.com (=?UTF-8?B?TWlsb3NsYXYgSMWvbGE=?=) Date: Tue, 31 Mar 2015 14:16:14 +0200 Subject: Migrating from Cyrus to Dovecot In-Reply-To: <715D864F-3DE2-4458-A293-B4F9BEF88C16@iki.fi> References: <5515127E.2050205@gmail.com> <715D864F-3DE2-4458-A293-B4F9BEF88C16@iki.fi> Message-ID: <551A900E.4030805@gmail.com> Hi Timo, thank you for the valuable answers! Milo Dne 27.3.2015 v 22:12 Timo Sirainen napsal(a): >> On 27 Mar 2015, at 10:19, Miloslav H?la wrote: >> >> Hi, >> >> we are migrating from Cyrus 2.3.7 to Dovecot 2.2.13. We have ~7000 maildirs with ~500GB. Our goal is to do the migration without users have notice and with the shortest service downtime. The users use IMAP (with shared folders and ACL), POP3 and sieve filters. >> >> As a first choice, we tried the Dovecot's dsync tool. First tests were great, but we are not able to change the Cyrus auth backend for migration. Moreover, this migration seems too slow for us. >> >> As a second try, we tried the cyrus2dovecot migrating Perl scripts (and their derivates) from Wiki2. More or less they works but we found we need more control during the migration. >> >> So, as a third try, we wrote own migrating scripts. And thanks to the cyrus2dovecot it wasn't too much complicated. And there are my questions: >> >> A) Files and dirs timestamps >> The mtime of email file is important as an internal date as I found on Wiki2. But what about timestamps of cur/new/tmp directories or Dovecot's internal files line dovecot-uidlist? Do they play some role here? > > No. > >> B) The 128 bit mailbox UID >> The Wiki2 speaks about 128 bit mailbox UID at first line of dovecot-uidlist. Cyrus preserves only 64 bit UID. Is this mailbox UID required by Dovecot? If so, can we use 0000000000000000501100008c4a11c1 (Cyrus UID padded by zeros)? > > The mailbox GUID is internal to Dovecot. There's no standard IMAP way to see it, so there's no need to migrate it. Better not to set it and let Dovecot generate it automatically. > >> C) Format of dovecot-uidlist records >> Wiki2 shows two examples: >> 25006 :1276528487.M364837P9451.kurkku,S=1355,W=1394:2, >> 25017 W2481 :1276533073.M242911P3632.kurkku:2,F >> >> Which format is preferred? Or what the benefits are? > > If W=size is in the filename, it never needs to be recalculated if dovecot-uidlist is lost. Of course, dovecot-uidlist should never be lost. So I don't think it makes a huge difference. If you care about performance, sdbox/mdbox mailbox format would behave much better. sdbox is a close match to Cyrus - so with Maildir you're actually likely making the disk I/O performance somewhat slower in Dovecot than in Cyrus, although that also depends on other things. > >> D) Converting between CRLF and LF >> If I understand correctly, Dovecot stores emails with LF only. We have all emails with CRLF now on Cyrus and converting them to LF only is a little more time consuming. Is there any benefit to do that? Or can we live with 'mail_save_crlf' without problems? > > Dovecot can automatically handle both mixed CRLF and LF mails, you can keep old mails as CRLF and new mails as LF. mail_save_crlf setting only controls what is used for new emails. If you want to save more disk space you can enable compression. > >> E) POP3 backend >> I found many informations about IMAP internals but few on POP3 internals. What do I need to do POP3 migration transparent for user? > > Just preserve the UIDL. See the pop3_uidl_format setting in http://wiki2.dovecot.org/Migration/Cyrus From robertfantini at gmail.com Tue Mar 31 17:19:29 2015 From: robertfantini at gmail.com (Robert Fantini) Date: Tue, 31 Mar 2015 13:19:29 -0400 Subject: multiple imap servers for high availability Message-ID: Hello, We run imap for our domain at our lan. I'd like to set up a backup mirror that has imap storage always duplicated. Then using something like heartbeat to share the imapd IP address . Could someone point me to documents , if any , that give ideas on how to set up the mirror? best regards, Rob From rgiles at arlut.utexas.edu Tue Mar 31 17:25:37 2015 From: rgiles at arlut.utexas.edu (Robert Giles) Date: Tue, 31 Mar 2015 12:25:37 -0500 (CDT) Subject: multiple imap servers for high availability In-Reply-To: References: Message-ID: You might take a look at Peer Heinlein's Dovecot book - chapter 16: Dovecot: POP3/IMAP servers for enterprises and ISPs ISBN-10: 3955391078 http://www.amazon.com/Dovecot-POP3-IMAP-servers-enterprises/dp/3955391078/ Robert On Tue, 31 Mar 2015 13:19 -0400, Robert Fantini wrote: > We run imap for our domain at our lan. > > I'd like to set up a backup mirror that has imap storage always > duplicated. Then using something like heartbeat to share the imapd IP > address . > > Could someone point me to documents , if any , that give ideas on how to > set up the mirror? > > best regards, Rob From robertfantini at gmail.com Tue Mar 31 20:30:53 2015 From: robertfantini at gmail.com (Robert Fantini) Date: Tue, 31 Mar 2015 16:30:53 -0400 Subject: multiple imap servers for high availability In-Reply-To: References: Message-ID: I'll order it now. Thank you . On Tue, Mar 31, 2015 at 1:25 PM, Robert Giles wrote: > You might take a look at Peer Heinlein's Dovecot book - chapter 16: > > Dovecot: POP3/IMAP servers for enterprises and ISPs > ISBN-10: 3955391078 > http://www.amazon.com/Dovecot-POP3-IMAP-servers-enterprises/dp/3955391078/ > > Robert > > > > On Tue, 31 Mar 2015 13:19 -0400, Robert Fantini wrote: > >> We run imap for our domain at our lan. >> >> I'd like to set up a backup mirror that has imap storage always >> duplicated. Then using something like heartbeat to share the imapd IP >> address . >> >> Could someone point me to documents , if any , that give ideas on how to >> set up the mirror? >> >> best regards, Rob >> > From space.ship.traveller at gmail.com Tue Mar 31 23:16:28 2015 From: space.ship.traveller at gmail.com (Samuel Williams) Date: Wed, 1 Apr 2015 12:16:28 +1300 Subject: Deleting empty folders In-Reply-To: <550DBD5D.60104@gmx.de> References: <550D2A58.3040504@gmx.de> <550DBD5D.60104@gmx.de> Message-ID: Hi Hardy, I had over 1600 folders, writing manually doveadm mailbox delete would be a real pain. I guess I could script it but that's even more of a pain. It would be nice if there was a command to remove empty mailboxes or something similar. On 22 March 2015 at 07:50, Hardy Flor wrote: > Hello Samuel, > > with mdbox as Maildir no direct file operations should beperforms. For > empty mailboxes that will ever happen, the "doveadm mailbox delete" deletes > the emails, if required in the mailbox. > > Hardy > > > Am 21.03.2015 um 11:48 schrieb Samuel Williams: > >> Thanks Hardy. I have 1600 folders to delete.. in the end I did find -type >> d >> -name "* 1" -exec rm -r {} \; >> >> From Merlin at DesktopMasters.com Sun Mar 29 18:58:36 2015 From: Merlin at DesktopMasters.com (Merlin at Desktop Masters) Date: Sun, 29 Mar 2015 11:58:36 -0700 Subject: Install process breaks its self by design Message-ID: <55184B5C.4000804@DesktopMasters.com> Greetings, I just tried to install Dovecot for the first time. It was on a virgin Ubuntu/Debian server (Version: 1:2.2.9-1ubuntu5) and the install failed. After a bit of struggling I worked out that the install fails BY DESIGN. Spoke about this with a few people and we decided it was best to report this issue and request that you redesign the packing so it does not fail. The reason it fails is as follows... When you install it asks you if you want to use self signed certificates. I have certificates so I said, "No". At the end of the install the script tries to start dovecot. It cannot because the service does not have certificates so it fails. This causes the script to abort and the other packages not to install and breaks the install process. I had to purge dovecot, then install it with the self signed certificates. So we have 2 issues here. 1. Why allow people to not install certificates if it is just going to break the install and confuse them. 2. Why are you starting the service right after install when dovecot clearly requires manual configuration as part of the setup. I recommend you resolve this by NOT starting the service as part of the install unless it is an upgrade and the service was already running. ~ Merlin From anthony at cajuntechie.org Tue Mar 31 23:37:34 2015 From: anthony at cajuntechie.org (Anthony Papillion) Date: Tue, 31 Mar 2015 18:37:34 -0500 Subject: Need a bit of help with the antispam plugin Message-ID: <551B2FBE.2030504@cajuntechie.org> Hello Everyone, I'm running the antispam plugin on Dovecot 2.0.19 on Ubuntu Server 14.04 and I can't seem to get it to work. In the IMAP section of dovecot.conf I have the following lines: protocol imap { mail_plugins = $mail_plugins imap_quota imap_acl antispam # mail_plugins = $mail_plugins imap_quota imap_acl imap_client_workarounds = tb-extra-mailbox-sep # Maximum number of IMAP connections allowed for a user from each IP address. # NOTE: The username is compared case-sensitively. # Default is 10. # Increase it to avoid issue like below: # "Maximum number of concurrent IMAP connections exceeded" mail_max_userip_connections = 20 plugin { antispam_debug_target = syslog antispam_verbose_debug = 1 antispam_backend = pipe antispam_trash = Trash antispam_spam = Junk antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham antispam_pipe_program = /usr/bin/sa-learn antispam_pipe_program_args = --username=%Lu } } For some reason, when I try to move something to Junk, I get the following in my syslog: Mar 31 18:27:16 anonymail imap: antispam: plugin initialising (2.0-notgit) Mar 31 18:27:16 anonymail imap: antispam: verbose debug enabled Mar 31 18:27:16 anonymail imap: antispam: "Junk" is exact match spam folder Mar 31 18:27:16 anonymail imap: antispam: no unsure folders Mar 31 18:27:16 anonymail imap: antispam: "Trash" is exact match trash folder Mar 31 18:27:16 anonymail imap: antispam: pipe backend spam argument = --spam Mar 31 18:27:16 anonymail imap: antispam: pipe backend not-spam argument = --ham Mar 31 18:27:16 anonymail imap: antispam: pipe backend program = /usr/bin/sa-learn Mar 31 18:27:16 anonymail imap: antispam: pipe backend program arg[0] = --username=anthony at cajuntechie.org Mar 31 18:27:16 anonymail imap: antispam: pipe backend tmpdir /tmp Mar 31 18:27:19 anonymail imap: antispam: mailbox_is_unsure(Junk): 0 Mar 31 18:27:19 anonymail imap: antispam: mailbox_is_trash(INBOX): 0 Mar 31 18:27:19 anonymail imap: antispam: mailbox_is_trash(Junk): 0 Mar 31 18:27:19 anonymail imap: antispam: mail copy: from trash: 0, to trash: 0 Mar 31 18:27:19 anonymail imap: antispam: mailbox_is_spam(INBOX): 0 Mar 31 18:27:19 anonymail imap: antispam: mailbox_is_spam(Junk): 1 Mar 31 18:27:19 anonymail imap: antispam: mailbox_is_unsure(INBOX): 0 Mar 31 18:27:19 anonymail imap: antispam: mail copy: src spam: 0, dst spam: 1, src unsure: 0 Mar 31 18:27:19 anonymail imap: antispam: running mailtrain backend program /usr/bin/sa-learn Mar 31 18:27:19 anonymail imap: antispam: running mailtrain backend program /usr/bin/sa-learn Mar 31 18:27:19 anonymail imap: antispam: running mailtrain backend program parameter 1 --username=anthony at cajuntechie.org Mar 31 18:27:19 anonymail imap: antispam: running mailtrain backend program parameter 2 --spam Mar 31 18:27:21 anonymail imap: antispam: run program failed with exit code -1 This is probably a permissions issue but I'm not able to debug it. Can anyone offer me any clues as to what I might be doing wrong or how I might fix it? Thanks, Anthony -- Anthony Papillion Phone: 1.918.631.7331 VoIP (SIP): 1259010 at localphone.com XMPP Chat: cypher at chat.cpunk.us Fingerprint: 65EF73EC 8B57F6B1 8C475BD4 426088AC FE21B251 PGP Key: http://www.cajuntechie.org/p/my-pgp-key.html