From amateo at um.es Thu Oct 1 10:08:38 2015 From: amateo at um.es (Angel L. Mateo) Date: Thu, 1 Oct 2015 12:08:38 +0200 Subject: SiS and attachments filename Message-ID: <560D0626.6030900@um.es> Hi, I've been working with dovecot SiS for a long time without any problem. This is my configuration for SiS: mail_attachment_dir = /mail/users/attachments mail_attachment_fs = sis posix mail_attachment_hash = %{sha1} mail_attachment_min_size = 128 k With this configuration, I have, for example: # ls -l /mail/users/attachments/27/d1/27d1904ac9947945bf48ea545ec278940d840ed1* -rw------- 12 vmail vmail 142914 may 6 2013 /mail/users/attachments/27/d1/27d1904ac9947945bf48ea545ec278940d840ed1-090d88269ad28851f14b0000a9061514 ... And every file is a hard link to /mail/users/attachments/27/d1/hashes/27d1904ac9947945bf48ea545ec278940d840ed1 For the documentation I have found, "27d1904ac9947945bf48ea545ec278940d840ed1" is a hash of the attach content, but what does the other part of the filename (the "090d88269ad28851f14b0000a9061514" means) My question has come to my mind because I need an easy way to find the mailbox where this attach is referenced from. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From marcin at mejor.pl Thu Oct 1 10:12:02 2015 From: marcin at mejor.pl (=?UTF-8?Q?Marcin_Miros=c5=82aw?=) Date: Thu, 1 Oct 2015 12:12:02 +0200 Subject: Does fts_lucene need fts_squat? Message-ID: <560D06F2.3000305@mejor.pl> Hi ML! Please look at this: - I've got no lucene-index: > # ls /dane/domeny/mejor.pl/mail/marcin/.maildir/lucene-indexes > ls: cannot access /dane/domeny/mejor.pl/mail/marcin/.maildir/lucene-indexes: No such file or directory - run search in body: > echo -e "a select bugzilla\na search body flto\na logout\n" | /usr/libexec/dovecot/imap -u marcin at mejor.pl > * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SEARCH=FUZZY NOTIFY SPECIAL-USE COMPRESS=DEFLATE QUOTA] Logged in as marcin at mejor.pl > * FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk Old $label1 $label2 $label3 $label4 $label5 ae_autoextract) > * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk Old $label1 $label2 $label3 $label4 $label5 ae_autoextract \*)] Flags permitted. > * 5311 EXISTS > * 0 RECENT > * OK [UNSEEN 4910] First unseen. > * OK [UIDVALIDITY 1246465819] UIDs valid > * OK [UIDNEXT 17871] Predicted next UID > * OK [HIGHESTMODSEQ 16914] Highest > a OK [READ-WRITE] Select completed (0.005 secs). > * SEARCH 366 380 2132 2515 2548 2551 2566 2567 2589 2612 2615 2617 3678 3696 4538 > a OK Search completed (2.768 secs). > * BYE Logging out > a OK Logout completed. > imap(marcin at mejor.pl) <>: Info: Disconnected: Logged out in=45 out=1084 - still doesn't have lucene indexes: > # ls /dane/domeny/mejor.pl/mail/marcin/.maildir/lucene-indexes > ls: cannot access /dane/domeny/mejor.pl/mail/marcin/.maildir/lucene-indexes: No such file or directory # doveadm index -A "*" # ls /dane/domeny/mejor.pl/mail/marcin/.maildir/lucene-indexes ls: cannot access /dane/domeny/mejor.pl/mail/marcin/.maildir/lucene-indexes: No such file or directory Now I'm changin 10-mail.conf: -mail_plugins = fts fts_lucene quota notify mail_log stats zlib +mail_plugins = fts fts_squat fts_lucene quota notify mail_log stats zlib Reloading dovecot and: - # doveadm index -A "*" - # ls /dane/domeny/mejor.pl/mail/marcin/.maildir/lucene-indexes _0.cfs segments_3 segments.gen Now something appears. But search returns no hit: > # echo -e "a select bugzilla\na search body flto\na logout\n" | /usr/libexec/dovecot/imap -u marcin at mejor.pl > * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SEARCH=FUZZY NOTIFY SPECIAL-USE COMPRESS=DEFLATE QUOTA] Logged in as marcin at mejor.pl > * FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk Old $label1 $label2 $label3 $label4 $label5 ae_autoextract) > * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk Old $label1 $label2 $label3 $label4 $label5 ae_autoextract \*)] Flags permitted. > * 5311 EXISTS > * 0 RECENT > * OK [UNSEEN 4910] First unseen. > * OK [UIDVALIDITY 1246465819] UIDs valid > * OK [UIDNEXT 17871] Predicted next UID > * OK [HIGHESTMODSEQ 16914] Highest > a OK [READ-WRITE] Select completed (0.008 secs). > * SEARCH > a OK Search completed (0.004 secs). > * BYE Logging out > a OK Logout completed. > imap(marcin at mejor.pl) <>: Info: Disconnected: Logged out in=45 out=1011 Also probably I mess something in index files. I've added fts_lucene earlier, then removed /lucene-indexes dir. Tried to do doveadm index, (does doeveadm fts rescan/optimize work? I didn't notice that mentioned commands do something with fts indexes (both squat and lucene). Marcin > # 2.2.18: /etc/dovecot/dovecot.conf > # Pigeonhole version 0.4.8 (0c4ae064f307+) > # OS: Linux 4.1.6-hardened x86_64 Gentoo Base System release 2.2 > auth_cache_size = 1 k > auth_master_user_separator = * > auth_mechanisms = login digest-md5 cram-md5 plain > deliver_log_format = msgid=%m: from=%f: phys=%p: virt=%w %$ > dict { > quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext > } > first_valid_uid = 8 > last_valid_uid = 8 > lda_mailbox_autocreate = yes > lda_mailbox_autosubscribe = yes > mail_attachment_dir = /dane/domeny/zalaczniki > mail_attachment_min_size = 10000 k > mail_cache_min_mail_count = 20 > mail_gid = mail > mail_log_prefix = "%s(%u) <%{session}>: " > mail_plugins = fts fts_squat fts_lucene quota notify mail_log stats zlib > mail_prefetch_count = 20 > mail_privileged_group = mail > mail_uid = mail > mailbox_list_index = yes > maildir_stat_dirs = yes > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include vari > ables body enotify environment mailbox date index ihave duplicate > mdbox_preallocate_space = yes > mdbox_rotate_interval = 60 days > mdbox_rotate_size = 50 M > namespace inbox { > inbox = yes > list = yes > location = > mailbox Drafts { > auto = subscribe > special_use = \Drafts > } > mailbox Junk { > auto = subscribe > special_use = \Junk > } > mailbox Sent { > auto = subscribe > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > auto = subscribe > special_use = \Trash > } > prefix = > separator = . > subscriptions = yes > } > passdb { > args = /etc/dovecot/.master.user > driver = passwd-file > master = yes > pass = yes > } > passdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > plugin { > fts = lucene > fts_lucene = whitespace_chars=@. > fts_squat = partial=4 full=10 > quota = dict:User quota::proxy::quota > sieve = file:~/sieve;active=~/.dovecot.sieve > stats_command_min_time = 1 mins > stats_domain_min_time = 12 hours > stats_ip_min_time = 12 hours > stats_memory_limit = 10k > stats_refresh = 30 s > stats_session_min_time = 15 mins > stats_track_cmds = yes > stats_user_min_time = 1 hours > zlib_save = bz2 > zlib_save_level = 2 > } > postmaster_address = postmaster at mejor.pl > protocols = imap pop3 sieve > service auth { > unix_listener auth-userdb { > group = mail > mode = 0660 > user = root > } > } > service dict { > unix_listener dict { > user = mail > } > } > service stats { > fifo_listener stats-mail { > mode = 0600 > user = mail > } > } > ssl_cert = ssl_key = userdb { > driver = prefetch > } > userdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > verbose_proctitle = yes > protocol lda { > mail_plugins = fts fts_squat fts_lucene quota notify mail_log stats zlib sieve > } > protocol imap { > mail_max_userip_connections = 20 > mail_plugins = fts fts_squat fts_lucene quota notify mail_log stats zlib imap_quota imap_stats imap_zlib > } From info at lpcom.de Thu Oct 1 11:25:26 2015 From: info at lpcom.de (Daniel Heitepriem) Date: Thu, 01 Oct 2015 13:25:26 +0200 Subject: Dovecot 1.0.10 - Authenticate against LDAP with multiple sub-trees Message-ID: <2a2120b56978da828f595c9b04176ece@lpcom.de> Hi guys, I'm using Dovecot 1.0.10 with LDAP authentication. Below is my dovecot-ldap.conf { hosts = *IP of LDAP server* auth_bind = yes auth_bind_userdn = uid=%u,ou=people,dc=*something*,dc=com ldap_version = 3 base = %u,ou=people,dc=*something*,dc=com deref = never scope = subtree user_filter = (&(objectClass=person)(uid=%u)) } And the current structure: Base | |-- people | | *user entries* | Now I want to update to a LDAP server with the following structure: Base | |-- people | |-- staff | | *user entries* |-- other location | | *user entries* | How can I keep my "auth_bind_userdn" using this new structure? Hopefully somebody can help me :) Regards, Daniel From Christian.Schmidt at chemie.uni-hamburg.de Thu Oct 1 12:07:29 2015 From: Christian.Schmidt at chemie.uni-hamburg.de (Christian Schmidt) Date: Thu, 1 Oct 2015 14:07:29 +0200 Subject: [IMAP] Nothing in INBOX In-Reply-To: References: Message-ID: <560D2201.7010201@chemie.uni-hamburg.de> Hello Yvon, On 30.09.2015 15:14, Yvon Thoraval wrote: > I've just installed Postfix + Dovecot on Xubunto 14.04 LTS. > Following this page : > Setup a mail server with Postfix and Dovecot on Ubuntu / Debian > > > although I'm able to send emails to GMail and other kind of domain, nothing > appears on Thunderbird INBOX, nor using openssl : > [..] > However, using mail, i can read the emails : > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > $ mail > > Heirloom mailx version 12.5 6/20/10. Type ? for help. > > "/var/mail/desktop": 20 messages 9 new 18 unread > [..] > my dovecot setup uses vhosts : > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > $ ls -al /var/mail/vhosts/// > [..] > with nothing in new subfolder: As you wrote, mail accesses and displays /var/mail/desktop, but you told dovecot to serve mails from var/mail/vhosts//. You probably want postfix to hand over mails to dovecot via lmtp. Mit freundlichen Gr??en Christian Schmidt -- No signature available. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5326 bytes Desc: S/MIME Cryptographic Signature URL: From pbatkov at hostcomm.ru Thu Oct 1 12:28:04 2015 From: pbatkov at hostcomm.ru (Pavel Batkov) Date: Thu, 1 Oct 2015 15:28:04 +0300 Subject: doveadm + ldap userdb Message-ID: <560D26D4.20805@hostcomm.ru> Hi! OS: CentOS Linux release 7.1.1503 (Core) Dovecot: 2.2.18 Run user listing - doveadm user '*', single user - work. auth[20540]: segfault at 8 ip 00007fd554340c6b sp 00007ffc17c886e0 error 4 in libauthdb_ldap.so[7fd55433b000+a000] dovecot: auth: Fatal: master: service(auth): child 20540 killed with signal 11 (core dumped) dovecot: auth: Error: net_connect_unix(auth-worker) failed: No such file or directory Backtrace: { "signal": 11 , "executable": "/usr/libexec/dovecot/auth" , "stacktrace": [ { "crash_thread": true , "frames": [ { "address": 139954408369259 , "build_id": "62d54c5834c6675c5ec229927d4c087a3277c1d2" , "build_id_offset": 23659 , "function_name": "ldap_input" , "file_name": "/usr/lib64/dovecot/auth/libauthdb_ldap.so" } , { "address": 139954486291015 , "build_id": "9ab58d2125033b7eef9f04560a7d7d28737585ce" , "build_id_offset": 502343 , "function_name": "io_loop_call_io" , "file_name": "/usr/lib64/dovecot/libdovecot.so.0" } , { "address": 139954486294735 , "build_id": "9ab58d2125033b7eef9f04560a7d7d28737585ce" , "build_id_offset": 506063 , "function_name": "io_loop_handler_run" , "file_name": "/usr/lib64/dovecot/libdovecot.so.0" } , { "address": 139954486289816 , "build_id": "9ab58d2125033b7eef9f04560a7d7d28737585ce" , "build_id_offset": 501144 , "function_name": "io_loop_run" , "file_name": "/usr/lib64/dovecot/libdovecot.so.0" } , { "address": 139954485950947 , "build_id": "9ab58d2125033b7eef9f04560a7d7d28737585ce" , "build_id_offset": 162275 , "function_name": "master_service_run" , "file_name": "/usr/lib64/dovecot/libdovecot.so.0" } , { "address": 139954491066353 , "build_id": "9e3bf53b9c2ecb0bec470c65ace47166336e650f" , "build_id_offset": 59377 , "function_name": "main" , "file_name": "/usr/libexec/dovecot/auth" } ] } ] } From gdb: Core was generated by `dovecot/auth'. Program terminated with signal 11, Segmentation fault. #0 0x00007f49accbac6b in ldap_input () from /usr/lib64/dovecot/auth/libauthdb_ldap.so (gdb) bt full #0 0x00007f49accbac6b in ldap_input () from /usr/lib64/dovecot/auth/libauthdb_ldap.so No symbol table info available. #1 0x00007f49b170aa47 in io_loop_call_io () from /usr/lib64/dovecot/libdovecot.so.0 No symbol table info available. #2 0x00007f49b170b8cf in io_loop_handler_run () from /usr/lib64/dovecot/libdovecot.so.0 No symbol table info available. #3 0x00007f49b170a598 in io_loop_run () from /usr/lib64/dovecot/libdovecot.so.0 No symbol table info available. #4 0x00007f49b16b79e3 in master_service_run () from /usr/lib64/dovecot/libdovecot.so.0 No symbol table info available. #5 0x00007f49b1b987f1 in main () No symbol table info available. From yvon.thoraval at gmail.com Thu Oct 1 13:36:52 2015 From: yvon.thoraval at gmail.com (Yvon Thoraval) Date: Thu, 1 Oct 2015 15:36:52 +0200 Subject: [IMAP] Nothing in INBOX In-Reply-To: <560D2201.7010201@chemie.uni-hamburg.de> References: <560D2201.7010201@chemie.uni-hamburg.de> Message-ID: ? 2015-10-01 14:07 GMT+02:00 Christian Schmidt < Christian.Schmidt at chemie.uni-hamburg.de>: > As you wrote, mail accesses and displays /var/mail/desktop, but you told > dovecot to serve mails from var/mail/vhosts//. > > You probably want postfix to hand over mails to dovecot via lmtp. > right ! Vielen dank ! From josh at joshtriplett.org Thu Oct 1 18:41:03 2015 From: josh at joshtriplett.org (Josh Triplett) Date: Thu, 1 Oct 2015 11:41:03 -0700 Subject: /usr/lib/dovecot/imap exits with an error if stderr closed Message-ID: <20151001184043.GA10636@jtriplet-mobl1> At the end of a direct IMAP conversation with the imap binary, it writes an informative message like "imap(josh): Info: Connection closed in=0 out=365" to stderr. With stderr closed, the imap binary will exit with exit code 81 after it gets EBADF from that write. While it makes sense to fail on failure to write an error message, I don't think it makes sense to fail if writing an info message fails. I encountered this when using /usr/lib/dovecot/imap with mutt's "tunnel" option. Mutt seems to run the configured tunnel program with stderr closed, which triggers this error. - Josh Triplett From pbatkov at hostcomm.ru Thu Oct 1 20:39:35 2015 From: pbatkov at hostcomm.ru (=?UTF-8?B?0J/QsNCy0LXQuyDQkdCw0YLRjNC60L7Qsg==?=) Date: Thu, 1 Oct 2015 23:39:35 +0300 Subject: doveadm + ldap userdb In-Reply-To: <560D26D4.20805@hostcomm.ru> References: <560D26D4.20805@hostcomm.ru> Message-ID: <560D9A07.5030608@hostcomm.ru> Sorry, I attached old backtrace.. auth[28503]: segfault at 28 ip 00007f6aadc0c831 sp 00007ffc368ae0e0 error 4 in auth[7f6aadbef000+49000] (gdb) bt full #0 auth_worker_server_resume_input (conn=0x0) at auth-worker-server.c:476 No locals. #1 0x00007f6aadc1a4ee in userdb_blocking_iter_deinit (_ctx=_ctx at entry=0x7f6aae0e0ac8) at userdb-blocking.c:126 ctx = ret = 0 #2 0x00007f6aadc008d1 in master_input_list_finish (ctx=0x7f6aae0e0ac0) at auth-master-connection.c:448 __FUNCTION__ = "master_input_list_finish" #3 0x00007f6aadc00e3d in auth_master_connection_destroy (_conn=_conn at entry=0x7ffc368ae140) at auth-master-connection.c:785 conn = 0x7f6aae0df650 #4 0x00007f6aadc01b34 in auth_master_connections_destroy_all () at auth-master-connection.c:834 conn = 0x0 #5 0x00007f6aadbfda5f in main_deinit () at main.c:279 l = #6 main (argc=1, argv=0x7f6aae09e3a0) at main.c:395 c = { "signal": 11 , "executable": "/usr/libexec/dovecot/auth" , "stacktrace": [ { "crash_thread": true , "frames": [ { "address": 140096158353457 , "build_id": "f367e23a12ce8b183a0d3abe4abfe69734c0a787" , "build_id_offset": 120881 , "function_name": "auth_worker_server_resume_input" , "file_name": "/usr/libexec/dovecot/auth" } , { "address": 140096158409966 , "build_id": "f367e23a12ce8b183a0d3abe4abfe69734c0a787" , "build_id_offset": 177390 , "function_name": "userdb_blocking_iter_deinit" , "file_name": "/usr/libexec/dovecot/auth" } , { "address": 140096158304465 , "build_id": "f367e23a12ce8b183a0d3abe4abfe69734c0a787" , "build_id_offset": 71889 , "function_name": "master_input_list_finish" , "file_name": "/usr/libexec/dovecot/auth" } , { "address": 140096158305853 , "build_id": "f367e23a12ce8b183a0d3abe4abfe69734c0a787" , "build_id_offset": 73277 , "function_name": "auth_master_connection_destroy" , "file_name": "/usr/libexec/dovecot/auth" } , { "address": 140096158309172 , "build_id": "f367e23a12ce8b183a0d3abe4abfe69734c0a787" , "build_id_offset": 76596 , "function_name": "auth_master_connections_destroy_all" , "file_name": "/usr/libexec/dovecot/auth" } , { "address": 140096158292575 , "build_id": "f367e23a12ce8b183a0d3abe4abfe69734c0a787" , "build_id_offset": 59999 , "function_name": "main" , "file_name": "/usr/libexec/dovecot/auth" } ] } ] } 01.10.2015 15:28, Pavel Batkov ?????: > Hi! > > OS: CentOS Linux release 7.1.1503 (Core) > Dovecot: 2.2.18 > Run user listing - doveadm user '*', single user - work. > > auth[20540]: segfault at 8 ip 00007fd554340c6b sp 00007ffc17c886e0 > error 4 in libauthdb_ldap.so[7fd55433b000+a000] > > dovecot: auth: Fatal: master: service(auth): child 20540 killed with > signal 11 (core dumped) > dovecot: auth: Error: net_connect_unix(auth-worker) failed: No such > file or directory > > Backtrace: > > { "signal": 11 > , "executable": "/usr/libexec/dovecot/auth" > , "stacktrace": > [ { "crash_thread": true > , "frames": > [ { "address": 139954408369259 > , "build_id": > "62d54c5834c6675c5ec229927d4c087a3277c1d2" > , "build_id_offset": 23659 > , "function_name": "ldap_input" > , "file_name": > "/usr/lib64/dovecot/auth/libauthdb_ldap.so" > } > , { "address": 139954486291015 > , "build_id": > "9ab58d2125033b7eef9f04560a7d7d28737585ce" > , "build_id_offset": 502343 > , "function_name": "io_loop_call_io" > , "file_name": "/usr/lib64/dovecot/libdovecot.so.0" > } > , { "address": 139954486294735 > , "build_id": > "9ab58d2125033b7eef9f04560a7d7d28737585ce" > , "build_id_offset": 506063 > , "function_name": "io_loop_handler_run" > , "file_name": "/usr/lib64/dovecot/libdovecot.so.0" > } > , { "address": 139954486289816 > , "build_id": > "9ab58d2125033b7eef9f04560a7d7d28737585ce" > , "build_id_offset": 501144 > , "function_name": "io_loop_run" > , "file_name": "/usr/lib64/dovecot/libdovecot.so.0" > } > , { "address": 139954485950947 > , "build_id": > "9ab58d2125033b7eef9f04560a7d7d28737585ce" > , "build_id_offset": 162275 > , "function_name": "master_service_run" > , "file_name": "/usr/lib64/dovecot/libdovecot.so.0" > } > , { "address": 139954491066353 > , "build_id": > "9e3bf53b9c2ecb0bec470c65ace47166336e650f" > , "build_id_offset": 59377 > , "function_name": "main" > , "file_name": "/usr/libexec/dovecot/auth" > } ] > } ] > } > > From gdb: > > Core was generated by `dovecot/auth'. > Program terminated with signal 11, Segmentation fault. > #0 0x00007f49accbac6b in ldap_input () from > /usr/lib64/dovecot/auth/libauthdb_ldap.so > > (gdb) bt full > #0 0x00007f49accbac6b in ldap_input () from > /usr/lib64/dovecot/auth/libauthdb_ldap.so > No symbol table info available. > #1 0x00007f49b170aa47 in io_loop_call_io () from > /usr/lib64/dovecot/libdovecot.so.0 > No symbol table info available. > #2 0x00007f49b170b8cf in io_loop_handler_run () from > /usr/lib64/dovecot/libdovecot.so.0 > No symbol table info available. > #3 0x00007f49b170a598 in io_loop_run () from > /usr/lib64/dovecot/libdovecot.so.0 > No symbol table info available. > #4 0x00007f49b16b79e3 in master_service_run () from > /usr/lib64/dovecot/libdovecot.so.0 > No symbol table info available. > #5 0x00007f49b1b987f1 in main () > No symbol table info available. From tss at iki.fi Fri Oct 2 16:13:08 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 2 Oct 2015 19:13:08 +0300 Subject: v2.2.19 released Message-ID: <82DE7170-1438-4A9E-9841-10EFDF324220@iki.fi> http://dovecot.org/releases/2.2/dovecot-2.2.19.tar.gz http://dovecot.org/releases/2.2/dovecot-2.2.19.tar.gz.sig * pop3_deleted_flag has been broken since v2.2.10. Using it would cause buffer overflows, which could be exploitable. However, this bug would have become visible quite soon after users had deleted some POP3 mails, because the pop3 processes would have started crashing all the time even in normal use. * "doveadm director flush" command has a changed meaning now: It safely moves users to their wanted backends, instead of simply forgetting the mapping entirely and leaving the existing connections untouched. Use -F parameter to get the original unsafe behavior. + Added imap-hibernate processes (see imap_hibernate_timeout setting). IDLEing IMAP connections can be hibernated, which saves memory. + Optimized tracking mailboxes' vsizes (= sum of all messages' sizes). If mailbox_list_index=yes, it's also stored in there. This makes it very efficient to look up vsizes for all mailboxes. + Added a quota "count" backend, which uses the mailbox vsizes to get the current quota usage. It requires using the new quota_vsizes=yes setting, which tracks the messages' "virtual sizes" rather than "physical sizes". Their distiction is minor and mostly irrelevant nowadays (if mail sizes should be counted with LF or CRLF newlines). + "doveadm director up/down" commands added. The monitoring script should be using these commands instead of changing the vhost count. This allows admin to manually disable a server by changing the vhost count to 0 without the monitoring script changing it back. + Added support for HAProxy protocol: http://wiki2.dovecot.org/HAProxy + Added push-notification plugin framework, which can be used to easily implement push notifications to various backends. Implemented "ox" backend for notifying Open-Xchange via HTTP/json. + imap_logout_format supports more variables now, e.g. number of deleted messages. + pop3: Added pop3_delete_type setting (related to pop3_deleted_flag). + plugin { fts_enforced=yes } setting now fails body searches unless it can be done via the full text search engine. + Added %{passdb:*} and %{userdb:*} variables to various places + auth: Added ":protected" suffix for passdb and userdb fields. If used, the field doesn't overwrite an existing field. + IMAP/POP3 proxy: If a backend server dies, avoid client reconnection spikes by slowly disconnecting clients over time. This is enabled by setting login_proxy_max_disconnect_delay=secs passdb extra field. + imap: Added new read-only METADATA entries: /private/specialuse, /shared/comment, /shared/admin + imap: If client disconnects in the middle of a command, log how long the command had been running. - mdbox: Rebuilding could have caused message's reference count to overflow the 16bit number in some situations, causing problems when trying to expunge the duplicates. - Various search fixes (fts, solr, tika, lib-charset, indexer) - Various virtual plugin fixes - Various fixes and optimizations to dsync, imapc and pop3-migration - imap: Various RFC compliancy and crash fixes to NOTIFY From sca at andreasschulze.de Fri Oct 2 19:31:44 2015 From: sca at andreasschulze.de (A. Schulze) Date: Fri, 2 Oct 2015 21:31:44 +0200 Subject: v2.2.19 released In-Reply-To: <82DE7170-1438-4A9E-9841-10EFDF324220@iki.fi> References: <82DE7170-1438-4A9E-9841-10EFDF324220@iki.fi> Message-ID: <560EDBA0.6000507@andreasschulze.de> Am 02.10.2015 um 18:13 schrieb Timo Sirainen: > http://dovecot.org/releases/2.2/dovecot-2.2.19.tar.gz > http://dovecot.org/releases/2.2/dovecot-2.2.19.tar.gz.sig Thanks Timo! From rs at sys4.de Fri Oct 2 20:28:38 2015 From: rs at sys4.de (Robert Schetterer) Date: Fri, 02 Oct 2015 22:28:38 +0200 Subject: v2.2.19 released In-Reply-To: <82DE7170-1438-4A9E-9841-10EFDF324220@iki.fi> References: <82DE7170-1438-4A9E-9841-10EFDF324220@iki.fi> Message-ID: <560EE8F6.40808@sys4.de> Am 02.10.2015 um 18:13 schrieb Timo Sirainen: > + Added imap-hibernate processes (see imap_hibernate_timeout setting). > IDLEing IMAP connections can be hibernated, which saves memory. Hm, i have Couldn't hibernate imap client: Couldn't export state: COMPRESS enabled does this mean if zlib plugin is enabled, hibernate dont works ? Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From sca at andreasschulze.de Fri Oct 2 22:35:30 2015 From: sca at andreasschulze.de (A. Schulze) Date: Sat, 3 Oct 2015 00:35:30 +0200 Subject: v2.2.19 released / imap-hibernate In-Reply-To: <82DE7170-1438-4A9E-9841-10EFDF324220@iki.fi> References: <82DE7170-1438-4A9E-9841-10EFDF324220@iki.fi> Message-ID: <560F06B2.30506@andreasschulze.de> Am 02.10.2015 um 18:13 schrieb Timo Sirainen: > + Added imap-hibernate processes (see imap_hibernate_timeout setting). > IDLEing IMAP connections can be hibernated, which saves memory. here is my config to enable the new function: ---- imap_hibernate_timeout = 60s service imap-hibernate { unix_listener imap-hibernate { user = vmail } } service imap { unix_listener imap-master { mode = 0666 } } ---- using this settings, one thunderbird connected to an otherwise idle mailbox, I have there processes: 10312 ? Ss 0:00 /usr/sbin/dovecot -F 10316 ? S 0:00 \_ dovecot/anvil [10 connections] 10317 ? S 0:00 \_ dovecot/log 10319 ? S 0:00 \_ dovecot/config 10329 ? S 0:00 \_ dovecot/imap-hibernate [10 connections] 10350 ? S 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] 10351 ? S 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] 10366 ? S 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] 10369 ? S 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] 10372 ? S 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] 10374 ? S 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] 10377 ? S 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] 10379 ? S 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] 10382 ? S 0:01 \_ dovecot/imap-login [1 connections (1 TLS)] 10385 ? S 0:01 \_ dovecot/imap-login [1 connections (1 TLS)] 11744 ? S 0:00 \_ dovecot/ssl-params 11746 ? S 0:00 \_ dovecot/lmtp is it OK that there are still 10 separte imap-login processes? are there concerns setting the mode for /var/run/dovecot/imap-master from 0600 to 0666? and are there suggestions for imap_hibernate_timeout? Thanks, Andreas From mkliewe at gmx.de Fri Oct 2 23:50:27 2015 From: mkliewe at gmx.de (Michael Kliewe) Date: Sat, 3 Oct 2015 01:50:27 +0200 Subject: imap_logout_format improvements / ideas In-Reply-To: <8CF3F5F9-0E00-401A-920D-74B2FF338338@iki.fi> References: <8CF3F5F9-0E00-401A-920D-74B2FF338338@iki.fi> Message-ID: <560F1843.1030100@gmx.de> Hi Timo, Am 15.05.2015 um 13:59 schrieb Timo Sirainen: > http://hg.dovecot.org/dovecot-2.2/rev/93bba97afb2a > http://hg.dovecot.org/dovecot-2.2/rev/e0a17714f0c9 > > So all in all, you can now add to imap_logout_format: > > # %i - total number of bytes read from client > # %o - total number of bytes sent to client > # %{fetch_hdr_count} - Number of mails with mail header data sent to client > # %{fetch_hdr_bytes} - Number of bytes with mail header data sent to client > # %{fetch_body_count} - Number of mails with mail body data sent to client > # %{fetch_body_bytes} - Number of bytes with mail body data sent to client > # %{deleted} - Number of mails where client added \Deleted flag > # %{expunged} - Number of mails that client expunged > # %{trashed} - Number of mails that client copied/moved to the > # special_use=\Trash mailbox. > > Any thoughts on what else would be useful? > > I was also thinking about separating (deletes and) expunges from Trash vs non-Trash, but I'm not sure how useful that would be (and what would it be called?) It's also a bit annoying that a copy+expunge to a Trash will return expunged=1 trashed=1, but a move to Trash (via IMAP MOVE) will return expunged=0 trashed=1. But trying to fix that would be too much trouble. when imap_zlib is enabled, the %i and %o still show the uncompressed bytes. It could be very helpful to access the compressed bytes if the client used COMPRESS, so you can see which clients use COMPRESS, you can see/calculate a percentage (for information only), and if you want to do accounting and/or limit a users traffic, this would be very helpful to get the real amount of bytes travelling the line. Currently if a client uses COMPRESS, your accounting (by parsing the logfile) is wrong by maybe 50% or even more... So maybe 4 new variables could be very helpful if imap_zlib is used: - total number of compressed bytes read from client - total number of compressed bytes sent to client - percentage of compressed to uncompressed bytes read from client - percentage of compressed to uncompressed bytes sent to client If either imap_zlib is not enabled, or the client did not use COMPRESS, set everything to 0 (or "-" or something). Thanks Michael From tss at iki.fi Sat Oct 3 08:50:21 2015 From: tss at iki.fi (Timo Sirainen) Date: Sat, 3 Oct 2015 11:50:21 +0300 Subject: v2.2.19 released In-Reply-To: <560EE8F6.40808@sys4.de> References: <82DE7170-1438-4A9E-9841-10EFDF324220@iki.fi> <560EE8F6.40808@sys4.de> Message-ID: On 02 Oct 2015, at 23:28, Robert Schetterer wrote: > > Am 02.10.2015 um 18:13 schrieb Timo Sirainen: >> + Added imap-hibernate processes (see imap_hibernate_timeout setting). >> IDLEing IMAP connections can be hibernated, which saves memory. > > Hm, i have > > Couldn't hibernate imap client: Couldn't export state: COMPRESS enabled > > does this mean if zlib plugin is enabled, hibernate dont works ? imap_zlib plugin enables COMPRESS. If COMPRESS is used, it can't currently be hibernated. The plan to get this fixed in future means moving the COMPRESS handling code to imap-login process similar to how SSL is handled. But this then needs some post-login communication between imap and imap-login processes, which is a bit larger change. From tss at iki.fi Sat Oct 3 08:53:04 2015 From: tss at iki.fi (Timo Sirainen) Date: Sat, 3 Oct 2015 11:53:04 +0300 Subject: v2.2.19 released / imap-hibernate In-Reply-To: <560F06B2.30506@andreasschulze.de> References: <82DE7170-1438-4A9E-9841-10EFDF324220@iki.fi> <560F06B2.30506@andreasschulze.de> Message-ID: On 03 Oct 2015, at 01:35, A. Schulze wrote: > > > > Am 02.10.2015 um 18:13 schrieb Timo Sirainen: >> + Added imap-hibernate processes (see imap_hibernate_timeout setting). >> IDLEing IMAP connections can be hibernated, which saves memory. > > using this settings, one thunderbird connected to an otherwise idle mailbox, I have there processes: > 10312 ? Ss 0:00 /usr/sbin/dovecot -F > 10316 ? S 0:00 \_ dovecot/anvil [10 connections] > 10317 ? S 0:00 \_ dovecot/log > 10319 ? S 0:00 \_ dovecot/config > 10329 ? S 0:00 \_ dovecot/imap-hibernate [10 connections] > 10350 ? S 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] > 10351 ? S 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] > 10366 ? S 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] > 10369 ? S 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] > 10372 ? S 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] > 10374 ? S 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] > 10377 ? S 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] > 10379 ? S 0:00 \_ dovecot/imap-login [1 connections (1 TLS)] > 10382 ? S 0:01 \_ dovecot/imap-login [1 connections (1 TLS)] > 10385 ? S 0:01 \_ dovecot/imap-login [1 connections (1 TLS)] > 11744 ? S 0:00 \_ dovecot/ssl-params > 11746 ? S 0:00 \_ dovecot/lmtp > > is it OK that there are still 10 separte imap-login processes? That's a separate issue. Switch to high-performance mode described in http://wiki2.dovecot.org/LoginProcess > are there concerns setting the mode for /var/run/dovecot/imap-master from 0600 to 0666? Ideally avoid that. You can probably just do: service imap { unix_listener imap-master { mode = 0600 user = vmail } } > and are there suggestions for imap_hibernate_timeout? I'm interested in some good answers to this too :) Although in future it'll likely become automatic/dynamic. From tss at iki.fi Sat Oct 3 08:57:46 2015 From: tss at iki.fi (Timo Sirainen) Date: Sat, 3 Oct 2015 11:57:46 +0300 Subject: imap_logout_format improvements / ideas In-Reply-To: <560F1843.1030100@gmx.de> References: <8CF3F5F9-0E00-401A-920D-74B2FF338338@iki.fi> <560F1843.1030100@gmx.de> Message-ID: > On 03 Oct 2015, at 02:50, Michael Kliewe wrote: > > when imap_zlib is enabled, the %i and %o still show the uncompressed bytes. It could be very helpful to access the compressed bytes if the client used COMPRESS, so you can see which clients use COMPRESS, you can see/calculate a percentage (for information only), and if you want to do accounting and/or limit a users traffic, this would be very helpful to get the real amount of bytes travelling the line. Currently if a client uses COMPRESS, your accounting (by parsing the logfile) is wrong by maybe 50% or even more... > > So maybe 4 new variables could be very helpful if imap_zlib is used: > - total number of compressed bytes read from client > - total number of compressed bytes sent to client > - percentage of compressed to uncompressed bytes read from client > - percentage of compressed to uncompressed bytes sent to client > > If either imap_zlib is not enabled, or the client did not use COMPRESS, set everything to 0 (or "-" or something). It's a bit problematic to get this information from plugin to imap core.. Also compression can be done in the SSL layer, which is handled by imap-login process so imap process doesn't know about it. And I just mentioned in earlier mail that I'm planning to move COMPRESS code to imap-login as well. So getting the information becomes even more difficult, since now at logout time imap-login needs to somehow tell the imap process how many bytes there were. A bit unnecessary work just for logging purposes.. But maybe it won't be too difficult then. Need to first implement all the new code related to that. From roundcube222 at alaadin.org Sat Oct 3 15:56:24 2015 From: roundcube222 at alaadin.org (Michael Peter) Date: Sat, 3 Oct 2015 18:56:24 +0300 Subject: dovecot quota service for postfix Message-ID: <4955a977fd8c830d0782f0b6868134e5.squirrel@mailer.coptics.org> Hello, Thanks for the clarifications plugin { quota = maildir:User quota quota_rule = *:storage=1G quota_grace = 10%% # 10% is the default quota_status_success = DUNNO quota_status_nouser = DUNNO quota_status_overquota = "552 5.2.2 Mailbox is full" } The above configurations is my default for all users. but my mail system is using mbox format and not maildir.... so what should i change in the above configuration to use mbox INSTEAD of maildir ?? Also for the quota paramater, can i use (myquota = maildir:User quota) INSTEAD of (quota = maildir:User quota) ?? or it must "QUOTA =" and not any thing else to work ? same for quota_rule parameter, should i add (_rule) after quota to work ??? can i add for example qouta_restriction INSTEAD of quota_rules ?? i am asking this question to better understand how dovecot work.. also last question when i use the following command to get quota for existing system user called michael , it gives me an error as follow, so why dovecot is not able to get the quota for the system users? doveadm quota get -u michael doveadm(michael): Fatal: User doesn't exist Many thanks >> Incase user is underquota, then the email is returned back to postfix for >> processing and delivering email to user mbox, incase user has no sufficient quota then dovecot will inform postfix to reject email. Please confirm if what i mentioned above is correct and that i >> understand >> correctly how dove-cot policy works with postfix. > Your understanding is mor or less correct, except that postfix never passes the email itself to dovecot when performing the quota check. It simply sends a message to the quota policy service asking if the recipient is over quota. If the recipient is over quota, dovecot provides postfix with the response configured in quota_status_overquota. If the recipient is under quota, dovecot provides postfix with the response configured in quota_status_success. If dovecot does not know the user, it provides postfix with the response configured in > quota_status_nouser. >> Kindly check my configuration below and advise the missing parts to add... > I don't see anything wrong with your configuration on first appraisal, but if it isn't working I would recommend you perform the following troubleshooting steps. > 1. Verify the user's mailbox is over or under quota in alignment with your expectations > - `doveadm quota -u $RECIPIENT` > 2. Test that the quota service provides the response you expect it to > - `echo "recipient=$RECIPIENT" | nc -q1 localhost 12340` > On 09/26/2015 01:40 AM, Michael Peter wrote: >> Hello, >> I am trying to use dovecot quota service for postfix in order set quota for some users. >> i configure postfix policy service to ask dovecot if user quota is sufficient to accept the email or not. >> (dovecot in this case acts only as policy server to postfix as it only ACCEPT/REJECT email according to user quota, but it is not used to deliver >> email , because postfix will deliver the email to user mailbox) Incase user is underquota, then the email is returned back to postfix for >> processing and delivering email to user mbox, incase user has no sufficient quota then dovecot will inform postfix to reject email. Please confirm if what i mentioned above is correct and that i >> understand >> correctly how dove-cot policy works with postfix. >> Kindly check my configuration below and advise the missing parts to add... >> plugin { >> quota_rule = *:storage=1G >> quota_grace = 10%% >> # 10% is the default >> quota_status_success = DUNNO >> quota_status_nouser = DUNNO >> quota_status_overquota = "552 5.2.2 Mailbox is full" >> ... >> } >> The above configurations is for all users which is not override by quota >> rules in userdb's extra fields in my password file below. >> Is the above configuration missing some thing ? i am using mbox format for >> emails storage. do you i have to add (quota= SOMETHING) before >> (quote_rule >> = *:storage=1G) and what to do add to make dovecot support mbox quota? smtpd_recipient_restrictions = >> check_policy_service inet:SERVER-HIDDEN-FOR-SECRUITY.com:12340 >> service quota-status { >> executable = quota-status -p postfix >> inet_listener { >> port = 12340 >> } >> client_limit = 1 >> } >> now in order to configure dovecot specific quota for user named (user1) i >> use passwd-file as follow >> user1:{plain}pass3:1002:1002::/home/user1::userdb_mail=mbox:~/mymail:INBOX=/var/mail/user1 userdb_quota_rule=*:bytes=300M >> All other users except users1 should have 1GB storage according to this configuration. which user1 will only have 300MB >> Please advise what is my configuration is missing and if i need to add any >> thing to run without issues. >> Many thanks >> Michael Peter From rs at sys4.de Sat Oct 3 16:53:10 2015 From: rs at sys4.de (Robert Schetterer) Date: Sat, 3 Oct 2015 18:53:10 +0200 Subject: v2.2.19 released In-Reply-To: References: <82DE7170-1438-4A9E-9841-10EFDF324220@iki.fi> <560EE8F6.40808@sys4.de> Message-ID: <561007F6.8030803@sys4.de> Am 03.10.2015 um 10:50 schrieb Timo Sirainen: > On 02 Oct 2015, at 23:28, Robert Schetterer wrote: >> >> Am 02.10.2015 um 18:13 schrieb Timo Sirainen: >>> + Added imap-hibernate processes (see imap_hibernate_timeout setting). >>> IDLEing IMAP connections can be hibernated, which saves memory. >> >> Hm, i have >> >> Couldn't hibernate imap client: Couldn't export state: COMPRESS enabled >> >> does this mean if zlib plugin is enabled, hibernate dont works ? > > imap_zlib plugin enables COMPRESS. If COMPRESS is used, it can't currently be hibernated. The plan to get this fixed in future means moving the COMPRESS handling code to imap-login process similar to how SSL is handled. But this then needs some post-login communication between imap and imap-login processes, which is a bit larger change. > Thx Timo, for explain, perhaps this should be in the wiki, meanwhile Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From stephan at rename-it.nl Sun Oct 4 21:22:16 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 4 Oct 2015 23:22:16 +0200 Subject: Released Pigeonhole v0.4.9 for Dovecot v2.2.19 Message-ID: <56119888.2040201@rename-it.nl> Hello Dovecot users, Here is the definitive 0.4.9 release. Only a few minor fixes were done since RC1. Changelog v0.4.9: * Properly implemented checking of ABI version for Sieve interpreter plugins, much like Dovecot itself does for plugins. This will prevent plugin ABI mismatches. + Implemented a vnd.dovecot.environment extension. This builds upon the standard environment extension and adds a few more environment items, such as username and default mailbox. It also creates a variables namespace so that environment items can be accessed directly. I am still thinking about more environment items that can be added. + Sieve extprograms plugin: Made line endings of the input passed to the external programs configurable. This can be configured separately for each of the three extensions. + ManageSieve: Implemented proxy XCLIENT support. This allows the proxy to pass client information to the back-end. - ManageSieve: Fixed an assert failure occurring when a client disconnects during the GETSCRIPT command. - doveadm sieve plugin: Fixed incorrect initialization of mail user. This caused a few memory leaks. - sieve-filter command line tool: Fixed handling of failure-related implicit keep when there is an explicit default destination folder. This caused message duplication. - lib-sieve: Fixed bug in RFC5322 header folding. Words longer than the optimal line length caused empty lines in the output, which would break the resulting message header. This surfaced in References: headers with very long message IDs. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.9.tar.gz http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.9.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From larryrtx at gmail.com Mon Oct 5 14:43:54 2015 From: larryrtx at gmail.com (Larry Rosenman) Date: Mon, 5 Oct 2015 09:43:54 -0500 Subject: Unknown cache fields? Message-ID: I assume this is expected, but just checking. I do NOT use POP3. Oct 5 09:41:20 thebighonker dovecot: indexer-worker(mrm): Debug: Ignoring unknown cache field: pop3.order Oct 5 09:41:20 thebighonker dovecot: indexer-worker(mrm): Debug: Ignoring unknown cache field: binary.parts Oct 5 09:41:20 thebighonker dovecot: indexer-worker(mrm): Debug: Ignoring unknown cache field: body.snippet -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 From guest0x at yahoo.com Mon Oct 5 18:04:11 2015 From: guest0x at yahoo.com (Guest-X) Date: Mon, 5 Oct 2015 18:04:11 +0000 (UTC) Subject: Compiling v2.2.19 - internal compiler error: segmentation fault (test-timing.c -> int64_t issue?) Message-ID: <1088330828.759743.1444068251186.JavaMail.yahoo@mail.yahoo.com> Hi, I am trying to compile a 64-build of the latest dovecot v2.2.19 on my SuSE Linux Enterprise Server 10 (x64) system. I've run ./configure with two parameters, the --prefix one and the --with-ssldir one. During the make stage, I am getting the following 'internal compiler error' message : --------------------------------------------- [...] gcc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib-test -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -MT test_lib-test-timing.o -MD -MP -MF .deps/test_lib-test-timing.Tpo -c -o test_lib-test-timing.o `test -f 'test-timing.c' || echo './'`test-timing.c test-timing.c: In function ?test_timing?: test-timing.c:48: internal compiler error: Segmentation fault Please submit a full bug report, with preprocessed source if appropriate. See for instructions. make[4]: *** [test_lib-test-timing.o] Error 1 ---------------------------------------------- The lines that seem to be causing the problem are the following ( from src/lib/test-timing.c ) : 45: void test_timing(void) 46: { 47: static int64_t test_input1[] = { 48: 20, 19, 18, 1, 2, 3, 4, 5, 6, 7, 8, 49: 9, 10, 11, 12, 13, 14, 15, 16, 17, -1 50: }; This is the first time that any version of dovecot (the v2.2 branch) couldn't compile on my system, and I've literally compiled every single release up to now. My gcc version is a very old one (4.1.2) : ---------------------------------------------- Using built-in specs. Target: x86_64-suse-linux Configured with: ../configure --enable-threads=posix --prefix=/usr --with-local-prefix=/usr/local --infodir=/usr/share/info --mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64 --enable-languages=c,c++,objc,fortran,obj-c++,java,ada --enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.1.2 --enable-ssp --disable-libssp --disable-libgcj --with-slibdir=/lib64 --with-system-zlib --enable-shared --enable-__cxa_atexit --enable-libstdcxx-allocator=new --program-suffix= --enable-version-specific-runtime-libs --without-system-libunwind --with-cpu=generic --host=x86_64-suse-linux Thread model: posix gcc version 4.1.2 20070115 (prerelease) (SUSE Linux) ----------------------------------------------- Don't know whether this helps, but I've noticed that if I change the int64_t to uint64_t in the following lines of the same file : 61: static int64_t *test_inputs[] = { 62: test_input1, test_input2, test_input3 63: }; ..then compilation finishes without errors, but I'm not certain what the practical consequences of this action might be. Can somebody help me please ? Thank you for your time. From nick+dovecot at bunbun.be Mon Oct 5 19:05:56 2015 From: nick+dovecot at bunbun.be (Nick Rosier) Date: Mon, 05 Oct 2015 21:05:56 +0200 Subject: doveadm index assertion failed Message-ID: <7cdb984d3bb0d4ede95a16a2b2fb8d9a@bunbun.be> Hi, one of my mailboxes returns following error when I run doveadm index on it: Panic: file charset-iconv.c: line 85 (charset_to_utf8_try): assertion failed: (srcleft <= CHARSET_MAX_PENDING_BUF_SIZE) OS: FreeBSD 10.2 Dovecot: 2.1.19 Tika: 1.10 SOLR: 5.3.1 Running doveadm -D index does not show any more information indicating what causes this error (mail/folder/...) How I can find what is causing this error? Rgds, N. From cbaptista at opensuse.us Mon Oct 5 19:22:11 2015 From: cbaptista at opensuse.us (Carlos Baptista) Date: Mon, 5 Oct 2015 20:22:11 +0100 Subject: Dovecot don't erase mails from storage. Message-ID: Hi, I have a dovecot installation with a strange problem: all the mails stays in storage after deleted from users mailbox. If user logon after the mails are deleted, they don't receive them any more, but the m.## files aren?t deleted. This happens with pop3 (default for this installation) and/or imap. The dovecot is installed along postfix with mysql and multi-domains support Here's the dovecot conf ######################## dovecot -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-042stab111.11 x86_64 Debian 7.9 auth_mechanisms = plain login first_valid_uid = 111 last_valid_uid = 111 lda_mailbox_autocreate = yes mail_debug = yes mail_gid = vmail mail_plugins = " quota" mail_privileged_group = vmail mail_uid = vmail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = dict:User quota::file:%h/mail/dovecot-quota quota_rule = *:storage=500MB } postmaster_address = sysadmin at inside.pt protocols = " imap pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } ssl_cert = Oct 02 17:50:38 auth-worker(11561): Debug: sql(carla.franjoso at xxxxxxx.xx,xxx.xxx.xxx.xxx): query: /* dovecot-sql.conf password_query */ SELECT '/var/vmail/xxxxxx.xx/carla.franjoso' AS userdb_home, 'mdbox:~/mail' AS userdb_mail, 111 AS userdb_uid, 114 AS userdb_gid, CONCAT('*:bytes=', CAST(m.quota AS CHAR)) AS userdb_quota_rule, CONCAT(m.local_part, '@', d.name) AS user, m.password AS password FROM mailboxes AS m LEFT JOIN domains AS d ON m.domain_id = d.id WHERE m.local_part = 'carla.franjoso' AND d.name = 'xxxxxxx.xx' AND m.is_active AND d.is_active Oct 02 17:50:38 auth: Debug: client out: OK 1 user=carla.franjoso at xxxxxx.xx Oct 02 17:50:38 auth: Debug: master in: REQUEST 3537633281 11621 1 41ef179d4a37a603ce38c43fab768f78 Oct 02 17:50:38 auth: Debug: prefetch(carla.franjoso at xxxxxx.xx,xxx.xxx.xxx.xxx,): success Oct 02 17:50:38 auth: Debug: master out: USER 3537633281 carla.franjoso at xxxxx.xx home=/var/vmail/xxxxxxxx/carla.franjoso mail=mdbox:~/mail uid=1 11 gid=114 quota_rule=*:bytes=104857600 Oct 02 17:50:38 pop3: Debug: Loading modules from directory: /usr/lib/dovecot/modules Oct 02 17:50:38 pop3: Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Oct 02 17:50:38 pop3: Debug: Added userdb setting: mail=mdbox:~/mail Oct 02 17:50:38 pop3: Debug: Added userdb setting: plugin/quota_rule=*:bytes=104857600 Oct 02 17:50:38 pop3(carla.franjoso at xxxxx.xx): Debug: Effective uid=111, gid=114, home=/var/vmail/xxxxxxxx.xx/carla.franjoso Oct 02 17:50:38 pop3(carla.franjoso at xxxxx.xx): Debug: Quota root: name=User quota backend=dict args=:file:/var/vmail/xxxxxxxxx.xx/carla.franjoso/mail/dovecot-quota Oct 02 17:50:38 pop3(carla.franjoso at xxxxx.xx): Debug: Quota rule: root=User quota mailbox=* bytes=104857600 messages=0 Oct 02 17:50:38 pop3(carla.franjoso at xxxxx.xx): Debug: dict quota: user=carla.franjoso at xxxx.xx, uri=file:/var/vmail/xxxxxx,xx/carla.franjoso/mail/dovecot-quot a, noenforcing=0 Oct 02 17:50:38 pop3(carla.franjoso at xxxxx.xx): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes locatio n=mdbox:~/mail Oct 02 17:50:38 pop3(carla.franjoso at xxxxx.xx): Debug: fs: root=/var/vmail/ assis.pt/carla.franjoso/mail, index=, control=, inbox=, alt= Oct 02 17:50:38 pop3(carla.franjoso at xxxxx.xx): Debug: Namespace : Using permissions from /var/vmail/xxxxxxx.xx/carla.franjoso/mail: mode=0700 gid=-1 What is "gid=-1" at the end of the log? Can anyone help? Thanks Carlos Baptista From bytesplit at gmail.com Mon Oct 5 19:32:49 2015 From: bytesplit at gmail.com (Philon) Date: Mon, 5 Oct 2015 21:32:49 +0200 Subject: Dovecot don't erase mails from storage. In-Reply-To: References: Message-ID: Dear Carlos, it looks like you are using mdbox. Those files only get a marker for deletion of mails. The actual deletion needs to be done by purging the mailboxes? See Timos writing: http://www.dovecot.org/list/dovecot/2010-July/050778.html And the wiki here: http://wiki2.dovecot.org/Tools/Doveadm/Purge This will rewrite the mdbox file without the deleted emails. Thus only then the emails are actually deleted. Philon > Am 05.10.2015 um 21:22 schrieb Carlos Baptista : > > Hi, > > I have a dovecot installation with a strange problem: all the mails stays > in storage after deleted from users mailbox. If user logon after the mails > are deleted, they don't receive them any more, but the m.## files aren?t > deleted. This happens with pop3 (default for this installation) and/or imap. > > The dovecot is installed along postfix with mysql and multi-domains support > > Here's the dovecot conf > ######################## > > > dovecot -n > # 2.1.7: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-042stab111.11 x86_64 Debian 7.9 > auth_mechanisms = plain login > first_valid_uid = 111 > last_valid_uid = 111 > lda_mailbox_autocreate = yes > mail_debug = yes > mail_gid = vmail > mail_plugins = " quota" > mail_privileged_group = vmail > mail_uid = vmail > namespace inbox { > inbox = yes > location = > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > prefix = > } > passdb { > args = /etc/dovecot/dovecot-sql.conf > driver = sql > } > plugin { > mail_log_events = delete undelete expunge copy mailbox_delete > mailbox_rename > mail_log_fields = uid box msgid size > quota = dict:User quota::file:%h/mail/dovecot-quota > quota_rule = *:storage=500MB > } > postmaster_address = sysadmin at inside.pt > protocols = " imap pop3" > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0660 > user = postfix > } > } > ssl_cert = ssl_key = userdb { > driver = prefetch > } > userdb { > args = /etc/dovecot/dovecot-sql.conf > driver = sql > } > protocol lmtp { > mail_plugins = " quota sieve" > } > protocol lda { > mail_plugins = " quota" > } > protocol imap { > mail_plugins = " quota imap_quota" > } > > ############################### > > Here is an example of log file with debug > > Oct 02 17:50:38 auth: Debug: auth client connected (pid=11621) > Oct 02 17:50:38 auth: Debug: client in: AUTH 1 PLAIN > service=pop3 secured session=Xiif9iEhmgBV80+u > lip=xxx.xxx.xxx.xxx rip=xxx.xxx.xxx.xxx lport=995 > rport=53914 resp= > Oct 02 17:50:38 auth-worker(11561): Debug: > sql(carla.franjoso at xxxxxxx.xx,xxx.xxx.xxx.xxx): > query: /* dovecot-sql.conf password_query */ SELECT > '/var/vmail/xxxxxx.xx/carla.franjoso' AS userdb_home, 'mdbox:~/mail' AS > userdb_mail, 111 AS userdb_uid, 114 AS userdb_gid, CONCAT('*:bytes=', > CAST(m.quota AS CHAR)) AS userdb_quota_rule, CONCAT(m.local_part, '@', > d.name) AS user, m.password AS password FROM mailboxes AS m LEFT JOIN > domains AS d ON m.domain_id = d.id WHERE m.local_part = 'carla.franjoso' > AND d.name = 'xxxxxxx.xx' AND m.is_active AND d.is_active > Oct 02 17:50:38 auth: Debug: client out: OK 1 > user=carla.franjoso at xxxxxx.xx > Oct 02 17:50:38 auth: Debug: master in: REQUEST 3537633281 11621 1 > 41ef179d4a37a603ce38c43fab768f78 > Oct 02 17:50:38 auth: Debug: > prefetch(carla.franjoso at xxxxxx.xx,xxx.xxx.xxx.xxx,): > success > Oct 02 17:50:38 auth: Debug: master out: USER 3537633281 > carla.franjoso at xxxxx.xx home=/var/vmail/xxxxxxxx/carla.franjoso > mail=mdbox:~/mail uid=1 > 11 gid=114 quota_rule=*:bytes=104857600 > Oct 02 17:50:38 pop3: Debug: Loading modules from directory: > /usr/lib/dovecot/modules > Oct 02 17:50:38 pop3: Debug: Module loaded: > /usr/lib/dovecot/modules/lib10_quota_plugin.so > Oct 02 17:50:38 pop3: Debug: Added userdb setting: mail=mdbox:~/mail > Oct 02 17:50:38 pop3: Debug: Added userdb setting: > plugin/quota_rule=*:bytes=104857600 > Oct 02 17:50:38 pop3(carla.franjoso at xxxxx.xx): Debug: Effective uid=111, > gid=114, home=/var/vmail/xxxxxxxx.xx/carla.franjoso > Oct 02 17:50:38 pop3(carla.franjoso at xxxxx.xx): Debug: Quota root: name=User > quota backend=dict > args=:file:/var/vmail/xxxxxxxxx.xx/carla.franjoso/mail/dovecot-quota > Oct 02 17:50:38 pop3(carla.franjoso at xxxxx.xx): Debug: Quota rule: root=User > quota mailbox=* bytes=104857600 messages=0 > Oct 02 17:50:38 pop3(carla.franjoso at xxxxx.xx): Debug: dict quota: > user=carla.franjoso at xxxx.xx, > uri=file:/var/vmail/xxxxxx,xx/carla.franjoso/mail/dovecot-quot > a, noenforcing=0 > Oct 02 17:50:38 pop3(carla.franjoso at xxxxx.xx): Debug: Namespace inbox: > type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, > subscriptions=yes locatio > n=mdbox:~/mail > Oct 02 17:50:38 pop3(carla.franjoso at xxxxx.xx): Debug: fs: root=/var/vmail/ > assis.pt/carla.franjoso/mail, index=, control=, inbox=, alt= > Oct 02 17:50:38 pop3(carla.franjoso at xxxxx.xx): Debug: Namespace : Using > permissions from /var/vmail/xxxxxxx.xx/carla.franjoso/mail: mode=0700 > gid=-1 > > > What is "gid=-1" at the end of the log? > > Can anyone help? > > Thanks > Carlos Baptista From milimber at gmail.com Tue Oct 6 15:53:00 2015 From: milimber at gmail.com (Damon) Date: Tue, 6 Oct 2015 11:53:00 -0400 Subject: separate passdb for unix_listener Message-ID: I want to use a separate passed for the unix_listener (postfix smtp-auth) so i can limit access to the smtp server by user/domain using the postfixadmin database. I want to let users access map to receive email but not be able to send. Any ideas? Thanks Damon From semfly at gmail.com Tue Oct 6 22:34:53 2015 From: semfly at gmail.com (Simon) Date: Tue, 6 Oct 2015 15:34:53 -0700 (PDT) Subject: More information about - Dovecot 2.2.x quota mysql and dict Message-ID: <1444170893263-54077.post@n4.nabble.com> I some add information about bug... Dovecot 2.2.19 version on server FreeBSD 10.2 Release. Config quota (dict + mysql) Database create Postfixadmin 2.3.7 Old version Dovecot 2.2.13 use method UPDATE in MySQL base + dict example mysql.log ----------------- Connect postfix at localhost on postfix 134 Query SELECT bytes FROM quota2 WHERE username = 'user at mydomain.lan' 135 Query SELECT bytes FROM quota2 WHERE username = 'user at mydomain.lan' 135 Query BEGIN 135 Query UPDATE quota2 SET bytes=bytes+2570,messages=messages+1 WHERE username = 'user at mydomain.lan' 136 Connect postfix at localhost on postfix 136 Query BEGIN 136 Query UPDATE domain SET quota=quota+2570 WHERE domain = 'mydomain.lan' 134 Query BEGIN 134 Query UPDATE quota2 SET bytes=bytes+1723,messages=messages+1 WHERE username = 'user at mydomain.lan' 137 Connect postfix at localhost on postfix 137 Query BEGIN 137 Query UPDATE domain SET quota=quota+1723 WHERE domain = 'mydomain.lan' 136 Query UPDATE quota2 SET messages=messages+1 WHERE username = 'mydomain.lan' - - - 137 Query COMMIT 151004 3:12:27 128 Quit ----------------- # cat dovecot-2.2.13/src/lib-dict/dict-sql.c === some text delete === static const char * sql_dict_update_query(const struct dict_sql_build_query *build) { struct sql_dict *dict = build->dict; const struct dict_sql_build_query_field *fields; unsigned int i, field_count; string_t *query; i_assert(build->inc); fields = array_get(&build->fields, &field_count); i_assert(field_count > 0); query = t_str_new(64); str_printfa(query, "UPDATE %s SET ", fields[0].map->table); #### this in code, work excellent for (i = 0; i < field_count; i++) { if (i > 0) str_append_c(query, ','); str_printfa(query, "%s=%s", fields[i].map->value_field, fields[i].map->value_field); if (fields[i].value[0] != '-') str_append_c(query, '+'); str_append(query, fields[i].value); } sql_dict_where_build(dict, fields[0].map, build->extra_values, build->key1, SQL_DICT_RECURSE_NONE, query); return str_c(query); } === === === Latest version Dovecot 2.2.19 use method DELETE/INSERT in MySQL base + dict 140 Connect postfix at localhost on postfix 140 Query SELECT quota FROM domain WHERE domain = 'mydomain.lan' 140 Query SELECT quota FROM domain WHERE domain = 'mydomain.lan' 140 Query SELECT messages FROM quota2 WHERE username = 'mydomain.lan' 140 Query BEGIN 140 Query DELETE FROM domain WHERE domain = 'mydomain.lan' 140 Query DELETE FROM quota2 WHERE username = 'mydomain.lan' 140 Query INSERT INTO domain (quota,domain) VALUES ('8581','mydomain.lan') ON DUPLICATE KEY UPDATE quota='8581' 151004 15:18:45 140 Quit # cat dovecot-2.2.19/src/lib-dict/dict-sql.c === some text delete === static void sql_dict_unset(struct dict_transaction_context *_ctx, const char *key) { struct sql_dict_transaction_context *ctx = (struct sql_dict_transaction_context *)_ctx; struct sql_dict *dict = (struct sql_dict *)_ctx->dict; const struct dict_sql_map *map; ARRAY_TYPE(const_string) values; if (ctx->prev_inc_map != NULL) sql_dict_prev_inc_flush(ctx); map = sql_dict_find_map(dict, key, &values); if (map == NULL) { i_error("sql dict unset: Invalid/unmapped key: %s", key); ctx->failed = TRUE; return; } T_BEGIN { string_t *query = t_str_new(256); const char *error; str_printfa(query, "DELETE FROM %s", map->table); #### here deleted all information about domain, this code i think need change if (sql_dict_where_build(dict, map, &values, key[0], SQL_DICT_RECURSE_NONE, query, &error) < 0) { i_error("dict-sql: Failed to delete %s: %s", key, error); ctx->failed = TRUE; } else { sql_update(ctx->sql_ctx, str_c(query)); } } T_END; } === === === Now dic + mysql doesn?t work in Dovecot 2.2.19, because first DELETE operator delete information from base about my domain mydomain.lan Thanks! Simon. -- View this message in context: http://dovecot.2317879.n4.nabble.com/More-information-about-Dovecot-2-2-x-quota-mysql-and-dict-tp54077.html Sent from the Dovecot mailing list archive at Nabble.com. From marco.fretz at gmail.com Wed Oct 7 06:10:48 2015 From: marco.fretz at gmail.com (Marco Fretz) Date: Wed, 7 Oct 2015 08:10:48 +0200 Subject: separate passdb for unix_listener In-Reply-To: References: Message-ID: <5614B768.3090205@gmail.com> You can have a separate dovecot instance for smtp-auth and use that socket in postfix config. Just use a small config like the one below for the "auth only instance" and fire it up with "dovecot -F -c /etc/dovecot/dovecot-auth.conf" you can then copy and adapt the systemd script or what ever to start it automatically. in that sql-conf you just need a password_query, no user_query. --- base_dir = /var/run/dovecot-auth instance_name = dovecot-auth passdb { driver = sql args = /etc/dovecot/dovecot-auth-sql.conf.ext } # disable listeners service imap-login { inet_listener imap { port = 0 } inet_listener imaps { port = 0 } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 0 } } service auth { unix_listener auth-userdb { #mode = 0666 #user = #group = virtual_mail } # Postfix smtp-auth socket unix_listener /var/spool/postfix/private/auth-smtp { mode = 0660 group = virtual_mail } # Auth process is run as this user. #user = $default_internal_user } --- On 06.10.2015 17:53, Damon wrote: > I want to use a separate passed for the unix_listener (postfix smtp-auth) so i can limit access to the smtp server by user/domain using the postfixadmin database. > I want to let users access map to receive email but not be able to send. > > Any ideas? > > Thanks > Damon From skdovecot at smail.inf.fh-brs.de Wed Oct 7 08:15:52 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 7 Oct 2015 10:15:52 +0200 (CEST) Subject: separate passdb for unix_listener In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 6 Oct 2015, Damon wrote: > I want to use a separate passed for the unix_listener (postfix smtp-auth) so i can limit access to the smtp server by user/domain using the postfixadmin database. > I want to let users access map to receive email but not be able to send. I use %s / %{service} to distinguish SMTP from other requests in the query. It depends on the capability of your passdb, however. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVhTUuHz1H7kL/d9rAQI80Af+MS/YrNsqpxmHiRRgVLrwJBE3rI9M/w6O CFCBQRShvKKtUU6Z0gmFJJB/PqS0I/UXaPImJNAvjD6lQqwRJE+eMKWZZIisdZ8j dhZVtDlwuMMcp+Xs0tkNkMWsHz2t0wGWiqhVl1WQdHpSeHORF4JOXXOaGCDw+PHM iVw16ekiDaVipQViso3bsVtY9S3hRQTBMdffqKQnJS3Igq6kDS8HIuZ2K50KtarH vjr49b/wpst6s4mg51ZaEbv5n/w2kAWlE/gTYSYENPsyG04oHNRLUQrLQLKM/w0o wgPnoKbt3QV4v6AEj5U+vhiKEFQKx11G5VqrxFXAOS2dLsSFPrT5KQ== =DV+5 -----END PGP SIGNATURE----- From talpus at gmail.com Wed Oct 7 11:32:55 2015 From: talpus at gmail.com (Nikolai Krot) Date: Wed, 7 Oct 2015 13:32:55 +0200 Subject: how to treat synonymous domain names as one Message-ID: Hallo List Users, I am curious if there is a way to rewrite domain name to another domain name. My goal is to provide authentication and mailboxes for the synonymous domains in a single source while keeping all other domains separate. I have setup: auth_username_format = %Lu mail_location = /var/vmail/%d/%u/Maildir in userdb/passdb (passwd-file): /etc/dovecot/%d/users Say I have two synonymous domain names my-domain-1.com my-long-named-domain-1.com I want both to resolve to my-domain-1.com when %d variable is used in Dovecot configs, so that passwd-file is /etc/dovecot/my-domain-1.com/users mailbox is /var/vmail/my-domain-1.com/user/Maildir while keeping other domains unchanged my-other-domain-2.com should resolve to: passwd-file is /etc/dovecot/my-other-domain-2.com/users mailbox is /var/vmail/my-other-domain-2.com/user/Maildir thanks, Nikolai From skdovecot at smail.inf.fh-brs.de Wed Oct 7 13:09:20 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 7 Oct 2015 15:09:20 +0200 (CEST) Subject: how to treat synonymous domain names as one In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 7 Oct 2015, Nikolai Krot wrote: > I am curious if there is a way to rewrite domain name to another domain > name. My goal is to provide authentication and mailboxes for the synonymous > domains in a single source while keeping all other domains separate. > > I have setup: > > auth_username_format = %Lu > mail_location = /var/vmail/%d/%u/Maildir > > in userdb/passdb (passwd-file): > /etc/dovecot/%d/users > > Say I have two synonymous domain names > > my-domain-1.com > my-long-named-domain-1.com symlink the passdb of both domains together, as well as the userdb files. If you use the same file for both passdb and userdb, symlink just this single file. Then add the "user" ExtraField to all users in passdb and userdb. E.g. for user XYZ add user=XYZ at my-domain-1.com see: http://wiki2.dovecot.org/AuthDatabase/PasswdFile > > I want both to resolve to my-domain-1.com when %d variable is used in > Dovecot configs, so that > > passwd-file is /etc/dovecot/my-domain-1.com/users > mailbox is /var/vmail/my-domain-1.com/user/Maildir > > while keeping other domains unchanged > > my-other-domain-2.com > > should resolve to: > > passwd-file is /etc/dovecot/my-other-domain-2.com/users > mailbox is /var/vmail/my-other-domain-2.com/user/Maildir > > thanks, > Nikolai > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVhUZgHz1H7kL/d9rAQIp/Af/X27OvJWg6QjlWeIYS9m/JmEuyiWGkBlH hzdIr3moczMejv1829tYIJzMPM9LiDcysf/crMwCCF56AXLkDOfjENh9dUpd8pDx thOJ1xD1oHG+VdAh/+YV0IX2VKruu8lA+ZW0h5HshaDgBaXXshgXMkvi7TI8cOYd 1G9nVMbip7qsjeCNmynNMgtwTEBb4HiBNEvkr7aVl+L4AF4z8L130EpX2BMVNdTX xqQuF9GK6xy24noLXBIxoNpt28y0bHK04S43zyuErHvWoI9tBMJrzq4MLPJq8coZ FaUemvGW5sDLbb86PRGtFUTXG7RVmmOJVrvrBZSVHysHszdvB9douA== =xkTg -----END PGP SIGNATURE----- From tkpapp at gmail.com Wed Oct 7 13:09:17 2015 From: tkpapp at gmail.com (Tamas Papp) Date: Wed, 07 Oct 2015 15:09:17 +0200 Subject: bug report: dovecot-imapd core dump Message-ID: <87mvvuddvm.fsf@tamas.ihs.ac.at> Typical log message: Oct 7 11:39:36 hostname dovecot: imap(username): Fatal: master: service(imap): child 14864 killed with signal 11 (core dumped) What triggers the problem: User moving multiple messages to a subfolder via IMAP (from another account, on another server, client: Thunderbird). Moving single messages either does not trigger the problem, or with much lower probability (could not reproduce). Version: $ dpkg -l 'dovecot*' Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-=====================================-=======================-=======================-================================================================================ un dovecot-common (no description available) ii dovecot-core 1:2.2.9-1ubuntu5 amd64 secure POP3/IMAP server - core files un dovecot-gssapi (no description available) ii dovecot-imapd 1:2.2.9-1ubuntu5 amd64 secure POP3/IMAP server - IMAP daemon un dovecot-ldap (no description available) un dovecot-lmtpd (no description available) un dovecot-managesieved (no description available) un dovecot-mysql (no description available) un dovecot-pgsql (no description available) un dovecot-pop3d (no description available) un dovecot-sieve (no description available) un dovecot-solr (no description available) un dovecot-sqlite (no description available) tamas at szob:~$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 15.04 Release: 15.04 Codename: vivid Also tried 2.2.18 from Ubuntu wiley, same results. Backtrace (I am not sure this is useful, extraced core from Ubuntu's Apport): (gdb) bt full #0 0x00007f404e58e3b6 in mail_index_keywords_unref () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #1 0x00007f404ea254ad in _start () No symbol table info available. Any help would be appreciated, I am happy to provide more info if necessary. Tamas From tkpapp at gmail.com Wed Oct 7 14:21:07 2015 From: tkpapp at gmail.com (Tamas Papp) Date: Wed, 07 Oct 2015 16:21:07 +0200 Subject: bug report: dovecot-imapd core dump In-Reply-To: <1444226911.12007.7.camel@dominikbreu.de> References: <87mvvuddvm.fsf@tamas.ihs.ac.at> <1444224426.12007.6.camel@dominikbreu.de> <87lhbedbt7.fsf@tamas.ihs.ac.at> <1444226911.12007.7.camel@dominikbreu.de> Message-ID: <87k2qydajw.fsf@tamas.ihs.ac.at> Hi, could not find it -- back to the list, maybe someone can suggest a fix/workaround. Best, Tamas On Wed, Oct 07 2015, Dominik Breu wrote: > Hello, > > yeah this one looks familiar to me can you search the list back in > september there was a somewhat similar bug with thunderbird imho timo > posted a notice about it. > > greetings, > > dominik > > Am Mittwoch, den 07.10.2015, 15:53 +0200 schrieb Tamas Papp: >> Hi Dominik, >> >> Thanks for the help, the backtrace is here (I did not sent it to the >> list since I am not sure if it contains password or message >> information, >> sorry but I am unfamiliar with these things). >> >> Core was generated by `dovecot/imap'. >> Program terminated with signal SIGSEGV, Segmentation fault. >> #0 0x00007f404e58e3b6 in mail_index_keywords_unref >> #(_keywords=_keywords at entry=0x7ffecb8adeb8) at mail-index.c:383 >> 383 mail-index.c: No such file or directory. >> (gdb) bt full >> #0 0x00007f404e58e3b6 in mail_index_keywords_unref >> #(_keywords=_keywords at entry=0x7ffecb8adeb8) at mail-index.c:383 >> keywords = 0x7f404e27066d >> __FUNCTION__ = "mail_index_keywords_unref" >> #1 0x00007f404e559975 in mailbox_keywords_unref >> (keywords=keywords at entry=0x7ffecb8adeb8) at mailbox-keywords.c:94 >> No locals. >> #2 0x00007f404ea254ad in cmd_append_handle_args >> (nonsync_r=0x7ffecb8ade8f, >> #args=, cmd=0x7f4050bc3250) at cmd-append.c:602 >> ctx = 0x7f4050bc3348 >> flags_list = 0x7f4050bcc840 >> flags = 9 >> keywords_list = 0x7f4050bc33f0 >> internal_date_str = 0x7f4050bcca00 "17-Sep-2015 14:19:23 >> +0200" >> internal_date = -1 >> timezone_offset = 0 >> keywords = 0x0 >> ret = >> client = 0x7f4050bc2670 >> cat_list = 0x0 >> input = >> valid = >> #3 cmd_append_parse_new_msg (cmd=cmd at entry=0x7f4050bc3250) at >> cmd-append.c:770 >> client = >> ctx = 0x7f4050bc3348 >> args = 0x7f4050bcc488 >> msg = >> arg_min_count = >> fatal = 255 >> nonsync = true >> last_literal = >> ret = >> __FUNCTION__ = "cmd_append_parse_new_msg" >> #4 0x00007f404ea25a17 in cmd_append (cmd=0x7f4050bc3250) at >> cmd-append.c:932 >> client = 0x7f4050bc2670 >> ctx = 0x7f4050bc3348 >> mailbox = 0x7f4050bafb78 "Cemex" >> #5 0x00007f404ea30e4c in command_exec (cmd=cmd at entry=0x7f4050bc3250) >> at >> imap-commands.c:158 >> hook = 0x7f4050bad0d0 >> ret = >> #6 0x00007f404ea2fd62 in client_command_input (cmd=0x7f4050bc3250) >> at >> imap-client.c:780 >> client = 0x7f4050bc2670 >> command = >> __FUNCTION__ = "client_command_input" >> #7 0x00007f404ea2fe16 in client_command_input (cmd=0x7f4050bc3250) >> at >> imap-client.c:841 >> client = 0x7f4050bc2670 >> command = >> __FUNCTION__ = "client_command_input" >> #8 0x00007f404ea3014d in client_handle_next_command >> (remove_io_r=> pointer>, client=0x7f4050bc2670) at imap-client.c:879 >> ---Type to continue, or q to quit--- >> No locals. >> #9 client_handle_input (client=0x7f4050bc2670) at imap-client.c:891 >> _data_stack_cur_id = 3 >> remove_io = false >> handled_commands = false >> client = 0x7f4050bc2670 >> #10 0x00007f404ea304f5 in client_input (client=0x7f4050bc2670) at >> imap-client.c:933 >> cmd = 0xf013c >> output = 0x7f4050bc30a0 >> bytes = 135 >> __FUNCTION__ = "client_input" >> #11 0x00007f404e285247 in io_loop_call_io (io=0x7f4050bc3170) at >> ioloop.c:388 >> ioloop = 0x7f4050bac720 >> t_id = 2 >> #12 0x00007f404e286079 in io_loop_handler_run >> (ioloop=ioloop at entry=0x7f4050bac720) at ioloop-epoll.c:220 >> ctx = 0x7f4050bad3b0 >> io = >> tv = {tv_sec = 1799, tv_usec = 999282} >> events_count = >> msecs = >> ret = 1 >> i = 0 >> j = >> call = >> __FUNCTION__ = "io_loop_handler_run" >> #13 0x00007f404e284d88 in io_loop_run (ioloop=0x7f4050bac720) at >> ioloop.c:412 >> __FUNCTION__ = "io_loop_run" >> #14 0x00007f404e231de3 in master_service_run (service=0x7f4050bac5b0, >> callback=) at master-service.c:566 >> No locals. >> #15 0x00007f404ea2424e in main (argc=1, argv=0x7f4050bac390) at >> main.c:400 >> set_roots = {0x7f404ec489c0 , 0x0} >> login_set = {auth_socket_path = 0x7f4050ba4048 "\001", >> postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = >> 0x7f404ea39f40 , >> failure_callback = 0x7f404ea39c20 , >> request_auth_token = 1} >> service_flags = >> storage_service_flags = >> username = 0x0 >> c = >> (gdb) quit >> >> Best, >> >> Tamas >> >> On Wed, Oct 07 2015, Dominik Breu wrote: >> >> > Hello >> > >> > Tamas it would be a better backtace when you have installed the >> > dovecot >> > debug symboly mostly located under dovecote-dbg in apt. Further >> > debug >> > help can you find in the wiki. >> > >> > greetings >> > >> > dominik >> > >> > Am Mittwoch, den 07.10.2015, 15:09 +0200 schrieb Tamas Papp: >> > > Typical log message: >> > > >> > > Oct 7 11:39:36 hostname dovecot: imap(username): Fatal: master: >> > > service(imap): child 14864 killed with signal 11 (core dumped) >> > > >> > > What triggers the problem: >> > > >> > > User moving multiple messages to a subfolder via IMAP (from >> > > another >> > > account, on another server, client: Thunderbird). Moving single >> > > messages >> > > either does not trigger the problem, or with much lower >> > > probability >> > > (could not reproduce). >> > > >> > > Version: >> > > $ dpkg -l 'dovecot*' >> > > Desired=Unknown/Install/Remove/Purge/Hold >> > > > Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig >> > > > -aWait/Trig-pend >> > > > / Err?=(none)/Reinst-required (Status,Err: uppercase=bad) >> > > > > / Name Version >> > > > > Architecture Description >> > > +++-=====================================-======================= >> > > -======================= >> > > -================================================================ >> > > ==== >> > > ============ >> > > un dovecot-common >> > > (no description available) >> > > ii dovecot-core 1:2.2.9-1ubuntu5 >> > > amd64 secure POP3/IMAP server - core files >> > > un dovecot-gssapi >> > > (no description available) >> > > ii dovecot-imapd 1:2.2.9-1ubuntu5 >> > > amd64 secure POP3/IMAP server - IMAP daemon >> > > un dovecot-ldap >> > > (no description available) >> > > un dovecot-lmtpd >> > > (no description available) >> > > un dovecot-managesieved >> > > (no description available) >> > > un dovecot-mysql >> > > (no description available) >> > > un dovecot-pgsql >> > > (no description available) >> > > un dovecot-pop3d >> > > (no description available) >> > > un dovecot-sieve >> > > (no description available) >> > > un dovecot-solr >> > > (no description available) >> > > un dovecot-sqlite >> > > (no description available) >> > > tamas at szob:~$ lsb_release -a >> > > No LSB modules are available. >> > > Distributor ID: Ubuntu >> > > Description: Ubuntu 15.04 >> > > Release: 15.04 >> > > Codename: vivid >> > > >> > > Also tried 2.2.18 from Ubuntu wiley, same results. >> > > >> > > Backtrace (I am not sure this is useful, extraced core from >> > > Ubuntu's >> > > Apport): >> > > >> > > (gdb) bt full >> > > #0 0x00007f404e58e3b6 in mail_index_keywords_unref () from >> > > /usr/lib/dovecot/libdovecot-storage.so.0 >> > > No symbol table info available. >> > > #1 0x00007f404ea254ad in _start () >> > > No symbol table info available. >> > > >> > > Any help would be appreciated, I am happy to provide more info if >> > > necessary. >> > > >> > > Tamas From steffan at hldns.com Thu Oct 8 15:23:16 2015 From: steffan at hldns.com (Steffan Cline) Date: Thu, 08 Oct 2015 08:23:16 -0700 Subject: Dovecot plugin and Push Mail Message-ID: <098D974C-DF36-4BA0-A93C-A9DCFDADBDD2@hldns.com> I?m posting this here and will in a couple other places to see if anyone has any experience with this and knows what I can do. My setup is CentOS 6 with postfix, procmail and dovecot administered via Virtualmin. Postfix of course receives the email. Procmail uses it?s hooks to run the email through spam assassin and clamd. After than procmail hands it off to dovecot/deliver to drop it in the user?s mail box. I use the following plugin for dovecot with the daemon to provide Apple push mail for iPhones. https://github.com/st3fan/dovecot-xaps-daemon https://github.com/st3fan/dovecot-xaps-plugin For the most part, this works awesome but have run into a bit of a race issue with notification of mail in a particular instance. On my computer I use Outlook for Mac which says it supports IDLE but is broken. When an email comes in, I ALWAYS get the notification via the dovecpt plugin for a push since Outlook is checking IMAP every 2 minutes. On a buddy?s setup, he uses Apple Mail which supports IDLE that actually works. When his email client is connected from his laptop, it does actually maintain a persistent connection and NEVER receives a notification from the dovecot plugin because I?m guessing dovecot/deliver sees that the email was received by the desktop client. I?m not sure how to make the plugin ALWAYS give a push. Is it a dovecot config? A bug in the plugin? I?m not sure. Suggestions? Thank you, Steffan Cline 602-793-0014 From talpus at gmail.com Thu Oct 8 17:24:10 2015 From: talpus at gmail.com (Nikolai Krot) Date: Thu, 8 Oct 2015 19:24:10 +0200 Subject: how to treat synonymous domain names as one In-Reply-To: References: Message-ID: Hallo Thanks for your reply. I thought about using symlinks, i do not like this solution. Anyway, it is acceptable for paths to passdb/userdb. Thanks for the hits about user=xxx ! I will use it for paths to mail directories. Thanks again! Best regards, Nikolai On Wed, Oct 7, 2015 at 3:09 PM, Steffen Kaiser < skdovecot at smail.inf.fh-brs.de> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Wed, 7 Oct 2015, Nikolai Krot wrote: > > I am curious if there is a way to rewrite domain name to another domain >> name. My goal is to provide authentication and mailboxes for the >> synonymous >> domains in a single source while keeping all other domains separate. >> >> I have setup: >> >> auth_username_format = %Lu >> mail_location = /var/vmail/%d/%u/Maildir >> >> in userdb/passdb (passwd-file): >> /etc/dovecot/%d/users >> >> Say I have two synonymous domain names >> >> my-domain-1.com >> my-long-named-domain-1.com >> > > symlink the passdb of both domains together, as well as the userdb files. > If you use the same file for both passdb and userdb, symlink just this > single file. Then add the "user" ExtraField to all users in passdb and > userdb. E.g. for user XYZ add user=XYZ at my-domain-1.com > > see: > http://wiki2.dovecot.org/AuthDatabase/PasswdFile > > >> I want both to resolve to my-domain-1.com when %d variable is used in >> Dovecot configs, so that >> >> passwd-file is /etc/dovecot/my-domain-1.com/users >> mailbox is /var/vmail/my-domain-1.com/user/Maildir >> >> while keeping other domains unchanged >> >> my-other-domain-2.com >> >> should resolve to: >> >> passwd-file is /etc/dovecot/my-other-domain-2.com/users >> mailbox is /var/vmail/my-other-domain-2.com/user/Maildir >> >> thanks, >> Nikolai >> >> > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVhUZgHz1H7kL/d9rAQIp/Af/X27OvJWg6QjlWeIYS9m/JmEuyiWGkBlH > hzdIr3moczMejv1829tYIJzMPM9LiDcysf/crMwCCF56AXLkDOfjENh9dUpd8pDx > thOJ1xD1oHG+VdAh/+YV0IX2VKruu8lA+ZW0h5HshaDgBaXXshgXMkvi7TI8cOYd > 1G9nVMbip7qsjeCNmynNMgtwTEBb4HiBNEvkr7aVl+L4AF4z8L130EpX2BMVNdTX > xqQuF9GK6xy24noLXBIxoNpt28y0bHK04S43zyuErHvWoI9tBMJrzq4MLPJq8coZ > FaUemvGW5sDLbb86PRGtFUTXG7RVmmOJVrvrBZSVHysHszdvB9douA== > =xkTg > -----END PGP SIGNATURE----- > From hs at schlittermann.de Thu Oct 8 19:46:57 2015 From: hs at schlittermann.de (Heiko Schlittermann) Date: Thu, 8 Oct 2015 21:46:57 +0200 Subject: Dovecot auth-ldap ignores tls_* settings when using ldaps:// Message-ID: <20151008194657.GZ4156@jumper.schlittermann.de> Hi, I'm using dovecot 2.2.9 (but after checking src/auth/db-ldap.c in 2.2.13 there seems to be the same bug/feature). The userdb and passdb use LDAP. All further configuration is done in auth-ldap.conf.ext. uri = ldaps:/// # tls = tls_cert_file = /etc/ssl/certs/client-cert.pem tls_key_file = /etc/ssl/certs/client-key.file Dovecot ignores the tls_* options. If I use an ldap:// URI and switch on TLS using tls=yes it works as expected. But I do not see any reason why LDAPS should not read the tls_* settings. This small patch solved it for me --- dovecot-2.2.9/src/auth/db-ldap.c 2013-11-24 14:37:39.000000000 +0100 +++ dovecot-2.2.9.hs12/src/auth/db-ldap.c 2015-10-08 21:24:47.051446465 +0200 @@ -1043,7 +1043,7 @@ static void db_ldap_set_tls_options(struct ldap_connection *conn) { - if (!conn->set.tls) + if (!(conn->set.tls || strncmp(conn->set.uris, "ldaps:", 6) == 0)) return; #ifdef OPENLDAP_TLS_OPTIONS It would be great, if somebody can confirm this and if this or some equivalent patch could make it upstream. Best regards from Dresden/Germany Viele Gr??e aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From steffan at hldns.com Thu Oct 8 19:50:22 2015 From: steffan at hldns.com (Steffan Cline) Date: Thu, 08 Oct 2015 12:50:22 -0700 Subject: Dovecot plugin and Push Mail In-Reply-To: <098D974C-DF36-4BA0-A93C-A9DCFDADBDD2@hldns.com> References: <098D974C-DF36-4BA0-A93C-A9DCFDADBDD2@hldns.com> Message-ID: <099C9CD4-A850-4545-8D21-27D6015EC311@hldns.com> I figured this might be handy for anyone to look over. # doveconf -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-71.29.1.el6.x86_64 x86_64 CentOS release 6.7 (Final) auth_mechanisms = plain login disable_plaintext_auth = no mail_debug = yes mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve xaps_socket = /var/run/xapsd/xapsd.sock } postmaster_address = postmaster at hldns.com protocols = imap pop3 sieve service managesieve-login { inet_listener sieve { port = 4190 } } ssl_ca = wrote: >I?m posting this here and will in a couple other places to see if anyone has any experience with this and knows what I can do. > >My setup is CentOS 6 with postfix, procmail and dovecot administered via Virtualmin. > >Postfix of course receives the email. Procmail uses it?s hooks to run the email through spam assassin and clamd. After than procmail hands it off to dovecot/deliver to drop it in the user?s mail box. > >I use the following plugin for dovecot with the daemon to provide Apple push mail for iPhones. > >https://github.com/st3fan/dovecot-xaps-daemon > >https://github.com/st3fan/dovecot-xaps-plugin > >For the most part, this works awesome but have run into a bit of a race issue with notification of mail in a particular instance. > >On my computer I use Outlook for Mac which says it supports IDLE but is broken. When an email comes in, I ALWAYS get the notification via the dovecpt plugin for a push since Outlook is checking IMAP every 2 minutes. > >On a buddy?s setup, he uses Apple Mail which supports IDLE that actually works. When his email client is connected from his laptop, it does actually maintain a persistent connection and NEVER receives a notification from the dovecot plugin because I?m guessing dovecot/deliver sees that the email was received by the desktop client. > >I?m not sure how to make the plugin ALWAYS give a push. Is it a dovecot config? A bug in the plugin? I?m not sure. > >Suggestions? > > >Thank you, >Steffan Cline >602-793-0014 > From mw at dermichi.com Fri Oct 9 09:47:49 2015 From: mw at dermichi.com (Michael Weissenbacher) Date: Fri, 9 Oct 2015 11:47:49 +0200 Subject: Double Dots In-Reply-To: <560BEDF9.6090800@mathcs.emory.edu> References: <560BEDF9.6090800@mathcs.emory.edu> Message-ID: <56178D45.10903@dermichi.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hello Ken! > I don't think its Thunderbird adding the dot, since K9 mail on my > Android phone shows the same issue. It's only after the IMAP > delivery that the extra dot appears. > I've seen similar behavior with some buggy versions of amavis in ancient times (like more than 5 years ago). I wouldn't suspect dovecot to do anything like that. With kind regards, Michael -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJWF41EAAoJEIxwm4JXrtWEHjwP/jggKQWmj2Uyf2SHpQcT1VVB aggOiN78rnvsiWzBL2cBbdwuAMGWvcJYdtXL04fesy626EMm0YtAxJZ+t1ob7Y/4 lpv8K1CdqFj0P45CS/22Ku8LVkW+4UC0GAJfAp+3DnkSCNG3Sz2XYRlfwxuZZ4G5 KKI7PB4qB0i+cldM6oo+fQFojn6eYTAr4iyIWXe+8cyc59AKrjEreyfs7buGCtmK ExGvTc6s6n6JLYEmoWb8dYPcY+ZY8dKB4+t3tWOSlDYxIRE94cd2B/6mUdbDdoFa X615ZNlUWERhpjBF757c2SOUTZnD+xvLa9+UVf2XEhXLT4FShPnjyxpEtkTpsCFT 7HX+nienuKOqIs8iY6bcMqDnF7SzEK+vaVLFNnFsRZM29J/lrar1btkSYC3joFJQ wjRz5nKWGg5AWlO/9B+mRFVfSYBXX4ws9LkeRy79pRGwX49o7Ng7ghbp29MAytKC WB38SPCYYRFYEzJVi5pP9EG85Ns1RiYuzglMbcp4VPRv88jtBwkF0BxCHjAvQ8RF Jy5tiFAhJMnQEalZFaGVa5lxLS0qpa6b8a78XK7eLStN6rtrpCXdblDUK73JxVio WMfRnTAjfYULRy/AmyktUMSiMtpksOfDmK+ZpddwpCUo/rMNwO24Qh1i6u+wrRtt cFhsz2o+F8Hjn73Cm1Gg =1+lp -----END PGP SIGNATURE----- From nathan at bravenet.com Fri Oct 9 17:33:55 2015 From: nathan at bravenet.com (Nathan Coulson) Date: Fri, 9 Oct 2015 10:33:55 -0700 Subject: Questions on supporting Shared Mailboxes using imapc:// Message-ID: <5617FA83.9060404@bravenet.com> I was not able to get a shared mailbox setup working using imapc. Tested using mailboxes on the same server, as well as other servers. We are using dovecot 2.2.10 from Centos 7, following the guide at http://wiki2.dovecot.org/SharedMailboxes/ClusterSetup imapc_host=192.168.5.5 #(Local dovecot server) imapc_master_user= %u imapc_password=TempPass #imapc_user=test2 at example.com # Added for testing. namespace { type = shared separator = / prefix = shared/%%u/ list = children location = imapc:~/shared/%%u/ } namespace { # used as a control to verify that shared mailboxes work type = shared separator = / prefix = shared2/%%u/ list = children location = sdbox:%%h:INDEXPVT=~/shared2/%%u } dict { acl = mysql:/etc/dovecot/dovecot-sql_aggelos_shares.conf.ext } plugin { acl = vfile acl_shared_dict = proxy::acl } mail_plugins = acl protocol imap { mail_plugins = acl imap_acl } passdb { driver = sql args = /etc/dovecot/dovecot-sql.conf.ext } passdb { # for master user logins driver = sql args = /etc/dovecot/dovecot-sql_aggelos_aclmaster.conf.ext master = yes pass = yes } userdb { driver = sql args = /etc/dovecot/dovecot-sql.conf.ext } I can login using test2 at example.com*test at example.com (and see all of test2's email), and it looks like it authenticates ok via imap, but shared never shows up. No issues using shared2 which uses direct access to the mailbox doveadm acl debug -u test at example.com shared/test2 at example.com (Fails, uses imapc) doveadm(test at example.com): Info: Mailbox 'INBOX' is in namespace 'shared/test2 at example.com/' doveadm(test at example.com): Info: Mailbox path: /misc/1/2/mail/test at example.com-4/shared/test2 at example.com/.INBOX doveadm(test at example.com): Info: All message flags are shared across users in mailbox doveadm(test at example.com): Info: User test at example.com has no rights for mailbox doveadm(test at example.com): Error: User test at example.com is missing 'lookup' right doveadm(test at example.com): Info: Mailbox shared/test2 at example.com is NOT visible in LIST doveadm acl debug -u test at example.com shared/test2 at example.com/Junk (Fails, contains a folder that exists) doveadm(test at example.com): Info: Mailbox 'Junk' is in namespace 'shared/test2 at example.com/' doveadm(test at example.com): Info: Mailbox path: /misc/1/2/mail/test at example.com-4/shared/test2 at example.com/.Junk doveadm(test at example.com): Info: All message flags are shared across users in mailbox doveadm(test at example.com): Info: User test at example.com has no rights for mailbox doveadm(test at example.com): Error: User test at example.com is missing 'lookup' right doveadm(test at example.com): Info: Mailbox shared/test2 at example.com/Junk is NOT visible in LIST doveadm acl debug -u test at example.com shared/test2 at example.com/z (Fails, contains a folder that does not exist. Expected result) doveadm(test at example.com): Error: Mailbox 'z' in namespace 'shared/test2 at example.com/' doesn't exist in /misc/1/2/mail/test at example.com-4/shared/test2 at example.com/.z doveadm acl debug -u test at example.com shared2/test2 at example.com (Works, using direct storage) doveadm(test at example.com): Info: Mailbox 'INBOX' is in namespace 'shared2/test2 at example.com/' doveadm(test at example.com): Info: Mailbox path: /misc/1/2/mail/test2 at example.com-87/mailboxes/INBOX/dbox-Mails doveadm(test at example.com): Info: Per-user private flags in mailbox: \Seen doveadm(test at example.com): Info: User test at example.com has rights: lookup read write-seen doveadm(test at example.com): Info: Mailbox found from dovecot-acl-list doveadm(test at example.com): Info: User test2 at example.com found from ACL shared dict doveadm(test at example.com): Info: Mailbox shared2/test2 at example.com is visible in LIST It feels like the acl information in the vfiles is not available when using imapc. Our setup is as follows: (Currently does not use Dovecot Director, but has the same issues where shared mailboxes may not be local on the server) * Front servers, that are running dovecot (proxy to backend servers) * Backend Servers, each handle their own local users. Thank you -- Nathan Coulson System Administrator for Bravenet www.bravenet.com nathan at bravenet.com From gayleard at eircom.net Sat Oct 10 11:22:19 2015 From: gayleard at eircom.net (Timothy Murphy) Date: Sat, 10 Oct 2015 13:22:19 +0200 Subject: Syncronizing two IMAP servers Message-ID: <3746968.ARlfglcyC8@william.gayleard.com> I spend time in two places, in different countries. I have a local network in each place, with IMAP servers on each network. I collect email by fetchmail from various servers, and read email on my laptop with KMail. I tried using just one IMAP server, but this proved impractical. So I have two collections of the same sub-folders, on the two servers. I have two questions: 1. Is there a simpler way of organizing my email? 2. Given the current method, is there a way of syncronizing folders on the two servers? -- Timothy Murphy gayleard /at/ eircom.net School of Mathematics, Trinity College, Dublin From allen at vo.lu Sat Oct 10 11:46:24 2015 From: allen at vo.lu (John Allen) Date: Sat, 10 Oct 2015 13:46:24 +0200 Subject: Syncronizing two IMAP servers In-Reply-To: <3746968.ARlfglcyC8@william.gayleard.com> References: <3746968.ARlfglcyC8@william.gayleard.com> Message-ID: <5618FA90.2040701@vo.lu> On 10/10/2015 13:22, Timothy Murphy wrote: > I spend time in two places, in different countries. > I have a local network in each place, > with IMAP servers on each network. > I collect email by fetchmail from various servers, > and read email on my laptop with KMail. > > I tried using just one IMAP server, but this proved impractical. > So I have two collections of the same sub-folders, on the two servers. > > I have two questions: > 1. Is there a simpler way of organizing my email? Why was it impractical? Remote access to a single IMAP server in one location ought to be possible. We spend extended periods in several countries. I keep all my email (collected via fetchmail/exim) on an IMAP server (dovecot) at home and it's never been a problem to access it remotely from different countries and different devices. > 2. Given the current method, is there a way of syncronizing folders > on the two servers? If you are using plain maildir, then I think you could just use rsync to synchronise the two sets of folders and files. I guess you would need to exclude from the rsync operation the dovecot-specific files (indexes and so forth) and allow dovecot to update these itself to reflect the new state of each set of stored messages. -- John Allen Bofferdange, Luxembourg allen at vo.lu From lists at marzocchi.net Sat Oct 10 16:09:05 2015 From: lists at marzocchi.net (Olaf Marzocchi) Date: Sat, 10 Oct 2015 18:09:05 +0200 Subject: Maildir: ACLs/Unix perms: unlink(...) failed: Permission denied In-Reply-To: <56086833.7090507@marzocchi.net> References: <55FD6E91.7020505@marzocchi.net> <839515024ef34c25a9bbe682a454855c@valo.at> <56086833.7090507@marzocchi.net> Message-ID: <56193821.7010503@marzocchi.net> I solved the issue I mentioned some days ago. I checked in the logs the datethe issue appeared, and I noticed it did not correspond to a dovecot update, dovecot was not the culprit. The date also did not correspond to a update of OmniOS, and in any case the previous OmniOS update contained only userland updates. Since the issue appeared when I assigned for the first time ACLs to my home folder on the fileserver to make it better compatible with SMB sharing, I decided the easiest way was to start a new ZFS dataset only for mail, splitting home folder and mail. $ zfs create -o compression=on tank/mail $ chgrp mail /tank/mail $ mkdir /tank/mail/olaf $ mv /tank/home/olaf/Maildir /tank/mail/olaf/ $ chown -R olaf:olaf /tank/mail/olaf $ find Maildir -type d -exec chmod 700 {} \; $ find Maildir -type f -exec chmod 600 {} \; $ svcadm enable dovecot This time in the dataset I did not set the options: -o aclinherit=passthrough-x -o aclmode=passthrough because dovecot does not need ACL anyway. I'm not even sure those two options are what I actually need, but the server is running so I won't change them. Anyway, the server is running fine now. I'm not sure why I cannot see in Thunderbird any folder "Trash" but if I try to create one it fails with "Folder already existing", but I will find out. I also wrote a summary of the issue and of the solution here, because other people had the same problem in the past (http://www.dovecot.org/list/dovecot/2013-November/093778.html) and there was no solution posted. http://www.marzocchi.net/Olafsen/Software/InstallationOfOmniOSAndBasicSetup Cheers, Olaf On 28/09/2015 00:05, Olaf Marzocchi wrote: > Hi, > I tried again with some other options. > > After finding > http://www.dovecot.org/list/dovecot/2013-November/093793.html > I deleted every ACL from the directory Maildir and I also assigned the > group "mail" to it, recursively: > > OmniOS-Xeon:/tank/home/olaf/Maildir/.Generiche $ ls -lV > total 903 > drwxrwxrwx 2 olaf mail 2 Sep 27 23:47 cur > owner@:rwxp--aARWcCos:-------:allow > group@:rwxp--a-R-c--s:-------:allow > everyone@:rwxp--a-R-c--s:-------:allow > (and so on) > > I tried also > mail_full_filesystem_access = yes > hoping that it would solve the issue, but nothing. Even with > mail_debug = yes > the log does not give any info besides > dovecot: [ID 583609 mail.error] imap(olaf): Error: > unlink(/tank/home/olaf/Maildir/.Generiche/dovecot-uidlist.tmp) failed: > Permission denied > > (it shows also "rename" instead of "unlink") > > With these additional info, has anyone any idea about the cause of the > problem? > > My doveconf -n: > > # 2.2.18: /etc/dovecot/dovecot.conf > # OS: SunOS 5.11 i86pc zfs > mail_debug = yes > mail_full_filesystem_access = yes > mail_location = maildir:/tank/home/%u/Maildir > mail_privileged_group = mail > namespace inbox { > inbox = yes > location = > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > prefix = > } > passdb { > driver = pam > } > protocols = imap > ssl = required > ssl_cert = ssl_key = userdb { > driver = passwd > } > > > Any help will be appreciated. > > Regards, > Olaf Marzocchi > > > > > On 19/09/2015 19:22, Christian Kivalo wrote: >> Hi, >> >> On 2015-09-19 16:17, Olaf Marzocchi wrote: >>> Dear Dovecot users, hello. >>> I will merge two issues I have into a single email because they may be >>> related. >>> >>> I used dovecot on a OmniOS server since 2014 (currently OmniOS >>> r151014) with the following configuration (it shows 2.2.18 because I >>> recently updated dovecot, skipping only the PostgreSQL plugin): >>> >>> # 2.2.18: /etc/dovecot/dovecot.conf >>> # OS: SunOS 5.11 i86pc zfs >>> mail_location = maildir:/tank/home/%u/Maildir >>> mail_privileged_group = mail >>> namespace inbox { >>> inbox = yes >>> location = >>> mailbox Drafts { >>> special_use = \Drafts >>> } >>> mailbox Junk { >>> special_use = \Junk >>> } >>> mailbox Sent { >>> special_use = \Sent >>> } >>> mailbox "Sent Messages" { >>> special_use = \Sent >>> } >>> mailbox Trash { >>> special_use = \Trash >>> } >>> prefix = >>> } >>> passdb { >>> driver = pam >>> } >>> protocols = imap >>> ssl = required >>> ssl_cert = >> ssl_key = >> userdb { >>> driver = passwd >>> } >>> >>> You can see that I set the Maildir folder inside the shared home >>> folders of my server (it is only one user, anyway). >>> It always worked perfectly, but one-two months ago I changed the >>> permissions of my whole home folder, recursively, to add proper ACLs. >>> I needed them because the clients started using illumos kernel SMB >>> (relying on ACLs) instead of Netatalk/AFP (relying on Unix perms >>> only). >>> I didn't realise I applied the ACLs also to the Maildir folder. >>> >>> Dovecot worked for several weeks fine, I noticed the issue only >>> yesterday when a mailbox (see below) appeared in Thunderbird >>> completely empty even if the "cur" subfolder on the server still >>> contains all the mails. >>> >>> Dovecot was throwing some errors like: >>> >>> dovecot: [ID 583609 mail.error] imap(olaf): Error: >>> rename(/tank/home/olaf/Maildir/.&A6k- Mailing >>> Lists.Log/dovecot.index.cache) failed: Permission denied >>> (euid=501(olaf) egid=501(olaf) UNIX perms appear ok (ACL/MAC wrong?)) >>> dovecot: [ID 583609 mail.error] imap(olaf): Error: >>> rename(/tank/home/olaf/Maildir/.&A6k- Mailing >>> Lists.Log/dovecot.index.tmp, /tank/home/olaf/Maildir/.&A6k- Mailing >>> Lists.Log/dovecot.index) failed: Permission denied >>> dovecot: [ID 583609 mail.error] imap(olaf): Error: >>> unlink(/tank/home/olaf/Maildir/subscriptions.lock) failed: Permission >>> denied >>> dovecot: [ID 583609 mail.error] imap(olaf): Error: >>> rename(/tank/home/olaf/Maildir/subscriptions.lock, >>> /tank/home/olaf/Maildir/subscriptions) failed: Permission denied >>> >>> I will post here the current permissions of the folder containing >>> Maildir, of the Maildir itself, of its contents, and of the folder >>> that appears empty when browsed with a client (Thunderbird). >>> >>> /tank/home/olaf $ ls -lV .. >>> drwx------+ 16 olaf olaf 17 Sep 19 01:52 olaf >>> user:olaf:rwxpdDaARWcCos:fd-----:allow >>> group:2147483648:rwxpdDaARWcCos:fd-----:allow >>> everyone@:rwxpdDaARWcCos:fd-----:deny >>> >>> /tank/home/olaf $ ls -lV >>> drwxrwx--- 348 olaf olaf 359 Sep 19 01:51 Maildir >>> owner@:rwxp--aARWcCos:-------:allow >>> group@:rwxp--a-R-c--s:-------:allow >>> everyone@:------a-R-c--s:-------:allow >>> >>> /tank/home/olaf $ ls -lV Maildir/ >>> drwxrwx--- 2 olaf olaf 2 Jan 30 2014 cur >>> owner@:rwxp--aARWcCos:-------:allow >>> group@:rwxp--a-R-c--s:-------:allow >>> everyone@:------a-R-c--s:-------:allow >>> -rwxrwx--- 1 olaf olaf 21 Jan 30 2014 dovecot-keywords >>> owner@:rwxp--aARWcCos:-------:allow >>> group@:rwxp--a-R-c--s:-------:allow >>> everyone@:------a-R-c--s:-------:allow >>> (ALL THE SAME PERMISSIONS FOR THE OTHER FILES EXCEPT...) >>> -rwxrwx--- 1 olaf olaf 13735 Jan 24 2015 subscriptions >>> owner@:rwxp--aARWcCos:-------:allow >>> group@:rwxp--a-R-c--s:-------:allow >>> everyone@:------a-R-c--s:-------:allow >>> -rw-rw---- 1 olaf olaf 13709 Sep 19 01:51 subscriptions.lock >>> owner@:rw-p--aARWcCos:-------:allow >>> group@:rw-p--a-R-c--s:-------:allow >>> everyone@:------a-R-c--s:-------:allow >>> >>> The folder that appears empty: >>> >>> /tank/home/olaf $ ls -lV Maildir/.Generiche/ >>> total 513 >>> drwxrwx--- 2 olaf olaf 949 Sep 18 01:42 cur >>> owner@:rwxp--aARWcCos:-------:allow >>> group@:rwxp--a-R-c--s:-------:allow >>> everyone@:------a-R-c--s:-------:allow >>> -rwxrwx--- 1 olaf olaf 46 May 18 2014 dovecot-keywords >>> owner@:rwxp--aARWcCos:-------:allow >>> group@:rwxp--a-R-c--s:-------:allow >>> everyone@:------a-R-c--s:-------:allow >>> (ALL THE SAME PERMISSIONS FOR THE OTHER FILES) >>> >>> >>> I really hope you will have the time to help me because I already >>> applied the permissions recursively and I removed the ACLs, almost as >>> it was before my mistake. >>> I specified "almost" because originally (I checked the backups) the >>> Maildir folder had an ACL that gave access permissions also to the >>> group "mail": >>> >>> drwxrwx---+349 olaf olaf 359 Feb 16 2014 Maildir >>> group:mail:rwxpdDaARWcCos:fd-----:allow >>> owner@:rwxpdDaARWcCos:fd----I:allow >>> group@:rwxpdDaARWcCos:fd----I:allow >>> everyone@:rwxpdDaARWcCos:fd----I:deny >>> >>> Yesterday I haven't replicated it because from the documentation I >>> understood it was not necessary. >> >> From my view the permissions seem to be set correctly, i have to admin, >> its been a while since i moved to virtual users so i may be wrong here... >> >> The log output also seems to support that permissions are correct. >> >> Have you tried adding the group:mail:.... ACLs back? >> >> Have you set mail_debug=yes or other more verbose logging settings? >> http://wiki2.dovecot.org/Logging From tkpapp at gmail.com Sat Oct 10 17:34:23 2015 From: tkpapp at gmail.com (Tamas Papp) Date: Sat, 10 Oct 2015 19:34:23 +0200 Subject: bug report: dovecot-imapd core dump In-Reply-To: <87k2qydajw.fsf@tamas.ihs.ac.at> References: <87mvvuddvm.fsf@tamas.ihs.ac.at> <1444224426.12007.6.camel@dominikbreu.de> <87lhbedbt7.fsf@tamas.ihs.ac.at> <1444226911.12007.7.camel@dominikbreu.de> <87k2qydajw.fsf@tamas.ihs.ac.at> Message-ID: <87mvvqbpb4.fsf@tamas.ihs.ac.at> Hi, Since I did not get any specific suggestions about this bug, I am attaching another backtrace. Any suggestions for workarounds etc would be appreciated -- should I regenerate the index? Or install a dovecot from the repo where this might be fixed? Is this a known bug? Best, Tamas -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-gdb Type: application/octet-stream Size: 4375 bytes Desc: not available URL: -------------- next part -------------- On Wed, Oct 07 2015, Tamas Papp wrote: > Hi, > > could not find it -- back to the list, maybe someone can suggest a > fix/workaround. > > Best, > > Tamas > > On Wed, Oct 07 2015, Dominik Breu wrote: > >> Hello, >> >> yeah this one looks familiar to me can you search the list back in >> september there was a somewhat similar bug with thunderbird imho timo >> posted a notice about it. >> >> greetings, >> >> dominik >> >> Am Mittwoch, den 07.10.2015, 15:53 +0200 schrieb Tamas Papp: >>> Hi Dominik, >>> >>> Thanks for the help, the backtrace is here (I did not sent it to the >>> list since I am not sure if it contains password or message >>> information, >>> sorry but I am unfamiliar with these things). >>> >>> Core was generated by `dovecot/imap'. >>> Program terminated with signal SIGSEGV, Segmentation fault. >>> #0 0x00007f404e58e3b6 in mail_index_keywords_unref >>> #(_keywords=_keywords at entry=0x7ffecb8adeb8) at mail-index.c:383 >>> 383 mail-index.c: No such file or directory. >>> (gdb) bt full >>> #0 0x00007f404e58e3b6 in mail_index_keywords_unref >>> #(_keywords=_keywords at entry=0x7ffecb8adeb8) at mail-index.c:383 >>> keywords = 0x7f404e27066d >>> __FUNCTION__ = "mail_index_keywords_unref" >>> #1 0x00007f404e559975 in mailbox_keywords_unref >>> (keywords=keywords at entry=0x7ffecb8adeb8) at mailbox-keywords.c:94 >>> No locals. >>> #2 0x00007f404ea254ad in cmd_append_handle_args >>> (nonsync_r=0x7ffecb8ade8f, >>> #args=, cmd=0x7f4050bc3250) at cmd-append.c:602 >>> ctx = 0x7f4050bc3348 >>> flags_list = 0x7f4050bcc840 >>> flags = 9 >>> keywords_list = 0x7f4050bc33f0 >>> internal_date_str = 0x7f4050bcca00 "17-Sep-2015 14:19:23 >>> +0200" >>> internal_date = -1 >>> timezone_offset = 0 >>> keywords = 0x0 >>> ret = >>> client = 0x7f4050bc2670 >>> cat_list = 0x0 >>> input = >>> valid = >>> #3 cmd_append_parse_new_msg (cmd=cmd at entry=0x7f4050bc3250) at >>> cmd-append.c:770 >>> client = >>> ctx = 0x7f4050bc3348 >>> args = 0x7f4050bcc488 >>> msg = >>> arg_min_count = >>> fatal = 255 >>> nonsync = true >>> last_literal = >>> ret = >>> __FUNCTION__ = "cmd_append_parse_new_msg" >>> #4 0x00007f404ea25a17 in cmd_append (cmd=0x7f4050bc3250) at >>> cmd-append.c:932 >>> client = 0x7f4050bc2670 >>> ctx = 0x7f4050bc3348 >>> mailbox = 0x7f4050bafb78 "Cemex" >>> #5 0x00007f404ea30e4c in command_exec (cmd=cmd at entry=0x7f4050bc3250) >>> at >>> imap-commands.c:158 >>> hook = 0x7f4050bad0d0 >>> ret = >>> #6 0x00007f404ea2fd62 in client_command_input (cmd=0x7f4050bc3250) >>> at >>> imap-client.c:780 >>> client = 0x7f4050bc2670 >>> command = >>> __FUNCTION__ = "client_command_input" >>> #7 0x00007f404ea2fe16 in client_command_input (cmd=0x7f4050bc3250) >>> at >>> imap-client.c:841 >>> client = 0x7f4050bc2670 >>> command = >>> __FUNCTION__ = "client_command_input" >>> #8 0x00007f404ea3014d in client_handle_next_command >>> (remove_io_r=>> pointer>, client=0x7f4050bc2670) at imap-client.c:879 >>> ---Type to continue, or q to quit--- >>> No locals. >>> #9 client_handle_input (client=0x7f4050bc2670) at imap-client.c:891 >>> _data_stack_cur_id = 3 >>> remove_io = false >>> handled_commands = false >>> client = 0x7f4050bc2670 >>> #10 0x00007f404ea304f5 in client_input (client=0x7f4050bc2670) at >>> imap-client.c:933 >>> cmd = 0xf013c >>> output = 0x7f4050bc30a0 >>> bytes = 135 >>> __FUNCTION__ = "client_input" >>> #11 0x00007f404e285247 in io_loop_call_io (io=0x7f4050bc3170) at >>> ioloop.c:388 >>> ioloop = 0x7f4050bac720 >>> t_id = 2 >>> #12 0x00007f404e286079 in io_loop_handler_run >>> (ioloop=ioloop at entry=0x7f4050bac720) at ioloop-epoll.c:220 >>> ctx = 0x7f4050bad3b0 >>> io = >>> tv = {tv_sec = 1799, tv_usec = 999282} >>> events_count = >>> msecs = >>> ret = 1 >>> i = 0 >>> j = >>> call = >>> __FUNCTION__ = "io_loop_handler_run" >>> #13 0x00007f404e284d88 in io_loop_run (ioloop=0x7f4050bac720) at >>> ioloop.c:412 >>> __FUNCTION__ = "io_loop_run" >>> #14 0x00007f404e231de3 in master_service_run (service=0x7f4050bac5b0, >>> callback=) at master-service.c:566 >>> No locals. >>> #15 0x00007f404ea2424e in main (argc=1, argv=0x7f4050bac390) at >>> main.c:400 >>> set_roots = {0x7f404ec489c0 , 0x0} >>> login_set = {auth_socket_path = 0x7f4050ba4048 "\001", >>> postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = >>> 0x7f404ea39f40 , >>> failure_callback = 0x7f404ea39c20 , >>> request_auth_token = 1} >>> service_flags = >>> storage_service_flags = >>> username = 0x0 >>> c = >>> (gdb) quit >>> >>> Best, >>> >>> Tamas >>> >>> On Wed, Oct 07 2015, Dominik Breu wrote: >>> >>> > Hello >>> > >>> > Tamas it would be a better backtace when you have installed the >>> > dovecot >>> > debug symboly mostly located under dovecote-dbg in apt. Further >>> > debug >>> > help can you find in the wiki. >>> > >>> > greetings >>> > >>> > dominik >>> > >>> > Am Mittwoch, den 07.10.2015, 15:09 +0200 schrieb Tamas Papp: >>> > > Typical log message: >>> > > >>> > > Oct 7 11:39:36 hostname dovecot: imap(username): Fatal: master: >>> > > service(imap): child 14864 killed with signal 11 (core dumped) >>> > > >>> > > What triggers the problem: >>> > > >>> > > User moving multiple messages to a subfolder via IMAP (from >>> > > another >>> > > account, on another server, client: Thunderbird). Moving single >>> > > messages >>> > > either does not trigger the problem, or with much lower >>> > > probability >>> > > (could not reproduce). >>> > > >>> > > Version: >>> > > $ dpkg -l 'dovecot*' >>> > > Desired=Unknown/Install/Remove/Purge/Hold >>> > > > Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig >>> > > > -aWait/Trig-pend >>> > > > / Err?=(none)/Reinst-required (Status,Err: uppercase=bad) >>> > > > > / Name Version >>> > > > > Architecture Description >>> > > +++-=====================================-======================= >>> > > -======================= >>> > > -================================================================ >>> > > ==== >>> > > ============ >>> > > un dovecot-common >>> > > (no description available) >>> > > ii dovecot-core 1:2.2.9-1ubuntu5 >>> > > amd64 secure POP3/IMAP server - core files >>> > > un dovecot-gssapi >>> > > (no description available) >>> > > ii dovecot-imapd 1:2.2.9-1ubuntu5 >>> > > amd64 secure POP3/IMAP server - IMAP daemon >>> > > un dovecot-ldap >>> > > (no description available) >>> > > un dovecot-lmtpd >>> > > (no description available) >>> > > un dovecot-managesieved >>> > > (no description available) >>> > > un dovecot-mysql >>> > > (no description available) >>> > > un dovecot-pgsql >>> > > (no description available) >>> > > un dovecot-pop3d >>> > > (no description available) >>> > > un dovecot-sieve >>> > > (no description available) >>> > > un dovecot-solr >>> > > (no description available) >>> > > un dovecot-sqlite >>> > > (no description available) >>> > > tamas at szob:~$ lsb_release -a >>> > > No LSB modules are available. >>> > > Distributor ID: Ubuntu >>> > > Description: Ubuntu 15.04 >>> > > Release: 15.04 >>> > > Codename: vivid >>> > > >>> > > Also tried 2.2.18 from Ubuntu wiley, same results. >>> > > >>> > > Backtrace (I am not sure this is useful, extraced core from >>> > > Ubuntu's >>> > > Apport): >>> > > >>> > > (gdb) bt full >>> > > #0 0x00007f404e58e3b6 in mail_index_keywords_unref () from >>> > > /usr/lib/dovecot/libdovecot-storage.so.0 >>> > > No symbol table info available. >>> > > #1 0x00007f404ea254ad in _start () >>> > > No symbol table info available. >>> > > >>> > > Any help would be appreciated, I am happy to provide more info if >>> > > necessary. >>> > > >>> > > Tamas From lists at marzocchi.net Sat Oct 10 17:54:53 2015 From: lists at marzocchi.net (Olaf Marzocchi) Date: Sat, 10 Oct 2015 19:54:53 +0200 Subject: Syncronizing two IMAP servers In-Reply-To: <5618FA90.2040701@vo.lu> References: <3746968.ARlfglcyC8@william.gayleard.com> <5618FA90.2040701@vo.lu> Message-ID: <561950ED.9040401@marzocchi.net> On 10/10/2015 13:46, John Allen wrote: > If you are using plain maildir, then I think you could just use rsync to > synchronise the two sets of folders and files. I guess you would need to > exclude from the rsync operation the dovecot-specific files (indexes and > so forth) and allow dovecot to update these itself to reflect the new > state of each set of stored messages. Another option would be to try imapfilter. Olaf From djk at tobit.co.uk Sat Oct 10 20:56:25 2015 From: djk at tobit.co.uk (djk) Date: Sat, 10 Oct 2015 21:56:25 +0100 Subject: Syncronizing two IMAP servers In-Reply-To: <3746968.ARlfglcyC8@william.gayleard.com> References: <3746968.ARlfglcyC8@william.gayleard.com> Message-ID: <56197B79.8080909@tobit.co.uk> On 10/10/15 12:22, Timothy Murphy wrote: > I spend time in two places, in different countries. > I have a local network in each place, > with IMAP servers on each network. > I collect email by fetchmail from various servers, > and read email on my laptop with KMail. > > I tried using just one IMAP server, but this proved impractical. > So I have two collections of the same sub-folders, on the two servers. > > I have two questions: > 1. Is there a simpler way of organizing my email? > 2. Given the current method, is there a way of syncronizing folders > on the two servers? > You can use bi-directional replication: http://wiki2.dovecot.org/Replication I personally use the SSL over TCP method and it works find *provided* you have a valid (ie non-self signed or otherwise correctly installed private CA) certificate to use. Dirk From dmiller at amfes.com Sun Oct 11 05:42:18 2015 From: dmiller at amfes.com (Daniel Miller) Date: Sat, 10 Oct 2015 22:42:18 -0700 Subject: Multiple core solr support Message-ID: Is it possible to use variables, such as domain or username, in the plugin url declaration? This would allow using multiple cores to separate the stored indexes. So when my own giant mailbox gets corrupt...I don't have to rebuild everybody's... Something like url=http://mysolr.server.com:8983/solr/%u -- Daniel From hs at schlittermann.de Sun Oct 11 17:04:19 2015 From: hs at schlittermann.de (Heiko Schlittermann) Date: Sun, 11 Oct 2015 19:04:19 +0200 Subject: dovecot as proxy and verification of the backends certificate Message-ID: <20151011170419.GL4156@jumper.schlittermann.de> Hello, I'm using a dovecot as proxy, connecting to one or more backends. The backends use X.509 certificates. The proxy's passdb returns extra fields: user=foo proxy host=backend1. ssl=yes nopassword=y Thus the proxy connects to the backend but can't verify the backends certificate. The following comment suggests using ssl_client_ca_file for that. # Directory and/or file for trusted SSL CA certificates. These are used only # when Dovecot needs to act as an SSL client (e.g. imapc backend). The # directory is usually /etc/ssl/certs in Debian-based systems and the file is # /etc/pki/tls/cert.pem in RedHat-based systems. #ssl_client_ca_dir = #ssl_client_ca_file = ssl_client_ca_file = /tmp/certs/ca-local.pem But that does not work! Instead I've to use ssl_ca # PEM encoded trusted certificate authority. Set this only if you intend to use # ssl_verify_client_cert=yes. The file should contain the CA certificate(s) # followed by the matching CRL(s). (e.g. ssl_ca = From eric at ericabrahamsen.net Mon Oct 12 15:14:29 2015 From: eric at ericabrahamsen.net (Eric Abrahamsen) Date: Mon, 12 Oct 2015 08:14:29 -0700 Subject: iterate users with passwd-file passdb? Message-ID: <87twpwunje.fsf@ericabrahamsen.net> Apologies for the basic question, but I've been googling and can't figure out if this is possible or not. Simply: Is it possible to iterate over users if I'm using the passwd-file passdb driver? Do I need a SQL-based driver if I want to iterate? Here's what I've got right now. I don't have many users, and would prefer to retain the simplicity of the passwd-file if possible. Thanks! passdb { driver = passwd-file args = username_format=%u scheme=ssha512 /etc/dovecot/passwd.db deny = no master = no pass = no skip = never result_failure = continue result_internalfail = continue result_success = return-ok } userdb { driver = static args = uid=5000 gid=5000 home=/var/mail/vmail/%d/%n } From giorgio.paolucci at unipd.it Mon Oct 12 16:39:52 2015 From: giorgio.paolucci at unipd.it (Giorgio Paolucci) Date: Mon, 12 Oct 2015 18:39:52 +0200 Subject: fts solr ignores new mailboxes until fts rescan Message-ID: <20151012183952.Horde.WJtw27aXzCGxZDGs_q_5gPe@mailweb.unipd.it> Hello, on my debian wheezy installation, I implemented fts plugin with solr as indexing system. I noticed that if a user creates a new folder, that folder it is ignored by dovecot indexing until a doveadm fts rescan is performed for that user. From that time on everything works ok on that folder. As a test I set fts_autoindex=yes and put in that folder a brand new mail never indexed before. in my logs indexer-worker says it indexed 0 messages Obviously this is a major issue as users often create new folders, but a search always returns empty? results. My system is a debian wheezy with dovecot packages from backports? dovecot 2.2.13-11~bpo70+1. Am I missing something? Any clue would be appreciated. Thank you all in advance. Bests Giorgio Paolucci ============================= doveconf -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8 auth_default_realm = xxx disable_plaintext_auth = no log_path = /var/log/dovecot/dovecot.log mail_fsync = always mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota? listescape mail_log notify fts fts_solr managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_interval = 1 days mdbox_rotate_size = 10 M mmap_disable = yes namespace inbox { ? inbox = yes ? location = ? mailbox Drafts { ??? special_use = \Drafts ? } ? mailbox Junk { ??? special_use = \Junk ? } ? mailbox Sent { ??? special_use = \Sent ? } ? mailbox "Sent Messages" { ??? special_use = \Sent ? } ? mailbox Trash { ??? special_use = \Trash ? } ? prefix = ? separator = / } passdb { ? args = /etc/dovecot/conf.d/dovecot-sql.conf.ext ? driver = sql } passdb { ? args = /etc/dovecot/conf.d/dovecot-sqlSSO.conf.ext ? driver = sql } plugin { ? fts = solr ? fts_solr = url=http://%n.%1n.solr.csia:8983/solr/ ? mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename ? mail_log_fields = uid box msgid size ? pop3_migration_mailbox = INBOX ? quota = dict:User quota::file:/SYSTEM/USERS/homes/%d/%1u/%u/%u.quota ? quota_rule = *:storage=7000M ? quota_rule2 = Trash:storage=+100M ? quota_warning = storage=95%% quota-warning 95 %u ? quota_warning2 = storage=80%% quota-warning 80 %u ? sieve = /SYSTEM/USERS/mailbox/%d/%1n/%u/.dovecotactive.sieve ? sieve_default = /etc/dovecot/default.sieve ? sieve_dir = /SYSTEM/USERS/mailbox/%d/%1n/%u/sieve } pop3c_host = 147.162.10.68 protocols = imap pop3 lmtp sieve service auth { ? unix_listener /var/spool/postfix/private/auth { ??? group = postfix ??? mode = 0660 ??? user = postfix ? } ? unix_listener auth-userdb { ??? group = vmail ??? mode = 0600 ??? user = vmail ? } } service imap-login { ? inet_listener imaps { ??? ssl = no ? } ? process_min_avail = 20 ? service_count = 1 } service imap { ? executable = imap postlogin ? process_limit = 4096 } service lmtp { ? executable = lmtp ? inet_listener lmtp { ??? port = 2223 ? } ? process_min_avail = 15 } service pop3-login { ? inet_listener pop3s { ??? ssl = no ? } } service pop3 { ? process_limit = 256 } service postlogin { ? executable = script-login -d rawlog ? user = vmail } service quota-warning { ? executable = script /SYSTEM/DOVECOT/scripts/quota-warning.sh ? unix_listener quota-warning { ??? user = vmail ? } ? user = vmail } ssl = no userdb { ? driver = prefetch } userdb { ? args = /etc/dovecot/conf.d/dovecot-sql.conf.ext ? driver = sql } userdb { ? driver = prefetch } userdb { ? args = /etc/dovecot/conf.d/dovecot-sqlSSO.conf.ext ? driver = sql } protocol lmtp { ? info_log_path = /tmp/dovecot-lmtp-info.log ? log_path = /tmp/dovecot-lmtp.log ? mail_plugins = quota? listescape mail_log notify sieve } protocol lda { ? mail_plugins = quota? listescape mail_log notify } protocol imap { ? imap_max_line_length = 128 k ? mail_max_userip_connections = 20 ? mail_plugins = quota? listescape mail_log notify fts fts_solr imap_quota } protocol doveadm { ? mail_plugins = quota? listescape mail_log notify fts fts_solr pop3_migration } protocol pop3 { ? pop3_no_flag_updates = yes ? pop3_uidl_format = %10v.%09u From grupodecorreo10 at gmail.com Tue Oct 13 07:20:31 2015 From: grupodecorreo10 at gmail.com (grupo correo) Date: Tue, 13 Oct 2015 09:20:31 +0200 Subject: Problems sync with mail client MAC and dovecot-2.2.10-4.el7_0.1.el7_0.1.x86_64 Message-ID: Hello, i have installed Centos 7.1.1503 with Dovecot 2.2.10-4.el7_0.1. With Thunderbird all works fine, but when i configure the email client of Mac (Yosemite) with IMAP i can see a error in the log and the client not see the emails: Oct 9 14:31:34 localhost dovecot: imap(xxxxxxx at xxxxxxx): Fatal: master: service(imap): child 18639 killed with signal 11 (core dumps disabled) I have read the instructions of http://www.dovecot.org/bugreport.html, but core file is not writing. Does anyone have this issues? I think that can be a problem in the configuration Best regards From ml+dovecot at valo.at Tue Oct 13 08:25:55 2015 From: ml+dovecot at valo.at (Christian Kivalo) Date: Tue, 13 Oct 2015 10:25:55 +0200 Subject: fts solr ignores new mailboxes until fts rescan In-Reply-To: <20151012183952.Horde.WJtw27aXzCGxZDGs_q_5gPe@mailweb.unipd.it> References: <20151012183952.Horde.WJtw27aXzCGxZDGs_q_5gPe@mailweb.unipd.it> Message-ID: <66f5a395191db3285a830c1a86a578ff@valo.at> Hi, On 2015-10-12 18:39, Giorgio Paolucci wrote: > Hello, > on my debian wheezy installation, I implemented fts plugin with solr as > indexing system. > I noticed that if a user creates a new folder, that folder it is > ignored by > dovecot indexing until a doveadm fts rescan is performed for that user. > From that time on everything works ok on that folder. > > As a test I set fts_autoindex=yes and put in that folder a brand new > mail > never indexed before. in my logs indexer-worker says it indexed 0 > messages > > Obviously this is a major issue as users often create new folders, but > a > search always returns empty? results. > > My system is a debian wheezy with dovecot packages from backports? > dovecot > 2.2.13-11~bpo70+1. > > Am I missing something? Any clue would be appreciated. Am using 2.2.19 with fts_autoindex = yes and Maildir storage. I create a folder and moved a message from inbox to the new folder and the log says it indexed 1 message in test, I don't have messages that where never indexed but i doubt this makes any difference. I'm using Roundcube and tried the search with both Subject and Body searches. Are the mails indexed when you perform a search? How do you search? > Thank you all in advance. > > Bests > Giorgio Paolucci > ============================= > > doveconf -n > # 2.2.13: /etc/dovecot/dovecot.conf > # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8 > auth_default_realm = xxx > disable_plaintext_auth = no > log_path = /var/log/dovecot/dovecot.log > mail_fsync = always > mail_nfs_index = yes > mail_nfs_storage = yes > mail_plugins = quota? listescape mail_log notify fts fts_solr > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character > vacation subaddress comparator-i;ascii-numeric relational regex > imap4flags > copy include variables body enotify environment mailbox date ihave > mdbox_rotate_interval = 1 days > mdbox_rotate_size = 10 M > mmap_disable = yes > namespace inbox { > ? inbox = yes > ? location = > ? mailbox Drafts { > ??? special_use = \Drafts > ? } > ? mailbox Junk { > ??? special_use = \Junk > ? } > ? mailbox Sent { > ??? special_use = \Sent > ? } > ? mailbox "Sent Messages" { > ??? special_use = \Sent > ? } > ? mailbox Trash { > ??? special_use = \Trash > ? } > ? prefix = > ? separator = / > } > passdb { > ? args = /etc/dovecot/conf.d/dovecot-sql.conf.ext > ? driver = sql > } > passdb { > ? args = /etc/dovecot/conf.d/dovecot-sqlSSO.conf.ext > ? driver = sql > } > plugin { > ? fts = solr > ? fts_solr = url=http://%n.%1n.solr.csia:8983/solr/ i have and fts_autoindex = yes and break-imap-search set, but afaik the lattter should only take effect on searches regards christian > ? mail_log_events = delete undelete expunge copy mailbox_delete > mailbox_rename > ? mail_log_fields = uid box msgid size > ? pop3_migration_mailbox = INBOX > ? quota = dict:User quota::file:/SYSTEM/USERS/homes/%d/%1u/%u/%u.quota > ? quota_rule = *:storage=7000M > ? quota_rule2 = Trash:storage=+100M > ? quota_warning = storage=95%% quota-warning 95 %u > ? quota_warning2 = storage=80%% quota-warning 80 %u > ? sieve = /SYSTEM/USERS/mailbox/%d/%1n/%u/.dovecotactive.sieve > ? sieve_default = /etc/dovecot/default.sieve > ? sieve_dir = /SYSTEM/USERS/mailbox/%d/%1n/%u/sieve > } > pop3c_host = 147.162.10.68 > protocols = imap pop3 lmtp sieve > service auth { > ? unix_listener /var/spool/postfix/private/auth { > ??? group = postfix > ??? mode = 0660 > ??? user = postfix > ? } > ? unix_listener auth-userdb { > ??? group = vmail > ??? mode = 0600 > ??? user = vmail > ? } > } > service imap-login { > ? inet_listener imaps { > ??? ssl = no > ? } > ? process_min_avail = 20 > ? service_count = 1 > } > service imap { > ? executable = imap postlogin > ? process_limit = 4096 > } > service lmtp { > ? executable = lmtp > ? inet_listener lmtp { > ??? port = 2223 > ? } > ? process_min_avail = 15 > } > service pop3-login { > ? inet_listener pop3s { > ??? ssl = no > ? } > } > service pop3 { > ? process_limit = 256 > } > service postlogin { > ? executable = script-login -d rawlog > ? user = vmail > } > service quota-warning { > ? executable = script /SYSTEM/DOVECOT/scripts/quota-warning.sh > ? unix_listener quota-warning { > ??? user = vmail > ? } > ? user = vmail > } > ssl = no > userdb { > ? driver = prefetch > } > userdb { > ? args = /etc/dovecot/conf.d/dovecot-sql.conf.ext > ? driver = sql > } > userdb { > ? driver = prefetch > } > userdb { > ? args = /etc/dovecot/conf.d/dovecot-sqlSSO.conf.ext > ? driver = sql > } > protocol lmtp { > ? info_log_path = /tmp/dovecot-lmtp-info.log > ? log_path = /tmp/dovecot-lmtp.log > ? mail_plugins = quota? listescape mail_log notify sieve > } > protocol lda { > ? mail_plugins = quota? listescape mail_log notify > } > protocol imap { > ? imap_max_line_length = 128 k > ? mail_max_userip_connections = 20 > ? mail_plugins = quota? listescape mail_log notify fts fts_solr > imap_quota > } > protocol doveadm { > ? mail_plugins = quota? listescape mail_log notify fts fts_solr > pop3_migration > } > protocol pop3 { > ? pop3_no_flag_updates = yes > ? pop3_uidl_format = %10v.%09u From ml+dovecot at valo.at Tue Oct 13 08:29:52 2015 From: ml+dovecot at valo.at (Christian Kivalo) Date: Tue, 13 Oct 2015 10:29:52 +0200 Subject: Problems sync with mail client MAC and dovecot-2.2.10-4.el7_0.1.el7_0.1.x86_64 In-Reply-To: References: Message-ID: <63037df1426dbe7c51adc039607ee103@valo.at> On 2015-10-13 09:20, grupo correo wrote: > Hello, > > i have installed Centos 7.1.1503 with Dovecot 2.2.10-4.el7_0.1. With > Thunderbird all works fine, but when i configure the email client of > Mac > (Yosemite) with IMAP i can see a error in the log and the client not > see > the emails: > > Oct 9 14:31:34 localhost dovecot: imap(xxxxxxx at xxxxxxx): Fatal: > master: > service(imap): child 18639 killed with signal 11 (core dumps disabled) Do you have more logs? Have you enabled mail_debug = yes to get more verbose logs? > > I have read the instructions of http://www.dovecot.org/bugreport.html, > but > core file is not writing. > > Does anyone have this issues? I think that can be a problem in the > configuration Please share your doveconf -n output. > Best regards regards christian From giorgio.paolucci at unipd.it Tue Oct 13 10:51:36 2015 From: giorgio.paolucci at unipd.it (Giorgio Paolucci) Date: Tue, 13 Oct 2015 12:51:36 +0200 Subject: fts solr ignores new mailboxes until fts rescan In-Reply-To: <66f5a395191db3285a830c1a86a578ff@valo.at> References: <20151012183952.Horde.WJtw27aXzCGxZDGs_q_5gPe@mailweb.unipd.it> <66f5a395191db3285a830c1a86a578ff@valo.at> Message-ID: <20151013125136.Horde.vToIErbLu2HCnuFcrxsmkhb@mailweb.unipd.it> Hi Christian, may be the issue is related to this other post: http://dovecot.org/pipermail/dovecot/2015-September/102094.html Anyway, what I can tell you is that for every? user account,? for every folders created after the last doveadm fts rescan for that user, dovecot does not invoke indexer. A new rescan fix the problem for all the folders created up to that moment. I was wondering if I was missing something or if there was a work around to force rescan only for new folders at the moment of their creation... Bests. Giorgio Quoting Christian Kivalo : > Hi, > > On 2015-10-12 18:39, Giorgio Paolucci wrote: >> Hello, >> on my debian wheezy installation, I implemented fts plugin with solr as >> indexing system. >> I noticed that if a user creates a new folder, that folder it is >> ignored by >> dovecot indexing until a doveadm fts rescan is performed for that user. >> From that time on everything works ok on that folder. >> >> As a test I set fts_autoindex=yes and put in that folder a brand new mail >> never indexed before. in my logs indexer-worker says it indexed 0 >> messages >> >> Obviously this is a major issue as users often create new folders, but a >> search always returns empty? results. >> >> My system is a debian wheezy with dovecot packages from backports? >> dovecot >> 2.2.13-11~bpo70+1. >> >> Am I missing something? Any clue would be appreciated. > > Am using 2.2.19 with fts_autoindex = yes and Maildir storage. > > I create a folder and moved a message from inbox to the new folder and > the log says it indexed 1 message in test, I don't have messages that > where never indexed but i doubt this makes any difference. > > I'm using Roundcube and tried the search with both Subject and Body > searches. > > Are the mails indexed when you perform a search? > How do you search? > >> Thank you all in advance. >> >> Bests >> Giorgio Paolucci >> ============================= >> >> doveconf -n >> # 2.2.13: /etc/dovecot/dovecot.conf >> # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8 >> auth_default_realm = xxx >> disable_plaintext_auth = no >> log_path = /var/log/dovecot/dovecot.log >> mail_fsync = always >> mail_nfs_index = yes >> mail_nfs_storage = yes >> mail_plugins = quota? listescape mail_log notify fts fts_solr >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope encoded-character >> vacation subaddress comparator-i;ascii-numeric relational regex >> imap4flags >> copy include variables body enotify environment mailbox date ihave >> mdbox_rotate_interval = 1 days >> mdbox_rotate_size = 10 M >> mmap_disable = yes >> namespace inbox { >> ? inbox = yes >> ? location = >> ? mailbox Drafts { >> ??? special_use = \Drafts >> ? } >> ? mailbox Junk { >> ??? special_use = \Junk >> ? } >> ? mailbox Sent { >> ??? special_use = \Sent >> ? } >> ? mailbox "Sent Messages" { >> ??? special_use = \Sent >> ? } >> ? mailbox Trash { >> ??? special_use = \Trash >> ? } >> ? prefix = >> ? separator = / >> } >> passdb { >> ? args = /etc/dovecot/conf.d/dovecot-sql.conf.ext >> ? driver = sql >> } >> passdb { >> ? args = /etc/dovecot/conf.d/dovecot-sqlSSO.conf.ext >> ? driver = sql >> } >> plugin { >> ? fts = solr >> ? fts_solr = url=http://%n.%1n.solr.csia:8983/solr/ > > i have and fts_autoindex = yes and break-imap-search set, but afaik the > lattter should only take effect on searches > > regards > christian > >> ? mail_log_events = delete undelete expunge copy mailbox_delete >> mailbox_rename >> ? mail_log_fields = uid box msgid size >> ? pop3_migration_mailbox = INBOX >> ? quota = dict:User quota::file:/SYSTEM/USERS/homes/%d/%1u/%u/%u.quota >> ? quota_rule = *:storage=7000M >> ? quota_rule2 = Trash:storage=+100M >> ? quota_warning = storage=95%% quota-warning 95 %u >> ? quota_warning2 = storage=80%% quota-warning 80 %u >> ? sieve = /SYSTEM/USERS/mailbox/%d/%1n/%u/.dovecotactive.sieve >> ? sieve_default = /etc/dovecot/default.sieve >> ? sieve_dir = /SYSTEM/USERS/mailbox/%d/%1n/%u/sieve >> } >> pop3c_host = 147.162.10.68 >> protocols = imap pop3 lmtp sieve >> service auth { >> ? unix_listener /var/spool/postfix/private/auth { >> ??? group = postfix >> ??? mode = 0660 >> ??? user = postfix >> ? } >> ? unix_listener auth-userdb { >> ??? group = vmail >> ??? mode = 0600 >> ??? user = vmail >> ? } >> } >> service imap-login { >> ? inet_listener imaps { >> ??? ssl = no >> ? } >> ? process_min_avail = 20 >> ? service_count = 1 >> } >> service imap { >> ? executable = imap postlogin >> ? process_limit = 4096 >> } >> service lmtp { >> ? executable = lmtp >> ? inet_listener lmtp { >> ??? port = 2223 >> ? } >> ? process_min_avail = 15 >> } >> service pop3-login { >> ? inet_listener pop3s { >> ??? ssl = no >> ? } >> } >> service pop3 { >> ? process_limit = 256 >> } >> service postlogin { >> ? executable = script-login -d rawlog >> ? user = vmail >> } >> service quota-warning { >> ? executable = script /SYSTEM/DOVECOT/scripts/quota-warning.sh >> ? unix_listener quota-warning { >> ??? user = vmail >> ? } >> ? user = vmail >> } >> ssl = no >> userdb { >> ? driver = prefetch >> } >> userdb { >> ? args = /etc/dovecot/conf.d/dovecot-sql.conf.ext >> ? driver = sql >> } >> userdb { >> ? driver = prefetch >> } >> userdb { >> ? args = /etc/dovecot/conf.d/dovecot-sqlSSO.conf.ext >> ? driver = sql >> } >> protocol lmtp { >> ? info_log_path = /tmp/dovecot-lmtp-info.log >> ? log_path = /tmp/dovecot-lmtp.log >> ? mail_plugins = quota? listescape mail_log notify sieve >> } >> protocol lda { >> ? mail_plugins = quota? listescape mail_log notify >> } >> protocol imap { >> ? imap_max_line_length = 128 k >> ? mail_max_userip_connections = 20 >> ? mail_plugins = quota? listescape mail_log notify fts fts_solr >> imap_quota >> } >> protocol doveadm { >> ? mail_plugins = quota? listescape mail_log notify fts fts_solr >> pop3_migration >> } >> protocol pop3 { >> ? pop3_no_flag_updates = yes? pop3_uidl_format = %10v.%09u > > ? -- ? ? ?Ing. Giorgio Paolucci ? ? ?Universita' di Padova ? ? ?Centro Servizi Informatici di Ateneo ? ? ?Responsabile Infrastrutture, ? ? ?Networking e Sicurezza ? ? ?Tel.+39-049-8273711 -- From ml+dovecot at valo.at Tue Oct 13 11:41:18 2015 From: ml+dovecot at valo.at (Christian Kivalo) Date: Tue, 13 Oct 2015 13:41:18 +0200 Subject: fts solr ignores new mailboxes until fts rescan In-Reply-To: <20151013125136.Horde.vToIErbLu2HCnuFcrxsmkhb@mailweb.unipd.it> References: <20151012183952.Horde.WJtw27aXzCGxZDGs_q_5gPe@mailweb.unipd.it> <66f5a395191db3285a830c1a86a578ff@valo.at> <20151013125136.Horde.vToIErbLu2HCnuFcrxsmkhb@mailweb.unipd.it> Message-ID: <6ca51046b5024ea5c81db20a7593e548@valo.at> On 2015-10-13 12:51, Giorgio Paolucci wrote: > Hi Christian, > may be the issue is related to this other post: > http://dovecot.org/pipermail/dovecot/2015-September/102094.html Looks like the same problem and Timo wrote 2.2.19 should fix this -> http://dovecot.org/pipermail/dovecot/2015-September/102097.html regards christian > Anyway, what I can tell you is that for every? user account,? for every > folders created after the last doveadm fts rescan for that user, > dovecot > does not invoke indexer. > A new rescan fix the problem for all the folders created up to that > moment. > > I was wondering if I was missing something or if there was a work > around to > force rescan only for new folders at the moment of their creation... > > Bests. > Giorgio From mjoigny at neteven.com Tue Oct 13 15:15:41 2015 From: mjoigny at neteven.com (Michael JOIGNY) Date: Tue, 13 Oct 2015 17:15:41 +0200 Subject: Dovecot - Postfix with HAproxy Message-ID: <561D201D.5040901@neteven.com> Hi Everyone, I wouldlike to set up a postfix-dovecot with HA using HAproxy but im facing issues. I've followed this documentation : http://wiki2.dovecot.org/HAProxy (pour dovecot) http://blog.haproxy.com/2012/06/30/efficient-smtp-relay-infrastructure-with-postfix-and-load-balancers/ (pour postfix) Package's version : dovecot : 2:2.2.19 (>= 2.2.19 pour proxy protocol) haproxy : 1.5.14 postfix : 2.11.2-1 (>2.10 pour postscreen) A part of my configuration : ##HAPROXY #postfix listen smtp bind mail.xx.xx:465 balance roundrobin timeout client 1m timeout connect 5s no option http-server-close mode tcp option smtpchk option tcplog server tst tst.xxx:10465 send-proxy server tst2 tst2.xxx:10465 send-proxy server tst3 tst3.xxx:10465 send-proxy #dovecot listen imap bind mail.xxx.xx:993 timeout client 1m no option http-server-close balance leastconn stick store-request src stick-table type ip size 200k expire 30m mode tcp option tcplog server tst tst.xxx:10993 send-proxy-v2 server tst2tst2.xxx:10993 send-proxy-v2 server tst3 tst3.xxx:10993 send-proxy-v2 ##POSTFIX postix main.cf #Haproxy proxy protocol postscreen_upstream_proxy_protocol = haproxy postfix master.cf #haproxy 10465 inet n ? n ? 1 postscreen smtpd pass ? ? n ? ? smtpd S ##DOVECOT # 2.2.19 (719e7f8fd70b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.9 # OS: Linux 2.6.32-41-pve x86_64 Debian 7.9 simfs auth_debug = yes auth_verbose = yes disable_plaintext_auth = no *haproxy_timeout = 5 secs** **haproxy_trusted_networks = x.x.x.x* log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/mailbox/%d/%n mail_max_userip_connections = 0 mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_global_path = /mailbox/globalsieverc } protocols = sieve pop3 imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { mode = 0660 user = mailboxes } unix_listener auth-userdb { group = mail mode = 0666 user = dovecot } user = root } service imap-login { inet_listener imap { port = 0 } inet_listener imap_haproxy { haproxy = yes port = 10993 } inet_listener imaps { address = * port = 993 } process_limit = 450 } service pop3-login { inet_listener pop3 { address = * port = 110 } process_limit = 180 } ssl_cert = was automatically rejected:%n%r rejection_subject = Rejected: %s sendmail_path = /usr/sbin/sendmail } With my mail client : With an IMAP connection, logs below, i don't understand why my login is empty ... dovecot: imap-login: Disconnected: Too many invalid commands (no auth attempts in 0 secs): *user=<>*, rip=mon_ip_publique, lip=ip_publique_haproxy, session= xxx With a SMTP connection, logs below, i have a timeout. postfix/postscreen[16654]: CONNECT from [my public ip]:49942 to [my haproxy public ip]:465 postfix/postscreen[16654]: PREGREET 166 after 0 from [mon ip publique]:49942: \22\3\1\161\1\157\3\3+0E\b\213\131\177\173>\r/\213\177i\223k?FjA#\144\145\153\vP\\\155HL\190 If someone could help me, thanks. Kind regards. -- -------------- next part -------------- A non-text attachment was scrubbed... Name: sign_neteven.png Type: image/png Size: 20734 bytes Desc: not available URL: From tss at iki.fi Tue Oct 13 18:19:54 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 13 Oct 2015 21:19:54 +0300 Subject: Dovecot auth-ldap ignores tls_* settings when using ldaps:// In-Reply-To: <20151008194657.GZ4156@jumper.schlittermann.de> References: <20151008194657.GZ4156@jumper.schlittermann.de> Message-ID: <41A57B4E-4D76-4D21-9758-6321684D9B07@iki.fi> On 08 Oct 2015, at 22:46, Heiko Schlittermann wrote: > > Hi, > > I'm using dovecot 2.2.9 (but after checking src/auth/db-ldap.c in 2.2.13 > there seems to be the same bug/feature). > > The userdb and passdb use LDAP. All further configuration is done in > auth-ldap.conf.ext. > > uri = ldaps:/// > # tls = > tls_cert_file = /etc/ssl/certs/client-cert.pem > tls_key_file = /etc/ssl/certs/client-key.file > > Dovecot ignores the tls_* options. If I use an ldap:// URI and > switch on TLS using tls=yes it works as expected. > > But I do not see any reason why LDAPS should not read the tls_* > settings. I guess. > This small patch solved it for me > > --- dovecot-2.2.9/src/auth/db-ldap.c 2013-11-24 14:37:39.000000000 +0100 > +++ dovecot-2.2.9.hs12/src/auth/db-ldap.c 2015-10-08 21:24:47.051446465 +0200 > @@ -1043,7 +1043,7 @@ > > static void db_ldap_set_tls_options(struct ldap_connection *conn) > { > - if (!conn->set.tls) > + if (!(conn->set.tls || strncmp(conn->set.uris, "ldaps:", 6) == 0)) > return; That's a bit ugly. I think also the URIs support multiple ones, so some ldap and some ldaps URLs could even be mixed, which of course would be quite ugly.. I think the fix is to just remove the if (tls)-check completely. I don't think setting those harms anything even if tls/ldaps isn't being used? From tss at iki.fi Tue Oct 13 18:27:25 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 13 Oct 2015 21:27:25 +0300 Subject: dovecot as proxy and verification of the backends certificate In-Reply-To: <20151011170419.GL4156@jumper.schlittermann.de> References: <20151011170419.GL4156@jumper.schlittermann.de> Message-ID: <4A0BBE04-CCC7-44ED-9A6A-486E2F4A2664@iki.fi> On 11 Oct 2015, at 20:04, Heiko Schlittermann wrote: > > Hello, > > I'm using a dovecot as proxy, connecting to one or more backends. > The backends use X.509 certificates. > > The proxy's passdb returns > > extra fields: > user=foo > proxy > host=backend1. > ssl=yes > nopassword=y > > Thus the proxy connects to the backend but can't verify the backends > certificate. > > The following comment suggests using ssl_client_ca_file for that. > > # Directory and/or file for trusted SSL CA certificates. These are used only > # when Dovecot needs to act as an SSL client (e.g. imapc backend). The > # directory is usually /etc/ssl/certs in Debian-based systems and the file is > # /etc/pki/tls/cert.pem in RedHat-based systems. > #ssl_client_ca_dir = > #ssl_client_ca_file = > ssl_client_ca_file = /tmp/certs/ca-local.pem > > But that does not work! Instead I've to use ssl_ca > > # PEM encoded trusted certificate authority. Set this only if you intend to use > # ssl_verify_client_cert=yes. The file should contain the CA certificate(s) > # followed by the matching CRL(s). (e.g. ssl_ca = # ssl_ca = > ssl_ca = > Bug or feature? Mainly I'm asking because the comments do not indicate > that I should have used ssl_ca for this type of operation (dovecot as a > SSL client) It's a missing feature. I updated http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/Proxy about this. I'm thinking that once login-common code uses lib-ssl-iostream instead of the duplicated SSL code this gets fixed more or less automatically. Note sure if that'll happen for v2.3 or not. From tss at iki.fi Tue Oct 13 18:31:07 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 13 Oct 2015 21:31:07 +0300 Subject: Multiple core solr support In-Reply-To: References: Message-ID: <47A6D1D5-D5DD-4CB9-9C7D-B523A2191093@iki.fi> > On 11 Oct 2015, at 08:42, Daniel Miller wrote: > > Is it possible to use variables, such as domain or username, in the plugin url declaration? This would allow using multiple cores to separate the stored indexes. So when my own giant mailbox gets corrupt...I don't have to rebuild everybody's... > > Something like url=http://mysolr.server.com:8983/solr/%u Exactly that. Or things like http://mysolr%4Nu.server.com to have it use mysolr0..3.server.com based on the username hash. From hs at schlittermann.de Tue Oct 13 18:35:26 2015 From: hs at schlittermann.de (Heiko Schlittermann) Date: Tue, 13 Oct 2015 20:35:26 +0200 Subject: Dovecot auth-ldap ignores tls_* settings when using ldaps:// In-Reply-To: <41A57B4E-4D76-4D21-9758-6321684D9B07@iki.fi> References: <20151008194657.GZ4156@jumper.schlittermann.de> <41A57B4E-4D76-4D21-9758-6321684D9B07@iki.fi> Message-ID: <20151013183526.GT4156@jumper.schlittermann.de> Timo Sirainen (Di 13 Okt 2015 20:19:54 CEST): .. > > --- dovecot-2.2.9/src/auth/db-ldap.c 2013-11-24 14:37:39.000000000 +0100 > > +++ dovecot-2.2.9.hs12/src/auth/db-ldap.c 2015-10-08 21:24:47.051446465 +0200 > > @@ -1043,7 +1043,7 @@ > > > > static void db_ldap_set_tls_options(struct ldap_connection *conn) > > { > > - if (!conn->set.tls) > > + if (!(conn->set.tls || strncmp(conn->set.uris, "ldaps:", 6) == 0)) > > return; > > That's a bit ugly. I think also the URIs support multiple ones, so some ldap and some ldaps URLs could even be mixed, which of course would be quite ugly.. I think the fix is to just remove the if (tls)-check completely. I don't think setting those harms anything even if tls/ldaps isn't being used? Yes, thinking about mixed schema in the URIs whould have been my next question :) Ok, I can test what happens if we set tls_options w/o using LDAP+TLS or LDAPS at all. Best regards from Dresden/Germany Viele Gr??e aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From hs at schlittermann.de Tue Oct 13 18:37:44 2015 From: hs at schlittermann.de (Heiko Schlittermann) Date: Tue, 13 Oct 2015 20:37:44 +0200 Subject: dovecot as proxy and verification of the backends certificate In-Reply-To: <4A0BBE04-CCC7-44ED-9A6A-486E2F4A2664@iki.fi> References: <20151011170419.GL4156@jumper.schlittermann.de> <4A0BBE04-CCC7-44ED-9A6A-486E2F4A2664@iki.fi> Message-ID: <20151013183744.GU4156@jumper.schlittermann.de> Timo Sirainen (Di 13 Okt 2015 20:27:25 CEST): ? > > # followed by the matching CRL(s). (e.g. ssl_ca = > # ssl_ca = > > ssl_ca = > > > Bug or feature? Mainly I'm asking because the comments do not indicate > > that I should have used ssl_ca for this type of operation (dovecot as a > > SSL client) > > It's a missing feature. I updated http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/Proxy about this. I'm thinking that once login-common code uses lib-ssl-iostream instead of the duplicated SSL code this gets fixed more or less automatically. Note sure if that'll happen for v2.3 or not. Thank you. Best regards from Dresden/Germany Viele Gr??e aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From tss at iki.fi Tue Oct 13 18:41:36 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 13 Oct 2015 21:41:36 +0300 Subject: bug report: dovecot-imapd core dump In-Reply-To: <87mvvqbpb4.fsf@tamas.ihs.ac.at> References: <87mvvuddvm.fsf@tamas.ihs.ac.at> <1444224426.12007.6.camel@dominikbreu.de> <87lhbedbt7.fsf@tamas.ihs.ac.at> <1444226911.12007.7.camel@dominikbreu.de> <87k2qydajw.fsf@tamas.ihs.ac.at> <87mvvqbpb4.fsf@tamas.ihs.ac.at> Message-ID: On 10 Oct 2015, at 20:34, Tamas Papp wrote: > > Hi, > > Since I did not get any specific suggestions about this bug, I am > attaching another backtrace. Any suggestions for workarounds etc would > be appreciated -- should I regenerate the index? Or install a dovecot > from the repo where this might be fixed? Is this a known bug? Fixed: http://hg.dovecot.org/dovecot-2.2/rev/5e48c5a29ddd The APPEND will still fail, but at least it won't crash anymore. > > Best, > > Tamas > > > On Wed, Oct 07 2015, Tamas Papp wrote: > >> Hi, >> >> could not find it -- back to the list, maybe someone can suggest a >> fix/workaround. >> >> Best, >> >> Tamas >> >> On Wed, Oct 07 2015, Dominik Breu wrote: >> >>> Hello, >>> >>> yeah this one looks familiar to me can you search the list back in >>> september there was a somewhat similar bug with thunderbird imho timo >>> posted a notice about it. >>> >>> greetings, >>> >>> dominik >>> >>> Am Mittwoch, den 07.10.2015, 15:53 +0200 schrieb Tamas Papp: >>>> Hi Dominik, >>>> >>>> Thanks for the help, the backtrace is here (I did not sent it to the >>>> list since I am not sure if it contains password or message >>>> information, >>>> sorry but I am unfamiliar with these things). >>>> >>>> Core was generated by `dovecot/imap'. >>>> Program terminated with signal SIGSEGV, Segmentation fault. >>>> #0 0x00007f404e58e3b6 in mail_index_keywords_unref >>>> #(_keywords=_keywords at entry=0x7ffecb8adeb8) at mail-index.c:383 >>>> 383 mail-index.c: No such file or directory. >>>> (gdb) bt full >>>> #0 0x00007f404e58e3b6 in mail_index_keywords_unref >>>> #(_keywords=_keywords at entry=0x7ffecb8adeb8) at mail-index.c:383 >>>> keywords = 0x7f404e27066d >>>> __FUNCTION__ = "mail_index_keywords_unref" >>>> #1 0x00007f404e559975 in mailbox_keywords_unref >>>> (keywords=keywords at entry=0x7ffecb8adeb8) at mailbox-keywords.c:94 >>>> No locals. >>>> #2 0x00007f404ea254ad in cmd_append_handle_args >>>> (nonsync_r=0x7ffecb8ade8f, >>>> #args=, cmd=0x7f4050bc3250) at cmd-append.c:602 >>>> ctx = 0x7f4050bc3348 >>>> flags_list = 0x7f4050bcc840 >>>> flags = 9 >>>> keywords_list = 0x7f4050bc33f0 >>>> internal_date_str = 0x7f4050bcca00 "17-Sep-2015 14:19:23 >>>> +0200" >>>> internal_date = -1 >>>> timezone_offset = 0 >>>> keywords = 0x0 >>>> ret = >>>> client = 0x7f4050bc2670 >>>> cat_list = 0x0 >>>> input = >>>> valid = >>>> #3 cmd_append_parse_new_msg (cmd=cmd at entry=0x7f4050bc3250) at >>>> cmd-append.c:770 >>>> client = >>>> ctx = 0x7f4050bc3348 >>>> args = 0x7f4050bcc488 >>>> msg = >>>> arg_min_count = >>>> fatal = 255 >>>> nonsync = true >>>> last_literal = >>>> ret = >>>> __FUNCTION__ = "cmd_append_parse_new_msg" >>>> #4 0x00007f404ea25a17 in cmd_append (cmd=0x7f4050bc3250) at >>>> cmd-append.c:932 >>>> client = 0x7f4050bc2670 >>>> ctx = 0x7f4050bc3348 >>>> mailbox = 0x7f4050bafb78 "Cemex" >>>> #5 0x00007f404ea30e4c in command_exec (cmd=cmd at entry=0x7f4050bc3250) >>>> at >>>> imap-commands.c:158 >>>> hook = 0x7f4050bad0d0 >>>> ret = >>>> #6 0x00007f404ea2fd62 in client_command_input (cmd=0x7f4050bc3250) >>>> at >>>> imap-client.c:780 >>>> client = 0x7f4050bc2670 >>>> command = >>>> __FUNCTION__ = "client_command_input" >>>> #7 0x00007f404ea2fe16 in client_command_input (cmd=0x7f4050bc3250) >>>> at >>>> imap-client.c:841 >>>> client = 0x7f4050bc2670 >>>> command = >>>> __FUNCTION__ = "client_command_input" >>>> #8 0x00007f404ea3014d in client_handle_next_command >>>> (remove_io_r=>>> pointer>, client=0x7f4050bc2670) at imap-client.c:879 >>>> ---Type to continue, or q to quit--- >>>> No locals. >>>> #9 client_handle_input (client=0x7f4050bc2670) at imap-client.c:891 >>>> _data_stack_cur_id = 3 >>>> remove_io = false >>>> handled_commands = false >>>> client = 0x7f4050bc2670 >>>> #10 0x00007f404ea304f5 in client_input (client=0x7f4050bc2670) at >>>> imap-client.c:933 >>>> cmd = 0xf013c >>>> output = 0x7f4050bc30a0 >>>> bytes = 135 >>>> __FUNCTION__ = "client_input" >>>> #11 0x00007f404e285247 in io_loop_call_io (io=0x7f4050bc3170) at >>>> ioloop.c:388 >>>> ioloop = 0x7f4050bac720 >>>> t_id = 2 >>>> #12 0x00007f404e286079 in io_loop_handler_run >>>> (ioloop=ioloop at entry=0x7f4050bac720) at ioloop-epoll.c:220 >>>> ctx = 0x7f4050bad3b0 >>>> io = >>>> tv = {tv_sec = 1799, tv_usec = 999282} >>>> events_count = >>>> msecs = >>>> ret = 1 >>>> i = 0 >>>> j = >>>> call = >>>> __FUNCTION__ = "io_loop_handler_run" >>>> #13 0x00007f404e284d88 in io_loop_run (ioloop=0x7f4050bac720) at >>>> ioloop.c:412 >>>> __FUNCTION__ = "io_loop_run" >>>> #14 0x00007f404e231de3 in master_service_run (service=0x7f4050bac5b0, >>>> callback=) at master-service.c:566 >>>> No locals. >>>> #15 0x00007f404ea2424e in main (argc=1, argv=0x7f4050bac390) at >>>> main.c:400 >>>> set_roots = {0x7f404ec489c0 , 0x0} >>>> login_set = {auth_socket_path = 0x7f4050ba4048 "\001", >>>> postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = >>>> 0x7f404ea39f40 , >>>> failure_callback = 0x7f404ea39c20 , >>>> request_auth_token = 1} >>>> service_flags = >>>> storage_service_flags = >>>> username = 0x0 >>>> c = >>>> (gdb) quit >>>> >>>> Best, >>>> >>>> Tamas >>>> >>>> On Wed, Oct 07 2015, Dominik Breu wrote: >>>> >>>>> Hello >>>>> >>>>> Tamas it would be a better backtace when you have installed the >>>>> dovecot >>>>> debug symboly mostly located under dovecote-dbg in apt. Further >>>>> debug >>>>> help can you find in the wiki. >>>>> >>>>> greetings >>>>> >>>>> dominik >>>>> >>>>> Am Mittwoch, den 07.10.2015, 15:09 +0200 schrieb Tamas Papp: >>>>>> Typical log message: >>>>>> >>>>>> Oct 7 11:39:36 hostname dovecot: imap(username): Fatal: master: >>>>>> service(imap): child 14864 killed with signal 11 (core dumped) >>>>>> >>>>>> What triggers the problem: >>>>>> >>>>>> User moving multiple messages to a subfolder via IMAP (from >>>>>> another >>>>>> account, on another server, client: Thunderbird). Moving single >>>>>> messages >>>>>> either does not trigger the problem, or with much lower >>>>>> probability >>>>>> (could not reproduce). >>>>>> >>>>>> Version: >>>>>> $ dpkg -l 'dovecot*' >>>>>> Desired=Unknown/Install/Remove/Purge/Hold >>>>>>> Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig >>>>>>> -aWait/Trig-pend >>>>>>> / Err?=(none)/Reinst-required (Status,Err: uppercase=bad) >>>>>>>> / Name Version >>>>>>>> Architecture Description >>>>>> +++-=====================================-======================= >>>>>> -======================= >>>>>> -================================================================ >>>>>> ==== >>>>>> ============ >>>>>> un dovecot-common >>>>>> (no description available) >>>>>> ii dovecot-core 1:2.2.9-1ubuntu5 >>>>>> amd64 secure POP3/IMAP server - core files >>>>>> un dovecot-gssapi >>>>>> (no description available) >>>>>> ii dovecot-imapd 1:2.2.9-1ubuntu5 >>>>>> amd64 secure POP3/IMAP server - IMAP daemon >>>>>> un dovecot-ldap >>>>>> (no description available) >>>>>> un dovecot-lmtpd >>>>>> (no description available) >>>>>> un dovecot-managesieved >>>>>> (no description available) >>>>>> un dovecot-mysql >>>>>> (no description available) >>>>>> un dovecot-pgsql >>>>>> (no description available) >>>>>> un dovecot-pop3d >>>>>> (no description available) >>>>>> un dovecot-sieve >>>>>> (no description available) >>>>>> un dovecot-solr >>>>>> (no description available) >>>>>> un dovecot-sqlite >>>>>> (no description available) >>>>>> tamas at szob:~$ lsb_release -a >>>>>> No LSB modules are available. >>>>>> Distributor ID: Ubuntu >>>>>> Description: Ubuntu 15.04 >>>>>> Release: 15.04 >>>>>> Codename: vivid >>>>>> >>>>>> Also tried 2.2.18 from Ubuntu wiley, same results. >>>>>> >>>>>> Backtrace (I am not sure this is useful, extraced core from >>>>>> Ubuntu's >>>>>> Apport): >>>>>> >>>>>> (gdb) bt full >>>>>> #0 0x00007f404e58e3b6 in mail_index_keywords_unref () from >>>>>> /usr/lib/dovecot/libdovecot-storage.so.0 >>>>>> No symbol table info available. >>>>>> #1 0x00007f404ea254ad in _start () >>>>>> No symbol table info available. >>>>>> >>>>>> Any help would be appreciated, I am happy to provide more info if >>>>>> necessary. >>>>>> >>>>>> Tamas From hs at schlittermann.de Tue Oct 13 18:44:37 2015 From: hs at schlittermann.de (Heiko Schlittermann) Date: Tue, 13 Oct 2015 20:44:37 +0200 Subject: TLS communication director -> backend with X.509 cert checks? Message-ID: <20151013184437.GV4156@jumper.schlittermann.de> Hello, using Dovecot 2.2.9 and a setup with directors and backends. The communication between directors and backends needs to be TLS secured. The director config contains a list of hostnames for the backends. (implicit list because of multiple A/AAAA records for a single hostname or explicit list of several host names) On connection setup from a client the director connects to the selected backend. But it seems (not checked in the source yet), that for SSL certificate verification the director doesn't know the original host name anymore. The certificate's CN gets compared to the IP address the director connects to. Oct 12 23:56:51 director2 dovecot: director: Error: director(2001:683:921:f33::5:1:9090/out): connect() failed: Connection reset by peer Oct 12 23:57:53 director2 dovecot: imap-login: Error: proxy: hostname doesn't match SSL certificate at 2001:683:921:f33::5:fe:993: user=, method=PLAIN, rip=2001:638:912:f33::1:1, lip=2001:638:912:f33::5:2, TLS, session= In 10-directors.conf I've: director_mail_servers = backend1. backend2. Should I create certificates with IP address in SAN? (Any hint about the correct syntax for the openssl.conf is welcome). Or is there any chance that this is fixed already or will be fixed in the near future or even better, that it's my fault? Best regards from Dresden/Germany Viele Gr??e aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From tss at iki.fi Tue Oct 13 18:50:49 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 13 Oct 2015 21:50:49 +0300 Subject: doveadm index assertion failed In-Reply-To: <7cdb984d3bb0d4ede95a16a2b2fb8d9a@bunbun.be> References: <7cdb984d3bb0d4ede95a16a2b2fb8d9a@bunbun.be> Message-ID: On 05 Oct 2015, at 22:05, Nick Rosier wrote: > > Hi, > > one of my mailboxes returns following error when I run doveadm index on it: > > Panic: file charset-iconv.c: line 85 (charset_to_utf8_try): assertion failed: (srcleft <= CHARSET_MAX_PENDING_BUF_SIZE) > > OS: FreeBSD 10.2 > Dovecot: 2.1.19 > Tika: 1.10 > SOLR: 5.3.1 I thought this would have definitely been fixed by now.. Can you find some specific email which is triggering this? It's either something strange that I didn't expect, or it's a bug in FreeBSD's iconv(). Also can you debug this like: gdb --args doveadm index -u user at domain INBOX run f 5 p src p ic_srcbuf p *src_size p srcleft From tss at iki.fi Tue Oct 13 18:58:46 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 13 Oct 2015 21:58:46 +0300 Subject: Unknown cache fields? In-Reply-To: References: Message-ID: <05379376-D0C4-40C3-9D2C-48F1D0F6D3F2@iki.fi> On 05 Oct 2015, at 17:43, Larry Rosenman wrote: > > I assume this is expected, but just checking. I do NOT use POP3. > > Oct 5 09:41:20 thebighonker dovecot: indexer-worker(mrm): Debug: Ignoring > unknown cache field: pop3.order > Oct 5 09:41:20 thebighonker dovecot: indexer-worker(mrm): Debug: Ignoring > unknown cache field: binary.parts > Oct 5 09:41:20 thebighonker dovecot: indexer-worker(mrm): Debug: Ignoring > unknown cache field: body.snippet Bug, but it just affected dsync optimization. Fixed: http://hg.dovecot.org/dovecot-2.2/rev/3b1e7941542f From tss at iki.fi Tue Oct 13 19:02:59 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 13 Oct 2015 22:02:59 +0300 Subject: TLS communication director -> backend with X.509 cert checks? In-Reply-To: <20151013184437.GV4156@jumper.schlittermann.de> References: <20151013184437.GV4156@jumper.schlittermann.de> Message-ID: <9831E5FF-3161-4CC9-ACA9-AB36F23C589B@iki.fi> On 13 Oct 2015, at 21:44, Heiko Schlittermann wrote: > > Hello, > > using Dovecot 2.2.9 and a setup with directors and backends. > The communication between directors and backends needs to be TLS > secured. > > The director config contains a list of hostnames for the backends. > (implicit list because of multiple A/AAAA records for a single hostname > or explicit list of several host names) > > On connection setup from a client the director connects to the > selected backend. But it seems (not checked in the source yet), > that for SSL certificate verification the director doesn't know the > original host name anymore. The certificate's CN gets compared to > the IP address the director connects to. Right. The hostnames are lost immediately at director startup. I've never really thought about needing this functionality for director, since they're usually in the same trusted network with backends.. > Should I create certificates with IP address in SAN? (Any hint about the > correct syntax for the openssl.conf is welcome). Or is there any chance > that this is fixed already or will be fixed in the near future or even > better, that it's my fault? I guess that could work for now. No idea about how to do such certificates. From hs at schlittermann.de Tue Oct 13 19:18:35 2015 From: hs at schlittermann.de (Heiko Schlittermann) Date: Tue, 13 Oct 2015 21:18:35 +0200 Subject: TLS communication director -> backend with X.509 cert checks? In-Reply-To: <9831E5FF-3161-4CC9-ACA9-AB36F23C589B@iki.fi> References: <20151013184437.GV4156@jumper.schlittermann.de> <9831E5FF-3161-4CC9-ACA9-AB36F23C589B@iki.fi> Message-ID: <20151013191835.GW4156@jumper.schlittermann.de> Timo Sirainen (Di 13 Okt 2015 21:02:59 CEST): ? > > On connection setup from a client the director connects to the > > selected backend. But it seems (not checked in the source yet), > > that for SSL certificate verification the director doesn't know the > > original host name anymore. The certificate's CN gets compared to > > the IP address the director connects to. > > Right. The hostnames are lost immediately at director startup. I've never really thought about needing this functionality for director, since they're usually in the same trusted network with backends.. > That's it? "ususally". And additionally local policy says that we should use secured connections whenever credentials are transported ? And since the director uses either a master password or the credentials obtained from the client, we want to use secured connections. And using TLS w/o verified certs is better than nothing, but it's far from being perfect. I see: a) pass the host *names* to the director too, for CN verification purpose May be in struct mail_host could be a field for the original hostname we used to obtain the adress(es)? or b) allow some kind of certificate pinning, that is loose the implied relation CN <=> hostname > > Should I create certificates with IP address in SAN? (Any hint about the > > correct syntax for the openssl.conf is welcome). Or is there any chance > > that this is fixed already or will be fixed in the near future or even > > better, that it's my fault? > > I guess that could work for now. No idea about how to do such certificates. I'll try that, but I think it's not a solution as soon as we reach out for "official" certs. And because it puts more details about the infrastructure into the configuration than would be necessary. Best regards from Dresden/Germany Viele Gr??e aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From hs at schlittermann.de Tue Oct 13 19:21:31 2015 From: hs at schlittermann.de (Heiko Schlittermann) Date: Tue, 13 Oct 2015 21:21:31 +0200 Subject: TLS communication director -> backend with X.509 cert checks? In-Reply-To: <9831E5FF-3161-4CC9-ACA9-AB36F23C589B@iki.fi> References: <20151013184437.GV4156@jumper.schlittermann.de> <9831E5FF-3161-4CC9-ACA9-AB36F23C589B@iki.fi> Message-ID: <20151013192131.GX4156@jumper.schlittermann.de> Timo Sirainen (Di 13 Okt 2015 21:02:59 CEST): > > the IP address the director connects to. > > Right. The hostnames are lost immediately at director startup. I've never really thought about needing this functionality for director, since they're usually in the same trusted network with backends.. > Ooo. What if director_mail_servers = backends. and the DNS entry for backends. gets updated? Does the director catch up the change automatically w/o restart? Best regards from Dresden/Germany Viele Gr??e aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From kgc at corp.sonic.net Tue Oct 13 19:24:53 2015 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Tue, 13 Oct 2015 12:24:53 -0700 Subject: Storage Design (Regligious War, Whatever) Message-ID: <561D5A85.7010809@corp.sonic.net> We've been running dovecot director setup using MAILDIR++ spools on NFS served by 4 (aging) Netapp clustered filers and are considering modernizing our storage. There's nothing particularly wrong with the existing storage except the age, density and power consumption don't compare well when against new systems. Refreshing the Netapps is an option but they come at a substantial cost. The last time we were looking one of the recommended hot DIY setups was Linux NFS backed by XFS on N mirrors but this predated the availability of stable ZFS in FreeBSD or Linux as well as products like FreeNAS. A FreeBSD NFS/ZFS filer on commodity hardware with zil/l2arc on a PCIe SSD seems like an attractive, affordable and easily scaled out solution which also would allow us to leverage compression at the filesystem layer. Does anyone have any experience running ZFS spool storage? If so, how do you handle DR/HA for spool storage? Thoughts on how this might compare to using DRBD? Any pitfalls to watch out for or general pointers? Suggestions on pool configuration? -K From hs at schlittermann.de Tue Oct 13 19:31:46 2015 From: hs at schlittermann.de (Heiko Schlittermann) Date: Tue, 13 Oct 2015 21:31:46 +0200 Subject: Dualstack IPv4/IPv6 setup with directors Message-ID: <20151013193146.GY4156@jumper.schlittermann.de> Hi, still using 2.2.9, I've two directors, and these directors use both IPv4/IPv6 addresses. `host directors.` returns one A and AAA for each of the two directors: directors. has address 149.x.y.96 (director1) directors. has address 149.x.y.97 (director2) directors. has IPv6 address 2001:x:y:f33::5:1 (director1) directors. has IPv6 address 2001:x:y:f33::5:2 (director2) But connecting the the neighbor director gives: Oct 13 21:23:29 director1 dovecot: director: Warning: Director 2001:x:y:f33::5:2:9090/left disconnected us with reason: Invalid input: ME 149.x.y.96 9090 Oct 13 21:23:29 director1 dovecot: director: Error: director(149.x.y.97:9090/out): connect() failed: Connection refused Oct 13 21:23:29 director1 dovecot: director: Warning: net_connect_ip(): ip->family != my_ip->family Oct 13 21:23:29 director1 dovecot: director: Error: Remote director thinks it's someone else (connected to 2001:x:y:f33::5:1:9090, remote says it's 149.x.y.96:9090) I assume it's because the director uses only one (IPv4 OR IPv6) address for it's own identity, right? Best regards from Dresden/Germany Viele Gr??e aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From tss at iki.fi Tue Oct 13 19:36:40 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 13 Oct 2015 22:36:40 +0300 Subject: TLS communication director -> backend with X.509 cert checks? In-Reply-To: <20151013191835.GW4156@jumper.schlittermann.de> References: <20151013184437.GV4156@jumper.schlittermann.de> <9831E5FF-3161-4CC9-ACA9-AB36F23C589B@iki.fi> <20151013191835.GW4156@jumper.schlittermann.de> Message-ID: <9E604417-8950-4426-89A7-172B27AA47E9@iki.fi> On 13 Oct 2015, at 22:18, Heiko Schlittermann wrote: > > Timo Sirainen (Di 13 Okt 2015 21:02:59 CEST): > ? >>> On connection setup from a client the director connects to the >>> selected backend. But it seems (not checked in the source yet), >>> that for SSL certificate verification the director doesn't know the >>> original host name anymore. The certificate's CN gets compared to >>> the IP address the director connects to. >> >> Right. The hostnames are lost immediately at director startup. I've never really thought about needing this functionality for director, since they're usually in the same trusted network with backends.. >> > > That's it? "ususally". And additionally local policy says that we should use > secured connections whenever credentials are transported ? And since the > director uses either a master password or the credentials obtained from > the client, we want to use secured connections. And using TLS w/o > verified certs is better than nothing, but it's far from being perfect. I've been planning to add support for non-plaintext SASL for Dovecot proxy. Probably SCRAM-SHA1. That would avoid sending credentials in plaintext, although it wouldn't prevent other kind of MITM. > I see: > > a) pass the host *names* to the director too, for CN verification > purpose > > May be in struct mail_host could be a field for the original > hostname we used to obtain the adress(es)? Does the attached patch work? Compiles, but untested. -------------- next part -------------- A non-text attachment was scrubbed... Name: director-host.diff Type: application/octet-stream Size: 4143 bytes Desc: not available URL: -------------- next part -------------- From tss at iki.fi Tue Oct 13 19:37:46 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 13 Oct 2015 22:37:46 +0300 Subject: TLS communication director -> backend with X.509 cert checks? In-Reply-To: <20151013192131.GX4156@jumper.schlittermann.de> References: <20151013184437.GV4156@jumper.schlittermann.de> <9831E5FF-3161-4CC9-ACA9-AB36F23C589B@iki.fi> <20151013192131.GX4156@jumper.schlittermann.de> Message-ID: <71EC90CF-4C6F-452F-9F28-19C88798A21B@iki.fi> > On 13 Oct 2015, at 22:21, Heiko Schlittermann wrote: > > Timo Sirainen (Di 13 Okt 2015 21:02:59 CEST): >>> the IP address the director connects to. >> >> Right. The hostnames are lost immediately at director startup. I've never really thought about needing this functionality for director, since they're usually in the same trusted network with backends.. >> > > Ooo. > What if > > director_mail_servers = backends. > > and the DNS entry for backends. gets updated? Does the director > catch up the change automatically w/o restart? No, and I'm not sure it even should. Use "doveadm director ring remove" to get rid of unwanted directors. From tss at iki.fi Tue Oct 13 19:42:41 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 13 Oct 2015 22:42:41 +0300 Subject: Dualstack IPv4/IPv6 setup with directors In-Reply-To: <20151013193146.GY4156@jumper.schlittermann.de> References: <20151013193146.GY4156@jumper.schlittermann.de> Message-ID: <38AC1A8A-E2D7-4BDC-BB9D-9638842BBB08@iki.fi> On 13 Oct 2015, at 22:31, Heiko Schlittermann wrote: > > Hi, > > still using 2.2.9, I've two directors, and these directors > use both IPv4/IPv6 addresses. > > `host directors.` returns one A and AAA for each > of the two directors: > > directors. has address 149.x.y.96 (director1) > directors. has address 149.x.y.97 (director2) > directors. has IPv6 address 2001:x:y:f33::5:1 (director1) > directors. has IPv6 address 2001:x:y:f33::5:2 (director2) > > But connecting the the neighbor director gives: > > Oct 13 21:23:29 director1 dovecot: director: Warning: Director 2001:x:y:f33::5:2:9090/left disconnected us with reason: Invalid input: ME 149.x.y.96 9090 > Oct 13 21:23:29 director1 dovecot: director: Error: director(149.x.y.97:9090/out): connect() failed: Connection refused > Oct 13 21:23:29 director1 dovecot: director: Warning: net_connect_ip(): ip->family != my_ip->family > Oct 13 21:23:29 director1 dovecot: director: Error: Remote director thinks it's someone else (connected to 2001:x:y:f33::5:1:9090, remote says it's 149.x.y.96:9090) > > I assume it's because the director uses only one (IPv4 OR IPv6) address > for it's own identity, right? Right. I thought I did something to fix this crash, but probably not. In any case it's not a supported configuration even if it didn't crash. From hs at schlittermann.de Tue Oct 13 19:49:05 2015 From: hs at schlittermann.de (Heiko Schlittermann) Date: Tue, 13 Oct 2015 21:49:05 +0200 Subject: Dualstack IPv4/IPv6 setup with directors In-Reply-To: <38AC1A8A-E2D7-4BDC-BB9D-9638842BBB08@iki.fi> References: <20151013193146.GY4156@jumper.schlittermann.de> <38AC1A8A-E2D7-4BDC-BB9D-9638842BBB08@iki.fi> Message-ID: <20151013194905.GZ4156@jumper.schlittermann.de> Timo Sirainen (Di 13 Okt 2015 21:42:41 CEST): ? > > Oct 13 21:23:29 director1 dovecot: director: Error: director(149.x.y.97:9090/out): connect() failed: Connection refused > > Oct 13 21:23:29 director1 dovecot: director: Warning: net_connect_ip(): ip->family != my_ip->family > > Oct 13 21:23:29 director1 dovecot: director: Error: Remote director thinks it's someone else (connected to 2001:x:y:f33::5:1:9090, remote says it's 149.x.y.96:9090) > > > > I assume it's because the director uses only one (IPv4 OR IPv6) address > > for it's own identity, right? > > Right. I thought I did something to fix this crash, but probably not. In any case it's not a supported configuration even if it didn't crash. Ok, maybe it should be mentioned in the Wiki, that having multiple addresses per directors isn't supported for director-director communication. But anyway, thanks for the information, it helps. -- Heiko -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From slitt at troubleshooters.com Tue Oct 13 20:00:37 2015 From: slitt at troubleshooters.com (Steve Litt) Date: Tue, 13 Oct 2015 16:00:37 -0400 Subject: Thanks for Dovecot Message-ID: <20151013160037.5f909e25@littlap.domain.cxm> Hi all, Thanks for making Dovecot. I just transitioned from Debian Wheezy to Void Linux. It was fairly easy to get Dovecot working on my Void box, and having Dovecot makes all of my email activities easier by doing one thing and doing it right. Thank you for such great software. SteveT From stephan at rename-it.nl Tue Oct 13 20:13:49 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 13 Oct 2015 22:13:49 +0200 Subject: Dovecot - Postfix with HAproxy In-Reply-To: <561D201D.5040901@neteven.com> References: <561D201D.5040901@neteven.com> Message-ID: <561D65FD.4010207@rename-it.nl> Op 10/13/2015 om 5:15 PM schreef Michael JOIGNY: (For some reason Thunderbird messes up this e-mail). You could try to do pre-login rawlog to see what HAProxy is sending to Dovecot. http://wiki2.dovecot.org/Debugging/Rawlog Regards, Stephan. From akbwiz+dovecot at gmail.com Tue Oct 13 20:25:12 2015 From: akbwiz+dovecot at gmail.com (Akash) Date: Wed, 14 Oct 2015 01:55:12 +0530 Subject: Indexing fails with .. FIELDS_INDEX_EXTENSION).c_str() )' failed Message-ID: Hi, Mailing list archives suggest that this problem has been fixed in the older versions itself but I am getting the error still while trying to doveadm index a large folder of emails. Sometimes it throws error after 30000, something 40000, the latest it gave up after 111000. But it just never completes. 111000/322080doveadm: /home/stephan/packages/wheezy/i386/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriter.cpp:210: std::string lucene::index::DocumentsWriter::closeDocStore(): Assertion `numDocsInStore*8 == directory->fileLength( (docStoreSegment + "." + IndexFileNames::FIELDS_INDEX_EXTENSION).c_str() )' failed. Aborted I am using dovecot 2:2.2.19-1~auto+7& libclucene-core1:i386 2.3.3.4-4 from debian wheezy backports. Please advice. -Akash From tss at iki.fi Tue Oct 13 20:28:42 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 13 Oct 2015 23:28:42 +0300 Subject: Indexing fails with .. FIELDS_INDEX_EXTENSION).c_str() )' failed In-Reply-To: References: Message-ID: On 13 Oct 2015, at 23:25, Akash wrote: > > Hi, > > Mailing list archives suggest that this problem has been fixed in the > older versions itself but I am getting the error still while trying to > doveadm index a large folder of emails. Sometimes it throws error > after 30000, something 40000, the latest it gave up after 111000. But > it just never completes. > > 111000/322080doveadm: > /home/stephan/packages/wheezy/i386/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriter.cpp:210: > std::string lucene::index::DocumentsWriter::closeDocStore(): Assertion > `numDocsInStore*8 == directory->fileLength( (docStoreSegment + "." + > IndexFileNames::FIELDS_INDEX_EXTENSION).c_str() )' failed. > Aborted > > I am using dovecot 2:2.2.19-1~auto+7& libclucene-core1:i386 2.3.3.4-4 > from debian wheezy backports. Please advice. Looks like a bug in clucene library. I don't know if there's anything that can be done about it on Dovecot side. I recommend switching to fts-solr. From hs at schlittermann.de Tue Oct 13 20:33:23 2015 From: hs at schlittermann.de (Heiko Schlittermann) Date: Tue, 13 Oct 2015 22:33:23 +0200 Subject: TLS communication director -> backend with X.509 cert checks? In-Reply-To: <9E604417-8950-4426-89A7-172B27AA47E9@iki.fi> References: <20151013184437.GV4156@jumper.schlittermann.de> <9831E5FF-3161-4CC9-ACA9-AB36F23C589B@iki.fi> <20151013191835.GW4156@jumper.schlittermann.de> <9E604417-8950-4426-89A7-172B27AA47E9@iki.fi> Message-ID: <20151013203323.GA4156@jumper.schlittermann.de> Timo Sirainen (Di 13 Okt 2015 21:36:40 CEST): ? > > I see: > > > > a) pass the host *names* to the director too, for CN verification > > purpose > > > > May be in struct mail_host could be a field for the original > > hostname we used to obtain the adress(es)? > > Does the attached patch work? Compiles, but untested. I'm about to test it. Best regards from Dresden/Germany Viele Gr??e aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From jtam.home at gmail.com Tue Oct 13 20:50:46 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 13 Oct 2015 13:50:46 -0700 (PDT) Subject: iterate users with passwd-file passdb? In-Reply-To: References: Message-ID: Eric Abrahamsen writes: > Simply: Is it possible to iterate over users if I'm using the > passwd-file passdb driver? Do I need a SQL-based driver if I want to > iterate? What do you mean by "iterate"? If you mean whether you can look up a password entry in a multi-entry file, then yes, definitely. If you mean to sequentially go through it and do a first/last/best match, probably not. Joseph Tam From eric at ericabrahamsen.net Tue Oct 13 21:01:30 2015 From: eric at ericabrahamsen.net (Eric Abrahamsen) Date: Tue, 13 Oct 2015 14:01:30 -0700 Subject: iterate users with passwd-file passdb? References: Message-ID: <87d1wiiitx.fsf@ericabrahamsen.net> Joseph Tam writes: > Eric Abrahamsen writes: > >> Simply: Is it possible to iterate over users if I'm using the >> passwd-file passdb driver? Do I need a SQL-based driver if I want to >> iterate? > > What do you mean by "iterate"? If you mean whether you can look up a > password entry in a multi-entry file, then yes, definitely. If you > mean to sequentially go through it and do a first/last/best match, > probably not. Basically I just mean the -A argument to the doveadm commands. For example, iterate through all the users and run sa-learn on each user's learn/spam folder. If it's got to be SQL, it won't kill me... Thanks! Eric From gedalya at gedalya.net Tue Oct 13 21:04:49 2015 From: gedalya at gedalya.net (Gedalya) Date: Tue, 13 Oct 2015 17:04:49 -0400 Subject: Thanks for Dovecot In-Reply-To: <20151013160037.5f909e25@littlap.domain.cxm> References: <20151013160037.5f909e25@littlap.domain.cxm> Message-ID: <561D71F1.9040001@gedalya.net> On 10/13/2015 04:00 PM, Steve Litt wrote: > Hi all, > > Thanks for making Dovecot. > > I just transitioned from Debian Wheezy to Void Linux. It was fairly > easy to get Dovecot working on my Void box, and having Dovecot makes > all of my email activities easier by doing one thing and doing it right. > > Thank you for such great software. > > SteveT Hey, you know what? It's never a bad time to join in and say a simple: Thank you! From mcguire at neurotica.com Tue Oct 13 21:08:55 2015 From: mcguire at neurotica.com (Dave McGuire) Date: Tue, 13 Oct 2015 17:08:55 -0400 Subject: Thanks for Dovecot In-Reply-To: <561D71F1.9040001@gedalya.net> References: <20151013160037.5f909e25@littlap.domain.cxm> <561D71F1.9040001@gedalya.net> Message-ID: <561D72E7.1000208@neurotica.com> On 10/13/2015 05:04 PM, Gedalya wrote: >> Thanks for making Dovecot. >> >> I just transitioned from Debian Wheezy to Void Linux. It was fairly >> easy to get Dovecot working on my Void box, and having Dovecot makes >> all of my email activities easier by doing one thing and doing it right. >> >> Thank you for such great software. >> >> SteveT > > Hey, you know what? It's never a bad time to join in and say a simple: > Thank you! Agreed! Thank you! -Dave -- Dave McGuire, AK4HZ New Kensington, PA From tss at iki.fi Tue Oct 13 21:20:30 2015 From: tss at iki.fi (Timo Sirainen) Date: Wed, 14 Oct 2015 00:20:30 +0300 Subject: iterate users with passwd-file passdb? In-Reply-To: <87d1wiiitx.fsf@ericabrahamsen.net> References: <87d1wiiitx.fsf@ericabrahamsen.net> Message-ID: On 14 Oct 2015, at 00:01, Eric Abrahamsen wrote: > > Joseph Tam writes: > >> Eric Abrahamsen writes: >> >>> Simply: Is it possible to iterate over users if I'm using the >>> passwd-file passdb driver? Do I need a SQL-based driver if I want to >>> iterate? >> >> What do you mean by "iterate"? If you mean whether you can look up a >> password entry in a multi-entry file, then yes, definitely. If you >> mean to sequentially go through it and do a first/last/best match, >> probably not. > > Basically I just mean the -A argument to the doveadm commands. For > example, iterate through all the users and run sa-learn on each user's > learn/spam folder. If it's got to be SQL, it won't kill me... Use userdb passwd-file to get iteration working. passdb isn't used for iteration. userdb static can't be used to iterate. From jtam.home at gmail.com Tue Oct 13 21:27:28 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 13 Oct 2015 14:27:28 -0700 (PDT) Subject: iterate users with passwd-file passdb? In-Reply-To: References: Message-ID: Eric Abrahamsen writes: >>> Simply: Is it possible to iterate over users if I'm using the >>> passwd-file passdb driver? Do I need a SQL-based driver if I want to >>> iterate? >> >> What do you mean by "iterate"? If you mean whether you can look up a >> password entry in a multi-entry file, then yes, definitely. If you >> mean to sequentially go through it and do a first/last/best match, >> probably not. > > Basically I just mean the -A argument to the doveadm commands. For > example, iterate through all the users and run sa-learn on each user's > learn/spam folder. If it's got to be SQL, it won't kill me... Ah. Yes, you can use -A and iterate through users, subject to any constraints such as first_valid_uid, last_valid_uid, etc. Joseph Tam From hs at schlittermann.de Tue Oct 13 21:34:47 2015 From: hs at schlittermann.de (Heiko Schlittermann) Date: Tue, 13 Oct 2015 23:34:47 +0200 Subject: TLS communication director -> backend with X.509 cert checks? In-Reply-To: <20151013203323.GA4156@jumper.schlittermann.de> References: <20151013184437.GV4156@jumper.schlittermann.de> <9831E5FF-3161-4CC9-ACA9-AB36F23C589B@iki.fi> <20151013191835.GW4156@jumper.schlittermann.de> <9E604417-8950-4426-89A7-172B27AA47E9@iki.fi> <20151013203323.GA4156@jumper.schlittermann.de> Message-ID: <20151013213447.GB4156@jumper.schlittermann.de> Hi Timo, Heiko Schlittermann (Di 13 Okt 2015 22:33:23 CEST): > > Does the attached patch work? Compiles, but untested. > I'm about to test it. It seems to update the struct mail_host, but it looks as if the data in mail_host do not propagate down to login_proxy_new(). In other words, in login_proxy_new() set->host contains the IP address, correctly, because the director choose it, but where can I find the hostname there? And we need a way to pass the host*name* further, to the SSL verifcation step, don't we? Best regards from Dresden/Germany Viele Gr??e aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From tss at iki.fi Tue Oct 13 21:49:20 2015 From: tss at iki.fi (Timo Sirainen) Date: Wed, 14 Oct 2015 00:49:20 +0300 Subject: TLS communication director -> backend with X.509 cert checks? In-Reply-To: <20151013213447.GB4156@jumper.schlittermann.de> References: <20151013184437.GV4156@jumper.schlittermann.de> <9831E5FF-3161-4CC9-ACA9-AB36F23C589B@iki.fi> <20151013191835.GW4156@jumper.schlittermann.de> <9E604417-8950-4426-89A7-172B27AA47E9@iki.fi> <20151013203323.GA4156@jumper.schlittermann.de> <20151013213447.GB4156@jumper.schlittermann.de> Message-ID: On 14 Oct 2015, at 00:34, Heiko Schlittermann wrote: > > Hi Timo, > > Heiko Schlittermann (Di 13 Okt 2015 22:33:23 CEST): >>> Does the attached patch work? Compiles, but untested. >> I'm about to test it. > > It seems to update the struct mail_host, but it looks as if the data > in mail_host do not propagate down to login_proxy_new(). > > In other words, in login_proxy_new() set->host contains the IP address, > correctly, because the director choose it, but where can I find the > hostname there? And we need a way to pass the host*name* further, to the > SSL verifcation step, don't we? Proxying in general does check that hostname matches the SSL certificate, because both the hostname and IP address are sent to login process. So it should work in a way that host= and hostip= is sent. I thought my patch did that.. Normally auth_debug=yes would be enough to debug this, but this happens between director and login process so I don't think it's going to be of much use. login process's client_auth_parse_args() is what should see these two parameters correctly. I can check this further tomorrow. From eric at ericabrahamsen.net Tue Oct 13 21:54:24 2015 From: eric at ericabrahamsen.net (Eric Abrahamsen) Date: Tue, 13 Oct 2015 14:54:24 -0700 Subject: iterate users with passwd-file passdb? References: <87d1wiiitx.fsf@ericabrahamsen.net> Message-ID: <87612aigdr.fsf@ericabrahamsen.net> Timo Sirainen writes: > On 14 Oct 2015, at 00:01, Eric Abrahamsen wrote: >> >> Joseph Tam writes: >> >>> Eric Abrahamsen writes: >>> >>>> Simply: Is it possible to iterate over users if I'm using the >>>> passwd-file passdb driver? Do I need a SQL-based driver if I want to >>>> iterate? >>> >>> What do you mean by "iterate"? If you mean whether you can look up a >>> password entry in a multi-entry file, then yes, definitely. If you >>> mean to sequentially go through it and do a first/last/best match, >>> probably not. >> >> Basically I just mean the -A argument to the doveadm commands. For >> example, iterate through all the users and run sa-learn on each user's >> learn/spam folder. If it's got to be SQL, it won't kill me... > > Use userdb passwd-file to get iteration working. passdb isn't used for iteration. userdb static can't be used to iterate. That did it, thank you! I had been staring at this: http://wiki2.dovecot.org/AuthDatabase/PasswdFile#line-38 But not getting it exactly right. For posterity, I edited my conf to look like this: passdb { driver = passwd-file args = username_format=%u scheme=ssha512 /etc/dovecot/passwd.db deny = no master = no pass = no skip = never result_failure = continue result_internalfail = continue result_success = return-ok } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/passwd.db default_fields = uid=vmail gid=vmail home=/var/mail/vmail/%d/%n } Then edited /etc/dovecot/passwd.db to add six colons at the end of each user line -- empty fields since presently everything I need is specified in default_fields. Thanks again, Eric From micah at riseup.net Tue Oct 13 21:56:43 2015 From: micah at riseup.net (micah) Date: Tue, 13 Oct 2015 17:56:43 -0400 Subject: Thanks for Dovecot In-Reply-To: <561D72E7.1000208@neurotica.com> References: <20151013160037.5f909e25@littlap.domain.cxm> <561D71F1.9040001@gedalya.net> <561D72E7.1000208@neurotica.com> Message-ID: <87612al9es.fsf@muck.riseup.net> Dave McGuire writes: > On 10/13/2015 05:04 PM, Gedalya wrote: >>> Thanks for making Dovecot. >>> >>> I just transitioned from Debian Wheezy to Void Linux. It was fairly >>> easy to get Dovecot working on my Void box, and having Dovecot makes >>> all of my email activities easier by doing one thing and doing it right. >>> >>> Thank you for such great software. >>> >>> SteveT >> >> Hey, you know what? It's never a bad time to join in and say a simple: >> Thank you! > > Agreed! Thank you! I want to jump on the puppy pile of love... I've used a lot of different IMAP software over the decades, and dovecot has been a breath of fresh air and a huge relief compared to what I've suffered through in the past! Thanks Timo for all your hard work, it really is appreciated! micah From hs at schlittermann.de Tue Oct 13 22:10:50 2015 From: hs at schlittermann.de (Heiko Schlittermann) Date: Wed, 14 Oct 2015 00:10:50 +0200 Subject: TLS communication director -> backend with X.509 cert checks? In-Reply-To: References: <20151013184437.GV4156@jumper.schlittermann.de> <9831E5FF-3161-4CC9-ACA9-AB36F23C589B@iki.fi> <20151013191835.GW4156@jumper.schlittermann.de> <9E604417-8950-4426-89A7-172B27AA47E9@iki.fi> <20151013203323.GA4156@jumper.schlittermann.de> <20151013213447.GB4156@jumper.schlittermann.de> Message-ID: <20151013221050.GC4156@jumper.schlittermann.de> Timo Sirainen (Di 13 Okt 2015 23:49:20 CEST): ? > > Proxying in general does check that hostname matches the SSL certificate, because both the hostname and IP address are sent to login process. So it should work in a way that host= and hostip= is sent. I thought my patch did that.. Normally auth_debug=yes would be enough to debug this, but this happens between director and login process so I don't think it's going to be of much use. login process's client_auth_parse_args() is what should see these two parameters correctly. > > I can check this further tomorrow. I've put an i_warning("*** %s: ...", __FUNCTION__, ...) into several places. Oct 14 00:02:33 director1 dovecot: director: Warning: *** login_host_callback: OK#0112#011user=foo#011proxy#011ssl=yes#011nopassword=y#011lip=2001:x.y:f33::5:1#011lport=993#011pass=x#011proxy_refresh=450#011host=2001:x.y:f33::5:fe Here it seems that the director doesn't send it's knowledge about the hostname. Here some other output, to show that the host list contains names and addresses: Oct 14 00:02:32 director1 dovecot: director: Warning: ** mail_host_add: added backends. [2001:x.y:f33::5:fe] Oct 14 00:02:32 director1 dovecot: director: Warning: ** mail_host_add: added backends. [2001:x.y:f33::5:ff] Oct 14 00:02:32 director1 dovecot: director: Warning: ** mail_host_add: added backends. [149.x.y.103] Oct 14 00:02:32 director1 dovecot: director: Warning: ** mail_host_add: added backends. [149.x.y.102] -- Heiko -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From hs at schlittermann.de Tue Oct 13 22:46:11 2015 From: hs at schlittermann.de (Heiko Schlittermann) Date: Wed, 14 Oct 2015 00:46:11 +0200 Subject: TLS communication director -> backend with X.509 cert checks? In-Reply-To: <20151013221050.GC4156@jumper.schlittermann.de> References: <20151013184437.GV4156@jumper.schlittermann.de> <9831E5FF-3161-4CC9-ACA9-AB36F23C589B@iki.fi> <20151013191835.GW4156@jumper.schlittermann.de> <9E604417-8950-4426-89A7-172B27AA47E9@iki.fi> <20151013203323.GA4156@jumper.schlittermann.de> <20151013213447.GB4156@jumper.schlittermann.de> <20151013221050.GC4156@jumper.schlittermann.de> Message-ID: <20151013224611.GD4156@jumper.schlittermann.de> Heiko Schlittermann (Mi 14 Okt 2015 00:10:50 CEST): > Timo Sirainen (Di 13 Okt 2015 23:49:20 CEST): > ? > > > > Proxying in general does check that hostname matches the SSL certificate, because both the hostname and IP address are sent to login process. So it should work in a way that host= and hostip= is sent. I thought my patch did that.. Normally auth_debug=yes would be enough to debug this, but this happens between director and login process so I don't think it's going to be of much use. login process's client_auth_parse_args() is what should see these two parameters correctly. > > > > I can check this further tomorrow. > > I've put an i_warning("*** %s: ...", __FUNCTION__, ...) into several places. > > Oct 14 00:02:33 director1 dovecot: director: Warning: *** login_host_callback: OK#0112#011user=foo#011proxy#011ssl=yes#011nopassword=y#011lip=2001:x.y:f33::5:1#011lport=993#011pass=x#011proxy_refresh=450#011host=2001:x.y:f33::5:fe > > Here it seems that the director doesn't send it's knowledge about the > hostname. > > Here some other output, to show that the host list contains names and addresses: > > Oct 14 00:02:32 director1 dovecot: director: Warning: ** mail_host_add: added backends. [2001:x.y:f33::5:fe] > Oct 14 00:02:32 director1 dovecot: director: Warning: ** mail_host_add: added backends. [2001:x.y:f33::5:ff] > Oct 14 00:02:32 director1 dovecot: director: Warning: ** mail_host_add: added backends. [149.x.y.103] > Oct 14 00:02:32 director1 dovecot: director: Warning: ** mail_host_add: added backends. [149.x.y.102] And if I add -D to the director service, I can see "Debug: request refreshed timeout to ?", but never I see "Debug: request added". And from what I understand this would be the place where the mail_host info comes into the game. But probably I do not understand how director_request_continue() is supposed to work. Best regards from Dresden/Germany Viele Gr??e aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From tim at timg.ws Tue Oct 13 23:03:47 2015 From: tim at timg.ws (Tim Groeneveld) Date: Wed, 14 Oct 2015 10:03:47 +1100 Subject: Indexing fails with .. FIELDS_INDEX_EXTENSION).c_str() )' failed In-Reply-To: References: Message-ID: <15063720163.ddcbb3c397513.7592718327009795148@timg.ws> ---- On Wed, 14 Oct 2015 07:28:42 +1100 Timo Sirainen wrote ---- > Looks like a bug in clucene library. I don't know if there's anything that can be done about it on Dovecot side. I recommend switching to fts-solr. Either that or fts-elasticsearch[1] Regards, Tim [1] https://github.com/ascendantcom/fts-elasticsearch/ From hs at schlittermann.de Tue Oct 13 23:10:20 2015 From: hs at schlittermann.de (Heiko Schlittermann) Date: Wed, 14 Oct 2015 01:10:20 +0200 Subject: TLS communication director -> backend with X.509 cert checks? In-Reply-To: <20151013224611.GD4156@jumper.schlittermann.de> References: <20151013184437.GV4156@jumper.schlittermann.de> <9831E5FF-3161-4CC9-ACA9-AB36F23C589B@iki.fi> <20151013191835.GW4156@jumper.schlittermann.de> <9E604417-8950-4426-89A7-172B27AA47E9@iki.fi> <20151013203323.GA4156@jumper.schlittermann.de> <20151013213447.GB4156@jumper.schlittermann.de> <20151013221050.GC4156@jumper.schlittermann.de> <20151013224611.GD4156@jumper.schlittermann.de> Message-ID: <20151013231020.GE4156@jumper.schlittermann.de> Heiko Schlittermann (Mi 14 Okt 2015 00:46:11 CEST): ? > > And if I add -D to the director service, I can see "Debug: request refreshed timeout to ?", > but never I see "Debug: request added". And from what I > understand this would be the place where the mail_host info comes into > the game. > > But probably I do not understand how director_request_continue() is > supposed to work. Ah, the information comes from the other director running. The other one is using an unpatched version of dovecot. If I shutdown the other director instance, it seems to work. Tomorrow I'll do more testing. Good work, thank you. BTW: I've put there an IPv6 address into the director_servers list (not an DNS name). director_servers = 2001:x:y:f33::5:1 ? inet_listener { address = :: port = 9090 } it doesn't recognize itself: Oct 14 01:06:13 director1 dovecot: director: Fatal: director_servers doesn't list ourself director_servers = 2001:x:y:f33::5:1:9090 ? inet_listener { address = :: port = 9090 } works, but is ambigous, isn't it? Shouldn't we use [2001:x:y:f33::5:1]:9090 in such a case? But: *Unknown director host: [2001:x:y:f33::5:1]* Best regards from Dresden/Germany Viele Gr??e aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From emailgrant at gmail.com Wed Oct 14 00:28:21 2015 From: emailgrant at gmail.com (Grant) Date: Tue, 13 Oct 2015 17:28:21 -0700 Subject: Thanks for Dovecot In-Reply-To: <20151013160037.5f909e25@littlap.domain.cxm> References: <20151013160037.5f909e25@littlap.domain.cxm> Message-ID: > Thanks for making Dovecot. > > I just transitioned from Debian Wheezy to Void Linux. It was fairly > easy to get Dovecot working on my Void box, and having Dovecot makes > all of my email activities easier by doing one thing and doing it right. > > Thank you for such great software. Same here. I switched from courier to dovecot and one of my users with a very heavy inbox called it "scary fast". Should have switched years ago. - Grant From jaldhar at debian.org Wed Oct 14 03:55:49 2015 From: jaldhar at debian.org (Jaldhar H. Vyas) Date: Tue, 13 Oct 2015 23:55:49 -0400 (EDT) Subject: Thanks for Dovecot In-Reply-To: <20151013160037.5f909e25@littlap.domain.cxm> References: <20151013160037.5f909e25@littlap.domain.cxm> Message-ID: On Tue, 13 Oct 2015, Steve Litt wrote: > Thanks for making Dovecot. +1 > > I just transitioned from Debian Wheezy to Void Linux. It was fairly > easy to get Dovecot working on my Void box, and having Dovecot makes > all of my email activities easier by doing one thing and doing it right. > I'm just curious, did you not find it easy to get Dovecot working on Debian? I ask because if you didn't I'm largely the one to thank :-) -- Jaldhar H. Vyas From grupodecorreo10 at gmail.com Wed Oct 14 07:43:40 2015 From: grupodecorreo10 at gmail.com (grupo correo) Date: Wed, 14 Oct 2015 09:43:40 +0200 Subject: Problems sync with mail client MAC and dovecot-2.2.10-4.el7_0.1.el7_0.1.x86_64 Message-ID: > > On 2015-10-13 09:20, grupo correo wrote: > > Hello, > > > > i have installed Centos 7.1.1503 with Dovecot 2.2.10-4.el7_0.1. With > > Thunderbird all works fine, but when i configure the email client of > > Mac > > (Yosemite) with IMAP i can see a error in the log and the client not > > see > > the emails: > > > > Oct 9 14:31:34 localhost dovecot: imap(xxxxxxx at xxxxxxx): Fatal: > > master: > > service(imap): child 18639 killed with signal 11 (core dumps disabled) > > Do you have more logs? > > Have you enabled mail_debug = yes to get more verbose logs? > > > > > I have read the instructions of http://www.dovecot.org/bugreport.html, > > but > > core file is not writing. > > > > Does anyone have this issues? I think that can be a problem in the > > configuration > > Please share your doveconf -n output. > > > Best regards > > regards > christian > Attached the doveconf -n Yes i have mail_debug = yes into file conf.d/10-logging.conf I paste the rest of log: Oct 13 21:27:17 localhost dovecot: imap: Debug: Loading modules from directory: /usr/lib64/dovecot Oct 13 21:27:17 localhost dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib01_acl_plugin.so Oct 13 21:27:17 localhost dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib02_imap_acl_plugin.so Oct 13 21:27:17 localhost dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib10_quota_plugin.so Oct 13 21:27:17 localhost dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib11_imap_quota_plugin.so Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: Effective uid=56396, gid=55000, home=/VMAIL/xxxxxxxxxxxxxx Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: Quota root: name=User quota backend=maildir args= Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: Quota grace: root=User quota bytes=0 (10%) Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: Namespace inbox: type=private, prefix=INBOX., sep=., inbox=yes, hidden=no, list=children, subscriptions=yes location=maildir:~/Maildir Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: maildir++: root=/VMAIL/xxxxxxxxxxxxxx/Maildir, index=, indexpvt=, control=, inbox=/VMAIL/xxxxxxxxxxxxxx/Maildir, alt= Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: acl: initializing backend with data: vfile Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: acl: acl username = xxxxxxxxxxxxxx Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: acl: owner = 1 Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: acl vfile: Global ACL directory: (none) Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Fatal: master: service(imap): child 2464 killed with signal 11 (core dumps disabled) -------------- next part -------------- # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-229.11.1.el7.x86_64 x86_64 CentOS Linux release 7.1.1503 (Core) base_dir = /var/run/dovecot/ default_vsz_limit = 512 M disable_plaintext_auth = no first_valid_gid = 0 first_valid_uid = 0 imap_idle_notify_interval = 1 mins lock_method = dotlock mail_debug = yes mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mbox_write_locks = dotlock mmap_disable = yes namespace inbox { inbox = yes list = children location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile acl_shared_dict = file:/VMAIL/%d/shared-mailboxes quota = maildir:User quota } protocols = imap pop3 service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service imap-postlogin { executable = script-login /usr/local/bin/postlogin } service imap { executable = imap imap-postlogin } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } ssl_cert = References: Message-ID: <33ba2eacef5dcfcb4c703615e559b094@valo.at> > Attached the doveconf -n > Yes i have mail_debug = yes into file conf.d/10-logging.conf > I paste the rest of log: > Oct 13 21:27:17 localhost dovecot: imap: Debug: Loading modules from > directory: /usr/lib64/dovecot > Oct 13 21:27:17 localhost dovecot: imap: Debug: Module loaded: > /usr/lib64/dovecot/lib01_acl_plugin.so > Oct 13 21:27:17 localhost dovecot: imap: Debug: Module loaded: > /usr/lib64/dovecot/lib02_imap_acl_plugin.so > Oct 13 21:27:17 localhost dovecot: imap: Debug: Module loaded: > /usr/lib64/dovecot/lib10_quota_plugin.so > Oct 13 21:27:17 localhost dovecot: imap: Debug: Module loaded: > /usr/lib64/dovecot/lib11_imap_quota_plugin.so > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: > Effective > uid=56396, gid=55000, home=/VMAIL/xxxxxxxxxxxxxx > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: Quota > root: > name=User quota backend=maildir args= > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: Quota > grace: root=User quota bytes=0 (10%) > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: > Namespace > inbox: type=private, prefix=INBOX., sep=., inbox=yes, hidden=no, > list=children, subscriptions=yes location=maildir:~/Maildir Your inbox namespace seems to be ok and is also defined in your doveconf -n output. > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: > maildir++: > root=/VMAIL/xxxxxxxxxxxxxx/Maildir, index=, indexpvt=, control=, > inbox=/VMAIL/xxxxxxxxxxxxxx/Maildir, alt= > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: acl: > initializing backend with data: vfile > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: acl: > acl > username = xxxxxxxxxxxxxx > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: acl: > owner > = 1 > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: acl > vfile: > Global ACL directory: (none) > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: > Namespace : > type=private, prefix=, sep=, inbox=no, hidden=yes, list=no, > subscriptions=no location=fail::LAYOUT=none Somehow there seems to be an empty namespace in addition to your inbox namespace. Is there a second namespace in your config somewhere? > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: none: > root=, index=, indexpvt=, control=, inbox=, alt= > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Fatal: master: > service(imap): child 2464 killed with signal 11 (core dumps disabled) Just guessing: Could it be with Thunderbird that the second namespace is ignored when there is no subscription to it and the Apple mailprogram behaves differently? regards christian From akbwiz+dovecot at gmail.com Wed Oct 14 10:20:09 2015 From: akbwiz+dovecot at gmail.com (Akash) Date: Wed, 14 Oct 2015 15:50:09 +0530 Subject: Strange indexing behavior on HTML emails .. Message-ID: <7fde63232b2bf2cb24f3b45f0b93b55d@mailjol.in> Hi, In continuation to the issue I had posted about long back: http://www.dovecot.org/list/dovecot/2014-August/097362.html I did further testing today on a fresh new Debian & latest Dovecot and observed a undesired behavior. I am using fts_lucene & following sequence of commands on a empty test account me at myself.com: doveadm expunge -u 'my at myself.com' mailbox 'INBOX' all cat test.eml | /usr/lib/dovecot/dovecot-lda -e -f you at yourself.com -d me at myself.com doveadm search -u 'akash at mailjol.in' mailbox 'INBOX' text ABCD Search command does or doesn't find the email with slight variation in the content of test.eml. Here are the results: test.eml content: ----------------------------- From: you at yourself.com To: me at myself.com Subject: Test Message Content-Type: text/html
ABCD 1234
----------------------------- RESULT: OK. The email is found. test.eml content (double quotes inside div tag replaced with single): ----------------------------- From: you at yourself.com To: me at myself.com Subject: Test Message Content-Type: text/html
ABCD 1234
----------------------------- RESULT: None. The email isn't found. test.eml content (single quotes in div but content/type header removed): ----------------------------- From: you at yourself.com To: me at myself.com Subject: Test Message
ABCD 1234
----------------------------- RESULT: OK. The email is found. What could be the reason for this? -Akash From akbwiz+dovecot at gmail.com Wed Oct 14 11:31:13 2015 From: akbwiz+dovecot at gmail.com (Akash) Date: Wed, 14 Oct 2015 17:01:13 +0530 Subject: Strange indexing behavior on HTML emails .. In-Reply-To: <7fde63232b2bf2cb24f3b45f0b93b55d@mailjol.in> References: <7fde63232b2bf2cb24f3b45f0b93b55d@mailjol.in> Message-ID: <844797630c34c97766625e0ed02380ba@mailjol.in> The issue is probably linked to: http://www.dovecot.org/list/dovecot-cvs/2014-May/024462.html But that change-set was in 2014 and I am using Dovecot 2.2.19 so don't understand why I am still seeing this behavior. -Akash From akbwiz+dovecot at gmail.com Wed Oct 14 15:03:56 2015 From: akbwiz+dovecot at gmail.com (Akash) Date: Wed, 14 Oct 2015 20:33:56 +0530 Subject: Strange indexing behavior on HTML emails .. In-Reply-To: <7fde63232b2bf2cb24f3b45f0b93b55d@mailjol.in> References: <7fde63232b2bf2cb24f3b45f0b93b55d@mailjol.in> Message-ID: <895954ff5f750afd71fe25b88fbb41c4@mailjol.in> Tried latest source from HG and with solr also apart from lucene which I tested previously. The problem with single quotes in HTML is still there. The revision: http://hg.dovecot.org/dovecot-2.2/rev/ad028a950248 should have solved it but the relevant code no longer exists in src/plugins/fts/fts-parser-html.c. Seems like it has been moved into lib-mail. The file src/lib-mail/mail-html2text.c does contain something about single quotes but to no avail. Can someone at-least confirm existence of this issue? From grupodecorreo10 at gmail.com Wed Oct 14 15:53:26 2015 From: grupodecorreo10 at gmail.com (grupo correo) Date: Wed, 14 Oct 2015 17:53:26 +0200 Subject: Emails with sender MISSING_MAILBOX@missing_domain Message-ID: Hello, we are working in the migration from courier-imap to dovecot 2.2.10-4.el7_0.1.x86_64. Some emails that send a internal application show as sender "MISSING_MAILBOX at missing_domain", i use like email client Horde using dovecot like server. But if i open that email with Horde and courier-imap i can see the correct email sender. I think that the email of the sender is no correct or not accomplish some RFC. Can anyone help me to understand this? Thank you From hs at schlittermann.de Wed Oct 14 21:28:43 2015 From: hs at schlittermann.de (Heiko Schlittermann) Date: Wed, 14 Oct 2015 23:28:43 +0200 Subject: TLS communication director -> backend with X.509 cert checks? In-Reply-To: <20151013231020.GE4156@jumper.schlittermann.de> References: <20151013184437.GV4156@jumper.schlittermann.de> <9831E5FF-3161-4CC9-ACA9-AB36F23C589B@iki.fi> <20151013191835.GW4156@jumper.schlittermann.de> <9E604417-8950-4426-89A7-172B27AA47E9@iki.fi> <20151013203323.GA4156@jumper.schlittermann.de> <20151013213447.GB4156@jumper.schlittermann.de> <20151013221050.GC4156@jumper.schlittermann.de> <20151013224611.GD4156@jumper.schlittermann.de> <20151013231020.GE4156@jumper.schlittermann.de> Message-ID: <20151014212843.GF4156@jumper.schlittermann.de> Hi Timo Heiko Schlittermann (Mi 14 Okt 2015 01:10:20 CEST): ? > Ah, the information comes from the other director running. The other one > is using an unpatched version of dovecot. Your patch for backend-certificate verification works. Thank you for the good and fast work. Is there any chance that this will make it into Dovecot's next release? BTW: The ambiguity of 2001:db8::9090 remains. Shouldn't you allow [2001:db8::]? resp [2001:db8::9090]? resp. [2001:db8::]:9090? for such cases? (In case one want's to use IPv6 addresses instead of hostnames in the director_servers option. (And probably in other places too.)) ?) Address ?) Address:Port Best regards from Dresden/Germany Viele Gr??e aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From semfly at gmail.com Wed Oct 14 22:51:22 2015 From: semfly at gmail.com (Simon) Date: Wed, 14 Oct 2015 15:51:22 -0700 (PDT) Subject: More information about - Dovecot 2.2.x quota mysql and dict In-Reply-To: <1444170893263-54077.post@n4.nabble.com> References: <1444170893263-54077.post@n4.nabble.com> Message-ID: <1444863082842-54152.post@n4.nabble.com> I am sorry... but may be already somebody solved this bug ? -- View this message in context: http://dovecot.2317879.n4.nabble.com/More-information-about-Dovecot-2-2-x-quota-mysql-and-dict-tp54077p54152.html Sent from the Dovecot mailing list archive at Nabble.com. From kilburna at gmail.com Thu Oct 15 03:26:36 2015 From: kilburna at gmail.com (Kilburn Abrahams) Date: Thu, 15 Oct 2015 14:26:36 +1100 Subject: Dovecot top stats Message-ID: Hi, I have being try to track down top email users to sometime. When I do a network traffic check I can see there is about 15 times more email traffic pulled from mail server than sent. The problem I am trying to track down is which users are the culprit. I have enabled doevcot stats and I can do a doveadm stats dump user but I get a 0 for disk_input, disk_output, read_bytes and write_bytes. I do get mail_read_bytes numbers but I am not sure these numbers are correct. I there a top utility that works well so I can see email traffic from users mailboxes. The doveadm top does not work well. I also tried the perl scripts and that does not do a much better job. Thanks Kilburn From skdovecot at smail.inf.fh-brs.de Thu Oct 15 06:52:52 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 15 Oct 2015 08:52:52 +0200 (CEST) Subject: Emails with sender MISSING_MAILBOX@missing_domain In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 14 Oct 2015, grupo correo wrote: > we are working in the migration from courier-imap to dovecot > 2.2.10-4.el7_0.1.x86_64. > > Some emails that send a internal application show as sender > "MISSING_MAILBOX at missing_domain", i use like email client Horde using > dovecot like server. But if i open that email with Horde and courier-imap i > can see the correct email sender. > > I think that the email of the sender is no correct or not accomplish some > RFC. Can anyone help me to understand this? Would you post the headers of the message? - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVh9NRHz1H7kL/d9rAQJBzQf+JIC12Ztzr9WLVJcQkj5qOWRd3Ow0M7/S g6Tp7LRum+m8rbylUaA0qnc1L+EGQnwXZms0EOEZiIGI8BeKGwoNKt90eq9k3AE3 Lf/XeGxV7jGc2Ll+zxiiI584mYqcJwU1k0IfNevPGnGSY4sDt1pkPywCuigqk498 z097RgwptOg6Wo3KmqrA5Cd/D2EwG6ZDHAM1MeJEP+ZkGmePjr5FLy0jpuBbr5ge KIcXtvDAkdR80PrVAZRP9pDzMlu3IbjJlBaV28l0slWZmBCh+ePAk5U+0l8V9Itm p9+y3b+6t9pgL7wsWnk0SLJdlEhHQkQH9tG2VI4gBqCyiFIEj/ZD8Q== =xtXO -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Thu Oct 15 07:17:30 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 15 Oct 2015 09:17:30 +0200 (CEST) Subject: Dovecot top stats In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 15 Oct 2015, Kilburn Abrahams wrote: > I have enabled doevcot stats and I can do a doveadm stats dump user but I > get a 0 for disk_input, disk_output, read_bytes and write_bytes. I do get > mail_read_bytes numbers but I am not sure these numbers are correct. my stats contain numbers greater than zero, except for very few users. Did you tested it with some account and waited a bit, incl. logins/logouts? - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVh9TCnz1H7kL/d9rAQIb4gf/T+sUh08end+qFaDozYpNM/grWmFYKL+S kkqR9xYOrZJePJkkkJIrGBOOzy9ie8ZQlfKkWXoyv4gu22Me6oSszzoDiaPekov7 P/RHVm/iUhKRFqINfMAbP91SlJwWK7Ht+MkMdSqGuwRoC2zovv1DjWUOsaMctJ/T c63nJL+Q7zlGJD1yfZwq9Tu69b4y19vn2uXGAi0yL4WgHdgK1JAk2R3U4JTyPjpx BjAd4txtTVHSKqvU3GVL83MsESg/3g80wMgZS9SkGmnRZv4BPd15EIgjNqo6fOLy 35hJ1ItgpqnApJETHEdUyD2f5y7GLrpoy3vLbWGRmUokd2OyV4DIcw== =hz1I -----END PGP SIGNATURE----- From alec at alec.pl Thu Oct 15 07:53:46 2015 From: alec at alec.pl (A.L.E.C) Date: Thu, 15 Oct 2015 09:53:46 +0200 Subject: Emails with sender MISSING_MAILBOX@missing_domain In-Reply-To: References: Message-ID: <561F5B8A.9020403@alec.pl> On 10/15/2015 08:52 AM, Steffen Kaiser wrote: >> I think that the email of the sender is no correct or not accomplish some >> RFC. Can anyone help me to understand this? https://bugs.horde.org/ticket/13436 -- Aleksander 'A.L.E.C' Machniak Kolab Groupware Developer [http://kolab.org] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From tlx at leuxner.net Thu Oct 15 09:22:02 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 15 Oct 2015 11:22:02 +0200 Subject: iterate users with passwd-file passdb? In-Reply-To: References: <87d1wiiitx.fsf@ericabrahamsen.net> Message-ID: <20151015092202.GA62903@nihlus.leuxner.net> * Timo Sirainen 2015.10.13 23:20: > Use userdb passwd-file to get iteration working. passdb isn't used for iteration. userdb static can't be used to iterate. I guess this works for a single file comprising of all users at the moment: Oct 15 11:13:58 nihlus dovecot: auth-worker(5545): Error: passwd-file: User iteration isn't currently supported with %variable paths Is variable support planned? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: Digital signature URL: From michal.zila at somi.sk Thu Oct 15 11:12:20 2015 From: michal.zila at somi.sk (=?utf-8?Q?Michal_=C5=BDila?=) Date: Thu, 15 Oct 2015 13:12:20 +0200 Subject: master user in dovecot 1.2 Message-ID: <172EDF1A-8BE5-4570-9540-63A36BAD8676@somi.sk> Hello list, i would like to use master user functionality in dovecot v1.2, but i just cant get it to work: Here?s my configuration: # dovecot -n # 1.2.9: /etc/dovecot.conf # OS: Linux 2.6.18-164.10.1.el5PAE i686 Red Hat Enterprise Linux Server release 5.4 (Tikanga) login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login first_valid_uid: 200 mail_privileged_group: mail mail_location: mbox:~/mail:INBOX=/var/mail/%u mail_executable(default): /root/dovecot-master-postlogin mail_executable(imap): /root/dovecot-master-postlogin mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 lda: postmaster_address: postmaster at example.com auth default: master_user_separator: * debug: yes passdb: driver: passwd-file args: /etc/dovecot.passwd.masterusers pass: yes master: yes passdb: driver: pam userdb: driver: passwd # cat /root/dovecot-master-postlogin #!/bin/bash MASTER_USER=$USER export MASTER_USER exec /usr/libexec/dovecot/imap "$@? # cat /etc/dovecot.passwd.masterusers masteruser:{PLAIN}Heslo42 Trying login as normal user: # telnet localhost imap Trying 127.0.0.1... Connected to localhost.localdomain (127.0.0.1). Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE STARTTLS AUTH=PLAIN] Dovecot ready. a1 login jozefkova jozefkova026 a1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH] Logged in Trying login as master user: # telnet localhost imap Trying 127.0.0.1... Connected to localhost.localdomain (127.0.0.1). Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE STARTTLS AUTH=PLAIN] Dovecot ready. a1 login jozefkova*masteruser Heslo42 a1 NO [AUTHORIZATIONFAILED] Authorization failed log sequence of failed auth: Oct 15 13:04:47 linux dovecot: auth(default): client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=34618 resp= Oct 15 13:04:47 linux dovecot: auth(default): passwd-file(masteruser,127.0.0.1,master): lookup: user=masteruser file=/etc/dovecot.passwd.masterusers Oct 15 13:04:47 linux dovecot: auth(default): passdb(masteruser,127.0.0.1,master): Master user logging in as jozefkova Oct 15 13:04:47 linux dovecot: auth-worker(default): pam(jozefkova,127.0.0.1): lookup service=dovecot Oct 15 13:04:47 linux dovecot: auth-worker(default): pam(jozefkova,127.0.0.1): #1/1 style=1 msg=Password: Oct 15 13:04:49 linux dovecot: auth-worker(default): pam(jozefkova,127.0.0.1): pam_authenticate() failed: Authentication failure (password mismatch?) Oct 15 13:04:51 linux dovecot: auth(default): client out: FAIL 1 user=jozefkova authz So i can see it?s trying to loggin in using pam mech. Is there something i can configure in dovecot userdb or is it something else? I use pam to authenticate my real users? Thank you in advance? ? Michal Zila -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4021 bytes Desc: not available URL: From phil at dovecot.fi Thu Oct 15 11:41:01 2015 From: phil at dovecot.fi (Phil Carmody) Date: Thu, 15 Oct 2015 14:41:01 +0300 Subject: Strange indexing behavior on HTML emails .. In-Reply-To: <895954ff5f750afd71fe25b88fbb41c4@mailjol.in> References: <7fde63232b2bf2cb24f3b45f0b93b55d@mailjol.in> <895954ff5f750afd71fe25b88fbb41c4@mailjol.in> Message-ID: <20151015114101.GF790@phil.dovecot.net> On Wed, Oct 14, 2015 at 08:33:56PM +0530, Akash wrote: > Tried latest source from HG and with solr also apart from lucene which I > tested previously. The problem with single quotes in HTML is still there. > > The revision: > > http://hg.dovecot.org/dovecot-2.2/rev/ad028a950248 > > should have solved it but the relevant code no longer exists in > src/plugins/fts/fts-parser-html.c. Seems like it has been moved into > lib-mail. The file src/lib-mail/mail-html2text.c does contain something > about single quotes but to no avail. Can someone at-least confirm existence > of this issue? Thanks for the report. Bug found. My bad. A patch is working its way through the internal process, and will be in the public tree soon. Cheers, Phil From akbwiz+dovecot at gmail.com Thu Oct 15 12:09:47 2015 From: akbwiz+dovecot at gmail.com (Akash) Date: Thu, 15 Oct 2015 17:39:47 +0530 Subject: Strange indexing behavior on HTML emails .. In-Reply-To: <20151015114101.GF790@phil.dovecot.net> References: <7fde63232b2bf2cb24f3b45f0b93b55d@mailjol.in> <895954ff5f750afd71fe25b88fbb41c4@mailjol.in> <20151015114101.GF790@phil.dovecot.net> Message-ID: <813ebcc86778a3ee1255d8231d6d05fa@mailjol.in> > > Thanks for the report. Bug found. My bad. A patch is working its way > through the internal process, and will be in the public tree soon. > > Cheers, > Phil That was fast :-) Thanks. Will wait for the patch. From flint42 at gmail.com Thu Oct 15 12:19:22 2015 From: flint42 at gmail.com (Jean-Baptiste Vignaud) Date: Thu, 15 Oct 2015 14:19:22 +0200 Subject: Strange indexing behavior on HTML emails .. In-Reply-To: <20151015114101.GF790@phil.dovecot.net> References: <7fde63232b2bf2cb24f3b45f0b93b55d@mailjol.in> <895954ff5f750afd71fe25b88fbb41c4@mailjol.in> <20151015114101.GF790@phil.dovecot.net> Message-ID: > > > > Thanks for the report. Bug found. My bad. A patch is working its way > through the internal process, and will be in the public tree soon. > > Cheers, > Phil > Hello; Does this patch will need to reindex lucene ? Thanks From kilburna at gmail.com Thu Oct 15 15:09:27 2015 From: kilburna at gmail.com (Kilburn Abrahams) Date: Fri, 16 Oct 2015 02:09:27 +1100 Subject: Dovecot top stats In-Reply-To: References: Message-ID: <561FC1A7.3020009@gmail.com> On 10/15/2015 06:17 PM, Steffen Kaiser wrote: > On Thu, 15 Oct 2015, Kilburn Abrahams wrote: > > > I have enabled doevcot stats and I can do a doveadm stats dump user > but I > > get a 0 for disk_input, disk_output, read_bytes and write_bytes. I > do get > > mail_read_bytes numbers but I am not sure these numbers are correct. > > my stats contain numbers greater than zero, except for very few users. > > Did you tested it with some account and waited a bit, incl. > logins/logouts? > > -- Steffen Kaiser Yes I waited more than 24 hrs but I would assume it would a few minutes would do it as well. I am using gentoo and dovecot 2.2.19. I get numbers for all columns except disk_input, disk_output, read_count, read_bytes, write_count and write_bytes which is 0. cat /proc/self/io rchar: 2068 wchar: 0 syscr: 6 syscw: 0 read_bytes: 0 write_bytes: 0 cancelled_write_bytes: 0 Is this a kernel setting that causes this read_bytes and write_bytes to be 0 as it happens on a test server as well. /Kilburn From rs at sys4.de Thu Oct 15 15:31:29 2015 From: rs at sys4.de (Robert Schetterer) Date: Thu, 15 Oct 2015 17:31:29 +0200 Subject: master user in dovecot 1.2 In-Reply-To: <172EDF1A-8BE5-4570-9540-63A36BAD8676@somi.sk> References: <172EDF1A-8BE5-4570-9540-63A36BAD8676@somi.sk> Message-ID: <561FC6D1.7050904@sys4.de> 1.2.9 is very outdated, install a recent version 2009-12-16 1.2.9 changeset | changelog | files vs 12 days ago Timo Sirainen Released v2.2.19. 2.2.19 Am 15.10.2015 um 13:12 schrieb Michal ?ila: > Hello list, > > i would like to use master user functionality in dovecot v1.2, but i just cant get it to work: > > Here?s my configuration: > > # dovecot -n > # 1.2.9: /etc/dovecot.conf > # OS: Linux 2.6.18-164.10.1.el5PAE i686 Red Hat Enterprise Linux Server release 5.4 (Tikanga) > login_dir: /var/run/dovecot/login > login_executable(default): /usr/libexec/dovecot/imap-login > login_executable(imap): /usr/libexec/dovecot/imap-login > login_executable(pop3): /usr/libexec/dovecot/pop3-login > first_valid_uid: 200 > mail_privileged_group: mail > mail_location: mbox:~/mail:INBOX=/var/mail/%u > mail_executable(default): /root/dovecot-master-postlogin > mail_executable(imap): /root/dovecot-master-postlogin > mail_executable(pop3): /usr/libexec/dovecot/pop3 > mail_plugin_dir(default): /usr/lib/dovecot/imap > mail_plugin_dir(imap): /usr/lib/dovecot/imap > mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 > lda: > postmaster_address: postmaster at example.com > auth default: > master_user_separator: * > debug: yes > passdb: > driver: passwd-file > args: /etc/dovecot.passwd.masterusers > pass: yes > master: yes > passdb: > driver: pam > userdb: > driver: passwd > > > # cat /root/dovecot-master-postlogin > #!/bin/bash > MASTER_USER=$USER > export MASTER_USER > exec /usr/libexec/dovecot/imap "$@? > > > # cat /etc/dovecot.passwd.masterusers > masteruser:{PLAIN}Heslo42 > > > Trying login as normal user: > > # telnet localhost imap > Trying 127.0.0.1... > Connected to localhost.localdomain (127.0.0.1). > Escape character is '^]'. > * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE STARTTLS AUTH=PLAIN] Dovecot ready. > a1 login jozefkova jozefkova026 > a1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH] Logged in > > > Trying login as master user: > > # telnet localhost imap > Trying 127.0.0.1... > Connected to localhost.localdomain (127.0.0.1). > Escape character is '^]'. > * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE STARTTLS AUTH=PLAIN] Dovecot ready. > a1 login jozefkova*masteruser Heslo42 > a1 NO [AUTHORIZATIONFAILED] Authorization failed > > > log sequence of failed auth: > > Oct 15 13:04:47 linux dovecot: auth(default): client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=34618 resp= > Oct 15 13:04:47 linux dovecot: auth(default): passwd-file(masteruser,127.0.0.1,master): lookup: user=masteruser file=/etc/dovecot.passwd.masterusers > Oct 15 13:04:47 linux dovecot: auth(default): passdb(masteruser,127.0.0.1,master): Master user logging in as jozefkova > Oct 15 13:04:47 linux dovecot: auth-worker(default): pam(jozefkova,127.0.0.1): lookup service=dovecot > Oct 15 13:04:47 linux dovecot: auth-worker(default): pam(jozefkova,127.0.0.1): #1/1 style=1 msg=Password: > Oct 15 13:04:49 linux dovecot: auth-worker(default): pam(jozefkova,127.0.0.1): pam_authenticate() failed: Authentication failure (password mismatch?) > Oct 15 13:04:51 linux dovecot: auth(default): client out: FAIL 1 user=jozefkova authz > > > > So i can see it?s trying to loggin in using pam mech. Is there something i can configure in dovecot userdb or is it something else? I use pam to authenticate my real users? > > > Thank you in advance? > > > > ? > Michal Zila > Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From me at junc.eu Thu Oct 15 15:40:17 2015 From: me at junc.eu (Benny Pedersen) Date: Thu, 15 Oct 2015 17:40:17 +0200 Subject: master user in dovecot 1.2 In-Reply-To: <561FC6D1.7050904@sys4.de> References: <172EDF1A-8BE5-4570-9540-63A36BAD8676@somi.sk> <561FC6D1.7050904@sys4.de> Message-ID: <6b4cad5ca138c57863f524966e6a066f@junc.eu> Robert Schetterer skrev den 2015-10-15 17:31: > 1.2.9 is very outdated, install a recent version > > 2009-12-16 1.2.9 changeset | changelog | files 1.2.17 is latest, if its not any problem with that version i just consider its stable but fair its outdated, but not unstable From rs at sys4.de Thu Oct 15 15:48:49 2015 From: rs at sys4.de (Robert Schetterer) Date: Thu, 15 Oct 2015 17:48:49 +0200 Subject: master user in dovecot 1.2 In-Reply-To: <6b4cad5ca138c57863f524966e6a066f@junc.eu> References: <172EDF1A-8BE5-4570-9540-63A36BAD8676@somi.sk> <561FC6D1.7050904@sys4.de> <6b4cad5ca138c57863f524966e6a066f@junc.eu> Message-ID: <561FCAE1.5080205@sys4.de> Am 15.10.2015 um 17:40 schrieb Benny Pedersen: > Robert Schetterer skrev den 2015-10-15 17:31: >> 1.2.9 is very outdated, install a recent version >> >> 2009-12-16 1.2.9 changeset | changelog | files > > 1.2.17 is latest, if its not any problem with that version i just > consider its stable > > but fair its outdated, but not unstable but dont expect wide list support and recent bugfixes included Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From me at junc.eu Thu Oct 15 15:58:54 2015 From: me at junc.eu (Benny Pedersen) Date: Thu, 15 Oct 2015 17:58:54 +0200 Subject: master user in dovecot 1.2 In-Reply-To: <561FCAE1.5080205@sys4.de> References: <172EDF1A-8BE5-4570-9540-63A36BAD8676@somi.sk> <561FC6D1.7050904@sys4.de> <6b4cad5ca138c57863f524966e6a066f@junc.eu> <561FCAE1.5080205@sys4.de> Message-ID: <4320ddea59d7e4010d106678b96136fe@junc.eu> Robert Schetterer skrev den 2015-10-15 17:48: > but dont expect wide list support and recent bugfixes included i have no problem with stable software even if its outdated, for me it changes if its unstable From rs at sys4.de Thu Oct 15 16:58:38 2015 From: rs at sys4.de (Robert Schetterer) Date: Thu, 15 Oct 2015 18:58:38 +0200 Subject: master user in dovecot 1.2 In-Reply-To: <4320ddea59d7e4010d106678b96136fe@junc.eu> References: <172EDF1A-8BE5-4570-9540-63A36BAD8676@somi.sk> <561FC6D1.7050904@sys4.de> <6b4cad5ca138c57863f524966e6a066f@junc.eu> <561FCAE1.5080205@sys4.de> <4320ddea59d7e4010d106678b96136fe@junc.eu> Message-ID: <561FDB3E.8020903@sys4.de> Am 15.10.2015 um 17:58 schrieb Benny Pedersen: > Robert Schetterer skrev den 2015-10-15 17:48: > >> but dont expect wide list support and recent bugfixes included > > i have no problem with stable software even if its outdated, for me it > changes if its unstable stable vs unstable is a very personal idea from software over the time related to dovecot there may existed versions called stable but there was never a bugfree one, depend to all features included, so its more like a rolling release in general. So using latest is always the best option Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From me at junc.eu Thu Oct 15 17:27:51 2015 From: me at junc.eu (Benny Pedersen) Date: Thu, 15 Oct 2015 19:27:51 +0200 Subject: master user in dovecot 1.2 In-Reply-To: <561FDB3E.8020903@sys4.de> References: <172EDF1A-8BE5-4570-9540-63A36BAD8676@somi.sk> <561FC6D1.7050904@sys4.de> <6b4cad5ca138c57863f524966e6a066f@junc.eu> <561FCAE1.5080205@sys4.de> <4320ddea59d7e4010d106678b96136fe@junc.eu> <561FDB3E.8020903@sys4.de> Message-ID: <64ac9f0c659791cf23c66932233bb215@junc.eu> Robert Schetterer skrev den 2015-10-15 18:58: > stable vs unstable is a very personal idea from software i know my gentoo overlay is unstalbe, but it still works for the ebuilds i still use from it, oh no :=) > over the time related to dovecot there may existed versions called > stable > but there was never a bugfree one, bugs can be resolved with patchs > depend to all features included, so > its more like a rolling release in general. yep we will see dovecot version 3.x some time in future, this will indeed make dovecot v1 and v2 depricated :=) > So using latest is always > the best option if needed yes, i just keept v1 long enough to make v2 setup stable to be used as replacement hosting for my emails, used imapsync to move mails, and it worked like a charm for me From rs at sys4.de Thu Oct 15 17:44:37 2015 From: rs at sys4.de (Robert Schetterer) Date: Thu, 15 Oct 2015 19:44:37 +0200 Subject: master user in dovecot 1.2 In-Reply-To: <64ac9f0c659791cf23c66932233bb215@junc.eu> References: <172EDF1A-8BE5-4570-9540-63A36BAD8676@somi.sk> <561FC6D1.7050904@sys4.de> <6b4cad5ca138c57863f524966e6a066f@junc.eu> <561FCAE1.5080205@sys4.de> <4320ddea59d7e4010d106678b96136fe@junc.eu> <561FDB3E.8020903@sys4.de> <64ac9f0c659791cf23c66932233bb215@junc.eu> Message-ID: <561FE605.3060600@sys4.de> Am 15.10.2015 um 19:27 schrieb Benny Pedersen: > bugs can be resolved with patchs i cant speak for Timo but 1.2.x latest patch was 17 months ago perhaps there will be some security patches, if needed, but i guess no one will fix feature bugs, so it makes no sense to answer list support questions ,cause they maybe results of unfixed bugs , allready fixed in latest Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From gcivitella at entermail.it Thu Oct 15 18:26:01 2015 From: gcivitella at entermail.it (Giuseppe Civitella) Date: Thu, 15 Oct 2015 20:26:01 +0200 Subject: Haproxy and Dovecot auth Message-ID: <561FEFB9.6020807@entermail.it> Hi all, I'd like to use Haproxy to balance an auth service on a couple of Dovecot directors to have a redundant sasl service for my Postfix instances. While I configure the Postfixes to use, as smtpd_sasl_path, a direct connection to one of the directors I notice no errors. When I redirect the smtpd_sasl_path through an Haproxy balanced connection, I receive an error the first time I try to sen an email. The error is: SASL PLAIN authentication failed: Connection lost to authentication server If then I send more emails I receive no errors. If I do not send emails for a few minutes, the error appears again. The relevant Haproxy configuration is the following: listen auth *:12345 mode tcp balance source log global option tcplog option log-health-checks stick-table type ip size 200k expire 30m stick on src default-server inter 1000 fall 3 rise 1 server dovecot-director01 dovecot-director01:12345 check Do anyone have an idea about what it's missing? Thanks a lot Giuseppe -- Giuseppe Civitella gcivitella at entermail.it From ad+lists at uni-x.org Thu Oct 15 19:16:10 2015 From: ad+lists at uni-x.org (Alexander Dalloz) Date: Thu, 15 Oct 2015 21:16:10 +0200 Subject: master user in dovecot 1.2 In-Reply-To: <172EDF1A-8BE5-4570-9540-63A36BAD8676@somi.sk> References: <172EDF1A-8BE5-4570-9540-63A36BAD8676@somi.sk> Message-ID: <561FFB7A.6030908@uni-x.org> Am 15.10.2015 um 13:12 schrieb Michal ?ila: > # OS: Linux 2.6.18-164.10.1.el5PAE i686 Red Hat Enterprise Linux Server release 5.4 (Tikanga) 3 years no updates, that's irresponsible. Your system has big security holes. Current RHEL is 5.11. "yum update" and reboot ASAP! Alexander From michal.zila at somi.sk Fri Oct 16 06:54:33 2015 From: michal.zila at somi.sk (=?utf-8?Q?Michal_=C5=BDila?=) Date: Fri, 16 Oct 2015 08:54:33 +0200 Subject: master user in dovecot 1.2 In-Reply-To: <561FFB7A.6030908@uni-x.org> References: <172EDF1A-8BE5-4570-9540-63A36BAD8676@somi.sk> <561FFB7A.6030908@uni-x.org> Message-ID: <52986E26-4D86-40C9-B641-FDAF826EAC24@somi.sk> Thank you for you answer. I got your point, anyways, this is not my system and i just want to migrate to RHEL7 ASAP and this is why i need master user to work (i want to use imap sync), because sysadmin doesnt have user passwords. So? is there any possibility to have working master user or not? Thanks. D?a 15.10.2015 o 21:16, Alexander Dalloz nap?sal: > Am 15.10.2015 um 13:12 schrieb Michal ?ila: >> # OS: Linux 2.6.18-164.10.1.el5PAE i686 Red Hat Enterprise Linux Server release 5.4 (Tikanga) > > 3 years no updates, that's irresponsible. Your system has big security holes. Current RHEL is 5.11. "yum update" and reboot ASAP! > > Alexander -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4021 bytes Desc: not available URL: From zava.zava at libero.it Fri Oct 16 08:10:26 2015 From: zava.zava at libero.it (zava.zava at libero.it) Date: Fri, 16 Oct 2015 10:10:26 +0200 (CEST) Subject: Problem with ldap - master user - sogo vacation Message-ID: <1060715825.1075941444983026424.JavaMail.defaultUser@defaultHost> Hi,I have a problem, i can't get the authentication for the master user using dovecot - ldap authentication, in combination with sogo.There is a script that is used by Sogo to check and disable the vacation messagge: "sogo /usr/sbin/sogo-tool expire-autoreply -p /etc/dovecot/master-users" Insidie this file there is: sogo:{SHA}gkodfgdk6yujkgdklkkdfkgjrg9ugr9p= if I run the script I get: Oct 16 09:59:12 auth: Debug: auth client connected (pid=6528)Oct 16 09:59:12 auth: Debug: client in: AUTH 1 PLAIN service=sieve secured session=aQDV64h65pAB/AAAB lip=127.0.0.1 rip=127.0.0.1 lport=4190 rport=36260 resp=Y2VkQGdyddgPQ== (previous base64 data may contain sensitive data)Oct 16 09:59:12 auth: Debug: passwd-file(sogo,127.0.0.1,master,): Master user lookup for login: user at domain.comOct 16 09:59:12 auth: Debug: passwd-file(sogo,127.0.0.1,master,): lookup: user=sogo file=/etc/dovecot/master-usersOct 16 09:59:12 auth: Info: passwd-file(sogo,127.0.0.1,master,): Password mismatch (given password: {SHA}4sdgkfgwe22=)Oct 16 09:59:12 auth: Debug: passwd-file(sogo,127.0.0.1,master,): SHA({SHA}kerwkekwrejkewerjrw=) != 'erwrewwweewrerewrwe='Oct 16 09:59:14 auth: Debug: client passdb out: FAIL 1 user=sogoOct 16 09:59:14 managesieve-login: Info: Disconnected (auth failed, 1 attempts in 2 secs): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, TLS: Disconnected, session= In the auth-master.conf.ext file I have: passdb { driver = passwd-file master = yes args = /etc/dovecot/master-users} Thanks in advance for help. # 2.2.13: /etc/dovecot/dovecot.conf# OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.2 ext4auth_debug = yesauth_debug_passwords = yesauth_master_user_separator = *auth_mechanisms = plain auth_verbose = yesauth_verbose_passwords = yesdefault_client_limit = 22000default_process_limit = 5000default_vsz_limit = 1 Gdisable_plaintext_auth = nofirst_valid_gid = 5000first_valid_uid = 5000last_valid_gid = 5000last_valid_uid = 5000log_path = /var/log/dovecot.logmail_debug = yesmail_location = maildir:/var/vmail/xxx/%Ln/Maildirmanagesieve_notify_capability = mailtomanagesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags notifynamespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = /}passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes}passdb { driver = pam}passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap}passdb { args = /etc/dovecot/dovecot-ldap.conf1.ext driver = ldap}passdb { args = /etc/dovecot/dovecot-ldap.conf2.ext driver = ldap}passdb { args = /etc/dovecot/dovecot-ldap.conf3.ext driver = ldap}passdb { args = /etc/dovecot/dovecot-ldap.conf4.ext driver = ldap}passdb { args = /etc/dovecot/dovecot-ldap.conf5.ext driver = ldap}passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap}plugin { sieve = /var/vmail/xxx/%n/.dovecot.sieve sieve_dir = /var/vmail/xxx/%n/ sieve_extensions = +enotify +vacation +copy +imapflags +notify}protocols = imap sieveservice auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0666 }}service imap { process_limit = 5000}service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 0 vsz_limit = 64 M}service managesieve { process_limit = 1024}ssl_cert = > Message: 6 > Date: Wed, 14 Oct 2015 11:43:22 +0200 > From: Christian Kivalo > To: Dovecot Mailing List > Subject: Re: Problems sync with mail client MAC and > dovecot-2.2.10-4.el7_0.1.el7_0.1.x86_64 > Message-ID: <33ba2eacef5dcfcb4c703615e559b094 at valo.at> > Content-Type: text/plain; charset=US-ASCII; format=flowed > > > > Attached the doveconf -n > > Yes i have mail_debug = yes into file conf.d/10-logging.conf > > I paste the rest of log: > > Oct 13 21:27:17 localhost dovecot: imap: Debug: Loading modules from > > directory: /usr/lib64/dovecot > > Oct 13 21:27:17 localhost dovecot: imap: Debug: Module loaded: > > /usr/lib64/dovecot/lib01_acl_plugin.so > > Oct 13 21:27:17 localhost dovecot: imap: Debug: Module loaded: > > /usr/lib64/dovecot/lib02_imap_acl_plugin.so > > Oct 13 21:27:17 localhost dovecot: imap: Debug: Module loaded: > > /usr/lib64/dovecot/lib10_quota_plugin.so > > Oct 13 21:27:17 localhost dovecot: imap: Debug: Module loaded: > > /usr/lib64/dovecot/lib11_imap_quota_plugin.so > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: > > Effective > > uid=56396, gid=55000, home=/VMAIL/xxxxxxxxxxxxxx > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: Quota > > root: > > name=User quota backend=maildir args= > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: Quota > > grace: root=User quota bytes=0 (10%) > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: > > Namespace > > inbox: type=private, prefix=INBOX., sep=., inbox=yes, hidden=no, > > list=children, subscriptions=yes location=maildir:~/Maildir > > Your inbox namespace seems to be ok and is also defined in your doveconf > -n output. > > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: > > maildir++: > > root=/VMAIL/xxxxxxxxxxxxxx/Maildir, index=, indexpvt=, control=, > > inbox=/VMAIL/xxxxxxxxxxxxxx/Maildir, alt= > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: acl: > > initializing backend with data: vfile > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: acl: > > acl > > username = xxxxxxxxxxxxxx > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: acl: > > owner > > = 1 > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: acl > > vfile: > > Global ACL directory: (none) > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: > > Namespace : > > type=private, prefix=, sep=, inbox=no, hidden=yes, list=no, > > subscriptions=no location=fail::LAYOUT=none > > Somehow there seems to be an empty namespace in addition to your inbox > namespace. > > Is there a second namespace in your config somewhere? > > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: none: > > root=, index=, indexpvt=, control=, inbox=, alt= > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Fatal: master: > > service(imap): child 2464 killed with signal 11 (core dumps disabled) > > Just guessing: > Could it be with Thunderbird that the second namespace is ignored when > there is no subscription to it and the Apple mailprogram behaves > differently? > > regards > christian > > > > Hi, i can reproduce the error, conencting opnessl to port 993: From phil at dovecot.fi Fri Oct 16 11:07:26 2015 From: phil at dovecot.fi (Phil Carmody) Date: Fri, 16 Oct 2015 14:07:26 +0300 Subject: Strange indexing behavior on HTML emails .. In-Reply-To: References: <7fde63232b2bf2cb24f3b45f0b93b55d@mailjol.in> <895954ff5f750afd71fe25b88fbb41c4@mailjol.in> <20151015114101.GF790@phil.dovecot.net> Message-ID: <20151016110726.GG790@phil.dovecot.net> On Thu, Oct 15, 2015 at 02:19:22PM +0200, Jean-Baptiste Vignaud wrote: > > Thanks for the report. Bug found. My bad. A patch is working its way > > through the internal process, and will be in the public tree soon. > > Hello; > Does this patch will need to reindex lucene ? Yes, unfortunately, it does. Phil From akbwiz+dovecot at gmail.com Fri Oct 16 11:10:32 2015 From: akbwiz+dovecot at gmail.com (Akash) Date: Fri, 16 Oct 2015 16:40:32 +0530 Subject: Strange indexing behavior on HTML emails .. In-Reply-To: <20151016110726.GG790@phil.dovecot.net> References: <7fde63232b2bf2cb24f3b45f0b93b55d@mailjol.in> <895954ff5f750afd71fe25b88fbb41c4@mailjol.in> <20151015114101.GF790@phil.dovecot.net> <20151016110726.GG790@phil.dovecot.net> Message-ID: <2c2aad55795f02dae2996bdc994a0abb@mailjol.in> >> Does this patch will need to reindex lucene ? > > Yes, unfortunately, it does. > > Phil Just confirming that the patch has solved it. Thanks. Though it was just a tiny glitch, it prevented so many mails from showing in the search results. And I feel even Solr will require re-indexing. From niyalevi at gmail.com Fri Oct 16 11:25:02 2015 From: niyalevi at gmail.com (niya levi) Date: Fri, 16 Oct 2015 12:25:02 +0100 Subject: quotas configuration a few questions Message-ID: <5620DE8E.9080000@gmail.com> hi everyone need a few questions answering if possible before i can complete some modifications on my mailserver 1 how do i modify my user_query line in dovecot-sql.conf for quotas 2 is my iterate_query line ok 3 should i use lmtp or lda to deliver the quota warnings in quota-warning.sh my server is using lmtp but i've only seen examples using lda 4 how do i break long lines in dovecot.conf, is it the same as postgresql where you leave a space on the next line before continuing the txt. i would like to break the cipher line in dovecot.conf over multiple lines 5 have i configured my namespace section correctly so that i can substitute for the autocreate plugin also could you give the files a quick look over to make sure there isn't any unnecessary additions that shouldn't be there or omissions that should be there thanks shadrock ============ /etc/dovecot/dovecot.conf disable_plaintext_auth = no mail_privileged_group = vmail log_timestamp = "%Y-%m-%d %H:%M:%S " log_path = /var/log/dovecot.log auth_debug_passwords=yes mail_debug=yes verbose_ssl=yes ssl_prefer_server_ciphers = yes ssl_dh_parameters_length = 2048 ssl_cert = References: <7cdb984d3bb0d4ede95a16a2b2fb8d9a@bunbun.be> <56210E1B.1090408@bunbun.be> Message-ID: <5621619A.5040501@bunbun.be> Timo Sirainen wrote: > On 05 Oct 2015, at 22:05, Nick Rosier wrote: >> Hi, >> >> one of my mailboxes returns following error when I run doveadm index on it: >> >> Panic: file charset-iconv.c: line 85 (charset_to_utf8_try): assertion failed: (srcleft<= CHARSET_MAX_PENDING_BUF_SIZE) >> >> OS: FreeBSD 10.2 >> Dovecot: 2.1.19 >> Tika: 1.10 >> SOLR: 5.3.1 > > I thought this would have definitely been fixed by now.. Can you find some specific email which is triggering this? It's either something strange that I didn't expect, or it's a bug in FreeBSD's iconv(). > > Also can you debug this like: > > gdb --args doveadm index -u user at domain INBOX > run > > f 5 > p src > p ic_srcbuf > p *src_size > p srcleft I recompiled Dovecot with Debug but I suspect I will have to do it for all the required libraries as well; I did manage to find the offending message is in Trash but have not been able to pin-point which message it is. Just in case, here is the debugging info. I will see if I can compile all dependencies in debug mode as well. GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... (gdb) run Starting program: /usr/local/bin/doveadm index -u xxxx at xxxx Trash doveadm(xxxx at xxxx): Panic: file charset-iconv.c: line 85 (charset_to_utf8_try): assertion failed: (srcleft <= CHARSET_MAX_PENDING_BUF_SIZE) Program received signal SIGABRT, Aborted. 0x000000080149f64a in thr_kill () from /lib/libc.so.7 (gdb) f 5 #5 0x0000000801143cd3 in i_panic (format=0x801191c30 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:275 275 failures.c: No such file or directory. in failures.c Current language: auto; currently minimal (gdb) p src No symbol "src" in current context. (gdb) p ic_srcbuf No symbol "ic_srcbuf" in current context. (gdb) p *src_size No symbol "src_size" in current context. (gdb) p srcleft No symbol "srcleft" in current context. From nick+dovecot at bunbun.be Fri Oct 16 20:44:23 2015 From: nick+dovecot at bunbun.be (Nick Rosier) Date: Fri, 16 Oct 2015 22:44:23 +0200 Subject: doveadm index assertion failed In-Reply-To: <56216136.2030506@bunbun.be> References: <7cdb984d3bb0d4ede95a16a2b2fb8d9a@bunbun.be> <56210E1B.1090408@bunbun.be> <56216136.2030506@bunbun.be> Message-ID: <562161A7.9080204@bunbun.be> Nick Rosier schreef: > > Timo Sirainen wrote: >> On 05 Oct 2015, at 22:05, Nick Rosier wrote: >>> Hi, >>> >>> one of my mailboxes returns following error when I run doveadm index on it: >>> >>> Panic: file charset-iconv.c: line 85 (charset_to_utf8_try): assertion failed: (srcleft<= CHARSET_MAX_PENDING_BUF_SIZE) >>> >>> OS: FreeBSD 10.2 >>> Dovecot: 2.1.19 >>> Tika: 1.10 >>> SOLR: 5.3.1 >> >> I thought this would have definitely been fixed by now.. Can you find some specific email which is triggering this? It's either something strange that I didn't expect, or it's a bug in FreeBSD's iconv(). >> >> Also can you debug this like: >> >> gdb --args doveadm index -u user at domain INBOX >> run >> >> f 5 >> p src >> p ic_srcbuf >> p *src_size >> p srcleft > I recompiled Dovecot with Debug but I suspect I will have to do it for > all the required libraries as well; I did manage to find the offending > message is in Trash but have not been able to pin-point which message > it is. Just in case, here is the debugging info. I will see if I can > compile all dependencies in debug mode as well. > > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and > you are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for > details. > This GDB was configured as "amd64-marcel-freebsd"... > (gdb) run > Starting program: /usr/local/bin/doveadm index -u xxxx at xxxx Trash > doveadm(xxxx at xxxx): Panic: file charset-iconv.c: line 85 > (charset_to_utf8_try): assertion failed: (srcleft <= > CHARSET_MAX_PENDING_BUF_SIZE) > > Program received signal SIGABRT, Aborted. > 0x000000080149f64a in thr_kill () from /lib/libc.so.7 > (gdb) f 5 > #5 0x0000000801143cd3 in i_panic (format=0x801191c30 "file %s: line > %d (%s): assertion failed: (%s)") at failures.c:275 > 275 failures.c: No such file or directory. > in failures.c > Current language: auto; currently minimal > (gdb) p src > No symbol "src" in current context. > (gdb) p ic_srcbuf > No symbol "ic_srcbuf" in current context. > (gdb) p *src_size > No symbol "src_size" in current context. > (gdb) p srcleft > No symbol "srcleft" in current context. > recompiled dovecot + all dependencies with debug but I get the same output. anything else I can try? From f-u.s at gmx.net Sat Oct 17 00:05:42 2015 From: f-u.s at gmx.net (Frank-Ulrich Sommer) Date: Sat, 17 Oct 2015 02:05:42 +0200 Subject: doveadm-search-query pattern format Message-ID: <562190D6.7080106@gmx.net> Hello, the doveadm-search-query documentation specifies a 'pattern' for many search keys. Unfortunately it does not mention the format specification of this pattern (special characters, wildcards, ...). I'm using dovecot 2.2.9 but also searched the current documention on dovecot.org. Where can I find further documentation concerning the exact specification of the pattern format? From akbwiz+dovecot at gmail.com Sat Oct 17 07:24:02 2015 From: akbwiz+dovecot at gmail.com (Akash) Date: Sat, 17 Oct 2015 12:54:02 +0530 Subject: Order in which UIDs are assigned.. Message-ID: <11462483ac51840662c0daaad4ab0640@mailjol.in> Hi, I just want some clarification on how Dovecot's IMAP assigns UIDs when it picks files from the "new" directory of a Maildir. What I am observing is that only ctime has a role to play in it. For example if there are two files in "new", a.msg & z.msg. Even when a.msg has lower mtime than z.msg and "a" comes before "z" alphabetically, dovecot will assign z.msg a lower UID if its ctime is lower than a.msg. Is this correct? Secondly, when mails are placed into Maildir through a external mechanism by-passing LDA, by any means is it possible for IMAP to run seive on it when it migrates the files from new to cur and indexes them? I understand that this might not be a valid question as mail is delivered already so there is nothing to filter :-) -Akash From tss at iki.fi Sat Oct 17 08:12:41 2015 From: tss at iki.fi (Timo Sirainen) Date: Sat, 17 Oct 2015 11:12:41 +0300 Subject: doveadm index assertion failed In-Reply-To: <5621619A.5040501@bunbun.be> References: <7cdb984d3bb0d4ede95a16a2b2fb8d9a@bunbun.be> <56210E1B.1090408@bunbun.be> <5621619A.5040501@bunbun.be> Message-ID: On 16 Oct 2015, at 23:44, Nick Rosier wrote: > > >> gdb --args doveadm index -u user at domain INBOX >> run >> >> f 5 >> p src >> p ic_srcbuf >> p *src_size >> p srcleft > I recompiled Dovecot with Debug but I suspect I will have to do it for all the required libraries as well; I did manage to find the offending message is in Trash but have not been able to pin-point which message it is. Just in case, here is the debugging info. I will see if I can compile all dependencies in debug mode as well. > > Program received signal SIGABRT, Aborted. > 0x000000080149f64a in thr_kill () from /lib/libc.so.7 > (gdb) f 5 > #5 0x0000000801143cd3 in i_panic (format=0x801191c30 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:275 I guessed the frame wrong. It's usually 5, but I guess in FreeBSD it's 6 then. So try the same commands, but "f 6" first before "f 5" (and if that doesn't work either, try f 7). From tss at iki.fi Sat Oct 17 08:16:06 2015 From: tss at iki.fi (Timo Sirainen) Date: Sat, 17 Oct 2015 11:16:06 +0300 Subject: Order in which UIDs are assigned.. In-Reply-To: <11462483ac51840662c0daaad4ab0640@mailjol.in> References: <11462483ac51840662c0daaad4ab0640@mailjol.in> Message-ID: On 17 Oct 2015, at 10:24, Akash wrote: > > Hi, > > I just want some clarification on how Dovecot's IMAP assigns UIDs when it picks files from the "new" directory of a Maildir. What I am observing is that only ctime has a role to play in it. For example if there are two files in "new", a.msg & z.msg. Even when a.msg has lower mtime than z.msg and "a" comes before "z" alphabetically, dovecot will assign z.msg a lower UID if its ctime is lower than a.msg. Is this correct? Dovecot doesn't look at the new files' mtimes or ctimes, because that would require stat()ing the file, which slows it down. So it's using the timestamp in the Maildir filename itself. > Secondly, when mails are placed into Maildir through a external mechanism by-passing LDA, by any means is it possible for IMAP to run seive on it when it migrates the files from new to cur and indexes them? I understand that this might not be a valid question as mail is delivered already so there is nothing to filter :-) Not easily. There is http://wiki2.dovecot.org/HowTo/RefilterMail From tss at iki.fi Sat Oct 17 08:18:36 2015 From: tss at iki.fi (Timo Sirainen) Date: Sat, 17 Oct 2015 11:18:36 +0300 Subject: doveadm-search-query pattern format In-Reply-To: <562190D6.7080106@gmx.net> References: <562190D6.7080106@gmx.net> Message-ID: > On 17 Oct 2015, at 03:05, Frank-Ulrich Sommer wrote: > > Hello, > > the doveadm-search-query documentation specifies a 'pattern' for many search keys. Unfortunately it does not mention the format specification of this pattern (special characters, wildcards, ...). I'm using dovecot 2.2.9 but also searched the current documention on dovecot.org. > > Where can I find further documentation concerning the exact specification of the pattern format? Well, it's basically copying the IMAP standard here, which specifies that searching happens with substring matching. Although if you have full text search enabled, it may be matching only full words. In any case it's just a text string and there is no wildcard support, so maybe "pattern" is the wrong word.. From akbwiz+dovecot at gmail.com Sat Oct 17 08:44:44 2015 From: akbwiz+dovecot at gmail.com (Akash) Date: Sat, 17 Oct 2015 14:14:44 +0530 Subject: Order in which UIDs are assigned.. In-Reply-To: References: <11462483ac51840662c0daaad4ab0640@mailjol.in> Message-ID: <7aaccab458a555716a9ae1f8df05acb3@mailjol.in> > Dovecot doesn't look at the new files' mtimes or ctimes, because that > would require stat()ing the file, which slows it down. So it's using > the timestamp in the Maildir filename itself. Thanks Timo for the clarification. Whats Dovecot's strategy when the file-name does not contain a time-stamp and its just a sequential long number such as 1494829006737776656. My problem is that I am manually copying a lot of mail into the "new" directory of a Maildir. The file names range from: 1280369043768241070 - 1514890738113019895 but Dovecot is always assigning first UID to the mail in the file 1494829006737776656. And I am failing to understand why. I tried this 3-4 times using different ways. First by simply hard linking the files into "new" dir, second by copying them over, third by copying and then touching the file in a way that lower in sequence has a lower mtime, fourth by copying them into "new" in proper ascending sequence (according to file name) and then giving them appropriate mtime related to their sequence. But no matter what i do, Dovecot assigns first UID to file 1494829006737776656 only. -Akash From tss at iki.fi Sat Oct 17 09:02:01 2015 From: tss at iki.fi (Timo Sirainen) Date: Sat, 17 Oct 2015 12:02:01 +0300 Subject: Order in which UIDs are assigned.. In-Reply-To: <7aaccab458a555716a9ae1f8df05acb3@mailjol.in> References: <11462483ac51840662c0daaad4ab0640@mailjol.in> <7aaccab458a555716a9ae1f8df05acb3@mailjol.in> Message-ID: <0A33E3BE-2B40-4FA7-A23C-7A3E1530C02B@iki.fi> > On 17 Oct 2015, at 11:44, Akash wrote: > >> Dovecot doesn't look at the new files' mtimes or ctimes, because that >> would require stat()ing the file, which slows it down. So it's using >> the timestamp in the Maildir filename itself. > > Thanks Timo for the clarification. Whats Dovecot's strategy when the file-name does not contain a time-stamp and its just a sequential long number such as 1494829006737776656. If it's less than 64 bits number and Dovecot is compiled as 64bit binary, I guess it should work. Otherwise it gets more or less random: http://hg.dovecot.org/dovecot-2.2/file/17aac21b303d/src/lib-storage/index/maildir/maildir-filename.c#l117 > My problem is that I am manually copying a lot of mail into the "new" directory of a Maildir. The file names range from: 1280369043768241070 - 1514890738113019895 but Dovecot is always assigning first UID to the mail in the file 1494829006737776656. And I am failing to understand why. I tried this 3-4 times using different ways. First by simply hard linking the files into "new" dir, second by copying them over, third by copying and then touching the file in a way that lower in sequence has a lower mtime, fourth by copying them into "new" in proper ascending sequence (according to file name) and then giving them appropriate mtime related to their sequence. But no matter what i do, Dovecot assigns first UID to file 1494829006737776656 only. I guess you have 32bit Dovecot. From akbwiz+dovecot at gmail.com Sat Oct 17 09:19:19 2015 From: akbwiz+dovecot at gmail.com (Akash) Date: Sat, 17 Oct 2015 14:49:19 +0530 Subject: Order in which UIDs are assigned.. In-Reply-To: <0A33E3BE-2B40-4FA7-A23C-7A3E1530C02B@iki.fi> References: <11462483ac51840662c0daaad4ab0640@mailjol.in> <7aaccab458a555716a9ae1f8df05acb3@mailjol.in> <0A33E3BE-2B40-4FA7-A23C-7A3E1530C02B@iki.fi> Message-ID: <08f7b97a4b58d4efe579cfd64694cd7b@mailjol.in> > If it's less than 64 bits number and Dovecot is compiled as 64bit > binary, I guess it should work. Otherwise it gets more or less random: > > http://hg.dovecot.org/dovecot-2.2/file/17aac21b303d/src/lib-storage/index/maildir/maildir-filename.c#l117 That explains it. Thanks. From tss at iki.fi Sat Oct 17 09:41:16 2015 From: tss at iki.fi (Timo Sirainen) Date: Sat, 17 Oct 2015 12:41:16 +0300 Subject: More information about - Dovecot 2.2.x quota mysql and dict In-Reply-To: <1444170893263-54077.post@n4.nabble.com> References: <1444170893263-54077.post@n4.nabble.com> Message-ID: <1E35FEAE-2CE8-492E-8F26-AFA3061F222A@iki.fi> On 07 Oct 2015, at 01:34, Simon wrote: > > I some add information about bug... > > > Dovecot 2.2.19 version on server FreeBSD 10.2 Release. > Config quota (dict + mysql) > Database create Postfixadmin 2.3.7 > > Old version Dovecot 2.2.13 use method UPDATE in MySQL base + dict > example mysql.log > ----------------- > Connect postfix at localhost on postfix > 134 Query SELECT bytes FROM quota2 WHERE username = > 'user at mydomain.lan' > 135 Query SELECT bytes FROM quota2 WHERE username = > 'user at mydomain.lan' > 135 Query BEGIN > 135 Query UPDATE quota2 SET > bytes=bytes+2570,messages=messages+1 WHERE username = 'user at mydomain.lan' Here's it's updating quota with user at mydomain.lan > 140 Query SELECT messages FROM quota2 WHERE username = > 'mydomain.lan' > 140 Query BEGIN > 140 Query DELETE FROM domain WHERE domain = 'mydomain.lan' > 140 Query DELETE FROM quota2 WHERE username = 'mydomain.lan' > 140 Query INSERT INTO domain (quota,domain) VALUES > ('8581','mydomain.lan') ON DUPLICATE KEY UPDATE quota='8581' > 151004 15:18:45 140 Quit Here it's only mydomain.lan. So something's not right. > Now dic + mysql doesn?t work in Dovecot 2.2.19, because first DELETE > operator delete information from base about my domain mydomain.lan I can't reproduce this. What's your doveconf -n output? And what's in the dovecot-dict-sql.conf.ext? From semfly at gmail.com Sat Oct 17 10:29:28 2015 From: semfly at gmail.com (Simon) Date: Sat, 17 Oct 2015 03:29:28 -0700 (PDT) Subject: More information about - Dovecot 2.2.x quota mysql and dict In-Reply-To: <1E35FEAE-2CE8-492E-8F26-AFA3061F222A@iki.fi> References: <1444170893263-54077.post@n4.nabble.com> <1E35FEAE-2CE8-492E-8F26-AFA3061F222A@iki.fi> Message-ID: <1445077768513-54189.post@n4.nabble.com> Thanks for your answer, Timo! :) I send to you my configoration for reproducing. Please, follow the link, because configuration is a very big, there will be more easy to read and understand My configuration Dovecot 2.2.19 Below the same my configuration Dovecot 2.2.19 here. BEGIN of configuration --------------------- # doveconf -n # 2.2.19: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 10.2-RELEASE amd64 ufs auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login cram-md5 auth_verbose = yes base_dir = /var/run/dovecot/ first_valid_gid = 1983 first_valid_uid = 1983 listen = * log_path = /var/log/dovecot.log mail_debug = yes mail_gid = virtual mail_location = maildir:/var/spool/mail/%d/%u mail_uid = virtual passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } dict { sqluserquota = mysql:/usr/local/etc/dovecot/dovecot-dict-sql-user.conf sqldomainquota = mysql:/usr/local/etc/dovecot/dovecot-dict-sql-domain.conf } plugin { quota = dict:domain_quota:%d:proxy::sqldomainquota quota2 = dict:user_quota::proxy::sqluserquota quota2_rule2 = Trash:storage=+20%% quota2_rule3 = Junk:storage=+20%% quota2_grace = 10%% quota2_warning = storage=100%% quota-exceeded 100 %u quota2_warning2 = storage=95%% quota-warning 95 %u quota2_warning3 = storage=90%% quota-warning 90 %u quota2_warning4 = storage=75%% quota-warning 75 %u } service quota-warning { executable = script /usr/local/etc/dovecot/quota_warning.sh unix_listener quota-warning { mode = 0660 user = virtual } } protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = virtual mode = 0660 user = virtual } unix_listener auth-userdb { group = virtual mode = 0660 user = virtual } } service dict { unix_listener dict { mode = 0660 user = virtual } } service imap-login { client_limit = 256 inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 993 } process_limit = 64 process_min_avail = 8 service_count = 1 } service pop3-login { inet_listener pop3 { address = * port = 110 } inet_listener pop3s { address = * port = 995 } } ssl_cert = References: <1444170893263-54077.post@n4.nabble.com> <1E35FEAE-2CE8-492E-8F26-AFA3061F222A@iki.fi> <1445077768513-54189.post@n4.nabble.com> Message-ID: <7BA39810-5B32-4DDB-85F4-208ABC0FE837@iki.fi> On 17 Oct 2015, at 13:29, Simon wrote: > > Thanks for your answer, Timo! :) > > I send to you my configoration for reproducing. .. > If need anymore about configuration, please write me. > Today I can respond quickly. What does Dovecot log when you deliver a new mail (with auth_debug=yes and mail_debug=yes enabled)? From semfly at gmail.com Sat Oct 17 14:02:25 2015 From: semfly at gmail.com (Simon) Date: Sat, 17 Oct 2015 07:02:25 -0700 (PDT) Subject: More information about - Dovecot 2.2.x quota mysql and dict In-Reply-To: <7BA39810-5B32-4DDB-85F4-208ABC0FE837@iki.fi> References: <1444170893263-54077.post@n4.nabble.com> <1E35FEAE-2CE8-492E-8F26-AFA3061F222A@iki.fi> <1445077768513-54189.post@n4.nabble.com> <7BA39810-5B32-4DDB-85F4-208ABC0FE837@iki.fi> Message-ID: <1445090545342-54191.post@n4.nabble.com> /192.168.7.184/ -/Client - Mozilla Thunderbird (latest version)/ *192.168.7.185* - *Server Dovecot 2.2.19* # vim dovecot.log =============== Oct 04 15:17:45 imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges Oct 04 15:17:45 imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges Oct 04 15:17:45 auth: Debug: auth client connected (pid=14842) Oct 04 15:17:45 imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges Oct 04 15:17:45 imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges Oct 04 15:17:45 auth: Debug: auth client connected (pid=14843) Oct 04 15:17:45 imap-login: Debug: SSL: where=0x10, ret=1: before/accept initialization [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: before/accept initialization [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client hello A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write server hello A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write certificate A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write key exchange A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write server done A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 flush data [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client certificate A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client certificate A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client key exchange A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read certificate verify A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read finished A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write session ticket A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write change cipher spec A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write finished A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 flush data [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x20, ret=1: SSL negotiation finished successfully [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2002, ret=1: SSL negotiation finished successfully [192.168.7.184] Oct 04 15:17:45 auth: Debug: client in: AUTH 1 CRAM-MD5 service=imap secured session=Zt5gYkYhjQDAqAe4 lip=192.168.7.185 rip=192.168.7.184 lport=143 rport=59277 Oct 04 15:17:45 auth: Debug: client passdb out: CONT 1 PDU2NzIwMjE3OTU1MDYwMjguMTQ0Mzk2MTA2NUBzZXJ2Nz4= Oct 04 15:17:45 auth: Debug: client in: CONT 1 c2VtZmx5QGZseTc3Ny5uZXQgNmY2Y2FiNTc3YjQ4ZTE1OGE4NzU4NzkwZjdkNzhmN2U= (previous base64 data may contain sensitive data) Oct 04 15:17:45 auth-worker(14845): Debug: Loading modules from directory: /usr/local/lib/dovecot/auth Oct 04 15:17:45 auth-worker(14845): Debug: sql(simon at mydomain.lan,192.168.7.184): query: SELECT username as user, password FROM mailbox WHERE username = 'simon at mydomain.lan' and active='1' Oct 04 15:17:45 auth: Debug: sql(simon at mydomain.lan,192.168.7.184,): Credentials: 6df356c95df134db8c7ad557c31c752ed2ffcb7a9ff613748af3c43036a7370c Oct 04 15:17:45 auth: Debug: client passdb out: OK 1 user=simon at mydomain.lan Oct 04 15:17:45 auth: Debug: master in: REQUEST 3765567489 14836 1 982fc0b6f6ecf6098e257ccac0abbe3d session_pid=14846 request_auth_token Oct 04 15:17:45 auth-worker(14845): Debug: sql(simon at mydomain.lan,192.168.7.184): SELECT '/var/spool/mail/mydomain.lan/simon at mydomain.lan' AS home, 'maildir:/var/spool/mail/mydomain.lan/simon at mydomain.lan' AS mail, 1983 AS uid, 1983 AS gid, CONCAT('*:bytes=', domain.maxquota*1048576) AS quota_rule, CONCAT('*:bytes=', mailbox.quota) AS quota2_rule FROM mailbox, domain WHERE username = 'simon at mydomain.lan' AND mailbox.active = '1' AND domain.domain = 'mydomain.lan' AND domain.active = '1' Oct 04 15:17:45 auth: Debug: master userdb out: USER 3765567489 simon at mydomain.lan home=/var/spool/mail/mydomain.lan/simon at mydomain.lan mail=maildir:/var/spool/mail/mydomain.lan/simon at mydomain.lan uid=1983 gid=1983 quota_rule=*:bytes=7516192768 quota2_rule=*:bytes=7516192768 auth_token=71e913f2c5640c009de725f121c5bf5a5aa31198 Oct 04 15:17:45 imap-login: Info: Login: user=, method=CRAM-MD5, rip=192.168.7.184, lip=192.168.7.185, mpid=14846, TLS, session= Oct 04 15:17:45 imap-login: Debug: SSL: where=0x10, ret=1: before/accept initialization [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: before/accept initialization [192.168.7.184] Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Loading modules from directory: /usr/local/lib/dovecot Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Module loaded: /usr/local/lib/dovecot/lib10_quota_plugin.so Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Module loaded: /usr/local/lib/dovecot/lib11_imap_quota_plugin.so Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Added userdb setting: mail=maildir:/var/spool/mail/mydomain.lan/simon at mydomain.lan Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Added userdb setting: plugin/quota2_rule=*:bytes=7516192768 Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Added userdb setting: plugin/quota_rule=*:bytes=7516192768 Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Effective uid=1983, gid=1983, home=/var/spool/mail/mydomain.lan/simon at mydomain.lan Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Quota root: name=domain_quota backend=dict args=mydomain.lan:proxy::sqldomainquota Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Quota rule: root=domain_quota mailbox=* bytes=7516192768 messages=0 Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Quota grace: root=domain_quota bytes=751619276 (10%) Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Quota root: name=user_quota backend=dict args=:proxy::sqluserquota Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Quota rule: root=user_quota mailbox=* bytes=7516192768 messages=0 Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Quota rule: root=user_quota mailbox=Trash bytes=+1503238553 (20%) messages=0 Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Quota rule: root=user_quota mailbox=Junk bytes=+1503238553 (20%) messages=0 Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Quota warning: bytes=7516192768 (100%) messages=0 reverse=no command=quota-exceeded 100 simon at mydomain.lan Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Quota warning: bytes=7140383129 (95%) messages=0 reverse=no command=quota-warning 95 simon at mydomain.lan Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Quota warning: bytes=6764573491 (90%) messages=0 reverse=no command=quota-warning 90 simon at mydomain.lan Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Quota warning: bytes=5637144576 (75%) messages=0 reverse=no command=quota-warning 75 simon at mydomain.lan Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: Quota grace: root=user_quota bytes=751619276 (10%) Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: dict quota: user=mydomain.lan, uri=proxy::sqldomainquota, noenforcing=0 Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: dict quota: user=simon at mydomain.lan, uri=proxy::sqluserquota, noenforcing=0 Oct 04 15:17:45 imap(simon at mydomain.lan): Debug: maildir++: root=/var/spool/mail/mydomain.lan/simon at mydomain.lan, index=, indexpvt=, control=, inbox=/var/spool/mail/mydomain.lan/simon at mydomain.lan, alt= Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client hello A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write server hello A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write certificate A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write key exchange A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write server done A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 flush data [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client certificate A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client certificate A [192.168.7.184] Oct 04 15:17:45 dict: Error: sql dict: commit failed: Field 'description' doesn't have a default value Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client key exchange A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read certificate verify A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read finished A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write session ticket A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write change cipher spec A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write finished A [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 flush data [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x20, ret=1: SSL negotiation finished successfully [192.168.7.184] Oct 04 15:17:45 imap-login: Debug: SSL: where=0x2002, ret=1: SSL negotiation finished successfully [192.168.7.184] Oct 04 15:17:45 auth: Debug: client in: AUTH 1 CRAM-MD5 service=imap secured session=gPtjYkYhjgDAqAe4 lip=192.168.7.185 rip=192.168.7.184 lport=143 rport=59278 Oct 04 15:17:45 auth: Debug: client passdb out: CONT 1 PDYwNDQwODUxNzAwMzE2NDEuMTQ0Mzk2MTA2NUBzZXJ2Nz4= Oct 04 15:17:45 auth: Debug: client in: CONT 1 c2VtZmx5QGZseTc3Ny5uZXQgZTAzYzdjY2I3OGRmMDE1MmZmOTMzNTdmMzMyMmFmZmU= (previous base64 data may contain sensitive data) Oct 04 15:17:45 auth-worker(14845): Debug: sql(simon at mydomain.lan,192.168.7.184): query: SELECT username as user, password FROM mailbox WHERE username = 'simon at mydomain.lan' and active='1' Oct 04 15:17:45 auth: Debug: sql(simon at mydomain.lan,192.168.7.184,): Credentials: 6df356c95df134db8c7ad557c31c752ed2ffcb7a9ff613748af3c43036a7370c Oct 04 15:17:45 auth: Debug: client passdb out: OK 1 user=simon at mydomain.lan Oct 04 15:17:45 auth: Debug: master in: REQUEST 592969729 13563 1 eda35ea2307a82f2dec1b894530f1d09 session_pid=14849 request_auth_token Oct 04 15:17:45 auth-worker(14845): Debug: sql(simon at mydomain.lan,192.168.7.184): SELECT '/var/spool/mail/mydomain.lan/simon at mydomain.lan' AS home, 'maildir:/var/spool/mail/mydomain.lan/simon at mydomain.lan' AS mail, 1983 AS uid, 1983 AS gid, CONCAT('*:bytes=', domain.maxquota*1048576) AS quota_rule, CONCAT('*:bytes=', mailbox.quota) AS quota2_rule FROM mailbox, domain WHERE username = 'simon at mydomain.lan' AND mailbox.active = '1' AND domain.domain = 'mydomain.lan' AND domain.active = '1' Oct 04 15:17:45 auth-worker(14845): Info: sql(simon at mydomain.lan,192.168.7.184): unknown user Oct 04 15:17:45 auth: Error: sql(simon at mydomain.lan,192.168.7.184,): user not found from userdb Oct 04 15:17:45 auth: Debug: master userdb out: NOTFOUND 592969729 Oct 04 15:17:45 imap: Error: Authenticated user not found from userdb, auth lookup id=592969729 (client-pid=13563 client-id=1) Oct 04 15:17:45 imap-login: Info: Internal login failure (pid=13563 id=1) (internal failure, 1 successful auths): user=, method=CRAM-MD5, rip=192.168.7.184, lip=192.168.7.185, mpid=14849, TLS, session= Oct 04 15:17:45 imap-login: Debug: SSL alert: close notify [192.168.7.184] Oct 04 15:17:52 imap(simon at mydomain.lan): Info: Disconnected: Logged out in=184 out=1432 Oct 04 15:17:52 imap-login: Debug: SSL alert: close notify [192.168.7.184] -= END of dovecot.log =- -- View this message in context: http://dovecot.2317879.n4.nabble.com/More-information-about-Dovecot-2-2-x-quota-mysql-and-dict-tp54077p54191.html Sent from the Dovecot mailing list archive at Nabble.com. From jtam.home at gmail.com Sun Oct 18 02:20:59 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Sat, 17 Oct 2015 19:20:59 -0700 (PDT) Subject: Hibernation disabled when mailbox selected Message-ID: After setting imap_hibernate_timeout to 60s, I could not find any hibernated connections after a few hours. I tested hibernation and made an observation: IDLE'd imap sessions only hibernate if they don't have a mailbox SELECT'd, otherwise they never hibernate. Is this the way it's supposed to work? Joseph Tam From tss at iki.fi Sun Oct 18 17:25:49 2015 From: tss at iki.fi (Timo Sirainen) Date: Sun, 18 Oct 2015 20:25:49 +0300 Subject: Hibernation disabled when mailbox selected In-Reply-To: References: Message-ID: <4989A281-52A3-41E6-A2AF-A1C776468458@iki.fi> On 18 Oct 2015, at 05:20, Joseph Tam wrote: > > > After setting imap_hibernate_timeout to 60s, I could not find any > hibernated connections after a few hours. I tested hibernation and made > an observation: IDLE'd imap sessions only hibernate if they don't have > a mailbox SELECT'd, otherwise they never hibernate. > > Is this the way it's supposed to work? What client is that? IDLE doesn't really make much sense when a mailbox isn't SELECTed, so I expected that all the clients would have something SELECTed.. But yeah, the code could be changed to support that. From leon at dexterous.org Sun Oct 18 21:42:50 2015 From: leon at dexterous.org (Leon Kyneur) Date: Sun, 18 Oct 2015 21:42:50 +0000 Subject: Doveadm client/server possible? Message-ID: Hi As my mail infrastructure grows I'm now looking at the possibility of running all my doveadm commands from a central location. Wondering if there is a way to configure an inet_listener on doveadm and have a client elsewhere connect to it? Have seen the inet_listener for replication but not sure if it can also be used for general client server communication. Thanks Leon From jtam.home at gmail.com Mon Oct 19 04:02:39 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Sun, 18 Oct 2015 21:02:39 -0700 (PDT) Subject: Hibernation disabled when mailbox selected In-Reply-To: <4989A281-52A3-41E6-A2AF-A1C776468458@iki.fi> References: <4989A281-52A3-41E6-A2AF-A1C776468458@iki.fi> Message-ID: On Sun, 18 Oct 2015, Timo Sirainen wrote: >> After setting imap_hibernate_timeout to 60s, I could not find any >> hibernated connections after a few hours. I tested hibernation and made >> an observation: IDLE'd imap sessions only hibernate if they don't have >> a mailbox SELECT'd, otherwise they never hibernate. >> >> Is this the way it's supposed to work? > > What client is that? That would be "openssl s_client". I do not recommend this as your regular mail reader -- the user interface sucks. > IDLE doesn't really make much sense when a mailbox isn't SELECTed, so I > expected that all the clients would have something SELECTed.. But > yeah, the code could be changed to support that. If hibernate only works when a mailbox is not selected, and all IMAP client connections select a mailbox, then yeah, hibernate wouldn't be a very useful feature. I would guess that most of the imap processes running in most installation are in IDLE state with s SELECTed mailbox, so extending hibernate to work with these connections these would more useful. Couldn't a small pool of imap-hibernate processes hold on to all these hibernated connections for maximal memory conservation, or would that break the security model dovecot uses? Anyways, thanks for clearing up my confusion. Joseph Tam From myworking82 at gmail.com Mon Oct 19 06:36:53 2015 From: myworking82 at gmail.com (MyWorking) Date: Mon, 19 Oct 2015 09:36:53 +0300 Subject: replication full documentation Message-ID: <56248F85.1000207@gmail.com> Hi! I am create replication between two Debian 8.2 servers with dovecot 2.2.13. It work, but wont now more about replication used and i didn't now where i can red it: 1. aggregator - documentation and what he done? 2. replicator - full description or man for him 3. dsync - full description or man for him 4. doveadm replicator status - full description for all his points: * Queued 'sync' requests - when it started and what it is mean? * Queued 'high' requests - when it started and what it is mean? * Queued 'low' requests - when it started and what it is mean? * Queued 'failed' requests - when it started and what it is mean? * Queued 'full resync' requests - when it started and what it is mean? (wherry often i see this point is active on status for all my mail accounts) * Waiting 'failed' requests - i now what is it! * Total number of known users - i now what is it! 5. can dovecot create replication in real-time???? From grupodecorreo10 at gmail.com Mon Oct 19 07:38:18 2015 From: grupodecorreo10 at gmail.com (grupo correo) Date: Mon, 19 Oct 2015 09:38:18 +0200 Subject: Problems sync with mail client MAC and dovecot-2.2.10-4.el7_0.1.el7_0.1.x86_64 Message-ID: > > > Date: Fri, 16 Oct 2015 10:22:37 +0200 > From: grupo correo > To: dovecot at dovecot.org > Subject: Re: Problems sync with mail client MAC and > dovecot-2.2.10-4.el7_0.1.el7_0.1.x86_64 > Message-ID: > < > CALxU0ypQfUkT5Df6ZeeN+_QrCfPcZF7tgDsOgGSM_84xVQF-eA at mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > > > Date: Wed, 14 Oct 2015 11:43:22 +0200 > > From: Christian Kivalo > > To: Dovecot Mailing List > > Subject: Re: Problems sync with mail client MAC and > > dovecot-2.2.10-4.el7_0.1.el7_0.1.x86_64 > > Message-ID: <33ba2eacef5dcfcb4c703615e559b094 at valo.at> > > Content-Type: text/plain; charset=US-ASCII; format=flowed > > > > > > > Attached the doveconf -n > > > Yes i have mail_debug = yes into file conf.d/10-logging.conf > > > I paste the rest of log: > > > Oct 13 21:27:17 localhost dovecot: imap: Debug: Loading modules from > > > directory: /usr/lib64/dovecot > > > Oct 13 21:27:17 localhost dovecot: imap: Debug: Module loaded: > > > /usr/lib64/dovecot/lib01_acl_plugin.so > > > Oct 13 21:27:17 localhost dovecot: imap: Debug: Module loaded: > > > /usr/lib64/dovecot/lib02_imap_acl_plugin.so > > > Oct 13 21:27:17 localhost dovecot: imap: Debug: Module loaded: > > > /usr/lib64/dovecot/lib10_quota_plugin.so > > > Oct 13 21:27:17 localhost dovecot: imap: Debug: Module loaded: > > > /usr/lib64/dovecot/lib11_imap_quota_plugin.so > > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: > > > Effective > > > uid=56396, gid=55000, home=/VMAIL/xxxxxxxxxxxxxx > > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: Quota > > > root: > > > name=User quota backend=maildir args= > > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: Quota > > > grace: root=User quota bytes=0 (10%) > > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: > > > Namespace > > > inbox: type=private, prefix=INBOX., sep=., inbox=yes, hidden=no, > > > list=children, subscriptions=yes location=maildir:~/Maildir > > > > Your inbox namespace seems to be ok and is also defined in your doveconf > > -n output. > > > > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: > > > maildir++: > > > root=/VMAIL/xxxxxxxxxxxxxx/Maildir, index=, indexpvt=, control=, > > > inbox=/VMAIL/xxxxxxxxxxxxxx/Maildir, alt= > > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: acl: > > > initializing backend with data: vfile > > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: acl: > > > acl > > > username = xxxxxxxxxxxxxx > > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: acl: > > > owner > > > = 1 > > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: acl > > > vfile: > > > Global ACL directory: (none) > > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: > > > Namespace : > > > type=private, prefix=, sep=, inbox=no, hidden=yes, list=no, > > > subscriptions=no location=fail::LAYOUT=none > > > > Somehow there seems to be an empty namespace in addition to your inbox > > namespace. > > > > Is there a second namespace in your config somewhere? > > > > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Debug: none: > > > root=, index=, indexpvt=, control=, inbox=, alt= > > > Oct 13 21:27:17 localhost dovecot: imap(xxxxxxxxxxxxxx): Fatal: master: > > > service(imap): child 2464 killed with signal 11 (core dumps disabled) > > > > Just guessing: > > Could it be with Thunderbird that the second namespace is ignored when > > there is no subscription to it and the Apple mailprogram behaves > > differently? > > > > regards > > christian > > > > > > > > > Hi, > i can reproduce the error, conencting opnessl to port 993: > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > dovecot mailing list > dovecot at dovecot.org > http://dovecot.org/cgi-bin/mailman/listinfo/dovecot > > ------------------------------ > > Hello, excuse me my last email, i clicked very fast..... I can reproduce the problem doing: - openssl s_client -host localhost -port 993 - 0 LOGIN xxxxxxxxxxxx xxxxxxxxxxxxxxxx ......... - 0 NAMESPACE * NAMESPACE (("INBOX." ".")) NIL (("#shared." ".")("shared." ".")) 0 OK NAMESPACE completed. 0 list "*" "" closed And the log show: Oct 19 09:35:58 localhost dovecot: imapxxxxxxxxxxxxxxxx): Fatal: master: service(imap): child 22048 killed with signal 11 (core dumps disabled) I can reproduce this problem with Mac email client and Outlook 2007. Best regards From nick+dovecot at bunbun.be Mon Oct 19 09:44:10 2015 From: nick+dovecot at bunbun.be (Nick Rosier) Date: Mon, 19 Oct 2015 11:44:10 +0200 Subject: doveadm index assertion failed In-Reply-To: References: <7cdb984d3bb0d4ede95a16a2b2fb8d9a@bunbun.be> <56210E1B.1090408@bunbun.be> <5621619A.5040501@bunbun.be> Message-ID: <5624BB6A.9060403@bunbun.be> Timo Sirainen wrote: > On 16 Oct 2015, at 23:44, Nick Rosier wrote: >>> gdb --args doveadm index -u user at domain INBOX >>> run >>> >>> f 5 >>> p src >>> p ic_srcbuf >>> p *src_size >>> p srcleft >> I recompiled Dovecot with Debug but I suspect I will have to do it for all the required libraries as well; I did manage to find the offending message is in Trash but have not been able to pin-point which message it is. Just in case, here is the debugging info. I will see if I can compile all dependencies in debug mode as well. >> >> Program received signal SIGABRT, Aborted. >> 0x000000080149f64a in thr_kill () from /lib/libc.so.7 >> (gdb) f 5 >> #5 0x0000000801143cd3 in i_panic (format=0x801191c30 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:275 > > I guessed the frame wrong. It's usually 5, but I guess in FreeBSD it's 6 then. So try the same commands, but "f 6" first before "f 5" (and if that doesn't work either, try f 7). Indeed Timo, this provides more info. Hope this helps to track down the bug. GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... (gdb) run Starting program: /usr/local/bin/doveadm index -u xxx at xxx.xxx Trash doveadm(robots at rkfomh.net): Panic: file charset-iconv.c: line 85 (charset_to_utf8_try): assertion failed: (srcleft <= CHARSET_MAX_PENDING_BUF_SIZE) Program received signal SIGABRT, Aborted. 0x000000080149f64a in thr_kill () from /lib/libc.so.7 (gdb) f 6 #6 0x0000000801138c2c in charset_to_utf8_try (t=0x6d0360, src=0x7e7cd6 "Your message\n\n From:+AAkAIgQgBEMEOgQ+BDIEPgQ0BDgEQgQ1BDsETAA- +BDgEPQREBD4EQAQ8BDAERgQ4BD4EPQQ9BD4EMwQ+AA- +BEYENQQ9BEIEQAQwACIA- \n\n To:+AAkA-efg at clasinfo.com\n\n Subj:+AAkEGgQwBDQEM"..., src_size=0x7fffffffdf18, dest=0x788680, result=0x7fffffffdf2c) at charset-iconv.c:85 85 charset-iconv.c: No such file or directory. in charset-iconv.c Current language: auto; currently minimal (gdb) p src $1 = ( const unsigned char *) 0x7e7cd6 "Your message\n\n From:+AAkAIgQgBEMEOgQ+BDIEPgQ0BDgEQgQ1BDsETAA- +BDgEPQREBD4EQAQ8BDAERgQ4BD4EPQQ9BD4EMwQ+AA- +BEYENQQ9BEIEQAQwACIA- \n\n To:+AAkA-efg at clasinfo.com\n\n Subj:+AAkEGgQwBDQEM"... (gdb) p ic_srcbuf $2 = 0x7e7d11 "A- +BDgEPQREBD4EQAQ8BDAERgQ4BD4EPQQ9BD4EMwQ+AA- +BEYENQQ9BEIEQAQwACIA- \n\n To:+AAkA-efg at clasinfo.com\n\n Subj:+AAkEGgQwBDQEMARBBEIEQAQ+BDIESwQ5AA- +BEMERwQ1BEIA-\n Sent:+AAkA-2012-06-1"... (gdb) p *src_size $3 = 901 (gdb) p srcleft $4 = 842 (gdb) From tss at iki.fi Mon Oct 19 09:58:12 2015 From: tss at iki.fi (Timo Sirainen) Date: Mon, 19 Oct 2015 12:58:12 +0300 Subject: Hibernation disabled when mailbox selected In-Reply-To: <4989A281-52A3-41E6-A2AF-A1C776468458@iki.fi> References: <4989A281-52A3-41E6-A2AF-A1C776468458@iki.fi> Message-ID: > On 18 Oct 2015, at 20:25, Timo Sirainen wrote: > > On 18 Oct 2015, at 05:20, Joseph Tam wrote: >> >> >> After setting imap_hibernate_timeout to 60s, I could not find any >> hibernated connections after a few hours. I tested hibernation and made >> an observation: IDLE'd imap sessions only hibernate if they don't have >> a mailbox SELECT'd, otherwise they never hibernate. >> >> Is this the way it's supposed to work? > > What client is that? IDLE doesn't really make much sense when a mailbox isn't SELECTed, so I expected that all the clients would have something SELECTed.. But yeah, the code could be changed to support that. Actually, it already works, so your problem is something different. Set mail_debug=yes and it'll log the reason why hiberation couldn't be done. I've heard that some clients issue STATUS commands periodically and that causes them not to be hibernated. This is something that probably needs to be implemented by imap-hibernate at some point. From tss at iki.fi Mon Oct 19 10:31:41 2015 From: tss at iki.fi (Timo Sirainen) Date: Mon, 19 Oct 2015 13:31:41 +0300 Subject: More information about - Dovecot 2.2.x quota mysql and dict In-Reply-To: <1E35FEAE-2CE8-492E-8F26-AFA3061F222A@iki.fi> References: <1444170893263-54077.post@n4.nabble.com> <1E35FEAE-2CE8-492E-8F26-AFA3061F222A@iki.fi> Message-ID: <2B32C2E2-DAAA-4300-8F6F-C57647C7AD10@iki.fi> On 17 Oct 2015, at 12:41, Timo Sirainen wrote: > >> Connect postfix at localhost on postfix >> 134 Query SELECT bytes FROM quota2 WHERE username = >> 'user at mydomain.lan' >> 135 Query SELECT bytes FROM quota2 WHERE username = >> 'user at mydomain.lan' >> 135 Query BEGIN >> 135 Query UPDATE quota2 SET >> bytes=bytes+2570,messages=messages+1 WHERE username = 'user at mydomain.lan' > > Here's it's updating quota with user at mydomain.lan > >> 140 Query SELECT messages FROM quota2 WHERE username = >> 'mydomain.lan' >> 140 Query BEGIN >> 140 Query DELETE FROM domain WHERE domain = 'mydomain.lan' >> 140 Query DELETE FROM quota2 WHERE username = 'mydomain.lan' >> 140 Query INSERT INTO domain (quota,domain) VALUES >> ('8581','mydomain.lan') ON DUPLICATE KEY UPDATE quota='8581' >> 151004 15:18:45 140 Quit > > Here it's only mydomain.lan. So something's not right. After seeing your configs, this makes more sense now, although it's a bit confusing. You're updating quota bytes to "domain" table, but quota messages to "quota2" table where both domain and user quotas are mixed. >> Now dic + mysql doesn?t work in Dovecot 2.2.19, because first DELETE >> operator delete information from base about my domain mydomain.lan > > I can't reproduce this. What's your doveconf -n output? And what's in the dovecot-dict-sql.conf.ext? I still couldn't reproduce this with your exact config. I think that there is no bug or change in behavior in v2.2.19. There are some situations when Dovecot wants to recalculate the quota (e.g. a message is expunged but its size isn't known). In these situations Dovecot recalculates the quota and does a DELETE + INSERT to dict. This code has been there since the beginning. Maybe you just happened to be finally triggering this situation. So it's not safe to place the quota updates to a table that has other information also. Also this means that the domain-quota doesn't work 100% in all situations and I guess some kind of a daily recalculation for them would be a good idea.. From tss at iki.fi Mon Oct 19 11:06:06 2015 From: tss at iki.fi (Timo Sirainen) Date: Mon, 19 Oct 2015 14:06:06 +0300 Subject: TLS communication director -> backend with X.509 cert checks? In-Reply-To: <20151014212843.GF4156@jumper.schlittermann.de> References: <20151013184437.GV4156@jumper.schlittermann.de> <9831E5FF-3161-4CC9-ACA9-AB36F23C589B@iki.fi> <20151013191835.GW4156@jumper.schlittermann.de> <9E604417-8950-4426-89A7-172B27AA47E9@iki.fi> <20151013203323.GA4156@jumper.schlittermann.de> <20151013213447.GB4156@jumper.schlittermann.de> <20151013221050.GC4156@jumper.schlittermann.de> <20151013224611.GD4156@jumper.schlittermann.de> <20151013231020.GE4156@jumper.schlittermann.de> <20151014212843.GF4156@jumper.schlittermann.de> Message-ID: <8D60B903-18BD-4E2A-B865-74E0D5CEFCDB@iki.fi> > On 15 Oct 2015, at 00:28, Heiko Schlittermann wrote: > > Hi Timo > > Heiko Schlittermann (Mi 14 Okt 2015 01:10:20 CEST): > ? >> Ah, the information comes from the other director running. The other one >> is using an unpatched version of dovecot. > > Your patch for backend-certificate verification works. Thank you for the > good and fast work. Is there any chance that this will make it into > Dovecot's next release? Implemented also support for sending the hostname within director ring: http://hg.dovecot.org/dovecot-2.2/rev/8e9cada0c8fc http://hg.dovecot.org/dovecot-2.2/rev/7f718c840aff http://hg.dovecot.org/dovecot-2.2/rev/5876ca2d63fb Although it's not possible right now to add hostname using "doveadm director add", so that probably needs to be implemented at some point. > BTW: The ambiguity of 2001:db8::9090 remains. Shouldn't you allow > [2001:db8::]? resp [2001:db8::9090]? resp. [2001:db8::]:9090? for such > cases? (In case one want's to use IPv6 addresses instead of hostnames in > the director_servers option. (And probably in other places too.)) http://hg.dovecot.org/dovecot-2.2/rev/c5c34c02fda3 From tss at iki.fi Mon Oct 19 11:09:22 2015 From: tss at iki.fi (Timo Sirainen) Date: Mon, 19 Oct 2015 14:09:22 +0300 Subject: doveadm index assertion failed In-Reply-To: References: <7cdb984d3bb0d4ede95a16a2b2fb8d9a@bunbun.be> <56210E1B.1090408@bunbun.be> <5621619A.5040501@bunbun.be> Message-ID: <4242E6B9-FA6D-4068-AABA-DA76F9315E11@iki.fi> > On 17 Oct 2015, at 11:12, Timo Sirainen wrote: > > On 16 Oct 2015, at 23:44, Nick Rosier wrote: >> >> >>> gdb --args doveadm index -u user at domain INBOX >>> run >>> >>> f 5 >>> p src >>> p ic_srcbuf >>> p *src_size >>> p srcleft >> I recompiled Dovecot with Debug but I suspect I will have to do it for all the required libraries as well; I did manage to find the offending message is in Trash but have not been able to pin-point which message it is. Just in case, here is the debugging info. I will see if I can compile all dependencies in debug mode as well. >> >> Program received signal SIGABRT, Aborted. >> 0x000000080149f64a in thr_kill () from /lib/libc.so.7 >> (gdb) f 5 >> #5 0x0000000801143cd3 in i_panic (format=0x801191c30 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:275 > > I guessed the frame wrong. It's usually 5, but I guess in FreeBSD it's 6 then. So try the same commands, but "f 6" first before "f 5" (and if that doesn't work either, try f 7). Uh. I meant of course: type "f 6" instead of "f 5". From jtam.home at gmail.com Mon Oct 19 23:10:44 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 19 Oct 2015 16:10:44 -0700 (PDT) Subject: Hibernation disabled when mailbox selected In-Reply-To: References: <4989A281-52A3-41E6-A2AF-A1C776468458@iki.fi> Message-ID: On Mon, 19 Oct 2015, Timo Sirainen wrote: > Actually, it already works, so your problem is something different. > Set mail_debug=yes and it'll log the reason why hiberation couldn't be > done. imap(ono): Debug: Couldn't hibernate imap client: Couldn't extract notifications fd: Filesystem notifications not supported You mentioned something about this, but I must have overlooked it. Mailbox changes are noticed only if inotify/kqueue is enabled, so without them the hibernation feature won't work well. Solaris10 does not have support for this (but Solaris11 does have File Event Notification). Pity. Joseph Tam From listas.correo at yahoo.es Tue Oct 20 09:26:26 2015 From: listas.correo at yahoo.es (mailing lists) Date: Tue, 20 Oct 2015 09:26:26 +0000 (UTC) Subject: Dovecot 2.2.19 - Panic: file dict-sql.c: line 670 (sql_dict_iterate): assertion failed: ((ctx->flags & DICT_ITERATE_FLAG_ASYNC) != 0) References: <1253844734.208105.1445333186188.JavaMail.yahoo@mail.yahoo.com> Message-ID: <1253844734.208105.1445333186188.JavaMail.yahoo@mail.yahoo.com> Hello everyone, For this installation I am using passdb against AD, userdb with MySQL and nfs storage but using simultaneously the vfile acl backend and the shared mailbox dictionary I get the following panic: Oct 19 12:34:51 server dovecot: imap-login: Login: user=, method=PLAIN, rip=10.112.99.52, lip=10.113.63.50, mpid=7538, session= Oct 19 12:34:51 server dovecot: dict(7540): Panic: file dict-sql.c: line 670 (sql_dict_iterate): assertion failed: ((ctx->flags & DICT_ITERATE_FLAG_ASYNC) != 0) Oct 19 12:34:51 server dovecot: dict(7540): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0() [0x31826800ca] -> /usr/lib64/dovecot/libdovecot.so.0() [0x3182680136] -> /usr/lib64/dovecot/libdovecot.so.0() [0x31826299ea] -> dovecot-mailbox/dict() [0x406abc] -> dovecot-mailbox/dict() [0x4047aa] -> dovecot-mailbox/dict() [0x404f32] -> dovecot-mailbox/dict(dict_command_input+0xab) [0x4050bb] -> dovecot-mailbox/dict() [0x40425b] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x3182692deb] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xb7) [0x31826943c7] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x3182692e95] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x3182693038] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x318262e953] -> dovecot-mailbox/dict(main+0x165) [0x405295] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x303be1ed5d] -> dovecot-mailbox/dict() [0x403da9] Oct 19 12:34:51 server dovecot: dict(7540): Fatal: master: service(dict): child 7540 killed with signal 6 (core dumps disabled) to reproduce the panic I only have to delete the dovecot-acl-list file and access the mailbox. It looks like that this commit http://hg.dovecot.org/dovecot-2.2/rev/7ccff6d5dd1b is causing the panic. # 2.2.19: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.9 (357ac0a0e68b+) # OS: Linux 2.6.32-573.1.1.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.7 (Santiago) auth_master_user_separator = * auth_username_format = %Ln dict { ? acl = mysql:/etc/dovecot/dovecot-dict-acl.conf.ext } disable_plaintext_auth = no instance_name = dovecot-mailbox lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * lmtp_rcpt_check_quota = yes lmtp_save_to_detail_mailbox = yes mail_fsync = always mail_gid = vmail mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = " mail_log notify acl mailbox_alias quota listescape" mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex im ap4flags copy include variables body enotify environment mailbox date index ihave duplicate imapflags notify mbox_write_locks = fcntl mmap_disable = yes namespace { ? list = children ? location = maildir:%%h/Mail:INDEX=~/shared/%%u:INDEXPVT=~/shared-pvt/%%u ? prefix = shared/%%u/ ? separator = / ? subscriptions = no ? type = shared } namespace inbox { ? inbox = yes ? location = ? mailbox Drafts { ??? special_use = \Drafts ? } ? mailbox Junk { ??? special_use = \Junk ? } ? mailbox Sent { ??? special_use = \Sent ? } ? mailbox "Sent Messages" { ??? special_use = \Sent ? } ? mailbox Trash { ??? special_use = \Trash ? } ? mailbox sent_mail_folder { ??? special_use = \Sent ? } ? mailbox trash_folder { ??? special_use = \Trash ? } ? prefix = ? separator = / ? type = private } passdb { ? args = /etc/dovecot/dovecot-pass_db-ldap.conf.ext ? driver = ldap } plugin { ? acl = vfile ? acl_shared_dict = proxy::acl ? mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append ? mail_log_fields = uid box msgid from subject size flags ? sieve = file:~/sieve;active=~/.dovecot.sieve ? sieve_extensions = +notify +imapflags ? sieve_max_actions = 32 ? sieve_max_redirects = 25 ? sieve_max_script_size = 1M ? sieve_quota_max_storage = 10M } protocols = imap pop3 lmtp sieve service auth { ? unix_listener auth-userdb { ??? group = vmail ? } } service dict { ? unix_listener dict { ??? group = dovecot ??? mode = 0660 ??? user = vmail ? } } service lmtp { ? inet_listener lmtp { ??? port = 24 ? } } service managesieve-login { ? inet_listener sieve { ??? port = 4190 ? } ? inet_listener sieve_deprecated { ??? port = 2000 ? } ? service_count = 1 } service managesieve { ? process_limit = 1024 } ssl = no userdb { ? args = /etc/dovecot/dovecot-user_db-sql.conf.ext ? driver = sql } verbose_proctitle = yes protocol lmtp { ? mail_plugins = " mail_log notify acl mailbox_alias quota listescape sieve" } protocol lda { ? mail_plugins = " mail_log notify acl mailbox_alias quota listescape sieve" } protocol imap { ? mail_plugins = " mail_log notify acl mailbox_alias quota listescape imap_acl imap_quota" } protocol sieve { ? managesieve_implementation_string = Dovecot Pigeonhole ? managesieve_logout_format = bytes=%i/%o ? managesieve_max_compile_errors = 1 ? managesieve_max_line_length = 65536 } protocol pop3 { ? mail_plugins = " mail_log notify acl mailbox_alias quota listescape" } From listas.correo at yahoo.es Mon Oct 19 14:37:43 2015 From: listas.correo at yahoo.es (mailing lists) Date: Mon, 19 Oct 2015 14:37:43 +0000 (UTC) Subject: Dovecot 2.2.19 - Panic: file dict-sql.c: line 670 (sql_dict_iterate): assertion failed: ((ctx->flags & DICT_ITERATE_FLAG_ASYNC) != 0) References: <872294195.3874003.1445265463840.JavaMail.yahoo@mail.yahoo.com> Message-ID: <872294195.3874003.1445265463840.JavaMail.yahoo@mail.yahoo.com> Hello everyone, For this installation I am using passdb against AD, userdb with MySQL and nfs storage but using simultaneously the vfile acl backend and the shared mailbox dictionary I get the following panic: Oct 19 12:34:51 server dovecot: imap-login: Login: user=, method=PLAIN, rip=10.112.99.52, lip=10.113.63.50, mpid=7538, session= Oct 19 12:34:51 server dovecot: dict(7540): Panic: file dict-sql.c: line 670 (sql_dict_iterate): assertion failed: ((ctx->flags & DICT_ITERATE_FLAG_ASYNC) != 0) Oct 19 12:34:51 server dovecot: dict(7540): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0() [0x31826800ca] -> /usr/lib64/dovecot/libdovecot.so.0() [0x3182680136] -> /usr/lib64/dovecot/libdovecot.so.0() [0x31826299ea] -> dovecot-mailbox/dict() [0x406abc] -> dovecot-mailbox/dict() [0x4047aa] -> dovecot-mailbox/dict() [0x404f32] -> dovecot-mailbox/dict(dict_command_input+0xab) [0x4050bb] -> dovecot-mailbox/dict() [0x40425b] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x3182692deb] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xb7) [0x31826943c7] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x3182692e95] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x3182693038] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x318262e953] -> dovecot-mailbox/dict(main+0x165) [0x405295] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x303be1ed5d] -> dovecot-mailbox/dict() [0x403da9] Oct 19 12:34:51 server dovecot: dict(7540): Fatal: master: service(dict): child 7540 killed with signal 6 (core dumps disabled) am I missing something obvious or this is a dovecot bug? # 2.2.19: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.9 (357ac0a0e68b+) # OS: Linux 2.6.32-573.1.1.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.7 (Santiago) auth_master_user_separator = * auth_username_format = %Ln dict { ? acl = mysql:/etc/dovecot/dovecot-dict-acl.conf.ext } disable_plaintext_auth = no instance_name = dovecot-mailbox lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * lmtp_rcpt_check_quota = yes lmtp_save_to_detail_mailbox = yes mail_fsync = always mail_gid = vmail mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = " mail_log notify acl mailbox_alias quota listescape" mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex im ap4flags copy include variables body enotify environment mailbox date index ihave duplicate imapflags notify mbox_write_locks = fcntl mmap_disable = yes namespace { ? list = children ? location = maildir:%%h/Mail:INDEX=~/shared/%%u:INDEXPVT=~/shared-pvt/%%u ? prefix = shared/%%u/ ? separator = / ? subscriptions = no ? type = shared } namespace inbox { ? inbox = yes ? location = ? mailbox Drafts { ??? special_use = \Drafts ? } ? mailbox Junk { ??? special_use = \Junk ? } ? mailbox Sent { ??? special_use = \Sent ? } ? mailbox "Sent Messages" { ??? special_use = \Sent ? } ? mailbox Trash { ??? special_use = \Trash ? } ? mailbox sent_mail_folder { ??? special_use = \Sent ? } ? mailbox trash_folder { ??? special_use = \Trash ? } ? prefix = ? separator = / ? type = private } passdb { ? args = /etc/dovecot/dovecot-pass_db-ldap.conf.ext ? driver = ldap } plugin { ? acl = vfile ? acl_shared_dict = proxy::acl ? mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append ? mail_log_fields = uid box msgid from subject size flags ? sieve = file:~/sieve;active=~/.dovecot.sieve ? sieve_extensions = +notify +imapflags ? sieve_max_actions = 32 ? sieve_max_redirects = 25 ? sieve_max_script_size = 1M ? sieve_quota_max_storage = 10M } protocols = imap pop3 lmtp sieve service auth { ? unix_listener auth-userdb { ??? group = vmail ? } } service dict { ? unix_listener dict { ??? group = dovecot ??? mode = 0660 ??? user = vmail ? } } service lmtp { ? inet_listener lmtp { ??? port = 24 ? } } service managesieve-login { ? inet_listener sieve { ??? port = 4190 ? } ? inet_listener sieve_deprecated { ??? port = 2000 ? } ? service_count = 1 } service managesieve { ? process_limit = 1024 } ssl = no userdb { ? args = /etc/dovecot/dovecot-user_db-sql.conf.ext ? driver = sql } verbose_proctitle = yes protocol lmtp { ? mail_plugins = " mail_log notify acl mailbox_alias quota listescape sieve" } protocol lda { ? mail_plugins = " mail_log notify acl mailbox_alias quota listescape sieve" } protocol imap { ? mail_plugins = " mail_log notify acl mailbox_alias quota listescape imap_acl imap_quota" } protocol sieve { ? managesieve_implementation_string = Dovecot Pigeonhole ? managesieve_logout_format = bytes=%i/%o ? managesieve_max_compile_errors = 1 ? managesieve_max_line_length = 65536 } protocol pop3 { ? mail_plugins = " mail_log notify acl mailbox_alias quota listescape" } From tss at iki.fi Tue Oct 20 10:13:17 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 20 Oct 2015 13:13:17 +0300 Subject: Dovecot 2.2.19 - Panic: file dict-sql.c: line 670 (sql_dict_iterate): assertion failed: ((ctx->flags & DICT_ITERATE_FLAG_ASYNC) != 0) In-Reply-To: <1253844734.208105.1445333186188.JavaMail.yahoo@mail.yahoo.com> References: <1253844734.208105.1445333186188.JavaMail.yahoo@mail.yahoo.com> <1253844734.208105.1445333186188.JavaMail.yahoo@mail.yahoo.com> Message-ID: On 20 Oct 2015, at 12:26, mailing lists wrote: > > Hello everyone, > For this installation I am using passdb against AD, userdb with MySQL and nfs storage but using simultaneously the vfile acl backend and the shared mailbox dictionary I get the following panic: > Oct 19 12:34:51 server dovecot: imap-login: Login: user=, method=PLAIN, rip=10.112.99.52, lip=10.113.63.50, mpid=7538, session= > Oct 19 12:34:51 server dovecot: dict(7540): Panic: file dict-sql.c: line 670 (sql_dict_iterate): assertion failed: ((ctx->flags & DICT_ITERATE_FLAG_ASYNC) != 0) http://hg.dovecot.org/dovecot-2.2/rev/467695fee373 probably fixes this? From jameszee13 at gmail.com Tue Oct 20 12:39:25 2015 From: jameszee13 at gmail.com (James) Date: Tue, 20 Oct 2015 08:39:25 -0400 Subject: help filtering messages Message-ID: Hello, I'm attempting to use sieve-filter on a local mailbox. I've defined mail_location as follows: % grep mail_location /etc/dovecot/conf.d/10-mail.conf mail_location = maildir:~/Mail I have a large number of mailboxes / mail in ~/Mail: ~ % find Mail | wc -l 63158 There are definitely a few messages in INBOX: ~ % find Mail/INBOX/cur | wc -l 5 I must admit, I'm completely new to sieve and have just begun experimenting with it. Based on a super helpful blog post[1], I quick set up the following ~/.sieve: % cat ~/.sieve require ["fileinto"]; # rule:[TEST] if header :contains "Subject" "Test Email" { fileinto "TEST"; stop; } As mentioned, there are numerous emails in INBOX. This email is located in ~/Mail/INBOX/cur/ and, for verbosity (and to ensure I'm not making a foolish mistake), I've displayed it here: From: Test Account To: "Me" Subject: Test Email Date: Mon, 19 Oct 2015 14:55:06 +0000 This is a test email. Looking at the sieve filter recipe and email message, it appears that I should have a match, no? sieve-filter appears to disagree: % sieve-filter -C ~/.sieve INBOX sieve-filter(test_user): Debug: Effective uid=1000, gid=1000, home=/home/test_user sieve-filter(test_user): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Mail sieve-filter(test_user): Debug: maildir++: root=/home/test_user/Mail, index=, indexpvt=, control=, inbox=/home/test_user/Mail, alt= Thinking this was because sieve will only filter on new messages, I moved the file from cur to new. No change. Any thoughts on why this may be happening? Thanks! 1 https://mebsd.com/configure-freebsd-servers/dovecot-pigeonhole-sieve-filter-refilter-delivered-email.html From listas.correo at yahoo.es Tue Oct 20 13:06:13 2015 From: listas.correo at yahoo.es (mailing lists) Date: Tue, 20 Oct 2015 13:06:13 +0000 (UTC) Subject: Dovecot 2.2.19 - Panic: file dict-sql.c: line 670 (sql_dict_iterate): assertion failed: ((ctx->flags & DICT_ITERATE_FLAG_ASYNC) != 0) In-Reply-To: References: Message-ID: <1562960979.433071.1445346373356.JavaMail.yahoo@mail.yahoo.com> > On 20 Oct 2015, at 12:26, mailing lists wrote: > >> Oct 19 12:34:51 server dovecot: dict(7540): Panic: file dict-sql.c: line 670 (sql_dict_iterate): assertion failed: ((ctx->flags & DICT_ITERATE_FLAG_ASYNC) != 0)> > > http://hg.dovecot.org/dovecot-2.2/rev/467695fee373 probably fixes this? yes, it does it!! thank you. From f.bosch at genkgo.nl Wed Oct 21 15:07:52 2015 From: f.bosch at genkgo.nl (Frederik Bosch | Genkgo) Date: Wed, 21 Oct 2015 17:07:52 +0200 Subject: events Message-ID: <5627AA48.5030809@genkgo.nl> Hello, We want to trigger a script after certain actions by the user (event). This script inserts the action into message queue (e.g. Rabbit MQ) accompanied with some data. Then one or more workers picks up the action from the message queue and do something with it. The question is: how can I trigger the script from dovecot? An action/event should at least emitted when moving an e-mail message. The amount of data in the event is unknown. My current goal I would like to at least know the sender and receiver address and the type of action (e.g. move). But I can imagine, when having an event driven application, that others might even want the complete e-mail message to be in data. Is such an event driven system possible? When looking at current features of Dovecot: it would the same as the mail_log feature, but instead of writing to a file, one should able to write to a script or socket. Ideally the format of the event would JSON, XML or another data format. Regards, Frederik Current version: Dovecot 2.2.16. From guestuser464 at gmail.com Wed Oct 21 16:47:55 2015 From: guestuser464 at gmail.com (guest user) Date: Wed, 21 Oct 2015 18:47:55 +0200 Subject: Dsync only indexes Message-ID: Hello, We are using dbox and we would like to put indexes on a separate storage, our actual setup have indexes on the same storage has the messages. Is there a way to only synchronise the indexes to easily change our setup ? Thanks. From micah at riseup.net Wed Oct 21 17:13:45 2015 From: micah at riseup.net (micah) Date: Wed, 21 Oct 2015 13:13:45 -0400 Subject: migration from zimbra to dovecot Message-ID: <87r3ko5ely.fsf@muck.riseup.net> Hello, Are there any migration scripts to handle UID conversion for moving from a zimbra setup to dovecot? It would be nice to move all the mail without having re-download it all! thanks! micah ps - asking for a friend From michael.slusarz at dovecot.fi Wed Oct 21 21:33:39 2015 From: michael.slusarz at dovecot.fi (Michael M Slusarz) Date: Wed, 21 Oct 2015 15:33:39 -0600 Subject: events In-Reply-To: <5627AA48.5030809@genkgo.nl> References: <5627AA48.5030809@genkgo.nl> Message-ID: <562804B3.2060405@dovecot.fi> On 10/21/2015 9:07 AM, Frederik Bosch | Genkgo wrote: > We want to trigger a script after certain actions by the user (event). > This script inserts the action into message queue (e.g. Rabbit MQ) > accompanied with some data. Then one or more workers picks up the action > from the message queue and do something with it. The question is: how > can I trigger the script from dovecot? This is precisely what the new push-notification plugin is for (2.2.19). (You will need to write a driver to interact with your notification handler, similar to the "dlog" or "ox" drivers.) michael From sami.ketola at dovecot.fi Thu Oct 22 06:38:08 2015 From: sami.ketola at dovecot.fi (Sami Ketola) Date: Thu, 22 Oct 2015 09:38:08 +0300 Subject: migration from zimbra to dovecot In-Reply-To: <87r3ko5ely.fsf@muck.riseup.net> References: <87r3ko5ely.fsf@muck.riseup.net> Message-ID: <0E9EB409-B550-452E-AF99-E9C81C3FDD1F@dovecot.fi> > On 21 Oct 2015, at 20:13, micah wrote: > Are there any migration scripts to handle UID conversion for moving from > a zimbra setup to dovecot? It would be nice to move all the mail without > having re-download it all! Depends on what is the imap backend behind Zimbra or if using Zimbra builtin imap backend, but usually UID:s are preserved when mails are migrated by using dsync over imapc connection when the dsync target platform is Dovecot. This however requires that master password can be set on Zimbra side or users passwords are known. Sami From f.bosch at genkgo.nl Thu Oct 22 06:46:26 2015 From: f.bosch at genkgo.nl (Frederik Bosch | Genkgo) Date: Thu, 22 Oct 2015 08:46:26 +0200 Subject: events In-Reply-To: <562804B3.2060405@dovecot.fi> References: <5627AA48.5030809@genkgo.nl> <562804B3.2060405@dovecot.fi> Message-ID: <56288642.2070803@genkgo.nl> Michael, Thanks a lot! After looking at the source, I guess the ox driver will do. Maybe, when other people find this thread, you could tell what dlog is. Because I do not know it, and googling came up with little results. Regards, Frederik On 21-10-15 23:33, Michael M Slusarz wrote: > On 10/21/2015 9:07 AM, Frederik Bosch | Genkgo wrote: > >> We want to trigger a script after certain actions by the user (event). >> This script inserts the action into message queue (e.g. Rabbit MQ) >> accompanied with some data. Then one or more workers picks up the action >> from the message queue and do something with it. The question is: how >> can I trigger the script from dovecot? > > This is precisely what the new push-notification plugin is for > (2.2.19). (You will need to write a driver to interact with your > notification handler, similar to the "dlog" or "ox" drivers.) > > michael From tss at iki.fi Thu Oct 22 11:00:01 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 22 Oct 2015 14:00:01 +0300 Subject: More information about - Dovecot 2.2.x quota mysql and dict In-Reply-To: <2B32C2E2-DAAA-4300-8F6F-C57647C7AD10@iki.fi> References: <1444170893263-54077.post@n4.nabble.com> <1E35FEAE-2CE8-492E-8F26-AFA3061F222A@iki.fi> <2B32C2E2-DAAA-4300-8F6F-C57647C7AD10@iki.fi> Message-ID: <60FECC87-B08E-4716-9876-38129D7EB5F7@iki.fi> On 19 Oct 2015, at 13:31, Timo Sirainen wrote: > > On 17 Oct 2015, at 12:41, Timo Sirainen wrote: >> >>> Connect postfix at localhost on postfix >>> 134 Query SELECT bytes FROM quota2 WHERE username = >>> 'user at mydomain.lan' >>> 135 Query SELECT bytes FROM quota2 WHERE username = >>> 'user at mydomain.lan' >>> 135 Query BEGIN >>> 135 Query UPDATE quota2 SET >>> bytes=bytes+2570,messages=messages+1 WHERE username = 'user at mydomain.lan' >> >> Here's it's updating quota with user at mydomain.lan >> >>> 140 Query SELECT messages FROM quota2 WHERE username = >>> 'mydomain.lan' >>> 140 Query BEGIN >>> 140 Query DELETE FROM domain WHERE domain = 'mydomain.lan' >>> 140 Query DELETE FROM quota2 WHERE username = 'mydomain.lan' >>> 140 Query INSERT INTO domain (quota,domain) VALUES >>> ('8581','mydomain.lan') ON DUPLICATE KEY UPDATE quota='8581' >>> 151004 15:18:45 140 Quit >> >> Here it's only mydomain.lan. So something's not right. > > After seeing your configs, this makes more sense now, although it's a bit confusing. You're updating quota bytes to "domain" table, but quota messages to "quota2" table where both domain and user quotas are mixed. This is pretty much the issue here. The current code works exactly as it should, so I think the old code was buggy.. So what's happening with domain quota is: - priv/quota/storage maps to domain.quota field - priv/quota/messages maps to quota2.messages field The domain table has all the domains, so updating it has worked fine. But apparently there were no domain entries in quota2 table? And apparently the old Dovecot code was simply ignoring this and never writing anything in there? So that would be a bug. When either the storage or the messages don't exist, Dovecot does a quota recalculation and writes the resulting recalculated fields. This recalculation does a DELETE + INSERT internally. So there are 2 things you could do now..: 1) If you manually INSERT all the missing domains to quota table, the existing code continues to work. Because the domains exist there, there's no need for Dovecot to do a quota recalculation and DELETE anything. 2) This has never been 100% safe though. There has always been the possibility of quota recalculation deleting the domain. I've now added a "no-unset" parameter to dict: quota = dict:domain_quota:%d:no-unset:proxy::sqldomainquota This disables the DELETEs completely. However quota recalculation still attempts to do: INSERT INTO domain (quota,domain) VALUES ('5404','postdomain.lan') ON DUPLICATE KEY UPDATE quota='5404' Which fails in your case, because MySQL wants the INSERT to be valid as well. You can kludge around it by changing MySQL schema with: alter table domain alter description set default ''; alter table domain alter transport set default 'dovecot'; Now quota recalculation works as well. Except of course domain quota recalculation doesn't actually work correctly, because it calculates only the single user's quota and places it to the domain quota. From jameszee13 at gmail.com Fri Oct 23 03:33:12 2015 From: jameszee13 at gmail.com (James) Date: Fri, 23 Oct 2015 03:33:12 +0000 Subject: help filtering messages In-Reply-To: References: Message-ID: Bump - hoping someone can point me in the right direction. Thanks. On Tue, Oct 20, 2015 at 8:39 AM James wrote: > Hello, > > I'm attempting to use sieve-filter on a local mailbox. > > I've defined mail_location as follows: > % grep mail_location /etc/dovecot/conf.d/10-mail.conf > mail_location = maildir:~/Mail > > I have a large number of mailboxes / mail in ~/Mail: > > ~ % find Mail | wc -l > 63158 > > There are definitely a few messages in INBOX: > > ~ % find Mail/INBOX/cur | wc -l > 5 > > I must admit, I'm completely new to sieve and have just begun > experimenting with it. Based on a super helpful blog post[1], I quick > set up the following ~/.sieve: > > % cat ~/.sieve > require ["fileinto"]; > # rule:[TEST] > if header :contains "Subject" "Test Email" > { > fileinto "TEST"; > stop; > } > > As mentioned, there are numerous emails in INBOX. This email is > located in ~/Mail/INBOX/cur/ and, for verbosity (and to ensure > I'm not making a foolish mistake), I've displayed it here: > > From: Test Account > To: "Me" > Subject: Test Email > Date: Mon, 19 Oct 2015 14:55:06 +0000 > > This is a test email. > > Looking at the sieve filter recipe and email message, it appears that > I should have a match, no? sieve-filter appears to disagree: > > % sieve-filter -C ~/.sieve INBOX > sieve-filter(test_user): Debug: Effective uid=1000, gid=1000, > home=/home/test_user > sieve-filter(test_user): Debug: Namespace inbox: type=private, > prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes > location=maildir:~/Mail > sieve-filter(test_user): Debug: maildir++: root=/home/test_user/Mail, > index=, indexpvt=, control=, inbox=/home/test_user/Mail, alt= > > Thinking this was because sieve will only filter on new messages, I > moved the file from cur to new. No change. > > Any thoughts on why this may be happening? > > Thanks! > > 1 > https://mebsd.com/configure-freebsd-servers/dovecot-pigeonhole-sieve-filter-refilter-delivered-email.html > From skdovecot at smail.inf.fh-brs.de Fri Oct 23 06:35:42 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 23 Oct 2015 08:35:42 +0200 (CEST) Subject: help filtering messages In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 23 Oct 2015, James wrote: >> I'm attempting to use sieve-filter on a local mailbox. >> >> I've defined mail_location as follows: >> % grep mail_location /etc/dovecot/conf.d/10-mail.conf >> mail_location = maildir:~/Mail >> >> I have a large number of mailboxes / mail in ~/Mail: >> >> ~ % find Mail | wc -l >> 63158 >> >> There are definitely a few messages in INBOX: >> >> ~ % find Mail/INBOX/cur | wc -l >> 5 Actually, plain Maildir in Dovecot uses "Mail/cur" and "Mail/new". So either your mail_location is wrong (maybe it comes from the userdb) or your layout. Perhaps rename that directory "Mail/INBOX" -> "Mail/.testbox" >> I must admit, I'm completely new to sieve and have just begun >> experimenting with it. Based on a super helpful blog post[1], I quick >> set up the following ~/.sieve: >> >> % cat ~/.sieve >> require ["fileinto"]; >> # rule:[TEST] >> if header :contains "Subject" "Test Email" >> { >> fileinto "TEST"; >> stop; >> } >> >> As mentioned, there are numerous emails in INBOX. This email is >> located in ~/Mail/INBOX/cur/ and, for verbosity (and to ensure >> I'm not making a foolish mistake), I've displayed it here: >> >> From: Test Account >> To: "Me" >> Subject: Test Email >> Date: Mon, 19 Oct 2015 14:55:06 +0000 >> >> This is a test email. >> >> Looking at the sieve filter recipe and email message, it appears that >> I should have a match, no? sieve-filter appears to disagree: >> >> % sieve-filter -C ~/.sieve INBOX >> sieve-filter(test_user): Debug: Effective uid=1000, gid=1000, >> home=/home/test_user >> sieve-filter(test_user): Debug: Namespace inbox: type=private, >> prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes >> location=maildir:~/Mail >> sieve-filter(test_user): Debug: maildir++: root=/home/test_user/Mail, >> index=, indexpvt=, control=, inbox=/home/test_user/Mail, alt= >> >> Thinking this was because sieve will only filter on new messages, I >> moved the file from cur to new. No change. >> >> Any thoughts on why this may be happening? my version of sieve-filter does not output anything, regardless of the settings -v and -D, if and only if it found no message. I suspect, that mailbox "INBOX" in /home/test_user/Mail does not belong to /home/test_user/Mail/INBOX/{cur,new}, but to /home/test_user/Mail/{cur,new}. Rename INBOX to .testbox and try with sieve-filter -C ~/.sieve testbox - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVinVPnz1H7kL/d9rAQIXaggAiJPHMB06mybOgWYQH3umgsr6HhHxnv5n 5CV6Yb1xZsW9fuPFlua6FWx+KoBlG24+FOCNRnug/YEnmU/1QhJkfBjhB3BIGBKC YkAk2iz52FlXdDIramh10S2u/mT62UT7iuFm2snU2se5A63fABBOWcIahZih5Pc5 fSPZud85Ms4qp567V7PNbhBEqDE+piQOVWlThGkUF7ARgm3d14j2F8HfEsTKrZu5 CzaTbHiRWBkRX2AGYVGGG1LpZ0ebcCFiZt2N8gJ+78x3savF67zZtQrpBteFeVfu juPSYz4ukqGzC8BX4pqBHC8RBiLcjBFuA1ZrKAq8GtA9clF+4eEOSQ== =zc79 -----END PGP SIGNATURE----- From bind at enas.net Fri Oct 23 12:51:55 2015 From: bind at enas.net (Urban Loesch) Date: Fri, 23 Oct 2015 14:51:55 +0200 Subject: Question about dovecot replication Message-ID: <562A2D6B.2020107@enas.net> Hi, last week I installed 2 servers with two dovecot nodes and replication in active/active mode located in different datacenters. Based on the howto from wiki "http://wiki2.dovecot.org/Replication" it works great. According to the wiki it is recomended to run "doveadm purge" on both systems continously, because "doveadm purge" will not be replicated by the replication service. No problem so far. But I have one doubt: is it also reccmended (for keeping maildata in sync) to run "doveadm replicator replicate '*'" continously on both nodes? Or is it enough on only one node? Or should I run "doveadm sync -A tcp:anotherhost.example.com" in regular intervalls? Perhaps once a day on both nodes? Thanks Urban Loesch From michael.slusarz at dovecot.fi Fri Oct 23 14:12:53 2015 From: michael.slusarz at dovecot.fi (Michael M Slusarz) Date: Fri, 23 Oct 2015 08:12:53 -0600 Subject: events In-Reply-To: <56288642.2070803@genkgo.nl> References: <5627AA48.5030809@genkgo.nl> <562804B3.2060405@dovecot.fi> <56288642.2070803@genkgo.nl> Message-ID: <562A4065.6010706@dovecot.fi> On 10/22/2015 12:46 AM, Frederik Bosch | Genkgo wrote: > Thanks a lot! After looking at the source, I guess the ox driver will > do. Maybe, when other people find this thread, you could tell what dlog > is. Because I do not know it, and googling came up with little results. "dlog" is nothing more than a push-notification backend that will log various information and hook triggers (at a DEBUG level) to the Dovecot log. It's meant for debugging and development purposes. "dlog" stands for either "Dovecot LOGging" or "Debug LOGging", whichever you prefer. michael > On 21-10-15 23:33, Michael M Slusarz wrote: >> On 10/21/2015 9:07 AM, Frederik Bosch | Genkgo wrote: >> >>> We want to trigger a script after certain actions by the user (event). >>> This script inserts the action into message queue (e.g. Rabbit MQ) >>> accompanied with some data. Then one or more workers picks up the action >>> from the message queue and do something with it. The question is: how >>> can I trigger the script from dovecot? >> >> This is precisely what the new push-notification plugin is for >> (2.2.19). (You will need to write a driver to interact with your >> notification handler, similar to the "dlog" or "ox" drivers.) >> >> michael From lukas.mika at web.de Fri Oct 23 17:46:58 2015 From: lukas.mika at web.de (Lukas Mika) Date: Fri, 23 Oct 2015 19:46:58 +0200 Subject: Notify plugin doesn't send mailbox_create notifications Message-ID: <562A7292.4020308@web.de> Hi there, I'm running dovecot 2.2.13 on my debian jessie server. When installing my own dovecot plugin, I noticed that there are no notifications coming from the notify plugin. I checked back with the mail_log plugin, but also this plugin doesn't log any notifications when a new mailbox is created; the mail is saved to the new inbox never the less. I ran dovecot 2.2.9 before on my Ubuntu Trusty (14.04) server, where with the same settings the plugins worked. Can you reproduce this issue and does anybody know a fix to this? Here are the relevant bits of my config: 20-lmtp.conf: protocol lmtp { mail_plugins). mail_plugins = $mail_plugins sieve mail_log notify } 10-logging.conf ... plugin { mail_log_events = mailbox_create mail_log_fields = uid box } ... Cheers From those.lists at gmail.com Fri Oct 23 18:43:17 2015 From: those.lists at gmail.com (Forrest) Date: Fri, 23 Oct 2015 14:43:17 -0400 Subject: 2.2.18 initial setup permissions problem Message-ID: <562A7FC5.5030004@gmail.com> I'm using Dovecot 2.2.18, compiled from source, on CentOS 6. I'm running into a permissions problem with my first rollout of Dovecot which has to do with directory permissions: Oct 23 14:34:51 mail dovecot: imap(username): Error: stat(/home/username/Maildir/tmp) failed: Permission denied (euid=596() egid=97(dovecot) missing +x perm: /home/username/Maildir, dir owned by 500:500 mode=0700) This being setup with the Dovecot wiki instructions. I may have misunderstood an initial setting. If I change permissions to my personal Maildir directory struct to be dovecot:dovecot it works fine, but this seems unusual. Dovecot is running as root: root 19176 0.0 0.1 2876 1096 ? Ss 14:28 0:00 /usr/local/sbin/dovecot dovecot 19178 0.0 0.0 2780 996 ? S 14:28 0:00 dovecot/anvil root 19179 0.0 0.1 2912 1144 ? S 14:28 0:00 dovecot/log root 19225 0.0 0.1 3544 1776 ? S 14:38 0:00 dovecot/config dovecot 19226 0.0 0.1 3332 1408 ? S 14:38 0:00 dovecot/auth root 19227 0.0 0.0 2776 916 ? S 14:38 0:00 dovecot/ssl-params I can't seem to figure this one out. Can someone point me in the right direction. Thanks. From fila at pruda.com Sat Oct 24 04:14:22 2015 From: fila at pruda.com (fila at pruda.com) Date: Fri, 23 Oct 2015 21:14:22 -0700 Subject: important message Message-ID: <0000994c1fd4$32423abf$f22ac741$@pruda.com> Hello! New message, please read fila at pruda.com From fila at pruda.com Sat Oct 24 04:14:34 2015 From: fila at pruda.com (dovecot) Date: Fri, 23 Oct 2015 21:14:34 -0700 Subject: important message Message-ID: <0000197e7b88$233d0061$23476a96$@pruda.com> Hello! New message, please read dovecot From fila at pruda.com Sat Oct 24 04:14:45 2015 From: fila at pruda.com (dovecot) Date: Fri, 23 Oct 2015 21:14:45 -0700 Subject: important message Message-ID: <000035471bb8$73997cca$b72ed2a1$@pruda.com> Hello! New message, please read dovecot From fila at pruda.com Sat Oct 24 04:15:02 2015 From: fila at pruda.com (dovecot) Date: Fri, 23 Oct 2015 21:15:02 -0700 Subject: important message Message-ID: <00000058d4b7$728804e7$3e70347f$@pruda.com> Hello! New message, please read dovecot From fila at pruda.com Sat Oct 24 04:15:06 2015 From: fila at pruda.com (dovecot) Date: Fri, 23 Oct 2015 21:15:06 -0700 Subject: important message Message-ID: <0000c1910b30$c847782d$fbc0de90$@pruda.com> Hello! New message, please read dovecot From fila at pruda.com Sat Oct 24 04:18:42 2015 From: fila at pruda.com (dovecot) Date: Fri, 23 Oct 2015 21:18:42 -0700 Subject: important message Message-ID: <0000986a05bc$d96414ee$864e8f3b$@pruda.com> Hello! New message, please read dovecot From fila at pruda.com Sat Oct 24 04:18:41 2015 From: fila at pruda.com (dovecot) Date: Fri, 23 Oct 2015 21:18:41 -0700 Subject: important message Message-ID: <0000897ee950$39ea12b5$7fd506fd$@pruda.com> Hello! New message, please read dovecot From fila at pruda.com Sat Oct 24 04:19:02 2015 From: fila at pruda.com (Vojtech Kodera) Date: Fri, 23 Oct 2015 21:19:02 -0700 Subject: important message Message-ID: <00000d6fe089$24a37749$fdd8a262$@pruda.com> Hello! New message, please read Vojtech Kodera From fila at pruda.com Sat Oct 24 04:18:58 2015 From: fila at pruda.com (Vojtech Kodera) Date: Fri, 23 Oct 2015 21:18:58 -0700 Subject: important message Message-ID: <000073a72a7a$0a1800c2$4e10cfb7$@pruda.com> Hello! New message, please read Vojtech Kodera From fila at pruda.com Sat Oct 24 04:20:10 2015 From: fila at pruda.com (fila at pruda.com) Date: Fri, 23 Oct 2015 21:20:10 -0700 Subject: important message Message-ID: <0000f6bc0bef$a76ded98$aeebccb1$@pruda.com> Hello! New message, please read fila at pruda.com From fila at pruda.com Sat Oct 24 04:20:29 2015 From: fila at pruda.com (dovecot) Date: Fri, 23 Oct 2015 21:20:29 -0700 Subject: important message Message-ID: <000038a2e2e8$7c757f84$ccb6c5d8$@pruda.com> Hello! New message, please read dovecot From sebastian at wolfgarten.com Sat Oct 24 11:59:50 2015 From: sebastian at wolfgarten.com (Sebastian Wolfgarten) Date: Sat, 24 Oct 2015 13:59:50 +0200 Subject: Memory leak or misconfiguration: Repeated allocation of very large block Message-ID: <8924437C-8FB3-4FA5-AEDA-2B7F136941E2@wolfgarten.com> Dear all, I am using Dovecot 2.2.19 and I keep on getting the following warnings in my mail.log file on FreeBSD 10: Oct 24 10:45:28 server1 dovecot: imap: Error: GC Warning: Repeated allocation of very large block (appr. size 20480): Oct 24 10:45:28 server1 dovecot: imap: Error: May lead to memory leak and poor performance. I am wondering whether this is a configuration issue (i.e. a mistake in my current config) or whether I am hitting a bug. What?s the best way of figuring this one out? I tried increasing the verbosity of the logs but it did not really provide me with further information. Any ideas? Here is my config: -- [root at server1 ~]# dovecot -n # 2.2.19: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.9 (357ac0a0e68b+) # OS: FreeBSD 10.1-RELEASE amd64 zfs auth_cache_size = 50 M auth_mechanisms = plain login default_vsz_limit = 512 M disable_plaintext_auth = no first_valid_uid = 1008 last_valid_uid = 1008 login_greeting = Ready. mail_gid = 1008 mail_location = maildir:/var/mail/vdomains/%d/%n mail_max_keyword_length = 128 mail_max_userip_connections = 20 mail_uid = 1008 maildir_stat_dirs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = sebastian at wolfgarten.com protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = vmail mode = 0666 user = vmail } user = $default_internal_user } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 3 service_count = 0 vsz_limit = 512 M } service imap-postlogin { executable = script-login /usr/local/etc/popafter.sh user = $default_internal_user } service imap { executable = imap imap-postlogin process_limit = 1024 vsz_limit = 512 M } service lmtp { unix_listener lmtp { mode = 0666 } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service pop3-postlogin { executable = script-login /usr/local/etc/popafter.sh user = $default_internal_user } service pop3 { executable = pop3 pop3-postlogin process_limit = 1024 } ssl_ca = From tss at iki.fi Sat Oct 24 21:13:11 2015 From: tss at iki.fi (Timo Sirainen) Date: Sun, 25 Oct 2015 00:13:11 +0300 Subject: Question about dovecot replication In-Reply-To: <562A2D6B.2020107@enas.net> References: <562A2D6B.2020107@enas.net> Message-ID: <5306FAC1-7F65-4D1C-AE6E-D842AA2E2D19@iki.fi> > On 23 Oct 2015, at 15:51, Urban Loesch wrote: > > Hi, > > last week I installed 2 servers with two dovecot nodes and replication in active/active mode located in different datacenters. > > Based on the howto from wiki "http://wiki2.dovecot.org/Replication" it works great. > According to the wiki it is recomended to run "doveadm purge" on both systems continously, because "doveadm purge" will not be replicated by the > replication service. No problem so far. > > But I have one doubt: > is it also reccmended (for keeping maildata in sync) to run "doveadm replicator replicate '*'" continously on both nodes? > Or is it enough on only one node? > > Or should I run "doveadm sync -A tcp:anotherhost.example.com" in regular intervalls? Perhaps once a day on both nodes? The replicator process iterates through all users periodically (replication_full_sync_interval setting) and does full replication for all of them. So there's no need to run either of those doveadm commands. From tss at iki.fi Sat Oct 24 21:15:48 2015 From: tss at iki.fi (Timo Sirainen) Date: Sun, 25 Oct 2015 00:15:48 +0300 Subject: Notify plugin doesn't send mailbox_create notifications In-Reply-To: <562A7292.4020308@web.de> References: <562A7292.4020308@web.de> Message-ID: <9D41464C-B342-46D7-81FF-E0C91FB24D1C@iki.fi> On 23 Oct 2015, at 20:46, Lukas Mika wrote: > > Hi there, > > I'm running dovecot 2.2.13 on my debian jessie server. > When installing my own dovecot plugin, I noticed that there are no > notifications coming from the notify plugin. > > I checked back with the mail_log plugin, but also this plugin doesn't > log any notifications when a new mailbox is created; the mail is saved > to the new inbox never the less. I don't know about 2.2.13, but at least with 2.2.19 it works: /usr/local/libexec/dovecot/imap -O -o mail_plugins=notify,mail_log -o plugin/mail_log_events=mailbox_create -o mail=maildir:~/Maildir -u tss a create testing imap(tss): Info: Mailbox created: testing > > I ran dovecot 2.2.9 before on my Ubuntu Trusty (14.04) server, where > with the same settings the plugins worked. > > Can you reproduce this issue and does anybody know a fix to this? > > Here are the relevant bits of my config: > 20-lmtp.conf: > protocol lmtp { > mail_plugins). > mail_plugins = $mail_plugins sieve mail_log notify > } > > 10-logging.conf > ... > plugin { > mail_log_events = mailbox_create > mail_log_fields = uid box > } > ... > > Cheers From tss at iki.fi Sat Oct 24 21:19:41 2015 From: tss at iki.fi (Timo Sirainen) Date: Sun, 25 Oct 2015 00:19:41 +0300 Subject: Memory leak or misconfiguration: Repeated allocation of very large block In-Reply-To: <8924437C-8FB3-4FA5-AEDA-2B7F136941E2@wolfgarten.com> References: <8924437C-8FB3-4FA5-AEDA-2B7F136941E2@wolfgarten.com> Message-ID: <619EF5DE-EB27-4069-9ADA-3042A5D7EA5A@iki.fi> On 24 Oct 2015, at 14:59, Sebastian Wolfgarten wrote: > > Dear all, > > I am using Dovecot 2.2.19 and I keep on getting the following warnings in my mail.log file on FreeBSD 10: > > Oct 24 10:45:28 server1 dovecot: imap: Error: GC Warning: Repeated allocation of very large block (appr. size 20480): > Oct 24 10:45:28 server1 dovecot: imap: Error: May lead to memory leak and poor performance. > > I am wondering whether this is a configuration issue (i.e. a mistake in my current config) or whether I am hitting a bug. What?s the best way of figuring this one out? I tried increasing the verbosity of the logs but it did not really provide me with further information. I guess you have configured Dovecot --with-gc? I guess it's possible this is a bug, but it could just as well be a false alarm. In general I don't recommend/support --with-gc option (and I should probably just remove it altogether). The easiest and the most usable way of finding memory leaks is to run with valgrind (which must be done without-gc): service imap { executable = /usr/bin/valgrind -q --show-reachable=yes --leak-check=full /usr/libexec/dovecot/imap } I'm not aware of any memory leaks right now. From sebastian at wolfgarten.com Sun Oct 25 10:54:15 2015 From: sebastian at wolfgarten.com (Sebastian Wolfgarten) Date: Sun, 25 Oct 2015 11:54:15 +0100 Subject: Memory leak or misconfiguration: Repeated allocation of very large block In-Reply-To: <619EF5DE-EB27-4069-9ADA-3042A5D7EA5A@iki.fi> References: <8924437C-8FB3-4FA5-AEDA-2B7F136941E2@wolfgarten.com> <619EF5DE-EB27-4069-9ADA-3042A5D7EA5A@iki.fi> Message-ID: Hi Timo, alright, thanks I will recompile without gc - this is a standard build option on FreeBSD but I will get the port maintainer to maybe change this. Best regards Sebastian > Am 24.10.2015 um 23:19 schrieb Timo Sirainen : > > On 24 Oct 2015, at 14:59, Sebastian Wolfgarten wrote: >> >> Dear all, >> >> I am using Dovecot 2.2.19 and I keep on getting the following warnings in my mail.log file on FreeBSD 10: >> >> Oct 24 10:45:28 server1 dovecot: imap: Error: GC Warning: Repeated allocation of very large block (appr. size 20480): >> Oct 24 10:45:28 server1 dovecot: imap: Error: May lead to memory leak and poor performance. >> >> I am wondering whether this is a configuration issue (i.e. a mistake in my current config) or whether I am hitting a bug. What?s the best way of figuring this one out? I tried increasing the verbosity of the logs but it did not really provide me with further information. > > I guess you have configured Dovecot --with-gc? I guess it's possible this is a bug, but it could just as well be a false alarm. In general I don't recommend/support --with-gc option (and I should probably just remove it altogether). The easiest and the most usable way of finding memory leaks is to run with valgrind (which must be done without-gc): > > service imap { > executable = /usr/bin/valgrind -q --show-reachable=yes --leak-check=full /usr/libexec/dovecot/imap > } > > I'm not aware of any memory leaks right now. > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 842 bytes Desc: Message signed with OpenPGP using GPGMail URL: From j.cubizolles at free.fr Sun Oct 25 13:50:19 2015 From: j.cubizolles at free.fr (Julien Cubizolles) Date: Sun, 25 Oct 2015 14:50:19 +0100 Subject: Can't access maildir with dovecot Message-ID: <877fmbdplw.fsf@free.fr> I'm a total newbie regarding imap servers. I'm trying to set up a Offlineimap/dovecot/Gnus system and Gnus complains that it cannot access the maildir... Here is my setup, put together by copying stuff from around the web without any prior knowledge of these tools... * Offlineimap is setup to use the $HOME/Maildir-free/, using --8<---------------cut here---------------start------------->8--- preauthtunnel = /usr/lib/dovecot/imap -o mail_location=maildir:$HOME/Maildir-free --8<---------------cut here---------------end--------------->8--- in the local maildir part of ~/.offlineimaprc. * The Maildir-free maildir has been correctly (I think) populated with a single "General" folder. /home/wilk/Maildir-free: total used in directory 52 available 5502852 drwxrwxr-x 6 wilk wilk 4096 oct. 23 11:10 . drwx------ 240 wilk wilk 20480 oct. 23 15:06 .. drwxrwxr-x 2 wilk wilk 4096 oct. 23 11:09 cur -rw-rw-r-- 1 wilk wilk 348 oct. 23 11:10 dovecot.index.log -rw-rw-r-- 1 wilk wilk 51 oct. 23 11:10 dovecot-uidlist -rw-rw-r-- 1 wilk wilk 8 oct. 23 11:09 dovecot-uidvalidity -r--r--r-- 1 wilk wilk 0 oct. 23 11:09 dovecot-uidvalidity.5629f967 drwx------ 6 wilk wilk 4096 oct. 23 11:16 General -rw-rw-r-- 1 wilk wilk 0 oct. 23 11:09 maildirfolder drwxrwxr-x 2 wilk wilk 4096 oct. 23 11:09 new drwxrwxr-x 2 wilk wilk 4096 oct. 23 11:09 tmp * dovecot is not setup to start as a server (no script in /etc/rc*.d/) and I didn't configure anything in any /etc/dovecot/ file (maybe I should have ?) * Gnus is supposed to access the maildir as an IMAP server through dovecot: I have: (nnimap "FreeOffline" (nnimap-stream shell) (nnimap-shell-program "/usr/lib/dovecot/imap -o mail_location=maildir:$HOME/Maildir-free") in gnus-secondary-select-methods * I try to access this maildir as follows /usr/lib/dovecot/imap -o mail_location=maildir:/home/wilk/Maildir-free/ * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE] Logged in as wilk * LIST "" * * LIST (\HasNoChildren) "." INBOX * OK List completed. As you can see, the "General" folder doesn't appear, and there is a mention of "INBOX" even though there is no folder under this name (but I guess that's a default for any IMAP system). Accordingly, Gnus only displays a INBOX folder when browsing the FreeOffline IMAP server, and complains that it contains no message. What's the correct way to setup dovecot? Thanks for your help. Julien. From mfoley at ohprs.org Mon Oct 26 05:34:21 2015 From: mfoley at ohprs.org (Mark Foley) Date: Mon, 26 Oct 2015 01:34:21 -0400 Subject: MAPI Properties? Message-ID: <201510260534.t9Q5YLrE030150@mail.hprs.local> I'm using Dovceot/IMAP on Linux and Outlook clients on WIN7 workstations. Mail on Linux is stored in Maildir format. I'm searching for where Outook keeps its information on color categories in IMAP. According to Diane Poremsky at slipstick.com, "Outlook stores it in the mapi properties of each message. If you use MFCMAPI to viuw the messages, you'll see the properties." MAPI is a Windows thing and the recommended MFCMAPI is for viewing these properties in Exchange. Not what I can use. Outlook must be storing these properties somewhere in the Dovecot/IMAP system as color categories can be set from Outlook. Can someone tell me where to look for these properties? THX - Mark From f.bosch at genkgo.nl Mon Oct 26 06:59:52 2015 From: f.bosch at genkgo.nl (Frederik Bosch | Genkgo) Date: Mon, 26 Oct 2015 07:59:52 +0100 Subject: events In-Reply-To: <562A4065.6010706@dovecot.fi> References: <5627AA48.5030809@genkgo.nl> <562804B3.2060405@dovecot.fi> <56288642.2070803@genkgo.nl> <562A4065.6010706@dovecot.fi> Message-ID: <562DCF68.90609@genkgo.nl> Thanks again. Final question: how do I configure this plugin? On 23-10-15 16:12, Michael M Slusarz wrote: > On 10/22/2015 12:46 AM, Frederik Bosch | Genkgo wrote: > >> Thanks a lot! After looking at the source, I guess the ox driver will >> do. Maybe, when other people find this thread, you could tell what dlog >> is. Because I do not know it, and googling came up with little results. > > "dlog" is nothing more than a push-notification backend that will log > various information and hook triggers (at a DEBUG level) to the > Dovecot log. It's meant for debugging and development purposes. > > "dlog" stands for either "Dovecot LOGging" or "Debug LOGging", > whichever you prefer. > > michael > > >> On 21-10-15 23:33, Michael M Slusarz wrote: >>> On 10/21/2015 9:07 AM, Frederik Bosch | Genkgo wrote: >>> >>>> We want to trigger a script after certain actions by the user (event). >>>> This script inserts the action into message queue (e.g. Rabbit MQ) >>>> accompanied with some data. Then one or more workers picks up the >>>> action >>>> from the message queue and do something with it. The question is: how >>>> can I trigger the script from dovecot? >>> >>> This is precisely what the new push-notification plugin is for >>> (2.2.19). (You will need to write a driver to interact with your >>> notification handler, similar to the "dlog" or "ox" drivers.) >>> >>> michael From teemu.huovila at dovecot.fi Mon Oct 26 07:36:37 2015 From: teemu.huovila at dovecot.fi (Teemu Huovila) Date: Mon, 26 Oct 2015 09:36:37 +0200 Subject: events In-Reply-To: <562DCF68.90609@genkgo.nl> References: <5627AA48.5030809@genkgo.nl> <562804B3.2060405@dovecot.fi> <56288642.2070803@genkgo.nl> <562A4065.6010706@dovecot.fi> <562DCF68.90609@genkgo.nl> Message-ID: <562DD805.2020301@dovecot.fi> On 26.10.2015 08:59, Frederik Bosch | Genkgo wrote: > Thanks again. Final question: how do I configure this plugin? As the only existing driver at the moment is the OX one, the plugin is documented in OX wiki at http://oxpedia.org/wiki/index.php?title=AppSuite:OX_Mail#Setup_of_the_Dovecot_Push br, Teemu Huovila > > > > On 23-10-15 16:12, Michael M Slusarz wrote: >> On 10/22/2015 12:46 AM, Frederik Bosch | Genkgo wrote: >> >>> Thanks a lot! After looking at the source, I guess the ox driver will >>> do. Maybe, when other people find this thread, you could tell what dlog >>> is. Because I do not know it, and googling came up with little results. >> >> "dlog" is nothing more than a push-notification backend that will log various information and hook triggers (at a DEBUG level) to the Dovecot log. It's meant for debugging and development purposes. >> >> "dlog" stands for either "Dovecot LOGging" or "Debug LOGging", whichever you prefer. >> >> michael >> >> >>> On 21-10-15 23:33, Michael M Slusarz wrote: >>>> On 10/21/2015 9:07 AM, Frederik Bosch | Genkgo wrote: >>>> >>>>> We want to trigger a script after certain actions by the user (event). >>>>> This script inserts the action into message queue (e.g. Rabbit MQ) >>>>> accompanied with some data. Then one or more workers picks up the action >>>>> from the message queue and do something with it. The question is: how >>>>> can I trigger the script from dovecot? >>>> >>>> This is precisely what the new push-notification plugin is for >>>> (2.2.19). (You will need to write a driver to interact with your >>>> notification handler, similar to the "dlog" or "ox" drivers.) >>>> >>>> michael From f.bosch at genkgo.nl Mon Oct 26 07:45:32 2015 From: f.bosch at genkgo.nl (Frederik Bosch | Genkgo) Date: Mon, 26 Oct 2015 08:45:32 +0100 Subject: events In-Reply-To: <562DD805.2020301@dovecot.fi> References: <5627AA48.5030809@genkgo.nl> <562804B3.2060405@dovecot.fi> <56288642.2070803@genkgo.nl> <562A4065.6010706@dovecot.fi> <562DCF68.90609@genkgo.nl> <562DD805.2020301@dovecot.fi> Message-ID: <562DDA1C.4030801@genkgo.nl> Ah fantastic. Now I guess I can use notify plugin without push_notification metadata plugins, right? On 26-10-15 08:36, Teemu Huovila wrote: > > On 26.10.2015 08:59, Frederik Bosch | Genkgo wrote: >> Thanks again. Final question: how do I configure this plugin? > As the only existing driver at the moment is the OX one, the plugin is documented in OX wiki at > http://oxpedia.org/wiki/index.php?title=AppSuite:OX_Mail#Setup_of_the_Dovecot_Push > > br, > Teemu Huovila > >> >> >> On 23-10-15 16:12, Michael M Slusarz wrote: >>> On 10/22/2015 12:46 AM, Frederik Bosch | Genkgo wrote: >>> >>>> Thanks a lot! After looking at the source, I guess the ox driver will >>>> do. Maybe, when other people find this thread, you could tell what dlog >>>> is. Because I do not know it, and googling came up with little results. >>> "dlog" is nothing more than a push-notification backend that will log various information and hook triggers (at a DEBUG level) to the Dovecot log. It's meant for debugging and development purposes. >>> >>> "dlog" stands for either "Dovecot LOGging" or "Debug LOGging", whichever you prefer. >>> >>> michael >>> >>> >>>> On 21-10-15 23:33, Michael M Slusarz wrote: >>>>> On 10/21/2015 9:07 AM, Frederik Bosch | Genkgo wrote: >>>>> >>>>>> We want to trigger a script after certain actions by the user (event). >>>>>> This script inserts the action into message queue (e.g. Rabbit MQ) >>>>>> accompanied with some data. Then one or more workers picks up the action >>>>>> from the message queue and do something with it. The question is: how >>>>>> can I trigger the script from dovecot? >>>>> This is precisely what the new push-notification plugin is for >>>>> (2.2.19). (You will need to write a driver to interact with your >>>>> notification handler, similar to the "dlog" or "ox" drivers.) >>>>> >>>>> michael From skdovecot at smail.inf.fh-brs.de Mon Oct 26 09:30:19 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 26 Oct 2015 10:30:19 +0100 (CET) Subject: 2.2.18 initial setup permissions problem In-Reply-To: <562A7FC5.5030004@gmail.com> References: <562A7FC5.5030004@gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 23 Oct 2015, Forrest wrote: > I'm using Dovecot 2.2.18, compiled from source, on CentOS 6. > > I'm running into a permissions problem with my first rollout of Dovecot which > has to do with directory permissions: > > Oct 23 14:34:51 mail dovecot: imap(username): Error: > stat(/home/username/Maildir/tmp) failed: Permission denied > (euid=596() egid=97(dovecot) missing +x perm: ^^^^^^^^^^^^^ > /home/username/Maildir, dir owned by 500:500 mode=0700) ^^^^^^^ you have configured your user or Dovecot to use uid 596, but the user's Maildir is owned by uid 500 . > I can't seem to figure this one out. Can someone point me in the right > direction. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVi3yq3z1H7kL/d9rAQKOlQf/SB/eakAhhwJJJHJQ6Lqzkslm4inj8aTg ZA2ldaMn9gn/eckpmEBiPGw1CpXQu1JXFY3LWcvA3GcRZse+5ZY2Ae6v6RvA7JaL vYzalFEdIMkkTDipvvDgtAzUCHGG1g3popZYj3WAWBtJoYUtmBuXX8gFZ1JQj5nC lJ+iV54SgyLEyCJtWbJpQ8NXDvp01HNW7/vdyfD5zXfecyecA72CILFLLYjztTkn 8Ih9ApTTi1AXdeGAXPvgRFa2ojLiJ1pt+gvh15APUDSP3QcHn8WQbaFgjoPWpoVQ 0oynQ1jxvMaRlKJWKBl6AlvrEZwBWkLm+T9y9mSeWvsvin+WjFnvMA== =bF/e -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Mon Oct 26 09:32:05 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 26 Oct 2015 10:32:05 +0100 (CET) Subject: Can't access maildir with dovecot In-Reply-To: <877fmbdplw.fsf@free.fr> References: <877fmbdplw.fsf@free.fr> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sun, 25 Oct 2015, Julien Cubizolles wrote: > Date: Sun, 25 Oct 2015 14:50:19 +0100 > From: Julien Cubizolles > To: dovecot at dovecot.org > Subject: Can't access maildir with dovecot > > I'm a total newbie regarding imap servers. I'm trying to set up a > Offlineimap/dovecot/Gnus system and Gnus complains that it cannot access > the maildir... Here is my setup, put together by copying stuff from > around the web without any prior knowledge of these tools... > > * Offlineimap is setup to use the $HOME/Maildir-free/, using > > --8<---------------cut here---------------start------------->8--- > preauthtunnel = /usr/lib/dovecot/imap -o mail_location=maildir:$HOME/Maildir-free > --8<---------------cut here---------------end--------------->8--- > > in the local maildir part of ~/.offlineimaprc. > > * The Maildir-free maildir has been correctly (I think) populated > with a single "General" folder. > > /home/wilk/Maildir-free: > total used in directory 52 available 5502852 > drwxrwxr-x 6 wilk wilk 4096 oct. 23 11:10 . > drwx------ 240 wilk wilk 20480 oct. 23 15:06 .. > drwxrwxr-x 2 wilk wilk 4096 oct. 23 11:09 cur > -rw-rw-r-- 1 wilk wilk 348 oct. 23 11:10 dovecot.index.log > -rw-rw-r-- 1 wilk wilk 51 oct. 23 11:10 dovecot-uidlist > -rw-rw-r-- 1 wilk wilk 8 oct. 23 11:09 dovecot-uidvalidity > -r--r--r-- 1 wilk wilk 0 oct. 23 11:09 dovecot-uidvalidity.5629f967 > drwx------ 6 wilk wilk 4096 oct. 23 11:16 General Maildir++ prefixes all Mailboxes with ".", hence it should read: .General > -rw-rw-r-- 1 wilk wilk 0 oct. 23 11:09 maildirfolder > drwxrwxr-x 2 wilk wilk 4096 oct. 23 11:09 new > drwxrwxr-x 2 wilk wilk 4096 oct. 23 11:09 tmp > > > * dovecot is not setup to start as a server (no script in /etc/rc*.d/) > and I didn't configure anything in any /etc/dovecot/ file (maybe I > should have ?) > > * Gnus is supposed to access the maildir as an IMAP server through > dovecot: I have: > > (nnimap "FreeOffline" > (nnimap-stream shell) > (nnimap-shell-program "/usr/lib/dovecot/imap -o mail_location=maildir:$HOME/Maildir-free") > > in gnus-secondary-select-methods > > * I try to access this maildir as follows > > /usr/lib/dovecot/imap -o mail_location=maildir:/home/wilk/Maildir-free/ > > * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE] Logged in as wilk > * LIST "" * > * LIST (\HasNoChildren) "." INBOX > * OK List completed. > > As you can see, the "General" folder doesn't appear, and there is a > mention of "INBOX" even though there is no folder under this name (but I > guess that's a default for any IMAP system). Accordingly, Gnus only > displays a INBOX folder when browsing the FreeOffline IMAP server, and > complains that it contains no message. > > What's the correct way to setup dovecot? Thanks for your help. > > Julien. > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVi3zFXz1H7kL/d9rAQJDlgf8D5fXwSY/YgFcFTW7ObWnle4l3LyssjYw M7tDim/Jq0XxphvGjF3QYJki9IibWVgeh6z82tOgOngS3pSWz1HtxnO3d8ghn7Ri KrIBCo5WB9qFqcjjpPzA2POCm9QZyH7/AVtqAPFajM3uInotz00UTZrrp4PtsgDc 6Qb8gaXhuqKACAUMJ8kM/UH7gzwMw+rbJpq3ZW0PLYKBInwtWmN58UURXM1uhyUx S5lzRa3pdLwJhBIX+nx9aeSSqOOZXMqYOOCrCwjrcDtpSBkp9ktKhOFdS9Qlh3xO tJZfrosxUVc7sdYsoThWquAfL+EqezA5bCt0o25UEWL1mVdWpQ1oJw== =AXDG -----END PGP SIGNATURE----- From arjan.wekking at pine.nl Mon Oct 26 09:55:49 2015 From: arjan.wekking at pine.nl (Arjan Wekking) Date: Mon, 26 Oct 2015 10:55:49 +0100 Subject: Haproxy and Dovecot auth In-Reply-To: <561FEFB9.6020807@entermail.it> References: <561FEFB9.6020807@entermail.it> Message-ID: <7BAECB9D-5CA0-4B65-BF4C-6ED3220EB9F1@pine.nl> On 15 Oct 2015, at 20:26, Giuseppe Civitella wrote: > When I redirect the smtpd_sasl_path through an Haproxy balanced > connection, I receive an error the first time I try to sen an email. > The error is: > SASL PLAIN authentication failed: Connection lost to authentication server > If then I send more emails I receive no errors. > If I do not send emails for a few minutes, the error appears again. We?ve seen this happen as well, and concluded that this is caused by the Postfix SASL client not recovering gracefully from lost TCP connections. If I remember correctly, Postfix keeps a SASL authentication connection open for re-use, but when this connection breaks down (e.g. due to a time out in the TCP connection itself or due to Dovecot closing it on the other end due to inactivity) then the next/first authentication attempt fails. Postfix will then not handle this error as a temporary failure, but as an authentication failure (i.e. ?wrong username/password?). After this, it will close the connection and the next authentication attempt will re-establish the connection, and that will of course succeed again. This explains why you get this error only the first time (because it failed due to the old/broken cnnection) and why after a few minutes the error appears again (because by then the connection is stale again). Now, I haven?t actually confirmed this, but I?m pretty sure the problem is in the Dovecot SASL client in Postfix. It is written with the assumption that the connection is over a UNIX socket. In those cases a broken connection is detected earlier/differently (EPIPE) and Postfix will actually recover by reconnecting and trying again. You might be able to confirm and possibly work around this issue by forwarding UNIX socket connections to TCP, with tools like socat, netcat, spiped etc. -Arjan -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 204 bytes Desc: Message signed with OpenPGP using GPGMail URL: From teemu.huovila at dovecot.fi Mon Oct 26 10:35:11 2015 From: teemu.huovila at dovecot.fi (Teemu Huovila) Date: Mon, 26 Oct 2015 12:35:11 +0200 Subject: events In-Reply-To: <562DDA1C.4030801@genkgo.nl> References: <5627AA48.5030809@genkgo.nl> <562804B3.2060405@dovecot.fi> <56288642.2070803@genkgo.nl> <562A4065.6010706@dovecot.fi> <562DCF68.90609@genkgo.nl> <562DD805.2020301@dovecot.fi> <562DDA1C.4030801@genkgo.nl> Message-ID: <562E01DF.9000806@dovecot.fi> On 26.10.2015 09:45, Frederik Bosch | Genkgo wrote: > Ah fantastic. Now I guess I can use notify plugin without push_notification metadata plugins, right? Im not sure I understand the question correctly. I understood from the thread that you would be writing a driver for the push-notification plugin, so you need to load that plugin. In case you mean the imap_metadata = yes setting, you do not need that, if your driver does not use metadata. Teemu > > On 26-10-15 08:36, Teemu Huovila wrote: >> >> On 26.10.2015 08:59, Frederik Bosch | Genkgo wrote: >>> Thanks again. Final question: how do I configure this plugin? >> As the only existing driver at the moment is the OX one, the plugin is documented in OX wiki at >> http://oxpedia.org/wiki/index.php?title=AppSuite:OX_Mail#Setup_of_the_Dovecot_Push >> >> br, >> Teemu Huovila >> >>> >>> >>> On 23-10-15 16:12, Michael M Slusarz wrote: >>>> On 10/22/2015 12:46 AM, Frederik Bosch | Genkgo wrote: >>>> >>>>> Thanks a lot! After looking at the source, I guess the ox driver will >>>>> do. Maybe, when other people find this thread, you could tell what dlog >>>>> is. Because I do not know it, and googling came up with little results. >>>> "dlog" is nothing more than a push-notification backend that will log various information and hook triggers (at a DEBUG level) to the Dovecot log. It's meant for debugging and development purposes. >>>> >>>> "dlog" stands for either "Dovecot LOGging" or "Debug LOGging", whichever you prefer. >>>> >>>> michael >>>> >>>> >>>>> On 21-10-15 23:33, Michael M Slusarz wrote: >>>>>> On 10/21/2015 9:07 AM, Frederik Bosch | Genkgo wrote: >>>>>> >>>>>>> We want to trigger a script after certain actions by the user (event). >>>>>>> This script inserts the action into message queue (e.g. Rabbit MQ) >>>>>>> accompanied with some data. Then one or more workers picks up the action >>>>>>> from the message queue and do something with it. The question is: how >>>>>>> can I trigger the script from dovecot? >>>>>> This is precisely what the new push-notification plugin is for >>>>>> (2.2.19). (You will need to write a driver to interact with your >>>>>> notification handler, similar to the "dlog" or "ox" drivers.) >>>>>> >>>>>> michael From f.bosch at genkgo.nl Mon Oct 26 10:44:06 2015 From: f.bosch at genkgo.nl (Frederik Bosch | Genkgo) Date: Mon, 26 Oct 2015 11:44:06 +0100 Subject: events In-Reply-To: <562E01DF.9000806@dovecot.fi> References: <5627AA48.5030809@genkgo.nl> <562804B3.2060405@dovecot.fi> <56288642.2070803@genkgo.nl> <562A4065.6010706@dovecot.fi> <562DCF68.90609@genkgo.nl> <562DD805.2020301@dovecot.fi> <562DDA1C.4030801@genkgo.nl> <562E01DF.9000806@dovecot.fi> Message-ID: <562E03F6.4010904@genkgo.nl> Teemu, If just need the http request, I will need something like the following configuration, right? So no meta data plugin, but with notify and push_notification? protocol lmtp { mail_plugins = $mail_plugins notify push_notification } plugin { push_notification_driver = ox:url=http://myurl/ } Regards, Frederik On 26-10-15 11:35, Teemu Huovila wrote: > > On 26.10.2015 09:45, Frederik Bosch | Genkgo wrote: >> Ah fantastic. Now I guess I can use notify plugin without push_notification metadata plugins, right? > Im not sure I understand the question correctly. I understood from the thread that you would be writing a driver for the push-notification plugin, so you need to load that plugin. In case you mean the imap_metadata = yes setting, you do not need that, if your driver does not use metadata. > > Teemu > >> On 26-10-15 08:36, Teemu Huovila wrote: >>> On 26.10.2015 08:59, Frederik Bosch | Genkgo wrote: >>>> Thanks again. Final question: how do I configure this plugin? >>> As the only existing driver at the moment is the OX one, the plugin is documented in OX wiki at >>> http://oxpedia.org/wiki/index.php?title=AppSuite:OX_Mail#Setup_of_the_Dovecot_Push >>> >>> br, >>> Teemu Huovila >>> >>>> >>>> On 23-10-15 16:12, Michael M Slusarz wrote: >>>>> On 10/22/2015 12:46 AM, Frederik Bosch | Genkgo wrote: >>>>> >>>>>> Thanks a lot! After looking at the source, I guess the ox driver will >>>>>> do. Maybe, when other people find this thread, you could tell what dlog >>>>>> is. Because I do not know it, and googling came up with little results. >>>>> "dlog" is nothing more than a push-notification backend that will log various information and hook triggers (at a DEBUG level) to the Dovecot log. It's meant for debugging and development purposes. >>>>> >>>>> "dlog" stands for either "Dovecot LOGging" or "Debug LOGging", whichever you prefer. >>>>> >>>>> michael >>>>> >>>>> >>>>>> On 21-10-15 23:33, Michael M Slusarz wrote: >>>>>>> On 10/21/2015 9:07 AM, Frederik Bosch | Genkgo wrote: >>>>>>> >>>>>>>> We want to trigger a script after certain actions by the user (event). >>>>>>>> This script inserts the action into message queue (e.g. Rabbit MQ) >>>>>>>> accompanied with some data. Then one or more workers picks up the action >>>>>>>> from the message queue and do something with it. The question is: how >>>>>>>> can I trigger the script from dovecot? >>>>>>> This is precisely what the new push-notification plugin is for >>>>>>> (2.2.19). (You will need to write a driver to interact with your >>>>>>> notification handler, similar to the "dlog" or "ox" drivers.) >>>>>>> >>>>>>> michael -- *Frederik Bosch* Partner - Genkgo telefoon: +31 (0)20 - 894 39 31 email: f.bosch at genkgo.nl skype: genkgo.support web: www.genkgo.nl *Postadres*: Postbus 15956 1001 NL Amsterdam *Bezoekadres*: Keizersgracht 253 Amsterdam Genkgo logo Genkgo B.V. staat geregistreerd bij de Kamer van Koophandel onder nummer 56501153 From teemu.huovila at dovecot.fi Mon Oct 26 11:04:20 2015 From: teemu.huovila at dovecot.fi (Teemu Huovila) Date: Mon, 26 Oct 2015 13:04:20 +0200 Subject: events In-Reply-To: <562E03F6.4010904@genkgo.nl> References: <5627AA48.5030809@genkgo.nl> <562804B3.2060405@dovecot.fi> <56288642.2070803@genkgo.nl> <562A4065.6010706@dovecot.fi> <562DCF68.90609@genkgo.nl> <562DD805.2020301@dovecot.fi> <562DDA1C.4030801@genkgo.nl> <562E01DF.9000806@dovecot.fi> <562E03F6.4010904@genkgo.nl> Message-ID: <562E08B4.2090405@dovecot.fi> On 26.10.2015 12:44, Frederik Bosch | Genkgo wrote: > Teemu, > > If just need the http request, I will need something like the following configuration, right? So no meta data plugin, but with notify and push_notification? > > protocol lmtp { > mail_plugins = $mail_plugins notify push_notification > } > > plugin { > push_notification_driver = ox:url=http://myurl/ > } You could test that, but my understanding of the ox push driver code is that it completely depends on metadata and will not do anything useful, if no metadata is set. Perhaps Michael can correct me, if Im wrong. If you want some subset of the ox driver functionality, you could try implementing your own driver, based on the existing code. br, Teemu > > > Regards, > Frederik > > > > On 26-10-15 11:35, Teemu Huovila wrote: >> >> On 26.10.2015 09:45, Frederik Bosch | Genkgo wrote: >>> Ah fantastic. Now I guess I can use notify plugin without push_notification metadata plugins, right? >> Im not sure I understand the question correctly. I understood from the thread that you would be writing a driver for the push-notification plugin, so you need to load that plugin. In case you mean the imap_metadata = yes setting, you do not need that, if your driver does not use metadata. >> >> Teemu >> >>> On 26-10-15 08:36, Teemu Huovila wrote: >>>> On 26.10.2015 08:59, Frederik Bosch | Genkgo wrote: >>>>> Thanks again. Final question: how do I configure this plugin? >>>> As the only existing driver at the moment is the OX one, the plugin is documented in OX wiki at >>>> http://oxpedia.org/wiki/index.php?title=AppSuite:OX_Mail#Setup_of_the_Dovecot_Push >>>> >>>> br, >>>> Teemu Huovila >>>> >>>>> >>>>> On 23-10-15 16:12, Michael M Slusarz wrote: >>>>>> On 10/22/2015 12:46 AM, Frederik Bosch | Genkgo wrote: >>>>>> >>>>>>> Thanks a lot! After looking at the source, I guess the ox driver will >>>>>>> do. Maybe, when other people find this thread, you could tell what dlog >>>>>>> is. Because I do not know it, and googling came up with little results. >>>>>> "dlog" is nothing more than a push-notification backend that will log various information and hook triggers (at a DEBUG level) to the Dovecot log. It's meant for debugging and development purposes. >>>>>> >>>>>> "dlog" stands for either "Dovecot LOGging" or "Debug LOGging", whichever you prefer. >>>>>> >>>>>> michael >>>>>> >>>>>> >>>>>>> On 21-10-15 23:33, Michael M Slusarz wrote: >>>>>>>> On 10/21/2015 9:07 AM, Frederik Bosch | Genkgo wrote: >>>>>>>> >>>>>>>>> We want to trigger a script after certain actions by the user (event). >>>>>>>>> This script inserts the action into message queue (e.g. Rabbit MQ) >>>>>>>>> accompanied with some data. Then one or more workers picks up the action >>>>>>>>> from the message queue and do something with it. The question is: how >>>>>>>>> can I trigger the script from dovecot? >>>>>>>> This is precisely what the new push-notification plugin is for >>>>>>>> (2.2.19). (You will need to write a driver to interact with your >>>>>>>> notification handler, similar to the "dlog" or "ox" drivers.) >>>>>>>> >>>>>>>> michael > From f.bosch at genkgo.nl Mon Oct 26 12:54:50 2015 From: f.bosch at genkgo.nl (Frederik Bosch | Genkgo) Date: Mon, 26 Oct 2015 13:54:50 +0100 Subject: events In-Reply-To: <562E08B4.2090405@dovecot.fi> References: <5627AA48.5030809@genkgo.nl> <562804B3.2060405@dovecot.fi> <56288642.2070803@genkgo.nl> <562A4065.6010706@dovecot.fi> <562DCF68.90609@genkgo.nl> <562DD805.2020301@dovecot.fi> <562DDA1C.4030801@genkgo.nl> <562E01DF.9000806@dovecot.fi> <562E03F6.4010904@genkgo.nl> <562E08B4.2090405@dovecot.fi> Message-ID: <562E229A.9020602@genkgo.nl> Teemu, After having another look at the source code, it seems that you are right. Thanks for making this clear. Regards, Frederik On 26-10-15 12:04, Teemu Huovila wrote: > > On 26.10.2015 12:44, Frederik Bosch | Genkgo wrote: >> Teemu, >> >> If just need the http request, I will need something like the following configuration, right? So no meta data plugin, but with notify and push_notification? >> >> protocol lmtp { >> mail_plugins = $mail_plugins notify push_notification >> } >> >> plugin { >> push_notification_driver = ox:url=http://myurl/ >> } > You could test that, but my understanding of the ox push driver code is that it completely depends on metadata and will not do anything useful, if no metadata is set. Perhaps Michael can correct me, if Im wrong. > > If you want some subset of the ox driver functionality, you could try implementing your own driver, based on the existing code. > > br, > Teemu > >> >> Regards, >> Frederik >> >> >> >> On 26-10-15 11:35, Teemu Huovila wrote: >>> On 26.10.2015 09:45, Frederik Bosch | Genkgo wrote: >>>> Ah fantastic. Now I guess I can use notify plugin without push_notification metadata plugins, right? >>> Im not sure I understand the question correctly. I understood from the thread that you would be writing a driver for the push-notification plugin, so you need to load that plugin. In case you mean the imap_metadata = yes setting, you do not need that, if your driver does not use metadata. >>> >>> Teemu >>> >>>> On 26-10-15 08:36, Teemu Huovila wrote: >>>>> On 26.10.2015 08:59, Frederik Bosch | Genkgo wrote: >>>>>> Thanks again. Final question: how do I configure this plugin? >>>>> As the only existing driver at the moment is the OX one, the plugin is documented in OX wiki at >>>>> http://oxpedia.org/wiki/index.php?title=AppSuite:OX_Mail#Setup_of_the_Dovecot_Push >>>>> >>>>> br, >>>>> Teemu Huovila >>>>> >>>>>> On 23-10-15 16:12, Michael M Slusarz wrote: >>>>>>> On 10/22/2015 12:46 AM, Frederik Bosch | Genkgo wrote: >>>>>>> >>>>>>>> Thanks a lot! After looking at the source, I guess the ox driver will >>>>>>>> do. Maybe, when other people find this thread, you could tell what dlog >>>>>>>> is. Because I do not know it, and googling came up with little results. >>>>>>> "dlog" is nothing more than a push-notification backend that will log various information and hook triggers (at a DEBUG level) to the Dovecot log. It's meant for debugging and development purposes. >>>>>>> >>>>>>> "dlog" stands for either "Dovecot LOGging" or "Debug LOGging", whichever you prefer. >>>>>>> >>>>>>> michael >>>>>>> >>>>>>> >>>>>>>> On 21-10-15 23:33, Michael M Slusarz wrote: >>>>>>>>> On 10/21/2015 9:07 AM, Frederik Bosch | Genkgo wrote: >>>>>>>>> >>>>>>>>>> We want to trigger a script after certain actions by the user (event). >>>>>>>>>> This script inserts the action into message queue (e.g. Rabbit MQ) >>>>>>>>>> accompanied with some data. Then one or more workers picks up the action >>>>>>>>>> from the message queue and do something with it. The question is: how >>>>>>>>>> can I trigger the script from dovecot? >>>>>>>>> This is precisely what the new push-notification plugin is for >>>>>>>>> (2.2.19). (You will need to write a driver to interact with your >>>>>>>>> notification handler, similar to the "dlog" or "ox" drivers.) >>>>>>>>> >>>>>>>>> michael From bob at computerisms.ca Mon Oct 26 16:50:23 2015 From: bob at computerisms.ca (Bob Miller) Date: Mon, 26 Oct 2015 09:50:23 -0700 Subject: MAPI Properties? In-Reply-To: <201510260534.t9Q5YLrE030150@mail.hprs.local> References: <201510260534.t9Q5YLrE030150@mail.hprs.local> Message-ID: <562E59CF.30304@computerisms.ca> The last time I investigated that was a couple years ago, but according to my understanding the IMAP protocol does not support the colour categories provided by Outlook, and therefore anything Outlook does in that respect is a "Local Only" feature. That is, the colour categories cannot be stored by an IMAP server, nor shared via the IMAP protocol between multiple Outlook clients... On 15-10-25 10:34 PM, Mark Foley wrote: > I'm using Dovceot/IMAP on Linux and Outlook clients on WIN7 workstations. Mail > on Linux is stored in Maildir format. > > I'm searching for where Outook keeps its information on color categories in > IMAP. According to Diane Poremsky at slipstick.com, "Outlook stores it in the > mapi properties of each message. If you use MFCMAPI to viuw the messages, you'll > see the properties." MAPI is a Windows thing and the recommended MFCMAPI is for > viewing these properties in Exchange. Not what I can use. > > Outlook must be storing these properties somewhere in the Dovecot/IMAP system as > color categories can be set from Outlook. Can someone tell me where to look for > these properties? > > THX - Mark > From j.cubizolles at free.fr Mon Oct 26 21:42:19 2015 From: j.cubizolles at free.fr (Julien Cubizolles) Date: Mon, 26 Oct 2015 22:42:19 +0100 Subject: Can't access maildir with dovecot In-Reply-To: (Steffen Kaiser's message of "Mon, 26 Oct 2015 10:32:05 +0100 (CET)") References: <877fmbdplw.fsf@free.fr> Message-ID: <87d1w15mtg.fsf@free.fr> Steffen Kaiser writes: > On Sun, 25 Oct 2015, Julien Cubizolles wrote: > >> Date: Sun, 25 Oct 2015 14:50:19 +0100 >> From: Julien Cubizolles >> To: dovecot at dovecot.org >> Subject: Can't access maildir with dovecot >> >> I'm a total newbie regarding imap servers. I'm trying to set up a >> Offlineimap/dovecot/Gnus system and Gnus complains that it cannot access >> the maildir... Here is my setup, put together by copying stuff from >> around the web without any prior knowledge of these tools... >> >> * Offlineimap is setup to use the $HOME/Maildir-free/, using >> >> --8<---------------cut here---------------start------------->8--- >> preauthtunnel = /usr/lib/dovecot/imap -o mail_location=maildir:$HOME/Maildir-free >> --8<---------------cut here---------------end--------------->8--- >> >> in the local maildir part of ~/.offlineimaprc. >> >> * The Maildir-free maildir has been correctly (I think) populated >> with a single "General" folder. >> >> /home/wilk/Maildir-free: >> total used in directory 52 available 5502852 >> drwxrwxr-x 6 wilk wilk 4096 oct. 23 11:10 . >> drwx------ 240 wilk wilk 20480 oct. 23 15:06 .. >> drwxrwxr-x 2 wilk wilk 4096 oct. 23 11:09 cur >> -rw-rw-r-- 1 wilk wilk 348 oct. 23 11:10 dovecot.index.log >> -rw-rw-r-- 1 wilk wilk 51 oct. 23 11:10 dovecot-uidlist >> -rw-rw-r-- 1 wilk wilk 8 oct. 23 11:09 dovecot-uidvalidity >> -r--r--r-- 1 wilk wilk 0 oct. 23 11:09 dovecot-uidvalidity.5629f967 >> drwx------ 6 wilk wilk 4096 oct. 23 11:16 General > > Maildir++ prefixes all Mailboxes with ".", hence it should read: .General It should be OfflineIMAP's job to name it that way, right?? I tried adding "sep = ." to the Remote Repository but it seems it only affects the subfolders, not the first-level one. Also I naively renamed the General folder to .General, and dovecot still can't access it. Thanks for your help. From those.lists at gmail.com Mon Oct 26 22:18:01 2015 From: those.lists at gmail.com (Forrest) Date: Mon, 26 Oct 2015 18:18:01 -0400 Subject: 2.2.18 initial setup permissions problem In-Reply-To: References: <562A7FC5.5030004@gmail.com> Message-ID: <562EA699.1010806@gmail.com> On 10/26/15 5:30 AM, Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Fri, 23 Oct 2015, Forrest wrote: > >> I'm using Dovecot 2.2.18, compiled from source, on CentOS 6. >> >> I'm running into a permissions problem with my first rollout of >> Dovecot which has to do with directory permissions: >> >> Oct 23 14:34:51 mail dovecot: imap(username): Error: >> stat(/home/username/Maildir/tmp) failed: Permission denied >> (euid=596() egid=97(dovecot) missing +x perm: > ^^^^^^^^^^^^^ >> /home/username/Maildir, dir owned by 500:500 mode=0700) > ^^^^^^^ > > you have configured your user or Dovecot to use uid 596, but the > user's Maildir is owned by uid 500 . > >> I can't seem to figure this one out. Can someone point me in the >> right direction. > Interesting, in our setup at work, the Maildir is owned by the user, not dovecot. So why does that work and the above does not? Is it a setting? I can grep through the configs -- just not sure what to look for here. I didn't see anything in the dovecot.conf, though. Thanks. From skdovecot at smail.inf.fh-brs.de Tue Oct 27 07:21:18 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Tue, 27 Oct 2015 08:21:18 +0100 (CET) Subject: Can't access maildir with dovecot In-Reply-To: <87d1w15mtg.fsf@free.fr> References: <877fmbdplw.fsf@free.fr> <87d1w15mtg.fsf@free.fr> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 26 Oct 2015, Julien Cubizolles wrote: > Steffen Kaiser writes: > >> On Sun, 25 Oct 2015, Julien Cubizolles wrote: >> >>> Date: Sun, 25 Oct 2015 14:50:19 +0100 >>> From: Julien Cubizolles >>> To: dovecot at dovecot.org >>> Subject: Can't access maildir with dovecot >>> >>> I'm a total newbie regarding imap servers. I'm trying to set up a >>> Offlineimap/dovecot/Gnus system and Gnus complains that it cannot access >>> the maildir... Here is my setup, put together by copying stuff from >>> around the web without any prior knowledge of these tools... >>> >>> * Offlineimap is setup to use the $HOME/Maildir-free/, using >>> >>> --8<---------------cut here---------------start------------->8--- >>> preauthtunnel = /usr/lib/dovecot/imap -o mail_location=maildir:$HOME/Maildir-free >>> --8<---------------cut here---------------end--------------->8--- >>> >>> in the local maildir part of ~/.offlineimaprc. >>> >>> * The Maildir-free maildir has been correctly (I think) populated >>> with a single "General" folder. >>> >>> /home/wilk/Maildir-free: >>> total used in directory 52 available 5502852 >>> drwxrwxr-x 6 wilk wilk 4096 oct. 23 11:10 . >>> drwx------ 240 wilk wilk 20480 oct. 23 15:06 .. >>> drwxrwxr-x 2 wilk wilk 4096 oct. 23 11:09 cur >>> -rw-rw-r-- 1 wilk wilk 348 oct. 23 11:10 dovecot.index.log >>> -rw-rw-r-- 1 wilk wilk 51 oct. 23 11:10 dovecot-uidlist >>> -rw-rw-r-- 1 wilk wilk 8 oct. 23 11:09 dovecot-uidvalidity >>> -r--r--r-- 1 wilk wilk 0 oct. 23 11:09 dovecot-uidvalidity.5629f967 >>> drwx------ 6 wilk wilk 4096 oct. 23 11:16 General >> >> Maildir++ prefixes all Mailboxes with ".", hence it should read: .General > > It should be OfflineIMAP's job to name it that way, right?? I tried > adding "sep = ." to the Remote Repository but it seems it only affects > the subfolders, not the first-level one. > > Also I naively renamed the General folder to .General, and dovecot still > can't access it. Add: -o mail_debug=yes run this command as the user "wilk" /usr/lib/dovecot/imap -o mail_location=maildir:/home/wilk/Maildir-free/ \ -o mail_debug=yes and check out the debug output. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVi8l7nz1H7kL/d9rAQLrjQgAvXe4EnsuH+qEISCdNS/Up741jQZuzcpu +v+8b+YcR5zlyCqOCbnVg9ckbmDQ2xse4BpMJFTq57nh/kNw6jujppqQQyC56Rw8 2TNaJu/+x0NRc4oCHNbOw4D4o+OZD6Cki5LYeYVZewJRFnGsaZAvKuaUsVNA4oir SyDkYaVVsR6jlXeVq22zvadve+JRiwN7h4PWI5nfa2agbiVGC2cJndK4MzyOg5re 4o4aVzKU44RTJ6Xx5vMdH5Gi/86DR9aNVAZ7SkfWPPcwFjJ2lU43inJTLJL0BmDS Hr3AJZtoV7D7sKfu8NU8JOsy9TsOZNIeRGcpHxVb8ut5HV775Uk0Ow== =TC5l -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Tue Oct 27 07:26:28 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Tue, 27 Oct 2015 08:26:28 +0100 (CET) Subject: 2.2.18 initial setup permissions problem In-Reply-To: <562EA699.1010806@gmail.com> References: <562A7FC5.5030004@gmail.com> <562EA699.1010806@gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 26 Oct 2015, Forrest wrote: > On 10/26/15 5:30 AM, Steffen Kaiser wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Fri, 23 Oct 2015, Forrest wrote: >> >>> I'm using Dovecot 2.2.18, compiled from source, on CentOS 6. >>> >>> I'm running into a permissions problem with my first rollout of Dovecot >>> which has to do with directory permissions: >>> >>> Oct 23 14:34:51 mail dovecot: imap(username): Error: >>> stat(/home/username/Maildir/tmp) failed: Permission denied >>> (euid=596() egid=97(dovecot) missing +x perm: >> ^^^^^^^^^^^^^ >>> /home/username/Maildir, dir owned by 500:500 mode=0700) >> ^^^^^^^ >> >> you have configured your user or Dovecot to use uid 596, but the user's >> Maildir is owned by uid 500 . >> >>> I can't seem to figure this one out. Can someone point me in the right >>> direction. >> > > Interesting, in our setup at work, the Maildir is owned by the user, not > dovecot. So why does that work and the above does not? Is it a setting? At work your userdb is configured to return system users: http://wiki2.dovecot.org/SystemUsers by returning uid/gid per user, see http://wiki2.dovecot.org/UserDatabase > I can grep through the configs -- just not sure what to look for here. I > didn't see anything in the dovecot.conf, though. At home you seem to have a static global mail_uid/_gid or static userdb or do not return no uid/gid per user. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVi8nJHz1H7kL/d9rAQJPJQgAvTeoq707orJt68D0hkSGlx4DCePLPR10 KRaIfcZRGkU1G7ILkAn8s3oH6NPwUc8GoG80JYWLEu9pWbOlLDjRYeie4WaSvdhG hfuQabSBHpFD/otdxi+qSM3wBXfMP8BYNWrtjJ7HG06sbnTDBMo+pZiQizkLMnj0 lBgGk4FDsh1jeMQuDyeE3xNTxvHmSdUwDNhmeDgti6IuyEx6iU8O3MjwTQVj/GMK hdtqDnfIStfp1Qgou6j48t3Nl/OvRmskJmMNjRSrC0EMUN/IJq+waV3sxGTgxlFw 6kRksx3zPt7pIaUpI3lFIZVWiR2HeDrsT9OdlgxS6lqh2vVXe14tdQ== =f78e -----END PGP SIGNATURE----- From j.cubizolles at free.fr Tue Oct 27 08:13:46 2015 From: j.cubizolles at free.fr (Julien Cubizolles) Date: Tue, 27 Oct 2015 09:13:46 +0100 Subject: Can't access maildir with dovecot In-Reply-To: (Steffen Kaiser's message of "Tue, 27 Oct 2015 08:21:18 +0100 (CET)") References: <877fmbdplw.fsf@free.fr> <87d1w15mtg.fsf@free.fr> Message-ID: <87eggg91ad.fsf@free.fr> Steffen Kaiser writes: > Add: -o mail_debug=yes > > run this command as the user "wilk" > > /usr/lib/dovecot/imap -o mail_location=maildir:/home/wilk/Maildir-free/ \ > -o mail_debug=yes > > and check out the debug output. Here is what I got: Debug: Effective uid=1000, gid=1000, home=/home/wilk Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/wilk/Maildir-free Debug: maildir++: root=/home/wilk/Maildir-free, index=, indexpvt=, control=, inbox=/home/wilk/Maildir-free, alt= But I also tried something different: I put a --8<---------------cut here---------------start------------->8--- preauthtunnel = /usr/lib/dovecot/imap -c ~/.dovecotrc --8<---------------cut here---------------end--------------->8--- line in .offlineimaprc and ~/.dovecotrc --8<---------------cut here---------------start------------->8--- protocols = imap mail_location = maildir:~/Maildir-free auth default { } --8<---------------cut here---------------end--------------->8--- And now it's working, also there are no differences in the debug logs --8<---------------cut here---------------start------------->8--- Debug: Effective uid=1000, gid=1000, home=/home/wilk Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/wilk/Maildir-free Debug: maildir++: root=/home/wilk/Maildir-free, index=, indexpvt=, control=, inbox=/home/wilk/Maildir-free, alt= --8<---------------cut here---------------end--------------->8--- Thanks for your help. Julien. From vitalif at yourcmc.ru Tue Oct 27 10:24:39 2015 From: vitalif at yourcmc.ru (vitalif at yourcmc.ru) Date: Tue, 27 Oct 2015 13:24:39 +0300 Subject: Dovecot frequently full-text reindexes the whole maildir Message-ID: <7ace0f02e5fc44b337e9b12595d7434b@yourcmc.ru> Hi! I'm using Dovecot 2.2.13 (Debian Jessie package 1:2.2.13-12~deb8u1) on my personal mail server (the address I'm writing from is on this server). I use Maildirs, I have fts + fts_squat enabled, and I have a problem with it for a long time - dovecot seems to not update the index always "incrementally". Yesterday I've finally made a test by telneting to imap port and issuing a search request. Dovecot started to reindex the mailbox. Full reindexing has taken around 15-20 minutes. After reindexing the fulltext searches were fast, just as they should be. And they were fast yesterday even after some new emails arrived. But I've tried to do a new search today and Dovecot started to reindex the whole maildir again! Why is it reindexing the whole maildir again? Could it be related to my anti-spam scripts removing messages from SPAM imap folder? And if yes, how to remove them correctly? -- With best regards, Vitaliy Filippov From jost+lists at dimejo.at Tue Oct 27 10:28:19 2015 From: jost+lists at dimejo.at (Alex JOST) Date: Tue, 27 Oct 2015 11:28:19 +0100 Subject: MAPI Properties? In-Reply-To: <201510260534.t9Q5YLrE030150@mail.hprs.local> References: <201510260534.t9Q5YLrE030150@mail.hprs.local> Message-ID: <562F51C3.4000000@dimejo.at> Am 26.10.2015 um 06:34 schrieb Mark Foley: > I'm using Dovceot/IMAP on Linux and Outlook clients on WIN7 workstations. Mail > on Linux is stored in Maildir format. > > I'm searching for where Outook keeps its information on color categories in > IMAP. According to Diane Poremsky at slipstick.com, "Outlook stores it in the > mapi properties of each message. If you use MFCMAPI to viuw the messages, you'll > see the properties." MAPI is a Windows thing and the recommended MFCMAPI is for > viewing these properties in Exchange. Not what I can use. > > Outlook must be storing these properties somewhere in the Dovecot/IMAP system as > color categories can be set from Outlook. Can someone tell me where to look for > these properties? The keywords are stored in the filename. See the Wiki[1] for more information about it. I'm personally not using Outlook and therefor can't tell for sure, but from what I've read on this topic Outlook understands IMAP keywords and is syncing them to the IMAP server. [1] http://wiki2.dovecot.org/MailboxFormat/Maildir -- Alex JOST From dovecot at mtfbwy.cz Tue Oct 27 11:22:33 2015 From: dovecot at mtfbwy.cz (=?UTF-8?Q?David_Cig=c3=a1nek?=) Date: Tue, 27 Oct 2015 12:22:33 +0100 Subject: How to use different SASL mechanisms for ssl connections Message-ID: <562F5E79.1040403@mtfbwy.cz> Hi I would like to set different SASL mechanisms for users trying to login (POP3/IMAP/Managesieve) on encrypted channel, than for those on unencrypted connection. Specificaly: For users on unencrypted channel, i need to allow login with CRAM-MD5 (to improve security - I have users who require unencrypted connection but i cant let tem use PLAIN login, of course). But for users on TLS/STARTTLS, there is no need to encrypt the password and I want to store their passwords more securely - crypted with SHA512 - but that prohibits the usage of CRAM-MD5 for login in. I know how to handle the passdb/userdb lookups but i need to prevent the POP3/IMAP server from announcing the CRAM-MD5 SASL method on encrypted channel. Because if the server announces the CRAM-MD5 method, MUAs will preffer to use that and for most users, this method is unsupported so they will get an error. Also some MUAs seems to change SASL method from PLAIN to CRAM-MD5 on their own (according to my logs - after enabling the CRAM-MD5, many MUAs started to login with that, even though they have logged in with LOGIN/PLAIN up until then). I need the same that can be achieved in Posfix (for authenticated sending via SMTP) with those settings: smtpd_sasl_security_options = noanonymous, noplaintext smtpd_sasl_tls_security_options = noanonymous, nodictionary Is there any way how to achieve this behavior in Dovecot? I'm using Dovecot ver. 2.2.16 (from source) on Centos 7. Since I have already done some research, i believe there is currently no way how to achieve this behavior. Now I'm looking at the source code here: src/login-common/sasl-server.c sasl_server_get_advertised_mechs() and I think it shouldn't be too hard for me to implement a little ad-hoc change to get this working. So I'd like to ask you if I'm on the righ way with that and if there is anyone out there who could help me with that, I would realy appreciate it. Thank you very much. David. From f-u.s at gmx.net Tue Oct 27 11:08:52 2015 From: f-u.s at gmx.net (Frank-Ulrich Sommer) Date: Tue, 27 Oct 2015 12:08:52 +0100 Subject: Dovecot frequently full-text reindexes the whole maildir In-Reply-To: <7ace0f02e5fc44b337e9b12595d7434b@yourcmc.ru> References: <7ace0f02e5fc44b337e9b12595d7434b@yourcmc.ru> Message-ID: Hi, I'm not an expert, but am doing something similar (moving messages and training the spam filter). To detect the existence of new messages looking directly at the maildir can be a quick and dirty solution to get a trigger event. But to move or delete I think you should use 'doveadm' to search for all messages in your spam directory and get a list of all uids an guids. Then you can call doveadm again to move vor delete the messages by id. Just look for "doveadm search/move/purge" in the dovecot wiki. This should ensure that the indexes stay up to date. Am 27. Oktober 2015 11:24:39 MEZ, schrieb vitalif at yourcmc.ru: >Hi! > >I'm using Dovecot 2.2.13 (Debian Jessie package 1:2.2.13-12~deb8u1) on >my personal mail server (the address I'm writing from is on this >server). > >I use Maildirs, I have fts + fts_squat enabled, and I have a problem >with it for a long time - dovecot seems to not update the index always >"incrementally". > >Yesterday I've finally made a test by telneting to imap port and >issuing >a search request. Dovecot started to reindex the mailbox. Full >reindexing has taken around 15-20 minutes. > >After reindexing the fulltext searches were fast, just as they should >be. And they were fast yesterday even after some new emails arrived. > >But I've tried to do a new search today and Dovecot started to reindex >the whole maildir again! > >Why is it reindexing the whole maildir again? Could it be related to my > >anti-spam scripts removing messages from SPAM imap folder? And if yes, >how to remove them correctly? > >-- >With best regards, > Vitaliy Filippov -- Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet. From mysql.jorge at decimal.pt Tue Oct 27 11:38:36 2015 From: mysql.jorge at decimal.pt (Jan-Frode Myklebust) Date: Tue, 27 Oct 2015 04:38:36 -0700 Subject: Fw: new message Message-ID: <000008664fc4$8db40a03$a9778c21$@decimal.pt> Hey! New message, please read Jan-Frode Myklebust From mysql.jorge at decimal.pt Tue Oct 27 11:45:07 2015 From: mysql.jorge at decimal.pt (Daniel Kerr) Date: Tue, 27 Oct 2015 04:45:07 -0700 Subject: Fw: new message Message-ID: <000027e317f3$2b347a3d$a96a5f67$@decimal.pt> Hey! New message, please read Daniel Kerr From mysql.jorge at decimal.pt Tue Oct 27 11:45:21 2015 From: mysql.jorge at decimal.pt (Gary K) Date: Tue, 27 Oct 2015 04:45:21 -0700 Subject: Fw: new message Message-ID: <0000addb590c$f967247d$fdf147b4$@decimal.pt> Hey! New message, please read Gary K From mysql.jorge at decimal.pt Tue Oct 27 11:45:21 2015 From: mysql.jorge at decimal.pt (Nick Howitt) Date: Tue, 27 Oct 2015 04:45:21 -0700 Subject: Fw: new message Message-ID: <00008d752c1c$2e39eb53$d886aaef$@decimal.pt> Hey! New message, please read Nick Howitt From mehrtens at decoit.de Tue Oct 27 12:26:17 2015 From: mehrtens at decoit.de (=?utf-8?Q?Frank_Mehrtens?=) Date: Tue, 27 Oct 2015 13:26:17 +0100 Subject: Dovecot 2.2.18.2: Panic: file fs-api.c: line 756 (fs_copy): assertion failed: (src->fs == dest->fs) Message-ID: Hello, a colleague found some strange messages in the logs: Oct 27 13:08:43 mail postfix/qmgr[1618]: E83C8180F56: from=, size=298937, nrcpt=2 (queue active) Oct 27 13:08:43 mail dovecot: lmtp(6740): Connect from local Oct 27 13:08:43 mail dovecot: lmtp(user2 at domain.de): gYFbBEdpL1ZUGgAA0nA0dQ: message-id=<035901d110b0$3929bc20$ab7d3460$@domain.de>: saved mail to INBOX (from: user1 at domain.de, subject: WG: Matthias, size: 295392) Oct 27 13:08:43 mail postfix/lmtp[6702]: E83C8180F56: to=, relay=mail.domain.de[private/dovecot-lmtp], delay=0.41, delays=0.36/0/0/0.05, dsn=2.0.0, status=sent (250 2.0.0 gYFbBEdpL1ZUGgAA0nA0dQ Saved) Oct 27 13:08:43 mail dovecot: lmtp(mailarchiv at domain.de): Panic: file fs-api.c: line 756 (fs_copy): assertion failed: (src->fs == dest->fs) Oct 27 13:08:43 mail dovecot: lmtp(mailarchiv at domain.de): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x7b53a) [0x7f10419e553a] -> /usr/lib64/dovecot/libdovecot.so.0(+0x7b5a6) [0x7f10419e55a6] -> /usr/lib64/dovecot/libdovecot.so.0(+0x281ea) [0x7f10419921ea] -> /usr/lib64/dovecot/libdovecot.so.0(+0x3163a) [0x7f104199b63a] -> /usr/lib64/dovecot/libdovecot-storage.so.0(sdbox_copy+0x4c4) [0x7f1041cbad94] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_copy+0x7a) [0x7f1041c9f37a] -> /usr/lib64/dovecot/libdovecot-lda.so.0(mail_deliver_save+0x1a7) [0x7f1041f891b7] -> /usr/lib64/dovecot/libdovecot-lda.so.0(mail_deliver+0x11b) [0x7f1041f8956b] -> dovecot/lmtp() [0x406922] -> dovecot/lmtp() [0x40730d] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7f10419f7fbb] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xaf) [0x7f10419f918f] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f10419f8049] -> /usr/lib64/dovecot/libdovec ot.so.0(io_loop_run+0x38) [0x7f10419f80c8] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f1041997043] -> dovecot/lmtp(main+0x18b) [0x404dbb] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f10415f4d5d] -> dovecot/lmtp() [0x404b49] Oct 27 13:08:43 mail dovecot: lmtp(mailarchiv at domain.de): Fatal: master: service(lmtp): child 6740 killed with signal 6 (core dumps disabled) Oct 27 13:08:43 mail postfix/lmtp[6702]: E83C8180F56: to=, relay=mail.domain.de[private/dovecot-lmtp], delay=0.41, delays=0.36/0/0/0.05, dsn=4.4.2, status=deferred (lost connection with mail.domain.de[private/dovecot-lmtp] while sending end of data -- message may be sent more than once) Postfix always_bcc set to mailarchiv at domain.de. [root at mail ~]# rpm -qa | grep dove dovecot-ee-2.2.18.2-1.el6.x86_64 dovecot-ee-managesieve-0.4.8-1.el6.x86_64 dovecot-ee-pigeonhole-0.4.8-1.el6.x86_64 [root at mail ~]# doveconf -n # 2.2.18.2 (866bffbafde7): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.8 (73d4f6cd676a) # OS: Linux 2.6.32-573.7.1.el6.x86_64 x86_64 CentOS release 6.7 (Final) ext4 auth_cache_negative_ttl = 0 auth_mechanisms = plain login deliver_log_format = message-id=%m: %$ (from: %f, subject: %s, size: %p) dict { ? quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no imap_client_workarounds = tb-extra-mailbox-sep mail_attachment_dir = /storage/mail/attachments mail_attachment_hash = %{sha512} mail_gid = vmail mail_location = sdbox:/storage/mail/%d/%n mail_plugins = acl quota mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate namespace { ? list = children ? location = sdbox:/storage/mail/%%d/%%n/ ? prefix = shared/%%u/ ? separator = / ? subscriptions = no ? type = shared } namespace { ? list = children ? location = maildir:/storage/mail/public:INDEX=/storage/mail/%d/%n/public ? prefix = public/ ? separator = / ? subscriptions = no ? type = public } namespace inbox { ? inbox = yes ? location = ? mailbox Drafts { ??? auto = subscribe ??? special_use = \Drafts ? } ? mailbox "Gel?schte Objekte" { ??? special_use = \Trash ? } ? mailbox "Gesendete Objekte" { ??? special_use = \Sent ? } ? mailbox Junk { ??? special_use = \Junk ? } ? mailbox Sent { ??? special_use = \Sent ? } ? mailbox "Sent Messages" { ??? special_use = \Sent ? } ? mailbox Trash { ??? auto = subscribe ??? special_use = \Trash ? } ? prefix = ? subscriptions = yes ? type = private } passdb { ? args = /etc/dovecot/dovecot-sql.conf.ext ? driver = sql } plugin { ? acl = vfile ? acl_shared_dict = file:/storage/mail/shared-mailboxes ? sieve = ~/.dovecot.sieve ? sieve_dir = ~/sieve } protocols = imap lmtp sieve service auth { ? unix_listener /var/spool/postfix/private/auth { ??? group = postfix ??? mode = 0666 ??? user = postfix ? } } service dict { ? unix_listener dict { ??? group = vmail ??? mode = 0660 ??? user = vmail ? } } service imap-login { ? inet_listener imaps { ??? ssl = yes ? } } service lmtp { ? unix_listener /var/spool/postfix/private/dovecot-lmtp { ??? group = postfix ??? mode = 0660 ??? user = postfix ? } } service pop3-login { ? inet_listener pop3s { ??? ssl = yes ? } } ssl_cert = Web: http://www.decoit.de Amtsgericht Bremen, HRB 21533 Gesch?ftsf?hrer: Prof. Dr.-Ing. Kai-Oliver Detken From bruce+dovecot at bmts.us Tue Oct 27 14:15:16 2015 From: bruce+dovecot at bmts.us (Bruce Marriner) Date: Tue, 27 Oct 2015 14:15:16 +0000 Subject: =?utf-8?q?Migrating_to_new_server=2C_questions_about_recommended_configuration.?= Message-ID: <00000150a9a72c14-8be49dfa-fabb-4f7e-bba4-52fb5ef03567-000000@us-west-2.amazonses.com> Hello! I'm moving Dovecot from a FreeBSD Jail (on ZFS using Maildir) to an Amazon EC2 Debian host. 1) I was on Dovecot 2.1 on my FreeBSD system but upgraded it to 2.2 so I could use dsync for the transfers. Is dsync the best way to migrate my data? 2) On the new Debian host, is there a better filesystem than EXT4 that I should use for the mail storage? 3) On the new host, is mdbox generally recommended over maildir? Especially when mixed with dsync? 4) Does dsync work as a good backup tool to keep a mirror of my live system? 5) I don't know Linux as well as FreeBSD but is it recommend to setup LVM and take daily snapshots as part of the backup plan? Do LVM snapshots cause any problems with Dovecot in the case of a rollback/restore? From infoomatic at gmx.at Tue Oct 27 14:54:21 2015 From: infoomatic at gmx.at (Infoo Matic) Date: Tue, 27 Oct 2015 15:54:21 +0100 Subject: =?utf-8?q?Re=3A_Migrating_to_new_server=2C_questions_about_recommended_configuration=2E?= In-Reply-To: <00000150a9a72c14-8be49dfa-fabb-4f7e-bba4-52fb5ef03567-000000@us-west-2.amazonses.com> Message-ID: <4d9d-562f9000-1-a9072c0@76889282> Hi, I can only write about my experience: > 2) On the new Debian host, is there a better filesystem than EXT4 that I should use for the mail storage? I have used ext4 and never had problems with it for ages. Since recent times I also use ZFS on linux, but it's a bit slower and eats lots of RAM as you probably know > 3) On the new host, is mdbox generally recommended over maildir? Especially when mixed with dsync? Generally yes, brings speed a little speed boost, however administration is not as simple as with maildir (no systemtools like cp, rsync etc; you have to use dovecots tools); don't know in combination with dsync since I have not used it except for backup. > 4) Does dsync work as a good backup tool to keep a mirror of my live system? yes, so far I have not had any troubles > 5) I don't know Linux as well as FreeBSD but is it recommend to setup LVM and take daily snapshots as part of the backup plan? Do LVM snapshots cause any problems with Dovecot in the case of a rollback/restore? Generally, LVM and snapshots are nice. However, We had some troubles with it: having snapshots will have a significant impact on your performance; we have also encountered problems with rollbacks where removing snapshots got stuck; this was one of the reasons why we switched to ZFS (or: we are in process of switching) and just for info: my systems are all quite small (<1000 users). regards, infoomatic From f0andrey at gmail.com Tue Oct 27 15:43:27 2015 From: f0andrey at gmail.com (Andrey Fesenko) Date: Tue, 27 Oct 2015 18:43:27 +0300 Subject: Proxy with director accept only plain login Message-ID: Hello, i'm test system dovecot (proxy with director) and backend storage, auth LDAP server (user plain passwords) If i use plain auth, work fine. If connect DIGEST-MD5 or CRAM-MD5 proxy not redirect connection (Requested DIGEST-MD5 scheme, but we have a NULL password) ### Frontend proxy+director # dovecot -n # 2.2.19: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 10.2-RELEASE amd64 auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login digest-md5 cram-md5 auth_username_format = %Ln auth_verbose = yes base_dir = /var/run/dovecot-proxy/ director_doveadm_port = 12347 director_mail_servers = 192.168.1.3 director_servers = 192.168.1.2 disable_plaintext_auth = no doveadm_port = 12347 first_valid_gid = 0 first_valid_uid = 1000 instance_name = proxy last_valid_gid = 6000 last_valid_uid = 6000 listen = 192.168.1.2 lmtp_proxy = yes mail_location = mbox:~/:INBOX=/var/mail/%u passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf driver = ldap } passdb { args = scheme=SSHA /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { driver = pam } protocols = imap lmtp service director { fifo_listener login/proxy-notify { mode = 0600 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service doveadm { inet_listener { port = 12347 } } service imap-login { executable = imap-login director } service ipc { unix_listener ipc { user = dovecot } } service lmtp { client_limit = 5 executable = lmtp idle_kill = 0 inet_listener lmtp { address = 192.168.1.2 port = 2003 } process_limit = 0 process_min_avail = 0 protocol = lmtp service_count = 0 } ssl_cert = , method=PLAIN, rip=192.168.1.3, lip=192.168.1.2, session= Oct 27 18:15:56 fbsd10 dovecot: imap-login: proxy(usertest): disconnecting 192.168.1.3 (Disconnected by server): user=, method=PLAIN, rip=192.168.1.3, lip=192.168.1.2, session= Oct 27 18:16:30 fbsd10 dovecot: auth: ldap(usertest,192.168.1.3,): Requested DIGEST-MD5 scheme, but we have a NULL password Oct 27 18:16:36 fbsd10 dovecot: imap-login: Aborted login (auth failed, 1 attempts in 18 secs): user=, method=DIGEST-MD5, rip=192.168.1.3, lip=192.168.1.2, session= Oct 27 18:15:40 fbsd9 dovecot: imap-login: Login: user=, method=PLAIN, rip=192.168.1.2, lip=192.168.1.3, mpid=62534, TLS, session= Oct 27 18:15:56 fbsd9 dovecot: imap(usertest): Disconnected: Logged out in=8 out=383 From toni at solu.fi Wed Oct 28 08:20:20 2015 From: toni at solu.fi (Toni Mattila) Date: Wed, 28 Oct 2015 10:20:20 +0200 Subject: 2.2.19 crash Message-ID: <56308544.1070709@solu.fi> Hi, I have one user constantly crashing with: Core was generated by `dovecot/imap'. Program terminated with signal 11, Segmentation fault. #0 0x007e5b7c in mailbox_list_notify_next (notify=0x0, rec_r=0xbfee8f00) at mailbox-list-notify.c:28 28 return notify->list->v.notify_next(notify, rec_r); (gdb) bt #0 0x007e5b7c in mailbox_list_notify_next (notify=0x0, rec_r=0xbfee8f00) at mailbox-list-notify.c:28 #1 0x080641f2 in imap_client_notify_ns (notify_ns=0x8859478) at imap-notify.c:248 #2 0x0806476a in imap_client_notify_more (client=) at imap-notify.c:297 #3 0x080648df in imap_client_notify_newmails (client=0x88555c0) at imap-notify.c:339 #4 0x080699fe in imap_sync_notify_more (ctx=0x8858d70) at imap-sync.c:356 #5 imap_sync_more (ctx=0x8858d70) at imap-sync.c:526 #6 0x08069db2 in cmd_sync_continue (sync_cmd=0x8856008) at imap-sync.c:658 #7 0x0806a206 in cmd_sync_delayed_real (client=0x88555c0) at imap-sync.c:748 #8 cmd_sync_delayed (client=0x88555c0) at imap-sync.c:886 #9 0x007e7c25 in notify_delay_callback (box=0x8864cd0) at mailbox-watch.c:26 #10 0x00caec60 in io_loop_handle_timeouts_real (ioloop=0x883d4a0) at ioloop.c:527 #11 io_loop_handle_timeouts (ioloop=0x883d4a0) at ioloop.c:540 #12 0x00cb0299 in io_loop_handler_run_internal (ioloop=0x883d4a0) at ioloop-epoll.c:193 #13 0x00caea8b in io_loop_handler_run (ioloop=0x883d4a0) at ioloop.c:607 #14 0x00caee08 in io_loop_run (ioloop=0x883d4a0) at ioloop.c:583 #15 0x00c4216d in master_service_run (service=0x883d3d0, callback=0x806a700 ) at master-service.c:640 #16 0x0806af22 in main (argc=Cannot access memory at address 0x0 (gdb) f 6 #6 0x08069db2 in cmd_sync_continue (sync_cmd=0x8856008) at imap-sync.c:658 658 if ((ret = imap_sync_more(ctx)) == 0) Best Regards, Toni Mattila From gabriele.bulfon at sonicle.com Wed Oct 28 10:53:21 2015 From: gabriele.bulfon at sonicle.com (Gabriele Bulfon) Date: Wed, 28 Oct 2015 11:53:21 +0100 (CET) Subject: Dovecot, JavaMail, UIDs and Message Numbers Message-ID: <12390852.77.1446029601297.JavaMail.sonicle@www> Hi, new to this list, so a little prelude to my issue with Dovecot. We have been using JavaMail against Cyrus for ages, and developed Webtop, a huge Java web collaboration application running on them in production in various installations for all this time. Recently we had to run the same software against Dovecot pre-existing accounts running on Nethesis NethServer solution. After some time of testing, we started having an intermittent problem, where Dovecot started reindexing when Webtop was trying to issue sort or search commands. The same accounts previously worked with Thunderbird clients and/or RoundCube interface, never facing this problem. Digging into the issue, we compared the imap protocol produced by Thunderbird, RoundCube and Webtop, discovering that JavaMail is internally all based on message numbers, so any sort/search/fetch is a normal SORT/SEARCH/FETCH, while RC and TB always use UID SORT/UID SEARCH/UID FETCH. We tried to convert our code to use UIDs instead of msgnums, but turned out to be a huge rewrite of JavaMail, because of how JavaMail imap implementation is tied to msgnums. So my questions are: - is it a known fact that Dovecot prefers and performs better with UIDs than with msgnums? - is there any tweak we can do on Dovecot to work better on msgnums? - anybody already faced this issue with JavaMail/Dovecot? Thanks for any suggestion, Gabriele. From Hajo.Locke at gmx.de Wed Oct 28 14:56:40 2015 From: Hajo.Locke at gmx.de (Hajo Locke) Date: Wed, 28 Oct 2015 15:56:40 +0100 Subject: sieve vacation - 2 questions Message-ID: <5630E228.5010404@gmx.de> Hello, still trying to move from procmail to sieve, but there are new problems. Currently we use procmail for autoresponder and let user choose timerange, in which responder should work. May be you set up today your responder for 2 weeks in future where responder start and stop automatically without further manuell intervention. In procmail this is done by simple scripting and comparison of timestamps. How to do this in sieve? 2nd questions concerns to used From-Address in responding mail. Header of a sieve-responder shows a minimal From-Address: From: But our Users want to add a custom name which is displayed at recipient of responder: From: John Doe In procmail this is easy with formail, how to do this in sieve? Both features we implemented because people asked about it. I should find a way to recreate this in sieve. Thanks, Hajo From stephan at rename-it.nl Wed Oct 28 16:34:10 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 28 Oct 2015 17:34:10 +0100 Subject: sieve vacation - 2 questions In-Reply-To: <5630E228.5010404@gmx.de> References: <5630E228.5010404@gmx.de> Message-ID: <5630F902.7060404@rename-it.nl> Op 10/28/2015 om 3:56 PM schreef Hajo Locke: > Hello, > > still trying to move from procmail to sieve, but there are new problems. > > Currently we use procmail for autoresponder and let user choose > timerange, in which responder should work. May be you set up today > your responder for 2 weeks in future where responder start and stop > automatically without further manuell intervention. > In procmail this is done by simple scripting and comparison of > timestamps. How to do this in sieve? You need to use the currentdate test, as provided by the date extension (http://tools.ietf.org/html/rfc5260). A useful example is here: http://tools.ietf.org/html/rfc5260#section-5.1 > 2nd questions concerns to used From-Address in responding mail. > Header of a sieve-responder shows a minimal From-Address: > From: > But our Users want to add a custom name which is displayed at > recipient of responder: > From: John Doe > In procmail this is easy with formail, how to do this in sieve? > Both features we implemented because people asked about it. I should > find a way to recreate this in sieve. Use the ":from" argument of the vacation command. This can also contain the name (phrase) part of the email address: require "vacation"; vacation :from "Stephan Bosch " "I am on off-world."; Regards, Stephan. From alessio at skye.it Wed Oct 28 17:11:39 2015 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 28 Oct 2015 18:11:39 +0100 Subject: events In-Reply-To: <562E08B4.2090405@dovecot.fi> References: <5627AA48.5030809@genkgo.nl> <562804B3.2060405@dovecot.fi> <56288642.2070803@genkgo.nl> <562A4065.6010706@dovecot.fi> <562DCF68.90609@genkgo.nl> <562DD805.2020301@dovecot.fi> <562DDA1C.4030801@genkgo.nl> <562E01DF.9000806@dovecot.fi> <562E03F6.4010904@genkgo.nl> <562E08B4.2090405@dovecot.fi> Message-ID: Il 26.10.2015 12:04 Teemu Huovila ha scritto: > On 26.10.2015 12:44, Frederik Bosch | Genkgo wrote: >> Teemu, >> >> If just need the http request, I will need something like the >> following configuration, right? So no meta data plugin, but with >> notify and push_notification? >> >> protocol lmtp { >> mail_plugins = $mail_plugins notify push_notification >> } >> >> plugin { >> push_notification_driver = ox:url=http://myurl/ >> >> } > You could test that, but my understanding of the ox push driver code > is that it completely depends on metadata and will not do anything > useful, if no metadata is set. Perhaps Michael can correct me, if Im > wrong. > > If you want some subset of the ox driver functionality, you could try > implementing your own driver, based on the existing code. Hi, I'm interested to testing push_notification with ox driver (I need only a GET when a new message arrived) but I don't understand how to insert METADATA information via IMAP for an user. Can somebody provide an example? Thanks -- Alessio Cecchi Postmaster AT http://www.qboxmail.it http://www.linkedin.com/in/alessice From Hajo.Locke at gmx.de Thu Oct 29 08:32:17 2015 From: Hajo.Locke at gmx.de (Hajo Locke) Date: Thu, 29 Oct 2015 09:32:17 +0100 Subject: sieve vacation - 2 questions In-Reply-To: <5630F902.7060404@rename-it.nl> References: <5630E228.5010404@gmx.de> <5630F902.7060404@rename-it.nl> Message-ID: <5631D991.5040408@gmx.de> Hello Stephan, this works great. Thanks a lot. Hajo Am 28.10.2015 um 17:34 schrieb Stephan Bosch: > Op 10/28/2015 om 3:56 PM schreef Hajo Locke: >> Hello, >> >> still trying to move from procmail to sieve, but there are new problems. >> >> Currently we use procmail for autoresponder and let user choose >> timerange, in which responder should work. May be you set up today >> your responder for 2 weeks in future where responder start and stop >> automatically without further manuell intervention. >> In procmail this is done by simple scripting and comparison of >> timestamps. How to do this in sieve? > You need to use the currentdate test, as provided by the date extension > (http://tools.ietf.org/html/rfc5260). A useful example is here: > > http://tools.ietf.org/html/rfc5260#section-5.1 > >> 2nd questions concerns to used From-Address in responding mail. >> Header of a sieve-responder shows a minimal From-Address: >> From: >> But our Users want to add a custom name which is displayed at >> recipient of responder: >> From: John Doe >> In procmail this is easy with formail, how to do this in sieve? >> Both features we implemented because people asked about it. I should >> find a way to recreate this in sieve. > Use the ":from" argument of the vacation command. This can also contain > the name (phrase) part of the email address: > > > require "vacation"; > > vacation :from "Stephan Bosch " "I am on off-world."; > > > > Regards, > > Stephan. > From teemu.huovila at dovecot.fi Thu Oct 29 09:42:02 2015 From: teemu.huovila at dovecot.fi (Teemu Huovila) Date: Thu, 29 Oct 2015 11:42:02 +0200 Subject: events In-Reply-To: References: <5627AA48.5030809@genkgo.nl> <562804B3.2060405@dovecot.fi> <56288642.2070803@genkgo.nl> <562A4065.6010706@dovecot.fi> <562DCF68.90609@genkgo.nl> <562DD805.2020301@dovecot.fi> <562DDA1C.4030801@genkgo.nl> <562E01DF.9000806@dovecot.fi> <562E03F6.4010904@genkgo.nl> <562E08B4.2090405@dovecot.fi> Message-ID: <5631E9EA.4010501@dovecot.fi> On 28.10.2015 19:11, Alessio Cecchi wrote: > Il 26.10.2015 12:04 Teemu Huovila ha scritto: >> On 26.10.2015 12:44, Frederik Bosch | Genkgo wrote: >>> Teemu, >>> >>> If just need the http request, I will need something like the following configuration, right? So no meta data plugin, but with notify and push_notification? >>> >>> protocol lmtp { >>> mail_plugins = $mail_plugins notify push_notification >>> } >>> >>> plugin { >>> push_notification_driver = ox:url=http://myurl/ >>> } >> You could test that, but my understanding of the ox push driver code >> is that it completely depends on metadata and will not do anything >> useful, if no metadata is set. Perhaps Michael can correct me, if Im >> wrong. >> >> If you want some subset of the ox driver functionality, you could try >> implementing your own driver, based on the existing code. > > Hi, I'm interested to testing push_notification with ox driver (I need only a GET when a new message arrived) but I don't understand how to insert METADATA information via IMAP for an user. The notification is done with a http PUT. The IMAP METADATA is set with the SETMETADATA command (https://tools.ietf.org/html/rfc5464#section-4.3). As to how to register for the notifications, the best documentation is probably the source. You can see it in either http://hg.dovecot.org/dovecot-2.2/file/9654ab4c337c/src/plugins/push-notification/push-notification-driver-ox.c or maybe more easily in the OX source code file backend/com.openexchange.push.dovecot/src/com/openexchange/push/dovecot/commands/RegistrationCommand.java You can get the backend source by git clone https://code.open-xchange.com/git/wd/backend An example would be something like: SETMETADATA "" (/private/vendor/vendor.dovecot/http-notify "user=myusername") br, Teemu Huovila From ml+dovecot at valo.at Thu Oct 29 10:15:10 2015 From: ml+dovecot at valo.at (Christian Kivalo) Date: Thu, 29 Oct 2015 11:15:10 +0100 Subject: Acl sql dict error, dict crashes Message-ID: <477fffe6f01600205ae986d8a09dfb3b@valo.at> Hi, I configured shared mailboxes and get an error when i use SQL as the acl_shared_dict. It works when i use a file instead. With SQL as the shared_dict, dict crashes. I have enabled sql query logs on the mariadb server, the table user_shares and anyone_shares are queried but there is no insert to the empty tables. The dovecot-acl files are created in the mailbox when i add an acl with doveadm acl add ... doveadm acl debug shows an error: doveadm acl debug -u my at example.net shared/other at example.net/INBOX doveadm(my at example.net): Info: Mailbox 'INBOX' is in namespace 'shared/other at example.net/' doveadm(my at example.net): Info: Mailbox path: /srv/mail/other at example.net/Maildir doveadm(my at example.net): Info: Per-user private flags in mailbox: \Seen doveadm(my at example.net): Info: User my at example.net has rights: lookup read doveadm(my at example.net): Info: Mailbox found from dovecot-acl-list doveadm(my at example.net): Error: User other at example.net not found from ACL shared dict, rebuilding doveadm(my at example.net): Error: read(/var/run/dovecot/dict) failed: Connection reset by peer doveadm(my at example.net): Error: acl: dict commit failed doveadm(my at example.net): Fatal: ACL lookup dict rebuild failed then the dict crashes Oct 29 10:12:13 uschi dovecot: auth: Debug: master in: USER#0111#011my at example.net#011service=doveadm Oct 29 10:12:13 uschi dovecot: auth-worker(27393): Debug: sql(my at example.net): SELECT CONCAT("/srv/mail/", dir) AS home, CONCAT("maildir:/srv/mail/", dir, "/Maildir:LAYOUT=fs") AS mail, 5000 AS uid, 5000 AS gid FROM virtual_users WHERE email = 'my at example.net'; Oct 29 10:12:13 uschi dovecot: auth: Debug: userdb out: USER#0111#011my at example.net#011home=/srv/mail/my at example.net#011mail=maildir:/srv/mail/my at example.net/Maildir:LAYOUT=fs#011uid=5000#011gid=5000 Oct 29 10:12:13 uschi dovecot: auth: Debug: master in: USER#0112#011other at example.net#011service=doveadm Oct 29 10:12:13 uschi dovecot: auth-worker(27393): Debug: sql(other at example.net): SELECT CONCAT("/srv/mail/", dir) AS home, CONCAT("maildir:/srv/mail/", dir, "/Maildir:LAYOUT=fs") AS mail, 5000 AS uid, 5000 AS gid FROM virtual_users WHERE email = 'other at example.net'; Oct 29 10:12:13 uschi dovecot: auth: Debug: userdb out: USER#0112#011other at example.net#011home=/srv/mail/other at example.net#011mail=maildir:/srv/mail/other at example.net/Maildir:LAYOUT=fs#011uid=5000#011gid=5000 Oct 29 10:12:13 uschi dovecot: dict(27421): Panic: file dict-sql.c: line 670 (sql_dict_iterate): assertion failed: ((ctx->flags & DICT_ITERATE_FLAG_ASYNC) != 0) Oct 29 10:12:13 uschi dovecot: dict(27421): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7e290) [0x7f823f032290] -> /usr/lib/dovecot/libdovecot.so.0(+0x7e37c) [0x7f823f03237c] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f823efdd65d] -> dovecot/dict() [0x407e84] -> dovecot/dict() [0x405890] -> dovecot/dict() [0x405b01] -> dovecot/dict(dict_command_input+0xa1) [0x405bd1] -> dovecot/dict() [0x404d96] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7f823f044f7c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xe1) [0x7f823f046231] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7f823f045005] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f823f0451a8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f823efe2c23] -> dovecot/dict(main+0x149) [0x404809] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f823e4cdb45] -> dovecot/dict() [0x40488d] Oct 29 10:12:13 uschi dovecot: dict(27421): Fatal: master: service(dict): child 27421 killed with signal 6 (core dumps disabled) The SQL query log from mariadb shows 151029 10:18:06 5318 Connect readuser at 10.1.1.50 as anonymous on mailserver 5318 Query SELECT CONCAT("/srv/mail/", dir) AS home, CONCAT("maildir:/srv/mail/", dir, "/Maildir:LAYOUT=fs") AS mail, 5000 AS uid, 5000 AS gid FROM virtual_users WHERE email = 'my at example.net' 5318 Query SELECT CONCAT("/srv/mail/", dir) AS home, CONCAT("maildir:/srv/mail/", dir, "/Maildir:LAYOUT=fs") AS mail, 5000 AS uid, 5000 AS gid FROM virtual_users WHERE email = 'other at example.net' 5319 Connect writeuser at 10.1.1.50 as anonymous on mailserver 5319 Query SELECT dummy,from_user FROM anyone_shares 5319 Query SELECT dummy,from_user FROM user_shares WHERE to_user = 'my at example.net' 5319 Query SELECT dummy,to_user,from_user FROM user_shares 5319 Query SELECT dummy,from_user FROM anyone_shares There is no insert. As far as i can tell everything works as it should with shared folders except that die sql based acl_shared_dict is not updated. The expire dict is properly updated. Is there something i can test? My Thunderbird shows me that the server does not support shareing of mailboxes so i can't share parts of my mailbox to other users. Is this a separate issue or related? doveconf -n # 2.2.19: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.9 (357ac0a0e68b+) # OS: Linux 4.2.1 x86_64 Debian 8.2 ext4 auth_debug = yes auth_mechanisms = plain login auth_verbose = yes default_vsz_limit = 512 M deliver_log_format = msgid=%m, from=%f, envelope from=%{from_envelope}, envelope to=%{to_envelope}, delivery time=%{delivery_time}ms, lmtp session time=%{session_time}ms, status=%$ dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } first_valid_gid = 5000 first_valid_uid = 5000 imap_client_workarounds = tb-extra-mailbox-sep tb-lsub-flags last_valid_gid = 5000 last_valid_uid = 5000 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k session=<%{session}> mail_access_groups = vmail mail_debug = yes mail_gid = vmail mail_location = maildir:/srv/mail/%u/Maildir:LAYOUT=fs mail_plugins = fts fts_solr virtual stats expire acl mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate namespace { list = children location = maildir:%%h/Maildir:INDEXPVT=%h/shared/%%u:LAYOUT=fs prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Archives { auto = subscribe special_use = \Archive } mailbox "Deleted Items" { special_use = \Trash } mailbox Draft { special_use = \Drafts } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox "Gel&APY-schte Objekte" { special_use = \Trash } mailbox "Gesendete Objekte" { special_use = \Sent } mailbox Junk-E-Mail { special_use = \Junk } mailbox Junk { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Items" { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / subscriptions = yes type = private } namespace virtual { hidden = no inbox = no list = children location = virtual:/srv/mail/%u/virtual mailbox All { special_use = \All } prefix = virtual/ separator = / subscriptions = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = proxy::acl antispam_backend = dspam antispam_debug_target = syslog antispam_dspam_args = --client;--user;%u;--source=error;--signature=%%s antispam_dspam_binary = /usr/bin/dspam antispam_dspam_notspam = --class=innocent antispam_dspam_spam = --class=spam antispam_signature = X-DSPAM-Signature antispam_signature_missing = error antispam_spam_pattern_ignorecase = Junk;SPAM antispam_trash_pattern_ignorecase = trash;Deleted *;Gel&APY-schte *;Gel??schte * expire = Trash expire2 = Spam expire_dict = proxy::expire fts = solr fts_autoindex = yes fts_solr = break-imap-search url=http://10.1.1.50:8983/solr/ mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename append mail_log_fields = uid box msgid size from vsize flags mailbox_alias_new = Sent Messages mailbox_alias_new10 = Draft mailbox_alias_new11 = Entw&APw-rfe mailbox_alias_new2 = Sent Items mailbox_alias_new3 = Gesendete Objekte mailbox_alias_new4 = Sent Messages mailbox_alias_new5 = Junk mailbox_alias_new6 = Junk-E-Mail mailbox_alias_new7 = Deleted Items mailbox_alias_new8 = Deleted Messages mailbox_alias_new9 = Gel&APY-schte Objekte mailbox_alias_old = Sent mailbox_alias_old10 = Drafts mailbox_alias_old11 = Drafts mailbox_alias_old2 = Sent mailbox_alias_old3 = Sent mailbox_alias_old4 = Sent mailbox_alias_old5 = Spam mailbox_alias_old6 = Spam mailbox_alias_old7 = Trash mailbox_alias_old8 = Trash mailbox_alias_old9 = Trash sieve = file:/srv/sieve/%u/;active=/srv/sieve/%u/.dovecot.sieve sieve_before = /srv/sieve/before stats_refresh = 30 secs stats_track_cmds = yes } postmaster_address = postmaster at sec-svcs.eu protocols = imap pop3 lmtp sieve service auth-worker { user = doveauth } service auth { inet_listener { address = 10.1.1.10 port = 12987 } unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = doveauth mode = 0666 user = doveauth } } service dict { unix_listener dict { group = vmail mode = 0660 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 2 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0666 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 0 } } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl = required ssl_cert = References: <12390852.77.1446029601297.JavaMail.sonicle@www> Message-ID: <5631F7D8.8080607@iki.fi> On 10/28/2015 12:53 PM, Gabriele Bulfon wrote: > Hi, > new to this list, so a little prelude to my issue with Dovecot. > We have been using JavaMail against Cyrus for ages, and developed Webtop, a huge Java web collaboration application running on them in production in various installations for all this time. > Recently we had to run the same software against Dovecot pre-existing accounts running on Nethesis NethServer solution. > After some time of testing, we started having an intermittent problem, where Dovecot started reindexing when Webtop was trying to issue sort or search commands. > The same accounts previously worked with Thunderbird clients and/or RoundCube interface, never facing this problem. > Digging into the issue, we compared the imap protocol produced by Thunderbird, RoundCube and Webtop, discovering > that JavaMail is internally all based on message numbers, so any sort/search/fetch is a normal SORT/SEARCH/FETCH, > while RC and TB always use UID SORT/UID SEARCH/UID FETCH. > We tried to convert our code to use UIDs instead of msgnums, but turned out to be a huge rewrite of JavaMail, because > of how JavaMail imap implementation is tied to msgnums. > So my questions are: > - is it a known fact that Dovecot prefers and performs better with UIDs than with msgnums? > - is there any tweak we can do on Dovecot to work better on msgnums? > - anybody already faced this issue with JavaMail/Dovecot? Dovecot doesn't care if you use msgnums of UIDs. They both work fine. Your problem has to be something else. What exactly do you mean by "Dovecot started reindexing"? Dovecot's normal indexes or full text search indexes? How have you configured everything? Does Dovecot log anything (especially any errors/warnings about anything)? From daniel at demus.dk Thu Oct 29 08:54:05 2015 From: daniel at demus.dk (Daniel Demus) Date: Thu, 29 Oct 2015 09:54:05 +0100 Subject: Indexer crash when fixing incorrectly named Maildir files Message-ID: <5631DEAD.7060808@demus.dk> Hi, If the indexer encounters, and fixes by renaming, an incorrectly named file, it crashes immediately afterwards. I would guess the old file-name is re-used after re-naming and but isn't checked because it is assumed to exist having come from a listing of directory contents. dovecot: indexer-worker(): Error: Maildir filename has wrong S value, renamed the file from /home//Maildir/.INBOX.NYTimes/cur/1095923655.M449995P13207V0000000000000302I01010085_0.server.demus.dk,S=11936:2,S to /home//Maildir/.INBOX.NYTimes/cur/1095923655.M449995P13207V0000000000000302I01010085_0.server.demus.dk,S=12378:2,S: 1 Time(s) dovecot: indexer-worker(): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x737c4) [0x76e327c4] -> /usr/lib/dovecot/libdovecot.so.0(i_warning+0) [0x76e328e4] -> /usr/lib/dovecot/libdovecot.so.0(i_panic+0x64) [0x76dda320] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x998b0) [0x76f388b0] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_init_stream+0x1c8) [0x76f38df4] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x5391c) [0x76ef291c] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_get_stream+0x68) [0x76ec482c] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x536f4) [0x76ef26f4] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_get_virtual_size+0x48) [0x76ec4454] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_precache+0x124) [0x76f3a51c] -> /usr/lib/dovecot/lib20_fts_plugin.so(+0xcf20) [0x76beff20] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_precache+0x40) [0x76ec4c84] -> dovecot/indexer-worker(+0x1eec) [0x54b78eec] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x68) [0x76e46794] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xe8) [0x76e47a1c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xc) [0x76e46824] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x50) [0x76e468c8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x14) [0x76de02f0] -> dovecot/indexer-worker(main+0xe8) [0x54b78880] -> /lib/libc.so.6(__libc_start_main+0x114) [0x76c51d3c]: 1 Time(s) dovecot: indexer-worker(): Error: read(/home//Maildir/.INBOX.NYTimes/cur/1095923655.M449995P13207V0000000000000302I01010085_0.server.demus.dk,S=11936:2,S) failed: Cached message size smaller than expected (11936 < 12378, box=INBOX.NYTimes, UID=1054): 1 Time(s) dovecot: indexer-worker(): Error: read(/home//Maildir/.INBOX.NYTimes/cur/1095923655.M449995P13207V0000000000000302I01010085_0.server.demus.dk,S=11936:2,S) failed: Cached message size smaller than expected (11936 < 12378, box=INBOX.NYTimes, UID=1054) (uid=1054, box=INBOX.NYTimes): 2 Time(s) dovecot: indexer-worker(): Fatal: master: service(indexer-worker): child 15655 killed with signal 6 (core dumps disabled): 1 Time(s) Regards, Daniel Demus From tss at iki.fi Thu Oct 29 10:43:30 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 29 Oct 2015 12:43:30 +0200 Subject: Acl sql dict error, dict crashes In-Reply-To: <477fffe6f01600205ae986d8a09dfb3b@valo.at> References: <477fffe6f01600205ae986d8a09dfb3b@valo.at> Message-ID: <26C5AEC6-3442-49AA-A9FF-F5152A777BF9@iki.fi> On 29 Oct 2015, at 12:15, Christian Kivalo wrote: > > Hi, > > I configured shared mailboxes and get an error when i use SQL as the acl_shared_dict. It works when i use a file instead. With SQL as the shared_dict, dict crashes. > > Oct 29 10:12:13 uschi dovecot: dict(27421): Panic: file dict-sql.c: line 670 (sql_dict_iterate): assertion failed: ((ctx->flags & DICT_ITERATE_FLAG_ASYNC) != 0) http://hg.dovecot.org/dovecot-2.2/rev/467695fee373 should fix this. From tss at iki.fi Thu Oct 29 10:48:15 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 29 Oct 2015 12:48:15 +0200 Subject: Proxy with director accept only plain login In-Reply-To: References: Message-ID: On 27 Oct 2015, at 17:43, Andrey Fesenko wrote: > > Hello, i'm test system dovecot (proxy with director) and backend > storage, auth LDAP server (user plain passwords) > > If i use plain auth, work fine. > > If connect DIGEST-MD5 or CRAM-MD5 proxy not redirect connection > (Requested DIGEST-MD5 scheme, but we have a NULL password) > > ### Frontend proxy+director .. > passdb { > args = /usr/local/etc/dovecot/dovecot-ldap.conf > driver = ldap > } So LDAP is the primary way of authenticating. > pass_attrs = uid=user,=password=,description=proxy,ipHostNumber=host,=nopassword=y,=starttls=any-cert But you set password to empty and nopassword=yes. CRAM-MD5 and DIGEST-MD5 authentication requires that the server already knows the password. The only way to make it work is to have proxy actually fully authenticate the user and then login to Dovecot backend with a master password. From cumc-4361-2 at chguadalquivir.es Thu Oct 29 10:50:42 2015 From: cumc-4361-2 at chguadalquivir.es (Fran) Date: Thu, 29 Oct 2015 11:50:42 +0100 Subject: My dovecot works fine against Active Directory 2003, but not against AD2008 In-Reply-To: <55F9709C.2090108@elyograg.org> References: <201509120631.t8C6VE2T005436@mail.hprs.local> <55F9709C.2090108@elyograg.org> Message-ID: <5631FA02.5040905@chguadalquivir.es> Exactly, that's what I meant. El 16/09/2015 a las 15:37, Shawn Heisey escribi?: > On 9/12/2015 12:31 AM, Mark Foley wrote: >> Hmmm, I've not heard of "Active Directory 2003" or 2008. The year numbers >> indicated to me you might be talking about Windows Small Business Server 2003 or >> 2008. Is your AD Server Windows? Linux? Something else? I'm using Samba4 AD/DC >> on Linux. > The OP probably is referring to AD functional levels: > > https://technet.microsoft.com/en-us/library/cc787290%28v=ws.10%29.aspx > > Thanks, > Shawn > From cumc-4361-2 at chguadalquivir.es Thu Oct 29 11:16:04 2015 From: cumc-4361-2 at chguadalquivir.es (Fran) Date: Thu, 29 Oct 2015 12:16:04 +0100 Subject: My dovecot works fine against Active Directory 2003, but not against AD2008 In-Reply-To: <201509120631.t8C6VE2T005436@mail.hprs.local> References: <201509120631.t8C6VE2T005436@mail.hprs.local> Message-ID: <5631FFF4.1020507@chguadalquivir.es> I'm sorry for the late response, I missed this mail. I'll answer your questions below. I'm sending a BCC of this mail to your personal address, but it seems to have some problem because your server bounces it: El 12/09/2015 a las 8:31, Mark Foley escribi?: > Fran - thanks for your reply. I'm cc'ing you directly on this as well as posting > to the list as I'm not sure how often you check the list and I'm down to hanging > by my last fingernail on this project. > > I have some preliminary questions interspersed below. > > Thanks, --Mark > > -----Original Message----- >> Subject: Re: My dovecot works fine against Active Directory 2003, but not >> against AD2008 >> To: dovecot at dovecot.org >> From: Fran >> Date: Thu, 10 Sep 2015 13:26:21 +0200 >> >> Hi Mark, >> >> when I say AD 2003/8 I mean Active Directory 2003/8. > Hmmm, I've not heard of "Active Directory 2003" or 2008. The year numbers > indicated to me you might be talking about Windows Small Business Server 2003 or > 2008. Is your AD Server Windows? Linux? Something else? I'm using Samba4 AD/DC > on Linux. https://technet.microsoft.com/en-us/library/cc787290%28v=ws.10%29.aspx >> My configuration is attached. > Thank you very much for that. If I make some headway, I'll likely have more > questions on specifics. > >> I based my installation (dovecot+postfix) in the guides of this site: >> http://www.linuxmail.info >> >> The LDAP part is this: >> http://www.linuxmail.info/postfix-dovecot-ldap-centos-5/ > If you were able to make sense out of these sites' tiny screen-shots and one-line > descriptions my hat's off to you. "Your a better man that I am Gunga-Din!" If > there was more detailed narrative somewhere I couldn't find it. Also, I don't > have jXplorer on my system, so probably I couldn't get too far anyway. You don't need jXplorer at all, in fact I didn't use it. If you need to browser throught your LDAP directory you can use any LDAP browser. The descriptions of that site are short, that's true, but it contains the essential info to adapt it to any similar environment. Don't take it like a step by step guide, unless you use exactly the same environment and versions, you won't find same files in same places. Try to understand how differents parts work and adapt it to your environment. > > BIG QUESTIONS: > > 1. Are you using MS Outlook IMAP clients in your environment? If so, how are you > making them connect with LDAP? By checking the SPA checkbox? There are Thunderbird, Roundcube, Outlook, IOS and Android clients on my environment. All of them use standard IMAP connections. I don't understand very well your question, the client doesn't need to connect with LDAP, it's dovecot itself who connect with AD to validate the IMAP user login. > > 2. The mail_gid/mail_uid as vmail confuses me. I see that setting a lot, > including in your config. http://wiki2.dovecot.org/VirtualUsers says, "You can > create, for example, one vmail user which owns all the mails, or you can assign > a separate UID for each user." I have assigned a separte UID for each based on > the UID returned by `wbinfo -u `. Does assigning separate UIDs mess > up my ability to adapt your configuration? I assigned one vmail user which owns all the mails. You can still use my configuration for many other parts though. > > little questions: > > 3. I'm not planning on using quotas. Can I safely omit your mail_plugins = " quota" > setting and all your plugin { quota_...} settings? I want to be as simple as > possible to start. You don't need that plugin if you don't plan to use it. > > 4. Likewise, dovecot seems to be able to find users' mailboxes just fine. Can I > omit the namespace inbox {} setting? I don't think so. This is my in /etc/dovecot/conf.d/10-mail.conf mail_home = /home/vmail//%Lu mail_location = maildir:~/Maildir mail_uid = 1000 mail_gid = 1000 namespace inbox { # Namespace type: private, shared or public type = private inbox = yes mailbox Trash { auto = subscribe special_use = \Trash } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Junk { auto = subscribe special_use = \Junk } } I think this is essential to have a minimal directory structure in any new mail account > > These may seem like amaturish questions, but little details have foiled me a lot > on this Dovecot project. > > If I feel confident with the answers you provide here, I'll move on to trying > some things. > > Thanks a lot for your help!!! > > --Mark > >> You can also use PAM to connect to AD >> (http://www.linuxmail.info/active-directory-dovecot-pam-authentication/) >> but that way doesn't allow to retrieve custom fields from the AD (ex. a >> field to set quota per user), so I'm using the standard LDAP method. >> >> Regards >> >> El 10/09/2015 a las 4:51, Mark Foley escribi?: >>> Fran and/or Matthias, >>> >>> Could you publish your doveconf -n? I can't get dovecot to authenticate with my >>> AD. Maybe you have a solution I could try. >>> >>> What mail client(s) are you using? I assume by "AD 2003/8" You mean SBS2003/8 >>> and are therefore using Outlook? >>> >>> --Mark >>> >>> -----Original Message----- > [deleted] > ? Regards From gabriele.bulfon at sonicle.com Thu Oct 29 12:37:01 2015 From: gabriele.bulfon at sonicle.com (Gabriele Bulfon) Date: Thu, 29 Oct 2015 13:37:01 +0100 (CET) Subject: Dovecot, JavaMail, UIDs and Message Numbers In-Reply-To: <5631F7D8.8080607@iki.fi> References: <12390852.77.1446029601297.JavaMail.sonicle@www> <5631F7D8.8080607@iki.fi> Message-ID: <2295612.72.1446122221612.JavaMail.sonicle@www> In this case, this is no more my field, I will ask the NethServer gurus to check what you asked. Thanks a lot Gabriele ---------------------------------------------------------------------------------- Da: Timo Sirainen A: Gabriele Bulfon dovecot at dovecot.org Cc: Cristian Manoni Giacomo Sanchietti Raffaele Fullone Matteo Albinola Data: 29 ottobre 2015 11.41.28 CET Oggetto: Re: Dovecot, JavaMail, UIDs and Message Numbers On 10/28/2015 12:53 PM, Gabriele Bulfon wrote: Hi, new to this list, so a little prelude to my issue with Dovecot. We have been using JavaMail against Cyrus for ages, and developed Webtop, a huge Java web collaboration application running on them in production in various installations for all this time. Recently we had to run the same software against Dovecot pre-existing accounts running on Nethesis NethServer solution. After some time of testing, we started having an intermittent problem, where Dovecot started reindexing when Webtop was trying to issue sort or search commands. The same accounts previously worked with Thunderbird clients and/or RoundCube interface, never facing this problem. Digging into the issue, we compared the imap protocol produced by Thunderbird, RoundCube and Webtop, discovering that JavaMail is internally all based on message numbers, so any sort/search/fetch is a normal SORT/SEARCH/FETCH, while RC and TB always use UID SORT/UID SEARCH/UID FETCH. We tried to convert our code to use UIDs instead of msgnums, but turned out to be a huge rewrite of JavaMail, because of how JavaMail imap implementation is tied to msgnums. So my questions are: - is it a known fact that Dovecot prefers and performs better with UIDs than with msgnums? - is there any tweak we can do on Dovecot to work better on msgnums? - anybody already faced this issue with JavaMail/Dovecot? Dovecot doesn't care if you use msgnums of UIDs. They both work fine. Your problem has to be something else. What exactly do you mean by "Dovecot started reindexing"? Dovecot's normal indexes or full text search indexes? How have you configured everything? Does Dovecot log anything (especially any errors/warnings about anything)? From ml+dovecot at valo.at Thu Oct 29 12:59:53 2015 From: ml+dovecot at valo.at (Christian Kivalo) Date: Thu, 29 Oct 2015 13:59:53 +0100 Subject: Acl sql dict error, dict crashes In-Reply-To: <26C5AEC6-3442-49AA-A9FF-F5152A777BF9@iki.fi> References: <477fffe6f01600205ae986d8a09dfb3b@valo.at> <26C5AEC6-3442-49AA-A9FF-F5152A777BF9@iki.fi> Message-ID: Hi Timo, On 2015-10-29 11:43, Timo Sirainen wrote: > On 29 Oct 2015, at 12:15, Christian Kivalo wrote: >> >> Hi, >> >> I configured shared mailboxes and get an error when i use SQL as the >> acl_shared_dict. It works when i use a file instead. With SQL as the >> shared_dict, dict crashes. >> >> Oct 29 10:12:13 uschi dovecot: dict(27421): Panic: file dict-sql.c: >> line 670 (sql_dict_iterate): assertion failed: ((ctx->flags & >> DICT_ITERATE_FLAG_ASYNC) != 0) > > http://hg.dovecot.org/dovecot-2.2/rev/467695fee373 should fix this. Thanks, it works now. regards christian From toni at solu.fi Thu Oct 29 13:01:15 2015 From: toni at solu.fi (Toni Mattila) Date: Thu, 29 Oct 2015 15:01:15 +0200 Subject: 2.2.19 crash In-Reply-To: <56308544.1070709@solu.fi> References: <56308544.1070709@solu.fi> Message-ID: <5632189B.60201@solu.fi> Hi All, I also tried with 16ff395c156e and 3d576d682137 patches that touch lib-storage/notify and still 2.2.19 cores: Core was generated by `dovecot/imap'. Program terminated with signal 11, Segmentation fault. #0 0x0043bb8c in mailbox_list_notify_next (notify=0x0, rec_r=0xbfdb34a0) at mailbox-list-notify.c:28 28 return notify->list->v.notify_next(notify, rec_r); (gdb) bt #0 0x0043bb8c in mailbox_list_notify_next (notify=0x0, rec_r=0xbfdb34a0) at mailbox-list-notify.c:28 #1 0x080641f2 in imap_client_notify_ns (notify_ns=0x9f5b470) at imap-notify.c:248 #2 0x0806476a in imap_client_notify_more (client=) at imap-notify.c:297 #3 0x080648df in imap_client_notify_newmails (client=0x9f575b8) at imap-notify.c:339 #4 0x080699fe in imap_sync_notify_more (ctx=0x9f85b10) at imap-sync.c:356 #5 imap_sync_more (ctx=0x9f85b10) at imap-sync.c:526 #6 0x08069db2 in cmd_sync_continue (sync_cmd=0x9f58000) at imap-sync.c:658 #7 0x0806a206 in cmd_sync_delayed_real (client=0x9f575b8) at imap-sync.c:748 #8 cmd_sync_delayed (client=0x9f575b8) at imap-sync.c:886 #9 0x0043dc35 in notify_delay_callback (box=0x9f84e88) at mailbox-watch.c:26 #10 0x001e9c60 in io_loop_handle_timeouts_real (ioloop=0x9f3f498) at ioloop.c:527 #11 io_loop_handle_timeouts (ioloop=0x9f3f498) at ioloop.c:540 #12 0x001eb299 in io_loop_handler_run_internal (ioloop=0x9f3f498) at ioloop-epoll.c:193 #13 0x001e9a8b in io_loop_handler_run (ioloop=0x9f3f498) at ioloop.c:607 #14 0x001e9e08 in io_loop_run (ioloop=0x9f3f498) at ioloop.c:583 #15 0x0017d16d in master_service_run (service=0x9f3f3c8, callback=0x806a700 ) at master-service.c:640 #16 0x0806af22 in main (argc=Cannot access memory at address 0x0 ) at main.c:442 Anything else I should try? Thanks in advance, Toni Mattila From tss at iki.fi Thu Oct 29 13:29:49 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 29 Oct 2015 15:29:49 +0200 Subject: 2.2.19 crash In-Reply-To: <5632189B.60201@solu.fi> References: <56308544.1070709@solu.fi> <5632189B.60201@solu.fi> Message-ID: <56321F4D.6000509@iki.fi> On 10/29/2015 03:01 PM, Toni Mattila wrote: > Hi All, > > I also tried with 16ff395c156e and 3d576d682137 patches that touch > lib-storage/notify and still 2.2.19 cores: > > Core was generated by `dovecot/imap'. > Program terminated with signal 11, Segmentation fault. > #0 0x0043bb8c in mailbox_list_notify_next (notify=0x0, > rec_r=0xbfdb34a0) at mailbox-list-notify.c:28 > 28 return notify->list->v.notify_next(notify, rec_r); > (gdb) bt > #0 0x0043bb8c in mailbox_list_notify_next (notify=0x0, > rec_r=0xbfdb34a0) at mailbox-list-notify.c:28 > #1 0x080641f2 in imap_client_notify_ns (notify_ns=0x9f5b470) at > imap-notify.c:248 http://hg.dovecot.org/dovecot-2.2/rev/787cfed25b0f should fix this. From toni at solu.fi Thu Oct 29 16:00:34 2015 From: toni at solu.fi (Toni Mattila) Date: Thu, 29 Oct 2015 18:00:34 +0200 Subject: 2.2.19 crash In-Reply-To: <56321F4D.6000509@iki.fi> References: <56308544.1070709@solu.fi> <5632189B.60201@solu.fi> <56321F4D.6000509@iki.fi> Message-ID: <563242A2.30102@solu.fi> Hi, On 29-Oct-15 15:29, Timo Sirainen wrote: >> #1 0x080641f2 in imap_client_notify_ns (notify_ns=0x9f5b470) at >> imap-notify.c:248 > http://hg.dovecot.org/dovecot-2.2/rev/787cfed25b0f should fix this. That seems to have fixed it. Atleast no cores for couple hours.. before it was crashing every 15-30minutes. Thanks again, Toni From dave.mehler at gmail.com Fri Oct 30 01:28:35 2015 From: dave.mehler at gmail.com (David Mehler) Date: Thu, 29 Oct 2015 21:28:35 -0400 Subject: Webmail accessive Dovecot logins Message-ID: Hello, I've seen this issue before, running a imap/smtp/database server on localhost and adding in a webmail interface, in this case Roundcube. In my maillog I'm seeing accessive Dovecot connections and logouts just from my own transaction of logging in, going to compose a message, sending, and logging out. I'm using Mysql as database backend and was wondering if there was something I could do to cut down on the amount of connections needed? I'm running 2.2.19 on a FreeBSD 10.2 system. I'm not sure what other information to provide, here's the relevant log and a doveconf -n. Second question, in the doveconf -n there's reference to my ssl_cipher am I using current tls ciphers that support pfs? Thanks. Dave. Oct 29 20:51:21 server dovecot: imap-login: Login: user=, method=PLAIN, rip=::1, lip=::1, mpid=71405, secured, session=<6Px600cja6cAAAAAAAAAAAAAAAAAAAAB> Oct 29 20:51:21 server dovecot: imap(xxx): Disconnected: Logged out in=82 out=763 Oct 29 20:51:22 server dovecot: imap-login: Login: user=, method=PLAIN, rip=::1, lip=::1, mpid=72189, secured, session= Oct 29 20:51:22 server dovecot: imap(xxx): Disconnected: Logged out in=70 out=932 Oct 29 20:51:29 server dovecot: imap-login: Login: user=, method=PLAIN, rip=::1, lip=::1, mpid=74281, secured, session= Oct 29 20:51:29 server dovecot: imap-login: Login: user=, method=PLAIN, rip=::1, lip=::1, mpid=74927, secured, session= Oct 29 20:51:29 server dovecot: imap(xxx): Disconnected: Logged out in=439 out=1702 Oct 29 20:51:29 server dovecot: imap(xxx): Disconnected: Logged out in=326 out=24327 Oct 29 20:51:45 server dovecot: imap-login: Login: user=, method=PLAIN, rip=::1, lip=::1, mpid=75557, secured, session=<3tjm1EcjsjUAAAAAAAAAAAAAAAAAAAAB> Oct 29 20:51:45 server dovecot: imap(xxx): Disconnected: Logged out in=32 out=521 Oct 29 20:51:46 server dovecot: imap-login: Login: user=, method=PLAIN, rip=::1, lip=::1, mpid=77051, secured, session= Oct 29 20:51:46 server dovecot: imap(xxx): Disconnected: Logged out in=44 out=799 doveconf -n # 2.2.19: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.9 (357ac0a0e68b+) # OS: FreeBSD 10.2-RELEASE amd64 ufs auth_default_realm = domain.com auth_mechanisms = plain login dict { sqlquota = mysql:/usr/local/etc/dovecot/dovecot-dict-sql.conf.ext } first_valid_gid = 999 first_valid_uid = 999 hostname = xxx at domain.com imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags last_valid_gid = 999 last_valid_uid = 999 lmtp_rcpt_check_quota = yes mail_gid = vmail mail_home = /home/vmail/%d/%n/home mail_location = maildir:/home/vmail/%d/%n:LAYOUT=fs mail_plugins = acl quota zlib mail_server_admin = mailto:postmaster at domain.com mail_uid = vmail mailbox_list_index = yes maildir_broken_filename_sizes = yes maildir_empty_new = yes maildir_stat_dirs = yes maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate namespace { list = yes location = maildir:/home/vmail/public:LAYOUT=fs prefix = public/ separator = / subscriptions = yes type = public } namespace inbox { hidden = no inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / subscriptions = yes type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::sqlquota quota_grace = 10%% quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO sieve_default = /home/vmail/conf.d/domain.com/sieve/default.sieve } postmaster_address = postmaster at domain.com protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { mode = 0600 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = } inet_listener imaps { ssl = yes } } service lmtp { unix_listener dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { address = 127.0.0.1 port = 12345 } } ssl = required ssl_cert = References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 29 Oct 2015, David Mehler wrote: > I've seen this issue before, running a imap/smtp/database server on > localhost and adding in a webmail interface, in this case Roundcube. > > In my maillog I'm seeing accessive Dovecot connections and logouts > just from my own transaction of logging in, going to compose a > message, sending, and logging out. > > I'm using Mysql as database backend and was wondering if there was > something I could do to cut down on the amount of connections needed? :-) don't connect so often. I guess with "from my own transaction" you mean a transaction in roundcube? http://trac.roundcube.net/wiki/Howto_Config/Performance "Use a caching IMAP proxy" - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVjMb+3z1H7kL/d9rAQIf7wgAuX4CLKqlScNUhbIdVLGHWjkJbdEsGlds TYvNvgMSzV/ga2yCZvXBQuSJgsAmPcw0u5r8cQ/giOg6RD8JNhDarCzW0EVsjcoh NwKv/WuRpMS5dOw5F3WWgtJnP55upulYN4EvmYilxuO9XGsVYdzmR/TlXae/Urw9 n31YLVrMkE7DzSm5JSUhUBOKT+u3yCLdQ7MYzDeH9G1Tt6el3ZOZSOWRhgwSngTz cwuPM84Q6kxifxdMibfMF8CwX2RwBbigstheQaJQH6l6vypvyjT7WMO3XRKxkBfZ MIQCoI6nv1iYArN3Cdlt25IfZEucTvzRRaYD86LtA9z3H4X+YXy8FA== =bbSN -----END PGP SIGNATURE----- From goetz.reinicke at filmakademie.de Fri Oct 30 10:18:01 2015 From: goetz.reinicke at filmakademie.de (=?UTF-8?Q?G=c3=b6tz_Reinicke_-_IT_Koordinator?=) Date: Fri, 30 Oct 2015 11:18:01 +0100 Subject: updating and wsitching repo to yum.dovecot.fi - Unknown protocol: sieve Message-ID: <563343D9.7040108@filmakademie.de> Hi, winter is coming and so I start to clean up some left overs of the year. One thing is to use the yum.dovecot.fi repository. After installing the current availabel dovecot and dovecot-ee-pigeonhole package and restarting dovecot I do get the error: doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf: protocols: Unknown protocol: sieve Is the sieve protocol an extra package? I thought in the 2.2. tree I dont have to do bigger config changes. We run already 2.2.something from city-fan.org and the switch was also the idea of going to the most recent release. Thanks for hints and feedback . G?tz -- G?tz Reinicke IT-Koordinator Tel. +49 7141 969 82420 E-Mail goetz.reinicke at filmakademie.de Filmakademie Baden-W?rttemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: J?rgen Walter MdL Staatssekret?r im Ministerium f?r Wissenschaft, Forschung und Kunst Baden-W?rttemberg Gesch?ftsf?hrer: Prof. Thomas Schadt -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5501 bytes Desc: S/MIME Cryptographic Signature URL: From teemu.huovila at dovecot.fi Fri Oct 30 10:49:09 2015 From: teemu.huovila at dovecot.fi (Teemu Huovila) Date: Fri, 30 Oct 2015 12:49:09 +0200 Subject: updating and wsitching repo to yum.dovecot.fi - Unknown protocol: sieve In-Reply-To: <563343D9.7040108@filmakademie.de> References: <563343D9.7040108@filmakademie.de> Message-ID: <56334B25.2060004@dovecot.fi> On 30.10.2015 12:18, G?tz Reinicke - IT Koordinator wrote: > Hi, > > winter is coming and so I start to clean up some left overs of the year. > > One thing is to use the yum.dovecot.fi repository. > > After installing the current availabel dovecot and dovecot-ee-pigeonhole > package and restarting dovecot I do get the error: > > > doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf: > protocols: Unknown protocol: sieve Could you please reply with the output of doveconf -n > > > Is the sieve protocol an extra package? I thought in the 2.2. tree I > dont have to do bigger config changes. > > We run already 2.2.something from city-fan.org and the switch was also > the idea of going to the most recent release. > > > Thanks for hints and feedback . G?tz > From vitalif at yourcmc.ru Fri Oct 30 10:51:45 2015 From: vitalif at yourcmc.ru (vitalif at yourcmc.ru) Date: Fri, 30 Oct 2015 13:51:45 +0300 Subject: Dovecot frequently full-text reindexes the whole maildir In-Reply-To: References: <7ace0f02e5fc44b337e9b12595d7434b@yourcmc.ru> Message-ID: Thanks, after rewriting anti-spam scripts to doveadm and changing the search engine to fts-lucene fulltext indexing started to work reliably (fts-squat has constantly corrupted its index files, but lucene is working OK). Now I have another small question: 'doveadm expunge' (even with -v) does not print any errors, but always returns exit code 68 in my case. What does it mean? I would like to get zero exit code in case of success... > Hi, > > I'm not an expert, but am doing something similar (moving messages and > training the spam filter). > > To detect the existence of new messages looking directly at the > maildir can be a quick and dirty solution to get a trigger event. > > But to move or delete I think you should use 'doveadm' to search for > all messages in your spam directory and get a list of all uids an > guids. Then you can call doveadm again to move vor delete the messages > by id. > > Just look for "doveadm search/move/purge" in the dovecot wiki. > This should ensure that the indexes stay up to date. > > Am 27. Oktober 2015 11:24:39 MEZ, schrieb vitalif at yourcmc.ru: >> Hi! >> >> I'm using Dovecot 2.2.13 (Debian Jessie package 1:2.2.13-12~deb8u1) on >> my personal mail server (the address I'm writing from is on this >> server). >> >> I use Maildirs, I have fts + fts_squat enabled, and I have a problem >> with it for a long time - dovecot seems to not update the index always >> "incrementally". >> >> Yesterday I've finally made a test by telneting to imap port and >> issuing >> a search request. Dovecot started to reindex the mailbox. Full >> reindexing has taken around 15-20 minutes. >> >> After reindexing the fulltext searches were fast, just as they should >> be. And they were fast yesterday even after some new emails arrived. >> >> But I've tried to do a new search today and Dovecot started to reindex >> the whole maildir again! >> >> Why is it reindexing the whole maildir again? Could it be related to >> my >> >> anti-spam scripts removing messages from SPAM imap folder? And if yes, >> how to remove them correctly? >> -- With best regards, Vitaliy Filippov From sca at andreasschulze.de Fri Oct 30 13:16:01 2015 From: sca at andreasschulze.de (A. Schulze) Date: Fri, 30 Oct 2015 14:16:01 +0100 Subject: Webmail accessive Dovecot logins In-Reply-To: Message-ID: <20151030141601.Horde.7RfYs5EIjiqKU43uxTDg3HV@andreasschulze.de> David Mehler: > Second question, in the doveconf -n there's reference to my ssl_cipher > am I using current tls ciphers that support pfs? > ssl_cipher_list = ALL:!LOW:!SSLv3:!SSLv2:!EXP:!aNULL some non pfs cipher would be still active. check yourself: # openssl ciphers -v 'ALL:!LOW:!SSLv3:!SSLv2:!EXP:!aNULL' | grep -v DH you disable the SSLv3 *cipher list* here. That's may be not the expected result. The *cipher list* SSLv3 is also used by TLSv1+ *protocols*. I suggest reading (again?) https://bettercrypto.org/static/applied-crypto-hardening.pdf > ssl_protocols = TLSv1 !SSLv3 !SSLv2 that disable SSLv2, SSLv3 but would also disable TLSv1.1 and TLSv1.2 your ssl library may support otherwise. better: ssl_protocols = !SSLv3 !SSLv2 finally you could use the service provided by ssllabs.com to scan your host. It's a little bit tricky. ssllabs allow only to scan on port 443 So you may configure your host for imaps ( not imap + STARTTLS ) on port 443 and use ssllabs.com. But that require you do not run a regular HTTPS webserver on the same host. Andreas From goetz.reinicke at filmakademie.de Fri Oct 30 13:35:11 2015 From: goetz.reinicke at filmakademie.de (=?UTF-8?Q?G=c3=b6tz_Reinicke_-_IT_Koordinator?=) Date: Fri, 30 Oct 2015 14:35:11 +0100 Subject: updating and wsitching repo to yum.dovecot.fi - Unknown protocol: sieve In-Reply-To: <56334B25.2060004@dovecot.fi> References: <563343D9.7040108@filmakademie.de> <56334B25.2060004@dovecot.fi> Message-ID: <5633720F.1030604@filmakademie.de> Am 30.10.15 um 11:49 schrieb Teemu Huovila: > > > On 30.10.2015 12:18, G?tz Reinicke - IT Koordinator wrote: >> Hi, >> >> winter is coming and so I start to clean up some left overs of the year. >> >> One thing is to use the yum.dovecot.fi repository. >> >> After installing the current availabel dovecot and dovecot-ee-pigeonhole >> package and restarting dovecot I do get the error: >> >> >> doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf: >> protocols: Unknown protocol: sieve > Could you please reply with the output of doveconf -n my guess: in the currently used rpms the "managesieve" libs are included; for the official dovecot repo I do have to install the dovecot-ee-managesieve.rpm too... # 2.2.18.2 (866bffbafde7): /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-371.6.1.el5xen x86_64 CentOS release 5.11 (Final) auth_debug = yes auth_master_user_separator = * auth_mechanisms = plain login auth_verbose = yes default_client_limit = 4000 default_process_limit = 4000 disable_plaintext_auth = no log_path = /var/log/dovecot.log login_trusted_networks = 193.196.129.21 mail_debug = yes mail_location = maildir:~/Maildir mail_plugins = mail_log notify quota acl mail_privileged_group = mail mdbox_rotate_size = 10 M namespace { list = children location = maildir:%%h/Maildir:INDEX=%h/shared/%%u:CONTROL=%h/shared/%%u prefix = shared/%%u/ separator = / subscriptions = yes type = shared } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes quota = dict:User quota::noenforcing:file:%h/dovecot-quota quota_rule = *:storage=5G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at filmakademie.de protocols = imap pop3 lmtp sieve sieve quota_full_tempfail = yes service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { group = vmail user = vmail } user = root } service imap-login { process_limit = 1024 process_min_avail = 16 service_count = 0 } service imap { process_limit = 1024 } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 } service managesieve { process_limit = 1024 } service pop3-login { process_limit = 1024 process_min_avail = 16 service_count = 0 } service pop3 { process_limit = 1024 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = dovecot } ssl_ca = From teemu.huovila at dovecot.fi Fri Oct 30 14:44:59 2015 From: teemu.huovila at dovecot.fi (Teemu Huovila) Date: Fri, 30 Oct 2015 16:44:59 +0200 Subject: updating and wsitching repo to yum.dovecot.fi - Unknown protocol: sieve In-Reply-To: <5633720F.1030604@filmakademie.de> References: <563343D9.7040108@filmakademie.de> <56334B25.2060004@dovecot.fi> <5633720F.1030604@filmakademie.de> Message-ID: <5633826B.1070701@dovecot.fi> On 30.10.2015 15:35, G?tz Reinicke - IT Koordinator wrote: > Am 30.10.15 um 11:49 schrieb Teemu Huovila: >> >> >> On 30.10.2015 12:18, G?tz Reinicke - IT Koordinator wrote: >>> Hi, >>> >>> winter is coming and so I start to clean up some left overs of the year. >>> >>> One thing is to use the yum.dovecot.fi repository. >>> >>> After installing the current availabel dovecot and dovecot-ee-pigeonhole >>> package and restarting dovecot I do get the error: >>> >>> >>> doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf: >>> protocols: Unknown protocol: sieve >> Could you please reply with the output of doveconf -n > > > my guess: in the currently used rpms the "managesieve" libs are > included; for the official dovecot repo I do have to install the > dovecot-ee-managesieve.rpm too... Yes, if you have "protocols = sieve .." then you need the managesieve package too. Also, to use sieve filtering, you need to load the sieve plugin in mail_plugins for lmtp or lda. Please refer to http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration http://wiki2.dovecot.org/Pigeonhole/ManageSieve/Configuration Teemu > > # 2.2.18.2 (866bffbafde7): /etc/dovecot/dovecot.conf > # OS: Linux 2.6.18-371.6.1.el5xen x86_64 CentOS release 5.11 (Final) > auth_debug = yes > auth_master_user_separator = * > auth_mechanisms = plain login > auth_verbose = yes > default_client_limit = 4000 > default_process_limit = 4000 > disable_plaintext_auth = no > log_path = /var/log/dovecot.log > login_trusted_networks = 193.196.129.21 > mail_debug = yes > mail_location = maildir:~/Maildir > mail_plugins = mail_log notify quota acl > mail_privileged_group = mail > mdbox_rotate_size = 10 M > namespace { > list = children > location = maildir:%%h/Maildir:INDEX=%h/shared/%%u:CONTROL=%h/shared/%%u > prefix = shared/%%u/ > separator = / > subscriptions = yes > type = shared > } > namespace inbox { > inbox = yes > location = > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > prefix = > separator = / > } > passdb { > args = /etc/dovecot/master-users > driver = passwd-file > master = yes > } > passdb { > args = /etc/dovecot/dovecot-ldap.conf.ext > driver = ldap > } > plugin { > acl = vfile > acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes > quota = dict:User quota::noenforcing:file:%h/dovecot-quota > quota_rule = *:storage=5G > quota_rule2 = Trash:storage=+100M > quota_warning = storage=95%% quota-warning 95 %u > quota_warning2 = storage=80%% quota-warning 80 %u > sieve = ~/.dovecot.sieve > sieve_dir = ~/sieve > } > postmaster_address = postmaster at filmakademie.de > protocols = imap pop3 lmtp sieve sieve > quota_full_tempfail = yes > service auth { > unix_listener /var/spool/postfix/private/auth { > mode = 0666 > } > unix_listener auth-userdb { > group = vmail > user = vmail > } > user = root > } > service imap-login { > process_limit = 1024 > process_min_avail = 16 > service_count = 0 > } > service imap { > process_limit = 1024 > } > service lmtp { > inet_listener lmtp { > address = 127.0.0.1 > port = 24 > } > } > service managesieve-login { > inet_listener sieve { > port = 4190 > } > service_count = 1 > } > service managesieve { > process_limit = 1024 > } > service pop3-login { > process_limit = 1024 > process_min_avail = 16 > service_count = 0 > } > service pop3 { > process_limit = 1024 > } > service quota-warning { > executable = script /usr/local/bin/quota-warning.sh > unix_listener quota-warning { > user = vmail > } > user = dovecot > } > ssl_ca = ssl_cert = ssl_cipher_list = > DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ALL:!LOW:!SSLv2:!EXP:!aNULL > ssl_key = ssl_prefer_server_ciphers = yes > userdb { > args = /etc/dovecot/dovecot-ldap.conf.ext > driver = ldap > } > verbose_proctitle = yes > doveconf: Error: protocols: Unknown protocol: sieve > protocol lmtp { > info_log_path = /var/log/dovecot-lmtp.log > log_path = /var/log/dovecot-lmtp-errors.log > mail_plugins = mail_log notify quota acl sieve > } > protocol imap { > mail_max_userip_connections = 20 > mail_plugins = mail_log notify quota acl imap_zlib imap_quota imap_acl > } > protocol pop3 { > mail_max_userip_connections = 20 > } > doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf: > protocols: Unknown protocol: sieve > > From goetz.reinicke at filmakademie.de Fri Oct 30 14:56:07 2015 From: goetz.reinicke at filmakademie.de (=?UTF-8?Q?G=c3=b6tz_Reinicke_-_IT_Koordinator?=) Date: Fri, 30 Oct 2015 15:56:07 +0100 Subject: updating and wsitching repo to yum.dovecot.fi - Unknown protocol: sieve In-Reply-To: <5633826B.1070701@dovecot.fi> References: <563343D9.7040108@filmakademie.de> <56334B25.2060004@dovecot.fi> <5633720F.1030604@filmakademie.de> <5633826B.1070701@dovecot.fi> Message-ID: <56338507.9050409@filmakademie.de> Am 30.10.15 um 15:44 schrieb Teemu Huovila: > > > On 30.10.2015 15:35, G?tz Reinicke - IT Koordinator wrote: >> Am 30.10.15 um 11:49 schrieb Teemu Huovila: >>> >>> >>> On 30.10.2015 12:18, G?tz Reinicke - IT Koordinator wrote: >>>> Hi, >>>> >>>> winter is coming and so I start to clean up some left overs of the year. >>>> >>>> One thing is to use the yum.dovecot.fi repository. >>>> >>>> After installing the current availabel dovecot and dovecot-ee-pigeonhole >>>> package and restarting dovecot I do get the error: >>>> >>>> >>>> doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf: >>>> protocols: Unknown protocol: sieve >>> Could you please reply with the output of doveconf -n >> >> >> my guess: in the currently used rpms the "managesieve" libs are >> included; for the official dovecot repo I do have to install the >> dovecot-ee-managesieve.rpm too... > Yes, if you have "protocols = sieve .." then you need the managesieve package too. Also, to use sieve filtering, you need to load the sieve plugin in mail_plugins for lmtp or lda. Please refer to > http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration > http://wiki2.dovecot.org/Pigeonhole/ManageSieve/Configuration Hi, currently sieve filtering is working (sieve plugin in mail_plugins for lmtp is enabled) with the old repo packages; do I have to install an other something-plugin.rpm from the dovecot repo? Thanks & Regards . G?tz -- G?tz Reinicke IT-Koordinator Tel. +49 7141 969 82420 E-Mail goetz.reinicke at filmakademie.de Filmakademie Baden-W?rttemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: J?rgen Walter MdL Staatssekret?r im Ministerium f?r Wissenschaft, Forschung und Kunst Baden-W?rttemberg Gesch?ftsf?hrer: Prof. Thomas Schadt -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5501 bytes Desc: S/MIME Cryptographic Signature URL: From bytesplit at gmail.com Fri Oct 30 17:00:08 2015 From: bytesplit at gmail.com (Philon) Date: Fri, 30 Oct 2015 18:00:08 +0100 Subject: updating and wsitching repo to yum.dovecot.fi - Unknown protocol: sieve In-Reply-To: <56338507.9050409@filmakademie.de> References: <563343D9.7040108@filmakademie.de> <56334B25.2060004@dovecot.fi> <5633720F.1030604@filmakademie.de> <5633826B.1070701@dovecot.fi> <56338507.9050409@filmakademie.de> Message-ID: <386412FC-693E-4BF6-8FD3-0A3AC274E9F1@googlemail.com> Hi G?tz, for my debian repository at least I find these packages: dovecot-ee-sieve - sieve filters support for Dovecot dovecot-ee-managesieved - secure ManageSieve server for Dovecot Could be that your third party yum package had these included in the default build. Gr??e aus T?bingen Philon > Am 30.10.2015 um 15:56 schrieb G?tz Reinicke - IT Koordinator : > > Am 30.10.15 um 15:44 schrieb Teemu Huovila: >> >> >> On 30.10.2015 15:35, G?tz Reinicke - IT Koordinator wrote: >>> Am 30.10.15 um 11:49 schrieb Teemu Huovila: >>>> >>>> >>>> On 30.10.2015 12:18, G?tz Reinicke - IT Koordinator wrote: >>>>> Hi, >>>>> >>>>> winter is coming and so I start to clean up some left overs of the year. >>>>> >>>>> One thing is to use the yum.dovecot.fi repository. >>>>> >>>>> After installing the current availabel dovecot and dovecot-ee-pigeonhole >>>>> package and restarting dovecot I do get the error: >>>>> >>>>> >>>>> doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf: >>>>> protocols: Unknown protocol: sieve >>>> Could you please reply with the output of doveconf -n >>> >>> >>> my guess: in the currently used rpms the "managesieve" libs are >>> included; for the official dovecot repo I do have to install the >>> dovecot-ee-managesieve.rpm too... >> Yes, if you have "protocols = sieve .." then you need the managesieve package too. Also, to use sieve filtering, you need to load the sieve plugin in mail_plugins for lmtp or lda. Please refer to >> http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration >> http://wiki2.dovecot.org/Pigeonhole/ManageSieve/Configuration > > Hi, > > currently sieve filtering is working (sieve plugin in mail_plugins for > lmtp is enabled) with the old repo packages; do I have to install an > other something-plugin.rpm from the dovecot repo? > > Thanks & Regards . G?tz > > -- > G?tz Reinicke > IT-Koordinator > > Tel. +49 7141 969 82420 > E-Mail goetz.reinicke at filmakademie.de > > Filmakademie Baden-W?rttemberg GmbH > Akademiehof 10 > 71638 Ludwigsburg > www.filmakademie.de > > Eintragung Amtsgericht Stuttgart HRB 205016 > > Vorsitzender des Aufsichtsrats: J?rgen Walter MdL > Staatssekret?r im Ministerium f?r Wissenschaft, > Forschung und Kunst Baden-W?rttemberg > > Gesch?ftsf?hrer: Prof. Thomas Schadt From jtam.home at gmail.com Fri Oct 30 20:23:04 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Fri, 30 Oct 2015 13:23:04 -0700 (PDT) Subject: Webmail accessive Dovecot logins In-Reply-To: References: Message-ID: "A. Schulze" writes: > David Mehler: > >> Second question, in the doveconf -n there's reference to my ssl_cipher >> am I using current tls ciphers that support pfs? > >> ssl_cipher_list = ALL:!LOW:!SSLv3:!SSLv2:!EXP:!aNULL > > some non pfs cipher would be still active. check yourself: > # openssl ciphers -v 'ALL:!LOW:!SSLv3:!SSLv2:!EXP:!aNULL' | grep -v DH You'll want the 'E' variation (ephemeral) of the DH algorithms, and preferably, the ECDHE variety as they are faster and supported on more browsers. The pattern to search for (or exclude) is "DHE" openssl ciphers -v {cipher-specs} | grep DHE If the OP wants preferentially use PFS ciphers (but keep the other ciphers around for very old browsers), maybe something like ssl_cipher_list = ECDH:ALL:!LOW:!SSLv2:!EXP:!aNULL ssl_prefer_server_ciphers = yes > finally you could use the service provided by ssllabs.com to scan your host. I second this recommendation, if you can work out the port issue. Maybe using a ncat | ncat pipe. Joseph Tam From lists at tigertech.com Fri Oct 30 22:06:56 2015 From: lists at tigertech.com (Robert L Mathews) Date: Fri, 30 Oct 2015 15:06:56 -0700 Subject: IMAP COPY creates invalid index data with zlib and mail_log plugins enabled but zlib_save off Message-ID: <5633EA00.2080603@tigertech.com> I've noticed that maildir IMAP COPY commands can generate invalid dovecot.index entries when all seven of the following are true: - The zlib plugin is enabled; - The zlib_save/zlib_save_level options are NOT enabled; - The source message being copied is compressed; - The mail_log plugin is logging "copy" events; - The mail_log_fields setting includes at least one message header; - The destination mailbox folder has an index file that is recording the logged headers; - The source mailbox folder does NOT have an index file recording the logged headers. In this situation, copying a message results in an index entry in the destination that has all the headers blank, like this: RECORD: seq=5, uid=182415, flags=0x08 (Seen) - ext 1 modseq : 3 (0300000000000000) - ext 2 cache : 2352 (30090000) - cache offset=2352 size=120, prev_offset = 0 - hdr.FROM: - hdr.MESSAGE-ID: - hdr.SUBJECT: - hdr.DATE: - hdr.BCC: - hdr.CC: - hdr.CONTENT-TYPE: - hdr.IN-REPLY-TO: - hdr.NEWSGROUPS: - hdr.PRIORITY: - hdr.REFERENCES: - hdr.REPLY-TO: - hdr.TO: - hdr.X-PRIORITY: For these copies, the mail_log plugin records blank header data in the log, too, like this (despite the message having a valid message-ID, from, and subject): Oct 29 21:11:42 wheezy dovecot: imap[31763]: user=<1 at example.com> ip=192.168.0.4 session=RV/MmUojyADAqAAE copy from INBOX.Saved: box=INBOX.Trash, uid=182412, msgid=, size=1357, vsize=1392, from=, subject=, flags=(\Seen) I've found this occurs in at least Dovecot 2.1.7, 2.2.13, and 2.2.18 (all packaged Debian versions). Changing any of the seven things I mentioned above fixes it. I've tried to find the code that causes this, but not yet been successful. My theory is that when the mail_log plugin calls mail_get_first_header(), and there's no index header record to read it from, it looks at the real message on disk. Perhaps it somehow operates on a compressed copy of the message although it expects uncompressed plaintext, so it doesn't find any of the headers. The "missing" headers then get cached to the destination index file as a side-effect. The copied maildir message itself is fine, by the way. It's an identical, still-compressed copy of the original message. And deleting the destination "dovecot.index*" files that contain the bad data causes Dovecot to rebuild the index with the correct header data. It's unusual to use the zlib plugin without the zlib_save/zlib_save_level options enabled, of course. That's probably why (as far as I can tell) there have been no reports of this before. I noticed it only because I was testing a small amount of manual mail compression before deploying it for all mail. Here's the "doveconf -n" of a system in the "has the problem" state: # 2.2.18: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.8 (0c4ae064f307+) # OS: Linux 3.2.0-4-686-pae i686 Debian stretch/sid auth_mechanisms = plain login auth_username_format = auth_verbose = yes auth_verbose_passwords = sha1 default_vsz_limit = 1 G disable_plaintext_auth = no lda_mailbox_autosubscribe = yes listen = * log_timestamp = "%Y-%m-%d %H:%M:%S " login_log_format_elements = user=<%u> pid=[%{mail_pid}] method=%m rip=%r lip=%l %c mail_fsync = never mail_location = maildir:~/ mail_log_prefix = "%s[%{pid}]: user=<%u> ip=%{rip} session=%{session} " mail_max_userip_connections = 100 mail_plugins = mail_log notify zlib managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate spamtest spamtestplus namespace { inbox = yes location = prefix = INBOX. separator = . type = private } namespace { hidden = yes inbox = no list = no location = prefix = separator = . type = private } passdb { args = /etc/dovecot/dovecot.users driver = passwd-file } plugin { fts = lucene fts_lucene = whitespace_chars=@. mail_log_events = copy delete undelete expunge mailbox_delete mailbox_rename flag_change save mailbox_create mail_log_fields = vsize size flags box uid box msgid size flags vsize from subject sieve = %h/mailbox.sieve sieve_after = %h/../../domain-after.sieve sieve_before = %h/../../domain-before.sieve sieve_dir = %h/sieve sieve_extensions = -reject +spamtest +spamtestplus sieve_global_dir = %h/../../sieve-global-include-scripts sieve_spamtest_max_value = 7 sieve_spamtest_status_header = X-Spam-Level sieve_spamtest_status_type = strlen } protocols = pop3 imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { process_min_avail = 8 service_count = 0 } service imap { process_limit = 2000 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service pop3-login { process_min_avail = 8 service_count = 0 } service pop3 { process_limit = 2000 } ssl = no userdb { args = /etc/dovecot/dovecot.users driver = passwd-file } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail imap_logout_format = %i bytes in, %o bytes out } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_logout_format = ; %i bytes in, %o bytes out, %t top (%p bytes), %r retr (%b bytes), %d of %m deleted, mailbox size %s bytes pop3_uidl_format = UID%u-%v } protocol lda { mail_fsync = optimized mail_plugins = mail_log notify zlib sieve } protocol lmtp { mail_fsync = optimized mail_plugins = mail_log notify zlib sieve } -- Robert L Mathews, Tiger Technologies, http://www.tigertech.net/ From brian.brady1982 at gmail.com Sat Oct 31 05:40:33 2015 From: brian.brady1982 at gmail.com (brian) Date: Sat, 31 Oct 2015 16:40:33 +1100 Subject: Initialization Failed: Message-ID: <56345451.9040304@gmail.com> Hi dovecotters I am trying to install dovecot on a mail server I am provisioning for myself in AWS. I am following an ArsTechnica guide, and at some stage in the process the author recommends and provides instructions to configure Dovecot (http://arstechnica.com/information-technology/2014/03/taking-e-mail-back-part-2-arming-your-server-with-postfix-dovecot/4/) So, now I have it installed and configured (I believe the way suggested) but when I try to connect via IMAP I am getting an error. If I try : root at ip-10-0-0-244:~# telnet mail.myactualdomain.com 143 Trying 127.0.0.1... Connected to mail.myactualdomain.com. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot (Ubuntu) ready. a1 LOGIN brian at myactualdomain.com plaintextpassword * BYE Internal error occurred. Refer to server log for more information. Connection closed by foreign host. root at ip-10-0-0-244:~# If I go to /var/log/syslog I can see the following lines: Oct 31 05:31:06 ip-10-0-0-244 dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=16934, secured, session= Oct 31 05:31:06 ip-10-0-0-244 dovecot: imap(brian at myactualdomain.com): Error: user brian at myactualdomain.com: Initialization failed: Namespace '': Unknown setting: /var/mail/vmail/myactualdomain.com/brian/mail Oct 31 05:31:06 ip-10-0-0-244 dovecot: imap(brian at myactualdomain.com): Error: Invalid user settings. Refer to server log for more information. root at ip-10-0-0-244:~# dovecot --version 2.2.9 So, I don't know why it's giving me an error about : Initialization failed: Namespace '': Unknown setting: /var/mail/vmail/myactualdomain.com/brian/mail The location is there, with the permissions with I believe the correct permissions. I'm a bit stuck as to where to even go from here. Any help would be greatly appreciated. Kind Regards Brian root at ip-10-0-0-244:~# dovecot -n # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.13.0-48-generic x86_64 Ubuntu 14.04.2 LTS auth_mechanisms = plain login auth_verbose = yes mail_home = /var/mail/vmail/%d/%n mail_location = maildir:~/Maildir:/var/mail/vmail/%d/%n/mail:LAYOUT=fs managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = username_format=%u scheme=ssha512 /etc/dovecot/passwd.db driver = passwd-file } plugin { sieve = ~/.dovecot.sieve sieve_after = /var/mail/vmail/sieve-after sieve_before = /var/mail/vmail/sieve-before sieve_dir = ~/sieve } protocols = imap sieve service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } ssl_cert = was automatically rejected:%n%r } protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_max_userip_connections = 10 } From ml+dovecot at valo.at Sat Oct 31 07:08:21 2015 From: ml+dovecot at valo.at (Christian Kivalo) Date: Sat, 31 Oct 2015 08:08:21 +0100 Subject: Initialization Failed: In-Reply-To: <56345451.9040304@gmail.com> References: <56345451.9040304@gmail.com> Message-ID: Hi, >Error: user brian at myactualdomain.com: Initialization failed: Namespace >'': Unknown setting: /var/mail/vmail/myactualdomain.com/brian/mail .... >mail_location = maildir:~/Maildir:/var/mail/vmail/%d/%n/mail:LAYOUT=fs You have double "maildir:" keyword. Regards Christian From filippo.carletti at gmail.com Sat Oct 31 10:08:35 2015 From: filippo.carletti at gmail.com (Filippo Carletti) Date: Sat, 31 Oct 2015 11:08:35 +0100 Subject: Dovecot, JavaMail, UIDs and Message Numbers Message-ID: We're using dovecot-2.1.16 on CentOS (rpm package from epel repo). Maildir format on ext4. > What exactly do you mean by "Dovecot started reindexing"? Selecting thread sort on the javamail client, the imap process opens all files in the folder and updates the following files on disk: dovecot.index dovecot.index.cache dovecot.index.log dovecot.index.log.2 dovecot.index.thread The folder has around 5000 mails and the operation takes some minutes (hw is not the bottleneck). Selecting thread view on Thunderbird or Roundcube takes less than a second, I can't see the imap process doing anything. > Dovecot's normal indexes or full text search indexes? Normal indexes. > How have you configured everything? What do you need to know in particular? doveconf output? Maildir infos: # time cat cur/* >/dev/null real 0m4.651s user 0m0.045s sys 0m0.141s # ls -f cur/ | wc -l 4170 # du -sh cur 49M cur Things go worse with a bigger maildir. > Does Dovecot log anything (especially any errors/warnings about anything)? No errors or warnings at all. I've enabled the mail_log plugin, but I think it can't log indexing operations. Thank you. -- Ciao, Filippo