June 2021 - dovecot - dovecot.org

Panic indexer-worker on Dovecot 2.3.15
by Alessio Cecchi 30 Jun '21

30 Jun '21

Hi, after the upgrade to dovecot 2.3.15 the Panic: indexer-worker error is still present. I don't know if is related to searching in a Virtual folder (look for my previous email on list with subject "Panic on indexer-worker and Dovecot stop to search on Solr" for more details). Here the relevant configuration of FTS plugin: plugin { fts = solr fts_autoindex = no fts_autoindex_exclude = \Junk fts_autoindex_exclude2 = \Drafts fts_decoder = decode2text fts_enforced = body fts_index_timeout = 5s fts_solr = url=http://10.0.0.32:8983/solr/dovecot/ [...] I'm running Dovecot with FTS Solr 7.7, here the log error and the bt full: Jun 29 13:06:26 Panic: indexer-worker(user1(a)company.net) session=<SLTLmOXFFIy5Ydlu:6BC4HLH+2mBHQwAAxbYlJw>: file http-clien t-request.c: line 1240 (http_client_request_send_more): assertion failed: (req->payload_input != NULL) Jun 29 13:06:26 Error: indexer-worker(user1(a)company.net) session=<SLTLmOXFFIy5Ydlu:6BC4HLH+2mBHQwAAxbYlJw>: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7f7212b36862] -> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f7212b3696e] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf50fe) [0x7f7212b440fe] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf51a1) [ 0x7f7212b441a1] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f7212a9460c] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_ request_send_more+0x3dd) [0x7f7212add54d] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_connection_output+0xf1) [0x7f7212ae1ca1] -> /usr/lib64/dovecot/libdovecot.so.0(+0x11e870) [0x7f7212b6d870] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x65) [0x7f 7212b5c425] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x12b) [0x7f7212b5ddab] -> /usr/lib64/dovecot/libdove cot.so.0(io_loop_handler_run+0x59) [0x7f7212b5c529] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f7212b5c768] -> /usr /lib64/dovecot/libdovecot.so.0(http_client_wait+0xcd) [0x7f7212aeadbd] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(solr_connectio n_select+0xe5) [0x7f7210a79265] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x4684) [0x7f7210a75684] -> /usr/lib64/dovecot/lib20 _fts_plugin.so(fts_backend_get_last_uid+0x6e) [0x7f7211e1944e] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xf9d2) [0x7f7211e1f9d2] - > /usr/lib64/dovecot/lib20_fts_plugin.so(+0x10f56) [0x7f7211e20f56] -> /usr/lib64/dovecot/lib20_virtual_plugin.so(+0x94da) [0x7f7211 a014da] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0x10ba6) [0x7f7211e20ba6] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_prec ache+0x2e) [0x7f7212e48fae] -> dovecot/indexer-worker [user1(a)company.net Virtual/All](+0x2bb1) [0x55559e522bb1] -> /usr/lib 64/dovecot/libdovecot.so.0(io_loop_call_io+0x65) [0x7f7212b5c425] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal +0x12b) [0x7f7212b5ddab] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x59) [0x7f7212b5c529] -> /usr/lib64/dovecot/libd ovecot.so.0(io_loop_run+0x38) [0x7f7212b5c768] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f7212ac63c3] -> do vecot/indexer-worker [user1(a)company.net Virtual/All](main+0xd7) [0x55559e522437] -> /lib64/libc.so.6(__libc_start_main+0xf5 ) [0x7f72126a3555] -> dovecot/indexer-worker [user1(a)company.net Virtual/All](+0x24fa) [0x55559e5224fa] Jun 29 13:06:26 Fatal: indexer-worker(user1(a)company.net) session=<SLTLmOXFFIy5Ydlu:6BC4HLH+2mBHQwAAxbYlJw>: master: service(indexer-worker): child 17223 killed with signal 6 (core dumped) # gdb /usr/libexec/dovecot/indexer-worker /var/core/core.indexer-worker.17223 GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-120.el7 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/libexec/dovecot/indexer-worker...Reading symbols from /usr/lib/debug/usr/libexec/dovecot/indexer-worker.debug...done. done. [New LWP 17223] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `dovecot/indexer-worker'. Program terminated with signal 6, Aborted. #0 0x00007f72126b7387 in raise () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-13.el7.x86_64 expat-2.1.0-12.el7.x86_64 glibc-2.17-324.el7_9.x86_64 libgcc-4.8.5-44.el7.x86_64 libzstd-1.5.0-1.el7.x86_64 xz-libs-5.2.2-1.el7.x86_64 zlib-1.2.7-19.el7_9.x86_64 (gdb) bt full #0 0x00007f72126b7387 in raise () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f72126b8a78 in abort () from /lib64/libc.so.6 No symbol table info available. #2 0x00007f7212b440b7 in default_fatal_finish (status=0, type=<optimized out>) at failures.c:459 backtrace = 0x55559eb75970 "/usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7f7212b36862] -> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f7212b3696e] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf50fe) [0"... recursed = 0 #3 fatal_handler_real (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at failures.c:471 status = 0 #4 0x00007f7212b441a1 in i_internal_fatal_handler (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at failures.c:866 No locals. #5 0x00007f7212a9460c in i_panic (format=format@entry=0x7f7212b85cf8 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:523 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0, timestamp_usecs = 0, log_prefix = 0x0, log_prefix_type_pos = 0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7ffe7b6e7150, reg_save_area = 0x7ffe7b6e7090}} #6 0x00007f7212add54d in http_client_request_send_more (req=req@entry=0x55559eca9db8, pipelined=pipelined@entry=false) at http-client-request.c:1240 conn = 0x55559ec8b750 cctx = 0x55559ebb63a8 output = 0x55559ecafa70 res = <optimized out> error = <optimized out> offset = <optimized out> __func__ = "http_client_request_send_more" #7 0x00007f7212ae1ca1 in http_client_connection_continue_request (conn=0x55559ec8b750) at http-client-connection.c:1342 tmp_conn = 0x55559ec8b750 count = 1 ret = <optimized out> reqs = <optimized out> req = 0x55559eca9db8 pipelined = false #8 http_client_connection_output (conn=0x55559ec8b750) at http-client-connection.c:1374 output = <optimized out> ret = <optimized out> #9 0x00007f7212b6d870 in stream_send_io (fstream=0x55559ec8cb20) at ostream-file.c:526 ostream = 0x55559ec8cbc0 am = 0x55559ec8cb20 use_cork = true ret = <optimized out> ---Type <return> to continue, or q <return> to quit--- #10 0x00007f7212b5c425 in io_loop_call_io (io=0x55559ec925f0) at ioloop.c:715 ioloop = 0x55559ecaf4d0 t_id = 6 __func__ = "io_loop_call_io" #11 0x00007f7212b5ddab in io_loop_handler_run_internal (ioloop=ioloop@entry=0x55559ecaf4d0) at ioloop-epoll.c:222 ctx = 0x55559ecab870 events = <optimized out> list = 0x55559ec8c8e0 io = <optimized out> tv = {tv_sec = 0, tv_usec = 0} events_count = <optimized out> msecs = <optimized out> ret = 1 i = 0 call = <optimized out> __func__ = "io_loop_handler_run_internal" #12 0x00007f7212b5c529 in io_loop_handler_run (ioloop=ioloop@entry=0x55559ecaf4d0) at ioloop.c:767 __func__ = "io_loop_handler_run" #13 0x00007f7212b5c768 in io_loop_run (ioloop=0x55559ecaf4d0) at ioloop.c:740 __func__ = "io_loop_run" #14 0x00007f7212aeadbd in http_client_wait () at http-client.c:339 prev_ioloop = 0x55559eb7b050 client_ioloop = 0x55559ecaf4d0 prev_client_ioloop = 0x55559eb7b050 __func__ = "http_client_wait" #15 0x00007f7210a79265 in solr_connection_select (conn=0x55559eb8b150, query=<optimized out>, pool=<optimized out>, box_results_r=0x7ffe7b6e7400) at solr-connection.c:211 lctx = {result_pool = 0x55559ec03910, payload = 0x0, io = 0x0, request_status = 0, parser = 0x0, results = 0x0} http_req = 0x55559ecb3b48 url = <optimized out> #16 0x00007f7210a75684 in fts_backend_solr_get_last_uid (_backend=0x55559eb8b090, box=0x55559ebe59a8, last_uid_r=0x7ffe7b6e74a4) at fts-backend-solr.c:238 hdr = {last_indexed_uid = 0, settings_checksum = 0, unused = 0} #17 0x00007f7211e1944e in fts_backend_get_last_uid (backend=<optimized out>, box=box@entry=0x55559ebe59a8, last_uid_r=last_uid_r@entry=0x7ffe7b6e74a4) at fts-api.c:106 hdr = {last_indexed_uid = 2, settings_checksum = 0, unused = 317606002} #18 0x00007f7211e1f9d2 in fts_mailbox_get_last_cached_seq (box=box@entry=0x55559ebe59a8, seq_r=seq_r@entry=0x7ffe7b6e74f4) at fts-storage.c:85 flist = <optimized out> seq1 = 21845 seq2 = 2663132384 last_uid = 21845 ---Type <return> to continue, or q <return> to quit--- #19 0x00007f7211e20f56 in fts_mail_precache_init (_mail=<optimized out>, _mail=<optimized out>) at fts-storage.c:478 ft = 0x55559ec8c820 flist = 0x55559ebca0a0 last_seq = 21845 #20 fts_mail_index (_mail=0x55559ecaed08) at fts-storage.c:501 ft = 0x55559ec8c820 flist = 0x55559ebca0a0 pmail = 0x55559ecaed08 #21 fts_mail_precache (_mail=0x55559ecaed08) at fts-storage.c:562 _data_stack_cur_id = 5 mail = 0x55559ecaed08 fmail = <optimized out> ft = 0x55559ec8c820 ret = 0 __func__ = "fts_mail_precache" #22 0x00007f7211a014da in virtual_mail_precache (mail=<optimized out>) at virtual-mail.c:243 vmail = <optimized out> backend_mail = 0x55559ecaed08 p = <optimized out> #23 0x00007f7211e20ba6 in fts_mail_precache (_mail=0x55559ec027f8) at fts-storage.c:555 mail = 0x55559ec027f8 fmail = 0x55559ec02f70 ft = 0x55559ebff760 ret = 0 __func__ = "fts_mail_precache" #24 0x00007f7212e48fae in mail_precache (mail=0x55559ec027f8) at mail.c:455 _data_stack_cur_id = 4 p = 0x55559ec027f8 ret = <optimized out> #25 0x000055559e522bb1 in index_mailbox_precache (conn=<optimized out>, box=0x55559eb9c498) at master-connection.c:119 counter = 1 max = 93 percentage_sent = 0 storage = <optimized out> status = {messages = 93, recent = 0, unseen = 0, uidvalidity = 1624964785, uidnext = 94, first_unseen_seq = 0, first_recent_uid = 94, last_cached_seq = 0, highest_modseq = 0, highest_pvt_modseq = 0, keywords = 0x0, permanent_flags = 0, flags = 0, permanent_keywords = false, allow_new_keywords = false, nonpermanent_modseqs = false, no_modseq_tracking = false, have_guids = true, have_save_guids = true, have_only_guid128 = false} uids = <optimized out> username = 0x55559eba0e18 "user1(a)company.net" first_uid = 1 percentage_str = "\261\376\332`" ---Type <return> to continue, or q <return> to quit--- percentage = <optimized out> error = MAIL_ERROR_NONE trans = 0x55559ebf1b00 ctx = 0x55559ebf3640 last_uid = 2 ret = 0 index_event = 0x55559ebf3250 box_vname = 0x55559eb9c830 "Virtual/All" errstr = <optimized out> search_args = 0x0 mail = 0x55559ec027f8 metadata = {guid = '\000' <repeats 15 times>, virtual_size = 0, physical_size = 0, first_save_date = 0, cache_fields = 0x0, precache_fields = (MAIL_FETCH_STREAM_HEADER | MAIL_FETCH_STREAM_BODY | MAIL_FETCH_RECEIVED_DATE | MAIL_FETCH_SAVE_DATE | MAIL_FETCH_PHYSICAL_SIZE | MAIL_FETCH_VIRTUAL_SIZE | MAIL_FETCH_UIDL_BACKEND | MAIL_FETCH_GUID | MAIL_FETCH_POP3_ORDER), backend_ns_prefix = 0x0, backend_ns_type = (unknown: 0)} seq = <optimized out> #26 index_mailbox (user=<optimized out>, user=<optimized out>, what=<optimized out>, max_recent_msgs=<optimized out>, mailbox=<optimized out>, conn=0x55559ebd8a80) at master-connection.c:238 box = 0x55559eb9c498 status = {messages = 2662839088, recent = 21845, unseen = 1, uidvalidity = 0, uidnext = 2662837936, first_unseen_seq = 21845, first_recent_uid = 313289502, last_cached_seq = 32626, highest_modseq = 140730969257744, highest_pvt_modseq = 93826223419056, keywords = 0x55559eb7aeb0, permanent_flags = 313289927, flags = 32626, permanent_keywords = false, allow_new_keywords = false, nonpermanent_modseqs = false, no_modseq_tracking = false, have_guids = true, have_save_guids = false, have_only_guid128 = false} errstr = <optimized out> ret = 0 ns = <optimized out> path = 0x55559eb74a50 "/home/vmail/domains/2/company.net/0/user1/Maildir/virtual/All" error = 93 sync_flags = <optimized out> #27 master_connection_input_line (line=<optimized out>, conn=0x55559ebd8a80) at master-connection.c:280 str = <optimized out> max_recent_msgs = 0 ret = <optimized out> args = <optimized out> input = {event_parent = 0x0, module = 0x55559e52343f "mail", service = 0x55559e523430 "indexer-worker", username = 0x55559eb72038 "user1(a)company.net", session_id = 0x0, session_id_prefix = 0x55559eb7205f "SLTLmOXFFIy5Ydlu", session_create_time = 0, local_ip = {family = 0, u = {ip6 = { __in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ---Type <return> to continue, or q <return> to quit--- ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, forward_fields = 0x0, flags_override_add = (unknown: 0), flags_override_remove = (unknown: 0), no_userdb_lookup = false, debug = false, conn_secured = false, conn_ssl_secured = false} service_user = 0x55559ebcb318 user = 0x55559eba0cf8 error = 0x0 #28 master_connection_input (conn=0x55559ebd8a80) at master-connection.c:320 _data_stack_cur_id = 3 line = <optimized out> #29 0x00007f7212b5c425 in io_loop_call_io (io=0x55559ec54760) at ioloop.c:715 ioloop = 0x55559eb7b050 t_id = 2 __func__ = "io_loop_call_io" #30 0x00007f7212b5ddab in io_loop_handler_run_internal (ioloop=ioloop@entry=0x55559eb7b050) at ioloop-epoll.c:222 ctx = 0x55559eb7bc40 events = <optimized out> list = 0x55559eb86770 io = <optimized out> tv = {tv_sec = 2147483, tv_usec = 0} events_count = <optimized out> msecs = <optimized out> ret = 1 i = 0 call = <optimized out> __func__ = "io_loop_handler_run_internal" #31 0x00007f7212b5c529 in io_loop_handler_run (ioloop=ioloop@entry=0x55559eb7b050) at ioloop.c:767 __func__ = "io_loop_handler_run" #32 0x00007f7212b5c768 in io_loop_run (ioloop=0x55559eb7b050) at ioloop.c:740 __func__ = "io_loop_run" #33 0x00007f7212ac63c3 in master_service_run (service=0x55559eb7aeb0, callback=callback@entry=0x55559e5225f0 <client_connected>) at master-service.c:862 No locals. #34 0x000055559e522437 in main (argc=1, argv=0x55559eb7ab90) at indexer-worker.c:76 storage_service_flags = (MAIL_STORAGE_SERVICE_FLAG_USERDB_LOOKUP | MAIL_STORAGE_SERVICE_FLAG_TEMP_PRIV_DROP | MAIL_STORAGE_SERVICE_FLAG_NO_IDLE_TIMEOUT) c = <optimized out> (gdb) quit Hope can be fixed. Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

1 0

Dovecot 2.3.15 compilation fails
by Andreas M. Kirchwitz 29 Jun '21

29 Jun '21

Hello Dovecot users and developers! Up to Dovecot 2.3.14.1 manual compilation of Dovecot sources was fine, I have an old 32-bit Linux server with GCC 4.4.7 where I compile almost all things manually. Works fine, never had problems with GCC versions. env LDFLAGS="-Wl,--no-as-needed" ./configure --prefix=/usr/local/dovecot --with-ssl=openssl --with-ssldir=/usr/local/dovecot/etc/dovecot/certs make Unfortunately, Dovecot 2.3.15 fails to compile at test-smtp-params.c (works with a recent GCC 11.1.1 on my desktop, but servers usually come with older stuff). I know a little bit of programming but that's above my head what goes wrong here and how I could fix it to continue (so I don't know if that's the only problem or if there will be more). See below (at the end) for the exact error message. I assume this compilation issue was not an intentional change/requirement between 2.3.14(.1) and 2.3.15, it just happened. I don't think that the problem is in "test-smtp-params.c" itself but some declarations somewhere else are missing, but I'm no expert here. There were too many changes in the src/lib-smtp source tree to understand what may have caused this (or maybe it's even somewhere else). Any help is highly appreciated. I'd be happy to help and try out any ideas or patches. I'd like to see that Dovecot can be compiled on as many platforms as possible, as long as there are no fundamental requirements making this impossible. Thanks a lot in advance and keep up the good work, Andreas [last lines of compilation process when error happens] gcc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test -I../../src/lib-settings -I../../src/lib-master -I../../src/lib-sasl -I../../src/lib-ssl-iostream -I../../src/lib-dns -I../../src/lib-program-client -I../../src/lib-mail -DTEST_BIN_DIR=\"/usr/local/src/dovecot-2.3.15/src/lib-smtp/test-bin\" -std=gnu99 -g -O2 -fstack-protector -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -mfunction-return=keep -mindirect-branch=keep -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/local/OpenSSL-1.1.1/include -MT test-smtp-params.o -MD -MP -MF .deps/test-smtp-params.Tpo -c -o test-smtp-params.o test-smtp-params.c test-smtp-params.c:29: error: unknown field ‘data’ specified in initializer test-smtp-params.c:29: warning: missing braces around initializer test-smtp-params.c:29: warning: (near initialization for ‘test_params_buffer1.<anonymous>’) test-smtp-params.c:30: warning: missing initializer test-smtp-params.c:30: warning: (near initialization for ‘test_params_buffer1.<anonymous>.<anonymous>.used’) test-smtp-params.c:30: error: unknown field ‘used’ specified in initializer test-smtp-params.c:31: warning: excess elements in struct initializer test-smtp-params.c:31: warning: (near initialization for ‘test_params_buffer1’) test-smtp-params.c:33: error: unknown field ‘data’ specified in initializer test-smtp-params.c:33: warning: missing braces around initializer test-smtp-params.c:33: warning: (near initialization for ‘test_params_buffer2.<anonymous>’) test-smtp-params.c:34: warning: missing initializer test-smtp-params.c:34: warning: (near initialization for ‘test_params_buffer2.<anonymous>.<anonymous>.used’) test-smtp-params.c:34: error: unknown field ‘used’ specified in initializer test-smtp-params.c:35: warning: excess elements in struct initializer test-smtp-params.c:35: warning: (near initialization for ‘test_params_buffer2’) make[3]: *** [test-smtp-params.o] Error 1 make[3]: Leaving directory `/usr/local/src/dovecot-2.3.15/src/lib-smtp' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/usr/local/src/dovecot-2.3.15/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/local/src/dovecot-2.3.15' make: *** [all] Error 2

7 6

acl_ignore_namespace & dovecot-acl-list files
by Vincent Brillault 29 Jun '21

29 Jun '21

Hi! After updating to 2.3.15, I was hoping to be able to get rid of some of the work-around I had in place when configuring shared folders in cluster mode in 2.3.14. Unfortunately it seems that I still need one. Running on CentOS7 with both a 'virtual' and 'shared' namespace, whenever I update the global vfile, I'm getting the following warning: ``` Error: open(/etc/dovecot/virtual/.temp.....8536.f1a78823779ee98d) failed: Read-only file system Error: dovecot-acl-list creation failed: safe_mkstemp(/etc/dovecot/virtual/.temp.....8536.) failed: Read-only file system Error: unlink(/etc/dovecot/virtual/dovecot-acl-list) failed: Read-only file system (in acl-backend-vfile-acllist.c:339) ``` I found it weird, as I'm using acl_ignore_namespace to ignore the virtual filesystem, but still dovecot seems to be trying to create a dovecot-acl-list file there. After digging into it and generating a backtrace where that error message is triggered, I believe I've trace it down to `acl_lookup_dict_rebuild`, which doesn't filter out "ignored" namespaces. The attached patch seems to solve the problem for me, but I must admit I didn't test it in depth so I'm not sure if I'm breaking something else... Let me know if I'm missing something, Thanks, Vincent PS: Some details: - Relevant configuration (let me know if I'm missing something): ``` mail_plugins = acl zlib virtual fts fts_solr namespace Virtual { prefix = Virtual/ separator = / hidden = yes list = no subscriptions = no location = virtual:/etc/dovecot/virtual:INDEX=~/virtual } namespace shared { type = shared prefix = shared/%%u/ list = children subscriptions = no location = imapc:~/shared/%%u } plugin { # ACLs acl = vfile:/etc/dovecot/dovecot-acl acl_ignore_namespace = Virtual/ acl_ignore_namespace2 = shared/* acl_shared_dict = proxy::acl } ``` - Backtrace generated manually (adding `backtrace_get(&backtrace)` & printing the result under the `dovecot-acl-list creation failed` error): ``` Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7ff99d364862] -> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7ff99d36496e] -> /usr/lib64/dovecot/lib01_acl_plugin.so(+0xb071) [0x7ff99ca9f071] -> /usr/lib64/dovecot/lib01_acl_plugin.so(acl_backend_vfile_acllist_rebuild+0x1d) [0x7ff99ca9f3ed] -> /usr/lib64/dovecot/lib01_acl_plugin.so(acl_backend_vfile_acllist_refresh+0x250) [0x7ff99ca9f6b0] -> /usr/lib64/dovecot/lib01_acl_plugin.so(acl_backend_vfile_nonowner_iter_init+0x9) [0x7ff99ca9f849] -> /usr/lib64/dovecot/lib01_acl_plugin.so(acl_lookup_dict_rebuild+0x13e) [0x7ff99caa179e] -> /usr/lib64/dovecot/lib01_acl_plugin.so(+0xb245) [0x7ff99ca9f245] -> /usr/lib64/dovecot/lib01_acl_plugin.so(acl_backend_vfile_acllist_rebuild+0x1d) [0x7ff99ca9f3ed] -> /usr/lib64/dovecot/lib01_acl_plugin.so(+0xa9ad) [0x7ff99ca9e9ad] -> /usr/lib64/dovecot/lib01_acl_plugin.so(+0x6e4c) [0x7ff99ca9ae4c] -> /usr/lib64/dovecot/lib01_acl_plugin.so(acl_object_get_my_rights+0x75) [0x7ff99ca9af35] -> /usr/lib64/dovecot/lib01_acl_plugin.so(+0xe44e) [0x7ff99caa244e] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_exists+0x6e) [0x7ff99d6845fe] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_list_mailbox+0x226) [0x7ff99d694ff6] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_list_iter_init_namespaces+0x282) [0x7ff99d6eecf2] -> dovecot/imap [vbrillau XXXXX LIST](cmd_list_full+0x55b) [0x563f9bb004bb] -> dovecot/imap [vbrillau XXXXX LIST](command_exec+0x64) [0x563f9bb0ab04] -> dovecot/imap [vbrillau XXXXX LIST](+0x1d9e2) [0x563f9bb089e2] -> dovecot/imap [vbrillau XXXXX LIST](+0x1da71) [0x563f9bb08a71] -> dovecot/imap [vbrillau XXXXX LIST](client_handle_input+0x205) [0x563f9bb08ef5] -> dovecot/imap [vbrillau XXXXX LIST](client_input+0x79) [0x563f9bb094d9] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x65) [0x7ff99d38a425] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x12b) [0x7ff99d38bdab] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x59) [0x7ff99d38a529] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7ff99d38a768] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7ff99d2f43c3] -> dovecot/imap [vbrillau XXXXX LIST](main+0x342) [0x563f9bafa2f2] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7ff99ced1555] -> dovecot/imap [vbrillau XXXXX LIST](+0xf4f5) [0x563f9bafa4f5] ```

2 1

SSL23_GET_CLIENT_HELLO:unknown protocol
by Voytek Eymont 29 Jun '21

29 Jun '21

I have a user failing to get mail , using TBird client is this due to his TBird email client backlevel TLS support ? I'm trying to find out his TBird version, might be an older TBird # dovecot --version 2.3.15 (0503334ab1) # egrep "ssl_protocols|ssl_min_protocol" /etc/dovecot/conf.d/10-ssl.conf #ssl_min_protocol = TLSv1.2 Jun 29 14:23:27 pop3-login: Info: Disconnected: Connection closed: SSL_accept() failed: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol (no auth attempts in 1 secs): user=<>, rip= 111.222.333.555, lip=ser.ver.ip.add, TLS handshaking: SSL_accept() failed: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol, session=<kZ6p99/FW9iTMgHi> Jun 29 14:23:54 pop3-login: Info: Disconnected: Connection closed: SSL_accept() failed: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol (no auth attempts in 1 secs): user=<>, rip= 111.222.333.555, lip=ser.ver.ip.add, TLS handshaking: SSL_accept() failed: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol, session=<Gq5G+d/Fu1STMgHi>

2 2

Increase FTS timeouts for slow(er) machines
by Bojan Čekrlić 29 Jun '21

29 Jun '21

Hello list, I have a question which doesn’t seem to be answered by the documentation. I’m running Dovecot on a Raspberry PI-like computer. I’ve enabled full-text search (FTS) and installed Solr and Tika. Now, Dovecot will timeout Tika after 60 seconds and some emails will not get indexed as there’s simply not enough processing power on the device to work so fast. I don’t mind the slowness, so the question here is — how do I increase this timeout? Looking at the commits it seems that this option is hardcoded in the code and cannot be increased via configuration change? Thanks, B

2 2

good options for Multiple users on a common email account
by Pat G 28 Jun '21

28 Jun '21

Hi, i ve a mail server to manage with some email accounts but with multiples users (+50) using a common email. it indicates sometimes that it can't connect cause too many connections. what are the good options to allow a lot of users for a specific account ? i modifyied these options : auth_worker_max_count = 60 mail_max_userip_connections = 60 is it sufficient ? dovecot -n : auth_worker_max_count = 60 mail_location = maildir:/home/%u/Maildir namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } protocols = " imap" service imap-login { inet_listener imaps { port = 993 ssl = yes } } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = # hidden, use -P to show it userdb { driver = passwd } protocol imap { mail_max_userip_connections = 60 } Regards, -- patg

3 3

Unable to run dovecot.2.3.15
by Joe Wong 28 Jun '21

28 Jun '21

Hello, I am trying to setup a new server on Centos 7 running 2.3.15, I compile the source with ./configure --with-pam , make then make install. I have another server using 2.3.8 so I copy my config files from there to /usr/local/etc/dovecot. I am using systemctl enable dovecot then systemctl start dovecot , however it failed to start. There is error found in /var/log/messages: Jun 28 09:14:37 new-ns1 systemd: Started Dovecot IMAP/POP3 email server. Jun 28 09:14:37 new-ns1 dovecot: Error: unlink(/usr/var/run/dovecot/login/login) failed: Read-only file system (in master-settings.c:739) Jun 28 09:14:37 new-ns1 dovecot: Error: unlink(/usr/var/run/dovecot/login/pop3) failed: Read-only file system (in master-settings.c:739) Jun 28 09:14:37 new-ns1 dovecot: Error: unlink(/usr/var/run/dovecot/login/imap) failed: Read-only file system (in master-settings.c:739) Jun 28 09:14:37 new-ns1 dovecot: Error: unlink(/usr/var/run/dovecot/login/dns-client) failed: Read-only file system (in master-settings.c:739) Jun 28 09:14:37 new-ns1 dovecot: Error: unlink(/usr/var/run/dovecot/login/ipc-proxy) failed: Read-only file system (in master-settings.c:739) Jun 28 09:14:37 new-ns1 dovecot: Error: unlink(/usr/var/run/dovecot/token-login/tokenlogin) failed: Read-only file system (in master-settings.c:739) Jun 28 09:14:37 new-ns1 dovecot: Error: unlink(/usr/var/run/dovecot/token-login/imap-urlauth) failed: Read-only file system (in master-settings.c:739) Jun 28 09:14:37 new-ns1 dovecot: Error: service(stats): unlink(/usr/var/run/dovecot/stats-reader) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(stats): unlink(/usr/var/run/dovecot/stats-writer) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(replicator): unlink(/usr/var/run/dovecot/replicator) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(aggregator): unlink(/usr/var/run/dovecot/replication-notify) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(aggregator): unlink(/usr/var/run/dovecot/replication-notify-fifo) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(pop3): unlink(/usr/var/run/dovecot/login/pop3) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(old-stats): unlink(/usr/var/run/dovecot/old-stats) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(old-stats): unlink(/usr/var/run/dovecot/old-stats-mail) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(old-stats): unlink(/usr/var/run/dovecot/old-stats-user) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(log): unlink(/usr/var/run/dovecot/log-errors) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(lmtp): unlink(/usr/var/run/dovecot/lmtp) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(ipc): unlink(/usr/var/run/dovecot/ipc) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(ipc): unlink(/usr/var/run/dovecot/login/ipc-proxy) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(indexer-worker): unlink(/usr/var/run/dovecot/indexer-worker) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(indexer): unlink(/usr/var/run/dovecot/indexer) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(imap): unlink(/usr/var/run/dovecot/login/imap) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(imap): unlink(/usr/var/run/dovecot/imap-master) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(imap-urlauth-worker): unlink(/usr/var/run/dovecot/imap-urlauth-worker) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(imap-urlauth): unlink(/usr/var/run/dovecot/token-login/imap-urlauth) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(imap-urlauth-login): unlink(/usr/var/run/dovecot/imap-urlauth) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(imap-hibernate): unlink(/usr/var/run/dovecot/imap-hibernate) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(doveadm): unlink(/usr/var/run/dovecot/doveadm-server) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(dns-client): unlink(/usr/var/run/dovecot/dns-client) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(dns-client): unlink(/usr/var/run/dovecot/login/dns-client) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(director): unlink(/usr/var/run/dovecot/director-admin) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(director): unlink(/usr/var/run/dovecot/director-userdb) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(dict): unlink(/usr/var/run/dovecot/dict) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(dict-async): unlink(/usr/var/run/dovecot/dict-async) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(config): unlink(/usr/var/run/dovecot/config) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(auth): unlink(/usr/var/run/dovecot/login/login) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(auth): unlink(/usr/var/run/dovecot/token-login/tokenlogin) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(auth): unlink(/usr/var/run/dovecot/auth-login) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(auth): unlink(/usr/var/run/dovecot/auth-client) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(auth): unlink(/usr/var/run/dovecot/auth-userdb) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(auth): unlink(/usr/var/run/dovecot/auth-master) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(auth-worker): unlink(/usr/var/run/dovecot/auth-worker) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(anvil): unlink(/usr/var/run/dovecot/anvil) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Error: service(anvil): unlink(/usr/var/run/dovecot/anvil-auth-penalty) failed: Read-only file system Jun 28 09:14:37 new-ns1 dovecot: Fatal: Failed to start listeners Jun 28 09:14:37 new-ns1 systemd: dovecot.service: main process exited, code=exited, status=89/n/a Jun 28 09:14:37 new-ns1 systemd: Unit dovecot.service entered failed state. Jun 28 09:14:37 new-ns1 systemd: dovecot.service failed. What did I miss here ?

4 5

Dovecot mailing list and DKIM
by Kevin N. 28 Jun '21

28 Jun '21

Hello everybody, Going through my mail logs I noticed that a couple of messages from the Dovecot mailing list failed DKIM validation. For example, this one has failed: https://markmail.org/message/te7tycmpiutw4kia 'opendkim-testkey' shows that the key is OK. The messages in the example comes from yahoo.com, however other messages from yahoo.com, ones that are not sent through the mailing list, are properly validated. The failure is not specific to yahoo.com since a couple of other messages from other domains, sent though the mailing list, also failed DKIM validation. However, the vast majority of the messages from the list properly pass DKIM validation. From what I have read, mailing lists in general do not play very well with DKIM or SPF and the Dovecot mailing list also had some issues with this in the past. With the Dovecot mailing list, I am a bit curious about why the vast majority of the messages from the list pass DKIM validation while a couple of them fail. Does the Dovecot mailing list software alter the message in a way that causes the failure? Or is there something else going on? Any thoughts? Cheers.

4 9

ZLIB/ZSTD Compressed MBOX in Maildir++ layout
by Pedro Ribeiro 28 Jun '21

28 Jun '21

Hello everyone! Our archive namespace is kept in MBOX format with Maildir++ layout due to: * Need to have the email from all users/folders self contained (our base namespace uses MDBOX with single instance storage of attachments) * MBOX/FS layout isn't supported because some folders have messages and subfolders. I'm trying to compress that namespace using zstd/gzip following the informations from Dovecot documentation about the zlib plugin but when the (MBOX/Maildir++) compress and become with .zst/.zstd/.gz suffix (both tested) the system can't read the mailboxes and incorrectly reports the suffix (.zstd/.gz) as a subfolder name (following the Maildir++ file structure). Is there any way to make the MBOX/Maildir++ compatible with the zlib compression? Using Dovecot 2.3.14 on Debian10 Regards. -- Best regards, =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Pedro Ribeiro Politécnico de Lisboa, Serviços da Presidência Departamento de Sistemas de Informação e Comunicações Phone: +351 210 464 700 (general) / VoIP: 80100 Helpdesk: helpdesk(a)net.ipl.pt / https://www.net.ipl.pt =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

2 5

Problem with Lua notification script since 2.3.15 update
by Ralf Becker 28 Jun '21

28 Jun '21

If the Lua-notifications are enabled, Dovecot imap dies immediately at authentication: Jun 28 09:17:29 imap-login: Info: Login: user=<ralf>, method=PLAIN, rip=172.18.0.12, lip=172.18.0.16, mpid=16, TLS, session=<dDhm9c/FYtqsEgAM> Jun 28 09:17:29 imap(ralf)<16><dDhm9c/FYtqsEgAM>: Panic: file dlua-script.c: line 224 (dlua_script_init): assertion failed: (lua_gettop(script->L) == 0) Jun 28 09:17:29 imap(ralf)<16><dDhm9c/FYtqsEgAM>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x41) [0x7f9f537ca5c1] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x22) [0x7f9f537ca6e2] -> /usr/lib/dovecot/libdovecot.so.0(+0x1070bb) [0x7f9f537d70bb] -> /usr/lib/dovecot/libdovecot.so.0(+0x107157) [0x7f9f537d7157] -> /usr/lib/dovecot/libdovecot.so.0(+0x5bb2b) [0x7f9f5372bb2b] -> /usr/lib/dovecot/libdovecot-lua.so.0(+0x5354) [0x7f9f53311354] -> /usr/lib/dovecot/modules/lib22_push_notification_lua_plugin.so(+0x3842) [0x7f9f532c6842] -> /usr/lib/dovecot/modules/lib20_push_notification_plugin.so(push_notification_driver_init+0x19c) [0x7f9f532d062c] -> /usr/lib/dovecot/modules/lib20_push_notification_plugin.so(+0x846f) [0x7f9f532d246f] -> /usr/lib/dovecot/modules/lib20_push_notification_plugin.so(+0x8b6a) [0x7f9f532d2b6a] -> /usr/lib/dovecot/libdovecot-storage.so.0(hook_mail_user_created+0x211) [0x7f9f539084e1] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_user_init+0x20b) [0x7f9f5390e80b] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_service_next_with_session_suffix+0x587) [0x7f9f5390bd57] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_service_lookup_next+0x53) [0x7f9f5390c4c3] -> dovecot/imap(client_create_from_input+0x180) [0x5626eb18fe40] -> dovecot/imap(+0x3112a) [0x5626eb19012a] -> /usr/lib/dovecot/libdovecot.so.0(+0x8d60e) [0x7f9f5375d60e] -> /usr/lib/dovecot/libdovecot.so.0(+0x8d97b) [0x7f9f5375d97b] -> /usr/lib/dovecot/libdovecot.so.0(+0x8e8be) [0x7f9f5375e8be] -> /usr/lib/dovecot/libdovecot.so.0(connection_input_default+0x15e) [0x7f9f537cef5e] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6d) [0x7f9f537ed3ed] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x145) [0x7f9f537eea15] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x54) [0x7f9f537ed494] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7f9f537ed600] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x17) [0x7f9f537609e7] -> dovecot/imap(main+0x469) [0x5626eb172c19] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7f9f535020b3] -> dovecot/imap(_start+0x2e) [0x5626eb172cae] Jun 28 09:17:29 imap(ralf)<16><dDhm9c/FYtqsEgAM>: Fatal: master: service(imap): child 16 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) I use the following Dovecot configuration: # Store METADATA information in a file dovecot-metadata in user's home mail_attribute_dict = file:%h/dovecot-metadata # enable metadata protocol imap { imap_metadata = yes } # add necessary plugins for Lua push notifications mail_plugins = $mail_plugins mail_lua notify push_notification push_notification_lua # Lua notification script and URL of EGroupware push server plugin { push_notification_driver = lua:file=/etc/dovecot/dovecot-push.lua push_lua_url = https://Bearer:secret@boulder.egroupware.org/egroupware/push } The Lua script is available under https://github.com/EGroupware/swoolepush/blob/master/doc/dovecot-push.lua The whole Dovecot configurations is available under https://github.com/EGroupware/build.opensuse.org/tree/master/server:eGroupW… Dovecot runs in a Ubuntu 20.04 based container and seems to use the correct liblua5.3: root@750978e5c0ee:/# dpkg -l|grep -i lua ii dovecot-lua 2:2.3.15-1+ubuntu20.04 amd64 secure POP3/IMAP server - LUA support ii liblua5.3-0:amd64 5.3.3-1.1ubuntu2 amd64 Shared library for the Lua interpreter version 5.3 ii lua-json 1.3.4-2 all JSON decoder/encoder for Lua ii lua-lpeg:amd64 1.0.2-1 amd64 LPeg library for the Lua language ii lua-socket:amd64 3.0~rc1+git+ac3201d-4 amd64 TCP/UDP socket library for the Lua language Everything was running fine with 2.3.13, have not tried with 2.3.14 yet. Any ideas? Ralf -- Ralf Becker EGroupware GmbH [www.egroupware.org] Handelsregister HRB Kaiserslautern 3587 Geschäftsführer Birgit und Ralf Becker Leibnizstr. 17, 67663 Kaiserslautern, Germany Telefon +49 631 31657-0

3 4