November 2020 - dovecot

Indexer error after upgrade to 2.3.11.3
by Alessio Cecchi 06 Nov '20

06 Nov '20

Hi, after the upgrade to Dovecot 2.3.11.3, from 2.3.10.1, I see frequently these errors from different users: Aug 18 11:02:35 Panic: indexer-worker(info(a)domain.com) session=<g71KISOttvS5LNVj:O3ahCyuZO18cYAAAEPCW+w>: file http-client-request.c: line 1232 (http_client_request_send_more): assertion failed: (req->payload_input != NULL) Aug 18 11:02:35 Error: indexer-worker(info(a)domain.com) session=<g71KISOttvS5LNVj:O3ahCyuZO18cYAAAEPCW+w>: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x2f) [0x7f0ee3c828bf] -> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x26) [0x7f0ee3c829d6] -> /usr/lib64/dovecot/libdovecot.so.0(+0xeb7ba) [0x7f0ee3c8d7ba] -> /usr/lib64/dovecot/libdovecot.so.0(+0xeb801) [0x7f0ee3c8d801] -> /usr/lib64/dovecot/libdovecot.so.0(+0x42ff1) [0x7f0ee3be4ff1] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_request_send_more+0x415) [0x7f0ee3c2ba25] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_connection_output+0x114) [0x7f0ee3c30994] -> /usr/lib64/dovecot/libdovecot.so.0(+0x115470) [0x7f0ee3cb7470] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x55) [0x7f0ee3ca4eb5] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xdc) [0x7f0ee3ca6ebc] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x5c) [0x7f0ee3ca4fac] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f0ee3ca51f8] -> /usr/lib64/dovecot/libdovecot.so.0(+0x8a955) [0x7f0ee3c2c955] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_request_finish_payload+0x21) [0x7f0ee3c2cbd1] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(solr_connection_post_end+0x45) [0x7f0ee1c85d15] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x3fa0) [0x7f0ee1c81fa0] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0x86cc) [0x7f0ee297f6cc] -> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_backend_update_deinit+0x2c) [0x7f0ee297f74c] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xfd04) [0x7f0ee2986d04] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xff3f) [0x7f0ee2986f3f] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0xf64b) [0x7f0ee2dc764b] -> /usr/lib64/dovecot/lib01_acl_plugin.so(+0xde43) [0x7f0ee2fdce43] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x54) [0x7f0ee3f91db4] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit+0x16) [0x7f0ee3f91e76] -> dovecot/indexer-worker [info(a)domain.com INBOX](+0x291c) [0x557584acb91c] -> dovecot/indexer-worker [info(a)domain.com INBOX](+0x2e54) [0x557584acbe54] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x55) [0x7f0ee3ca4eb5] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xdc) [0x7f0ee3ca6ebc] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x5c) [0x7f0ee3ca4fac] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f0ee3ca51f8] Aug 18 11:02:35 Error: indexer: Indexer worker disconnected, discarding 1 requests for info(a)domain.com Aug 18 11:02:35 Error: imap(info(a)domain.com) session=<g71KISOttvS5LNVj>: indexer failed to index mailbox INBOX Aug 18 11:02:35 Fatal: indexer-worker(info(a)domain.com) session=<g71KISOttvS5LNVj:O3ahCyuZO18cYAAAEPCW+w>: master: service(indexer-worker): child 24604 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) I'm using FTS with Solr 6.6.5. What is it? Thanks -- Alessio Cecchi Postmaster @http://www.qboxmail.it https://www.linkedin.com/in/alessice

9 25

Understanding the limitations of Dovecot Submission Service
by R. Diez 04 Nov '20

04 Nov '20

Hi all: The Dovecot Submission Service is very enticing for laz... I mean for busy people like me. A couple of lines in dovecot.conf , and you have a local SMTP server that does authentication. No SASL loops to jump through! Very cool people like me also appreciate cool new features like BURL. But now I want to understand its limitations. Say I just bought e-mail domain xxx(a)example.org . Say employee A has e-mail address a(a)example.org . It this employee turns evil, will he/she be able to send e-mails impersonating b(a)example.org ? Or does the Dovecot Submission Service check that he only has permission to send as a(a)example.org ? My requirements are pretty modest. Basically e-mail should just be sent all right. But are there any other reasons why I should avoid the Dovecot Submission Server anyway? The documentation does not really say much. Thanks in advance, rdiez

1 0

Re: API support for placing a message for a specific user
by Aki Tuomi 04 Nov '20

04 Nov '20

> On 03/11/2020 02:24 Oren Yehezkely <orenyny(a)gmail.com> wrote: > > > Hello, > > I am trying to find out if Dovecot API allows placing a message directly in a customer's mailbox, without needing to send the email via SMTP. > > Thanks for your help, > > Oren `doveadm save` is available from HTTP API, but seems to be missing from documentation. Example: curl --user doveadm:password http://localhost:1122/doveadm/v1 -d '[["save",{"user":"testuser","file":"Hello, world"},"c0d0"]]' note that despite being called 'file', is it actually the mail contents. Aki API for the SAVE command: { "command": "save", "parameters": [ { "name": "allUsers", "type": "boolean" }, { "name": "socketPath", "type": "string" }, { "name": "user", "type": "string" }, { "name": "transFlags", "type": "integer" }, { "name": "userFile", "type": "string" }, { "name": "mailbox", "type": "string" }, { "name": "file", "type": "string" } ] }

2 1

Re: Panic-Error (ctx->nested_parts_count > 0)
by Aki Tuomi 04 Nov '20

04 Nov '20

You should be able to workaround this by enabling fts. Aki > On 04/11/2020 09:22 Andreas Piper <piper(a)hrz.uni-marburg.de> wrote: > > > Hello, > > on my dovecot-replicator-backend I see lots of panic-messages "assertion > failed: (ctx->nested_parts_count > 0)", produced from a single user. I > assume that this is correlated with > https://github.com/dovecot/core/commit/a668d767a710ca18ab6e7177d8e8be22a6b0… > > How can I identify the incorrect formatted message in the user's > mailbox? I want to remove it, as the error-processes drive up the > backend's load incredibly. > > Will a fixed dovecot-package be out soon? > > Any help is appreciated. > > Best regards, > Andreas > > My system-data: > # 2.3.11.3 (502c39af9): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.5.11 (6c69c917) > # OS: Linux 4.15.0-122-generic x86_64 Ubuntu 18.04.5 LTS > > An example error-message: > > Nov 04 08:04:25 vhrz311 dovecot[6163]: > imap(xxx(a)staff.uni-marburg.de)<63176><ad/jlkKzSs2sGQEp>: Panic: file > message-parser.c: line 174 (message_part_finish): assertion failed: > (ctx->nested_parts_count > 0) > Nov 04 08:04:25 vhrz311 dovecot[6163]: > imap(xxx(a)staff.uni-marburg.de)<63173><ORvjlkKzRs2sGQEp>: Fatal: master: > service(imap): child 63173 killed with signal 6 (core dumps disabled - > https://dovecot.org/bugreport.html#coredumps) > Nov 04 08:04:25 vhrz311 dovecot[6163]: > imap(xxx(a)staff.uni-marburg.de)<63176><ad/jlkKzSs2sGQEp>: Error: Raw > backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x42) > [0x7f1dd28d6d22] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) > [0x7f1dd28d6e3e] -> /usr/lib/dovecot/libdovecot.so.0(+0xed75e) > [0x7f1dd28e175e] -> /usr/lib/dovecot/libdovecot.so.0(+0xed801) > [0x7f1dd28e1801] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) > [0x7f1dd2836beb] -> /usr/lib/dovecot/libdovecot.so.0(+0xcd40e) > [0x7f1dd28c140e] -> > /usr/lib/dovecot/libdovecot.so.0(message_parser_parse_next_block+0x15c) > [0x7f1dd28c2b2c] -> > /usr/lib/dovecot/libdovecot.so.0(message_search_msg+0xa0) > [0x7f1dd28c52a0] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xc8f9e) > [0x7f1dd2c67f9e] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mail_search_args_foreach+0x45) > [0x7f1dd2be31e5] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xca111) > [0x7f1dd2c69111] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xcb214) > [0x7f1dd2c6a214] -> > /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblock+0x10d) > [0x7f1dd2c6a9ed] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x22) > [0x7f1dd2becdf2] -> dovecot/imap(+0x25eb1) [0x561689529eb1] -> > dovecot/imap(command_exec+0x64) [0x561689522bc4] -> > dovecot/imap(+0x25592) [0x561689529592] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0x12f) > [0x7f1dd28fa3ff] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbe) > [0x7f1dd28fbebe] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) > [0x7f1dd28fa75c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) > [0x7f1dd28fa988] -> > /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) > [0x7f1dd2867c23] -> dovecot/imap(main+0x339) [0x561689512d19] -> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f1dd2424bf7] > -> dovecot/imap(_start+0x2a) [0x561689512f2a]

1 0

Panic-Error (ctx->nested_parts_count > 0)
by Andreas Piper 04 Nov '20

04 Nov '20

Hello, on my dovecot-replicator-backend I see lots of panic-messages "assertion failed: (ctx->nested_parts_count > 0)", produced from a single user. I assume that this is correlated with https://github.com/dovecot/core/commit/a668d767a710ca18ab6e7177d8e8be22a6b0… How can I identify the incorrect formatted message in the user's mailbox? I want to remove it, as the error-processes drive up the backend's load incredibly. Will a fixed dovecot-package be out soon? Any help is appreciated. Best regards, Andreas My system-data: # 2.3.11.3 (502c39af9): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.11 (6c69c917) # OS: Linux 4.15.0-122-generic x86_64 Ubuntu 18.04.5 LTS An example error-message: Nov 04 08:04:25 vhrz311 dovecot[6163]: imap(xxx(a)staff.uni-marburg.de)<63176><ad/jlkKzSs2sGQEp>: Panic: file message-parser.c: line 174 (message_part_finish): assertion failed: (ctx->nested_parts_count > 0) Nov 04 08:04:25 vhrz311 dovecot[6163]: imap(xxx(a)staff.uni-marburg.de)<63173><ORvjlkKzRs2sGQEp>: Fatal: master: service(imap): child 63173 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) Nov 04 08:04:25 vhrz311 dovecot[6163]: imap(xxx(a)staff.uni-marburg.de)<63176><ad/jlkKzSs2sGQEp>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7f1dd28d6d22] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f1dd28d6e3e] -> /usr/lib/dovecot/libdovecot.so.0(+0xed75e) [0x7f1dd28e175e] -> /usr/lib/dovecot/libdovecot.so.0(+0xed801) [0x7f1dd28e1801] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f1dd2836beb] -> /usr/lib/dovecot/libdovecot.so.0(+0xcd40e) [0x7f1dd28c140e] -> /usr/lib/dovecot/libdovecot.so.0(message_parser_parse_next_block+0x15c) [0x7f1dd28c2b2c] -> /usr/lib/dovecot/libdovecot.so.0(message_search_msg+0xa0) [0x7f1dd28c52a0] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xc8f9e) [0x7f1dd2c67f9e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_search_args_foreach+0x45) [0x7f1dd2be31e5] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xca111) [0x7f1dd2c69111] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xcb214) [0x7f1dd2c6a214] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblock+0x10d) [0x7f1dd2c6a9ed] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x22) [0x7f1dd2becdf2] -> dovecot/imap(+0x25eb1) [0x561689529eb1] -> dovecot/imap(command_exec+0x64) [0x561689522bc4] -> dovecot/imap(+0x25592) [0x561689529592] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0x12f) [0x7f1dd28fa3ff] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbe) [0x7f1dd28fbebe] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7f1dd28fa75c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f1dd28fa988] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f1dd2867c23] -> dovecot/imap(main+0x339) [0x561689512d19] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f1dd2424bf7] -> dovecot/imap(_start+0x2a) [0x561689512f2a]

1 0

Re: vsz_limit
by Aki Tuomi 04 Nov '20

04 Nov '20

> On 04/11/2020 05:19 Joan Moreau <jom(a)grosjo.net> wrote: > > > Hello > I am looking for help around memory management > 1 - How to get the current value of "vsz_limit" from inside a plugin (namely https://github.com/grosjo/fts-xapian/ ) , especially for indexer-worker > 2 - Is there a macro or function in dovecot to get the remaining free memory from this vsz value ? > Thank you > > Hi Joan, I don't think there is a feasible way to access this setting as of now. Is there a reason you need this? We usually recommend setting vsz_limit=0 for indexer-worker. Aki

2 1

Re: vsz_limit
by Aki Tuomi 04 Nov '20

04 Nov '20

> On 04/11/2020 05:19 Joan Moreau <jom(a)grosjo.net> wrote: > > > Hello > I am looking for help around memory management > 1 - How to get the current value of "vsz_limit" from inside a plugin (namely https://github.com/grosjo/fts-xapian/ ) , especially for indexer-worker > 2 - Is there a macro or function in dovecot to get the remaining free memory from this vsz value ? > Thank you > >

1 0

vsz_limit
by Joan Moreau 04 Nov '20

04 Nov '20

Hello I am looking for help around memory management 1 - How to get the current value of "vsz_limit" from inside a plugin (namely https://github.com/grosjo/fts-xapian/ ) , especially for indexer-worker 2 - Is there a macro or function in dovecot to get the remaining free memory from this vsz value ? Thank you

1 0

Feature Request: Redis support for username and TLS
by Mark Moseley 04 Nov '20

04 Nov '20

I was wondering if there was any imminent support in 2.3.12+ for using a username to log into Redis, as well as support for using TLS to connect to Redis. And if not, I'd like to put in a feature request for those two things (AUTH with username/password, and TLS connections to Redis). Specifically, I was looking at using a username/password combo to log into Redis for the quota_clone plugin. I found the 'password' param in the source (not documented at https://wiki.dovecot.org/Dictionary) There's no 'username' param (the 'username' in the source seems to refer to the mailbox, for the purpose of building the key name). Redis 6 supports authenticating with a username and password, as well as the ability to listen on a TLS-enabled port. Both of these significantly improve security, combined with the new ACL system. Obviously, these Redis 6 features are brand new, so I'd be shocked if they were already supported. But it'd be awesome if those were added to Dovecot :) Currently, I've got a localhost Envoy proxy doing TCP proxying from localhost+non-TLS to my Redis TLS port, which is a kludge at best. There's a neat Envoy Redis proxy that *almost* does the trick but the Envoy Redis proxy unfortunately doesn't support MULTI/EXEC, which Dovecot quota_clone uses, or I'd be using that instead of a plain TCP proxy (since the Envoy Redis proxy can use a username/password+tls to connect to the upstream Redis).

1 0

Delivering locally through the Submission Server
by R. Diez 03 Nov '20

03 Nov '20

Hi all: I am learning Dovecot step by step. I have enabled the Submission Server, in the hope that I would not need to learn other MTAs like Postfix. The Submission Server is very comfortable: it picks up the existing Dovecot configuration, so that you do not need to configure any user authentication separately. It is working fine on my test setup. My first thought was that, if the recipient is a local mailbox, the Submission Server would not need to relay the message to any external SMTP server, as it could just deliver it locally. After all, it is running on the same Dovecot. But it cannot do that, can it? Is there a work-around? I am still hoping that the mail server still works locally if the Internet connection fails. But I would need to learn some Postfix magic for that, would I? Thanks in advance, rdiez

5 7