August 2024 - dovecot

SQL Multiple passwords
by Aubry 19 Aug '24

19 Aug '24

Hi, From what I understood from the archive and from my tests, we cannot have multiple passwords for a given account. (I get the error: Password query returned multiple matches) But it looks like it can be done via a PAM module. Does anyone succeeded setup multiple password with PAM or any other method with a SQL backend ? Regards, Aubry

6 7

Panic with FTS_BACKEND_FLAG_TOKENIZED_INPUT and virtual mailboxes: file index-search-result.c: line 132
by markus-dovecot＠doits.de 17 Aug '24

17 Aug '24

Originally posted and debugged here: https://github.com/slusarz/dovecot-fts-flatcurve/issues/66 I have to following panic with fts flatcurve enabled: ``` Panic: file index-search-result.c: line 132 (index_search_result_update_flags): assertion failed: (result->search_args->args == &search_arg) Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x46) [0x74e9890a2846] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x22) [0x74e9890a2962] -> /usr/lib/dovecot/libdovecot.so.0(+0x10d81b) [0x74e9890af81b] -> /usr/lib/dovecot/libdovecot.so.0(+0x10d8b7) [0x74e9890af8b7] -> /usr/lib/dovecot/libdovecot.so.0(+0x5e2e2) [0x74e9890002e2] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x518a4) [0x74e9891d38a4] -> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x274e) [0x74e988bf8a9e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x5c) [0x74e9891eff9c] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x39) [0x74e9891f0039] -> dovecot/imap(cmd_select_full+0x1df) [0x62354bd8326f] -> dovecot/imap(command_exec+0xa4) [0x62354bd8b444] -> dovecot/imap(+0x22392) [0x62354bd89392] -> dovecot/imap(+0x22444) [0x62354bd89444] -> dovecot/imap(client_handle_input+0x1bd) [0x62354bd8989d] -> dovecot/imap(client_input+0x74) [0x62354bd89e64] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6d) [0x74e9890c603d] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x13a) [0x74e9890c77aa] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x54) [0x74e9890c60e4] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x74e9890c62a0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x17) [0x74e989036ad7] -> dovecot/imap(main+0x570) [0x62354bd7ad20] -> /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x74e988c29d90] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x74e988c29e40] -> dovecot/imap(_start+0x25) [0x62354bd7ade5] ``` It happens when accessing virtual folder with filters like this: ``` something-*/excep-dev* header x-abc-exception "1" ``` The error happens on subsequent access of the folder: - Removing dovecot.index{,.log} files allows to access the virtual folder again - dovecot.index gets recreated on access and the folder can be accessed multiple times - Marking an e-mail read triggers the error, though! Now each next access raises the error - Removing dovecot.index{,.log} restores access again and the e-mails are still marked as read (so this information is not lost), until the next e-mail is marked as read: then the folder cannot be accessed again Slusarz debugged the backtrace and wrote a detailed description at https://github.com/slusarz/dovecot-fts-flatcurve/issues/66#issuecomment-229… which I'll quote here: > This is not a flatcurve bug - it is a bug within Dovecot's indexing/fts handling. > > (Detailed debug below, so that core team can understand the problem.) > > As discussed above, this bug is triggered with virtual mailboxes on subsequent accesses. The initial access will work fine; the assert crash only happens when the Dovecot lib-storage index code reads the indexes created for the virtual mailboxes. > > `index_search_result_update_appends()` in lib-storage/index/index-search-result.c > > * Inside this function, a "temporary search parameter to limit the search only to the new messages" is added to the mail search args. Later, it is checked that the mail_search args points to this same search parameter after the search is executed. It is this assert check that is failing, because the mail args is changing during the course of the search. > > * `i_assert(result->search_args->args == &search_arg);` > > > The search args are being altered in plugins/fts/fts-search.args.c, `fts_search_args_expand()`. Specifically, in that function, either `fts_search_args_expand_language_top_level()` or `fts_search_args_expand_tree()` is being called, which can replace the current args parameter with a new pointer. > > `fts_search_args_expand()` is being called in plugins//fts/fts-search.c by `fts_search_try_lookup()` when the FTS backend has the FTS_BACKEND_FLAG_TOKENIZED_INPUT flag set. Flatcurve has this flag set. > > This flag is defined in plugins/fts/fts-api-private.h. Comments say: > > ``` > /* Tokenize all the input. update_build_more() will be called a single > directly indexable token at a time. Searching will modify the search > args so that lookup() sees only tokens that can be directly > searched. */ > ``` > > Thus, the comments explicitly say the search args will be modified if this flag is set. This modification is incompatible with the assert check in `index_search_result_update_appends()`, which requires that the search args not be modified. > > The core team will have to determine whether this behavior is correct, or if the assert check is invalid. You can find some more discussion at https://github.com/slusarz/dovecot-fts-flatcurve/issues/66, but I think the most important part (the very detailed analysis from Slusarz, thanks again!) might be enough to understand and maybe fix the problem. Relevant information about the mail system: ``` # 2.3.21 (47349e2482): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.21 (f6cd4b8e) # OS: Linux 6.5.0-45-generic x86_64 Ubuntu 22.04.4 LTS [...] namespace { location = virtual:~/virtual:LAYOUT=maildir++ prefix = _/ separator = / } [...] plugin { acl = vfile acl_shared_dict = file:/var/lib/dovecot/db/shared fts = flatcurve fts_autoindex = yes fts_filters = normalizer-icu snowball stopwords fts_filters_en = lowercase snowball stopwords fts_languages = de en fts_tokenizer_email_address = maxlen=100 fts_tokenizer_generic = algorithm=simple maxlen=30 fts_tokenizers = generic email-address sieve = file:~/sieve;active=~/.dovecot.sieve } [...] ``` I can provide more information if you need. Thanks for looking into it!

1 1

Re: Frequency Control for IMAP connection
by Joseph Tam 16 Aug '24

16 Aug '24

From: Corey H <coreyh(a)alwaysdata.net> > $ sudo tail -10000 /var/log/mail.log|grep **(a)**mail.com|wc -l > 3544 > > logs like this: > > Aug 15 19:54:10 mx dovecot: imap-login: Login: user=<**(a)**mail.com>, > method=PLAIN, rip=**, lip=**, mpid=4151830, TLS, > session=<UPotgLcfTOjYUzTN> > > How can I implement a Frequency Control on client connections to slow > down his requests? Is this a MacOSX client? I think they have a habit of doing this when doing global mailbox searches. Instead of doing a sane serial march through mailboxes, it opens up as many mailboxes as it can, then closes them when the IMAP server denies then when they hit the max. If Aki doesn't can't offer a solution, neither can I, but maybe you can play around with mail_max_userip_connections to stop them from swamping all your connections. You might have to resort to firewall or other general network controls to actually rate limit connections. Joseph Tam <jtam.home(a)gmail.com>

2 1

dovecot-2.3.2.1 and dovecot-pigeonhole-0.5.2 bug?
by Jan Nowak 16 Aug '24

16 Aug '24

Hello, we have a problem after updating the software with the operation of sieve scripts sending a copy of the e-mails, e-mails are not transferred. Configuration: dovecot-2.3.2.1 + dovecot-pigeonhole-0.5.2 + FreeBSD 11.2 log: Sep 06 13:28:17 lda(XXX@XXX)<38615><XXX+XXX>: Info: sieve: msgid=<XXX-XXX-XXX-7788-XXX@XXXXnet>: stored mail into mailbox 'INBOX' Sep 06 13:28:17 lda(XXX@XXX)<38615><XXX+XXX>: Info: sieve: Execution of script /usr/home/XXX/mail/XXX(a)XXX/.dovecot.sieve failed, but implicit keep was successful (user logfile /usr/home/XXX/mail/XXX(a)XXX/.dovecot.sieve.log may reveal additional details) .dovecot.sieve.log: error: msgid=<XXX-XXX-XXX-7788-XXX@XXXXnet>: failed to redirect message to <XXXXX(a)XXXXXX.com>: smtp(127.0.0.1:25): DATA failed: 552 Message header not CRLF terminated (permanent failure). On old configuration: dovecot 2.2 + dovecot-pigeonhole- 0.4 + FreeBSD 11.2 we don't have problem. Has anyone had a similar problem, knows the solution?

5 12

The future of SIS
by Sebastian Marsching 16 Aug '24

16 Aug '24

Hi, I am currently in the process of planning a new deployment of Dovecot. I was planning to use mdbox or sdbox with “mail_attachment_fs = sis posix”, but I stumbled across the following notice in the documentation for Dovecot 3.0 (https://doc.dovecot.org/3.0/settings/core/#core_setting-mail_attachment_fs) > Changed in version 2.4.0 (CE): SIS is deprecated and writing of SIS files is disabled. Reading is supported for now, any missing SIS attachments are replaced with files filled with spaces. > > Changed in version 3.0.0 (Pro): SIS is deprecated and writing of SIS files is disabled. Reading is supported for now, any missing SIS attachments are replaced with files filled with spaces. And https://doc.dovecot.org/3.0/installation_guide/upgrading/from-2.3-to-3.0/ says: > Saving new mails’ attachments via fs-sis is disabled, but reading SIS attachments is still supported. Missing SIS attachments are replaced with files filled with spaces. What is not entirely clear from this comment is whether this only applied to “sis posix” or also to “sis-queue posix”. Does anybody whether sis-queue is also affected by this? I saw that the notice was added to the dovecot/documentation Git repository in April 2023, but I couldn’t find any further information there or the main branch of the dovecot/core repository. Maybe, the code for 2.4 / 3.0 has not made it into that branch or repository yet. SIS sounds like a neat concept and I would like to use it, but obviously I do not want to use a soon-to-be-deprecated feature in a completely new deployment. So, does anybody know an alternative solution for attachment deduplication (except file-system level deduplication)? Thanks, Sebastian

19 43

vacation is not using from in envelope
by Marc 16 Aug '24

16 Aug '24

How to get dovecot use the from in sieve in the envelope?

3 5

Frequency Control for IMAP connection
by Corey H 15 Aug '24

15 Aug '24

Hello one of our users makes too quick IMAP connections to the server. $ sudo tail -10000 /var/log/mail.log|grep **(a)**mail.com|wc -l 3544 logs like this: Aug 15 19:54:10 mx dovecot: imap-login: Login: user=<**(a)**mail.com>, method=PLAIN, rip=**, lip=**, mpid=4151830, TLS, session=<UPotgLcfTOjYUzTN> How can I implement a Frequency Control on client connections to slow down his requests? Thank you.

2 1

2.3.21.1 build failure on MacOS X 14.6.1
by Daniel J. Luke 15 Aug '24

15 Aug '24

Hi - I get a build failure from dbox-storage.c because the stat struct doesn't have st_atim/st_ctim This simple patch fixes it (and should also work on linux/glibc hosts, but I haven't tested it there). --- src/lib-storage/index/dbox-common/dbox-storage.c.orig 2024-08-14 14:59:57 +++ src/lib-storage/index/dbox-common/dbox-storage.c 2024-08-14 15:01:23 @@ -293,8 +293,8 @@ if the directory exists. In case, get also the ctime */ struct stat stats; if (stat(path, &stats) == 0) { - last_temp_file_scan = stats.st_atim.tv_sec; - change_time = stats.st_ctim.tv_sec; + last_temp_file_scan = stats.st_atime; + change_time = stats.st_ctime; } else { if (errno != ENOENT) e_error(user->event, "stat(%s) failed: %m", path); -- Daniel J. Luke

2 1

v2.3.21 compile error
by Goetz Schultz 14 Aug '24

14 Aug '24

Hi, I am trying to compile said version above under FreeBSD 14.1 . Config setting is: ./configure --with-pgsql --with-sql However, during compile the process halts with below errors: <quote> test-mail-index-transaction-update.c:648:17: error: incompatible pointer to integer conversion assigning to 'uint32_t' (aka 'unsigned int') from 'char *(*)(int, int)' [-Wint-conversion] 648 | hdr.day_stamp = tests[i].old_day_stamp + timezone; | ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ test-mail-index-transaction-update.c:650:49: warning: arithmetic on a pointer to the function type 'char *(int, int)' is a GNU extension [-Wgnu-pointer-arith] 650 | mail_index_update_day_headers(t, tests[i].now + timezone); | ^ ~~~~~~~~ test-mail-index-transaction-update.c:650:36: error: incompatible pointer to integer conversion passing 'char *(*)(int, int)' to parameter of type 'time_t' (aka 'long') [-Wint-conversion] 650 | mail_index_update_day_headers(t, tests[i].now + timezone); | ^~~~~~~~~~~~~~~~~~~~~~~ </quote> Other info: gcc -v FreeBSD clang version 18.1.5 (https://github.com/llvm/llvm-project.git llvmorg-18.1.5-0-g617a15a9eac9) Target: x86_64-unknown-freebsd14.1 Thread model: posix InstalledDir: /usr/bin Any clue or pointer (no pun intended) what's going wrong and how to fix it? -- Thanks and regards Goetz R Schultz Quis custodiet ipsos custodes? ---------------------------->8------------------------------ /"\ \ / ASCII Ribbon Campaign X against HTML e-mail / \ ---------------------------->8------------------------------ ---------------------------->8------------------------------ /"\ \ / ASCII Ribbon Campaign X against HTML e-mail / \ This message is transmitted on 100% recycled electrons. ---------------------------->8------------------------------ Unsigned message - no responsibillity that content is not altered

3 5

[bug?] doveadm -f json who output
by Jasper Spaans 14 Aug '24

14 Aug '24

Hi, I'm trying to parse the output of `doveadm -f json who`[0] and am running into a small issue when it returns data about multiple users: the stream that is emitted in that case is not valid json. dovecot -version tells me dovecot 2.3.21 (47349e2482), more specifically, this is the version shipped in ubuntu 24.04, 1:2.3.21+dfsg1-2ubuntu5 doveadm -f json who [{"username":"<REDACTED>","connections":"4","service":"imap","pids":"(272307 270876 270872 270869)","ips":"(2001:<REDACTED>:7488)"}],{"username":"<REDACTED>","connections":"2","service":"imap","pids":"(271469 271465)","ips":"(2a02:<REDACTED>:9c28)"} When trying to pretty print this with jq, it complains about invalid json: doveadm -f json who|jq [ { "username": "<REDACTED>", "connections": "4", "service": "imap", "pids": "(272307 270876 270872 270869)", "ips": "(2001:<REDACTED>0:7488)" } ] jq: parse error: Expected value before ',' at line 1, column 167 Is this expected behaviour? I would have expected a list of user objects, and then it seems like the list-closing bracket that is currently at the end of the first user object should have been at the end of the last user object. I can code around this in my parser, but would rather not. Best, Jasper [0] actually I'm parsing the output of the doveadm http api, but that just seems to copy the data from the json format in there verbatim

2 1