[dovecot/core] 1e7f85: lib-storage: Do not drop autocreate boxes when lis...

Fri Jun 9 11:00:15 EEST 2017

  Branch: refs/heads/master-2.2
  Home:   https://github.com/dovecot/core
  Commit: 1e7f85e6d741036598f94cb369991e5ef0da8cae
      https://github.com/dovecot/core/commit/1e7f85e6d741036598f94cb369991e5ef0da8cae
  Author: Aki Tuomi <aki.tuomi at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/list/mailbox-list-iter.c

  Log Message:
  -----------
  lib-storage: Do not drop autocreate boxes when listing them

Otherwise we skip folders, because array_delete() is removing boxes while they're being iterated.

  Commit: ff6e6aedec2896f183605474e5435f602b62180c
      https://github.com/dovecot/core/commit/ff6e6aedec2896f183605474e5435f602b62180c
  Author: Aki Tuomi <aki.tuomi at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/list/mailbox-list-iter.c
    M src/lib-storage/mailbox-list-private.h

  Log Message:
  -----------
  lib-storage: Rename autocreate_iter_existing to mailbox_list_iter_autocreate_filter

It makes next commit possible

  Commit: c55cb326426bb35b8f0f7158f0d131d06e0d43cf
      https://github.com/dovecot/core/commit/c55cb326426bb35b8f0f7158f0d131d06e0d43cf
  Author: Aki Tuomi <aki.tuomi at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/list/mailbox-list-index-iter.c

  Log Message:
  -----------
  lib-storage: Apply mailbox_list_iter_autocreate_filter on mailbox_list_index_iter_next

It skips mailbox_list_iter_next, so it needs to be applied here as well.

  Commit: 98567fced5dd783e7df2ebf1430fab2d2d698519
      https://github.com/dovecot/core/commit/98567fced5dd783e7df2ebf1430fab2d2d698519
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/config/config-parser-private.h
    M src/config/config-parser.c

  Log Message:
  -----------
  config: When showing an "Unknown setting" error, show the full section path

For example with:

service foo {
  inet_listener bar {
    key = value
  }
}

Instead of showing just:

Unknown setting: key

Show the entire path:

Unknown setting: service { inet_listener { key

Any filters won't be shown, because they don't affect the result.

  Commit: 9de37e8198464da95ab1a3de2a28b739b2cbaf85
      https://github.com/dovecot/core/commit/9de37e8198464da95ab1a3de2a28b739b2cbaf85
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/list/Makefile.am
    A src/lib-storage/list/mailbox-list-iter-private.h
    M src/lib-storage/list/mailbox-list-iter.c

  Log Message:
  -----------
  lib-storage: Add mailbox-list-iter-private.h with autocreate context structs

Needed for the following ACL change.

  Commit: 71c823b56938ee856e6ff018ac3af548d7079311
      https://github.com/dovecot/core/commit/71c823b56938ee856e6ff018ac3af548d7079311
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/plugins/acl/Makefile.am
    M src/plugins/acl/acl-mailbox-list.c

  Log Message:
  -----------
  acl: Skip auto=create|subscribe mailboxes that don't have lookup-right

Removing them entirely from the boxes and box_sets arrays in the list
context treats them as if they weren't configured at all. This way they
don't need any other special code.

  Commit: d10dd118c1df15223ac1fd5e158bcdeac4178370
      https://github.com/dovecot/core/commit/d10dd118c1df15223ac1fd5e158bcdeac4178370
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/list/mailbox-list-iter-private.h
    M src/lib-storage/list/mailbox-list-iter.c

  Log Message:
  -----------
  lib-storage: Don't list duplicate autocreated mailboxes' parents

  Commit: c3472c8b698525a6427abe6f8d97157132d89dcc
      https://github.com/dovecot/core/commit/c3472c8b698525a6427abe6f8d97157132d89dcc
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/list/mailbox-list-iter.c

  Log Message:
  -----------
  lib-storage: Add \NoInferiors flag to autocreated mailboxes with mbox

Once the mailboxes get autocreated, the \NoInferiors flag is added anyway.

  Commit: 6b1bdc40b026986256de2b10c53a91f20ba6af11
      https://github.com/dovecot/core/commit/6b1bdc40b026986256de2b10c53a91f20ba6af11
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/list/mailbox-list-index-iter.c

  Log Message:
  -----------
  lib-storage: Cleanup - Don't use ctx in iter_use_index()

Shrinks the following change.

  Commit: 88470ba4a37f09a48a39bcc98c601f6b416a4124
      https://github.com/dovecot/core/commit/88470ba4a37f09a48a39bcc98c601f6b416a4124
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/list/mailbox-list-index-iter.c
    M src/lib-storage/list/mailbox-list-index.h
    M src/lib-storage/mailbox-list-private.h

  Log Message:
  -----------
  lib-storage: Fix mailbox_list_index=yes + ACLs + auto=subscribe mailboxes

If the auto=subscribe mailbox didn't have a lookup ACL, it shouldn't have
been listed. This didn't work properly, because ACL plugin initialized the
autocreate_ctx only for the index iterator, while the autoboxes were
listed using the backend iterator. Fixed by not creating index iterator at
all when doing a passthrough iteration.

  Commit: 475db673340db8ac189b5ced6e27c68f26e3bc72
      https://github.com/dovecot/core/commit/475db673340db8ac189b5ced6e27c68f26e3bc72
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/list/mailbox-list-iter.c
    M src/lib-storage/mailbox-list-private.h

  Log Message:
  -----------
  lib-storage: Make mailbox_list_iter_init_autocreate() static again

It's no longer needed elsewhere.

  Commit: 830af9a96d4f08c0165afbfd6bf31099d8753d13
      https://github.com/dovecot/core/commit/830af9a96d4f08c0165afbfd6bf31099d8753d13
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/plugins/acl/acl-global-file.c

  Log Message:
  -----------
  acl: Global ACL file was parsed as if it was local ACL file

This caused some of the ACL handling not work exactly as expected.

  Commit: 92e25c1bfe759f94b46922ba15d6c503bbc9ddc8
      https://github.com/dovecot/core/commit/92e25c1bfe759f94b46922ba15d6c503bbc9ddc8
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/list/mailbox-list-fs-flags.c

  Log Message:
  -----------
  lib-storage: fs_list_get_mailbox_flags() - Reorganize code

No functional changes.

  Commit: 4d1d65aeb5a6c8948cf1fa85d4f6a1ab2f6cd7e2
      https://github.com/dovecot/core/commit/4d1d65aeb5a6c8948cf1fa85d4f6a1ab2f6cd7e2
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/list/mailbox-list-fs-flags.c

  Log Message:
  -----------
  lib-storage: fs_list_get_mailbox_flags() - Add comments

  Commit: 788671da1365d872a20a9c3e60742652e43a2fd0
      https://github.com/dovecot/core/commit/788671da1365d872a20a9c3e60742652e43a2fd0
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/list/mailbox-list-fs-flags.c

  Log Message:
  -----------
  lib-storage: fs_list_get_mailbox_flags() - Don't assume directories always have children.

This makes the directory handling code equivalent for type==UNKNOWN and
type==DIR.

  Commit: e323ab7963f4fdfe018a3d6c6ef5057c5c4c9656
      https://github.com/dovecot/core/commit/e323ab7963f4fdfe018a3d6c6ef5057c5c4c9656
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/list/mailbox-list-fs-flags.c

  Log Message:
  -----------
  lib-storage: fs_list_get_mailbox_flags() - Return mailbox files as selectable

Avoids a second stat() later on, which would do the same anyway.

  Commit: b3f3151b5ff4eb61a27bc0936c037f9969528015
      https://github.com/dovecot/core/commit/b3f3151b5ff4eb61a27bc0936c037f9969528015
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/list/mailbox-list-fs-iter.c

  Log Message:
  -----------
  lib-storage: Fix ACL in parent mailbox potentially hiding its children.

fs_get_existence_info_flag() was only being called with LAYOUT=fs and
Maildir combination, so only it was broken if the parent mailbox wasn't
visible but its children were.

  Commit: 7ef5f45749506c4b2609471c4c250a2a3d9e890b
      https://github.com/dovecot/core/commit/7ef5f45749506c4b2609471c4c250a2a3d9e890b
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/list/mailbox-list-fs-iter.c

  Log Message:
  -----------
  lib-storage: Fix LAYOUT=fs potentially not ignoring ACLs when requested

fs_get_existence_info_flag() was only being called with LAYOUT=fs and
Maildir combination. This mainly affected that combination when using
mailbox_list_index=yes and trying to list mailboxes with ACL checking
disabled.

  Commit: 2c1b3c1c8f26ee76090ce56e14c6877f139210ad
      https://github.com/dovecot/core/commit/2c1b3c1c8f26ee76090ce56e14c6877f139210ad
  Author: Aki Tuomi <aki.tuomi at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-mail/Makefile.am

  Log Message:
  -----------
  lib-mail: Link test binaries to libmail.la instead of .lo files

  Commit: ee502bb6c2d4c58be8e1fc24722981d1887c4c5d
      https://github.com/dovecot/core/commit/ee502bb6c2d4c58be8e1fc24722981d1887c4c5d
  Author: Aki Tuomi <aki.tuomi at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-mail/Makefile.am
    A src/lib-mail/istream-qp-encoder.c
    M src/lib-mail/istream-qp.h
    A src/lib-mail/qp-encoder.c
    A src/lib-mail/qp-encoder.h
    A src/lib-mail/test-istream-qp-encoder.c
    A src/lib-mail/test-qp-encoder.c

  Log Message:
  -----------
  lib-mail: Add qp encoder

  Commit: f02f204080b70d88190b0c9234de5c5b001c4c75
      https://github.com/dovecot/core/commit/f02f204080b70d88190b0c9234de5c5b001c4c75
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/index/index-sort.c

  Log Message:
  -----------
  lib-storage: Fix SORT error message when reaching mail_sort_max_read_count

It should have been:
NO [LIMIT] Requested sort would have taken too long

Instead it was just:
NO [SERVERBUG] Mail field not cached

It was broken for CC, FROM, TO, SUBJECT, DISPLAYFROM and DISPLAYTO.
The numeric sorting was working correctly.

  Commit: f2450945b4a359a8e9399932138f9e6a32e73e4b
      https://github.com/dovecot/core/commit/f2450945b4a359a8e9399932138f9e6a32e73e4b
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/index/istream-mail.c

  Log Message:
  -----------
  lib-storage: Make sure istream-mail doesn't replace an existing storage error

I'm not aware of it actually having broken anything, so this is just for
extra safety.

  Commit: cb001ddb2604440d8835abd131cc9e04c1a64027
      https://github.com/dovecot/core/commit/cb001ddb2604440d8835abd131cc9e04c1a64027
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/index/index-sort-string.c

  Log Message:
  -----------
  lib-storage: Sort code cleanup - pass node to index_sort_get_string()

  Commit: 521d502c7c3cb3fb8a61e7cf3832111a1cdef42d
      https://github.com/dovecot/core/commit/521d502c7c3cb3fb8a61e7cf3832111a1cdef42d
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/index/index-sort-string.c

  Log Message:
  -----------
  lib-storage: Sort code cleanup - Change index_sort_get_string() to return bool

None of the callers cared about the difference between "expunged" vs
"error".

  Commit: 5954998a7d70032c13eeb769f812395f6db9f2bb
      https://github.com/dovecot/core/commit/5954998a7d70032c13eeb769f812395f6db9f2bb
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/index/index-sort-string.c

  Log Message:
  -----------
  lib-storage: Optimize error handling for SORT

If a lot of mails have failed in a large mailbox, this check reduces the
CPU usage significantly.

  Commit: 3641f03046163006629f8047cf67ebd6ea6ab142
      https://github.com/dovecot/core/commit/3641f03046163006629f8047cf67ebd6ea6ab142
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-index/mail-cache-lookup.c

  Log Message:
  -----------
  lib-index: mail_cache_get_missing_reason() - Minor code cleanup

  Commit: 29e2a26bd3a85ce7d28c81142db184ee2aa860ea
      https://github.com/dovecot/core/commit/29e2a26bd3a85ce7d28c81142db184ee2aa860ea
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-index/mail-cache-lookup.c
    M src/lib-index/mail-cache-private.h

  Log Message:
  -----------
  lib-index: Add cache to mail_cache_get_missing_reason()

This avoids excessive CPU usage when it's called in a loop for many mails.

  Commit: 8bcb5474740bbb0f74633e8c5fabbf123859342b
      https://github.com/dovecot/core/commit/8bcb5474740bbb0f74633e8c5fabbf123859342b
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/index/index-sort-string.c

  Log Message:
  -----------
  lib-storage: Fix assert-crash in SORT caused by earlier changes

Broken by 0a6a527f0c42b5478d80ac53ab357885676fd516

Fixes:
Panic: file index-sort-string.c: line 668 (index_sort_add_ids_range): assertion failed: (str != NULL)

  Commit: 5afa083d859fbce1ab14514322f91edc5d7533df
      https://github.com/dovecot/core/commit/5afa083d859fbce1ab14514322f91edc5d7533df
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-index/mail-cache-lookup.c
    M src/lib-index/mail-cache-private.h

  Log Message:
  -----------
  lib-index: Fix reset_id handling in mail_cache_get_missing_reason() cache

Broken by bd897f82a5d9b00ba256b462d2056c2dc7df257c

  Commit: f2cdf641586e59cfaa14a468493e116f8b2735b8
      https://github.com/dovecot/core/commit/f2cdf641586e59cfaa14a468493e116f8b2735b8
  Author: Josef 'Jeff' Sipek <jeff.sipek at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib/guid.c

  Log Message:
  -----------
  lib: guid_128_generate() needs to correctly convert usecs to nsecs

The timestamp in the guid starts off with real time, but "runs ahead" if a lot
of guids are generated.  If not many guids are being generated, the timestamp
is fast-forwarded to the current ioloop_timeval.  The buggy comparison prevented
fast forwarding when tv_sec matched, but tv_nsec value was > 0 (which
was true most of the time).

  Commit: 26ee65a7df62ad7eb6a421983ce43750abbac682
      https://github.com/dovecot/core/commit/26ee65a7df62ad7eb6a421983ce43750abbac682
  Author: Josef 'Jeff' Sipek <jeff.sipek at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib/guid.c

  Log Message:
  -----------
  lib: guid_128_generate() shouldn't generate invalid timestamps

The nsec validity check must take into account the soon-to-be-done increment.

  Commit: 2744ab671fb9e914a9bc3ebc8355e46db4597611
      https://github.com/dovecot/core/commit/2744ab671fb9e914a9bc3ebc8355e46db4597611
  Author: Josef 'Jeff' Sipek <jeff.sipek at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib/test-guid.c

  Log Message:
  -----------
  lib: test for guid_128_generate() time handling

Make sure that guids always increase regardless of what is happening to
the time.

  Commit: 0229904b5dab1d0377342bcba88ae9a4fe3906e8
      https://github.com/dovecot/core/commit/0229904b5dab1d0377342bcba88ae9a4fe3906e8
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lib-storage/index/imapc/imapc-sync.c

  Log Message:
  -----------
  imapc: Fix potential crash if initial sync fails.

Happened at least if mailbox had 0 mails and a failing mailbox_sync() was
called twice.

Fixes:
Panic: file imapc-sync.c: line 477 (imapc_sync_index): assertion failed: (mbox->sync_fetch_first_uid == 1)

  Commit: d9b9c96c0d2ff489907d53e2b2eb98d0cf0bf80b
      https://github.com/dovecot/core/commit/d9b9c96c0d2ff489907d53e2b2eb98d0cf0bf80b
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-06-09 (Fri, 09 Jun 2017)

  Changed paths:
    M src/lmtp/commands.c

  Log Message:
  -----------
  lmtp: Removed (Dovecot) from Received header.

Some people want to hide it, and I don't really see much benefit in
including it anyway. So lets just hide it from everyone.

Compare: https://github.com/dovecot/core/compare/d830a99520ee...d9b9c96c0d2f