From dovecot at dovecot.org Tue May 1 03:02:19 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Tue, 01 May 2012 03:02:19 +0300 Subject: dovecot-2.1: aqueue: Reversed recent changes, they didn't work a... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/99a30e40dce9 changeset: 14493:99a30e40dce9 user: Timo Sirainen date: Tue May 01 03:01:49 2012 +0300 description: aqueue: Reversed recent changes, they didn't work as intended. diffstat: src/lib/aqueue.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diffs (23 lines): diff -r 6c381b3e0a3d -r 99a30e40dce9 src/lib/aqueue.c --- a/src/lib/aqueue.c Mon Apr 30 23:04:08 2012 +0300 +++ b/src/lib/aqueue.c Tue May 01 03:01:49 2012 +0300 @@ -10,7 +10,8 @@ aqueue = i_new(struct aqueue, 1); aqueue->arr = array; - aqueue->area_size = array_count_i(array); + aqueue->area_size = buffer_get_size(aqueue->arr->buffer) / + aqueue->arr->element_size; i_assert(aqueue->area_size > 0); return aqueue; } @@ -31,7 +32,8 @@ orig_area_size = aqueue->area_size; (void)array_append_space_i(aqueue->arr); - aqueue->area_size = array_count_i(aqueue->arr); + aqueue->area_size = buffer_get_size(aqueue->arr->buffer) / + aqueue->arr->element_size; i_assert(orig_area_size < aqueue->area_size); count = I_MIN(aqueue->area_size - orig_area_size, aqueue->head); From dovecot at dovecot.org Tue May 1 03:15:55 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Tue, 01 May 2012 03:15:55 +0300 Subject: dovecot-2.1: lib-storage: shared/$user/INBOX is now visible only... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/3bb6d153c9c3 changeset: 14494:3bb6d153c9c3 user: Timo Sirainen date: Tue May 01 03:15:46 2012 +0300 description: lib-storage: shared/$user/INBOX is now visible only as shared/$user diffstat: src/lib-storage/mailbox-list.c | 12 ++++++++++++ 1 files changed, 12 insertions(+), 0 deletions(-) diffs (29 lines): diff -r 99a30e40dce9 -r 3bb6d153c9c3 src/lib-storage/mailbox-list.c --- a/src/lib-storage/mailbox-list.c Tue May 01 03:01:49 2012 +0300 +++ b/src/lib-storage/mailbox-list.c Tue May 01 03:15:46 2012 +0300 @@ -427,6 +427,12 @@ list_sep = mailbox_list_get_hierarchy_sep(list); ns_sep = mail_namespace_get_sep(ns); + if (*storage_name == '\0' && ns->type == NAMESPACE_SHARED && + (ns->flags & NAMESPACE_FLAG_INBOX_ANY) != 0) { + /* opening shared/$user. it's the same as INBOX. */ + storage_name = "INBOX"; + } + if (list_sep == ns_sep) return storage_name; if (ns->type == NAMESPACE_SHARED && @@ -501,6 +507,12 @@ and /inBox. */ return vname; } + if (strcmp(vname, "INBOX") == 0 && + (list->ns->flags & NAMESPACE_FLAG_INBOX_ANY) != 0) { + /* convert to shared/$user, we don't really care about the + INBOX suffix here. */ + vname = ""; + } if (*vname == '\0') { /* return namespace prefix without the separator */ if (list->ns->prefix_len == 0) From dovecot at dovecot.org Tue May 1 03:49:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Tue, 01 May 2012 03:49:27 +0300 Subject: dovecot-2.1: Fix the build failure on AIX Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/d5bbb1d203c5 changeset: 14495:d5bbb1d203c5 user: SATOH Fumiyasu date: Fri Apr 27 16:47:30 2012 +0900 description: Fix the build failure on AIX diffstat: src/lib-storage/index/index-search-private.h | 2 ++ src/plugins/stats/stats-plugin.h | 2 ++ 2 files changed, 4 insertions(+), 0 deletions(-) diffs (24 lines): diff -r 3bb6d153c9c3 -r d5bbb1d203c5 src/lib-storage/index/index-search-private.h --- a/src/lib-storage/index/index-search-private.h Tue May 01 03:15:46 2012 +0300 +++ b/src/lib-storage/index/index-search-private.h Fri Apr 27 16:47:30 2012 +0900 @@ -3,6 +3,8 @@ #include "mail-storage-private.h" +#include + struct index_search_context { struct mail_search_context mail_ctx; struct mail_index_view *view; diff -r 3bb6d153c9c3 -r d5bbb1d203c5 src/plugins/stats/stats-plugin.h --- a/src/plugins/stats/stats-plugin.h Tue May 01 03:15:46 2012 +0300 +++ b/src/plugins/stats/stats-plugin.h Fri Apr 27 16:47:30 2012 +0900 @@ -6,6 +6,8 @@ #include "mail-user.h" #include "mail-storage-private.h" +#include + #define STATS_USER_CONTEXT(obj) \ MODULE_CONTEXT(obj, stats_user_module) From pigeonhole at rename-it.nl Wed May 2 02:18:02 2012 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Wed, 02 May 2012 01:18:02 +0200 Subject: dovecot-2.1-pigeonhole: Fixed several small issues, including a ... Message-ID: details: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/7f9218fe9fac changeset: 1619:7f9218fe9fac user: Stephan Bosch date: Wed May 02 01:17:49 2012 +0200 description: Fixed several small issues, including a few potential segfault bugs, based on static analysis. Clang static analyzer is happy now. diffstat: src/lib-sieve-tool/mail-raw.c | 4 +- src/lib-sieve/cmd-redirect.c | 5 +- src/lib-sieve/edit-mail.c | 30 +++++++------ src/lib-sieve/mcht-matches.c | 1 - src/lib-sieve/plugins/body/ext-body-common.c | 16 +++++-- src/lib-sieve/plugins/editheader/ext-editheader-common.c | 3 +- src/lib-sieve/plugins/imap4flags/tag-flags.c | 3 + src/lib-sieve/plugins/include/cmd-include.c | 3 +- src/lib-sieve/plugins/notify/ext-notify-common.c | 10 +++- src/lib-sieve/plugins/variables/ext-variables-modifiers.c | 4 +- src/lib-sieve/plugins/variables/ext-variables-namespaces.c | 2 +- src/lib-sieve/sieve-address.c | 23 ++++------ src/lib-sieve/sieve-ast.c | 20 +++++---- src/lib-sieve/sieve-ast.h | 6 +- src/lib-sieve/sieve-commands.h | 2 +- src/lib-sieve/sieve-script.c | 6 ++- src/lib-sieve/sieve-validator.c | 8 ++- src/lib-sieve/sieve.c | 3 +- src/lib-sievestorage/sieve-storage-quota.c | 3 +- src/managesieve-login/managesieve-login-settings-plugin.c | 2 +- src/plugins/lda-sieve/lda-sieve-plugin.c | 8 +-- src/sieve-tools/sieve-dump.c | 2 +- src/sieve-tools/sieve-filter.c | 3 +- src/sieve-tools/sieve-test.c | 6 +- src/sieve-tools/sievec.c | 2 +- src/testsuite/cmd-test-message.c | 3 +- src/testsuite/testsuite.c | 2 +- 27 files changed, 95 insertions(+), 85 deletions(-) diffs (truncated from 666 to 300 lines): diff -r b2ff597c2279 -r 7f9218fe9fac src/lib-sieve-tool/mail-raw.c --- a/src/lib-sieve-tool/mail-raw.c Mon Apr 30 09:10:44 2012 +0200 +++ b/src/lib-sieve-tool/mail-raw.c Wed May 02 01:17:49 2012 +0200 @@ -115,7 +115,7 @@ if (ret > 0 && size >= 5 && memcmp(data, "From ", 5) == 0) { /* skip until the first LF */ i_stream_skip(input, 5); - while ((ret = i_stream_read_data(input, &data, &size, 0)) > 0) { + while ( i_stream_read_data(input, &data, &size, 0) > 0 ) { for (i = 0; i < size; i++) { if (data[i] == '\n') break; @@ -223,7 +223,7 @@ { struct mail_raw *mailr; struct istream *input = NULL; - time_t mtime; + time_t mtime = (time_t)-1; const char *sender = NULL; if ( path == NULL || strcmp(path, "-") == 0 ) { diff -r b2ff597c2279 -r 7f9218fe9fac src/lib-sieve/cmd-redirect.c --- a/src/lib-sieve/cmd-redirect.c Mon Apr 30 09:10:44 2012 +0200 +++ b/src/lib-sieve/cmd-redirect.c Wed May 02 01:17:49 2012 +0200 @@ -323,7 +323,6 @@ FILE *f; const unsigned char *data; size_t size; - int ret; /* Just to be sure */ if ( !sieve_smtp_available(senv) ) { @@ -352,12 +351,14 @@ rfc2822_header_field_write(f, "X-Sieve-Redirected-From", recipient); /* Pipe the message to the outgoing SMTP transport */ - while ((ret = i_stream_read_data(crlf_input, &data, &size, 0)) > 0) { + while (i_stream_read_data(crlf_input, &data, &size, 0) > 0) { if (fwrite(data, size, 1, f) == 0) break; i_stream_skip(crlf_input, size); } + // FIXME: handle stream error. Currently, we have no means to abort here. + i_stream_unref(&crlf_input); i_stream_unref(&input); diff -r b2ff597c2279 -r 7f9218fe9fac src/lib-sieve/edit-mail.c --- a/src/lib-sieve/edit-mail.c Mon Apr 30 09:10:44 2012 +0200 +++ b/src/lib-sieve/edit-mail.c Wed May 02 01:17:49 2012 +0200 @@ -657,21 +657,23 @@ } /* Insert header field index items in main list */ - if ( edmail->header_fields_appended != NULL ) { - if ( edmail->header_fields_appended->prev != NULL ) { - edmail->header_fields_appended->prev->next = head; - head->prev = edmail->header_fields_appended->prev; + if ( head != NULL && tail != NULL ) { + if ( edmail->header_fields_appended != NULL ) { + if ( edmail->header_fields_appended->prev != NULL ) { + edmail->header_fields_appended->prev->next = head; + head->prev = edmail->header_fields_appended->prev; + } + + tail->next = edmail->header_fields_appended; + edmail->header_fields_appended->prev = tail; + } else if ( edmail->header_fields_tail != NULL ) { + edmail->header_fields_tail->next = head; + head->prev = edmail->header_fields_tail; + edmail->header_fields_tail = tail; + } else { + edmail->header_fields_head = head; + edmail->header_fields_tail = tail; } - - tail->next = edmail->header_fields_appended; - edmail->header_fields_appended->prev = tail; - } else if ( edmail->header_fields_tail != NULL ) { - edmail->header_fields_tail->next = head; - head->prev = edmail->header_fields_tail; - edmail->header_fields_tail = tail; - } else { - edmail->header_fields_head = head; - edmail->header_fields_tail = tail; } /* Rebuild header index */ diff -r b2ff597c2279 -r 7f9218fe9fac src/lib-sieve/mcht-matches.c --- a/src/lib-sieve/mcht-matches.c Mon Apr 30 09:10:44 2012 +0200 +++ b/src/lib-sieve/mcht-matches.c Wed May 02 01:17:49 2012 +0200 @@ -111,7 +111,6 @@ vp = val; /* Value pointer */ kp = key; /* Key pointer */ wp = key; /* Wildcard (key) pointer */ - pvp = val; /* Previous value Pointer */ /* Start match values list if requested */ if ( (mvalues = sieve_match_values_start(mctx->runenv)) != NULL ) { diff -r b2ff597c2279 -r 7f9218fe9fac src/lib-sieve/plugins/body/ext-body-common.c --- a/src/lib-sieve/plugins/body/ext-body-common.c Mon Apr 30 09:10:44 2012 +0200 +++ b/src/lib-sieve/plugins/body/ext-body-common.c Wed May 02 01:17:49 2012 +0200 @@ -204,7 +204,6 @@ struct istream *input; unsigned int idx = 0; bool save_body = FALSE, have_all; - int ret; /* First check whether any are missing */ if (ext_body_get_return_parts(ctx, content_types, decode_to_plain)) { @@ -226,7 +225,7 @@ //parser = message_parser_init_from_parts(parts, input, 0, 0); parser = message_parser_init(ctx->pool, input, 0, 0); - while ( (ret = message_parser_parse_next_block(parser, &block)) > 0 ) { + while ( message_parser_parse_next_block(parser, &block) > 0 ) { if ( block.part != prev_part ) { bool message_rfc822 = FALSE; @@ -277,6 +276,7 @@ } /* Save bodies only if we have a wanted content-type */ + i_assert( body_part != NULL ); save_body = _is_wanted_content_type (content_types, body_part->content_type); continue; @@ -285,9 +285,10 @@ /* Encountered the empty line that indicates the end of the headers and * the start of the body */ - if ( block.hdr->eoh ) + if ( block.hdr->eoh ) { + i_assert( body_part != NULL ); body_part->have_body = TRUE; - else if ( header_part != NULL ) { + } else if ( header_part != NULL ) { /* Save message/rfc822 header as part content */ if ( block.hdr->continued ) { buffer_append(ctx->tmp_buffer, block.hdr->value, block.hdr->value_len); @@ -312,6 +313,8 @@ block.hdr->use_full_value = TRUE; continue; } + + i_assert( body_part != NULL ); /* Parse the content type from the Content-type header */ T_BEGIN { @@ -437,11 +440,14 @@ i_stream_skip(input, hdr_size.physical_size); /* Read raw message body */ - while ( (ret = i_stream_read_data(input, &data, &size, 0)) > 0 ) { + while ( (ret=i_stream_read_data(input, &data, &size, 0)) > 0 ) { buffer_append(buf, data, size); i_stream_skip(input, size); } + + if ( ret == -1 && input->stream_errno != 0 ) + return FALSE; } else { buf = ctx->raw_body; } diff -r b2ff597c2279 -r 7f9218fe9fac src/lib-sieve/plugins/editheader/ext-editheader-common.c --- a/src/lib-sieve/plugins/editheader/ext-editheader-common.c Mon Apr 30 09:10:44 2012 +0200 +++ b/src/lib-sieve/plugins/editheader/ext-editheader-common.c Wed May 02 01:17:49 2012 +0200 @@ -52,8 +52,7 @@ bool ext_editheader_load (const struct sieve_extension *ext, void **context) { - struct ext_editheader_config *ext_config = - (struct ext_editheader_config *) *context; + struct ext_editheader_config *ext_config; struct sieve_instance *svinst = ext->svinst; const char *protected; size_t max_header_size; diff -r b2ff597c2279 -r 7f9218fe9fac src/lib-sieve/plugins/imap4flags/tag-flags.c --- a/src/lib-sieve/plugins/imap4flags/tag-flags.c Mon Apr 30 09:10:44 2012 +0200 +++ b/src/lib-sieve/plugins/imap4flags/tag-flags.c Wed May 02 01:17:49 2012 +0200 @@ -299,6 +299,9 @@ } } + if ( ret < 0 ) + return flag_list->exec_status; + *se_context = (void *) ctx; return SIEVE_EXEC_OK; diff -r b2ff597c2279 -r 7f9218fe9fac src/lib-sieve/plugins/include/cmd-include.c --- a/src/lib-sieve/plugins/include/cmd-include.c Mon Apr 30 09:10:44 2012 +0200 +++ b/src/lib-sieve/plugins/include/cmd-include.c Wed May 02 01:17:49 2012 +0200 @@ -279,8 +279,7 @@ ctx_data->script = script; } - arg = sieve_ast_arguments_detach(arg, 1); - + (void)sieve_ast_arguments_detach(arg, 1); return TRUE; } diff -r b2ff597c2279 -r 7f9218fe9fac src/lib-sieve/plugins/notify/ext-notify-common.c --- a/src/lib-sieve/plugins/notify/ext-notify-common.c Mon Apr 30 09:10:44 2012 +0200 +++ b/src/lib-sieve/plugins/notify/ext-notify-common.c Wed May 02 01:17:49 2012 +0200 @@ -3,6 +3,7 @@ #include "lib.h" #include "str.h" +#include "istream.h" #include "rfc822-parser.h" #include "message-parser.h" #include "message-decoder.h" @@ -158,7 +159,7 @@ struct message_block block, decoded; struct istream *input; bool is_text, save_body; - int ret; + int ret = 1; /* Return cached result if available */ mctx = ext_notify_get_message_context(this_ext, renv->msgctx); @@ -171,7 +172,7 @@ /* Get the message stream */ if ( mail_get_stream(renv->msgdata->mail, NULL, NULL, &input) < 0 ) - return FALSE; + return NULL; /* Initialize body decoder */ decoder = message_decoder_init(FALSE); @@ -179,7 +180,7 @@ parser = message_parser_init(mctx->pool, input, 0, 0); is_text = TRUE; save_body = FALSE; - while ( (ret = message_parser_parse_next_block(parser, &block)) > 0 ) { + while ( (ret=message_parser_parse_next_block(parser, &block)) > 0 ) { if ( block.hdr != NULL || block.size == 0 ) { /* Decode block */ (void)message_decoder_decode_next_block(decoder, &block, &decoded); @@ -222,6 +223,9 @@ (void)message_parser_deinit(&parser, &parts); message_decoder_deinit(&decoder); + if ( ret < 0 && input->stream_errno != 0 ) + return NULL; + /* Return status */ return mctx->body_text; } diff -r b2ff597c2279 -r 7f9218fe9fac src/lib-sieve/plugins/variables/ext-variables-modifiers.c --- a/src/lib-sieve/plugins/variables/ext-variables-modifiers.c Mon Apr 30 09:10:44 2012 +0200 +++ b/src/lib-sieve/plugins/variables/ext-variables-modifiers.c Wed May 02 01:17:49 2012 +0200 @@ -220,7 +220,7 @@ str_append_str(*result, in); content = str_c_modifiable(*result); - content = str_ucase(content); + (void)str_ucase(content); return TRUE; } @@ -233,7 +233,7 @@ str_append_str(*result, in); content = str_c_modifiable(*result); - content = str_lcase(content); + (void)str_lcase(content); return TRUE; } diff -r b2ff597c2279 -r 7f9218fe9fac src/lib-sieve/plugins/variables/ext-variables-namespaces.c --- a/src/lib-sieve/plugins/variables/ext-variables-namespaces.c Mon Apr 30 09:10:44 2012 +0200 +++ b/src/lib-sieve/plugins/variables/ext-variables-namespaces.c Wed May 02 01:17:49 2012 +0200 @@ -90,7 +90,7 @@ const struct sieve_variables_namespace *nspc; struct arg_namespace_variable *var; const struct sieve_variable_name *name_element = array_idx(var_name, 0); - void *var_data; + void *var_data = NULL; nspc = ext_variables_namespace_create_instance (this_ext, valdtr, cmd, str_c(name_element->identifier)); diff -r b2ff597c2279 -r 7f9218fe9fac src/lib-sieve/sieve-address.c --- a/src/lib-sieve/sieve-address.c Mon Apr 30 09:10:44 2012 +0200 +++ b/src/lib-sieve/sieve-address.c Wed May 02 01:17:49 2012 +0200 @@ -348,7 +348,7 @@ return ret; } - if ((ret = parse_addr_spec(ctx)) < 0) + if (parse_addr_spec(ctx) < 0) return -1; if (*ctx->parser.data != '>') { @@ -367,8 +367,6 @@ (struct sieve_message_address_parser *ctx, const unsigned char *address, From dovecot at dovecot.org Thu May 3 23:26:55 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Thu, 03 May 2012 23:26:55 +0300 Subject: dovecot-2.1: lib-index: Fixed MAIL_INDEX_SYNC_FLAG_AVOID_FLAG_UP... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/5d45870e2e4a changeset: 14496:5d45870e2e4a user: Timo Sirainen date: Thu May 03 23:26:44 2012 +0300 description: lib-index: Fixed MAIL_INDEX_SYNC_FLAG_AVOID_FLAG_UPDATES with concurrent keyword changes. If session 1 saw keywords="a", session 2 added keyword "b" and session 1 (without the client syncing session) replaced flags with "a", the "b" keyword wasn't removed. diffstat: src/lib-index/mail-index-transaction-private.h | 4 ++++ src/lib-index/mail-index-transaction-update.c | 2 +- src/lib-index/mail-index-transaction.c | 17 +++++++++++++++++ src/lib-index/test-mail-index-transaction-update.c | 6 ++++++ 4 files changed, 28 insertions(+), 1 deletions(-) diffs (83 lines): diff -r d5bbb1d203c5 -r 5d45870e2e4a src/lib-index/mail-index-transaction-private.h --- a/src/lib-index/mail-index-transaction-private.h Fri Apr 27 16:47:30 2012 +0900 +++ b/src/lib-index/mail-index-transaction-private.h Thu May 03 23:26:44 2012 +0300 @@ -35,6 +35,7 @@ enum mail_index_transaction_flags flags; struct mail_index_transaction_vfuncs v; struct mail_index_view *view; + struct mail_index_view *latest_view; /* NOTE: If you add anything new, remember to update mail_index_transaction_reset_v() to reset it. */ @@ -121,6 +122,9 @@ unsigned int left_idx, unsigned int right_idx, uint32_t seq); +void mail_index_transaction_lookup_latest_keywords(struct mail_index_transaction *t, + uint32_t seq, + ARRAY_TYPE(keyword_indexes) *keywords); bool mail_index_cancel_flag_updates(struct mail_index_transaction *t, uint32_t seq); diff -r d5bbb1d203c5 -r 5d45870e2e4a src/lib-index/mail-index-transaction-update.c --- a/src/lib-index/mail-index-transaction-update.c Fri Apr 27 16:47:30 2012 +0900 +++ b/src/lib-index/mail-index-transaction-update.c Thu May 03 23:26:44 2012 +0300 @@ -977,7 +977,7 @@ t_array_init(&existing, 32); if (seq < t->first_new_seq) - mail_index_lookup_keywords(t->view, seq, &existing); + mail_index_transaction_lookup_latest_keywords(t, seq, &existing); existing_idx = array_get(&existing, &existing_count); if (modify_type == MODIFY_REPLACE && existing_count != keywords->count) diff -r d5bbb1d203c5 -r 5d45870e2e4a src/lib-index/mail-index-transaction.c --- a/src/lib-index/mail-index-transaction.c Fri Apr 27 16:47:30 2012 +0900 +++ b/src/lib-index/mail-index-transaction.c Thu May 03 23:26:44 2012 +0300 @@ -52,6 +52,8 @@ array_free(&t->module_contexts); mail_index_view_transaction_unref(t->view); + if (t->latest_view != NULL) + mail_index_view_close(&t->latest_view); mail_index_view_close(&t->view); i_free(t); } @@ -89,6 +91,21 @@ return next_uid; } +void mail_index_transaction_lookup_latest_keywords(struct mail_index_transaction *t, + uint32_t seq, + ARRAY_TYPE(keyword_indexes) *keywords) +{ + uint32_t uid, latest_seq; + + if (t->latest_view == NULL) { + (void)mail_index_refresh(t->view->index); + t->latest_view = mail_index_view_open(t->view->index); + } + mail_index_lookup_uid(t->view, seq, &uid); + if (mail_index_lookup_seq(t->view, uid, &latest_seq)) + mail_index_lookup_keywords(t->view, latest_seq, keywords); +} + static int mail_transaction_log_file_refresh(struct mail_index_transaction *t, struct mail_transaction_log_append_ctx *ctx) diff -r d5bbb1d203c5 -r 5d45870e2e4a src/lib-index/test-mail-index-transaction-update.c --- a/src/lib-index/test-mail-index-transaction-update.c Fri Apr 27 16:47:30 2012 +0900 +++ b/src/lib-index/test-mail-index-transaction-update.c Thu May 03 23:26:44 2012 +0300 @@ -43,6 +43,12 @@ return hdr.messages_count; } +void mail_index_transaction_lookup_latest_keywords(struct mail_index_transaction *t ATTR_UNUSED, + uint32_t seq ATTR_UNUSED, + ARRAY_TYPE(keyword_indexes) *keywords ATTR_UNUSED) +{ +} + static struct mail_index_transaction * mail_index_transaction_new(void) { From dovecot at dovecot.org Fri May 4 00:49:42 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 04 May 2012 00:49:42 +0300 Subject: dovecot-2.2: lib-index: Updated dovecot.index file only by recre... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/4c996a5737b0 changeset: 14230:4c996a5737b0 user: Timo Sirainen date: Fri May 04 00:41:51 2012 +0300 description: lib-index: Updated dovecot.index file only by recreating it, never write to it directly. This is safer, and nowadays there shouldn't be much of a performance loss with it either, since dovecot.index isn't updated very often. This also allows removing all locking from dovecot.index file, although for now we'll keep it in case old Dovecot versions are simultaneously writing to the index. diffstat: src/lib-index/mail-index-fsck.c | 3 +- src/lib-index/mail-index-map.c | 9 +- src/lib-index/mail-index-modseq.c | 5 +- src/lib-index/mail-index-private.h | 10 +- src/lib-index/mail-index-sync-ext.c | 11 +- src/lib-index/mail-index-sync-keywords.c | 4 +- src/lib-index/mail-index-sync-private.h | 2 - src/lib-index/mail-index-sync-update.c | 23 +---- src/lib-index/mail-index-write.c | 143 +----------------------------- src/lib-index/test-mail-index-sync-ext.c | 3 - 10 files changed, 24 insertions(+), 189 deletions(-) diffs (truncated from 422 to 300 lines): diff -r 96e3b3ef59dc -r 4c996a5737b0 src/lib-index/mail-index-fsck.c --- a/src/lib-index/mail-index-fsck.c Sun Mar 11 12:42:53 2012 +0200 +++ b/src/lib-index/mail-index-fsck.c Fri May 04 00:41:51 2012 +0300 @@ -454,8 +454,7 @@ mail_index_fsck_map(index, map); } T_END; - map->write_base_header = TRUE; - map->write_atomic = TRUE; + map->header_changed = TRUE; mail_index_write(index, FALSE); if (!orig_locked) diff -r 96e3b3ef59dc -r 4c996a5737b0 src/lib-index/mail-index-map.c --- a/src/lib-index/mail-index-map.c Sun Mar 11 12:42:53 2012 +0200 +++ b/src/lib-index/mail-index-map.c Fri May 04 00:41:51 2012 +0300 @@ -335,10 +335,7 @@ dest->records = buffer_get_modifiable_data(dest->buffer, NULL); dest->records_count = src->records_count; - /* if the map is ever written back to disk, we need to keep track of - what has changed. */ - dest->write_seq_first = src->write_seq_first; - dest->write_seq_last = src->write_seq_last; + dest->records_changed = src->records_changed; } static void mail_index_map_copy_header(struct mail_index_map *dest, @@ -401,9 +398,7 @@ mail_index_map_copy_header(mem_map, map); - mem_map->write_atomic = map->write_atomic; - mem_map->write_base_header = map->write_base_header; - mem_map->write_ext_header = map->write_ext_header; + mem_map->header_changed = map->header_changed; /* copy extensions */ if (array_is_created(&map->ext_id_map)) { diff -r 96e3b3ef59dc -r 4c996a5737b0 src/lib-index/mail-index-modseq.c --- a/src/lib-index/mail-index-modseq.c Sun Mar 11 12:42:53 2012 +0200 +++ b/src/lib-index/mail-index-modseq.c Fri May 04 00:41:51 2012 +0300 @@ -458,8 +458,7 @@ } T_END; } } - mail_index_sync_write_seq_update(ctx->sync_map_ctx, 1, - map->hdr.messages_count); + map->rec_map->records_changed = TRUE; mail_transaction_log_view_close(&ctx->log_view); } @@ -510,7 +509,7 @@ buffer_write(map->hdr_copy_buf, ext->hdr_offset, &new_modseq_hdr, sizeof(new_modseq_hdr)); map->hdr_base = map->hdr_copy_buf->data; - map->write_ext_header = TRUE; + map->header_changed = TRUE; } } diff -r 96e3b3ef59dc -r 4c996a5737b0 src/lib-index/mail-index-private.h --- a/src/lib-index/mail-index-private.h Sun Mar 11 12:42:53 2012 +0200 +++ b/src/lib-index/mail-index-private.h Fri May 04 00:41:51 2012 +0300 @@ -129,10 +129,8 @@ struct mail_index_map_modseq *modseq; uint32_t last_appended_uid; - /* If this mapping is written to disk and write_atomic=FALSE, - write_seq_* specify the message sequence range that needs to be - written. */ - uint32_t write_seq_first, write_seq_last; + /* The records have changed since it was read */ + bool records_changed; }; struct mail_index_map { @@ -151,9 +149,7 @@ struct mail_index_record_map *rec_map; - unsigned int write_base_header:1; - unsigned int write_ext_header:1; - unsigned int write_atomic:1; /* write to a new file and rename() */ + unsigned int header_changed:1; }; struct mail_index_module_register { diff -r 96e3b3ef59dc -r 4c996a5737b0 src/lib-index/mail-index-sync-ext.c --- a/src/lib-index/mail-index-sync-ext.c Sun Mar 11 12:42:53 2012 +0200 +++ b/src/lib-index/mail-index-sync-ext.c Fri May 04 00:41:51 2012 +0300 @@ -310,7 +310,7 @@ i_assert((map->hdr_copy_buf->used % sizeof(uint64_t)) == 0); map->hdr_base = map->hdr_copy_buf->data; map->hdr.header_size = map->hdr_copy_buf->used; - map->write_base_header = map->write_ext_header = TRUE; + map->header_changed = TRUE; ext_hdr = get_ext_header(map, ext); ext_hdr->reset_id = ext->reset_id; @@ -570,8 +570,7 @@ memset(PTR_OFFSET(rec, ext->record_offset), 0, ext->record_size); } - map->rec_map->write_seq_first = 1; - map->rec_map->write_seq_last = view->map->rec_map->records_count; + map->rec_map->records_changed = TRUE; } int mail_index_sync_ext_reset(struct mail_index_sync_map_ctx *ctx, @@ -635,7 +634,7 @@ if (ext->index_idx == ctx->view->index->modseq_ext_id) mail_index_modseq_hdr_update(ctx->modseq_ctx); - map->write_ext_header = TRUE; + map->header_changed = TRUE; return 1; } @@ -684,7 +683,7 @@ return ret; } - mail_index_sync_write_seq_update(ctx, seq, seq); + view->map->rec_map->records_changed = TRUE; /* @UNSAFE */ memcpy(old_data, u + 1, ext->record_size); @@ -786,6 +785,6 @@ return -1; } - mail_index_sync_write_seq_update(ctx, seq, seq); + view->map->rec_map->records_changed = TRUE; return 1; } diff -r 96e3b3ef59dc -r 4c996a5737b0 src/lib-index/mail-index-sync-keywords.c --- a/src/lib-index/mail-index-sync-keywords.c Sun Mar 11 12:42:53 2012 +0200 +++ b/src/lib-index/mail-index-sync-keywords.c Fri May 04 00:41:51 2012 +0300 @@ -215,7 +215,7 @@ if (!mail_index_lookup_seq_range(view, uid1, uid2, &seq1, &seq2)) return 1; - mail_index_sync_write_seq_update(ctx, seq1, seq2); + view->map->rec_map->records_changed = TRUE; mail_index_modseq_update_keyword(ctx->modseq_ctx, keyword_idx, seq1, seq2); @@ -337,7 +337,7 @@ &seq1, &seq2)) continue; - mail_index_sync_write_seq_update(ctx, seq1, seq2); + map->rec_map->records_changed = TRUE; mail_index_modseq_reset_keywords(ctx->modseq_ctx, seq1, seq2); for (seq1--; seq1 < seq2; seq1++) { rec = MAIL_INDEX_MAP_IDX(map, seq1); diff -r 96e3b3ef59dc -r 4c996a5737b0 src/lib-index/mail-index-sync-private.h --- a/src/lib-index/mail-index-sync-private.h Sun Mar 11 12:42:53 2012 +0200 +++ b/src/lib-index/mail-index-sync-private.h Fri May 04 00:41:51 2012 +0300 @@ -60,8 +60,6 @@ struct mail_index_map * mail_index_sync_get_atomic_map(struct mail_index_sync_map_ctx *ctx); -void mail_index_sync_write_seq_update(struct mail_index_sync_map_ctx *ctx, - uint32_t seq1, uint32_t seq2); void mail_index_sync_init_expunge_handlers(struct mail_index_sync_map_ctx *ctx); void diff -r 96e3b3ef59dc -r 4c996a5737b0 src/lib-index/mail-index-sync-update.c --- a/src/lib-index/mail-index-sync-update.c Sun Mar 11 12:42:53 2012 +0200 +++ b/src/lib-index/mail-index-sync-update.c Fri May 04 00:41:51 2012 +0300 @@ -95,7 +95,6 @@ mail_index_sync_move_to_private_memory(ctx); mail_index_record_map_move_to_private(ctx->view->map); mail_index_modseq_sync_map_replaced(ctx->modseq_ctx); - ctx->view->map->write_atomic = TRUE; return ctx->view->map; } @@ -344,18 +343,6 @@ return 1; } -void mail_index_sync_write_seq_update(struct mail_index_sync_map_ctx *ctx, - uint32_t seq1, uint32_t seq2) -{ - struct mail_index_map *map = ctx->view->map; - - if (map->rec_map->write_seq_first == 0 || - map->rec_map->write_seq_first > seq1) - map->rec_map->write_seq_first = seq1; - if (map->rec_map->write_seq_last < seq2) - map->rec_map->write_seq_last = seq2; -} - static int sync_append(const struct mail_index_record *rec, struct mail_index_sync_map_ctx *ctx) { @@ -396,9 +383,7 @@ map->rec_map->last_appended_uid = rec->uid; new_flags = rec->flags; - mail_index_sync_write_seq_update(ctx, - map->rec_map->records_count, - map->rec_map->records_count); + map->rec_map->records_changed = TRUE; mail_index_modseq_append(ctx->modseq_ctx, map->rec_map->records_count); } @@ -426,7 +411,7 @@ if (!mail_index_lookup_seq_range(view, u->uid1, u->uid2, &seq1, &seq2)) return 1; - mail_index_sync_write_seq_update(ctx, seq1, seq2); + view->map->rec_map->records_changed = TRUE; if (!MAIL_TRANSACTION_FLAG_UPDATE_IS_INTERNAL(u)) { mail_index_modseq_update_flags(ctx->modseq_ctx, u->add_flags | u->remove_flags, @@ -483,7 +468,7 @@ buffer_write(map->hdr_copy_buf, u->offset, u + 1, u->size); map->hdr_base = map->hdr_copy_buf->data; - map->write_base_header = TRUE; + map->header_changed = TRUE; /* @UNSAFE */ if ((uint32_t)(u->offset + u->size) <= sizeof(map->hdr)) { @@ -979,7 +964,7 @@ had_dirty = (map->hdr.flags & MAIL_INDEX_HDR_FLAG_HAVE_DIRTY) != 0; if (had_dirty) { map->hdr.flags &= ~MAIL_INDEX_HDR_FLAG_HAVE_DIRTY; - map->write_base_header = TRUE; + map->header_changed = TRUE; } if (map->hdr_base != map->hdr_copy_buf->data) { diff -r 96e3b3ef59dc -r 4c996a5737b0 src/lib-index/mail-index-write.c --- a/src/lib-index/mail-index-write.c Sun Mar 11 12:42:53 2012 +0200 +++ b/src/lib-index/mail-index-write.c Fri May 04 00:41:51 2012 +0300 @@ -119,152 +119,21 @@ return ret; } -static int mail_index_write_map_over(struct mail_index *index) -{ - struct mail_index_map *map = index->map; - struct mail_index_record_map *rec_map = map->rec_map; - unsigned int base_size; - - if (MAIL_INDEX_IS_IN_MEMORY(index)) - return 0; - - /* write extended headers */ - if (map->write_ext_header) { - base_size = map->hdr.base_header_size; - if (pwrite_full(index->fd, - CONST_PTR_OFFSET(map->hdr_base, base_size), - map->hdr.header_size - base_size, - base_size) < 0) - return -1; - } - - /* write records. */ - if (rec_map->write_seq_first != 0) { - size_t rec_offset = - (rec_map->write_seq_first-1) * map->hdr.record_size; - size_t recs_size = map->hdr.record_size * - (rec_map->write_seq_last - - rec_map->write_seq_first + 1); - - if (pwrite_full(index->fd, - CONST_PTR_OFFSET(rec_map->records, rec_offset), - recs_size, - map->hdr.header_size + rec_offset) < 0) - return -1; - } - - /* Write base header last. If we happen to crash in above pwrites, it - doesn't matter because we haven't yet written log file offsets, so - all the changes will be re-applied and the header/data state will - stay valid. - - The base header changes practically always, so - map->write_base_header might not be TRUE here in all situations. - It's used only to figure out if we want to write the map at all. */ - base_size = I_MIN(map->hdr.base_header_size, sizeof(map->hdr)); - if (pwrite_full(index->fd, &map->hdr, base_size, 0) < 0) - return -1; - return 0; -} - -static bool mail_index_has_last_changed(struct mail_index *index) -{ - struct mail_index_header hdr; - int ret; - - if ((ret = pread_full(index->fd, &hdr, sizeof(hdr), 0)) <= 0) { - if (ret < 0 && errno != ESTALE) - mail_index_set_syscall_error(index, "pread_full()"); From dovecot at dovecot.org Fri May 4 00:49:42 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 04 May 2012 00:49:42 +0300 Subject: dovecot-2.2: lib-index: Removed unused locking code. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/10a475d725a1 changeset: 14231:10a475d725a1 user: Timo Sirainen date: Fri May 04 00:48:45 2012 +0300 description: lib-index: Removed unused locking code. diffstat: src/lib-index/mail-index-lock.c | 91 ++++++++---------------------------- src/lib-index/mail-index-map-read.c | 1 - src/lib-index/mail-index-private.h | 5 +- src/lib-index/mail-index.c | 3 - 4 files changed, 21 insertions(+), 79 deletions(-) diffs (208 lines): diff -r 4c996a5737b0 -r 10a475d725a1 src/lib-index/mail-index-lock.c --- a/src/lib-index/mail-index-lock.c Fri May 04 00:41:51 2012 +0300 +++ b/src/lib-index/mail-index-lock.c Fri May 04 00:48:45 2012 +0300 @@ -36,27 +36,15 @@ timeout_secs, lock_r); } -static int mail_index_lock(struct mail_index *index, int lock_type, +static int mail_index_lock(struct mail_index *index, unsigned int timeout_secs, unsigned int *lock_id_r) { int ret; - i_assert(lock_type == F_RDLCK || lock_type == F_WRLCK); - - if (lock_type == F_RDLCK && index->lock_type != F_UNLCK) { + if (index->lock_type != F_UNLCK) { + /* file is already locked */ index->shared_lock_count++; *lock_id_r = index->lock_id_counter; - ret = 1; - } else if (lock_type == F_WRLCK && index->lock_type == F_WRLCK) { - index->excl_lock_count++; - *lock_id_r = index->lock_id_counter + 1; - ret = 1; - } else { - ret = 0; - } - - if (ret > 0) { - /* file is already locked */ return 1; } @@ -65,39 +53,25 @@ /* FIXME: exclusive locking will rewrite the index file every time. shouldn't really be needed.. reading doesn't require locks then, though */ - if (lock_type == F_WRLCK) - return 0; - index->shared_lock_count++; index->lock_type = F_RDLCK; *lock_id_r = index->lock_id_counter; return 1; } - if (index->file_lock == NULL) { - i_assert(index->lock_type == F_UNLCK); - ret = mail_index_lock_fd(index, index->filepath, index->fd, - lock_type, timeout_secs, - &index->file_lock); - } else { - i_assert(index->lock_type == F_RDLCK && lock_type == F_WRLCK); - ret = file_lock_try_update(index->file_lock, lock_type); - } + i_assert(index->lock_type == F_UNLCK); + ret = mail_index_lock_fd(index, index->filepath, index->fd, + F_RDLCK, timeout_secs, + &index->file_lock); if (ret <= 0) return ret; if (index->lock_type == F_UNLCK) index->lock_id_counter += 2; - index->lock_type = lock_type; + index->lock_type = F_RDLCK; - if (lock_type == F_RDLCK) { - index->shared_lock_count++; - *lock_id_r = index->lock_id_counter; - } else { - index->excl_lock_count++; - *lock_id_r = index->lock_id_counter + 1; - } - + index->shared_lock_count++; + *lock_id_r = index->lock_id_counter; return 1; } @@ -125,7 +99,7 @@ timeout_secs = I_MIN(MAIL_INDEX_SHARED_LOCK_TIMEOUT, index->max_lock_timeout_secs); - ret = mail_index_lock(index, F_RDLCK, timeout_secs, lock_id_r); + ret = mail_index_lock(index, timeout_secs, lock_id_r); if (ret > 0) { mail_index_flush_read_cache(index, index->filepath, index->fd, TRUE); @@ -141,48 +115,23 @@ return -1; } -int mail_index_try_lock_exclusive(struct mail_index *index, - unsigned int *lock_id_r) -{ - int ret; - - if ((ret = mail_index_lock(index, F_WRLCK, 0, lock_id_r)) > 0) { - mail_index_flush_read_cache(index, index->filepath, - index->fd, TRUE); - } - return ret; -} - void mail_index_unlock(struct mail_index *index, unsigned int *_lock_id) { unsigned int lock_id = *_lock_id; *_lock_id = 0; - if ((lock_id & 1) == 0) { - /* shared lock */ - if (!mail_index_is_locked(index, lock_id)) { - /* unlocking some older generation of the index file. - we've already closed the file so just ignore this. */ - return; - } - - i_assert(index->shared_lock_count > 0); - index->shared_lock_count--; - } else { - /* exclusive lock */ - i_assert(lock_id == index->lock_id_counter + 1); - i_assert(index->excl_lock_count > 0); - i_assert(index->lock_type == F_WRLCK); - if (--index->excl_lock_count == 0 && - index->shared_lock_count > 0) { - /* drop back to a shared lock. */ - index->lock_type = F_RDLCK; - (void)file_lock_try_update(index->file_lock, F_RDLCK); - } + /* shared lock */ + if (!mail_index_is_locked(index, lock_id)) { + /* unlocking some older generation of the index file. + we've already closed the file so just ignore this. */ + return; } - if (index->shared_lock_count == 0 && index->excl_lock_count == 0) { + i_assert(index->shared_lock_count > 0); + index->shared_lock_count--; + + if (index->shared_lock_count == 0) { index->lock_id_counter += 2; index->lock_type = F_UNLCK; if (index->lock_method != FILE_LOCK_METHOD_DOTLOCK) { diff -r 4c996a5737b0 -r 10a475d725a1 src/lib-index/mail-index-map-read.c --- a/src/lib-index/mail-index-map-read.c Fri May 04 00:41:51 2012 +0300 +++ b/src/lib-index/mail-index-map-read.c Fri May 04 00:48:45 2012 +0300 @@ -400,7 +400,6 @@ { int ret; - i_assert(index->lock_type != F_WRLCK); i_assert(!index->mapping); index->mapping = TRUE; diff -r 4c996a5737b0 -r 10a475d725a1 src/lib-index/mail-index-private.h --- a/src/lib-index/mail-index-private.h Fri May 04 00:41:51 2012 +0300 +++ b/src/lib-index/mail-index-private.h Fri May 04 00:48:45 2012 +0300 @@ -205,7 +205,7 @@ /* syncing will update this if non-NULL */ struct mail_index_transaction_commit_result *sync_commit_result; - int lock_type, shared_lock_count, excl_lock_count; + int lock_type, shared_lock_count; unsigned int lock_id_counter; enum file_lock_method lock_method; unsigned int max_lock_timeout_secs; @@ -275,9 +275,6 @@ /* Returns 0 = ok, -1 = error. */ int mail_index_lock_shared(struct mail_index *index, unsigned int *lock_id_r); -/* Returns 1 = ok, 0 = already locked, -1 = error. */ -int mail_index_try_lock_exclusive(struct mail_index *index, - unsigned int *lock_id_r); void mail_index_unlock(struct mail_index *index, unsigned int *lock_id); /* Returns TRUE if given lock_id is valid. */ bool mail_index_is_locked(struct mail_index *index, unsigned int lock_id); diff -r 4c996a5737b0 -r 10a475d725a1 src/lib-index/mail-index.c --- a/src/lib-index/mail-index.c Fri May 04 00:41:51 2012 +0300 +++ b/src/lib-index/mail-index.c Fri May 04 00:48:45 2012 +0300 @@ -570,7 +570,6 @@ i_strconcat(index->dir, "/", index->prefix, NULL); index->shared_lock_count = 0; - index->excl_lock_count = 0; index->lock_type = F_UNLCK; index->lock_id_counter = 2; @@ -629,7 +628,6 @@ index->lock_id_counter += 2; index->lock_type = F_UNLCK; index->shared_lock_count = 0; - index->excl_lock_count = 0; } void mail_index_close(struct mail_index *index) @@ -695,7 +693,6 @@ i_assert(index->shared_lock_count == 0 || (index->flags & MAIL_INDEX_OPEN_FLAG_NFS_FLUSH) == 0); - i_assert(index->excl_lock_count == 0); if (MAIL_INDEX_IS_IN_MEMORY(index)) return 0; From pigeonhole at rename-it.nl Fri May 4 12:40:51 2012 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Fri, 04 May 2012 11:40:51 +0200 Subject: dovecot-2.1-pigeonhole: LDA Sieve plugin: fixed sieve_before bug... Message-ID: details: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/aa611f912da6 changeset: 1620:aa611f912da6 user: Stephan Bosch date: Fri May 04 11:40:39 2012 +0200 description: LDA Sieve plugin: fixed sieve_before bug that made it only work when mail_debug=yes. diffstat: src/plugins/lda-sieve/lda-sieve-plugin.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (15 lines): diff -r 7f9218fe9fac -r aa611f912da6 src/plugins/lda-sieve/lda-sieve-plugin.c --- a/src/plugins/lda-sieve/lda-sieve-plugin.c Wed May 02 01:17:49 2012 +0200 +++ b/src/plugins/lda-sieve/lda-sieve-plugin.c Fri May 04 11:40:39 2012 +0200 @@ -653,9 +653,9 @@ setting_name = "sieve_before"; sieve_before = mail_user_plugin_getenv(mdctx->dest_user, setting_name); while ( sieve_before != NULL && *sieve_before != '\0' ) { - if ( debug && lda_sieve_multiscript_get_scripts + if ( lda_sieve_multiscript_get_scripts (svinst, setting_name, sieve_before, master_ehandler, - &script_sequence) == 0 ) { + &script_sequence) == 0 && debug ) { sieve_sys_debug(svinst, "%s location not found: %s", setting_name, sieve_before); } From dovecot at dovecot.org Fri May 4 19:01:57 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 04 May 2012 19:01:57 +0300 Subject: dovecot-2.1: lib-storage: Don't set mail_save_context->saving=TR... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/5d9a5011c1bf changeset: 14497:5d9a5011c1bf user: Timo Sirainen date: Fri May 04 19:00:04 2012 +0300 description: lib-storage: Don't set mail_save_context->saving=TRUE when copying via save. diffstat: src/lib-storage/mail-storage.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (13 lines): diff -r 5d45870e2e4a -r 5d9a5011c1bf src/lib-storage/mail-storage.c --- a/src/lib-storage/mail-storage.c Thu May 03 23:26:44 2012 +0300 +++ b/src/lib-storage/mail-storage.c Fri May 04 19:00:04 2012 +0300 @@ -1630,7 +1630,8 @@ return -1; } - (*ctx)->saving = TRUE; + if (!(*ctx)->copying_via_save) + (*ctx)->saving = TRUE; if (box->v.save_begin == NULL) { mail_storage_set_error(box->storage, MAIL_ERROR_NOTPOSSIBLE, "Saving messages not supported"); From dovecot at dovecot.org Fri May 4 19:01:57 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 04 May 2012 19:01:57 +0300 Subject: dovecot-2.1: notify plugin: mailbox_save_using_mail() now sends ... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/8a4d314ef7d0 changeset: 14498:8a4d314ef7d0 user: Timo Sirainen date: Fri May 04 19:01:19 2012 +0300 description: notify plugin: mailbox_save_using_mail() now sends "save" events, not "copy". diffstat: src/plugins/notify/notify-storage.c | 9 ++++++++- 1 files changed, 8 insertions(+), 1 deletions(-) diffs (20 lines): diff -r 5d9a5011c1bf -r 8a4d314ef7d0 src/plugins/notify/notify-storage.c --- a/src/plugins/notify/notify-storage.c Fri May 04 19:00:04 2012 +0300 +++ b/src/plugins/notify/notify-storage.c Fri May 04 19:01:19 2012 +0300 @@ -102,8 +102,15 @@ ctx->dest_mail = lt->tmp_mail; } - if ((ret = lbox->super.copy(ctx, mail)) == 0) + if ((ret = lbox->super.copy(ctx, mail)) < 0) + return -1; + + if (ctx->saving) { + /* we came from mailbox_save_using_mail() */ + notify_contexts_mail_save(ctx->dest_mail); + } else { notify_contexts_mail_copy(mail, ctx->dest_mail); + } return ret; } From dovecot at dovecot.org Fri May 4 19:01:57 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 04 May 2012 19:01:57 +0300 Subject: dovecot-2.1: mail-log: Removed broken "are we saving?" check fro... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/b89b98ebe52b changeset: 14499:b89b98ebe52b user: Timo Sirainen date: Fri May 04 19:01:46 2012 +0300 description: mail-log: Removed broken "are we saving?" check from copy event handling. diffstat: src/plugins/mail-log/mail-log-plugin.c | 16 +++++----------- 1 files changed, 5 insertions(+), 11 deletions(-) diffs (26 lines): diff -r 8a4d314ef7d0 -r b89b98ebe52b src/plugins/mail-log/mail-log-plugin.c --- a/src/plugins/mail-log/mail-log-plugin.c Fri May 04 19:01:19 2012 +0300 +++ b/src/plugins/mail-log/mail-log-plugin.c Fri May 04 19:01:46 2012 +0300 @@ -360,17 +360,11 @@ (struct mail_log_mail_txn_context *)txn; const char *desc; - if (dst->saving) { - /* we came from mailbox_save_using_mail() */ - mail_log_append_mail_message(ctx, dst, - MAIL_LOG_EVENT_SAVE, "save"); - } else { - desc = t_strdup_printf("copy from %s", - str_sanitize(mailbox_get_name(src->box), - MAILBOX_NAME_LOG_LEN)); - mail_log_append_mail_message(ctx, dst, - MAIL_LOG_EVENT_COPY, desc); - } + desc = t_strdup_printf("copy from %s", + str_sanitize(mailbox_get_name(src->box), + MAILBOX_NAME_LOG_LEN)); + mail_log_append_mail_message(ctx, dst, + MAIL_LOG_EVENT_COPY, desc); } static void mail_log_mail_expunge(void *txn, struct mail *mail) From dovecot at dovecot.org Fri May 4 20:07:10 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 04 May 2012 20:07:10 +0300 Subject: dovecot-2.1: doveadm mailbox mutf7: If input data is invalid, ex... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/3689eced9381 changeset: 14500:3689eced9381 user: Timo Sirainen date: Fri May 04 20:07:05 2012 +0300 description: doveadm mailbox mutf7: If input data is invalid, exit with EX_DATAERR. diffstat: src/doveadm/doveadm-mutf7.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (17 lines): diff -r b89b98ebe52b -r 3689eced9381 src/doveadm/doveadm-mutf7.c --- a/src/doveadm/doveadm-mutf7.c Fri May 04 19:01:46 2012 +0300 +++ b/src/doveadm/doveadm-mutf7.c Fri May 04 20:07:05 2012 +0300 @@ -40,11 +40,13 @@ if (imap_utf8_to_utf7(argv[i], str) < 0) { i_error("Mailbox name not valid UTF-8: %s", argv[i]); + doveadm_exit_code = EX_DATAERR; } } else { if (imap_utf7_to_utf8(argv[i], str) < 0) { i_error("Mailbox name not valid mUTF-7: %s", argv[i]); + doveadm_exit_code = EX_DATAERR; } } printf("%s\n", str_c(str)); From dovecot at dovecot.org Fri May 4 20:14:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 04 May 2012 20:14:30 +0300 Subject: dovecot-2.1: dbox: Metadata read buffer's size was supposed to b... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/fa6662ab4df3 changeset: 14501:fa6662ab4df3 user: Timo Sirainen date: Fri May 04 20:14:20 2012 +0300 description: dbox: Metadata read buffer's size was supposed to be unlimited, not zero. This was broken by 59f5238c6e61, which itself had fixed another bug. diffstat: src/lib-storage/index/dbox-common/dbox-file.c | 5 +++-- src/lib-storage/index/dbox-multi/mdbox-purge.c | 6 ++++-- 2 files changed, 7 insertions(+), 4 deletions(-) diffs (45 lines): diff -r 3689eced9381 -r fa6662ab4df3 src/lib-storage/index/dbox-common/dbox-file.c --- a/src/lib-storage/index/dbox-common/dbox-file.c Fri May 04 20:07:05 2012 +0300 +++ b/src/lib-storage/index/dbox-common/dbox-file.c Fri May 04 20:14:20 2012 +0300 @@ -407,7 +407,7 @@ /* skip over the actual metadata */ buf_size = i_stream_get_max_buffer_size(file->input); - i_stream_set_max_buffer_size(file->input, 0); + i_stream_set_max_buffer_size(file->input, (size_t)-1); while ((line = i_stream_read_next_line(file->input)) != NULL) { if (*line == DBOX_METADATA_OLDV1_SPACE || *line == '\0') { /* end of metadata */ @@ -657,7 +657,8 @@ ret = 0; buf_size = i_stream_get_max_buffer_size(file->input); - i_stream_set_max_buffer_size(file->input, 0); + /* use unlimited line length for metadata */ + i_stream_set_max_buffer_size(file->input, (size_t)-1); while ((line = i_stream_read_next_line(file->input)) != NULL) { if (*line == DBOX_METADATA_OLDV1_SPACE || *line == '\0') { /* end of metadata */ diff -r 3689eced9381 -r fa6662ab4df3 src/lib-storage/index/dbox-multi/mdbox-purge.c --- a/src/lib-storage/index/dbox-multi/mdbox-purge.c Fri May 04 20:07:05 2012 +0300 +++ b/src/lib-storage/index/dbox-multi/mdbox-purge.c Fri May 04 20:14:20 2012 +0300 @@ -107,7 +107,8 @@ o_stream_send(output, &meta_hdr, sizeof(meta_hdr)); buf_size = i_stream_get_max_buffer_size(file->input); - i_stream_set_max_buffer_size(file->input, 0); + /* use unlimited line length for metadata */ + i_stream_set_max_buffer_size(file->input, (size_t)-1); while ((line = i_stream_read_next_line(file->input)) != NULL) { if (*line == '\0') { /* end of metadata */ @@ -140,7 +141,8 @@ return ret; buf_size = i_stream_get_max_buffer_size(file->input); - i_stream_set_max_buffer_size(file->input, 0); + /* use unlimited line length for metadata */ + i_stream_set_max_buffer_size(file->input, (size_t)-1); while ((line = i_stream_read_next_line(file->input)) != NULL) { if (*line == '\0') { /* end of metadata */ From dovecot at dovecot.org Fri May 4 21:41:15 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 04 May 2012 21:41:15 +0300 Subject: dovecot-2.1: Added mail_shared_explicit_inbox setting. Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/40a544fc4778 changeset: 14502:40a544fc4778 user: Timo Sirainen date: Fri May 04 21:40:57 2012 +0300 description: Added mail_shared_explicit_inbox setting. diffstat: doc/example-config/conf.d/10-mail.conf | 2 ++ src/lib-storage/mail-storage-settings.c | 2 ++ src/lib-storage/mail-storage-settings.h | 1 + src/lib-storage/mailbox-list.c | 6 ++++-- 4 files changed, 9 insertions(+), 2 deletions(-) diffs (65 lines): diff -r fa6662ab4df3 -r 40a544fc4778 doc/example-config/conf.d/10-mail.conf --- a/doc/example-config/conf.d/10-mail.conf Fri May 04 20:14:20 2012 +0300 +++ b/doc/example-config/conf.d/10-mail.conf Fri May 04 21:40:57 2012 +0300 @@ -97,6 +97,8 @@ # List the shared/ namespace only if there are visible shared mailboxes. #list = children #} +# Should shared INBOX be visible as "shared/user" or "shared/user/INBOX"? +#mail_shared_explicit_inbox = yes # System user and group used to access mails. If you use multiple, userdb # can override these by returning uid or gid fields. You can use either numbers diff -r fa6662ab4df3 -r 40a544fc4778 src/lib-storage/mail-storage-settings.c --- a/src/lib-storage/mail-storage-settings.c Fri May 04 20:14:20 2012 +0300 +++ b/src/lib-storage/mail-storage-settings.c Fri May 04 21:40:57 2012 +0300 @@ -48,6 +48,7 @@ DEF(SET_BOOL, mail_debug), DEF(SET_BOOL, mail_full_filesystem_access), DEF(SET_BOOL, maildir_stat_dirs), + DEF(SET_BOOL, mail_shared_explicit_inbox), DEF(SET_ENUM, lock_method), DEF(SET_STR, pop3_uidl_format), @@ -78,6 +79,7 @@ .mail_debug = FALSE, .mail_full_filesystem_access = FALSE, .maildir_stat_dirs = FALSE, + .mail_shared_explicit_inbox = TRUE, .lock_method = "fcntl:flock:dotlock", .pop3_uidl_format = "%08Xu%08Xv" }; diff -r fa6662ab4df3 -r 40a544fc4778 src/lib-storage/mail-storage-settings.h --- a/src/lib-storage/mail-storage-settings.h Fri May 04 20:14:20 2012 +0300 +++ b/src/lib-storage/mail-storage-settings.h Fri May 04 21:40:57 2012 +0300 @@ -33,6 +33,7 @@ bool mail_debug; bool mail_full_filesystem_access; bool maildir_stat_dirs; + bool mail_shared_explicit_inbox; const char *lock_method; const char *pop3_uidl_format; diff -r fa6662ab4df3 -r 40a544fc4778 src/lib-storage/mailbox-list.c --- a/src/lib-storage/mailbox-list.c Fri May 04 20:14:20 2012 +0300 +++ b/src/lib-storage/mailbox-list.c Fri May 04 21:40:57 2012 +0300 @@ -428,7 +428,8 @@ ns_sep = mail_namespace_get_sep(ns); if (*storage_name == '\0' && ns->type == NAMESPACE_SHARED && - (ns->flags & NAMESPACE_FLAG_INBOX_ANY) != 0) { + (ns->flags & NAMESPACE_FLAG_INBOX_ANY) != 0 && + !list->mail_set->mail_shared_explicit_inbox) { /* opening shared/$user. it's the same as INBOX. */ storage_name = "INBOX"; } @@ -508,7 +509,8 @@ return vname; } if (strcmp(vname, "INBOX") == 0 && - (list->ns->flags & NAMESPACE_FLAG_INBOX_ANY) != 0) { + (list->ns->flags & NAMESPACE_FLAG_INBOX_ANY) != 0 && + !list->mail_set->mail_shared_explicit_inbox) { /* convert to shared/$user, we don't really care about the INBOX suffix here. */ vname = ""; From dovecot at dovecot.org Fri May 4 21:52:39 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 04 May 2012 21:52:39 +0300 Subject: dovecot-2.1: systemd: If a socket is enabled in systemd but not ... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/4a3bf567da54 changeset: 14503:4a3bf567da54 user: Timo Sirainen date: Fri May 04 21:52:05 2012 +0300 description: systemd: If a socket is enabled in systemd but not in Dovecot config, close it. Based on patch by Michal Hlavinka. diffstat: src/master/service-listen.c | 90 +++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 90 insertions(+), 0 deletions(-) diffs (114 lines): diff -r 40a544fc4778 -r 4a3bf567da54 src/master/service-listen.c --- a/src/master/service-listen.c Fri May 04 21:40:57 2012 +0300 +++ b/src/master/service-listen.c Fri May 04 21:52:05 2012 +0300 @@ -14,6 +14,7 @@ #include #include #include +#include #define MIN_BACKLOG 4 #define MAX_BACKLOG 511 @@ -242,6 +243,90 @@ return ret; } +#ifdef HAVE_SYSTEMD +static int get_socket_info(int fd, unsigned int *family, unsigned int *port) +{ + union sockaddr_union { + struct sockaddr sa; + struct sockaddr_in in4; + struct sockaddr_in6 in6; + } sockaddr; + socklen_t l; + + if (port) *port = -1; + if (family) *family = -1; + + memset(&sockaddr, 0, sizeof(sockaddr)); + l = sizeof(sockaddr); + + if (getsockname(fd, &sockaddr.sa, &l) < 0) + return -errno; + + if (family) *family = sockaddr.sa.sa_family; + if (port) { + if (sockaddr.sa.sa_family == AF_INET) { + if (l < sizeof(struct sockaddr_in)) + return -EINVAL; + + *port = ntohs(sockaddr.in4.sin_port); + } else { + if (l < sizeof(struct sockaddr_in6)) + return -EINVAL; + + *port = ntohs(sockaddr.in6.sin6_port); + } + } + return 0; +} + +static int services_verify_systemd(struct service_list *service_list) +{ + struct service *const *services; + static int sd_fds = -1; + int fd, fd_max; + + if (sd_fds < 0) { + sd_fds = sd_listen_fds(0); + if (sd_fds == -1) { + i_error("sd_listen_fds() failed: %m"); + return -1; + } + } + + fd_max = SD_LISTEN_FDS_START + sd_fds - 1; + for (fd = SD_LISTEN_FDS_START; fd <= fd_max; fd++) { + if (sd_is_socket_inet(fd, 0, SOCK_STREAM, 1, 0) > 0) { + int found = FALSE; + unsigned int port, family; + get_socket_info(fd, &family, &port); + + array_foreach(&service_list->services, services) { + struct service_listener *const *listeners; + + array_foreach(&(*services)->listeners, listeners) { + struct service_listener *l = *listeners; + if (l->type != SERVICE_LISTENER_INET) + continue; + if (l->set.inetset.set->port == port && + l->set.inetset.ip.family == family) { + found = TRUE; + break; + } + } + if (found) break; + } + if (!found) { + i_error("systemd listens on port %d, but it's not configured in Dovecot. Closing.",port); + if (shutdown(fd, SHUT_RDWR) < 0 && errno != ENOTCONN) + i_error("shutdown() failed: %m"); + if (dup2(null_fd, fd) < 0) + i_error("dup2() failed: %m"); + } + } + } +} +#endif + int services_listen(struct service_list *service_list) { struct service *const *services; @@ -252,6 +337,11 @@ if (ret2 < ret) ret = ret2; } + +#ifdef HAVE_SYSTEMD + if (ret == 0) + services_verify_systemd(service_list); +#endif return ret; } From dovecot at dovecot.org Fri May 4 22:55:39 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 04 May 2012 22:55:39 +0300 Subject: dovecot-2.1: shared mailboxes: If successful userdb lookup doesn... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/6dabd9545d48 changeset: 14504:6dabd9545d48 user: Timo Sirainen date: Fri May 04 22:55:29 2012 +0300 description: shared mailboxes: If successful userdb lookup doesn't return home, fallback to mail_home setting for %%h. diffstat: src/lib-storage/index/shared/shared-storage.c | 14 +++--- src/lib-storage/mail-user.c | 57 ++++++++++++++++++--------- 2 files changed, 45 insertions(+), 26 deletions(-) diffs (139 lines): diff -r 4a3bf567da54 -r 6dabd9545d48 src/lib-storage/index/shared/shared-storage.c --- a/src/lib-storage/index/shared/shared-storage.c Fri May 04 21:52:05 2012 +0300 +++ b/src/lib-storage/index/shared/shared-storage.c Fri May 04 22:55:29 2012 +0300 @@ -256,6 +256,13 @@ owner = mail_user_alloc(userdomain, user->set_info, user->unexpanded_set); owner->autocreated = TRUE; + if (mail_user_init(owner, &error) < 0) { + mailbox_list_set_critical(list, + "Couldn't create namespace '%s' for user %s: %s", + ns->prefix, userdomain, error); + mail_user_unref(&owner); + return -1; + } if (!var_has_key(storage->location, 'h', "home")) ret = 1; else { @@ -268,13 +275,6 @@ return -1; } } - if (mail_user_init(owner, &error) < 0) { - mailbox_list_set_critical(list, - "Couldn't create namespace '%s' for user %s: %s", - ns->prefix, userdomain, error); - mail_user_unref(&owner); - return -1; - } /* create the new namespace */ new_ns = i_new(struct mail_namespace, 1); diff -r 4a3bf567da54 -r 6dabd9545d48 src/lib-storage/mail-user.c --- a/src/lib-storage/mail-user.c Fri May 04 21:52:05 2012 +0300 +++ b/src/lib-storage/mail-user.c Fri May 04 22:55:29 2012 +0300 @@ -96,19 +96,23 @@ { const struct mail_storage_settings *mail_set; const char *home, *key, *value; + bool need_home_dir; - if (user->_home == NULL && - settings_vars_have_key(user->set_info, user->set, - 'h', "home", &key, &value) && - mail_user_get_home(user, &home) <= 0) { + need_home_dir = user->_home == NULL && + settings_vars_have_key(user->set_info, user->set, + 'h', "home", &key, &value); + + /* expand mail_home setting before calling mail_user_get_home() */ + settings_var_expand(user->set_info, user->set, + user->pool, mail_user_var_expand_table(user)); + + if (need_home_dir && mail_user_get_home(user, &home) <= 0) { *error_r = t_strdup_printf( "userdb didn't return a home directory, " "but %s used it (%%h): %s", key, value); return -1; } - settings_var_expand(user->set_info, user->set, - user->pool, mail_user_var_expand_table(user)); if (mail_user_expand_plugins_envs(user, error_r) < 0) return -1; @@ -265,7 +269,7 @@ return path; } -int mail_user_get_home(struct mail_user *user, const char **home_r) +static int mail_user_userdb_lookup_home(struct mail_user *user) { struct auth_user_info info; struct auth_user_reply reply; @@ -273,6 +277,8 @@ const char *username, *const *fields; int ret; + i_assert(!user->home_looked_up); + memset(&info, 0, sizeof(info)); info.service = user->service; if (user->local_ip != NULL) @@ -280,12 +286,6 @@ if (user->remote_ip != NULL) info.remote_ip = *user->remote_ip; - if (user->home_looked_up) { - *home_r = user->_home; - return user->_home != NULL ? 1 : 0; - } - *home_r = NULL; - if (mail_user_auth_master_conn == NULL) return 0; @@ -293,18 +293,37 @@ ret = auth_master_user_lookup(mail_user_auth_master_conn, user->username, &info, userdb_pool, &username, &fields); - if (ret >= 0) { + if (ret > 0) { auth_user_fields_parse(fields, userdb_pool, &reply); - user->_home = ret == 0 ? NULL : - p_strdup(user->pool, reply.home); - user->home_looked_up = TRUE; - ret = user->_home != NULL ? 1 : 0; - *home_r = user->_home; + user->_home = p_strdup(user->pool, reply.home); } pool_unref(&userdb_pool); return ret; } +int mail_user_get_home(struct mail_user *user, const char **home_r) +{ + int ret; + + if (user->home_looked_up) { + *home_r = user->_home; + return user->_home != NULL ? 1 : 0; + } + + ret = mail_user_userdb_lookup_home(user); + if (ret < 0) + return -1; + + if (ret > 0 && user->_home == NULL && *user->set->mail_home != '\0') { + /* no home in userdb, fallback to mail_home setting */ + user->_home = user->set->mail_home; + } + user->home_looked_up = TRUE; + + *home_r = user->_home; + return user->_home != NULL ? 1 : 0; +} + bool mail_user_is_plugin_loaded(struct mail_user *user, struct module *module) { const char *const *plugins; From dovecot at dovecot.org Fri May 4 23:50:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 04 May 2012 23:50:27 +0300 Subject: dovecot-2.1: mail-log: Memory leak fix Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/2a42fe2d42f3 changeset: 14505:2a42fe2d42f3 user: Timo Sirainen date: Fri May 04 23:50:22 2012 +0300 description: mail-log: Memory leak fix diffstat: src/plugins/mail-log/mail-log-plugin.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff -r 6dabd9545d48 -r 2a42fe2d42f3 src/plugins/mail-log/mail-log-plugin.c --- a/src/plugins/mail-log/mail-log-plugin.c Fri May 04 22:55:29 2012 +0300 +++ b/src/plugins/mail-log/mail-log-plugin.c Fri May 04 23:50:22 2012 +0300 @@ -228,6 +228,8 @@ wanted_fields |= MAIL_FETCH_VIRTUAL_SIZE; mail_add_temp_wanted_fields(mail, wanted_fields, wanted_headers); + if (wanted_headers != NULL) + mailbox_header_lookup_unref(&wanted_headers); } static void From dovecot at dovecot.org Fri May 4 23:56:56 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 04 May 2012 23:56:56 +0300 Subject: dovecot-2.1: acl: Memory leak fixes Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/5df8bac9a0c6 changeset: 14506:5df8bac9a0c6 user: Timo Sirainen date: Fri May 04 23:56:51 2012 +0300 description: acl: Memory leak fixes diffstat: src/plugins/acl/acl-backend-vfile-acllist.c | 2 +- src/plugins/acl/acl-cache.c | 4 ++++ 2 files changed, 5 insertions(+), 1 deletions(-) diffs (26 lines): diff -r 2a42fe2d42f3 -r 5df8bac9a0c6 src/plugins/acl/acl-backend-vfile-acllist.c --- a/src/plugins/acl/acl-backend-vfile-acllist.c Fri May 04 23:50:22 2012 +0300 +++ b/src/plugins/acl/acl-backend-vfile-acllist.c Fri May 04 23:56:51 2012 +0300 @@ -81,7 +81,7 @@ path = acl_list_get_path(backend); if (path == NULL) { /* we're never going to build acllist for this namespace. */ - i_array_init(&backend->acllist, 1); + acllist_clear(backend, 0); return 0; } diff -r 2a42fe2d42f3 -r 5df8bac9a0c6 src/plugins/acl/acl-cache.c --- a/src/plugins/acl/acl-cache.c Fri May 04 23:50:22 2012 +0300 +++ b/src/plugins/acl/acl-cache.c Fri May 04 23:56:51 2012 +0300 @@ -75,6 +75,10 @@ if (obj_cache->my_current_rights != NULL && obj_cache->my_current_rights != &negative_cache_entry) acl_cache_mask_deinit(&obj_cache->my_current_rights); + if (obj_cache->my_rights != NULL) + acl_cache_mask_deinit(&obj_cache->my_rights); + if (obj_cache->my_neg_rights != NULL) + acl_cache_mask_deinit(&obj_cache->my_neg_rights); i_free(obj_cache->name); i_free(obj_cache); } From dovecot at dovecot.org Sat May 5 00:01:38 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sat, 05 May 2012 00:01:38 +0300 Subject: dovecot-2.1: auth: Added %{session} variable for passdb/userdb q... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/3d8a25a4394d changeset: 14507:3d8a25a4394d user: Timo Sirainen date: Sat May 05 00:01:32 2012 +0300 description: auth: Added %{session} variable for passdb/userdb queries. diffstat: src/auth/auth-request.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (19 lines): diff -r 5df8bac9a0c6 -r 3d8a25a4394d src/auth/auth-request.c --- a/src/auth/auth-request.c Fri May 04 23:56:51 2012 +0300 +++ b/src/auth/auth-request.c Sat May 05 00:01:32 2012 +0300 @@ -1773,6 +1773,7 @@ { '\0', NULL, "login_user" }, { '\0', NULL, "login_username" }, { '\0', NULL, "login_domain" }, + { '\0', NULL, "session" }, { '\0', NULL, NULL } }; @@ -1832,6 +1833,7 @@ auth_request); } } + tab[18].value = escape_func(auth_request->session_id, auth_request); return tab; } From dovecot at dovecot.org Sun May 6 06:44:15 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 06 May 2012 06:44:15 +0300 Subject: dovecot-2.1: auth: Previous change caused crashing with userdb l... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/9da556b9a902 changeset: 14508:9da556b9a902 user: Timo Sirainen date: Sun May 06 06:44:04 2012 +0300 description: auth: Previous change caused crashing with userdb lookups diffstat: src/auth/auth-request.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (13 lines): diff -r 3d8a25a4394d -r 9da556b9a902 src/auth/auth-request.c --- a/src/auth/auth-request.c Sat May 05 00:01:32 2012 +0300 +++ b/src/auth/auth-request.c Sun May 06 06:44:04 2012 +0300 @@ -1833,7 +1833,8 @@ auth_request); } } - tab[18].value = escape_func(auth_request->session_id, auth_request); + tab[18].value = auth_request->session_id == NULL ? NULL : + escape_func(auth_request->session_id, auth_request); return tab; } From dovecot at dovecot.org Mon May 7 04:06:56 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 07 May 2012 04:06:56 +0300 Subject: dovecot-2.1: Makefile: Remove dovecot-config on make uninstall Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/f7bb8d105710 changeset: 14509:f7bb8d105710 user: Timo Sirainen date: Mon May 07 04:06:49 2012 +0300 description: Makefile: Remove dovecot-config on make uninstall diffstat: Makefile.am | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diffs (13 lines): diff -r 9da556b9a902 -r f7bb8d105710 Makefile.am --- a/Makefile.am Sun May 06 06:44:04 2012 +0300 +++ b/Makefile.am Mon May 07 04:06:49 2012 +0300 @@ -70,6 +70,9 @@ -e "s|^\(LIBDOVECOT_INCLUDE\)=.*$$|\1=-I$(pkgincludedir)|" \ > $(DESTDIR)$(pkglibdir)/dovecot-config +uninstall-hook: + rm $(DESTDIR)$(pkglibdir)/dovecot-config + CLEANFILES = $(datafiles) if HAVE_SYSTEMD CLEANFILES += $systedmsystemunit_DATA From dovecot at dovecot.org Mon May 7 08:45:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 07 May 2012 08:45:27 +0300 Subject: dovecot-2.1: Released v2.1.6. Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/7c249e2a82a9 changeset: 14510:7c249e2a82a9 user: Timo Sirainen date: Mon May 07 08:36:43 2012 +0300 description: Released v2.1.6. diffstat: NEWS | 18 ++++++++++++++++++ configure.in | 2 +- 2 files changed, 19 insertions(+), 1 deletions(-) diffs (35 lines): diff -r f7bb8d105710 -r 7c249e2a82a9 NEWS --- a/NEWS Mon May 07 04:06:49 2012 +0300 +++ b/NEWS Mon May 07 08:36:43 2012 +0300 @@ -1,3 +1,21 @@ +v2.1.6 2012-05-07 Timo Sirainen + + * Session ID is now included by default in auth and login process + log lines. It can be added to mail processes also by adding + %{session} to mail_log_prefix. + + + Added ssl_require_crl setting, which specifies if CRL check must + be successful when verifying client certificates. + + Added mail_shared_explicit_inbox setting to specify if a shared INBOX + should be accessible as "shared/$user" or "shared/$user/INBOX". + - v2.1.5: Using "~/" as mail_location or elsewhere failed to actually + expand it to home directory. + - dbox: Fixed potential assert-crash when reading dbox files. + - trash plugin: Fixed behavior when quota is already over limit. + - mail_log plugin: Logging "copy" event didn't work. + - Proxying to backend server with SSL: Verifying server certificate + name always failed, because it was compared to an IP address. + v2.1.5 2012-04-23 Timo Sirainen * IMAP: When neither the session nor the mailbox has modseq tracking diff -r f7bb8d105710 -r 7c249e2a82a9 configure.in --- a/configure.in Mon May 07 04:06:49 2012 +0300 +++ b/configure.in Mon May 07 08:36:43 2012 +0300 @@ -1,5 +1,5 @@ AC_PREREQ([2.59]) -AC_INIT([Dovecot],[2.1.5],[dovecot at dovecot.org]) +AC_INIT([Dovecot],[2.1.6],[dovecot at dovecot.org]) AC_CONFIG_SRCDIR([src]) AM_INIT_AUTOMAKE([foreign]) From dovecot at dovecot.org Mon May 7 08:45:28 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 07 May 2012 08:45:28 +0300 Subject: dovecot-2.1: Added tag 2.1.6 for changeset 7c249e2a82a9 Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/4fc570461e88 changeset: 14511:4fc570461e88 user: Timo Sirainen date: Mon May 07 08:36:43 2012 +0300 description: Added tag 2.1.6 for changeset 7c249e2a82a9 diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r 7c249e2a82a9 -r 4fc570461e88 .hgtags --- a/.hgtags Mon May 07 08:36:43 2012 +0300 +++ b/.hgtags Mon May 07 08:36:43 2012 +0300 @@ -82,3 +82,4 @@ b9adfd52cb66d5d89d291b76b9845d6361216d12 2.1.3 2c21c940e19d97a772128a7f281cea302e157d73 2.1.4 469cee314d9c54d2d7101ec9e38579fdc9610eaf 2.1.5 +7c249e2a82a9cd33ae15ead6443c3499e16da623 2.1.6 From dovecot at dovecot.org Mon May 7 08:45:28 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 07 May 2012 08:45:28 +0300 Subject: dovecot-2.1: Added signature for changeset 7c249e2a82a9 Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/d3acee47cee4 changeset: 14512:d3acee47cee4 user: Timo Sirainen date: Mon May 07 08:36:54 2012 +0300 description: Added signature for changeset 7c249e2a82a9 diffstat: .hgsigs | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r 4fc570461e88 -r d3acee47cee4 .hgsigs --- a/.hgsigs Mon May 07 08:36:43 2012 +0300 +++ b/.hgsigs Mon May 07 08:36:54 2012 +0300 @@ -45,3 +45,4 @@ b9adfd52cb66d5d89d291b76b9845d6361216d12 0 iEYEABECAAYFAk9jbugACgkQyUhSUUBVislSgwCgpo3f0bsSujItBum/M6js8SzF06YAmwftHlwaOstKeALdjLR5vtF2c5F7 2c21c940e19d97a772128a7f281cea302e157d73 0 iEYEABECAAYFAk+CtkYACgkQyUhSUUBVisknxgCfTJw2YPGJ17HbHRGmbwmCyLqepbMAn17j7IYzUfEU0xkXhCgwEAmk7XO4 469cee314d9c54d2d7101ec9e38579fdc9610eaf 0 iEYEABECAAYFAk+VWqkACgkQyUhSUUBVislnXACfVjPqMmPUvYtXQXwqff0h7N76mZUAn02lPeUCyuyr1TF9e1hGM/sKgmko +7c249e2a82a9cd33ae15ead6443c3499e16da623 0 iEYEABECAAYFAk+nX2sACgkQyUhSUUBVisn7uwCbBD3boxBOGEJ8OYsIJ57n5Cr09FAAoIvhxL6EHRB15AMOw4sPaALJ3/bB From dovecot at dovecot.org Mon May 7 23:41:02 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 07 May 2012 23:41:02 +0300 Subject: dovecot-2.1: acl: Fixed listing users who have only INBOX shared... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/20c9446e537e changeset: 14513:20c9446e537e user: Timo Sirainen date: Mon May 07 23:39:01 2012 +0300 description: acl: Fixed listing users who have only INBOX shared and mail_shared_explicit_inbox=no diffstat: src/plugins/acl/acl-shared-storage.c | 20 +++++++++++++++++++- 1 files changed, 19 insertions(+), 1 deletions(-) diffs (37 lines): diff -r d3acee47cee4 -r 20c9446e537e src/plugins/acl/acl-shared-storage.c --- a/src/plugins/acl/acl-shared-storage.c Mon May 07 08:36:54 2012 +0300 +++ b/src/plugins/acl/acl-shared-storage.c Mon May 07 23:39:01 2012 +0300 @@ -12,6 +12,24 @@ #define SHARED_NS_RETRY_SECS (60*60) +static bool acl_ns_prefix_exists(struct mail_namespace *ns) +{ + struct mailbox *box; + const char *vname; + enum mailbox_existence existence; + bool ret; + + if (ns->list->mail_set->mail_shared_explicit_inbox) + return FALSE; + + vname = t_strndup(ns->prefix, ns->prefix_len-1); + box = mailbox_alloc(ns->list, vname, 0); + ret = mailbox_exists(box, FALSE, &existence) == 0 && + existence == MAILBOX_EXISTENCE_SELECT; + mailbox_free(&box); + return ret; +} + static void acl_shared_namespace_add(struct mail_namespace *ns, struct mail_storage *storage, const char *userdomain) @@ -56,7 +74,7 @@ break; (void)mailbox_list_iter_deinit(&iter); - if (info == NULL) { + if (info == NULL && !acl_ns_prefix_exists(new_ns)) { /* no visible mailboxes, remove the namespace */ mail_namespace_destroy(new_ns); } From dovecot at dovecot.org Mon May 7 23:41:02 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 07 May 2012 23:41:02 +0300 Subject: dovecot-2.1: lib-storage: Don't auto-drop namespaces with only I... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/41f2bcb43dad changeset: 14514:41f2bcb43dad user: Timo Sirainen date: Mon May 07 23:39:34 2012 +0300 description: lib-storage: Don't auto-drop namespaces with only INBOX shared and mail_shared_explicit_inbox=no diffstat: src/lib-storage/mail-storage.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diffs (14 lines): diff -r 20c9446e537e -r 41f2bcb43dad src/lib-storage/mail-storage.c --- a/src/lib-storage/mail-storage.c Mon May 07 23:39:01 2012 +0300 +++ b/src/lib-storage/mail-storage.c Mon May 07 23:39:34 2012 +0300 @@ -765,6 +765,10 @@ return 0; } + /* if this is a shared namespace with only INBOX and + mail_shared_explicit_inbox=no, we'll need to mark the namespace as + usable here since nothing else will. */ + box->list->ns->flags |= NAMESPACE_FLAG_USABLE; return 0; } From dovecot at dovecot.org Tue May 8 01:25:17 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Tue, 08 May 2012 01:25:17 +0300 Subject: dovecot-2.1: layout=fs: Fixes to mail_shared_explicit_inbox=no Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/8cdc7c13d6f2 changeset: 14515:8cdc7c13d6f2 user: Timo Sirainen date: Tue May 08 01:22:32 2012 +0300 description: layout=fs: Fixes to mail_shared_explicit_inbox=no diffstat: src/lib-storage/list/mailbox-list-fs-iter.c | 8 +++++++- 1 files changed, 7 insertions(+), 1 deletions(-) diffs (25 lines): diff -r 41f2bcb43dad -r 8cdc7c13d6f2 src/lib-storage/list/mailbox-list-fs-iter.c --- a/src/lib-storage/list/mailbox-list-fs-iter.c Mon May 07 23:39:34 2012 +0300 +++ b/src/lib-storage/list/mailbox-list-fs-iter.c Tue May 08 01:22:32 2012 +0300 @@ -123,7 +123,8 @@ return t_strconcat("/", fname, NULL); } else { /* child */ - return t_strconcat(dir->storage_name, "/", fname, NULL); + return *fname == '\0' ? dir->storage_name : + t_strconcat(dir->storage_name, "/", fname, NULL); } } @@ -403,6 +404,11 @@ we just want to see its contents (not the INBOX's children). */ root = ""; + } else if (*prefix_vname == '\0') { + /* we need to handle "" explicitly here, because getting + storage name with mail_shared_explicit_inbox=no + would return root=INBOX. */ + root = ""; } else { root = mailbox_list_get_storage_name(ctx->ctx.list, prefix_vname); From dovecot at dovecot.org Tue May 8 01:25:17 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Tue, 08 May 2012 01:25:17 +0300 Subject: dovecot-2.1: lib-storage: Extra check to mail_shared_explicit_in... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/17a635cbb63d changeset: 14516:17a635cbb63d user: Timo Sirainen date: Tue May 08 01:24:01 2012 +0300 description: lib-storage: Extra check to mail_shared_explicit_inbox=no diffstat: src/lib-storage/mailbox-list.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 8cdc7c13d6f2 -r 17a635cbb63d src/lib-storage/mailbox-list.c --- a/src/lib-storage/mailbox-list.c Tue May 08 01:22:32 2012 +0300 +++ b/src/lib-storage/mailbox-list.c Tue May 08 01:24:01 2012 +0300 @@ -508,7 +508,7 @@ and /inBox. */ return vname; } - if (strcmp(vname, "INBOX") == 0 && + if (strcmp(vname, "INBOX") == 0 && list->ns->type == NAMESPACE_SHARED && (list->ns->flags & NAMESPACE_FLAG_INBOX_ANY) != 0 && !list->mail_set->mail_shared_explicit_inbox) { /* convert to shared/$user, we don't really care about the From dovecot at dovecot.org Tue May 8 08:31:37 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Tue, 08 May 2012 08:31:37 +0300 Subject: dovecot-2.1: Compiler warning fixes Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/cb80c575b00c changeset: 14517:cb80c575b00c user: Timo Sirainen date: Tue May 08 08:31:27 2012 +0300 description: Compiler warning fixes diffstat: src/auth/db-checkpassword.c | 14 +++--- src/auth/db-checkpassword.h | 4 +- src/auth/passdb-checkpassword.c | 8 +- src/auth/userdb-checkpassword.c | 4 +- src/lib-storage/mail-search-register-human.c | 18 ++++---- src/lib-storage/mail-search-register-imap.c | 60 ++++++++++++++-------------- src/lib/test-array.c | 2 +- src/lib/test-base64.c | 8 +- 8 files changed, 59 insertions(+), 59 deletions(-) diffs (truncated from 302 to 300 lines): diff -r 17a635cbb63d -r cb80c575b00c src/auth/db-checkpassword.c --- a/src/auth/db-checkpassword.c Tue May 08 01:24:01 2012 +0300 +++ b/src/auth/db-checkpassword.c Tue May 08 08:31:27 2012 +0300 @@ -29,7 +29,7 @@ char *auth_password; db_checkpassword_callback_t *callback; - void *context; + void (*request_callback)(); pid_t pid; int fd_out, fd_in; @@ -115,7 +115,7 @@ extra_fields = t_strsplit_tabescaped(str_c(request->input_buf)); request->callback(request->request, status, extra_fields, - request->context); + request->request_callback); checkpassword_request_free(&request); } @@ -448,7 +448,7 @@ struct auth_request *request, const char *auth_password, db_checkpassword_callback_t *callback, - void *context) + void (*request_callback)()) { struct chkpw_auth_request *chkpw_auth_request; unsigned int output_len; @@ -464,7 +464,7 @@ "Username+password combination too long (%u bytes)", output_len); callback(request, DB_CHECKPASSWORD_STATUS_FAILURE, - NULL, context); + NULL, request_callback); return; } @@ -477,7 +477,7 @@ (void)close(fd_in[1]); } callback(request, DB_CHECKPASSWORD_STATUS_INTERNAL_FAILURE, - NULL, context); + NULL, request_callback); return; } @@ -490,7 +490,7 @@ (void)close(fd_out[0]); (void)close(fd_out[1]); callback(request, DB_CHECKPASSWORD_STATUS_INTERNAL_FAILURE, - NULL, context); + NULL, request_callback); return; } @@ -523,7 +523,7 @@ chkpw_auth_request->output_len = output_len; chkpw_auth_request->input_buf = str_new(default_pool, 256); chkpw_auth_request->callback = callback; - chkpw_auth_request->context = context; + chkpw_auth_request->request_callback = request_callback; chkpw_auth_request->io_in = io_add(fd_in[0], IO_READ, checkpassword_child_input, diff -r 17a635cbb63d -r cb80c575b00c src/auth/db-checkpassword.h --- a/src/auth/db-checkpassword.h Tue May 08 01:24:01 2012 +0300 +++ b/src/auth/db-checkpassword.h Tue May 08 08:31:27 2012 +0300 @@ -13,7 +13,7 @@ typedef void db_checkpassword_callback_t(struct auth_request *request, enum db_checkpassword_status status, const char *const *extra_fields, - void *context); + void (*request_callback)()); struct db_checkpassword * db_checkpassword_init(const char *checkpassword_path, @@ -24,6 +24,6 @@ struct auth_request *request, const char *auth_password, db_checkpassword_callback_t *callback, - void *context); + void (*request_callback)()); #endif diff -r 17a635cbb63d -r cb80c575b00c src/auth/passdb-checkpassword.c --- a/src/auth/passdb-checkpassword.c Tue May 08 01:24:01 2012 +0300 +++ b/src/auth/passdb-checkpassword.c Tue May 08 08:31:27 2012 +0300 @@ -17,9 +17,9 @@ auth_checkpassword_callback(struct auth_request *request, enum db_checkpassword_status status, const char *const *extra_fields, - void *context) + void (*request_callback)()) { - verify_plain_callback_t *callback = context; + verify_plain_callback_t *callback = request_callback; const char *scheme, *crypted_pass = NULL; unsigned int i; @@ -71,9 +71,9 @@ credentials_checkpassword_callback(struct auth_request *request, enum db_checkpassword_status status, const char *const *extra_fields, - void *context) + void (*request_callback)()) { - lookup_credentials_callback_t *callback = context; + lookup_credentials_callback_t *callback = request_callback; const char *scheme, *crypted_pass = NULL; unsigned int i; diff -r 17a635cbb63d -r cb80c575b00c src/auth/userdb-checkpassword.c --- a/src/auth/userdb-checkpassword.c Tue May 08 01:24:01 2012 +0300 +++ b/src/auth/userdb-checkpassword.c Tue May 08 08:31:27 2012 +0300 @@ -16,9 +16,9 @@ userdb_checkpassword_callback(struct auth_request *request, enum db_checkpassword_status status, const char *const *extra_fields, - void *context) + void (*request_callback)()) { - userdb_callback_t *callback = context; + userdb_callback_t *callback = request_callback; unsigned int i; switch (status) { diff -r 17a635cbb63d -r cb80c575b00c src/lib-storage/mail-search-register-human.c --- a/src/lib-storage/mail-search-register-human.c Tue May 08 01:24:01 2012 +0300 +++ b/src/lib-storage/mail-search-register-human.c Tue May 08 08:31:27 2012 +0300 @@ -83,17 +83,17 @@ { \ return arg_new_human_date(ctx, _type, _date_type); \ } -CALLBACK_DATE(before, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_RECEIVED); -CALLBACK_DATE(on, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_RECEIVED); -CALLBACK_DATE(since, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_RECEIVED); +CALLBACK_DATE(before, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_RECEIVED) +CALLBACK_DATE(on, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_RECEIVED) +CALLBACK_DATE(since, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_RECEIVED) -CALLBACK_DATE(sentbefore, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_SENT); -CALLBACK_DATE(senton, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_SENT); -CALLBACK_DATE(sentsince, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_SENT); +CALLBACK_DATE(sentbefore, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_SENT) +CALLBACK_DATE(senton, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_SENT) +CALLBACK_DATE(sentsince, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_SENT) -CALLBACK_DATE(savedbefore, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_SAVED); -CALLBACK_DATE(savedon, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_SAVED); -CALLBACK_DATE(savedsince, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_SAVED); +CALLBACK_DATE(savedbefore, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_SAVED) +CALLBACK_DATE(savedon, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_SAVED) +CALLBACK_DATE(savedsince, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_SAVED) static struct mail_search_arg * arg_new_human_size(struct mail_search_build_context *ctx, diff -r 17a635cbb63d -r cb80c575b00c src/lib-storage/mail-search-register-imap.c --- a/src/lib-storage/mail-search-register-imap.c Tue May 08 01:24:01 2012 +0300 +++ b/src/lib-storage/mail-search-register-imap.c Tue May 08 08:31:27 2012 +0300 @@ -116,18 +116,18 @@ sarg->match_not = _not; \ return sarg; \ } -CALLBACK_FLAG(answered, MAIL_ANSWERED, FALSE); -CALLBACK_FLAG(unanswered, MAIL_ANSWERED, TRUE); -CALLBACK_FLAG(deleted, MAIL_DELETED, FALSE); -CALLBACK_FLAG(undeleted, MAIL_DELETED, TRUE); -CALLBACK_FLAG(draft, MAIL_DRAFT, FALSE); -CALLBACK_FLAG(undraft, MAIL_DRAFT, TRUE); -CALLBACK_FLAG(flagged, MAIL_FLAGGED, FALSE); -CALLBACK_FLAG(unflagged, MAIL_FLAGGED, TRUE); -CALLBACK_FLAG(seen, MAIL_SEEN, FALSE); -CALLBACK_FLAG(unseen, MAIL_SEEN, TRUE); -CALLBACK_FLAG(recent, MAIL_RECENT, FALSE); -CALLBACK_FLAG(old, MAIL_RECENT, TRUE); +CALLBACK_FLAG(answered, MAIL_ANSWERED, FALSE) +CALLBACK_FLAG(unanswered, MAIL_ANSWERED, TRUE) +CALLBACK_FLAG(deleted, MAIL_DELETED, FALSE) +CALLBACK_FLAG(undeleted, MAIL_DELETED, TRUE) +CALLBACK_FLAG(draft, MAIL_DRAFT, FALSE) +CALLBACK_FLAG(undraft, MAIL_DRAFT, TRUE) +CALLBACK_FLAG(flagged, MAIL_FLAGGED, FALSE) +CALLBACK_FLAG(unflagged, MAIL_FLAGGED, TRUE) +CALLBACK_FLAG(seen, MAIL_SEEN, FALSE) +CALLBACK_FLAG(unseen, MAIL_SEEN, TRUE) +CALLBACK_FLAG(recent, MAIL_RECENT, FALSE) +CALLBACK_FLAG(old, MAIL_RECENT, TRUE) static struct mail_search_arg * imap_search_new(struct mail_search_build_context *ctx) @@ -141,7 +141,7 @@ return sarg; } -CALLBACK_STR(keyword, SEARCH_KEYWORDS); +CALLBACK_STR(keyword, SEARCH_KEYWORDS) static struct mail_search_arg * imap_search_unkeyword(struct mail_search_build_context *ctx) @@ -179,17 +179,17 @@ { \ return arg_new_date(ctx, _type, _date_type); \ } -CALLBACK_DATE(before, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_RECEIVED); -CALLBACK_DATE(on, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_RECEIVED); -CALLBACK_DATE(since, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_RECEIVED); +CALLBACK_DATE(before, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_RECEIVED) +CALLBACK_DATE(on, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_RECEIVED) +CALLBACK_DATE(since, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_RECEIVED) -CALLBACK_DATE(sentbefore, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_SENT); -CALLBACK_DATE(senton, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_SENT); -CALLBACK_DATE(sentsince, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_SENT); +CALLBACK_DATE(sentbefore, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_SENT) +CALLBACK_DATE(senton, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_SENT) +CALLBACK_DATE(sentsince, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_SENT) -CALLBACK_DATE(x_savedbefore, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_SAVED); -CALLBACK_DATE(x_savedon, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_SAVED); -CALLBACK_DATE(x_savedsince, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_SAVED); +CALLBACK_DATE(x_savedbefore, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_SAVED) +CALLBACK_DATE(x_savedon, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_SAVED) +CALLBACK_DATE(x_savedsince, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_SAVED) static struct mail_search_arg * arg_new_size(struct mail_search_build_context *ctx, @@ -245,11 +245,11 @@ { \ return arg_new_header(ctx, _type, #_name); \ } -CALLBACK_HDR(bcc, SEARCH_HEADER_ADDRESS); -CALLBACK_HDR(cc, SEARCH_HEADER_ADDRESS); -CALLBACK_HDR(from, SEARCH_HEADER_ADDRESS); -CALLBACK_HDR(to, SEARCH_HEADER_ADDRESS); -CALLBACK_HDR(subject, SEARCH_HEADER_COMPRESS_LWSP); +CALLBACK_HDR(bcc, SEARCH_HEADER_ADDRESS) +CALLBACK_HDR(cc, SEARCH_HEADER_ADDRESS) +CALLBACK_HDR(from, SEARCH_HEADER_ADDRESS) +CALLBACK_HDR(to, SEARCH_HEADER_ADDRESS) +CALLBACK_HDR(subject, SEARCH_HEADER_COMPRESS_LWSP) static struct mail_search_arg * imap_search_header(struct mail_search_build_context *ctx) @@ -291,8 +291,8 @@ { \ return arg_new_body(ctx, _type); \ } -CALLBACK_BODY(body, SEARCH_BODY); -CALLBACK_BODY(text, SEARCH_TEXT); +CALLBACK_BODY(body, SEARCH_BODY) +CALLBACK_BODY(text, SEARCH_TEXT) static struct mail_search_arg * arg_new_interval(struct mail_search_build_context *ctx, @@ -478,7 +478,7 @@ return sarg; } -CALLBACK_STR(x_guid, SEARCH_GUID); +CALLBACK_STR(x_guid, SEARCH_GUID) static struct mail_search_arg * imap_search_x_mailbox(struct mail_search_build_context *ctx) diff -r 17a635cbb63d -r cb80c575b00c src/lib/test-array.c --- a/src/lib/test-array.c Tue May 08 01:24:01 2012 +0300 +++ b/src/lib/test-array.c Tue May 08 08:31:27 2012 +0300 @@ -33,7 +33,7 @@ static void test_array_reverse(void) { ARRAY_DEFINE(intarr, int); - int input[] = { -1234567890, -272585721, 2724859223U, 824725652 }; + int input[] = { -1234567890, -272585721, 272485922, 824725652 }; const int *output; unsigned int i, j; diff -r 17a635cbb63d -r cb80c575b00c src/lib/test-base64.c --- a/src/lib/test-base64.c Tue May 08 01:24:01 2012 +0300 +++ b/src/lib/test-base64.c Tue May 08 08:31:27 2012 +0300 @@ -48,9 +48,9 @@ "aGVs!!!!!" }; static const struct test_base64_decode_output output[] = { - { "hello world", 0, -1 }, - { "foo barits", 0, -1 }, - { "just niin", 1, -1 }, + { "hello world", 0, -1U }, + { "foo barits", 0, -1U }, + { "just niin", 1, -1U }, { "hel", 1, 4 }, { "hel", -1, 4 }, { "hel", -1, 4 } @@ -71,7 +71,7 @@ test_assert(output[i].ret == ret && strcmp(output[i].text, str_c(str)) == 0 && (src_pos == output[i].src_pos || - (output[i].src_pos == (unsigned int)-1 && + (output[i].src_pos == -1U && src_pos == strlen(input[i])))); From dovecot at dovecot.org Tue May 8 08:49:53 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Tue, 08 May 2012 08:49:53 +0300 Subject: dovecot-2.1: Compiler warning fixes Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/328a3cd837f8 changeset: 14518:328a3cd837f8 user: Timo Sirainen date: Tue May 08 08:49:47 2012 +0300 description: Compiler warning fixes diffstat: src/lib/test-bsearch-insert-pos.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diffs (16 lines): diff -r cb80c575b00c -r 328a3cd837f8 src/lib/test-bsearch-insert-pos.c --- a/src/lib/test-bsearch-insert-pos.c Tue May 08 08:31:27 2012 +0300 +++ b/src/lib/test-bsearch-insert-pos.c Tue May 08 08:49:47 2012 +0300 @@ -13,9 +13,9 @@ void test_bsearch_insert_pos(void) { static const unsigned int input[] = { - 1, 5, 9, 15, 16, -1, - 1, 5, 9, 15, 16, 17, -1, - -1 + 1, 5, 9, 15, 16, -1U, + 1, 5, 9, 15, 16, 17, -1U, + -1U }; static const unsigned int max_key = 18; const unsigned int *cur; From dovecot at dovecot.org Wed May 9 06:12:23 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 09 May 2012 06:12:23 +0300 Subject: dovecot-2.1: lib-storage: mailbox_list_get_storage_name/_get_vna... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/5a4eb9955d29 changeset: 14519:5a4eb9955d29 user: Timo Sirainen date: Wed May 09 04:41:47 2012 +0300 description: lib-storage: mailbox_list_get_storage_name/_get_vname() are now public. diffstat: src/lib-storage/mailbox-list-private.h | 3 --- src/lib-storage/mailbox-list.h | 4 ++++ 2 files changed, 4 insertions(+), 3 deletions(-) diffs (27 lines): diff -r 328a3cd837f8 -r 5a4eb9955d29 src/lib-storage/mailbox-list-private.h --- a/src/lib-storage/mailbox-list-private.h Tue May 08 08:49:47 2012 +0300 +++ b/src/lib-storage/mailbox-list-private.h Wed May 09 04:41:47 2012 +0300 @@ -180,9 +180,6 @@ const char *storage_name); const char *mailbox_list_get_unexpanded_path(struct mailbox_list *list, enum mailbox_list_path_type type); -const char *mailbox_list_get_storage_name(struct mailbox_list *list, - const char *vname); -const char *mailbox_list_get_vname(struct mailbox_list *list, const char *name); const char * mailbox_list_get_root_path(const struct mailbox_list_settings *set, enum mailbox_list_path_type type); diff -r 328a3cd837f8 -r 5a4eb9955d29 src/lib-storage/mailbox-list.h --- a/src/lib-storage/mailbox-list.h Tue May 08 08:49:47 2012 +0300 +++ b/src/lib-storage/mailbox-list.h Wed May 09 04:41:47 2012 +0300 @@ -230,6 +230,10 @@ bool mailbox_list_is_valid_create_name(struct mailbox_list *list, const char *name); +const char *mailbox_list_get_storage_name(struct mailbox_list *list, + const char *vname); +const char *mailbox_list_get_vname(struct mailbox_list *list, const char *name); + /* Return full path for the given mailbox name. The name must be a valid existing mailbox name, or NULL to get the root directory. For INDEX=MEMORY it returns "" as the path. */ From dovecot at dovecot.org Wed May 9 06:12:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 09 May 2012 06:12:24 +0300 Subject: dovecot-2.1: lib-storage: Added a mailbox.synced flag, which is ... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/eab6ce7a5ad6 changeset: 14520:eab6ce7a5ad6 user: Timo Sirainen date: Wed May 09 06:11:58 2012 +0300 description: lib-storage: Added a mailbox.synced flag, which is set when mailbox sync has succeeded. diffstat: src/lib-storage/mail-storage-private.h | 2 ++ src/lib-storage/mail-storage.c | 2 ++ 2 files changed, 4 insertions(+), 0 deletions(-) diffs (24 lines): diff -r 5a4eb9955d29 -r eab6ce7a5ad6 src/lib-storage/mail-storage-private.h --- a/src/lib-storage/mail-storage-private.h Wed May 09 04:41:47 2012 +0300 +++ b/src/lib-storage/mail-storage-private.h Wed May 09 06:11:58 2012 +0300 @@ -279,6 +279,8 @@ unsigned int disable_reflink_copy_to:1; /* Don't allow creating any new keywords */ unsigned int disallow_new_keywords:1; + /* Mailbox has been synced at least once */ + unsigned int synced:1; }; struct mail_vfuncs { diff -r 5a4eb9955d29 -r eab6ce7a5ad6 src/lib-storage/mail-storage.c --- a/src/lib-storage/mail-storage.c Wed May 09 04:41:47 2012 +0300 +++ b/src/lib-storage/mail-storage.c Wed May 09 06:11:58 2012 +0300 @@ -1349,6 +1349,8 @@ i_error("Syncing INBOX failed: %s", errormsg); } } + if (ret == 0) + box->synced = TRUE; return ret; } From dovecot at dovecot.org Wed May 9 15:01:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 09 May 2012 15:01:30 +0300 Subject: dovecot-2.1: master: systemd socket verification wasn't actually... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/17b20880453c changeset: 14521:17b20880453c user: Timo Sirainen date: Wed May 09 15:01:20 2012 +0300 description: master: systemd socket verification wasn't actually being called normally. diffstat: src/master/service-listen.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r eab6ce7a5ad6 -r 17b20880453c src/master/service-listen.c --- a/src/master/service-listen.c Wed May 09 06:11:58 2012 +0300 +++ b/src/master/service-listen.c Wed May 09 15:01:20 2012 +0300 @@ -339,7 +339,7 @@ } #ifdef HAVE_SYSTEMD - if (ret == 0) + if (ret > 0) services_verify_systemd(service_list); #endif return ret; From dovecot at dovecot.org Thu May 10 11:27:03 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Thu, 10 May 2012 11:27:03 +0300 Subject: dovecot-2.1: imapc: Use imapc_list_prefix also for listing subsc... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/2f7406e838b0 changeset: 14522:2f7406e838b0 user: Timo Sirainen date: Thu May 10 11:26:53 2012 +0300 description: imapc: Use imapc_list_prefix also for listing subscriptions diffstat: src/lib-storage/index/imapc/imapc-list.c | 10 +++++++++- 1 files changed, 9 insertions(+), 1 deletions(-) diffs (27 lines): diff -r 17b20880453c -r 2f7406e838b0 src/lib-storage/index/imapc/imapc-list.c --- a/src/lib-storage/index/imapc/imapc-list.c Wed May 09 15:01:20 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-list.c Thu May 10 11:26:53 2012 +0300 @@ -518,6 +518,7 @@ (struct imapc_mailbox_list *)_src_list; struct imapc_simple_context ctx; struct imapc_command *cmd; + const char *pattern; char sep; i_assert(src_list->tmp_subscriptions == NULL); @@ -537,7 +538,14 @@ src_list->tmp_subscriptions = mailbox_tree_init(src_list->sep); cmd = imapc_list_simple_context_init(&ctx, src_list); - imapc_command_send(cmd, "LSUB \"\" *"); + if (*src_list->storage->set->imapc_list_prefix == '\0') + pattern = "*"; + else { + pattern = t_strdup_printf("%s%c*", + src_list->storage->set->imapc_list_prefix, + src_list->sep); + } + imapc_command_sendf(cmd, "LSUB \"\" %s", pattern); imapc_simple_run(&ctx); /* replace subscriptions tree in destination */ From pigeonhole at rename-it.nl Fri May 11 03:55:09 2012 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Fri, 11 May 2012 02:55:09 +0200 Subject: dovecot-2.1-pigeonhole: Incorporated sieve_duplicate plugin into... Message-ID: details: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/12bcfe5e748a changeset: 1622:12bcfe5e748a user: Stephan Bosch date: Fri May 11 02:54:42 2012 +0200 description: Incorporated sieve_duplicate plugin into main Pigeonhole tree as a normal extension (vnd.dovecot.duplicate). Also restructured doc dir a bit in the process. diffstat: INSTALL | 13 +- Makefile.am | 2 + configure.in | 2 + doc/Makefile.am | 7 +- doc/editheader.txt | 51 -- doc/extensions/editheader.txt | 51 ++ doc/extensions/include.txt | 32 + doc/extensions/spamtest-virustest.txt | 140 ++++++ doc/extensions/vacation.txt | 95 ++++ doc/extensions/vnd.dovecot.duplicate.txt | 56 ++ doc/include.txt | 32 - doc/rfc/Makefile.am | 32 + doc/rfc/spec-bosch-sieve-duplicate.txt | 224 ++++++++++ doc/rfc/xml/reference.KEYWORDS.xml | 44 + doc/rfc/xml/reference.SIEVE.xml | 17 + doc/rfc/xml/spec-bosch-sieve-duplicate.xml | 159 +++++++ doc/spamtest-virustest.txt | 136 ------ doc/vacation.txt | 95 ---- src/lib-sieve/Makefile.am | 1 + src/lib-sieve/plugins/vnd.dovecot/Makefile.am | 2 +- src/lib-sieve/plugins/vnd.dovecot/duplicate/Makefile.am | 20 + src/lib-sieve/plugins/vnd.dovecot/duplicate/ext-duplicate-common.c | 120 +++++ src/lib-sieve/plugins/vnd.dovecot/duplicate/ext-duplicate-common.h | 40 + src/lib-sieve/plugins/vnd.dovecot/duplicate/ext-duplicate.c | 53 ++ src/lib-sieve/plugins/vnd.dovecot/duplicate/tst-duplicate.c | 151 ++++++ src/lib-sieve/sieve-extensions.c | 3 +- tests/extensions/vnd.dovecot/duplicate/errors.svtest | 18 + tests/extensions/vnd.dovecot/duplicate/errors/syntax.sieve | 16 + tests/extensions/vnd.dovecot/duplicate/execute.svtest | 12 + 29 files changed, 1298 insertions(+), 326 deletions(-) diffs (truncated from 1822 to 300 lines): diff -r f362b56485fa -r 12bcfe5e748a INSTALL --- a/INSTALL Fri May 11 02:45:33 2012 +0200 +++ b/INSTALL Fri May 11 02:54:42 2012 +0200 @@ -346,7 +346,8 @@ to delete and add header fields. The editheader extension requires explicit configuration and is not enabled - for use by default. Refer to doc/editheader.txt for configuration information. + for use by default. Refer to doc/extensions/editheader.txt for configuration + information. - Vacation extension: @@ -354,8 +355,8 @@ automatic replies to incoming email messages. The vacation extension is available by default, but it has its own specific - configuration options. Refer to doc/vacation.txt for settings specific to the - vacation extension. + configuration options. Refer to doc/extensions/vacation.txt for settings + specific to the vacation extension. - Include extension: @@ -363,8 +364,8 @@ into another. The include extension is available by default, but it has its own specific - configuration options. Refer to doc/include.txt for settings specific to the - include extension. + configuration options. Refer to doc/extensions/include.txt for settings + specific to the include extension. - Spamtest and Virustest extensions: @@ -379,7 +380,7 @@ The spamtest, spamtestplus and virustest extensions require explicit configuration and are not enabled for use by default. Refer to - doc/spamtest-virustest.txt for configuration information. + doc/extensions/spamtest-virustest.txt for configuration information. Sieve Interpreter - Migration from CMUSieve (Dovecot v1.0/v1.1) --------------------------------------------------------------- diff -r f362b56485fa -r 12bcfe5e748a Makefile.am --- a/Makefile.am Fri May 11 02:45:33 2012 +0200 +++ b/Makefile.am Fri May 11 02:54:42 2012 +0200 @@ -133,6 +133,8 @@ tests/extensions/editheader/protected.svtest \ tests/extensions/editheader/errors.svtest \ tests/extensions/vnd.dovecot/debug/execute.svtest \ + tests/extensions/vnd.dovecot/duplicate/errors.svtest \ + tests/extensions/vnd.dovecot/duplicate/execute.svtest \ tests/deprecated/notify/basic.svtest \ tests/deprecated/notify/mailto.svtest \ tests/deprecated/notify/errors.svtest \ diff -r f362b56485fa -r 12bcfe5e748a configure.in --- a/configure.in Fri May 11 02:45:33 2012 +0200 +++ b/configure.in Fri May 11 02:54:42 2012 +0200 @@ -97,6 +97,7 @@ doc/man/Makefile doc/example-config/Makefile doc/example-config/conf.d/Makefile +doc/rfc/Makefile src/Makefile src/lib-sieve/Makefile src/lib-sieve/plugins/Makefile @@ -121,6 +122,7 @@ src/lib-sieve/plugins/editheader/Makefile src/lib-sieve/plugins/vnd.dovecot/Makefile src/lib-sieve/plugins/vnd.dovecot/debug/Makefile +src/lib-sieve/plugins/vnd.dovecot/duplicate/Makefile src/lib-sieve-tool/Makefile src/lib-sievestorage/Makefile src/lib-managesieve/Makefile diff -r f362b56485fa -r 12bcfe5e748a doc/Makefile.am --- a/doc/Makefile.am Fri May 11 02:45:33 2012 +0200 +++ b/doc/Makefile.am Fri May 11 02:54:42 2012 +0200 @@ -1,15 +1,14 @@ -SUBDIRS = man example-config +SUBDIRS = man example-config rfc docfiles = \ - vacation.txt \ - spamtest-virustest.txt + script-location-dict.txt if BUILD_DOCS sieve_doc_DATA = $(docfiles) endif EXTRA_DIST = \ - rfc \ devel \ + extensions \ $(docfiles) diff -r f362b56485fa -r 12bcfe5e748a doc/editheader.txt --- a/doc/editheader.txt Fri May 11 02:45:33 2012 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,51 +0,0 @@ -Editheader Extension - -Relevant specifications -======================= - - RFC5293 - doc/rfc/editheader.rfc5293.txt - -Description -=========== - -The editheader extension [RFC5293] enables sieve scripts to interact with other -components that consume or produce header fields by allowing the script to -delete and add header fields. - -Configuration -============= - -The editheader is not available by default and needs to be enabled explicitly by -adding it to the sieve_extensions setting. - -The following settings can be configured for the editheader extension (default -values are indicated): - -sieve_editheader_max_header_size = 2048 - The maximum size in bytes of a header field value passed to the addheader - command. The minimum value for this setting is 1024 bytes. The value is in - bytes, unless followed by a k(ilo). - -sieve_editheader_protected = - A space-separated list of headers that cannot be added to nor removed from the - message header. The `Received:' and `Auto-Submitted:' fields are always - protected and the `Subject:' header cannot be protected, as required by the - RFC specification; adding one of these headers to this setting has no effect. - -Invalid values for the settings above will make the Sieve interpreter log -a warning and revert to the default values. - -Example -======= - -plugin { - # Use editheader - sieve_extensions = +editheader - - # Header fiels must not exceed one 1k - sieve_editheader_max_header_size = 1k - - # Protect special header - sieve_editheader_protected = X-Verified -} - diff -r f362b56485fa -r 12bcfe5e748a doc/extensions/editheader.txt --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/extensions/editheader.txt Fri May 11 02:54:42 2012 +0200 @@ -0,0 +1,51 @@ +Editheader Extension + +Relevant specifications +======================= + + RFC5293 - doc/rfc/editheader.rfc5293.txt + +Description +=========== + +The editheader extension [RFC5293] enables sieve scripts to interact with other +components that consume or produce header fields by allowing the script to +delete and add header fields. + +Configuration +============= + +The editheader is not available by default and needs to be enabled explicitly by +adding it to the sieve_extensions setting. + +The following settings can be configured for the editheader extension (default +values are indicated): + +sieve_editheader_max_header_size = 2048 + The maximum size in bytes of a header field value passed to the addheader + command. The minimum value for this setting is 1024 bytes. The value is in + bytes, unless followed by a k(ilo). + +sieve_editheader_protected = + A space-separated list of headers that cannot be added to nor removed from the + message header. The `Received:' and `Auto-Submitted:' fields are always + protected and the `Subject:' header cannot be protected, as required by the + RFC specification; adding one of these headers to this setting has no effect. + +Invalid values for the settings above will make the Sieve interpreter log +a warning and revert to the default values. + +Example +======= + +plugin { + # Use editheader + sieve_extensions = +editheader + + # Header fiels must not exceed one 1k + sieve_editheader_max_header_size = 1k + + # Protect special header + sieve_editheader_protected = X-Verified +} + diff -r f362b56485fa -r 12bcfe5e748a doc/extensions/include.txt --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/extensions/include.txt Fri May 11 02:54:42 2012 +0200 @@ -0,0 +1,32 @@ +Include Extension + +Relevant Specifications +======================= + + draft-ietf-sieve-include-05 - doc/rfc/draft-ietf-sieve-include-05.txt + +Description +=========== + +The Sieve include extension permits users to include one Sieve script into +another. This can make managing large scripts or multiple sets of scripts much +easier, and allows a site and its users to build up libraries of scripts. Users +are able to include their own personal scripts or site-wide scripts. + +Included scripts can include more scripts of their own, yielding a tree of +included scripts with the main script (typically the user's personal script) at +its root. + +Configuration +============= + +The include extension is available by default. The include extension has its own +specific settings. The following settings can be configured for the include +extension (default values are indicated): + +sieve_include_max_includes = 255 + The maximum number of scripts that may be included. This is the total number + of scripts involved in the include tree. + +sieve_include_max_nesting_depth = 10 + The maximum nesting depth for the include tree. diff -r f362b56485fa -r 12bcfe5e748a doc/extensions/spamtest-virustest.txt --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/extensions/spamtest-virustest.txt Fri May 11 02:54:42 2012 +0200 @@ -0,0 +1,140 @@ +Spamtest and Virustest Extensions + +Relevant Specifications +======================= + + RFC5235 - doc/rfc/spamvirustest.rfc5235.txt + +Description +=========== + +Using the spamtest and virustest extensions (RFC 5235), the Sieve language +provides a uniform and standardized command interface for evaluating spam and +virus tests performed on the message. Users no longer need to know what headers +need to be checked and how the scanner's verdict is represented in the header +field value. They only need to know how to use the spamtest (spamtestplus) and +virustest extensions. This also gives GUI-based Sieve editors the means to +provide a portable and easy to install interface for spam and virus filter +configuration. The burden of specifying which headers need to be checked and how +the scanner output is represented falls onto the Sieve administrator. + +Configuration +============= + +The spamtest, spamtestplus and virustest extensions are not enabled by default +and thus need to be enabled explicitly using the sieve_extensions setting. + +The following settings need to be configured for using the spamtest and +spamtestplus extensions. The virustest extension has identical configuration +settings, but with a `sieve_virustest_' prefix instead of a `sieve_spamtest_' +prefix: + +sieve_spamtest_status_type = "score" / "strlen" / "text" + This specifies the type of status result that the spam/virus scanner produces. + This can either be a numeric score ("score"), a string of identical characters + ("strlen"), e.g. '*******', or a textual description, e.g. `Spam' + or `Not Spam'. + +sieve_spamtest_status_header = [ ":" ] + This specifies the header field that contains the result information of the + spam scanner and it may express the syntax of the content of the header. If no + matching header is found in the message, the spamtest command will match + against "0". + + This is a structured setting. The first part specifies the header field name. + Optionally, a POSIX regular expression follows the header field name, + separated by a colon. Any whitespace directly following the colon is not part + of the regular expression. If the regular expression is omitted, any header + content is accepted and the full header value is used. When a regular + expression is used, it must specify one match value (inside brackets) that + yields the desired spam scanner result. If the header does not match the + regular expression or if no value match is found, the spamtest will match + against "0". + From pigeonhole at rename-it.nl Fri May 11 03:55:09 2012 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Fri, 11 May 2012 02:55:09 +0200 Subject: dovecot-2.1-pigeonhole: Fixed compiler warning. Message-ID: details: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/f362b56485fa changeset: 1621:f362b56485fa user: Stephan Bosch date: Fri May 11 02:45:33 2012 +0200 description: Fixed compiler warning. diffstat: src/lib-sieve/sieve-validator.c | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-) diffs (12 lines): diff -r aa611f912da6 -r f362b56485fa src/lib-sieve/sieve-validator.c --- a/src/lib-sieve/sieve-validator.c Fri May 04 11:40:39 2012 +0200 +++ b/src/lib-sieve/sieve-validator.c Fri May 11 02:45:33 2012 +0200 @@ -1182,8 +1182,6 @@ /* Identifier = "" when the command was previously marked as unknown */ if ( *(cmd_def->identifier) != '\0' ) { - struct sieve_command *cmd; - if ( (cmd_def->type == SCT_COMMAND && ast_type == SAT_TEST) || (cmd_def->type == SCT_TEST && ast_type == SAT_COMMAND) ) { sieve_validator_error( From dovecot at dovecot.org Mon May 14 18:09:21 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 14 May 2012 18:09:21 +0300 Subject: dovecot-2.1: lib-master: Added more default ignore prefixes Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/fac49069993a changeset: 14523:fac49069993a user: Timo Sirainen date: Mon May 14 18:09:10 2012 +0300 description: lib-master: Added more default ignore prefixes diffstat: src/lib-master/mountpoint-list.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff -r 2f7406e838b0 -r fac49069993a src/lib-master/mountpoint-list.c --- a/src/lib-master/mountpoint-list.c Thu May 10 11:26:53 2012 +0300 +++ b/src/lib-master/mountpoint-list.c Mon May 14 18:09:10 2012 +0300 @@ -53,6 +53,8 @@ const char *const mountpoint_list_default_ignore_prefixes[] = { "/cdrom", "/media", + "/sys", + "/proc", NULL }; From dovecot at dovecot.org Mon May 14 18:43:08 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 14 May 2012 18:43:08 +0300 Subject: dovecot-2.1: master: Fix to systemd support Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/20d98b848f37 changeset: 14524:20d98b848f37 user: Timo Sirainen date: Mon May 14 18:42:58 2012 +0300 description: master: Fix to systemd support Patch by Joop Boonen diffstat: src/master/service-listen.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff -r fac49069993a -r 20d98b848f37 src/master/service-listen.c --- a/src/master/service-listen.c Mon May 14 18:09:10 2012 +0300 +++ b/src/master/service-listen.c Mon May 14 18:42:58 2012 +0300 @@ -324,6 +324,7 @@ } } } + return 0; } #endif From dovecot at dovecot.org Mon May 14 19:01:56 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 14 May 2012 19:01:56 +0300 Subject: dovecot-2.1: maildir: If dovecot-uidlist can't be opened as read... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/7f238016db81 changeset: 14525:7f238016db81 user: Timo Sirainen date: Mon May 14 19:01:46 2012 +0300 description: maildir: If dovecot-uidlist can't be opened as read-write, try opening as read-only. diffstat: src/lib-storage/index/maildir/maildir-uidlist.c | 14 ++++++++++++++ 1 files changed, 14 insertions(+), 0 deletions(-) diffs (57 lines): diff -r 20d98b848f37 -r 7f238016db81 src/lib-storage/index/maildir/maildir-uidlist.c --- a/src/lib-storage/index/maildir/maildir-uidlist.c Mon May 14 18:42:58 2012 +0300 +++ b/src/lib-storage/index/maildir/maildir-uidlist.c Mon May 14 19:01:46 2012 +0300 @@ -97,6 +97,7 @@ unsigned int locked_refresh:1; unsigned int unsorted:1; unsigned int have_mailbox_guid:1; + unsigned int opened_readonly:1; }; struct maildir_uidlist_sync_ctx { @@ -702,11 +703,16 @@ struct stat st; uoff_t last_read_offset; int fd, ret; + bool readonly = FALSE; *retry_r = FALSE; if (uidlist->fd == -1) { fd = nfs_safe_open(uidlist->path, O_RDWR); + if (fd == -1 && errno == EACCES) { + fd = nfs_safe_open(uidlist->path, O_RDONLY); + readonly = TRUE; + } if (fd == -1) { if (errno != ENOENT) { mail_storage_set_critical(storage, @@ -807,6 +813,8 @@ (void)unlink(uidlist->path); } else if (ret > 0) { /* success */ + if (readonly) + uidlist->recreate_on_change = TRUE; uidlist->fd = fd; uidlist->fd_dev = st.st_dev; uidlist->fd_ino = st.st_ino; @@ -917,6 +925,10 @@ } uidlist->fd = nfs_safe_open(uidlist->path, O_RDWR); + if (uidlist->fd == -1 && errno == EACCES) { + uidlist->fd = nfs_safe_open(uidlist->path, O_RDONLY); + uidlist->recreate_on_change = TRUE; + } if (uidlist->fd == -1 && errno != ENOENT) { mail_storage_set_critical(uidlist->box->storage, "open(%s) failed: %m", uidlist->path); @@ -1510,6 +1522,8 @@ i_assert(uidlist->initial_hdr_read); if (maildir_uidlist_open_latest(uidlist) < 0) return -1; + if (uidlist->recreate_on_change) + return maildir_uidlist_recreate(uidlist); } i_assert(ctx->first_unwritten_pos != (unsigned int)-1); From dovecot at dovecot.org Mon May 14 19:30:07 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 14 May 2012 19:30:07 +0300 Subject: dovecot-2.1: ldap: If attributes contain ldapAttr=key=template%$... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/4bbc12a87a29 changeset: 14526:4bbc12a87a29 user: Timo Sirainen date: Mon May 14 19:30:03 2012 +0300 description: ldap: If attributes contain ldapAttr=key=template%$ and ldapAttr doesn't exist, skip the field. This makes it work again the same as with v2.0. diffstat: src/auth/db-ldap.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diffs (15 lines): diff -r 7f238016db81 -r 4bbc12a87a29 src/auth/db-ldap.c --- a/src/auth/db-ldap.c Mon May 14 19:01:46 2012 +0300 +++ b/src/auth/db-ldap.c Mon May 14 19:30:03 2012 +0300 @@ -1252,6 +1252,11 @@ /* use the LDAP attribute's value */ } else { /* template */ + if (values[0] == NULL && *field->ldap_attr_name != '\0') { + /* ldapAttr=key=template%$, but ldapAttr doesn't + exist. */ + return values; + } if (values[0] != NULL && values[1] != NULL) { auth_request_log_warning(ctx->auth_request, "ldap", "Multiple values found for '%s', " From dovecot at dovecot.org Mon May 14 21:07:52 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 14 May 2012 21:07:52 +0300 Subject: dovecot-2.1: pop3: Added pop3_uidl_duplicates setting. Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/3a095892242b changeset: 14527:3a095892242b user: Timo Sirainen date: Mon May 14 21:07:43 2012 +0300 description: pop3: Added pop3_uidl_duplicates setting. diffstat: doc/example-config/conf.d/20-pop3.conf | 5 + src/pop3/pop3-client.c | 37 ++- src/pop3/pop3-client.h | 5 +- src/pop3/pop3-commands.c | 272 ++++++++++++++++++++++---------- src/pop3/pop3-settings.c | 4 +- src/pop3/pop3-settings.h | 1 + 6 files changed, 220 insertions(+), 104 deletions(-) diffs (truncated from 509 to 300 lines): diff -r 4bbc12a87a29 -r 3a095892242b doc/example-config/conf.d/20-pop3.conf --- a/doc/example-config/conf.d/20-pop3.conf Mon May 14 19:30:03 2012 +0300 +++ b/doc/example-config/conf.d/20-pop3.conf Mon May 14 21:07:43 2012 +0300 @@ -54,6 +54,11 @@ # won't change those UIDLs. Currently this works only with Maildir. #pop3_save_uidl = no + # What to do about duplicate UIDLs if they exist? + # allow: Show duplicates to clients. + # rename: Append a temporary -2, -3, etc. counter after the UIDL. + #pop3_uidl_duplicates = allow + # POP3 logout format string: # %i - total number of bytes read from client # %o - total number of bytes sent to client diff -r 4bbc12a87a29 -r 3a095892242b src/pop3/pop3-client.c --- a/src/pop3/pop3-client.c Mon May 14 19:30:03 2012 +0300 +++ b/src/pop3/pop3-client.c Mon May 14 21:07:43 2012 +0300 @@ -6,6 +6,7 @@ #include "network.h" #include "istream.h" #include "ostream.h" +#include "crc32.h" #include "str.h" #include "llist.h" #include "hostpid.h" @@ -344,15 +345,20 @@ return NULL; } - if (var_has_key(set->pop3_logout_format, 'u', "uidl_change") && - client->messages_count > 0) - client->message_uidl_hashes_save = TRUE; - client->uidl_keymask = parse_uidl_keymask(client->mail_set->pop3_uidl_format); if (client->uidl_keymask == 0) i_fatal("Invalid pop3_uidl_format"); + if (var_has_key(set->pop3_logout_format, 'u', "uidl_change")) { + /* logging uidl_change. we need hashes of the UIDLs */ + client->message_uidls_save = TRUE; + } else if (strcmp(set->pop3_uidl_duplicates, "allow") != 0) { + /* UIDL duplicates aren't allowed, so we'll need to + keep track of them */ + client->message_uidls_save = TRUE; + } + if (!set->pop3_no_flag_updates && client->messages_count > 0) client->seen_bitmask = i_malloc(MSGS_BITMASK_SIZE(client)); @@ -375,12 +381,8 @@ uint32_t i, old_hash, new_hash; unsigned int old_msg_count, new_msg_count; - if (client->message_uidl_hashes == NULL) { - /* UIDL command not given or %u not actually used in format */ - return ""; - } - if (client->message_uidl_hashes_save) { - /* UIDL command not finished */ + if (client->message_uidls == NULL) { + /* UIDL command not given */ return ""; } @@ -388,18 +390,18 @@ old_msg_count = client->lowest_retr_pop3_msn > 0 ? client->lowest_retr_pop3_msn - 1 : client->messages_count; for (i = 0, old_hash = 0; i < old_msg_count; i++) - old_hash ^= client->message_uidl_hashes[i]; + old_hash ^= crc32_str(client->message_uidls[i]); /* assume all except deleted messages were sent to POP3 client */ if (!client->deleted) { for (i = 0, new_hash = 0; i < client->messages_count; i++) - new_hash ^= client->message_uidl_hashes[i]; + new_hash ^= crc32_str(client->message_uidls[i]); } else { for (i = 0, new_hash = 0; i < client->messages_count; i++) { if (client->deleted_bitmask[i / CHAR_BIT] & (1 << (i % CHAR_BIT))) continue; - new_hash ^= client->message_uidl_hashes[i]; + new_hash ^= crc32_str(client->message_uidls[i]); } } @@ -444,7 +446,11 @@ tab[6].value = dec2str(client->total_size); tab[7].value = dec2str(client->input->v_offset); tab[8].value = dec2str(client->output->offset); - tab[9].value = client_build_uidl_change_string(client); + if (var_has_key(client->set->pop3_logout_format, + tab[9].key, tab[9].long_key)) + tab[9].value = client_build_uidl_change_string(client); + else + tab[9].value = ""; tab[10].value = client->session_id; str = t_str_new(128); @@ -498,8 +504,9 @@ } mail_user_unref(&client->user); + if (client->uidl_pool != NULL) + pool_unref(&client->uidl_pool); i_free(client->message_sizes); - i_free(client->message_uidl_hashes); i_free(client->deleted_bitmask); i_free(client->seen_bitmask); i_free(client->msgnum_to_seq_map); diff -r 4bbc12a87a29 -r 3a095892242b src/pop3/pop3-client.h --- a/src/pop3/pop3-client.h Mon May 14 19:30:03 2012 +0300 +++ b/src/pop3/pop3-client.h Mon May 14 21:07:43 2012 +0300 @@ -55,7 +55,7 @@ unsigned int retr_count; /* [msgnum] */ - uint32_t *message_uidl_hashes; + const char **message_uidls; uoff_t *message_sizes; /* [msgnum/8] & msgnum%8 */ unsigned char *deleted_bitmask; @@ -64,13 +64,14 @@ /* settings: */ const struct pop3_settings *set; const struct mail_storage_settings *mail_set; + pool_t uidl_pool; enum uidl_keys uidl_keymask; unsigned int disconnected:1; unsigned int deleted:1; unsigned int waiting_input:1; unsigned int anvil_sent:1; - unsigned int message_uidl_hashes_save:1; + unsigned int message_uidls_save:1; }; extern struct client *pop3_clients; diff -r 4bbc12a87a29 -r 3a095892242b src/pop3/pop3-commands.c --- a/src/pop3/pop3-commands.c Mon May 14 19:30:03 2012 +0300 +++ b/src/pop3/pop3-commands.c Mon May 14 21:07:43 2012 +0300 @@ -4,8 +4,8 @@ #include "array.h" #include "istream.h" #include "ostream.h" +#include "hash.h" #include "str.h" -#include "crc32.h" #include "var-expand.h" #include "message-size.h" #include "mail-storage.h" @@ -550,62 +550,9 @@ bool list_all; }; -static bool pop3_get_uid(struct client *client, struct cmd_uidl_context *ctx, - struct var_expand_table *tab, string_t *str) -{ - char uid_str[MAX_INT_STRLEN]; - const char *uidl; - - if (mail_get_special(ctx->mail, MAIL_FETCH_UIDL_BACKEND, &uidl) == 0 && - *uidl != '\0') { - str_append(str, uidl); - return TRUE; - } - - if (client->set->pop3_reuse_xuidl && - mail_get_first_header(ctx->mail, "X-UIDL", &uidl) > 0) { - str_append(str, uidl); - return FALSE; - } - - if ((client->uidl_keymask & UIDL_UID) != 0) { - i_snprintf(uid_str, sizeof(uid_str), "%u", - ctx->mail->uid); - tab[1].value = uid_str; - } - if ((client->uidl_keymask & UIDL_MD5) != 0) { - if (mail_get_special(ctx->mail, MAIL_FETCH_HEADER_MD5, - &tab[2].value) < 0 || - *tab[2].value == '\0') { - /* broken */ - i_fatal("UIDL: Header MD5 not found " - "(pop3_uidl_format=%%m not supported by storage?)"); - } - } - if ((client->uidl_keymask & UIDL_FILE_NAME) != 0) { - if (mail_get_special(ctx->mail, - MAIL_FETCH_UIDL_FILE_NAME, - &tab[3].value) < 0 || - *tab[3].value == '\0') { - /* broken */ - i_fatal("UIDL: File name not found " - "(pop3_uidl_format=%%f not supported by storage?)"); - } - } - if ((client->uidl_keymask & UIDL_GUID) != 0) { - if (mail_get_special(ctx->mail, MAIL_FETCH_GUID, - &tab[4].value) < 0 || - *tab[4].value == '\0') { - /* broken */ - i_fatal("UIDL: Message GUID not found " - "(pop3_uidl_format=%%g not supported by storage?)"); - } - } - var_expand(str, client->mail_set->pop3_uidl_format, tab); - return FALSE; -} - -static bool list_uids_iter(struct client *client, struct cmd_uidl_context *ctx) +static void +pop3_get_uid(struct client *client, struct mail *mail, string_t *str, + bool *permanent_uidl_r) { static struct var_expand_table static_tab[] = { { 'v', NULL, "uidvalidity" }, @@ -616,20 +563,108 @@ { '\0', NULL, NULL } }; struct var_expand_table *tab; - string_t *str; - int ret; - unsigned int uidl_pos; - bool save_hashes, found = FALSE; + char uid_str[MAX_INT_STRLEN]; + const char *uidl; + + if (mail_get_special(mail, MAIL_FETCH_UIDL_BACKEND, &uidl) == 0 && + *uidl != '\0') { + str_append(str, uidl); + /* UIDL is already permanent */ + *permanent_uidl_r = TRUE; + return; + } + + *permanent_uidl_r = FALSE; + + if (client->set->pop3_reuse_xuidl && + mail_get_first_header(mail, "X-UIDL", &uidl) > 0) { + str_append(str, uidl); + return; + } tab = t_malloc(sizeof(static_tab)); memcpy(tab, static_tab, sizeof(static_tab)); tab[0].value = t_strdup_printf("%u", client->uid_validity); - save_hashes = client->message_uidl_hashes_save && ctx->list_all; - if (save_hashes && client->message_uidl_hashes == NULL) { - client->message_uidl_hashes = - i_new(uint32_t, client->messages_count); + if ((client->uidl_keymask & UIDL_UID) != 0) { + i_snprintf(uid_str, sizeof(uid_str), "%u", + mail->uid); + tab[1].value = uid_str; } + if ((client->uidl_keymask & UIDL_MD5) != 0) { + if (mail_get_special(mail, MAIL_FETCH_HEADER_MD5, + &tab[2].value) < 0 || + *tab[2].value == '\0') { + /* broken */ + i_fatal("UIDL: Header MD5 not found " + "(pop3_uidl_format=%%m not supported by storage?)"); + } + } + if ((client->uidl_keymask & UIDL_FILE_NAME) != 0) { + if (mail_get_special(mail, MAIL_FETCH_UIDL_FILE_NAME, + &tab[3].value) < 0 || + *tab[3].value == '\0') { + /* broken */ + i_fatal("UIDL: File name not found " + "(pop3_uidl_format=%%f not supported by storage?)"); + } + } + if ((client->uidl_keymask & UIDL_GUID) != 0) { + if (mail_get_special(mail, MAIL_FETCH_GUID, + &tab[4].value) < 0 || + *tab[4].value == '\0') { + /* broken */ + i_fatal("UIDL: Message GUID not found " + "(pop3_uidl_format=%%g not supported by storage?)"); + } + } + var_expand(str, client->mail_set->pop3_uidl_format, tab); +} + +static bool +list_uidls_saved_iter(struct client *client, struct cmd_uidl_context *ctx) +{ + bool found = FALSE; + int ret; + + while (ctx->msgnum < client->messages_count) { + uint32_t msgnum = ctx->msgnum++; + + if (client->deleted) { + if (client->deleted_bitmask[msgnum / CHAR_BIT] & From dovecot at dovecot.org Wed May 16 15:34:18 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 16 May 2012 15:34:18 +0300 Subject: dovecot-2.1: mdbox: mailbox_update() didn't look at the latest i... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/7dd1dd742825 changeset: 14528:7dd1dd742825 user: Timo Sirainen date: Tue May 15 18:52:41 2012 +0300 description: mdbox: mailbox_update() didn't look at the latest index data. So setting min_next_uid, min_first_recent_uid or min_highest_modseq may have actually shrank them. diffstat: src/lib-storage/index/dbox-multi/mdbox-storage.c | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diffs (36 lines): diff -r 3a095892242b -r 7dd1dd742825 src/lib-storage/index/dbox-multi/mdbox-storage.c --- a/src/lib-storage/index/dbox-multi/mdbox-storage.c Mon May 14 21:07:43 2012 +0300 +++ b/src/lib-storage/index/dbox-multi/mdbox-storage.c Tue May 15 18:52:41 2012 +0300 @@ -249,6 +249,7 @@ { struct mdbox_mailbox *mbox = (struct mdbox_mailbox *)box; struct mail_index_transaction *new_trans = NULL; + struct mail_index_view *view; const struct mail_index_header *hdr; uint32_t uid_validity, uid_next; @@ -260,7 +261,8 @@ trans = new_trans; } - hdr = mail_index_get_header(box->view); + view = mail_index_view_open(box->index); + hdr = mail_index_get_header(view); uid_validity = hdr->uid_validity; if (update != NULL && update->uid_validity != 0) uid_validity = update->uid_validity; @@ -293,12 +295,12 @@ &first_recent_uid, sizeof(first_recent_uid), FALSE); } if (update != NULL && update->min_highest_modseq != 0 && - mail_index_modseq_get_highest(box->view) < - update->min_highest_modseq) { + mail_index_modseq_get_highest(view) < update->min_highest_modseq) { mail_index_modseq_enable(box->index); mail_index_update_highest_modseq(trans, update->min_highest_modseq); } + mail_index_view_close(&view); mdbox_update_header(mbox, trans, update); if (new_trans != NULL) { From dovecot at dovecot.org Wed May 16 16:15:45 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 16 May 2012 16:15:45 +0300 Subject: dovecot-2.1: auth: If user is disabled or password expired, tell... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/d6f06ce44b0b changeset: 14529:d6f06ce44b0b user: Timo Sirainen date: Wed May 16 16:14:01 2012 +0300 description: auth: If user is disabled or password expired, tell about it to auth-client. diffstat: src/auth/auth-request-handler.c | 15 +++++++++++++++ src/auth/auth-request.c | 2 ++ src/auth/auth-request.h | 2 ++ 3 files changed, 19 insertions(+), 0 deletions(-) diffs (56 lines): diff -r 7dd1dd742825 -r d6f06ce44b0b src/auth/auth-request-handler.c --- a/src/auth/auth-request-handler.c Tue May 15 18:52:41 2012 +0300 +++ b/src/auth/auth-request-handler.c Wed May 16 16:14:01 2012 +0300 @@ -287,6 +287,21 @@ auth_stream_reply_add(reply, "nodelay", NULL); get_client_extra_fields(request, reply); + switch (request->passdb_result) { + case PASSDB_RESULT_INTERNAL_FAILURE: + case PASSDB_RESULT_SCHEME_NOT_AVAILABLE: + case PASSDB_RESULT_USER_UNKNOWN: + case PASSDB_RESULT_PASSWORD_MISMATCH: + case PASSDB_RESULT_OK: + break; + case PASSDB_RESULT_USER_DISABLED: + auth_stream_reply_add(reply, "user_disabled", NULL); + break; + case PASSDB_RESULT_PASS_EXPIRED: + auth_stream_reply_add(reply, "pass_expired", NULL); + break; + } + auth_request_handle_failure(request, reply); } diff -r 7dd1dd742825 -r d6f06ce44b0b src/auth/auth-request.c --- a/src/auth/auth-request.c Tue May 15 18:52:41 2012 +0300 +++ b/src/auth/auth-request.c Wed May 16 16:14:01 2012 +0300 @@ -557,6 +557,7 @@ request->private_callback.verify_plain); } else { auth_request_ref(request); + request->passdb_result = result; request->private_callback.verify_plain(result, request); auth_request_unref(&request); } @@ -691,6 +692,7 @@ but the user was unknown there */ result = PASSDB_RESULT_USER_UNKNOWN; } + request->passdb_result = result; request->private_callback. lookup_credentials(result, credentials, size, request); } diff -r 7dd1dd742825 -r d6f06ce44b0b src/auth/auth-request.h --- a/src/auth/auth-request.h Tue May 15 18:52:41 2012 +0300 +++ b/src/auth/auth-request.h Wed May 16 16:14:01 2012 +0300 @@ -56,6 +56,8 @@ struct auth_stream_reply *extra_cache_fields; /* the whole userdb result reply */ struct auth_stream_reply *userdb_reply; + /* Result of passdb lookup */ + enum passdb_result passdb_result; const struct mech_module *mech; const struct auth_settings *set; From dovecot at dovecot.org Wed May 16 16:15:45 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 16 May 2012 16:15:45 +0300 Subject: dovecot-2.1: login: If user is disabled or password expired, say... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/ca0fef559d43 changeset: 14530:ca0fef559d43 user: Timo Sirainen date: Wed May 16 16:14:31 2012 +0300 description: login: If user is disabled or password expired, say it in logout reason. diffstat: src/login-common/client-common-auth.c | 4 ++++ src/login-common/client-common.c | 4 ++++ src/login-common/client-common.h | 2 ++ 3 files changed, 10 insertions(+), 0 deletions(-) diffs (40 lines): diff -r d6f06ce44b0b -r ca0fef559d43 src/login-common/client-common-auth.c --- a/src/login-common/client-common-auth.c Wed May 16 16:14:01 2012 +0300 +++ b/src/login-common/client-common-auth.c Wed May 16 16:14:31 2012 +0300 @@ -82,6 +82,10 @@ reply_r->temp = TRUE; else if (strcmp(key, "authz") == 0) reply_r->authz_failure = TRUE; + else if (strcmp(key, "user_disabled") == 0) + client->auth_user_disabled = TRUE; + else if (strcmp(key, "pass_expired") == 0) + client->auth_pass_expired = TRUE; else if (strcmp(key, "reason") == 0) reply_r->reason = value; else if (strcmp(key, "host") == 0) diff -r d6f06ce44b0b -r ca0fef559d43 src/login-common/client-common.c --- a/src/login-common/client-common.c Wed May 16 16:14:01 2012 +0300 +++ b/src/login-common/client-common.c Wed May 16 16:14:31 2012 +0300 @@ -645,6 +645,10 @@ return t_strdup_printf("(internal failure, %u succesful auths)", client->auth_successes); } + if (client->auth_user_disabled) + return "(user disabled)"; + if (client->auth_pass_expired) + return "(password expired)"; return t_strdup_printf("(auth failed, %u attempts in %u secs)", client->auth_attempts, auth_secs); } diff -r d6f06ce44b0b -r ca0fef559d43 src/login-common/client-common.h --- a/src/login-common/client-common.h Wed May 16 16:14:01 2012 +0300 +++ b/src/login-common/client-common.h Wed May 16 16:14:31 2012 +0300 @@ -139,6 +139,8 @@ unsigned int auth_process_comm_fail:1; unsigned int proxy_auth_failed:1; unsigned int auth_waiting:1; + unsigned int auth_user_disabled:1; + unsigned int auth_pass_expired:1; /* ... */ }; From dovecot at dovecot.org Wed May 16 18:57:28 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 16 May 2012 18:57:28 +0300 Subject: dovecot-2.1: sdbox: Avoid using too many fds when copying messages. Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/6d11513cdd0d changeset: 14531:6d11513cdd0d user: Timo Sirainen date: Wed May 16 18:57:05 2012 +0300 description: sdbox: Avoid using too many fds when copying messages. This happened at least with quota plugin. diffstat: src/lib-storage/index/dbox-single/sdbox-save.c | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-) diffs (22 lines): diff -r ca0fef559d43 -r 6d11513cdd0d src/lib-storage/index/dbox-single/sdbox-save.c --- a/src/lib-storage/index/dbox-single/sdbox-save.c Wed May 16 16:14:31 2012 +0300 +++ b/src/lib-storage/index/dbox-single/sdbox-save.c Wed May 16 18:57:05 2012 +0300 @@ -78,9 +78,18 @@ void sdbox_save_add_file(struct mail_save_context *_ctx, struct dbox_file *file) { struct sdbox_save_context *ctx = (struct sdbox_save_context *)_ctx; + struct dbox_file *const *files; + unsigned int count; if (ctx->first_saved_seq == 0) ctx->first_saved_seq = ctx->ctx.seq; + + files = array_get(&ctx->files, &count); + if (count > 0) { + /* a plugin may leave a previously saved file open. + we'll close it here to avoid eating too many fds. */ + dbox_file_close(files[count-1]); + } array_append(&ctx->files, &file, 1); } From dovecot at dovecot.org Wed May 16 18:57:38 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 16 May 2012 18:57:38 +0300 Subject: dovecot-2.0: sdbox: Avoid using too many fds when copying messages. Message-ID: details: http://hg.dovecot.org/dovecot-2.0/rev/3fff8093ab5b changeset: 13093:3fff8093ab5b user: Timo Sirainen date: Wed May 16 18:57:05 2012 +0300 description: sdbox: Avoid using too many fds when copying messages. This happened at least with quota plugin. diffstat: src/lib-storage/index/dbox-single/sdbox-save.c | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-) diffs (22 lines): diff -r 5830e7292909 -r 3fff8093ab5b src/lib-storage/index/dbox-single/sdbox-save.c --- a/src/lib-storage/index/dbox-single/sdbox-save.c Wed Apr 25 02:50:47 2012 +0300 +++ b/src/lib-storage/index/dbox-single/sdbox-save.c Wed May 16 18:57:05 2012 +0300 @@ -78,9 +78,18 @@ void sdbox_save_add_file(struct mail_save_context *_ctx, struct dbox_file *file) { struct sdbox_save_context *ctx = (struct sdbox_save_context *)_ctx; + struct dbox_file *const *files; + unsigned int count; if (ctx->first_saved_seq == 0) ctx->first_saved_seq = ctx->ctx.seq; + + files = array_get(&ctx->files, &count); + if (count > 0) { + /* a plugin may leave a previously saved file open. + we'll close it here to avoid eating too many fds. */ + dbox_file_close(files[count-1]); + } array_append(&ctx->files, &file, 1); } From dovecot at dovecot.org Thu May 17 18:09:04 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Thu, 17 May 2012 18:09:04 +0300 Subject: dovecot-2.1: update-version.sh: Use mv -f to avoid questions in ... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/bb572accd1af changeset: 14532:bb572accd1af user: Timo Sirainen date: Thu May 17 18:08:58 2012 +0300 description: update-version.sh: Use mv -f to avoid questions in some systems. diffstat: update-version.sh | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (9 lines): diff -r 6d11513cdd0d -r bb572accd1af update-version.sh --- a/update-version.sh Wed May 16 18:57:05 2012 +0300 +++ b/update-version.sh Thu May 17 18:08:58 2012 +0300 @@ -64,4 +64,4 @@ cmp -s "${BUILDDIR}/${VERSION_H}" "${BUILDDIR}/${VERSION_HT}" && \ rm -f "${BUILDDIR}/${VERSION_HT}" || \ - mv "${BUILDDIR}/${VERSION_HT}" "${BUILDDIR}/${VERSION_H}" + mv -f "${BUILDDIR}/${VERSION_HT}" "${BUILDDIR}/${VERSION_H}" From dovecot at dovecot.org Sat May 19 18:43:18 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sat, 19 May 2012 18:43:18 +0300 Subject: dovecot-2.1: lib-storage: If chroot is used and mail_debug=yes, ... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/a7dd1edee318 changeset: 14533:a7dd1edee318 user: Timo Sirainen date: Sat May 19 18:43:13 2012 +0300 description: lib-storage: If chroot is used and mail_debug=yes, log the chroot path. diffstat: src/lib-storage/mail-storage-service.c | 25 +++++++++++++++++++++++-- 1 files changed, 23 insertions(+), 2 deletions(-) diffs (49 lines): diff -r bb572accd1af -r a7dd1edee318 src/lib-storage/mail-storage-service.c --- a/src/lib-storage/mail-storage-service.c Thu May 17 18:08:58 2012 +0300 +++ b/src/lib-storage/mail-storage-service.c Sat May 19 18:43:13 2012 +0300 @@ -604,8 +604,13 @@ mail_set = mail_user_set_get_storage_set(mail_user); if (mail_set->mail_debug) { - i_debug("Effective uid=%s, gid=%s, home=%s", - dec2str(geteuid()), dec2str(getegid()), home); + string_t *str = t_str_new(64); + + str_printfa(str, "Effective uid=%s, gid=%s, home=%s", + dec2str(geteuid()), dec2str(getegid()), home); + if (*priv->chroot != '\0') + str_printfa(str, ", chroot=%s", priv->chroot); + i_debug("%s", str_c(str)); } if ((user->flags & MAIL_STORAGE_SERVICE_FLAG_TEMP_PRIV_DROP) != 0 && @@ -909,6 +914,21 @@ user_set->mail_plugins, &mod_set); } +static int extra_field_key_cmp_p(const char *const *s1, const char *const *s2) +{ + const char *p1 = *s1, *p2 = *s2; + + for (; *p1 == *p2; p1++, p2++) { + if (*p1 == '\0') + return 0; + } + if (*p1 == '=') + return -1; + if (*p2 == '=') + return 1; + return *p1 - *p2; +} + int mail_storage_service_lookup(struct mail_storage_service_ctx *ctx, const struct mail_storage_service_input *input, struct mail_storage_service_user **user_r, @@ -994,6 +1014,7 @@ if (userdb_fields != NULL) { auth_user_fields_parse(userdb_fields, temp_pool, &reply); + array_sort(&reply.extra_fields, extra_field_key_cmp_p); if (user_reply_handle(ctx, user, &reply, &error) < 0) { i_error("user %s: Invalid settings in userdb: %s", username, error); From dovecot at dovecot.org Sat May 19 21:18:29 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sat, 19 May 2012 21:18:29 +0300 Subject: dovecot-2.1: director: Don't crash with quickly disconnecting in... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/40f958c7643b changeset: 14534:40f958c7643b user: Timo Sirainen date: Sat May 19 21:16:42 2012 +0300 description: director: Don't crash with quickly disconnecting incoming director connections. diffstat: src/director/director-connection.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (13 lines): diff -r a7dd1edee318 -r 40f958c7643b src/director/director-connection.c --- a/src/director/director-connection.c Sat May 19 18:43:13 2012 +0300 +++ b/src/director/director-connection.c Sat May 19 21:16:42 2012 +0300 @@ -1503,7 +1503,8 @@ struct director_connection *conn = *_conn; struct director *dir = conn->dir; - if (conn->created + DIRECTOR_SUCCESS_MIN_CONNECT_SECS > ioloop_time) { + if (conn->created + DIRECTOR_SUCCESS_MIN_CONNECT_SECS > ioloop_time && + conn->host != NULL) { /* connection didn't exist for very long, assume it has a network problem */ conn->host->last_network_failure = ioloop_time; From dovecot at dovecot.org Sat May 19 21:18:29 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sat, 19 May 2012 21:18:29 +0300 Subject: dovecot-2.1: director: Implemented ability to remove directors f... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/42cca8a1d179 changeset: 14535:42cca8a1d179 user: Timo Sirainen date: Sat May 19 21:18:04 2012 +0300 description: director: Implemented ability to remove directors from a running ring. Also added doveadm command for adding a new director to a running ring. diffstat: src/director/director-connection.c | 44 ++++++++++++-- src/director/director-host.c | 35 +++++++++++- src/director/director-host.h | 9 ++- src/director/director.c | 110 ++++++++++++++++++++++++++++++++++-- src/director/director.h | 15 ++++- src/director/doveadm-connection.c | 60 +++++++++++++++++++- src/director/main.c | 5 +- src/doveadm/doveadm-director.c | 67 ++++++++++++++++++++++ 8 files changed, 326 insertions(+), 19 deletions(-) diffs (truncated from 614 to 300 lines): diff -r 40f958c7643b -r 42cca8a1d179 src/director/director-connection.c --- a/src/director/director-connection.c Sat May 19 21:16:42 2012 +0300 +++ b/src/director/director-connection.c Sat May 19 21:18:04 2012 +0300 @@ -383,7 +383,12 @@ elsewhere with CONNECT. however, before disconnecting it verify first that our left side is actually still functional. */ + i_assert(conn->host == NULL); conn->host = director_host_get(dir, &ip, port); + /* the host shouldn't be removed at this point, but if for some + reason it is we don't want to crash */ + conn->host->removed = FALSE; + director_host_ref(conn->host); /* make sure we don't keep old sequence values across restarts */ conn->host->last_seq = 0; @@ -587,6 +592,10 @@ /* ignore updates to ourself */ return TRUE; } + if (host->removed) { + /* ignore re-adds of removed directors */ + return TRUE; + } /* already have this. just reset its last_network_failure timestamp, since it might be up now. */ @@ -598,18 +607,32 @@ } } else { /* save the director and forward it */ - director_host_add(conn->dir, &ip, port); + host = director_host_add(conn->dir, &ip, port); forward = TRUE; } if (forward) { - director_update_send(conn->dir, - director_connection_get_host(conn), - t_strdup_printf("DIRECTOR\t%s\t%u\n", - net_ip2addr(&ip), port)); + director_notify_ring_added(host, + director_connection_get_host(conn)); } return TRUE; } +static bool director_cmd_director_remove(struct director_connection *conn, + const char *const *args) +{ + struct director_host *host; + struct ip_addr ip; + unsigned int port; + + if (!director_args_parse_ip_port(conn, args, &ip, &port)) + return FALSE; + + host = director_host_lookup(conn->dir, &ip, port); + if (host != NULL && !host->removed) + director_ring_remove(host, director_connection_get_host(conn)); + return TRUE; +} + static bool director_cmd_host_hand_start(struct director_connection *conn, const char *const *args) @@ -659,7 +682,7 @@ *_args = args + 3; host = director_host_lookup(conn->dir, &ip, port); - if (host == NULL) { + if (host == NULL || host->removed) { /* director is already gone, but we can't be sure if this command was sent everywhere. re-send it as if it was from ourself. */ @@ -1191,6 +1214,8 @@ return director_cmd_user_killed_everywhere(conn, args); if (strcmp(cmd, "DIRECTOR") == 0) return director_cmd_director(conn, args); + if (strcmp(cmd, "DIRECTOR-REMOVE") == 0) + return director_cmd_director_remove(conn, args); if (strcmp(cmd, "SYNC") == 0) return director_connection_sync(conn, args); if (strcmp(cmd, "CONNECT") == 0) @@ -1279,6 +1304,8 @@ struct director_host *const *hostp; array_foreach(&conn->dir->dir_hosts, hostp) { + if ((*hostp)->removed) + continue; str_printfa(str, "DIRECTOR\t%s\t%u\n", net_ip2addr(&(*hostp)->ip), (*hostp)->port); } @@ -1433,12 +1460,15 @@ { struct director_connection *conn; + i_assert(!host->removed); + /* make sure we don't keep old sequence values across restarts */ host->last_seq = 0; conn = director_connection_init_common(dir, fd); conn->name = i_strdup_printf("%s/out", host->name); conn->host = host; + director_host_ref(host); conn->io = io_add(conn->fd, IO_WRITE, director_connection_connected, conn); return conn; @@ -1471,6 +1501,8 @@ } if (dir->right == conn) dir->right = NULL; + if (conn->host != NULL) + director_host_unref(conn->host); if (conn->user_iter != NULL) user_directory_iter_deinit(&conn->user_iter); diff -r 40f958c7643b -r 42cca8a1d179 src/director/director-host.c --- a/src/director/director-host.c Sat May 19 21:16:42 2012 +0300 +++ b/src/director/director-host.c Sat May 19 21:18:04 2012 +0300 @@ -29,6 +29,8 @@ struct director_host *host; host = i_new(struct director_host, 1); + host->dir = dir; + host->refcount = 1; host->ip = *ip; host->port = port; host->name = i_strdup_printf("%s:%u", net_ip2addr(ip), port); @@ -41,8 +43,39 @@ return host; } -void director_host_free(struct director_host *host) +void director_host_free(struct director_host **_host) { + struct director_host *host = *_host; + + i_assert(host->refcount == 1); + + *_host = NULL; + director_host_unref(host); +} + +void director_host_ref(struct director_host *host) +{ + i_assert(host->refcount > 0); + host->refcount++; +} + +void director_host_unref(struct director_host *host) +{ + struct director_host *const *hosts; + unsigned int i, count; + + i_assert(host->refcount > 0); + + if (--host->refcount > 0) + return; + + hosts = array_get(&host->dir->dir_hosts, &count); + for (i = 0; i < count; i++) { + if (hosts[i] == host) { + array_delete(&host->dir->dir_hosts, i, 1); + break; + } + } i_free(host->name); i_free(host); } diff -r 40f958c7643b -r 42cca8a1d179 src/director/director-host.h --- a/src/director/director-host.h Sat May 19 21:16:42 2012 +0300 +++ b/src/director/director-host.h Sat May 19 21:18:04 2012 +0300 @@ -6,6 +6,9 @@ struct director; struct director_host { + struct director *dir; + int refcount; + struct ip_addr ip; unsigned int port; @@ -22,12 +25,16 @@ time_t last_protocol_failure; /* we are this director */ unsigned int self:1; + unsigned int removed:1; }; struct director_host * director_host_add(struct director *dir, const struct ip_addr *ip, unsigned int port); -void director_host_free(struct director_host *host); +void director_host_free(struct director_host **host); + +void director_host_ref(struct director_host *host); +void director_host_unref(struct director_host *host); struct director_host * director_host_get(struct director *dir, const struct ip_addr *ip, diff -r 40f958c7643b -r 42cca8a1d179 src/director/director.c --- a/src/director/director.c Sat May 19 21:16:42 2012 +0300 +++ b/src/director/director.c Sat May 19 21:18:04 2012 +0300 @@ -20,6 +20,7 @@ #define DIRECTOR_SYNC_TIMEOUT_MSECS (5*1000) #define DIRECTOR_RING_MIN_WAIT_SECS 20 #define DIRECTOR_QUICK_RECONNECT_TIMEOUT_MSECS 1000 +#define DIRECTOR_DELAYED_DIR_REMOVE_MSECS (1000*30) static bool director_is_self_ip_set(struct director *dir) { @@ -127,8 +128,8 @@ static struct director_host * director_get_preferred_right_host(struct director *dir) { - struct director_host *const *hosts; - unsigned int count, self_idx; + struct director_host *const *hosts, *host; + unsigned int i, count, self_idx; hosts = array_get(&dir->dir_hosts, &count); if (count == 1) { @@ -137,7 +138,13 @@ } self_idx = director_find_self_idx(dir); - return hosts[(self_idx + 1) % count]; + for (i = 0; i < count; i++) { + host = hosts[(self_idx + i + 1) % count]; + if (!host->removed) + return host; + } + /* self, with some removed hosts */ + return NULL; } static bool director_wait_for_others(struct director *dir) @@ -177,6 +184,9 @@ for (i = 1; i < count; i++) { unsigned int idx = (self_idx + i) % count; + if (hosts[idx]->removed) + continue; + if (hosts[idx]->last_network_failure + DIRECTOR_RECONNECT_RETRY_SECS > ioloop_time) { /* connection failed recently, don't try retrying here */ @@ -408,6 +418,79 @@ director_connection_uncork(*connp); } +void director_notify_ring_added(struct director_host *added_host, + struct director_host *src) +{ + const char *cmd; + + cmd = t_strdup_printf("DIRECTOR\t%s\t%u\n", + net_ip2addr(&added_host->ip), added_host->port); + director_update_send(added_host->dir, src, cmd); +} + +static void director_delayed_dir_remove_timeout(struct director *dir) +{ + struct director_host *const *hosts, *host; + unsigned int i, count; + + timeout_remove(&dir->to_remove_dirs); + + hosts = array_get(&dir->dir_hosts, &count); + for (i = 0; i < count; ) { + if (hosts[i]->removed) { + host = hosts[i]; + director_host_free(&host); + hosts = array_get(&dir->dir_hosts, &count); + } else { + i++; + } + } +} + +void director_ring_remove(struct director_host *removed_host, + struct director_host *src) +{ + struct director *dir = removed_host->dir; + struct director_connection *const *conns, *conn; + unsigned int i, count; + const char *cmd; + + if (removed_host->self) { + /* others will just disconnect us */ + return; + } + + /* mark the host as removed and fully remove it later. this delay is + needed, because the removal may trigger director reconnections, + which may send the director back and we don't want to re-add it */ + removed_host->removed = TRUE; + if (dir->to_remove_dirs == NULL) { + dir->to_remove_dirs = From dovecot at dovecot.org Sat May 19 22:11:58 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sat, 19 May 2012 22:11:58 +0300 Subject: dovecot-2.1: expire: Handle dict errors. Don't crash if lookup f... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/8f72002cb394 changeset: 14536:8f72002cb394 user: Timo Sirainen date: Sat May 19 22:11:41 2012 +0300 description: expire: Handle dict errors. Don't crash if lookup fails. Based on patch by Michal Grzedzicki diffstat: src/plugins/expire/expire-plugin.c | 13 ++++++++++--- 1 files changed, 10 insertions(+), 3 deletions(-) diffs (30 lines): diff -r 42cca8a1d179 -r 8f72002cb394 src/plugins/expire/expire-plugin.c --- a/src/plugins/expire/expire-plugin.c Sat May 19 21:18:04 2012 +0300 +++ b/src/plugins/expire/expire-plugin.c Sat May 19 22:11:41 2012 +0300 @@ -162,8 +162,14 @@ this is the first mail in the database */ ret = dict_lookup(euser->db, pool_datastack_create(), key, &value); - if (ret == 0) { - /* first time saving here with expire enabled */ + if (ret <= 0) { + /* first time saving here with expire enabled. + also handle lookup errors by just assuming + it didn't exist */ + if (ret < 0) { + i_warning("expire: dict lookup failed, " + "assuming update is needed"); + } first_save_timestamp(box, &new_stamp); update_dict = TRUE; } else if (strcmp(value, "0") == 0) { @@ -188,7 +194,8 @@ dctx = dict_transaction_begin(euser->db); dict_set(dctx, key, dec2str(new_stamp)); - dict_transaction_commit(&dctx); + if (dict_transaction_commit(&dctx) < 0) + i_error("expire: dict commit failed"); } } T_END; i_free(xt); From dovecot at dovecot.org Sat May 19 22:25:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sat, 19 May 2012 22:25:35 +0300 Subject: dovecot-2.1: auth: Fixed crash with DIGEST-MD5 when attempting t... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/23543c407e81 changeset: 14537:23543c407e81 user: Timo Sirainen date: Sat May 19 22:25:27 2012 +0300 description: auth: Fixed crash with DIGEST-MD5 when attempting to do master user login without master passdbs. diffstat: src/auth/auth-request.c | 27 ++++++++++++++++++++------- 1 files changed, 20 insertions(+), 7 deletions(-) diffs (51 lines): diff -r 8f72002cb394 -r 23543c407e81 src/auth/auth-request.c --- a/src/auth/auth-request.c Sat May 19 22:11:41 2012 +0300 +++ b/src/auth/auth-request.c Sat May 19 22:25:27 2012 +0300 @@ -609,6 +609,20 @@ return FALSE; } +static bool auth_request_is_disabled_master_user(struct auth_request *request) +{ + if (request->passdb != NULL) + return FALSE; + + /* no masterdbs, master logins not supported */ + i_assert(request->requested_login_user != NULL); + auth_request_log_info(request, "passdb", + "Attempted master login with no master passdbs " + "(trying to log in as user: %s)", + request->requested_login_user); + return TRUE; +} + void auth_request_verify_plain(struct auth_request *request, const char *password, verify_plain_callback_t *callback) @@ -619,13 +633,7 @@ i_assert(request->state == AUTH_REQUEST_STATE_MECH_CONTINUE); - if (request->passdb == NULL) { - /* no masterdbs, master logins not supported */ - i_assert(request->requested_login_user != NULL); - auth_request_log_info(request, "passdb", - "Attempted master login with no master passdbs " - "(trying to log in as user: %s)", - request->requested_login_user); + if (auth_request_is_disabled_master_user(request)) { callback(PASSDB_RESULT_USER_UNKNOWN, request); return; } @@ -746,6 +754,11 @@ i_assert(request->state == AUTH_REQUEST_STATE_MECH_CONTINUE); + if (auth_request_is_disabled_master_user(request)) { + callback(PASSDB_RESULT_USER_UNKNOWN, NULL, 0, request); + return; + } + request->credentials_scheme = p_strdup(request->pool, scheme); request->private_callback.lookup_credentials = callback; From dovecot at dovecot.org Sat May 19 22:28:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sat, 19 May 2012 22:28:25 +0300 Subject: dovecot-2.2: lib-storage: Sort userdb fields before importing them. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/875eeb3052a9 changeset: 14232:875eeb3052a9 user: Timo Sirainen date: Sat May 19 22:28:19 2012 +0300 description: lib-storage: Sort userdb fields before importing them. This avoids requiring manual sorting in some situations, such as when defining new namespaces. diffstat: src/lib-storage/mail-storage-service.c | 16 ++++++++++++++++ 1 files changed, 16 insertions(+), 0 deletions(-) diffs (33 lines): diff -r 10a475d725a1 -r 875eeb3052a9 src/lib-storage/mail-storage-service.c --- a/src/lib-storage/mail-storage-service.c Fri May 04 00:48:45 2012 +0300 +++ b/src/lib-storage/mail-storage-service.c Sat May 19 22:28:19 2012 +0300 @@ -907,6 +907,21 @@ user_set->mail_plugins, &mod_set); } +static int extra_field_key_cmp_p(const char *const *s1, const char *const *s2) +{ + const char *p1 = *s1, *p2 = *s2; + + for (; *p1 == *p2; p1++, p2++) { + if (*p1 == '\0') + return 0; + } + if (*p1 == '=') + return -1; + if (*p2 == '=') + return 1; + return *p1 - *p2; +} + int mail_storage_service_lookup(struct mail_storage_service_ctx *ctx, const struct mail_storage_service_input *input, struct mail_storage_service_user **user_r, @@ -992,6 +1007,7 @@ if (userdb_fields != NULL) { auth_user_fields_parse(userdb_fields, temp_pool, &reply); + array_sort(&reply.extra_fields, extra_field_key_cmp_p); if (user_reply_handle(ctx, user, &reply, &error) < 0) { i_error("user %s: Invalid settings in userdb: %s", username, error); From dovecot at dovecot.org Sat May 19 22:40:19 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sat, 19 May 2012 22:40:19 +0300 Subject: dovecot-2.2: Extends struct net_unix_cred with pid field and mod... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/80688ab1ea3d changeset: 14233:80688ab1ea3d user: Timo Sirainen date: Sat May 19 22:40:08 2012 +0300 description: Extends struct net_unix_cred with pid field and modifies net_getunixcred() to fill it in if possible. Depends: none diffstat: configure.in | 4 ++- src/lib/network.c | 61 ++++++++++++++++++++++++++++++++++++++++++++++-------- src/lib/network.h | 4 ++- 3 files changed, 58 insertions(+), 11 deletions(-) diffs (137 lines): diff -r 875eeb3052a9 -r 80688ab1ea3d configure.in --- a/configure.in Sat May 19 22:28:19 2012 +0300 +++ b/configure.in Sat May 19 22:40:08 2012 +0300 @@ -292,7 +292,7 @@ sys/quota.h sys/fs/ufs_quota.h ufs/ufs/quota.h jfs/quota.h sys/fs/quota_common.h \ mntent.h sys/mnttab.h sys/event.h sys/time.h sys/mkdev.h linux/dqblk_xfs.h \ xfs/xqm.h execinfo.h ucontext.h malloc_np.h sys/utsname.h sys/vmount.h \ - sys/utsname.h glob.h linux/falloc.h ucred.h) + sys/utsname.h glob.h linux/falloc.h ucred.h sys/ucred.h) dnl * clang check have_clang=no @@ -408,6 +408,8 @@ walkcontext dirfd clearenv malloc_usable_size glob fallocate \ posix_fadvise getpeereid getpeerucred) +AC_CHECK_TYPES([struct sockpeercred]) + AC_CHECK_LIB(rt, clock_gettime, [ AC_DEFINE(HAVE_CLOCK_GETTIME,, Define if you have the clock_gettime function) LIBS="$LIBS -lrt" diff -r 875eeb3052a9 -r 80688ab1ea3d src/lib/network.c --- a/src/lib/network.c Sat May 19 22:28:19 2012 +0300 +++ b/src/lib/network.c Sat May 19 22:40:08 2012 +0300 @@ -13,8 +13,10 @@ #include #include #include -#ifdef HAVE_UCRED_H +#if defined(HAVE_UCRED_H) # include /* for getpeerucred() */ +#elif defined(HAVE_SYS_UCRED_H) +# include /* for FreeBSD struct xucred */ #endif union sockaddr_union { @@ -694,16 +696,14 @@ int net_getunixcred(int fd, struct net_unix_cred *cred_r) { -#if defined(HAVE_GETPEEREID) - /* OSX 10.4+, FreeBSD 4.6+, OpenBSD 3.0+, NetBSD 5.0+ */ - if (getpeereid(fd, &cred_r->uid, &cred_r->gid) < 0) { - i_error("getpeereid() failed: %m"); - return -1; - } - return 0; -#elif defined(SO_PEERCRED) +#if defined(SO_PEERCRED) +# if defined(HAVE_STRUCT_SOCKPEERCRED) + /* OpenBSD (may also provide getpeereid, but we also want pid) */ + struct sockpeercred ucred; +# else /* Linux */ struct ucred ucred; +# endif socklen_t len = sizeof(ucred); if (getsockopt(fd, SOL_SOCKET, SO_PEERCRED, &ucred, &len) < 0) { @@ -712,6 +712,48 @@ } cred_r->uid = ucred.uid; cred_r->gid = ucred.gid; + cred_r->pid = ucred.pid; + return 0; +#elif defined(LOCAL_PEEREID) + /* NetBSD (may also provide getpeereid, but we also want pid) */ + struct unpcbid ucred; + socklen_t len = sizeof(ucred); + + if (getsockopt(s, 0, LOCAL_PEEREID, &ucred, &len) < 0) { + i_error("getsockopt(LOCAL_PEEREID) failed: %m"); + return -1; + } + + cred_r->uid = ucred.unp_euid; + cred_r->gid = ucred.unp_egid; + cred_r->pid = ucred.unp_pid; + return 0; +#elif defined(HAVE_GETPEEREID) + /* OSX 10.4+, FreeBSD 4.6+, OpenBSD 3.0+, NetBSD 5.0+ */ + if (getpeereid(fd, &cred_r->uid, &cred_r->gid) < 0) { + i_error("getpeereid() failed: %m"); + return -1; + } + cred_r->pid = (pid_t)-1; + return 0; +#elif defined(LOCAL_PEERCRED) + /* Older FreeBSD */ + struct xucred ucred; + socklen_t len = sizeof(ucred); + + if (getsockopt(fd, 0, LOCAL_PEERCRED, &ucred, &len) < 0) { + i_error("getsockopt(LOCAL_PEERCRED) failed: %m"); + return -1; + } + + if (ucred.cr_version != XUCRED_VERSION) { + errno = EINVAL; + return -1; + } + + cred_r->uid = ucred.cr_uid; + cred_r->gid = ucred.cr_gid; + cred_r->pid = (pid_t)-1; return 0; #elif defined(HAVE_GETPEERUCRED) /* Solaris */ @@ -723,6 +765,7 @@ } cred_r->uid = ucred_geteuid(ucred); cred_r->gid = ucred_getrgid(ucred); + cred_r->pid = ucred_getpid(ucred); ucred_free(ucred); if (cred_r->uid == (uid_t)-1 || diff -r 875eeb3052a9 -r 80688ab1ea3d src/lib/network.h --- a/src/lib/network.h Sat May 19 22:28:19 2012 +0300 +++ b/src/lib/network.h Sat May 19 22:40:08 2012 +0300 @@ -34,6 +34,7 @@ struct net_unix_cred { uid_t uid; gid_t gid; + pid_t pid; }; /* maxmimum string length of IP address */ @@ -115,7 +116,8 @@ int net_getpeername(int fd, struct ip_addr *addr, unsigned int *port); /* Get UNIX socket name. */ int net_getunixname(int fd, const char **name_r); -/* Get UNIX socket peer process's credentials. */ +/* Get UNIX socket peer process's credentials. The pid may be (pid_t)-1 if + unavailable. */ int net_getunixcred(int fd, struct net_unix_cred *cred_r); /* Returns ip_addr as string, or NULL if ip is invalid. */ From dovecot at dovecot.org Sun May 20 02:41:18 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 02:41:18 +0300 Subject: dovecot-2.1: lib-master: Anvil reconnect timeout should be 5 sec... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/1c2c1d75d3b2 changeset: 14538:1c2c1d75d3b2 user: Timo Sirainen date: Sun May 20 02:41:08 2012 +0300 description: lib-master: Anvil reconnect timeout should be 5 seconds, not 5 milliseconds. diffstat: src/lib-master/anvil-client.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 23543c407e81 -r 1c2c1d75d3b2 src/lib-master/anvil-client.c --- a/src/lib-master/anvil-client.c Sat May 19 22:25:27 2012 +0300 +++ b/src/lib-master/anvil-client.c Sun May 20 02:41:08 2012 +0300 @@ -80,7 +80,7 @@ if (ioloop_time - client->last_reconnect < ANVIL_RECONNECT_MIN_SECS) { if (client->to_reconnect == NULL) { client->to_reconnect = - timeout_add(ANVIL_RECONNECT_MIN_SECS, + timeout_add(ANVIL_RECONNECT_MIN_SECS*1000, anvil_reconnect, client); } } else { From dovecot at dovecot.org Sun May 20 03:08:08 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:08:08 +0300 Subject: dovecot-2.1: director: When we find unwanted connection, wait fo... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/dbe6d05fd595 changeset: 14539:dbe6d05fd595 user: Timo Sirainen date: Sun May 20 03:08:01 2012 +0300 description: director: When we find unwanted connection, wait for 10s for it to disconnect us, not 10ms. diffstat: src/director/director-connection.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (21 lines): diff -r 1c2c1d75d3b2 -r dbe6d05fd595 src/director/director-connection.c --- a/src/director/director-connection.c Sun May 20 02:41:08 2012 +0300 +++ b/src/director/director-connection.c Sun May 20 03:08:01 2012 +0300 @@ -70,7 +70,7 @@ /* If outgoing director connection exists for less than this many seconds, mark the host as failed so we won't try to reconnect to it immediately */ #define DIRECTOR_SUCCESS_MIN_CONNECT_SECS 40 -#define DIRECTOR_WAIT_DISCONNECT_MSECS 10 +#define DIRECTOR_WAIT_DISCONNECT_SECS 10 #if DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS <= DIRECTOR_CONNECTION_PING_TIMEOUT_MSECS # error DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS is too low @@ -187,7 +187,7 @@ o_stream_uncork(conn->output); conn->to_disconnect = - timeout_add(DIRECTOR_WAIT_DISCONNECT_MSECS, + timeout_add(DIRECTOR_WAIT_DISCONNECT_SECS*1000, director_connection_wait_timeout, conn); } From dovecot at dovecot.org Sun May 20 03:26:23 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:23 +0300 Subject: dovecot-2.2: lib-ssl-iostream: Use SSL_LIBS when linking the sha... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/c07415305d9e changeset: 14234:c07415305d9e user: Timo Sirainen date: Mon Feb 27 13:34:20 2012 +0200 description: lib-ssl-iostream: Use SSL_LIBS when linking the shared library. diffstat: src/lib-ssl-iostream/Makefile.am | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (11 lines): diff -r 32ffa616f249 -r c07415305d9e src/lib-ssl-iostream/Makefile.am --- a/src/lib-ssl-iostream/Makefile.am Sat Feb 25 07:29:15 2012 +0200 +++ b/src/lib-ssl-iostream/Makefile.am Mon Feb 27 13:34:20 2012 +0200 @@ -30,6 +30,6 @@ pkglib_LTLIBRARIES = libdovecot-ssl.la libdovecot_ssl_la_SOURCES = -libdovecot_ssl_la_LIBADD = libssl_iostream.la ../lib/liblib.la $(MODULE_LIBS) +libdovecot_ssl_la_LIBADD = libssl_iostream.la ../lib/liblib.la $(MODULE_LIBS) $(SSL_LIBS) libdovecot_ssl_la_DEPENDENCIES = libssl_iostream.la libdovecot_ssl_la_LDFLAGS = -export-dynamic From dovecot at dovecot.org Sun May 20 03:26:23 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:23 +0300 Subject: dovecot-2.2: doveadm: Allow subcommands to specify '+' getopt pa... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/0d502dc7d265 changeset: 14235:0d502dc7d265 user: Timo Sirainen date: Mon Feb 27 14:37:45 2012 +0200 description: doveadm: Allow subcommands to specify '+' getopt parameter. diffstat: src/doveadm/doveadm-mail.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diffs (15 lines): diff -r c07415305d9e -r 0d502dc7d265 src/doveadm/doveadm-mail.c --- a/src/doveadm/doveadm-mail.c Mon Feb 27 13:34:20 2012 +0200 +++ b/src/doveadm/doveadm-mail.c Mon Feb 27 14:37:45 2012 +0200 @@ -455,7 +455,10 @@ if (doveadm_debug) ctx->service_flags |= MAIL_STORAGE_SERVICE_FLAG_DEBUG; - getopt_args = t_strconcat("AS:u:", ctx->getopt_args, NULL); + getopt_args = "AS:u:"; + /* keep context's getopt_args first in case it contains '+' */ + if (ctx->getopt_args != NULL) + getopt_args = t_strconcat(ctx->getopt_args, getopt_args, NULL); ctx->cur_username = getenv("USER"); wildcard_user = NULL; while ((c = getopt(argc, argv, getopt_args)) > 0) { From dovecot at dovecot.org Sun May 20 03:26:23 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:23 +0300 Subject: dovecot-2.2: dsync: Removed accidentally committed debug code. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/47dcf6a607a9 changeset: 14237:47dcf6a607a9 user: Timo Sirainen date: Mon Feb 27 14:40:19 2012 +0200 description: dsync: Removed accidentally committed debug code. diffstat: src/doveadm/dsync/doveadm-dsync.c | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diffs (11 lines): diff -r d75b807afadc -r 47dcf6a607a9 src/doveadm/dsync/doveadm-dsync.c --- a/src/doveadm/dsync/doveadm-dsync.c Mon Feb 27 14:38:05 2012 +0200 +++ b/src/doveadm/dsync/doveadm-dsync.c Mon Feb 27 14:40:19 2012 +0200 @@ -93,7 +93,6 @@ p = "server"; } else { /* we're executing doveadm */ - abort(); p = "dsync-server"; } array_append(&cmd_args, &p, 1); From dovecot at dovecot.org Sun May 20 03:26:23 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:23 +0300 Subject: dovecot-2.2: dsync: Fixes to handling legacy dsync parameters. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d75b807afadc changeset: 14236:d75b807afadc user: Timo Sirainen date: Mon Feb 27 14:38:05 2012 +0200 description: dsync: Fixes to handling legacy dsync parameters. diffstat: src/doveadm/dsync/doveadm-dsync.c | 33 +++++++++++++++++++++++++-------- 1 files changed, 25 insertions(+), 8 deletions(-) diffs (94 lines): diff -r 0d502dc7d265 -r d75b807afadc src/doveadm/dsync/doveadm-dsync.c --- a/src/doveadm/dsync/doveadm-dsync.c Mon Feb 27 14:37:45 2012 +0200 +++ b/src/doveadm/dsync/doveadm-dsync.c Mon Feb 27 14:38:05 2012 +0200 @@ -35,6 +35,7 @@ }; static const char *ssh_cmd = "ssh"; +static bool legacy_dsync = FALSE; static void run_cmd(const char *const *args, int *fd_in_r, int *fd_out_r) { @@ -87,13 +88,12 @@ array_append(&cmd_args, &p, 1); } - p = strchr(argv[0], '/'); - if (p == NULL) p = argv[0]; - if (strstr(p, "dsync") != NULL) { + if (legacy_dsync) { /* we're executing dsync */ p = "server"; } else { /* we're executing doveadm */ + abort(); p = "dsync-server"; } array_append(&cmd_args, &p, 1); @@ -307,7 +307,8 @@ switch (c) { case 'E': - /* dsync backup wrapper detection flag */ + /* dsync wrapper detection flag */ + legacy_dsync = TRUE; break; case 'f': ctx->brain_flags |= DSYNC_BRAIN_FLAG_FULL_SYNC; @@ -329,7 +330,7 @@ struct dsync_cmd_context *ctx; ctx = doveadm_mail_cmd_alloc(struct dsync_cmd_context); - ctx->ctx.getopt_args = "EfRm:"; + ctx->ctx.getopt_args = "+EfRm:"; ctx->ctx.v.parse_arg = cmd_mailbox_dsync_parse_arg; ctx->ctx.v.preinit = cmd_dsync_preinit; ctx->ctx.v.init = cmd_dsync_init; @@ -369,11 +370,27 @@ return 0; } +static bool +cmd_mailbox_dsync_server_parse_arg(struct doveadm_mail_cmd_context *_ctx, int c) +{ + switch (c) { + case 'E': + /* dsync wrapper detection flag */ + legacy_dsync = TRUE; + break; + default: + return FALSE; + } + return TRUE; +} + static struct doveadm_mail_cmd_context *cmd_dsync_server_alloc(void) { struct doveadm_mail_cmd_context *ctx; ctx = doveadm_mail_cmd_alloc(struct doveadm_mail_cmd_context); + ctx->getopt_args = "E"; + ctx->v.parse_arg = cmd_mailbox_dsync_server_parse_arg; ctx->v.run = cmd_dsync_server_run; return ctx; } @@ -500,9 +517,8 @@ } /* dsync flags */ - new_flags[0] = '-'; i = 1; - if (backup_flag) - new_flags[i++] = 'E'; + new_flags[0] = '-'; + new_flags[1] = 'E'; i = 2; if (flag_f) new_flags[i++] = 'f'; if (flag_R) @@ -528,6 +544,7 @@ i_assert(dest < max_argc); new_argv[dest] = NULL; + legacy_dsync = TRUE; *_argc = dest; *_argv = new_argv; optind = 1; From dovecot at dovecot.org Sun May 20 03:26:23 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:23 +0300 Subject: dovecot-2.2: mysql: Log idle time also for CR_SERVER_LOST errors. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/1999ae5c9699 changeset: 14238:1999ae5c9699 user: Timo Sirainen date: Mon Feb 27 16:31:33 2012 +0200 description: mysql: Log idle time also for CR_SERVER_LOST errors. diffstat: src/lib-sql/driver-mysql.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diffs (16 lines): diff -r 47dcf6a607a9 -r 1999ae5c9699 src/lib-sql/driver-mysql.c --- a/src/lib-sql/driver-mysql.c Mon Feb 27 14:40:19 2012 +0200 +++ b/src/lib-sql/driver-mysql.c Mon Feb 27 16:31:33 2012 +0200 @@ -464,9 +464,11 @@ struct mysql_db *db = (struct mysql_db *)_result->db; const char *errstr; unsigned int idle_time; + int err; + err = mysql_errno(db->mysql); errstr = mysql_error(db->mysql); - if (mysql_errno(db->mysql) == CR_SERVER_GONE_ERROR && + if ((err == CR_SERVER_GONE_ERROR || err == CR_SERVER_LOST) && db->last_success != 0) { idle_time = ioloop_time - db->last_success; errstr = t_strdup_printf("%s (idled for %u secs)", From dovecot at dovecot.org Sun May 20 03:26:23 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:23 +0300 Subject: dovecot-2.2: auth worker: When idle_kill timeout is reached, cha... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/5074680f321a changeset: 14239:5074680f321a user: Timo Sirainen date: Mon Feb 27 16:52:44 2012 +0200 description: auth worker: When idle_kill timeout is reached, change process title to indicate it. diffstat: src/auth/auth-worker-client.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (17 lines): diff -r 1999ae5c9699 -r 5074680f321a src/auth/auth-worker-client.c --- a/src/auth/auth-worker-client.c Mon Feb 27 16:31:33 2012 +0200 +++ b/src/auth/auth-worker-client.c Mon Feb 27 16:52:44 2012 +0200 @@ -19,6 +19,7 @@ #define CLIENT_STATE_HANDSHAKE "handshaking" #define CLIENT_STATE_IDLE "idling" +#define CLIENT_STATE_STOP "waiting for shutdown" struct auth_worker_client { int refcount; @@ -776,4 +777,5 @@ { if (auth_worker_client != NULL) o_stream_send_str(auth_worker_client->output, "SHUTDOWN\n"); + auth_worker_refresh_proctitle(CLIENT_STATE_STOP); } From dovecot at dovecot.org Sun May 20 03:26:23 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:23 +0300 Subject: dovecot-2.2: fs layout: Mailbox listing returned duplicates when... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/07fc8baa0c9b changeset: 14241:07fc8baa0c9b user: Timo Sirainen date: Tue Feb 28 04:29:12 2012 +0200 description: fs layout: Mailbox listing returned duplicates when patterns contained a parent and its children. diffstat: src/lib-storage/list/mailbox-list-fs-iter.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (13 lines): diff -r 3b328ffffdd4 -r 07fc8baa0c9b src/lib-storage/list/mailbox-list-fs-iter.c --- a/src/lib-storage/list/mailbox-list-fs-iter.c Mon Feb 27 16:53:50 2012 +0200 +++ b/src/lib-storage/list/mailbox-list-fs-iter.c Tue Feb 28 04:29:12 2012 +0200 @@ -360,7 +360,8 @@ childp = array_idx(&ctx->roots, i); parentlen = strlen(*parentp); if (strncmp(*parentp, *childp, parentlen) == 0 && - ((*childp)[parentlen] == ctx->sep || + (parentlen == 0 || + (*childp)[parentlen] == ctx->sep || (*childp)[parentlen] == '\0')) array_delete(&ctx->roots, i, 1); } From dovecot at dovecot.org Sun May 20 03:26:23 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:23 +0300 Subject: dovecot-2.2: auth worker: reset idle time after running a comman... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/3b328ffffdd4 changeset: 14240:3b328ffffdd4 user: Timo Sirainen date: Mon Feb 27 16:53:50 2012 +0200 description: auth worker: reset idle time after running a command, not before. diffstat: src/auth/auth-worker-client.c | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diffs (22 lines): diff -r 5074680f321a -r 3b328ffffdd4 src/auth/auth-worker-client.c --- a/src/auth/auth-worker-client.c Mon Feb 27 16:52:44 2012 +0200 +++ b/src/auth/auth-worker-client.c Mon Feb 27 16:53:50 2012 +0200 @@ -596,9 +596,6 @@ char *line; bool ret; - if (client->to_idle != NULL) - timeout_reset(client->to_idle); - switch (i_stream_read(client->input)) { case 0: return; @@ -656,6 +653,8 @@ break; } } + if (client->to_idle != NULL) + timeout_reset(client->to_idle); auth_worker_client_unref(&client); } From dovecot at dovecot.org Sun May 20 03:26:23 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:23 +0300 Subject: dovecot-2.2: Compiler warning fixes Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/54aeb3853bae changeset: 14242:54aeb3853bae user: Timo Sirainen date: Tue Feb 28 05:14:00 2012 +0200 description: Compiler warning fixes diffstat: src/doveadm/dsync/doveadm-dsync.c | 9 ++++----- 1 files changed, 4 insertions(+), 5 deletions(-) diffs (34 lines): diff -r 07fc8baa0c9b -r 54aeb3853bae src/doveadm/dsync/doveadm-dsync.c --- a/src/doveadm/dsync/doveadm-dsync.c Tue Feb 28 04:29:12 2012 +0200 +++ b/src/doveadm/dsync/doveadm-dsync.c Tue Feb 28 05:14:00 2012 +0200 @@ -370,7 +370,8 @@ } static bool -cmd_mailbox_dsync_server_parse_arg(struct doveadm_mail_cmd_context *_ctx, int c) +cmd_mailbox_dsync_server_parse_arg(struct doveadm_mail_cmd_context *_ctx ATTR_UNUSED, + int c) { switch (c) { case 'E': @@ -414,7 +415,6 @@ char *p, *dup, new_flags[6]; int max_argc, src, dest, i, j; bool flag_f = FALSE, flag_R = FALSE, flag_m, flag_u, flag_C, has_arg; - bool backup_flag = FALSE; p = strrchr(argv[0], '/'); if (p == NULL) p = argv[0]; @@ -501,10 +501,9 @@ } if (strcmp(argv[src], "mirror") == 0) new_argv[dest] = "sync"; - else if (strcmp(argv[src], "backup") == 0) { - backup_flag = TRUE; + else if (strcmp(argv[src], "backup") == 0) new_argv[dest] = "backup"; - } else if (strcmp(argv[src], "server") == 0) + else if (strcmp(argv[src], "server") == 0) new_argv[dest] = "dsync-server"; else i_fatal("Invalid parameter: %s", argv[src]); From dovecot at dovecot.org Sun May 20 03:26:23 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:23 +0300 Subject: dovecot-2.2: mailbox list indexes: Avoid unnecessary refresh fla... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/ba2b4f8a1bb1 changeset: 14243:ba2b4f8a1bb1 user: Timo Sirainen date: Tue Feb 28 06:52:43 2012 +0200 description: mailbox list indexes: Avoid unnecessary refresh flag writes. diffstat: src/lib-storage/list/mailbox-list-index-status.c | 2 ++ src/lib-storage/list/mailbox-list-index-sync.c | 2 +- src/lib-storage/list/mailbox-list-index.c | 5 ++--- src/lib-storage/list/mailbox-list-index.h | 2 ++ 4 files changed, 7 insertions(+), 4 deletions(-) diffs (51 lines): diff -r 54aeb3853bae -r ba2b4f8a1bb1 src/lib-storage/list/mailbox-list-index-status.c --- a/src/lib-storage/list/mailbox-list-index-status.c Tue Feb 28 05:14:00 2012 +0200 +++ b/src/lib-storage/list/mailbox-list-index-status.c Tue Feb 28 06:52:43 2012 +0200 @@ -289,6 +289,8 @@ struct mailbox_status status; uint32_t seq, seq1, seq2; + (void)mailbox_list_index_refresh(box->list); + node = mailbox_list_index_lookup(box->list, box->name); if (node == NULL) { mailbox_list_index_refresh_later(box->list); diff -r 54aeb3853bae -r ba2b4f8a1bb1 src/lib-storage/list/mailbox-list-index-sync.c --- a/src/lib-storage/list/mailbox-list-index-sync.c Tue Feb 28 05:14:00 2012 +0200 +++ b/src/lib-storage/list/mailbox-list-index-sync.c Tue Feb 28 06:52:43 2012 +0200 @@ -301,7 +301,7 @@ T_BEGIN { mailbox_list_index_sync_names(&sync_ctx); } T_END; - } else { + } else if (mailbox_list_index_need_refresh(ilist, sync_ctx.view)) { /* we're synced, reset refresh flag */ struct mailbox_list_index_header new_hdr; diff -r 54aeb3853bae -r ba2b4f8a1bb1 src/lib-storage/list/mailbox-list-index.c --- a/src/lib-storage/list/mailbox-list-index.c Tue Feb 28 05:14:00 2012 +0200 +++ b/src/lib-storage/list/mailbox-list-index.c Tue Feb 28 06:52:43 2012 +0200 @@ -197,9 +197,8 @@ return 0; } -static bool -mailbox_list_index_need_refresh(struct mailbox_list_index *ilist, - struct mail_index_view *view) +bool mailbox_list_index_need_refresh(struct mailbox_list_index *ilist, + struct mail_index_view *view) { const struct mailbox_list_index_header *hdr; const void *data; diff -r 54aeb3853bae -r ba2b4f8a1bb1 src/lib-storage/list/mailbox-list-index.h --- a/src/lib-storage/list/mailbox-list-index.h Tue Feb 28 05:14:00 2012 +0200 +++ b/src/lib-storage/list/mailbox-list-index.h Tue Feb 28 06:52:43 2012 +0200 @@ -124,6 +124,8 @@ struct mailbox_list_index_node * mailbox_list_index_lookup(struct mailbox_list *list, const char *name); +bool mailbox_list_index_need_refresh(struct mailbox_list_index *ilist, + struct mail_index_view *view); int mailbox_list_index_refresh(struct mailbox_list *list); void mailbox_list_index_refresh_later(struct mailbox_list *list); From dovecot at dovecot.org Sun May 20 03:26:23 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:23 +0300 Subject: dovecot-2.2: Show SEARCH=FUZZY in IMAP capabilities only when FT... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/bdc881838b00 changeset: 14244:bdc881838b00 user: Timo Sirainen date: Tue Feb 28 18:50:46 2012 +0200 description: Show SEARCH=FUZZY in IMAP capabilities only when FTS backend actually supports it. diffstat: configure.in | 2 +- src/imap/imap-client.c | 5 +++++ src/lib-storage/mail-user.h | 2 ++ src/plugins/fts-lucene/fts-backend-lucene.c | 3 ++- src/plugins/fts-solr/fts-backend-solr.c | 2 +- src/plugins/fts/fts-api-private.h | 4 +++- src/plugins/fts/fts-storage.c | 3 +++ 7 files changed, 17 insertions(+), 4 deletions(-) diffs (91 lines): diff -r ba2b4f8a1bb1 -r bdc881838b00 configure.in --- a/configure.in Tue Feb 28 06:52:43 2012 +0200 +++ b/configure.in Tue Feb 28 18:50:46 2012 +0200 @@ -2697,7 +2697,7 @@ dnl IDLE doesn't really belong to banner. It's there just to make Blackberries dnl happy, because otherwise BIS server disables push email. capability_banner="IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE" -capability="$capability_banner SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SEARCH=FUZZY SPECIAL-USE" +capability="$capability_banner SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE" AC_DEFINE_UNQUOTED(CAPABILITY_STRING, "$capability", IMAP capabilities) AC_DEFINE_UNQUOTED(CAPABILITY_BANNER_STRING, "$capability_banner", IMAP capabilities advertised in banner) diff -r ba2b4f8a1bb1 -r bdc881838b00 src/imap/imap-client.c --- a/src/imap/imap-client.c Tue Feb 28 06:52:43 2012 +0200 +++ b/src/imap/imap-client.c Tue Feb 28 18:50:46 2012 +0200 @@ -82,6 +82,11 @@ str_append_c(client->capability_string, ' '); str_append(client->capability_string, set->imap_capability + 1); } + if (user->fuzzy_search) { + /* Enable FUZZY capability only when it actually has + a chance of working */ + str_append(client->capability_string, " SEARCH=FUZZY"); + } ident = mail_user_get_anvil_userip_ident(client->user); if (ident != NULL) { diff -r ba2b4f8a1bb1 -r bdc881838b00 src/lib-storage/mail-user.h --- a/src/lib-storage/mail-user.h Tue Feb 28 06:52:43 2012 +0200 +++ b/src/lib-storage/mail-user.h Tue Feb 28 18:50:46 2012 +0200 @@ -54,6 +54,8 @@ unsigned int mail_debug:1; /* If INBOX can't be opened, log an error, but only once. */ unsigned int inbox_open_error_logged:1; + /* Fuzzy search works for this user (FTS enabled) */ + unsigned int fuzzy_search:1; }; struct mail_user_module_register { diff -r ba2b4f8a1bb1 -r bdc881838b00 src/plugins/fts-lucene/fts-backend-lucene.c --- a/src/plugins/fts-lucene/fts-backend-lucene.c Tue Feb 28 06:52:43 2012 +0200 +++ b/src/plugins/fts-lucene/fts-backend-lucene.c Tue Feb 28 18:50:46 2012 +0200 @@ -552,7 +552,8 @@ struct fts_backend fts_backend_lucene = { .name = "lucene", - .flags = FTS_BACKEND_FLAG_BUILD_FULL_WORDS, + .flags = FTS_BACKEND_FLAG_BUILD_FULL_WORDS | + FTS_BACKEND_FLAG_FUZZY_SEARCH, { fts_backend_lucene_alloc, diff -r ba2b4f8a1bb1 -r bdc881838b00 src/plugins/fts-solr/fts-backend-solr.c --- a/src/plugins/fts-solr/fts-backend-solr.c Tue Feb 28 06:52:43 2012 +0200 +++ b/src/plugins/fts-solr/fts-backend-solr.c Tue Feb 28 18:50:46 2012 +0200 @@ -833,7 +833,7 @@ struct fts_backend fts_backend_solr = { .name = "solr", - .flags = 0, + .flags = FTS_BACKEND_FLAG_FUZZY_SEARCH, { fts_backend_solr_alloc, diff -r ba2b4f8a1bb1 -r bdc881838b00 src/plugins/fts/fts-api-private.h --- a/src/plugins/fts/fts-api-private.h Tue Feb 28 06:52:43 2012 +0200 +++ b/src/plugins/fts/fts-api-private.h Tue Feb 28 18:50:46 2012 +0200 @@ -57,7 +57,9 @@ preserving original case */ FTS_BACKEND_FLAG_BUILD_DTCASE = 0x02, /* Send only fully indexable words rather than randomly sized blocks */ - FTS_BACKEND_FLAG_BUILD_FULL_WORDS = 0x04 + FTS_BACKEND_FLAG_BUILD_FULL_WORDS = 0x04, + /* Fuzzy search works */ + FTS_BACKEND_FLAG_FUZZY_SEARCH = 0x08 }; struct fts_backend { diff -r ba2b4f8a1bb1 -r bdc881838b00 src/plugins/fts/fts-storage.c --- a/src/plugins/fts/fts-storage.c Tue Feb 28 06:52:43 2012 +0200 +++ b/src/plugins/fts/fts-storage.c Tue Feb 28 18:50:46 2012 +0200 @@ -637,6 +637,9 @@ struct fts_mailbox_list *flist; struct mailbox_list_vfuncs *v = list->vlast; + if ((backend->flags & FTS_BACKEND_FLAG_FUZZY_SEARCH) != 0) + list->ns->user->fuzzy_search = TRUE; + flist = p_new(list->pool, struct fts_mailbox_list, 1); flist->module_ctx.super = *v; flist->backend = backend; From dovecot at dovecot.org Sun May 20 03:26:23 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:23 +0300 Subject: dovecot-2.2: fts-squat: Fixed search to actually work. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/ee6f3f375dd3 changeset: 14245:ee6f3f375dd3 user: Timo Sirainen date: Wed Feb 29 12:53:21 2012 +0200 description: fts-squat: Fixed search to actually work. diffstat: src/plugins/fts-squat/fts-backend-squat.c | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diffs (28 lines): diff -r bdc881838b00 -r ee6f3f375dd3 src/plugins/fts-squat/fts-backend-squat.c --- a/src/plugins/fts-squat/fts-backend-squat.c Tue Feb 28 18:50:46 2012 +0200 +++ b/src/plugins/fts-squat/fts-backend-squat.c Wed Feb 29 12:53:21 2012 +0200 @@ -436,6 +436,7 @@ { struct squat_fts_backend *backend = (struct squat_fts_backend *)_backend; + bool first = TRUE; int ret; fts_backend_squat_set_box(backend, box); @@ -446,14 +447,14 @@ } for (; args != NULL; args = args->next) { - ret = squat_lookup_arg(backend, args, and_args, + ret = squat_lookup_arg(backend, args, first ? FALSE : and_args, &result->definite_uids, &result->maybe_uids); if (ret < 0) return -1; if (ret > 0) args->match_always = TRUE; - + first = FALSE; } return 0; } From dovecot at dovecot.org Sun May 20 03:26:23 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:23 +0300 Subject: dovecot-2.2: mailbox_list_index=yes: Fixed mailbox listing when ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/99cde8ce9991 changeset: 14246:99cde8ce9991 user: Timo Sirainen date: Wed Feb 29 13:04:24 2012 +0200 description: mailbox_list_index=yes: Fixed mailbox listing when not using default namespace settings. diffstat: src/lib-storage/list/mailbox-list-index-iter.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r ee6f3f375dd3 -r 99cde8ce9991 src/lib-storage/list/mailbox-list-index-iter.c --- a/src/lib-storage/list/mailbox-list-index-iter.c Wed Feb 29 12:53:21 2012 +0200 +++ b/src/lib-storage/list/mailbox-list-index-iter.c Wed Feb 29 13:04:24 2012 +0200 @@ -51,7 +51,7 @@ str_append_c(ctx->path, ctx->sep); str_append(ctx->path, node->name); - ctx->info.name = str_c(ctx->path); + ctx->info.name = mailbox_list_get_vname(ctx->ctx.list, str_c(ctx->path)); ctx->info.flags = 0; if ((node->flags & MAILBOX_LIST_INDEX_FLAG_NONEXISTENT) != 0) ctx->info.flags |= MAILBOX_NONEXISTENT; From dovecot at dovecot.org Sun May 20 03:26:23 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:23 +0300 Subject: dovecot-2.2: login: If session timeouts after authentication, lo... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/855856a9f139 changeset: 14247:855856a9f139 user: Timo Sirainen date: Fri Mar 02 12:23:00 2012 +0200 description: login: If session timeouts after authentication, log a better error about it. diffstat: src/login-common/client-common.c | 33 ++++++++++++++++++++++++++++++--- src/login-common/client-common.h | 2 +- src/login-common/sasl-server.c | 1 + 3 files changed, 32 insertions(+), 4 deletions(-) diffs (66 lines): diff -r 99cde8ce9991 -r 855856a9f139 src/login-common/client-common.c --- a/src/login-common/client-common.c Wed Feb 29 13:04:24 2012 +0200 +++ b/src/login-common/client-common.c Fri Mar 02 12:23:00 2012 +0200 @@ -26,9 +26,36 @@ static void client_idle_disconnect_timeout(struct client *client) { - client_send_line(client, CLIENT_CMD_REPLY_BYE, - "Disconnected for inactivity."); - client_destroy(client, "Disconnected: Inactivity"); + const char *user_reason, *destroy_reason; + unsigned int secs; + + if (client->master_tag != 0) { + secs = ioloop_time - client->auth_finished; + user_reason = "Timeout while finishing login."; + destroy_reason = t_strdup_printf( + "Timeout while finishing login (waited %u secs)", secs); + client_log_err(client, destroy_reason); + } else if (client->auth_request != NULL) { + user_reason = + "Disconnected for inactivity during authentication."; + destroy_reason = + "Disconnected: Inactivity during authentication"; + } else if (client->login_proxy != NULL) { + secs = ioloop_time - client->created; + user_reason = "Timeout while finishing login."; + destroy_reason = t_strdup_printf( + "proxy: Logging in to %s:%u timed out " + "(state=%u, duration=%us)", + login_proxy_get_host(client->login_proxy), + login_proxy_get_port(client->login_proxy), + client->proxy_state, secs); + client_log_err(client, destroy_reason); + } else { + user_reason = "Disconnected for inactivity."; + destroy_reason = "Disconnected: Inactivity"; + } + client_send_line(client, CLIENT_CMD_REPLY_BYE, user_reason); + client_destroy(client, destroy_reason); } static void client_open_streams(struct client *client) diff -r 99cde8ce9991 -r 855856a9f139 src/login-common/client-common.h --- a/src/login-common/client-common.h Wed Feb 29 13:04:24 2012 +0200 +++ b/src/login-common/client-common.h Fri Mar 02 12:23:00 2012 +0200 @@ -107,7 +107,7 @@ char *auth_mech_name; struct auth_client_request *auth_request; string_t *auth_response; - time_t auth_first_started; + time_t auth_first_started, auth_finished; const char *sasl_final_resp; unsigned int master_auth_id; diff -r 99cde8ce9991 -r 855856a9f139 src/login-common/sasl-server.c --- a/src/login-common/sasl-server.c Wed Feb 29 13:04:24 2012 +0200 +++ b/src/login-common/sasl-server.c Fri Mar 02 12:23:00 2012 +0200 @@ -144,6 +144,7 @@ buffer_append(buf, data, size); req.data_size = buf->used; + client->auth_finished = ioloop_time; client->master_auth_id = req.auth_id; master_auth_request(master_auth, client->fd, &req, buf->data, master_auth_callback, client, &client->master_tag); From dovecot at dovecot.org Sun May 20 03:26:23 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:23 +0300 Subject: dovecot-2.2: doveadm: Added prerun() method for mail commands. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/1a33e3651c6d changeset: 14248:1a33e3651c6d user: Timo Sirainen date: Fri Mar 02 13:15:15 2012 +0200 description: doveadm: Added prerun() method for mail commands. diffstat: src/doveadm/doveadm-mail.c | 8 ++++++++ src/doveadm/doveadm-mail.h | 3 +++ 2 files changed, 11 insertions(+), 0 deletions(-) diffs (38 lines): diff -r 855856a9f139 -r 1a33e3651c6d src/doveadm/doveadm-mail.c --- a/src/doveadm/doveadm-mail.c Fri Mar 02 12:23:00 2012 +0200 +++ b/src/doveadm/doveadm-mail.c Fri Mar 02 13:15:15 2012 +0200 @@ -292,6 +292,13 @@ return ret; } + if (ctx->v.prerun != NULL) { + if (ctx->v.prerun(ctx, ctx->cur_service_user, error_r) < 0) { + mail_storage_service_user_free(&ctx->cur_service_user); + return -1; + } + } + ret = mail_storage_service_next(ctx->storage_service, ctx->cur_service_user, &ctx->cur_mail_user); @@ -372,6 +379,7 @@ continue; } input.username = user; + ctx->cur_username = user; doveadm_print_sticky("username", user); T_BEGIN { ret = doveadm_mail_next_user(ctx, &input, &error); diff -r 855856a9f139 -r 1a33e3651c6d src/doveadm/doveadm-mail.h --- a/src/doveadm/doveadm-mail.h Fri Mar 02 12:23:00 2012 +0200 +++ b/src/doveadm/doveadm-mail.h Fri Mar 02 13:15:15 2012 +0200 @@ -20,6 +20,9 @@ const char *const args[]); int (*get_next_user)(struct doveadm_mail_cmd_context *ctx, const char **username_r); + int (*prerun)(struct doveadm_mail_cmd_context *ctx, + struct mail_storage_service_user *service_user, + const char **error_r); int (*run)(struct doveadm_mail_cmd_context *ctx, struct mail_user *mail_user); void (*deinit)(struct doveadm_mail_cmd_context *ctx); From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: dsync: Added -l parameter to lock the syn... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d4922e80ca05 changeset: 14249:d4922e80ca05 user: Timo Sirainen date: Fri Mar 02 13:56:37 2012 +0200 description: dsync: Added -l parameter to lock the sync (via ~/.dovecot-sync.lock) diffstat: src/doveadm/dsync/doveadm-dsync.c | 169 +++++++++++++++++++++++++++++-------- 1 files changed, 131 insertions(+), 38 deletions(-) diffs (286 lines): diff -r 1a33e3651c6d -r d4922e80ca05 src/doveadm/dsync/doveadm-dsync.c --- a/src/doveadm/dsync/doveadm-dsync.c Fri Mar 02 13:15:15 2012 +0200 +++ b/src/doveadm/dsync/doveadm-dsync.c Fri Mar 02 13:56:37 2012 +0200 @@ -21,6 +21,8 @@ #include #include +#define DSYNC_LOCK_FILENAME ".dovecot-sync.lock" + struct dsync_cmd_context { struct doveadm_mail_cmd_context ctx; enum dsync_brain_flags brain_flags; @@ -31,6 +33,9 @@ int fd_in, fd_out; + unsigned int lock_timeout; + + unsigned int lock:1; unsigned int reverse_workers:1; }; @@ -100,7 +105,8 @@ *cmd_args_r = array_idx(&cmd_args, 0); } -static bool mirror_get_remote_cmd(const char *const *argv, const char *user, +static bool mirror_get_remote_cmd(struct dsync_cmd_context *ctx, + const char *const *argv, const char *user, const char *const **cmd_args_r) { ARRAY_TYPE(const_string) cmd_args; @@ -146,6 +152,10 @@ array_append(&cmd_args, &host, 1); p = "doveadm"; array_append(&cmd_args, &p, 1); p = "dsync-server"; array_append(&cmd_args, &p, 1); + if (ctx->lock) { + p = "-l"; array_append(&cmd_args, &p, 1); + p = dec2str(ctx->lock_timeout); array_append(&cmd_args, &p, 1); + } if (*user != '\0') { p = "-u"; array_append(&cmd_args, &p, 1); array_append(&cmd_args, &user, 1); @@ -208,13 +218,75 @@ return dsync_worker_init_proxy_client(ctx->fd_in, ctx->fd_out); } +static int dsync_lock(struct mail_user *user, unsigned int lock_timeout, + const char **path_r, struct file_lock **lock_r) +{ + const char *home, *path; + int ret, fd; + + if ((ret = mail_user_get_home(user, &home)) < 0) { + i_error("Couldn't look up user's home dir"); + return -1; + } + if (ret == 0) { + i_error("User has no home directory"); + return -1; + } + + path = t_strconcat(home, "/"DSYNC_LOCK_FILENAME, NULL); + fd = creat(path, 0600); + if (fd == -1) { + i_error("Couldn't create lock %s: %m", path); + return -1; + } + + if (file_wait_lock(fd, path, F_WRLCK, FILE_LOCK_METHOD_FCNTL, + lock_timeout, lock_r) <= 0) { + i_error("Couldn't lock %s: %m", path); + (void)close(fd); + return -1; + } + *path_r = path; + return fd; +} + +static int +cmd_dsync_start(struct dsync_cmd_context *ctx, struct dsync_worker *worker1, + struct dsync_worker *worker2) +{ + struct dsync_brain *brain; + + /* create and run the brain */ + brain = dsync_brain_init(worker1, worker2, ctx->mailbox, + ctx->brain_flags); + if (ctx->remote_cmd_args == NULL) + dsync_brain_sync_all(brain); + else { + dsync_brain_sync(brain); + if (!dsync_brain_has_failed(brain)) + io_loop_run(current_ioloop); + } + /* deinit */ + if (dsync_brain_has_unexpected_changes(brain)) { + i_warning("Mailbox changes caused a desync. " + "You may want to run dsync again."); + ctx->ctx.exit_code = 2; + } + if (dsync_brain_deinit(&brain) < 0) { + ctx->ctx.exit_code = EX_TEMPFAIL; + return -1; + } + return 0; +} + static int cmd_dsync_run(struct doveadm_mail_cmd_context *_ctx, struct mail_user *user) { struct dsync_cmd_context *ctx = (struct dsync_cmd_context *)_ctx; struct dsync_worker *worker1, *worker2, *workertmp; - struct dsync_brain *brain; - int ret = 0; + const char *lock_path; + struct file_lock *lock; + int lock_fd, ret = 0; user->admin = TRUE; @@ -230,27 +302,20 @@ worker2 = workertmp; } - /* create and run the brain */ - brain = dsync_brain_init(worker1, worker2, ctx->mailbox, - ctx->brain_flags); - if (ctx->remote_cmd_args == NULL) - dsync_brain_sync_all(brain); + if (!ctx->lock) + ret = cmd_dsync_start(ctx, worker1, worker2); else { - dsync_brain_sync(brain); - if (!dsync_brain_has_failed(brain)) - io_loop_run(current_ioloop); + lock_fd = dsync_lock(user, ctx->lock_timeout, &lock_path, &lock); + if (lock_fd == -1) { + _ctx->exit_code = EX_TEMPFAIL; + ret = -1; + } else { + ret = cmd_dsync_start(ctx, worker1, worker2); + file_lock_free(&lock); + if (close(lock_fd) < 0) + i_error("close(%s) failed: %m", lock_path); + } } - /* deinit */ - if (dsync_brain_has_unexpected_changes(brain)) { - i_warning("Mailbox changes caused a desync. " - "You may want to run dsync again."); - _ctx->exit_code = 2; - } - if (dsync_brain_deinit(&brain) < 0) { - _ctx->exit_code = EX_TEMPFAIL; - ret = -1; - } - dsync_worker_deinit(&worker1); dsync_worker_deinit(&worker2); return ret; @@ -276,7 +341,7 @@ if ((_ctx->service_flags & MAIL_STORAGE_SERVICE_FLAG_USERDB_LOOKUP) != 0 && _ctx->cur_username != NULL) username = _ctx->cur_username; - if (!mirror_get_remote_cmd(args, username, &ctx->remote_cmd_args)) { + if (!mirror_get_remote_cmd(ctx, args, username, &ctx->remote_cmd_args)) { /* it's a mail_location */ if (args[1] != NULL) doveadm_mail_help_name(_ctx->cmd->name); @@ -312,6 +377,11 @@ case 'f': ctx->brain_flags |= DSYNC_BRAIN_FLAG_FULL_SYNC; break; + case 'l': + ctx->lock = TRUE; + if (str_to_uint(optarg, &ctx->lock_timeout) < 0) + i_error("Invalid -l parameter: %s", optarg); + break; case 'm': ctx->mailbox = optarg; break; @@ -329,7 +399,7 @@ struct dsync_cmd_context *ctx; ctx = doveadm_mail_cmd_alloc(struct dsync_cmd_context); - ctx->ctx.getopt_args = "+EfRm:"; + ctx->ctx.getopt_args = "+Efl:m:R"; ctx->ctx.v.parse_arg = cmd_mailbox_dsync_parse_arg; ctx->ctx.v.preinit = cmd_dsync_preinit; ctx->ctx.v.init = cmd_dsync_init; @@ -349,35 +419,58 @@ } static int -cmd_dsync_server_run(struct doveadm_mail_cmd_context *ctx, +cmd_dsync_server_run(struct doveadm_mail_cmd_context *_ctx, struct mail_user *user) { + struct dsync_cmd_context *ctx = (struct dsync_cmd_context *)_ctx; struct dsync_proxy_server *server; struct dsync_worker *worker; + struct file_lock *lock; + const char *lock_path; + int lock_fd, ret = 0; user->admin = TRUE; i_set_failure_prefix(t_strdup_printf("dsync-remote(%s): ", user->username)); - worker = dsync_worker_init_local(user, *ctx->set->dsync_alt_char); + worker = dsync_worker_init_local(user, *_ctx->set->dsync_alt_char); server = dsync_proxy_server_init(STDIN_FILENO, STDOUT_FILENO, worker); - io_loop_run(current_ioloop); + if (!ctx->lock) + io_loop_run(current_ioloop); + else { + lock_fd = dsync_lock(user, ctx->lock_timeout, &lock_path, &lock); + if (lock_fd == -1) { + _ctx->exit_code = EX_TEMPFAIL; + ret = -1; + } else { + io_loop_run(current_ioloop); + file_lock_free(&lock); + if (close(lock_fd) < 0) + i_error("close(%s) failed: %m", lock_path); + } + } dsync_proxy_server_deinit(&server); dsync_worker_deinit(&worker); - return 0; + return ret; } static bool -cmd_mailbox_dsync_server_parse_arg(struct doveadm_mail_cmd_context *_ctx ATTR_UNUSED, - int c) +cmd_mailbox_dsync_server_parse_arg(struct doveadm_mail_cmd_context *_ctx, int c) { + struct dsync_cmd_context *ctx = (struct dsync_cmd_context *)_ctx; + switch (c) { case 'E': /* dsync wrapper detection flag */ legacy_dsync = TRUE; break; + case 'l': + ctx->lock = TRUE; + if (str_to_uint(optarg, &ctx->lock_timeout) < 0) + i_error("Invalid -l parameter: %s", optarg); + break; default: return FALSE; } @@ -386,21 +479,21 @@ static struct doveadm_mail_cmd_context *cmd_dsync_server_alloc(void) { - struct doveadm_mail_cmd_context *ctx; + struct dsync_cmd_context *ctx; - ctx = doveadm_mail_cmd_alloc(struct doveadm_mail_cmd_context); - ctx->getopt_args = "E"; - ctx->v.parse_arg = cmd_mailbox_dsync_server_parse_arg; - ctx->v.run = cmd_dsync_server_run; - return ctx; + ctx = doveadm_mail_cmd_alloc(struct dsync_cmd_context); + ctx->ctx.getopt_args = "El:"; + ctx->ctx.v.parse_arg = cmd_mailbox_dsync_server_parse_arg; + ctx->ctx.v.run = cmd_dsync_server_run; + return &ctx->ctx; } struct doveadm_mail_cmd cmd_dsync_mirror = { - cmd_dsync_alloc, "sync", "[-fR] [-m ] " + cmd_dsync_alloc, "sync", "[-fR] [-l ] [-m ] " }; struct doveadm_mail_cmd cmd_dsync_backup = { cmd_dsync_backup_alloc, "backup", - "[-fR] [-m ] " + "[-fR] [-l ] [-m ] " }; struct doveadm_mail_cmd cmd_dsync_server = { cmd_dsync_server_alloc, "dsync-server", &doveadm_mail_cmd_hide From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: dsync: Prefix remote dsync's error messages with "r... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/523a6f3e0713 changeset: 14250:523a6f3e0713 user: Timo Sirainen date: Fri Mar 02 14:06:18 2012 +0200 description: dsync: Prefix remote dsync's error messages with "remote:". diffstat: src/doveadm/dsync/doveadm-dsync.c | 60 +++++++++++++++++++++++++++++--------- 1 files changed, 45 insertions(+), 15 deletions(-) diffs (121 lines): diff -r d4922e80ca05 -r 523a6f3e0713 src/doveadm/dsync/doveadm-dsync.c --- a/src/doveadm/dsync/doveadm-dsync.c Fri Mar 02 13:56:37 2012 +0200 +++ b/src/doveadm/dsync/doveadm-dsync.c Fri Mar 02 14:06:18 2012 +0200 @@ -31,7 +31,8 @@ const char *const *remote_cmd_args; const char *local_location; - int fd_in, fd_out; + int fd_in, fd_out, fd_err; + struct io *io_err; unsigned int lock_timeout; @@ -42,39 +43,61 @@ static const char *ssh_cmd = "ssh"; static bool legacy_dsync = FALSE; -static void run_cmd(const char *const *args, int *fd_in_r, int *fd_out_r) +static void remote_error_input(struct dsync_cmd_context *ctx) { - int fd_in[2], fd_out[2]; + char buf[1024]; + ssize_t ret; - if (pipe(fd_in) < 0 || pipe(fd_out) < 0) + ret = read(ctx->fd_err, buf, sizeof(buf)-1); + if (ret == -1) { + io_remove(&ctx->io_err); + return; + } + if (ret > 0) { + buf[ret-1] = '\0'; + i_error("remote: %s", buf); + } +} + +static void +run_cmd(struct dsync_cmd_context *ctx, const char *const *args) +{ + int fd_in[2], fd_out[2], fd_err[2]; + + if (pipe(fd_in) < 0 || pipe(fd_out) < 0 || pipe(fd_err) < 0) i_fatal("pipe() failed: %m"); switch (fork()) { case -1: i_fatal("fork() failed: %m"); - break; case 0: /* child, which will execute the proxy server. stdin/stdout goes to pipes which we'll pass to proxy client. */ if (dup2(fd_in[0], STDIN_FILENO) < 0 || - dup2(fd_out[1], STDOUT_FILENO) < 0) + dup2(fd_out[1], STDOUT_FILENO) < 0 || + dup2(fd_err[1], STDERR_FILENO) < 0) i_fatal("dup2() failed: %m"); (void)close(fd_in[0]); (void)close(fd_in[1]); (void)close(fd_out[0]); (void)close(fd_out[1]); + (void)close(fd_err[0]); + (void)close(fd_err[1]); execvp_const(args[0], args); - break; default: /* parent */ - (void)close(fd_in[0]); - (void)close(fd_out[1]); - *fd_in_r = fd_out[0]; - *fd_out_r = fd_in[1]; break; } + + (void)close(fd_in[0]); + (void)close(fd_out[1]); + (void)close(fd_err[1]); + ctx->fd_in = fd_out[0]; + ctx->fd_out = fd_in[1]; + ctx->fd_err = fd_err[0]; + ctx->io_err = io_add(ctx->fd_err, IO_READ, remote_error_input, ctx); } static void @@ -318,6 +341,12 @@ } dsync_worker_deinit(&worker1); dsync_worker_deinit(&worker2); + if (ctx->io_err != NULL) + io_remove(&ctx->io_err); + if (ctx->fd_err != -1) { + (void)close(ctx->fd_err); + ctx->fd_err = -1; + } return ret; } @@ -327,6 +356,10 @@ struct dsync_cmd_context *ctx = (struct dsync_cmd_context *)_ctx; const char *username = ""; + ctx->fd_in = STDIN_FILENO; + ctx->fd_out = STDOUT_FILENO; + ctx->fd_err = -1; + if (args[0] == NULL) doveadm_mail_help_name(_ctx->cmd->name); @@ -351,10 +384,7 @@ if (ctx->remote_cmd_args != NULL) { /* do this before mail_storage_service_next() in case it drops process privileges */ - run_cmd(ctx->remote_cmd_args, &ctx->fd_in, &ctx->fd_out); - } else { - ctx->fd_in = STDIN_FILENO; - ctx->fd_out = STDOUT_FILENO; + run_cmd(ctx, ctx->remote_cmd_args); } } From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: dsync: Added dsync_remote_cmd setting, which is use... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/69ed88beb12f changeset: 14251:69ed88beb12f user: Timo Sirainen date: Fri Mar 02 14:18:29 2012 +0200 description: dsync: Added dsync_remote_cmd setting, which is used to execute remote dsync. This is used for the old style "host" and "mailuser at host" parameters, as well as for a new "remote:[user@]host" parameter. diffstat: src/doveadm/doveadm-settings.c | 2 + src/doveadm/doveadm-settings.h | 1 + src/doveadm/dsync/doveadm-dsync.c | 165 +++++++++++++++++++++++++++---------- 3 files changed, 122 insertions(+), 46 deletions(-) diffs (284 lines): diff -r 523a6f3e0713 -r 69ed88beb12f src/doveadm/doveadm-settings.c --- a/src/doveadm/doveadm-settings.c Fri Mar 02 14:06:18 2012 +0200 +++ b/src/doveadm/doveadm-settings.c Fri Mar 02 14:18:29 2012 +0200 @@ -61,6 +61,7 @@ DEF(SET_STR, doveadm_password), DEF(SET_STR, doveadm_allowed_commands), DEF(SET_STR, dsync_alt_char), + DEF(SET_STR, dsync_remote_cmd), { SET_STRLIST, "plugin", offsetof(struct doveadm_settings, plugin_envs), NULL }, @@ -77,6 +78,7 @@ .doveadm_password = "", .doveadm_allowed_commands = "", .dsync_alt_char = "_", + .dsync_remote_cmd = "ssh -l%{login} %{host} doveadm dsync-server -u%u -l%{lock_timeout}", .plugin_envs = ARRAY_INIT }; diff -r 523a6f3e0713 -r 69ed88beb12f src/doveadm/doveadm-settings.h --- a/src/doveadm/doveadm-settings.h Fri Mar 02 14:06:18 2012 +0200 +++ b/src/doveadm/doveadm-settings.h Fri Mar 02 14:18:29 2012 +0200 @@ -11,6 +11,7 @@ const char *doveadm_password; const char *doveadm_allowed_commands; const char *dsync_alt_char; + const char *dsync_remote_cmd; ARRAY_DEFINE(plugin_envs, const char *); }; diff -r 523a6f3e0713 -r 69ed88beb12f src/doveadm/dsync/doveadm-dsync.c --- a/src/doveadm/dsync/doveadm-dsync.c Fri Mar 02 14:06:18 2012 +0200 +++ b/src/doveadm/dsync/doveadm-dsync.c Fri Mar 02 14:18:29 2012 +0200 @@ -4,6 +4,8 @@ #include "lib-signals.h" #include "array.h" #include "execv-const.h" +#include "str.h" +#include "var-expand.h" #include "settings-parser.h" #include "master-service.h" #include "mail-storage-service.h" @@ -28,7 +30,6 @@ enum dsync_brain_flags brain_flags; const char *mailbox; - const char *const *remote_cmd_args; const char *local_location; int fd_in, fd_out, fd_err; @@ -38,9 +39,9 @@ unsigned int lock:1; unsigned int reverse_workers:1; + unsigned int remote:1; }; -static const char *ssh_cmd = "ssh"; static bool legacy_dsync = FALSE; static void remote_error_input(struct dsync_cmd_context *ctx) @@ -128,12 +129,61 @@ *cmd_args_r = array_idx(&cmd_args, 0); } +static const char *const * +get_ssh_cmd_args(struct dsync_cmd_context *ctx, + const char *host, const char *login, const char *mail_user) +{ + static struct var_expand_table static_tab[] = { + { 'u', NULL, "user" }, + { '\0', NULL, "login" }, + { '\0', NULL, "host" }, + { '\0', NULL, "lock_timeout" }, + { '\0', NULL, NULL } + }; + struct var_expand_table *tab; + ARRAY_TYPE(const_string) cmd_args; + string_t *str, *str2; + const char *value, *const *args; + + tab = t_malloc(sizeof(static_tab)); + memcpy(tab, static_tab, sizeof(static_tab)); + + tab[0].value = mail_user; + tab[1].value = login; + tab[2].value = host; + tab[3].value = dec2str(ctx->lock_timeout); + + t_array_init(&cmd_args, 8); + str = t_str_new(128); + str2 = t_str_new(128); + args = t_strsplit(doveadm_settings->dsync_remote_cmd, " "); + for (; *args != NULL; args++) { + if (strchr(*args, '%') == NULL) + value = *args; + else { + /* some automation: if parameter's all %variables + expand to empty, but the %variable isn't the only + text in the parameter, skip it. */ + str_truncate(str, 0); + str_truncate(str2, 0); + var_expand(str, *args, tab); + var_expand(str2, *args, static_tab); + if (strcmp(str_c(str), str_c(str2)) == 0 && + str_len(str) > 0) + continue; + value = t_strdup(str_c(str)); + } + array_append(&cmd_args, &value, 1); + } + (void)array_append_space(&cmd_args); + return array_idx(&cmd_args, 0); +} + static bool mirror_get_remote_cmd(struct dsync_cmd_context *ctx, - const char *const *argv, const char *user, + const char *user, const char *const **cmd_args_r) { - ARRAY_TYPE(const_string) cmd_args; - const char *p, *host; + const char *p, *host, *const *argv = ctx->ctx.args; if (argv[1] != NULL) { /* more than one parameter, so it contains a full command @@ -170,21 +220,7 @@ /* we'll assume virtual users, so in user at host it really means not to give ssh a username, but to give dsync -u user parameter. */ - t_array_init(&cmd_args, 8); - array_append(&cmd_args, &ssh_cmd, 1); - array_append(&cmd_args, &host, 1); - p = "doveadm"; array_append(&cmd_args, &p, 1); - p = "dsync-server"; array_append(&cmd_args, &p, 1); - if (ctx->lock) { - p = "-l"; array_append(&cmd_args, &p, 1); - p = dec2str(ctx->lock_timeout); array_append(&cmd_args, &p, 1); - } - if (*user != '\0') { - p = "-u"; array_append(&cmd_args, &p, 1); - array_append(&cmd_args, &user, 1); - } - (void)array_append_space(&cmd_args); - *cmd_args_r = array_idx(&cmd_args, 0); + *cmd_args_r = get_ssh_cmd_args(ctx, host, "", user); return TRUE; } @@ -241,6 +277,22 @@ return dsync_worker_init_proxy_client(ctx->fd_in, ctx->fd_out); } +static const char *const * +parse_ssh_location(struct dsync_cmd_context *ctx, + const char *location, const char *username) +{ + const char *host, *login; + + host = strchr(location, '@'); + if (host != NULL) + login = t_strdup_until(location, host++); + else { + host = location; + login = ""; + } + return get_ssh_cmd_args(ctx, host, login, username); +} + static int dsync_lock(struct mail_user *user, unsigned int lock_timeout, const char **path_r, struct file_lock **lock_r) { @@ -282,7 +334,7 @@ /* create and run the brain */ brain = dsync_brain_init(worker1, worker2, ctx->mailbox, ctx->brain_flags); - if (ctx->remote_cmd_args == NULL) + if (!ctx->remote) dsync_brain_sync_all(brain); else { dsync_brain_sync(brain); @@ -315,7 +367,7 @@ /* create workers */ worker1 = dsync_worker_init_local(user, *_ctx->set->dsync_alt_char); - if (ctx->remote_cmd_args == NULL) + if (!ctx->remote) worker2 = cmd_dsync_run_local(ctx, user); else worker2 = cmd_dsync_run_remote(ctx, user); @@ -350,15 +402,54 @@ return ret; } +static int cmd_dsync_prerun(struct doveadm_mail_cmd_context *_ctx, + struct mail_storage_service_user *service_user, + const char **error_r ATTR_UNUSED) +{ + struct dsync_cmd_context *ctx = (struct dsync_cmd_context *)_ctx; + const char *const *remote_cmd_args = NULL; + const struct mail_user_settings *user_set; + const char *username = ""; + + user_set = mail_storage_service_user_get_set(service_user)[0]; + + ctx->fd_in = STDIN_FILENO; + ctx->fd_out = STDOUT_FILENO; + ctx->fd_err = -1; + ctx->remote = FALSE; + + /* if we're executing remotely, give -u parameter if we also + did a userdb lookup. */ + if ((_ctx->service_flags & MAIL_STORAGE_SERVICE_FLAG_USERDB_LOOKUP) != 0) + username = _ctx->cur_username; + + if (!mirror_get_remote_cmd(ctx, username, &remote_cmd_args)) { + /* it's a mail_location */ + if (_ctx->args[1] != NULL) + doveadm_mail_help_name(_ctx->cmd->name); + ctx->local_location = _ctx->args[0]; + } + + if (remote_cmd_args == NULL && ctx->local_location != NULL && + strncmp(ctx->local_location, "remote:", 7) == 0) { + /* this is a remote (ssh) command */ + remote_cmd_args = parse_ssh_location(ctx, ctx->local_location+7, + _ctx->cur_username); + } + + if (remote_cmd_args != NULL) { + /* do this before mail_storage_service_next() in case it + drops process privileges */ + run_cmd(ctx, remote_cmd_args); + ctx->remote = TRUE; + } + return 0; +} + static void cmd_dsync_init(struct doveadm_mail_cmd_context *_ctx, const char *const args[]) { struct dsync_cmd_context *ctx = (struct dsync_cmd_context *)_ctx; - const char *username = ""; - - ctx->fd_in = STDIN_FILENO; - ctx->fd_out = STDOUT_FILENO; - ctx->fd_err = -1; if (args[0] == NULL) doveadm_mail_help_name(_ctx->cmd->name); @@ -367,25 +458,6 @@ if (doveadm_debug || doveadm_verbose) ctx->brain_flags |= DSYNC_BRAIN_FLAG_VERBOSE; - - /* if we're executing remotely, give -u parameter if we also - did a userdb lookup. this works only when we're handling a - single user */ - if ((_ctx->service_flags & MAIL_STORAGE_SERVICE_FLAG_USERDB_LOOKUP) != 0 && - _ctx->cur_username != NULL) - username = _ctx->cur_username; - if (!mirror_get_remote_cmd(ctx, args, username, &ctx->remote_cmd_args)) { - /* it's a mail_location */ - if (args[1] != NULL) - doveadm_mail_help_name(_ctx->cmd->name); - ctx->local_location = args[0]; - } - - if (ctx->remote_cmd_args != NULL) { - /* do this before mail_storage_service_next() in case it - drops process privileges */ - run_cmd(ctx, ctx->remote_cmd_args); - } } static void cmd_dsync_preinit(struct doveadm_mail_cmd_context *ctx) @@ -433,6 +505,7 @@ ctx->ctx.v.parse_arg = cmd_mailbox_dsync_parse_arg; ctx->ctx.v.preinit = cmd_dsync_preinit; ctx->ctx.v.init = cmd_dsync_init; + ctx->ctx.v.prerun = cmd_dsync_prerun; ctx->ctx.v.run = cmd_dsync_run; return &ctx->ctx; } From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: dsync: Added -d parameter to sync to plugin/mail_re... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/dc0038843cc7 changeset: 14252:dc0038843cc7 user: Timo Sirainen date: Fri Mar 02 14:25:49 2012 +0200 description: dsync: Added -d parameter to sync to plugin/mail_replica destination. This could be one global default or overridden by userdb. diffstat: src/doveadm/dsync/doveadm-dsync.c | 50 +++++++++++++++++++++++++++----------- 1 files changed, 35 insertions(+), 15 deletions(-) diffs (106 lines): diff -r 69ed88beb12f -r dc0038843cc7 src/doveadm/dsync/doveadm-dsync.c --- a/src/doveadm/dsync/doveadm-dsync.c Fri Mar 02 14:18:29 2012 +0200 +++ b/src/doveadm/dsync/doveadm-dsync.c Fri Mar 02 14:25:49 2012 +0200 @@ -38,6 +38,7 @@ unsigned int lock_timeout; unsigned int lock:1; + unsigned int default_replica_location:1; unsigned int reverse_workers:1; unsigned int remote:1; }; @@ -404,7 +405,7 @@ static int cmd_dsync_prerun(struct doveadm_mail_cmd_context *_ctx, struct mail_storage_service_user *service_user, - const char **error_r ATTR_UNUSED) + const char **error_r) { struct dsync_cmd_context *ctx = (struct dsync_cmd_context *)_ctx; const char *const *remote_cmd_args = NULL; @@ -418,16 +419,27 @@ ctx->fd_err = -1; ctx->remote = FALSE; - /* if we're executing remotely, give -u parameter if we also - did a userdb lookup. */ - if ((_ctx->service_flags & MAIL_STORAGE_SERVICE_FLAG_USERDB_LOOKUP) != 0) - username = _ctx->cur_username; + if (ctx->default_replica_location) { + ctx->local_location = + mail_user_set_plugin_getenv(user_set, "mail_replica"); + if (ctx->local_location == NULL || + *ctx->local_location == '\0') { + *error_r = "User has no mail_replica in userdb"; + _ctx->exit_code = DOVEADM_EX_NOTFOUND; + return -1; + } + } else { + /* if we're executing remotely, give -u parameter if we also + did a userdb lookup. */ + if ((_ctx->service_flags & MAIL_STORAGE_SERVICE_FLAG_USERDB_LOOKUP) != 0) + username = _ctx->cur_username; - if (!mirror_get_remote_cmd(ctx, username, &remote_cmd_args)) { - /* it's a mail_location */ - if (_ctx->args[1] != NULL) - doveadm_mail_help_name(_ctx->cmd->name); - ctx->local_location = _ctx->args[0]; + if (!mirror_get_remote_cmd(ctx, username, &remote_cmd_args)) { + /* it's a mail_location */ + if (_ctx->args[1] != NULL) + doveadm_mail_help_name(_ctx->cmd->name); + ctx->local_location = _ctx->args[0]; + } } if (remote_cmd_args == NULL && ctx->local_location != NULL && @@ -451,8 +463,13 @@ { struct dsync_cmd_context *ctx = (struct dsync_cmd_context *)_ctx; - if (args[0] == NULL) - doveadm_mail_help_name(_ctx->cmd->name); + if (ctx->default_replica_location) { + if (args[0] != NULL) + i_error("Don't give mail location with -d parameter"); + } else { + if (args[0] == NULL) + doveadm_mail_help_name(_ctx->cmd->name); + } lib_signals_ignore(SIGHUP, TRUE); @@ -472,6 +489,9 @@ struct dsync_cmd_context *ctx = (struct dsync_cmd_context *)_ctx; switch (c) { + case 'd': + ctx->default_replica_location = TRUE; + break; case 'E': /* dsync wrapper detection flag */ legacy_dsync = TRUE; @@ -501,7 +521,7 @@ struct dsync_cmd_context *ctx; ctx = doveadm_mail_cmd_alloc(struct dsync_cmd_context); - ctx->ctx.getopt_args = "+Efl:m:R"; + ctx->ctx.getopt_args = "+dEfl:m:R"; ctx->ctx.v.parse_arg = cmd_mailbox_dsync_parse_arg; ctx->ctx.v.preinit = cmd_dsync_preinit; ctx->ctx.v.init = cmd_dsync_init; @@ -592,11 +612,11 @@ } struct doveadm_mail_cmd cmd_dsync_mirror = { - cmd_dsync_alloc, "sync", "[-fR] [-l ] [-m ] " + cmd_dsync_alloc, "sync", "[-dfR] [-l ] [-m ] " }; struct doveadm_mail_cmd cmd_dsync_backup = { cmd_dsync_backup_alloc, "backup", - "[-fR] [-l ] [-m ] " + "[-dfR] [-l ] [-m ] " }; struct doveadm_mail_cmd cmd_dsync_server = { cmd_dsync_server_alloc, "dsync-server", &doveadm_mail_cmd_hide From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: Increased initial memory pool sizes and marked some... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/91e035840dc6 changeset: 14253:91e035840dc6 user: Timo Sirainen date: Fri Mar 02 14:33:41 2012 +0200 description: Increased initial memory pool sizes and marked some of them as "growing". diffstat: src/config/config-request.c | 2 +- src/doveadm/dsync/dsync-brain.c | 6 ++++-- src/doveadm/dsync/dsync-proxy-server.c | 2 +- src/lib-settings/settings-parser.c | 6 ++++-- 4 files changed, 10 insertions(+), 6 deletions(-) diffs (70 lines): diff -r dc0038843cc7 -r 91e035840dc6 src/config/config-request.c --- a/src/config/config-request.c Fri Mar 02 14:25:49 2012 +0200 +++ b/src/config/config-request.c Fri Mar 02 14:33:41 2012 +0200 @@ -350,7 +350,7 @@ i_assert(module != NULL); - pool = pool_alloconly_create("config export", 1024*64); + pool = pool_alloconly_create(MEMPOOL_GROWING"config export", 1024*64); ctx = p_new(pool, struct config_export_context, 1); ctx->pool = pool; diff -r dc0038843cc7 -r 91e035840dc6 src/doveadm/dsync/dsync-brain.c --- a/src/doveadm/dsync/dsync-brain.c Fri Mar 02 14:25:49 2012 +0200 +++ b/src/doveadm/dsync/dsync-brain.c Fri Mar 02 14:33:41 2012 +0200 @@ -230,7 +230,8 @@ struct dsync_brain_subs_list *list; pool_t pool; - pool = pool_alloconly_create("dsync brain subs list", 1024*4); + pool = pool_alloconly_create(MEMPOOL_GROWING"dsync brain subs list", + 1024*4); list = p_new(pool, struct dsync_brain_subs_list, 1); list->pool = pool; list->brain = brain; @@ -724,7 +725,8 @@ pool_t pool; bool ret; - pool = pool_alloconly_create("dsync changed mailboxes", 10240); + pool = pool_alloconly_create(MEMPOOL_GROWING"dsync changed mailboxes", + 10240); p_array_init(&mailboxes, pool, 128); dsync_brain_get_changed_mailboxes(brain, &mailboxes, (brain->flags & DSYNC_BRAIN_FLAG_FULL_SYNC) != 0); diff -r dc0038843cc7 -r 91e035840dc6 src/doveadm/dsync/dsync-proxy-server.c --- a/src/doveadm/dsync/dsync-proxy-server.c Fri Mar 02 14:25:49 2012 +0200 +++ b/src/doveadm/dsync/dsync-proxy-server.c Fri Mar 02 14:33:41 2012 +0200 @@ -167,7 +167,7 @@ server->fd_in = fd_in; server->fd_out = fd_out; - server->cmd_pool = pool_alloconly_create("worker server cmd", 1024); + server->cmd_pool = pool_alloconly_create("worker server cmd", 2048); server->io = io_add(fd_in, IO_READ, proxy_server_input, server); server->input = i_stream_create_fd(fd_in, (size_t)-1, FALSE); server->output = o_stream_create_fd(fd_out, (size_t)-1, FALSE); diff -r dc0038843cc7 -r 91e035840dc6 src/lib-settings/settings-parser.c --- a/src/lib-settings/settings-parser.c Fri Mar 02 14:25:49 2012 +0200 +++ b/src/lib-settings/settings-parser.c Fri Mar 02 14:33:41 2012 +0200 @@ -200,7 +200,8 @@ i_assert(count > 0); - parser_pool = pool_alloconly_create("settings parser", 16384); + parser_pool = pool_alloconly_create(MEMPOOL_GROWING"settings parser", + 8192); ctx = p_new(parser_pool, struct setting_parser_context, 1); ctx->set_pool = set_pool; ctx->parser_pool = parser_pool; @@ -1720,7 +1721,8 @@ pool_t parser_pool; pool_ref(new_pool); - parser_pool = pool_alloconly_create("dup settings parser", 8192); + parser_pool = pool_alloconly_create(MEMPOOL_GROWING"dup settings parser", + 8192); new_ctx = p_new(parser_pool, struct setting_parser_context, 1); new_ctx->set_pool = new_pool; new_ctx->parser_pool = parser_pool; From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: lib-storage: mail_transaction_commit_changes.change... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/3e9f91d8b2af changeset: 14254:3e9f91d8b2af user: Timo Sirainen date: Fri Mar 02 14:36:13 2012 +0200 description: lib-storage: mail_transaction_commit_changes.changed=TRUE only when something actually changed. diffstat: src/lib-storage/index/index-transaction.c | 8 ++++++++ src/lib-storage/index/maildir/maildir-mail.c | 1 + src/lib-storage/mail-storage-private.h | 2 ++ src/lib-storage/mail-storage.h | 3 +++ 4 files changed, 14 insertions(+), 0 deletions(-) diffs (65 lines): diff -r 91e035840dc6 -r 3e9f91d8b2af src/lib-storage/index/index-transaction.c --- a/src/lib-storage/index/index-transaction.c Fri Mar 02 14:33:41 2012 +0200 +++ b/src/lib-storage/index/index-transaction.c Fri Mar 02 14:36:13 2012 +0200 @@ -21,10 +21,15 @@ MAIL_STORAGE_CONTEXT(index_trans); int ret = 0; + if (t->nontransactional_changes) + t->changes->changed = TRUE; + if (t->save_ctx != NULL) { if (t->box->v.transaction_save_commit_pre(t->save_ctx) < 0) { t->save_ctx = NULL; ret = -1; + } else { + t->changes->changed = TRUE; } } @@ -35,6 +40,9 @@ if (t->super.commit(index_trans, result_r) < 0) { mail_storage_set_index_error(t->box); ret = -1; + } else if (result_r->commit_size > 0) { + /* something was written to the transaction log */ + t->changes->changed = TRUE; } } diff -r 91e035840dc6 -r 3e9f91d8b2af src/lib-storage/index/maildir/maildir-mail.c --- a/src/lib-storage/index/maildir/maildir-mail.c Fri Mar 02 14:33:41 2012 +0200 +++ b/src/lib-storage/index/maildir/maildir-mail.c Fri Mar 02 14:36:13 2012 +0200 @@ -593,6 +593,7 @@ uidl = ""; } + _mail->transaction->nontransactional_changes = TRUE; maildir_uidlist_set_ext(mbox->uidlist, _mail->uid, MAILDIR_UIDLIST_REC_EXT_POP3_UIDL, uidl); } diff -r 91e035840dc6 -r 3e9f91d8b2af src/lib-storage/mail-storage-private.h --- a/src/lib-storage/mail-storage-private.h Fri Mar 02 14:33:41 2012 +0200 +++ b/src/lib-storage/mail-storage-private.h Fri Mar 02 14:36:13 2012 +0200 @@ -395,6 +395,8 @@ struct mailbox_transaction_stats stats; /* Set to TRUE to update stats_* fields */ unsigned int stats_track:1; + /* We've done some non-transactional (e.g. dovecot-uidlist updates) */ + unsigned int nontransactional_changes:1; }; union mail_search_module_context { diff -r 91e035840dc6 -r 3e9f91d8b2af src/lib-storage/mail-storage.h --- a/src/lib-storage/mail-storage.h Fri Mar 02 14:33:41 2012 +0200 +++ b/src/lib-storage/mail-storage.h Fri Mar 02 14:36:13 2012 +0200 @@ -265,6 +265,9 @@ /* number of modseq changes that couldn't be changed as requested */ unsigned int ignored_modseq_changes; + + /* TRUE if anything actually changed with this commit */ + bool changed; }; struct mailbox_sync_rec { From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: eacces_error_get*(): Added ',' to message to improv... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/095abcfab7f1 changeset: 14256:095abcfab7f1 user: Timo Sirainen date: Fri Mar 02 15:34:22 2012 +0200 description: eacces_error_get*(): Added ',' to message to improve readability. diffstat: src/lib/eacces-error.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diffs (27 lines): diff -r d247d53f80b9 -r 095abcfab7f1 src/lib/eacces-error.c --- a/src/lib/eacces-error.c Fri Mar 02 15:31:25 2012 +0200 +++ b/src/lib/eacces-error.c Fri Mar 02 15:34:22 2012 +0200 @@ -113,12 +113,12 @@ write_eacces_error(errmsg, path, access_mode); if (test_manual_access(path, access_mode, FALSE, errmsg) == 0) { - str_append(errmsg, " UNIX perms appear ok " + str_append(errmsg, ", UNIX perms appear ok " "(ACL/MAC wrong?)"); } errno = EACCES; } else { - str_printfa(errmsg, " access(%s, %d) failed: %m", + str_printfa(errmsg, ", access(%s, %d) failed: %m", path, access_mode); } return -1; @@ -135,7 +135,7 @@ if (errno == EACCES) write_eacces_error(errmsg, path, access_mode); else - str_printfa(errmsg, " stat(%s/test) failed: %m", path); + str_printfa(errmsg, ", stat(%s/test) failed: %m", path); return -1; case R_OK: case W_OK: From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: eacces_error_get*(): Suggest ACL/MAC error if UNIX ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d247d53f80b9 changeset: 14255:d247d53f80b9 user: Timo Sirainen date: Fri Mar 02 15:31:25 2012 +0200 description: eacces_error_get*(): Suggest ACL/MAC error if UNIX permission bits look ok. diffstat: src/lib/eacces-error.c | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diffs (20 lines): diff -r 3e9f91d8b2af -r d247d53f80b9 src/lib/eacces-error.c --- a/src/lib/eacces-error.c Fri Mar 02 14:36:13 2012 +0200 +++ b/src/lib/eacces-error.c Fri Mar 02 15:31:25 2012 +0200 @@ -109,11 +109,13 @@ if (getuid() == geteuid()) { if (access(path, access_mode) == 0) return 0; - if (errno == EACCES) { write_eacces_error(errmsg, path, access_mode); - (void)test_manual_access(path, access_mode, - FALSE, errmsg); + if (test_manual_access(path, access_mode, + FALSE, errmsg) == 0) { + str_append(errmsg, " UNIX perms appear ok " + "(ACL/MAC wrong?)"); + } errno = EACCES; } else { str_printfa(errmsg, " access(%s, %d) failed: %m", From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: login proxy: Don't assume host is down unless last ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/53b23557ec54 changeset: 14258:53b23557ec54 user: Timo Sirainen date: Fri Mar 02 17:58:37 2012 +0200 description: login proxy: Don't assume host is down unless last success was >30s before last failure. This avoids thinking that host is down, simply because one connection for it failed. diffstat: src/login-common/login-proxy.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (19 lines): diff -r e86198d749ec -r 53b23557ec54 src/login-common/login-proxy.c --- a/src/login-common/login-proxy.c Fri Mar 02 15:49:34 2012 +0200 +++ b/src/login-common/login-proxy.c Fri Mar 02 17:58:37 2012 +0200 @@ -23,6 +23,7 @@ #define LOGIN_PROXY_IPC_PATH "ipc-proxy" #define LOGIN_PROXY_IPC_NAME "proxy" #define KILLED_BY_ADMIN_REASON "Killed by admin" +#define PROXY_IMMEDIATE_FAILURE_SECS 30 struct login_proxy { struct login_proxy *prev, *next; @@ -240,6 +241,7 @@ rec = login_proxy_state_get(proxy_state, &proxy->ip, proxy->port); if (timeval_cmp(&rec->last_failure, &rec->last_success) > 0 && + rec->last_failure.tv_sec - rec->last_success.tv_sec > PROXY_IMMEDIATE_FAILURE_SECS && rec->num_waiting_connections != 0) { /* the server is down. fail immediately */ i_error("proxy(%s): Host %s:%u is down", From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: dsync: Added -n parameter to dsync a specific names... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/e86198d749ec changeset: 14257:e86198d749ec user: Timo Sirainen date: Fri Mar 02 15:49:34 2012 +0200 description: dsync: Added -n parameter to dsync a specific namespace. diffstat: src/doveadm/doveadm-settings.c | 2 +- src/doveadm/dsync/doveadm-dsync.c | 28 ++++++++++++++++++------- src/doveadm/dsync/dsync-worker-local.c | 37 ++++++++++++++++++++++++--------- src/doveadm/dsync/dsync-worker.h | 3 +- 4 files changed, 50 insertions(+), 20 deletions(-) diffs (238 lines): diff -r 095abcfab7f1 -r e86198d749ec src/doveadm/doveadm-settings.c --- a/src/doveadm/doveadm-settings.c Fri Mar 02 15:34:22 2012 +0200 +++ b/src/doveadm/doveadm-settings.c Fri Mar 02 15:49:34 2012 +0200 @@ -78,7 +78,7 @@ .doveadm_password = "", .doveadm_allowed_commands = "", .dsync_alt_char = "_", - .dsync_remote_cmd = "ssh -l%{login} %{host} doveadm dsync-server -u%u -l%{lock_timeout}", + .dsync_remote_cmd = "ssh -l%{login} %{host} doveadm dsync-server -u%u -l%{lock_timeout} -n%{namespace}", .plugin_envs = ARRAY_INIT }; diff -r 095abcfab7f1 -r e86198d749ec src/doveadm/dsync/doveadm-dsync.c --- a/src/doveadm/dsync/doveadm-dsync.c Fri Mar 02 15:34:22 2012 +0200 +++ b/src/doveadm/dsync/doveadm-dsync.c Fri Mar 02 15:49:34 2012 +0200 @@ -28,7 +28,7 @@ struct dsync_cmd_context { struct doveadm_mail_cmd_context ctx; enum dsync_brain_flags brain_flags; - const char *mailbox; + const char *mailbox, *namespace_prefix; const char *local_location; @@ -139,6 +139,7 @@ { '\0', NULL, "login" }, { '\0', NULL, "host" }, { '\0', NULL, "lock_timeout" }, + { '\0', NULL, "namespace" }, { '\0', NULL, NULL } }; struct var_expand_table *tab; @@ -153,6 +154,7 @@ tab[1].value = login; tab[2].value = host; tab[3].value = dec2str(ctx->lock_timeout); + tab[4].value = ctx->namespace_prefix; t_array_init(&cmd_args, 8); str = t_str_new(128); @@ -265,7 +267,8 @@ "points to same directory: %s", path1); } - worker2 = dsync_worker_init_local(user2, *ctx->ctx.set->dsync_alt_char); + worker2 = dsync_worker_init_local(user2, ctx->namespace_prefix, + *ctx->ctx.set->dsync_alt_char); mail_user_unref(&user2); return worker2; } @@ -367,7 +370,8 @@ user->admin = TRUE; /* create workers */ - worker1 = dsync_worker_init_local(user, *_ctx->set->dsync_alt_char); + worker1 = dsync_worker_init_local(user, ctx->namespace_prefix, + *_ctx->set->dsync_alt_char); if (!ctx->remote) worker2 = cmd_dsync_run_local(ctx, user); else @@ -507,6 +511,9 @@ case 'm': ctx->mailbox = optarg; break; + case 'n': + ctx->namespace_prefix = optarg; + break; case 'R': ctx->reverse_workers = TRUE; break; @@ -521,7 +528,7 @@ struct dsync_cmd_context *ctx; ctx = doveadm_mail_cmd_alloc(struct dsync_cmd_context); - ctx->ctx.getopt_args = "+dEfl:m:R"; + ctx->ctx.getopt_args = "+dEfl:m:n:R"; ctx->ctx.v.parse_arg = cmd_mailbox_dsync_parse_arg; ctx->ctx.v.preinit = cmd_dsync_preinit; ctx->ctx.v.init = cmd_dsync_init; @@ -556,7 +563,8 @@ i_set_failure_prefix(t_strdup_printf("dsync-remote(%s): ", user->username)); - worker = dsync_worker_init_local(user, *_ctx->set->dsync_alt_char); + worker = dsync_worker_init_local(user, ctx->namespace_prefix, + *_ctx->set->dsync_alt_char); server = dsync_proxy_server_init(STDIN_FILENO, STDOUT_FILENO, worker); if (!ctx->lock) @@ -594,6 +602,9 @@ if (str_to_uint(optarg, &ctx->lock_timeout) < 0) i_error("Invalid -l parameter: %s", optarg); break; + case 'n': + ctx->namespace_prefix = optarg; + break; default: return FALSE; } @@ -605,18 +616,19 @@ struct dsync_cmd_context *ctx; ctx = doveadm_mail_cmd_alloc(struct dsync_cmd_context); - ctx->ctx.getopt_args = "El:"; + ctx->ctx.getopt_args = "El:n:"; ctx->ctx.v.parse_arg = cmd_mailbox_dsync_server_parse_arg; ctx->ctx.v.run = cmd_dsync_server_run; return &ctx->ctx; } struct doveadm_mail_cmd cmd_dsync_mirror = { - cmd_dsync_alloc, "sync", "[-dfR] [-l ] [-m ] " + cmd_dsync_alloc, "sync", + "[-dfR] [-l ] [-m ] [-n ] " }; struct doveadm_mail_cmd cmd_dsync_backup = { cmd_dsync_backup_alloc, "backup", - "[-dfR] [-l ] [-m ] " + "[-dfR] [-l ] [-m ] [-n ] " }; struct doveadm_mail_cmd cmd_dsync_server = { cmd_dsync_server_alloc, "dsync-server", &doveadm_mail_cmd_hide diff -r 095abcfab7f1 -r e86198d749ec src/doveadm/dsync/dsync-worker-local.c --- a/src/doveadm/dsync/dsync-worker-local.c Fri Mar 02 15:34:22 2012 +0200 +++ b/src/doveadm/dsync/dsync-worker-local.c Fri Mar 02 15:49:34 2012 +0200 @@ -94,7 +94,7 @@ struct hash_table *dir_changes_hash; char alt_char; - ARRAY_DEFINE(wanted_namespaces, struct mail_namespace *); + const char *namespace_prefix; mailbox_guid_t selected_box_guid; struct mailbox *selected_box; @@ -151,18 +151,33 @@ return h; } -static bool local_worker_want_namespace(struct mail_namespace *ns) +static bool local_worker_want_namespace(struct local_dsync_worker *worker, + struct mail_namespace *ns) { - return strcmp(ns->unexpanded_set->location, - SETTING_STRVAR_UNEXPANDED) == 0; + if (worker->namespace_prefix == NULL) { + return strcmp(ns->unexpanded_set->location, + SETTING_STRVAR_UNEXPANDED) == 0; + } else { + return strcmp(ns->prefix, worker->namespace_prefix) == 0; + } } static void dsync_check_namespaces(struct local_dsync_worker *worker) { struct mail_namespace *ns; + if (worker->namespace_prefix != NULL) { + ns = mail_namespace_find_prefix(worker->user->namespaces, + worker->namespace_prefix); + if (ns == NULL) { + i_fatal("Namespace prefix '%s' not found", + worker->namespace_prefix); + } + return; + } + for (ns = worker->user->namespaces; ns != NULL; ns = ns->next) { - if (local_worker_want_namespace(ns)) + if (local_worker_want_namespace(worker, ns)) return; } i_fatal("All your namespaces have a location setting. " @@ -171,7 +186,8 @@ } struct dsync_worker * -dsync_worker_init_local(struct mail_user *user, char alt_char) +dsync_worker_init_local(struct mail_user *user, const char *namespace_prefix, + char alt_char) { struct local_dsync_worker *worker; pool_t pool; @@ -181,13 +197,13 @@ worker->worker.v = local_dsync_worker; worker->user = user; worker->pool = pool; + worker->namespace_prefix = p_strdup(pool, namespace_prefix); worker->alt_char = alt_char; worker->mailbox_hash = hash_table_create(default_pool, pool, 0, mailbox_guid_hash, mailbox_guid_cmp); i_array_init(&worker->saved_uids, 128); i_array_init(&worker->msg_get_queue, 32); - p_array_init(&worker->wanted_namespaces, pool, 8); dsync_check_namespaces(worker); mail_user_ref(worker->user); @@ -382,7 +398,8 @@ hash_table_create(default_pool, worker->pool, 0, dir_change_hash, dir_change_cmp); for (ns = worker->user->namespaces; ns != NULL; ns = ns->next) { - if (ns->alias_for != NULL || !local_worker_want_namespace(ns)) + if (ns->alias_for != NULL || + !local_worker_want_namespace(worker, ns)) continue; if (dsync_worker_get_list_mailbox_log(worker, ns->list) < 0) @@ -503,7 +520,7 @@ memset(dsync_box_r, 0, sizeof(*dsync_box_r)); while ((info = mailbox_list_iter_next(iter->list_iter)) != NULL) { - if (local_worker_want_namespace(info->ns)) + if (local_worker_want_namespace(worker, info->ns)) break; } if (info == NULL) @@ -646,7 +663,7 @@ memset(rec_r, 0, sizeof(*rec_r)); while ((info = mailbox_list_iter_next(iter->list_iter)) != NULL) { - if (local_worker_want_namespace(info->ns) || + if (local_worker_want_namespace(worker, info->ns) || (info->ns->flags & NAMESPACE_FLAG_SUBSCRIPTIONS) == 0) break; } diff -r 095abcfab7f1 -r e86198d749ec src/doveadm/dsync/dsync-worker.h --- a/src/doveadm/dsync/dsync-worker.h Fri Mar 02 15:34:22 2012 +0200 +++ b/src/doveadm/dsync/dsync-worker.h Fri Mar 02 15:49:34 2012 +0200 @@ -30,7 +30,8 @@ typedef void dsync_worker_finish_callback_t(bool success, void *context); struct dsync_worker * -dsync_worker_init_local(struct mail_user *user, char alt_char); +dsync_worker_init_local(struct mail_user *user, const char *namespace_prefix, + char alt_char); struct dsync_worker *dsync_worker_init_proxy_client(int fd_in, int fd_out); void dsync_worker_deinit(struct dsync_worker **worker); From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: lib-sql: If failed query is retried, log it as as w... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/1002733ca266 changeset: 14259:1002733ca266 user: Timo Sirainen date: Fri Mar 02 19:24:57 2012 +0200 description: lib-sql: If failed query is retried, log it as as warning instead of as error. diffstat: src/lib-sql/driver-sqlpool.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff -r 53b23557ec54 -r 1002733ca266 src/lib-sql/driver-sqlpool.c --- a/src/lib-sql/driver-sqlpool.c Fri Mar 02 17:58:37 2012 +0200 +++ b/src/lib-sql/driver-sqlpool.c Fri Mar 02 19:24:57 2012 +0200 @@ -614,8 +614,8 @@ if (result->failed_try_retry && request->retry_count < array_count(&db->hosts)) { - i_error("%s: Query failed, retrying: %s", - db->driver->name, sql_result_get_error(result)); + i_warning("%s: Query failed, retrying: %s", + db->driver->name, sql_result_get_error(result)); request->retry_count++; driver_sqlpool_prepend_request(db, request); From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: Makefile: Fixed compiling with automake 1.11.2+ Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/a452e5f616a2 changeset: 14260:a452e5f616a2 user: Timo Sirainen date: Sun Mar 04 09:39:45 2012 +0200 description: Makefile: Fixed compiling with automake 1.11.2+ diffstat: Makefile.am | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diffs (21 lines): diff -r 1002733ca266 -r a452e5f616a2 Makefile.am --- a/Makefile.am Fri Mar 02 19:24:57 2012 +0200 +++ b/Makefile.am Sun Mar 04 09:39:45 2012 +0200 @@ -1,5 +1,8 @@ aclocaldir = $(datadir)/aclocal +# automake 1.11.2+ doesn't like allow using pkglib_DATA, but we really it. +dovecotlibdir = $(libdir)/dovecot + SUBDIRS = \ . \ src \ @@ -21,7 +24,7 @@ dovecot-config nodist_pkginclude_HEADERS = config.h -pkglib_DATA = $(datafiles) +dovecotlib_DATA = $(datafiles) if MAINTAINER_MODE ChangeLog: .hg/dirstate From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: Initial implementation of dsync-based replication. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/14ff849dc266 changeset: 14261:14ff849dc266 user: Timo Sirainen date: Sun Mar 04 09:50:21 2012 +0200 description: Initial implementation of dsync-based replication. diffstat: .hgignore | 2 + configure.in | 4 + src/Makefile.am | 1 + src/doveadm/dsync/doveadm-dsync.c | 1 + src/lib-storage/mail-user.h | 2 + src/plugins/replication/Makefile.am | 25 + src/plugins/replication/replication-plugin.c | 353 ++++++++++++++++++++ src/plugins/replication/replication-plugin.h | 9 + src/replication/Makefile.am | 4 + src/replication/aggregator/Makefile.am | 26 + src/replication/aggregator/aggregator-settings.c | 85 ++++ src/replication/aggregator/aggregator-settings.h | 12 + src/replication/aggregator/aggregator.c | 75 ++++ src/replication/aggregator/notify-connection.c | 154 ++++++++ src/replication/aggregator/notify-connection.h | 9 + src/replication/aggregator/replicator-connection.c | 321 ++++++++++++++++++ src/replication/aggregator/replicator-connection.h | 25 + src/replication/replication-common.h | 30 + src/replication/replicator/Makefile.am | 30 + src/replication/replicator/doveadm-connection.c | 194 +++++++++++ src/replication/replicator/doveadm-connection.h | 20 + src/replication/replicator/notify-connection.c | 197 +++++++++++ src/replication/replicator/notify-connection.h | 13 + src/replication/replicator/replicator-brain.c | 164 +++++++++ src/replication/replicator/replicator-brain.h | 11 + src/replication/replicator/replicator-queue.c | 363 +++++++++++++++++++++ src/replication/replicator/replicator-queue.h | 60 +++ src/replication/replicator/replicator-settings.c | 80 ++++ src/replication/replicator/replicator-settings.h | 15 + src/replication/replicator/replicator.c | 117 ++++++ 30 files changed, 2402 insertions(+), 0 deletions(-) diffs (truncated from 2559 to 300 lines): diff -r a452e5f616a2 -r 14ff849dc266 .hgignore --- a/.hgignore Sun Mar 04 09:39:45 2012 +0200 +++ b/.hgignore Sun Mar 04 09:50:21 2012 +0200 @@ -85,6 +85,8 @@ src/plugins/fts-squat/squat-test src/pop3-login/pop3-login src/pop3/pop3 +src/replication/replicator/replicator +src/replication/aggregator/aggregator src/util/gdbhelper src/util/listview src/util/maildirlock diff -r a452e5f616a2 -r 14ff849dc266 configure.in --- a/configure.in Sun Mar 04 09:39:45 2012 +0200 +++ b/configure.in Sun Mar 04 09:50:21 2012 +0200 @@ -2785,6 +2785,9 @@ src/master/Makefile src/pop3/Makefile src/pop3-login/Makefile +src/replication/Makefile +src/replication/aggregator/Makefile +src/replication/replicator/Makefile src/ssl-params/Makefile src/stats/Makefile src/util/Makefile @@ -2803,6 +2806,7 @@ src/plugins/notify/Makefile src/plugins/quota/Makefile src/plugins/imap-quota/Makefile +src/plugins/replication/Makefile src/plugins/snarf/Makefile src/plugins/stats/Makefile src/plugins/imap-stats/Makefile diff -r a452e5f616a2 -r 14ff849dc266 src/Makefile.am --- a/src/Makefile.am Sun Mar 04 09:39:45 2012 +0200 +++ b/src/Makefile.am Sun Mar 04 09:50:21 2012 +0200 @@ -39,6 +39,7 @@ log \ config \ director \ + replication \ util \ doveadm \ ssl-params \ diff -r a452e5f616a2 -r 14ff849dc266 src/doveadm/dsync/doveadm-dsync.c --- a/src/doveadm/dsync/doveadm-dsync.c Sun Mar 04 09:39:45 2012 +0200 +++ b/src/doveadm/dsync/doveadm-dsync.c Sun Mar 04 09:50:21 2012 +0200 @@ -368,6 +368,7 @@ int lock_fd, ret = 0; user->admin = TRUE; + user->dsyncing = TRUE; /* create workers */ worker1 = dsync_worker_init_local(user, ctx->namespace_prefix, diff -r a452e5f616a2 -r 14ff849dc266 src/lib-storage/mail-user.h --- a/src/lib-storage/mail-user.h Sun Mar 04 09:39:45 2012 +0200 +++ b/src/lib-storage/mail-user.h Sun Mar 04 09:50:21 2012 +0200 @@ -56,6 +56,8 @@ unsigned int inbox_open_error_logged:1; /* Fuzzy search works for this user (FTS enabled) */ unsigned int fuzzy_search:1; + /* We're running dsync */ + unsigned int dsyncing:1; }; struct mail_user_module_register { diff -r a452e5f616a2 -r 14ff849dc266 src/plugins/replication/Makefile.am --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/plugins/replication/Makefile.am Sun Mar 04 09:50:21 2012 +0200 @@ -0,0 +1,25 @@ +AM_CPPFLAGS = \ + -I$(top_srcdir)/src/lib \ + -I$(top_srcdir)/src/lib-mail \ + -I$(top_srcdir)/src/lib-imap \ + -I$(top_srcdir)/src/lib-index \ + -I$(top_srcdir)/src/lib-storage \ + -I$(top_srcdir)/src/replication \ + -I$(top_srcdir)/src/plugins/notify + +NOPLUGIN_LDFLAGS = +lib20_replication_plugin_la_LDFLAGS = -module -avoid-version + +module_LTLIBRARIES = \ + lib20_replication_plugin.la + +if DOVECOT_PLUGIN_DEPS +lib20_replication_plugin_la_LIBADD = \ + ../notify/lib15_notify_plugin.la +endif + +lib20_replication_plugin_la_SOURCES = \ + replication-plugin.c + +noinst_HEADERS = \ + replication-plugin.h diff -r a452e5f616a2 -r 14ff849dc266 src/plugins/replication/replication-plugin.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/plugins/replication/replication-plugin.c Sun Mar 04 09:50:21 2012 +0200 @@ -0,0 +1,353 @@ +/* Copyright (c) 2012 Dovecot authors, see the included COPYING file */ + +#include "lib.h" +#include "array.h" +#include "str.h" +#include "strescape.h" +#include "fd-set-nonblock.h" +#include "ioloop.h" +#include "network.h" +#include "write-full.h" +#include "mail-user.h" +#include "mail-storage-private.h" +#include "notify-plugin.h" +#include "replication-common.h" +#include "replication-plugin.h" + +#include + +#define REPLICATION_SOCKET_NAME "replication-notify" +#define REPLICATION_FIFO_NAME "replication-notify-fifo" +#define REPLICATION_NOTIFY_DELAY_MSECS 500 +#define REPLICATION_SYNC_TIMEOUT_SECS 10 + +#define REPLICATION_USER_CONTEXT(obj) \ + MODULE_CONTEXT(obj, replication_user_module) + +struct replication_user { + union mail_user_module_context module_ctx; + + const char *fifo_path; + const char *socket_path; + + int fifo_fd; + + struct timeout *to; + enum replication_priority priority; + unsigned int sync_secs; + + bool fifo_failed; +}; + +struct replication_mail_txn_context { + struct mail_user *user; + bool new_messages; +}; + +static MODULE_CONTEXT_DEFINE_INIT(replication_user_module, + &mail_user_module_register); + +static int +replication_fifo_notify(struct mail_user *user, + enum replication_priority priority) +{ + struct replication_user *ruser = REPLICATION_USER_CONTEXT(user); + string_t *str; + ssize_t ret; + + if (ruser->fifo_failed) + return -1; + if (ruser->fifo_fd == -1) { + ruser->fifo_fd = open(ruser->fifo_path, O_WRONLY); + if (ruser->fifo_fd == -1) { + i_error("open(%s) failed: %m", ruser->fifo_path); + ruser->fifo_failed = TRUE; + return -1; + } + fd_set_nonblock(ruser->fifo_fd, TRUE); + } + /* \t */ + str = t_str_new(256); + str_tabescape_write(str, user->username); + str_append_c(str, '\t'); + switch (priority) { + case REPLICATION_PRIORITY_NONE: + case REPLICATION_PRIORITY_SYNC: + i_unreached(); + case REPLICATION_PRIORITY_LOW: + str_append(str, "low"); + break; + case REPLICATION_PRIORITY_HIGH: + str_append(str, "high"); + break; + } + str_append_c(str, '\n'); + ret = write(ruser->fifo_fd, str_data(str), str_len(str)); + if (ret == 0) { + /* busy, try again later */ + return 0; + } + if (ret != (ssize_t)str_len(str)) { + if (ret < 0) + i_error("write(%s) failed: %m", ruser->fifo_path); + else { + i_error("write(%s) wrote partial data", + ruser->fifo_path); + } + if (close(ruser->fifo_fd) < 0) + i_error("close(%s) failed: %m", ruser->fifo_path); + ruser->fifo_fd = -1; + return -1; + } + return 1; +} + +static void replication_notify_now(struct mail_user *user) +{ + struct replication_user *ruser = REPLICATION_USER_CONTEXT(user); + int ret; + + i_assert(ruser->priority != REPLICATION_PRIORITY_NONE); + i_assert(ruser->priority != REPLICATION_PRIORITY_SYNC); + + if ((ret = replication_fifo_notify(user, ruser->priority)) < 0 && + !ruser->fifo_failed) { + /* retry once, in case replication server was restarted */ + ret = replication_fifo_notify(user, ruser->priority); + } + if (ret != 0) { + timeout_remove(&ruser->to); + ruser->priority = REPLICATION_PRIORITY_NONE; + } +} + +static int replication_notify_sync(struct mail_user *user) +{ + struct replication_user *ruser = REPLICATION_USER_CONTEXT(user); + string_t *str; + char buf[1024]; + int fd; + ssize_t ret; + + fd = net_connect_unix(ruser->socket_path); + if (fd == -1) { + i_error("net_connect_unix(%s) failed: %m", ruser->socket_path); + return -1; + } + net_set_nonblock(fd, FALSE); + + /* \t "sync" */ + str = t_str_new(256); + str_tabescape_write(str, user->username); + str_append(str, "\tsync\n"); + alarm(ruser->sync_secs); + if (write_full(fd, str_data(str), str_len(str)) < 0) { + i_error("write(%s) failed: %m", ruser->socket_path); + ret = -1; + } else { + /* + | - */ + ret = read(fd, buf, sizeof(buf)); + if (ret < 0) { + if (ret != EINTR) { + i_error("read(%s) failed: %m", + ruser->socket_path); + } else { + i_warning("replication(%s): Sync failure: " + "Timeout in %u secs", + user->username, ruser->sync_secs); + } + } else if (ret == 0) { + i_error("read(%s) failed: EOF", ruser->socket_path); + ret = -1; + } else if (buf[0] == '+') { + /* success */ + ret = 0; + } else if (buf[0] == '-') { + /* failure */ + if (buf[ret-1] == '\n') ret--; + i_warning("replication(%s): Sync failure: %s", + user->username, t_strndup(buf+1, ret-1)); + ret = -1; + } else { + i_warning("replication(%s): " + "Remote sent invalid input: %s", + user->username, t_strndup(buf, ret)); + } + } + alarm(0); + if (close(fd) < 0) + i_error("close(%s) failed: %m", ruser->socket_path); + return ret; +} + +static void replication_notify(struct mail_user *user, + enum replication_priority priority) +{ + struct replication_user *ruser = REPLICATION_USER_CONTEXT(user); + + if (user->dsyncing) { + /* we're running dsync, which means that the remote is telling + us about a change. don't trigger a replication back to it */ + return; + } + + if (priority == REPLICATION_PRIORITY_SYNC) { + if (replication_notify_sync(user) == 0) { + timeout_remove(&ruser->to); + ruser->priority = REPLICATION_PRIORITY_NONE; + return; + } + /* sync replication failed, try as "high" via fifo */ From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: doveadm server now returns unknown users with -NOUS... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/9693521aa153 changeset: 14262:9693521aa153 user: Timo Sirainen date: Sun Mar 04 10:11:55 2012 +0200 description: doveadm server now returns unknown users with -NOUSER error. diffstat: src/doveadm/client-connection.c | 57 ++++++++++++++++-------- src/doveadm/doveadm-mail.c | 24 ++++----- src/doveadm/doveadm-mail.h | 5 +- src/replication/replicator/doveadm-connection.c | 6 +- 4 files changed, 56 insertions(+), 36 deletions(-) diffs (211 lines): diff -r 14ff849dc266 -r 9693521aa153 src/doveadm/client-connection.c --- a/src/doveadm/client-connection.c Sun Mar 04 09:50:21 2012 +0200 +++ b/src/doveadm/client-connection.c Sun Mar 04 10:11:55 2012 +0200 @@ -35,22 +35,22 @@ unsigned int authenticated:1; }; -static bool -doveadm_mail_cmd_server(const char *cmd_name, - const struct doveadm_settings *set, - const struct mail_storage_service_input *input, - int argc, char *argv[]) +static struct doveadm_mail_cmd_context * +doveadm_mail_cmd_server_parse(const char *cmd_name, + const struct doveadm_settings *set, + const struct mail_storage_service_input *input, + int argc, char *argv[]) { struct doveadm_mail_cmd_context *ctx; const struct doveadm_mail_cmd *cmd; const char *getopt_args; - bool ret, add_username_header = FALSE; + bool add_username_header = FALSE; int c; cmd = doveadm_mail_cmd_find(cmd_name); if (cmd == NULL) { i_error("doveadm: Client sent unknown command: %s", cmd_name); - return FALSE; + return NULL; } ctx = doveadm_mail_cmd_init(cmd, set); @@ -83,7 +83,7 @@ "Client sent unknown parameter: %c", cmd->name, c); ctx->v.deinit(ctx); - return FALSE; + return NULL; } } } @@ -95,8 +95,9 @@ i_error("doveadm %s: Client sent unknown parameter: %s", cmd->name, argv[0]); ctx->v.deinit(ctx); - return FALSE; + return NULL; } + ctx->args = (const void *)argv; if (doveadm_print_is_initialized() && add_username_header) { doveadm_print_header("username", "Username", @@ -104,20 +105,38 @@ DOVEADM_PRINT_HEADER_FLAG_HIDE_TITLE); doveadm_print_sticky("username", input->username); } + return ctx; +} - ctx->args = (const void *)argv; +static void +doveadm_mail_cmd_server_run(struct client_connection *conn, + struct doveadm_mail_cmd_context *ctx, + const struct mail_storage_service_input *input) +{ + const char *error; + int ret; + if (ctx->v.preinit != NULL) ctx->v.preinit(ctx); - doveadm_mail_single_user(ctx, input); + ret = doveadm_mail_single_user(ctx, input, &error); doveadm_mail_server_flush(); ctx->v.deinit(ctx); doveadm_print_flush(); mail_storage_service_deinit(&ctx->storage_service); - ret = ctx->exit_code == 0; + + if (ret < 0) { + i_error("%s: %s", ctx->cmd->name, error); + o_stream_send(conn->output, "\n-\n", 3); + } else if (ret == 0) { + o_stream_send_str(conn->output, "\n-NOUSER\n"); + } else if (ctx->exit_code != 0) { + /* maybe not an error, but not a full success either */ + o_stream_send(conn->output, "\n-\n", 3); + } else { + o_stream_send(conn->output, "\n+\n", 3); + } pool_unref(&ctx->pool); - - return ret; } static bool client_is_allowed_command(const struct doveadm_settings *set, @@ -144,9 +163,9 @@ static bool client_handle_command(struct client_connection *conn, char **args) { struct mail_storage_service_input input; + struct doveadm_mail_cmd_context *ctx; const char *flags, *cmd_name; unsigned int argc; - bool ret; memset(&input, 0, sizeof(input)); input.service = "doveadm"; @@ -190,11 +209,11 @@ } o_stream_cork(conn->output); - ret = doveadm_mail_cmd_server(cmd_name, conn->set, &input, argc, args); - if (ret) - o_stream_send(conn->output, "\n+\n", 3); + ctx = doveadm_mail_cmd_server_parse(cmd_name, conn->set, &input, argc, args); + if (ctx == NULL) + o_stream_send(conn->output, "\n-\n", 3); else - o_stream_send(conn->output, "\n-\n", 3); + doveadm_mail_cmd_server_run(conn, ctx, &input); o_stream_uncork(conn->output); /* flush the output and disconnect */ diff -r 14ff849dc266 -r 9693521aa153 src/doveadm/doveadm-mail.c --- a/src/doveadm/doveadm-mail.c Sun Mar 04 09:50:21 2012 +0200 +++ b/src/doveadm/doveadm-mail.c Sun Mar 04 10:11:55 2012 +0200 @@ -316,12 +316,10 @@ return 1; } -void doveadm_mail_single_user(struct doveadm_mail_cmd_context *ctx, - const struct mail_storage_service_input *input) +int doveadm_mail_single_user(struct doveadm_mail_cmd_context *ctx, + const struct mail_storage_service_input *input, + const char **error_r) { - const char *error; - int ret; - i_assert(input->username != NULL); ctx->cur_username = input->username; @@ -331,11 +329,7 @@ if (hook_doveadm_mail_init != NULL) hook_doveadm_mail_init(ctx); - ret = doveadm_mail_next_user(ctx, input, &error); - if (ret < 0) - i_fatal("%s", error); - else if (ret == 0) - i_fatal_status(EX_NOUSER, "User doesn't exist"); + return doveadm_mail_next_user(ctx, input, error_r); } static void sig_die(const siginfo_t *si, void *context ATTR_UNUSED) @@ -453,8 +447,8 @@ doveadm_mail_cmd(const struct doveadm_mail_cmd *cmd, int argc, char *argv[]) { struct doveadm_mail_cmd_context *ctx; - const char *getopt_args, *wildcard_user; - int c; + const char *getopt_args, *wildcard_user, *error; + int ret, c; ctx = doveadm_mail_cmd_init(cmd, doveadm_settings); ctx->full_args = (const void *)(argv + 1); @@ -521,7 +515,11 @@ memset(&input, 0, sizeof(input)); input.service = "doveadm"; input.username = ctx->cur_username; - doveadm_mail_single_user(ctx, &input); + ret = doveadm_mail_single_user(ctx, &input, &error); + if (ret < 0) + i_fatal("%s", error); + else if (ret == 0) + i_fatal_status(EX_NOUSER, "User doesn't exist"); } else { ctx->service_flags |= MAIL_STORAGE_SERVICE_FLAG_TEMP_PRIV_DROP; doveadm_mail_all_users(ctx, argv, wildcard_user); diff -r 14ff849dc266 -r 9693521aa153 src/doveadm/doveadm-mail.h --- a/src/doveadm/doveadm-mail.h Sun Mar 04 09:50:21 2012 +0200 +++ b/src/doveadm/doveadm-mail.h Sun Mar 04 10:11:55 2012 +0200 @@ -95,8 +95,9 @@ struct doveadm_mail_cmd_context * doveadm_mail_cmd_init(const struct doveadm_mail_cmd *cmd, const struct doveadm_settings *set); -void doveadm_mail_single_user(struct doveadm_mail_cmd_context *ctx, - const struct mail_storage_service_input *input); +int doveadm_mail_single_user(struct doveadm_mail_cmd_context *ctx, + const struct mail_storage_service_input *input, + const char **error_r); int doveadm_mail_server_user(struct doveadm_mail_cmd_context *ctx, const struct mail_storage_service_input *input, const char **error_r); diff -r 14ff849dc266 -r 9693521aa153 src/replication/replicator/doveadm-connection.c --- a/src/replication/replicator/doveadm-connection.c Sun Mar 04 09:50:21 2012 +0200 +++ b/src/replication/replicator/doveadm-connection.c Sun Mar 04 10:11:55 2012 +0200 @@ -105,8 +105,10 @@ if (line[0] == '+') doveadm_callback(conn, DOVEADM_REPLY_OK); else if (line[0] == '-') { - /* FIXME: handle DOVEADM_REPLY_NOUSER */ - doveadm_callback(conn, DOVEADM_REPLY_FAIL); + if (strcmp(line+1, "NOUSER") == 0) + doveadm_callback(conn, DOVEADM_REPLY_NOUSER); + else + doveadm_callback(conn, DOVEADM_REPLY_FAIL); } else { i_error("%s: Invalid input: %s", conn->path, line); return -1; From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: replication plugin: Handle shared namespaces better... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/21d5e258d69c changeset: 14264:21d5e258d69c user: Timo Sirainen date: Sun Mar 04 10:26:32 2012 +0200 description: replication plugin: Handle shared namespaces better, skip public namespaces. These will be handled better later, maybe in v2.2. diffstat: src/plugins/replication/replication-plugin.c | 38 ++++++++++++++++++--------- 1 files changed, 25 insertions(+), 13 deletions(-) diffs (122 lines): diff -r b44922333957 -r 21d5e258d69c src/plugins/replication/replication-plugin.c --- a/src/plugins/replication/replication-plugin.c Sun Mar 04 10:25:23 2012 +0200 +++ b/src/plugins/replication/replication-plugin.c Sun Mar 04 10:26:32 2012 +0200 @@ -9,6 +9,7 @@ #include "network.h" #include "write-full.h" #include "mail-user.h" +#include "mail-namespace.h" #include "mail-storage-private.h" #include "notify-plugin.h" #include "replication-common.h" @@ -35,7 +36,7 @@ }; struct replication_mail_txn_context { - struct mail_user *user; + struct mail_namespace *ns; bool new_messages; }; @@ -175,19 +176,25 @@ return ret; } -static void replication_notify(struct mail_user *user, +static void replication_notify(struct mail_namespace *ns, enum replication_priority priority) { - struct replication_user *ruser = REPLICATION_USER_CONTEXT(user); + struct replication_user *ruser; - if (user->dsyncing) { + if (ns->user->dsyncing) { /* we're running dsync, which means that the remote is telling us about a change. don't trigger a replication back to it */ return; } + if (ns->owner == NULL) { + /* public namespace. we can't handle this for now. */ + return; + } + ruser = REPLICATION_USER_CONTEXT(ns->owner); + if (priority == REPLICATION_PRIORITY_SYNC) { - if (replication_notify_sync(user) == 0) { + if (replication_notify_sync(ns->owner) == 0) { timeout_remove(&ruser->to); ruser->priority = REPLICATION_PRIORITY_NONE; return; @@ -200,7 +207,7 @@ ruser->priority = priority; if (ruser->to == NULL) { ruser->to = timeout_add(REPLICATION_NOTIFY_DELAY_MSECS, - replication_notify_now, user); + replication_notify_now, ns->owner); } } @@ -210,7 +217,7 @@ struct replication_mail_txn_context *ctx; ctx = i_new(struct replication_mail_txn_context, 1); - ctx->user = t->box->storage->user; + ctx->ns = mailbox_get_namespace(t->box); return ctx; } @@ -237,28 +244,31 @@ { struct replication_mail_txn_context *ctx = (struct replication_mail_txn_context *)txn; - struct replication_user *ruser = REPLICATION_USER_CONTEXT(ctx->user); + struct replication_user *ruser = + REPLICATION_USER_CONTEXT(ctx->ns->user); enum replication_priority priority; if (ctx->new_messages || changes->changed) { priority = !ctx->new_messages ? REPLICATION_PRIORITY_LOW : ruser->sync_secs == 0 ? REPLICATION_PRIORITY_HIGH : REPLICATION_PRIORITY_SYNC; - replication_notify(ctx->user, priority); + replication_notify(ctx->ns, priority); } i_free(ctx); } static void replication_mailbox_create(struct mailbox *box) { - replication_notify(box->storage->user, REPLICATION_PRIORITY_LOW); + replication_notify(mailbox_get_namespace(box), + REPLICATION_PRIORITY_LOW); } static void replication_mailbox_delete_commit(void *txn ATTR_UNUSED, struct mailbox *box) { - replication_notify(box->storage->user, REPLICATION_PRIORITY_LOW); + replication_notify(mailbox_get_namespace(box), + REPLICATION_PRIORITY_LOW); } static void @@ -266,13 +276,15 @@ struct mailbox *dest, bool rename_children ATTR_UNUSED) { - replication_notify(dest->storage->user, REPLICATION_PRIORITY_LOW); + replication_notify(mailbox_get_namespace(dest), + REPLICATION_PRIORITY_LOW); } static void replication_mailbox_set_subscribed(struct mailbox *box, bool subscribed ATTR_UNUSED) { - replication_notify(box->storage->user, REPLICATION_PRIORITY_LOW); + replication_notify(mailbox_get_namespace(box), + REPLICATION_PRIORITY_LOW); } static void replication_user_deinit(struct mail_user *user) From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: replication plugin: Use one shared notification fif... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/b44922333957 changeset: 14263:b44922333957 user: Timo Sirainen date: Sun Mar 04 10:25:23 2012 +0200 description: replication plugin: Use one shared notification fifo for all users. diffstat: src/plugins/replication/replication-plugin.c | 58 ++++++++++++++------------- 1 files changed, 31 insertions(+), 27 deletions(-) diffs (138 lines): diff -r 9693521aa153 -r b44922333957 src/plugins/replication/replication-plugin.c --- a/src/plugins/replication/replication-plugin.c Sun Mar 04 10:11:55 2012 +0200 +++ b/src/plugins/replication/replication-plugin.c Sun Mar 04 10:25:23 2012 +0200 @@ -27,16 +27,11 @@ struct replication_user { union mail_user_module_context module_ctx; - const char *fifo_path; const char *socket_path; - int fifo_fd; - struct timeout *to; enum replication_priority priority; unsigned int sync_secs; - - bool fifo_failed; }; struct replication_mail_txn_context { @@ -46,25 +41,27 @@ static MODULE_CONTEXT_DEFINE_INIT(replication_user_module, &mail_user_module_register); +static int fifo_fd; +static bool fifo_failed; +static char *fifo_path; static int replication_fifo_notify(struct mail_user *user, enum replication_priority priority) { - struct replication_user *ruser = REPLICATION_USER_CONTEXT(user); string_t *str; ssize_t ret; - if (ruser->fifo_failed) + if (fifo_failed) return -1; - if (ruser->fifo_fd == -1) { - ruser->fifo_fd = open(ruser->fifo_path, O_WRONLY); - if (ruser->fifo_fd == -1) { - i_error("open(%s) failed: %m", ruser->fifo_path); - ruser->fifo_failed = TRUE; + if (fifo_fd == -1) { + fifo_fd = open(fifo_path, O_WRONLY); + if (fifo_fd == -1) { + i_error("open(%s) failed: %m", fifo_path); + fifo_failed = TRUE; return -1; } - fd_set_nonblock(ruser->fifo_fd, TRUE); + fd_set_nonblock(fifo_fd, TRUE); } /* \t */ str = t_str_new(256); @@ -82,21 +79,19 @@ break; } str_append_c(str, '\n'); - ret = write(ruser->fifo_fd, str_data(str), str_len(str)); + ret = write(fifo_fd, str_data(str), str_len(str)); if (ret == 0) { /* busy, try again later */ return 0; } if (ret != (ssize_t)str_len(str)) { if (ret < 0) - i_error("write(%s) failed: %m", ruser->fifo_path); - else { - i_error("write(%s) wrote partial data", - ruser->fifo_path); - } - if (close(ruser->fifo_fd) < 0) - i_error("close(%s) failed: %m", ruser->fifo_path); - ruser->fifo_fd = -1; + i_error("write(%s) failed: %m", fifo_path); + else + i_error("write(%s) wrote partial data", fifo_path); + if (close(fifo_fd) < 0) + i_error("close(%s) failed: %m", fifo_path); + fifo_fd = -1; return -1; } return 1; @@ -111,7 +106,7 @@ i_assert(ruser->priority != REPLICATION_PRIORITY_SYNC); if ((ret = replication_fifo_notify(user, ruser->priority)) < 0 && - !ruser->fifo_failed) { + !fifo_failed) { /* retry once, in case replication server was restarted */ ret = replication_fifo_notify(user, ruser->priority); } @@ -288,7 +283,7 @@ replication_notify_now(user); if (ruser->to != NULL) { i_warning("%s: Couldn't send final notification " - "due to fifo being busy", ruser->fifo_path); + "due to fifo being busy", fifo_path); timeout_remove(&ruser->to); } } @@ -308,9 +303,12 @@ v->deinit = replication_user_deinit; MODULE_CONTEXT_SET(user, replication_user_module, ruser); - ruser->fifo_fd = -1; - ruser->fifo_path = p_strconcat(user->pool, user->set->base_dir, - "/"REPLICATION_FIFO_NAME, NULL); + if (fifo_path == NULL) { + /* we'll assume that all users have the same base_dir. + they really should. */ + fifo_path = i_strconcat(user->set->base_dir, + "/"REPLICATION_FIFO_NAME, NULL); + } ruser->socket_path = p_strconcat(user->pool, user->set->base_dir, "/"REPLICATION_SOCKET_NAME, NULL); value = mail_user_plugin_getenv(user, "replication_sync_timeout"); @@ -340,12 +338,18 @@ void replication_plugin_init(struct module *module) { + fifo_fd = -1; replication_ctx = notify_register(&replication_vfuncs); mail_storage_hooks_add(module, &replication_mail_storage_hooks); } void replication_plugin_deinit(void) { + if (close(fifo_fd) < 0) + i_error("close(%s) failed: %m", fifo_path); + fifo_fd = -1; + i_free_and_null(fifo_path); + mail_storage_hooks_remove(&replication_mail_storage_hooks); notify_unregister(replication_ctx); } From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: doveadm-server: If socket is 0600 mode there's neve... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/ed641f7da63b changeset: 14265:ed641f7da63b user: Timo Sirainen date: Sun Mar 04 10:35:25 2012 +0200 description: doveadm-server: If socket is 0600 mode there's never no need to authenticate. Even if the socket's owner doesn't match the process's effective UID. This could be e.g. because socket owner is root, while doveadm-server runs as vmail. diffstat: src/doveadm/client-connection.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 21d5e258d69c -r ed641f7da63b src/doveadm/client-connection.c --- a/src/doveadm/client-connection.c Sun Mar 04 10:26:32 2012 +0200 +++ b/src/doveadm/client-connection.c Sun Mar 04 10:35:25 2012 +0200 @@ -364,7 +364,7 @@ fstat() always returns mode as 0777 */ if (net_getunixname(listen_fd, &listen_path) == 0 && stat(listen_path, &st) == 0 && S_ISSOCK(st.st_mode) && - (st.st_mode & 0777) == 0600 && st.st_uid == geteuid()) { + (st.st_mode & 0777) == 0600) { /* no need for client to authenticate */ conn->authenticated = TRUE; o_stream_send(conn->output, "+\n", 2); From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: replicator: Crashfix Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/fed306bef481 changeset: 14266:fed306bef481 user: Timo Sirainen date: Sun Mar 04 10:40:19 2012 +0200 description: replicator: Crashfix diffstat: src/replication/replicator/doveadm-connection.c | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diffs (13 lines): diff -r ed641f7da63b -r fed306bef481 src/replication/replicator/doveadm-connection.c --- a/src/replication/replicator/doveadm-connection.c Sun Mar 04 10:35:25 2012 +0200 +++ b/src/replication/replicator/doveadm-connection.c Sun Mar 04 10:40:19 2012 +0200 @@ -116,8 +116,7 @@ conn->end_of_print = FALSE; /* FIXME: disconnect after each request for now. doveadm server's getopt() handling seems to break otherwise */ - doveadm_disconnect(conn); - return 0; + return -1; } static void doveadm_input(struct doveadm_connection *conn) From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: auth: userdb passwd iteration now skips users with ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/85a8d582d37f changeset: 14267:85a8d582d37f user: Timo Sirainen date: Sun Mar 04 11:17:45 2012 +0200 description: auth: userdb passwd iteration now skips users with shell set to /bin/false or /sbin/nologin diffstat: src/auth/userdb-passwd.c | 24 +++++++++++++++++++----- 1 files changed, 19 insertions(+), 5 deletions(-) diffs (41 lines): diff -r fed306bef481 -r 85a8d582d37f src/auth/userdb-passwd.c --- a/src/auth/userdb-passwd.c Sun Mar 04 10:40:19 2012 +0200 +++ b/src/auth/userdb-passwd.c Sun Mar 04 11:17:45 2012 +0200 @@ -137,6 +137,24 @@ return &ctx->ctx; } +static bool +passwd_iterate_want_pw(struct passwd *pw, const struct auth_settings *set) +{ + /* skip entries not in valid UID range. + they're users for daemons and such. */ + if (pw->pw_uid < (uid_t)set->first_valid_uid) + return FALSE; + if (pw->pw_uid > (uid_t)set->last_valid_uid && set->last_valid_uid != 0) + return FALSE; + + /* skip entries that don't have a valid shell. + they're again probably not real users. */ + if (strcmp(pw->pw_shell, "/bin/false") == 0 || + strcmp(pw->pw_shell, "/sbin/nologin") == 0) + return FALSE; + return TRUE; +} + static void passwd_iterate_next(struct userdb_iterate_context *_ctx) { struct passwd_userdb_iterate_context *ctx = @@ -154,11 +172,7 @@ errno = 0; while ((pw = getpwent()) != NULL) { - /* skip entries not in valid UID range. - they're users for daemons and such. */ - if (pw->pw_uid >= (uid_t)set->first_valid_uid && - (set->last_valid_uid == 0 || - pw->pw_uid <= (uid_t)set->last_valid_uid)) { + if (passwd_iterate_want_pw(pw, set)) { _ctx->callback(pw->pw_name, _ctx->context); return; } From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: Makefile: Added missing replication plugin directory Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/a422bd8ed511 changeset: 14268:a422bd8ed511 user: Timo Sirainen date: Sun Mar 04 11:19:10 2012 +0200 description: Makefile: Added missing replication plugin directory diffstat: src/plugins/Makefile.am | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff -r 85a8d582d37f -r a422bd8ed511 src/plugins/Makefile.am --- a/src/plugins/Makefile.am Sun Mar 04 11:17:45 2012 +0200 +++ b/src/plugins/Makefile.am Sun Mar 04 11:19:10 2012 +0200 @@ -23,6 +23,7 @@ mail-log \ quota \ imap-quota \ + replication \ snarf \ stats \ imap-stats \ From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: replication plugin: Don't try to close fifo if it w... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/9d47be37afeb changeset: 14269:9d47be37afeb user: Timo Sirainen date: Sun Mar 04 11:23:14 2012 +0200 description: replication plugin: Don't try to close fifo if it wasn't even opened. diffstat: src/plugins/replication/replication-plugin.c | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diffs (18 lines): diff -r a422bd8ed511 -r 9d47be37afeb src/plugins/replication/replication-plugin.c --- a/src/plugins/replication/replication-plugin.c Sun Mar 04 11:19:10 2012 +0200 +++ b/src/plugins/replication/replication-plugin.c Sun Mar 04 11:23:14 2012 +0200 @@ -357,9 +357,11 @@ void replication_plugin_deinit(void) { - if (close(fifo_fd) < 0) - i_error("close(%s) failed: %m", fifo_path); - fifo_fd = -1; + if (fifo_fd != -1) { + if (close(fifo_fd) < 0) + i_error("close(%s) failed: %m", fifo_path); + fifo_fd = -1; + } i_free_and_null(fifo_path); mail_storage_hooks_remove(&replication_mail_storage_hooks); From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: replicator: Queue handling was pretty broken Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/633e587446d8 changeset: 14270:633e587446d8 user: Timo Sirainen date: Sun Mar 04 12:04:46 2012 +0200 description: replicator: Queue handling was pretty broken diffstat: src/replication/replicator/replicator-queue.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diffs (20 lines): diff -r 9d47be37afeb -r 633e587446d8 src/replication/replicator/replicator-queue.c --- a/src/replication/replicator/replicator-queue.c Sun Mar 04 11:23:14 2012 +0200 +++ b/src/replication/replicator/replicator-queue.c Sun Mar 04 12:04:46 2012 +0200 @@ -111,6 +111,7 @@ if (user == NULL) { user = i_new(struct replicator_user, 1); user->username = i_strdup(username); + hash_table_insert(queue->user_hash, user->username, user); } else { if (user->priority > priority) { /* user already has a higher priority than this */ @@ -237,6 +238,8 @@ void replicator_queue_push(struct replicator_queue *queue, struct replicator_user *user) { + i_assert(user->popped); + priorityq_add(queue->user_queue, &user->item); user->popped = FALSE; From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: replicator: Fixed crash on deinit Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/3b258bb9a03f changeset: 14271:3b258bb9a03f user: Timo Sirainen date: Sun Mar 04 12:07:21 2012 +0200 description: replicator: Fixed crash on deinit diffstat: src/replication/replicator/replicator-queue.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff -r 633e587446d8 -r 3b258bb9a03f src/replication/replicator/replicator-queue.c --- a/src/replication/replicator/replicator-queue.c Sun Mar 04 12:04:46 2012 +0200 +++ b/src/replication/replicator/replicator-queue.c Sun Mar 04 12:07:21 2012 +0200 @@ -83,6 +83,8 @@ while ((item = priorityq_pop(queue->user_queue)) != NULL) { struct replicator_user *user = (struct replicator_user *)item; + + user->popped = TRUE; replicator_queue_remove(queue, &user); } From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: replicator: Another deinit crashfix Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/f7a3714dc390 changeset: 14272:f7a3714dc390 user: Timo Sirainen date: Sun Mar 04 12:09:35 2012 +0200 description: replicator: Another deinit crashfix diffstat: src/replication/replicator/replicator-queue.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff -r 3b258bb9a03f -r f7a3714dc390 src/replication/replicator/replicator-queue.c --- a/src/replication/replicator/replicator-queue.c Sun Mar 04 12:07:21 2012 +0200 +++ b/src/replication/replicator/replicator-queue.c Sun Mar 04 12:09:35 2012 +0200 @@ -81,6 +81,8 @@ *_queue = NULL; + queue->change_callback = NULL; + while ((item = priorityq_pop(queue->user_queue)) != NULL) { struct replicator_user *user = (struct replicator_user *)item; From dovecot at dovecot.org Sun May 20 03:26:24 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:24 +0300 Subject: dovecot-2.2: log: Don't shutdown until all log writers have gone. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/174a22a052fe changeset: 14273:174a22a052fe user: Timo Sirainen date: Sun Mar 04 12:24:29 2012 +0200 description: log: Don't shutdown until all log writers have gone. This fixed losing log messages during shutdown. diffstat: src/log/log-connection.c | 2 ++ src/log/main.c | 8 ++++++-- 2 files changed, 8 insertions(+), 2 deletions(-) diffs (31 lines): diff -r f7a3714dc390 -r 174a22a052fe src/log/log-connection.c --- a/src/log/log-connection.c Sun Mar 04 12:09:35 2012 +0200 +++ b/src/log/log-connection.c Sun Mar 04 12:24:29 2012 +0200 @@ -341,6 +341,8 @@ i_error("close(log connection fd) failed: %m"); i_free(log->default_prefix); i_free(log); + + master_service_client_connection_destroyed(master_service); } void log_connections_init(void) diff -r f7a3714dc390 -r 174a22a052fe src/log/main.c --- a/src/log/main.c Sun Mar 04 12:09:35 2012 +0200 +++ b/src/log/main.c Sun Mar 04 12:24:29 2012 +0200 @@ -37,9 +37,13 @@ static void client_connected(struct master_service_connection *conn) { - if (conn->fifo) + if (conn->fifo) { log_connection_create(errorbuf, conn->fd, conn->listen_fd); - else if (strcmp(conn->name, "log-errors") == 0) + /* kludge: normally FIFOs aren't counted as connections, + but here we want log process to stay open until all writers + have closed */ + conn->fifo = FALSE; + } else if (strcmp(conn->name, "log-errors") == 0) doveadm_connection_create(errorbuf, conn->fd); else { i_error("Unknown listener name: %s", conn->name); From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: log: Avoid an "master input for invalid service_fd"... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/509206065d33 changeset: 14274:509206065d33 user: Timo Sirainen date: Sun Mar 04 12:31:26 2012 +0200 description: log: Avoid an "master input for invalid service_fd" error at deinit. diffstat: src/log/log-connection.c | 21 +++++++++++++-------- 1 files changed, 13 insertions(+), 8 deletions(-) diffs (58 lines): diff -r 174a22a052fe -r 509206065d33 src/log/log-connection.c --- a/src/log/log-connection.c Sun Mar 04 12:24:29 2012 +0200 +++ b/src/log/log-connection.c Sun Mar 04 12:31:26 2012 +0200 @@ -136,7 +136,7 @@ { struct log_connection *const *logs, *log; struct log_client *client; - const char *p, *p2; + const char *p, *p2, *cmd; unsigned int count; int service_fd; long pid; @@ -148,33 +148,38 @@ } service_fd = atoi(t_strcut(line, ' ')); pid = strtol(t_strcut(p, ' '), NULL, 10); + cmd = p2 + 1; logs = array_get(&logs_by_fd, &count); if (service_fd >= (int)count || logs[service_fd] == NULL) { + if (strcmp(cmd, "BYE") == 0 && service_fd < (int)count) { + /* master is probably shutting down and we already + noticed the log fd closing */ + return; + } i_error("Received master input for invalid service_fd %d: %s", service_fd, line); return; } log = logs[service_fd]; client = hash_table_lookup(log->clients, POINTER_CAST(pid)); - line = p2 + 1; - if (strcmp(line, "BYE") == 0) { + if (strcmp(cmd, "BYE") == 0) { if (client == NULL) { /* we haven't seen anything important from this client. it's not an error. */ return; } log_client_free(log, client, pid); - } else if (strncmp(line, "FATAL ", 6) == 0) { - client_log_fatal(log, client, line + 6, log_time, tm); - } else if (strncmp(line, "DEFAULT-FATAL ", 14) == 0) { + } else if (strncmp(cmd, "FATAL ", 6) == 0) { + client_log_fatal(log, client, cmd + 6, log_time, tm); + } else if (strncmp(cmd, "DEFAULT-FATAL ", 14) == 0) { /* If the client has logged a fatal/panic, don't log this message. */ if (client == NULL || !client->fatal_logged) - client_log_fatal(log, client, line + 14, log_time, tm); + client_log_fatal(log, client, cmd + 14, log_time, tm); } else { - i_error("Received unknown command from master: %s", line); + i_error("Received unknown command from master: %s", cmd); } } From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: imap: Fixed error handling in APPEND parameters. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/5a62b55914c4 changeset: 14275:5a62b55914c4 user: Timo Sirainen date: Sun Mar 04 12:50:02 2012 +0200 description: imap: Fixed error handling in APPEND parameters. diffstat: src/imap/cmd-append.c | 13 +++++++++---- 1 files changed, 9 insertions(+), 4 deletions(-) diffs (33 lines): diff -r 509206065d33 -r 5a62b55914c4 src/imap/cmd-append.c --- a/src/imap/cmd-append.c Sun Mar 04 12:31:26 2012 +0200 +++ b/src/imap/cmd-append.c Sun Mar 04 12:50:02 2012 +0200 @@ -227,11 +227,11 @@ enum mail_flags flags; const char *const *keywords_list; struct mail_keywords *keywords; - const char *internal_date_str; + const char *internal_date_str, *msg; time_t internal_date; int ret, timezone_offset; unsigned int save_count; - bool nonsync; + bool nonsync, fatal; if (cmd->cancel) { cmd_append_finish(ctx); @@ -245,8 +245,13 @@ ret = imap_parser_read_args(ctx->save_parser, 0, IMAP_PARSE_FLAG_LITERAL_SIZE, &args); if (ret == -1) { - if (!ctx->failed) - client_send_command_error(cmd, NULL); + if (!ctx->failed) { + msg = imap_parser_get_error(ctx->save_parser, &fatal); + if (fatal) + client_disconnect_with_error(client, msg); + else + client_send_command_error(cmd, msg); + } cmd_append_finish(ctx); return TRUE; } From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: dsync server: Ignore -f, -R and -m parameters inste... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/9c6eeeb810c0 changeset: 14276:9c6eeeb810c0 user: Timo Sirainen date: Sun Mar 04 13:01:09 2012 +0200 description: dsync server: Ignore -f, -R and -m parameters instead of failing. v2.0 dsync did this as well. diffstat: src/doveadm/dsync/doveadm-dsync.c | 20 ++++++++++++-------- 1 files changed, 12 insertions(+), 8 deletions(-) diffs (45 lines): diff -r 5a62b55914c4 -r 9c6eeeb810c0 src/doveadm/dsync/doveadm-dsync.c --- a/src/doveadm/dsync/doveadm-dsync.c Sun Mar 04 12:50:02 2012 +0200 +++ b/src/doveadm/dsync/doveadm-dsync.c Sun Mar 04 13:01:09 2012 +0200 @@ -644,6 +644,7 @@ char *p, *dup, new_flags[6]; int max_argc, src, dest, i, j; bool flag_f = FALSE, flag_R = FALSE, flag_m, flag_u, flag_C, has_arg; + bool dsync_server = FALSE; p = strrchr(argv[0], '/'); if (p == NULL) p = argv[0]; @@ -732,9 +733,10 @@ new_argv[dest] = "sync"; else if (strcmp(argv[src], "backup") == 0) new_argv[dest] = "backup"; - else if (strcmp(argv[src], "server") == 0) + else if (strcmp(argv[src], "server") == 0) { new_argv[dest] = "dsync-server"; - else + dsync_server = TRUE; + } else i_fatal("Invalid parameter: %s", argv[src]); src++; dest++; @@ -746,12 +748,14 @@ /* dsync flags */ new_flags[0] = '-'; new_flags[1] = 'E'; i = 2; - if (flag_f) - new_flags[i++] = 'f'; - if (flag_R) - new_flags[i++] = 'R'; - if (mailbox != NULL) - new_flags[i++] = 'm'; + if (!dsync_server) { + if (flag_f) + new_flags[i++] = 'f'; + if (flag_R) + new_flags[i++] = 'R'; + if (mailbox != NULL) + new_flags[i++] = 'm'; + } i_assert((unsigned int)i < sizeof(new_flags)); new_flags[i] = '\0'; From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: lib-storage: mailbox_enable() shouldn't actually op... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/8cbc130c2b72 changeset: 14277:8cbc130c2b72 user: Timo Sirainen date: Sun Mar 04 13:19:03 2012 +0200 description: lib-storage: mailbox_enable() shouldn't actually open the mailbox. This combined with mailbox_get_status() never synced the mailbox, so if there were any changes they weren't visible in STATUS reply. diffstat: src/lib-storage/index/index-storage.c | 11 ++++------- 1 files changed, 4 insertions(+), 7 deletions(-) diffs (29 lines): diff -r 9c6eeeb810c0 -r 8cbc130c2b72 src/lib-storage/index/index-storage.c --- a/src/lib-storage/index/index-storage.c Sun Mar 04 13:01:09 2012 +0200 +++ b/src/lib-storage/index/index-storage.c Sun Mar 04 13:19:03 2012 +0200 @@ -272,6 +272,9 @@ box->opened = TRUE; + if ((box->enabled_features & MAILBOX_FEATURE_CONDSTORE) != 0) + mail_index_modseq_enable(box->index); + index_thread_mailbox_opened(box); hook_mailbox_opened(box); return 0; @@ -313,14 +316,8 @@ int index_storage_mailbox_enable(struct mailbox *box, enum mailbox_feature feature) { - if ((feature & MAILBOX_FEATURE_CONDSTORE) != 0) { + if ((feature & MAILBOX_FEATURE_CONDSTORE) != 0) box->enabled_features |= MAILBOX_FEATURE_CONDSTORE; - if (mailbox_open(box) < 0) - return -1; - T_BEGIN { - mail_index_modseq_enable(box->index); - } T_END; - } return 0; } From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: auth: userdb passwd iteration skips now also users ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/694ea6e13a86 changeset: 14279:694ea6e13a86 user: Timo Sirainen date: Sun Mar 04 13:56:33 2012 +0200 description: auth: userdb passwd iteration skips now also users with /usr/sbin/nologin shell diffstat: src/auth/userdb-passwd.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (13 lines): diff -r 31ae11fe18b2 -r 694ea6e13a86 src/auth/userdb-passwd.c --- a/src/auth/userdb-passwd.c Sun Mar 04 13:20:21 2012 +0200 +++ b/src/auth/userdb-passwd.c Sun Mar 04 13:56:33 2012 +0200 @@ -150,7 +150,8 @@ /* skip entries that don't have a valid shell. they're again probably not real users. */ if (strcmp(pw->pw_shell, "/bin/false") == 0 || - strcmp(pw->pw_shell, "/sbin/nologin") == 0) + strcmp(pw->pw_shell, "/sbin/nologin") == 0 || + strcmp(pw->pw_shell, "/usr/sbin/nologin") == 0) return FALSE; return TRUE; } From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: lib-storage: If mailbox is already open in mailbox_... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/31ae11fe18b2 changeset: 14278:31ae11fe18b2 user: Timo Sirainen date: Sun Mar 04 13:20:21 2012 +0200 description: lib-storage: If mailbox is already open in mailbox_enable(), enable modseqs. diffstat: src/lib-storage/index/index-storage.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diffs (16 lines): diff -r 8cbc130c2b72 -r 31ae11fe18b2 src/lib-storage/index/index-storage.c --- a/src/lib-storage/index/index-storage.c Sun Mar 04 13:19:03 2012 +0200 +++ b/src/lib-storage/index/index-storage.c Sun Mar 04 13:20:21 2012 +0200 @@ -316,8 +316,11 @@ int index_storage_mailbox_enable(struct mailbox *box, enum mailbox_feature feature) { - if ((feature & MAILBOX_FEATURE_CONDSTORE) != 0) + if ((feature & MAILBOX_FEATURE_CONDSTORE) != 0) { box->enabled_features |= MAILBOX_FEATURE_CONDSTORE; + if (box->opened) + mail_index_modseq_enable(box->index); + } return 0; } From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: lib-mail: rfc822_parse_quoted_string() didn't remov... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/92c6e963176e changeset: 14280:92c6e963176e user: Timo Sirainen date: Sun Mar 04 14:12:19 2012 +0200 description: lib-mail: rfc822_parse_quoted_string() didn't remove '\' from the strings. diffstat: src/lib-mail/rfc822-parser.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 694ea6e13a86 -r 92c6e963176e src/lib-mail/rfc822-parser.c --- a/src/lib-mail/rfc822-parser.c Sun Mar 04 13:56:33 2012 +0200 +++ b/src/lib-mail/rfc822-parser.c Sun Mar 04 14:12:19 2012 +0200 @@ -231,7 +231,7 @@ if (ctx->data == ctx->end) return -1; - str_append_n(str, start, ctx->data - start); + str_append_n(str, start, ctx->data - start - 1); start = ctx->data; break; } From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: lib-storage: mailbox_list_iter_init_namespaces() du... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/bbe6b6c2ee99 changeset: 14281:bbe6b6c2ee99 user: Timo Sirainen date: Sun Mar 04 14:31:23 2012 +0200 description: lib-storage: mailbox_list_iter_init_namespaces() duplicated INBOX if it was also ns prefix. diffstat: src/lib-storage/mailbox-list-iter.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diffs (16 lines): diff -r 92c6e963176e -r bbe6b6c2ee99 src/lib-storage/mailbox-list-iter.c --- a/src/lib-storage/mailbox-list-iter.c Sun Mar 04 14:12:19 2012 +0200 +++ b/src/lib-storage/mailbox-list-iter.c Sun Mar 04 14:31:23 2012 +0200 @@ -334,6 +334,12 @@ unsigned int i; bool ret = FALSE; + if (strncasecmp(ns->prefix, "INBOX", ns->prefix_len-1) == 0) { + /* INBOX is going to be listed in any case, + don't duplicate it */ + return FALSE; + } + for (i = 0; ctx->patterns_ns_match[i] != NULL; i++) { T_BEGIN { ret = iter_next_try_prefix_pattern(ctx, ns, From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: Makefile: Fixed dovecot-config installation Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/2f3c83f28dfe changeset: 14283:2f3c83f28dfe user: Timo Sirainen date: Sun Mar 04 15:11:20 2012 +0200 description: Makefile: Fixed dovecot-config installation diffstat: Makefile.am | 9 +-------- 1 files changed, 1 insertions(+), 8 deletions(-) diffs (33 lines): diff -r d4f40b5c080c -r 2f3c83f28dfe Makefile.am --- a/Makefile.am Sun Mar 04 14:33:00 2012 +0200 +++ b/Makefile.am Sun Mar 04 15:11:20 2012 +0200 @@ -1,8 +1,5 @@ aclocaldir = $(datadir)/aclocal -# automake 1.11.2+ doesn't like allow using pkglib_DATA, but we really it. -dovecotlibdir = $(libdir)/dovecot - SUBDIRS = \ . \ src \ @@ -20,11 +17,7 @@ update-version.sh \ $(conf_DATA) -datafiles = \ - dovecot-config - nodist_pkginclude_HEADERS = config.h -dovecotlib_DATA = $(datafiles) if MAINTAINER_MODE ChangeLog: .hg/dirstate @@ -64,7 +57,7 @@ endif install-exec-hook: - rm $(DESTDIR)$(pkglibdir)/dovecot-config && \ + $(MKDIR_P) $(DESTDIR)$(pkglibdir); \ grep -v '^LIBDOVECOT_.*_INCLUDE' dovecot-config | \ grep -v '^LIBDOVECOT.*_DEPS' | sed \ -e "s|^\(LIBDOVECOT\)=.*$$|\1='-L$(pkglibdir) -ldovecot'|" \ From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: Makefile: Fix to previous change to get dovecot-con... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/a743cc250208 changeset: 14284:a743cc250208 user: Timo Sirainen date: Sun Mar 04 15:21:21 2012 +0200 description: Makefile: Fix to previous change to get dovecot-config file built again. diffstat: Makefile.am | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff -r 2f3c83f28dfe -r a743cc250208 Makefile.am --- a/Makefile.am Sun Mar 04 15:11:20 2012 +0200 +++ b/Makefile.am Sun Mar 04 15:21:21 2012 +0200 @@ -17,6 +17,8 @@ update-version.sh \ $(conf_DATA) +noinst_DATA = dovecot-config + nodist_pkginclude_HEADERS = config.h if MAINTAINER_MODE From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: lib-storage: Removed MAILBOX_LIST_ITER_LIST_PREFIXE... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d4f40b5c080c changeset: 14282:d4f40b5c080c user: Timo Sirainen date: Sun Mar 04 14:33:00 2012 +0200 description: lib-storage: Removed MAILBOX_LIST_ITER_LIST_PREFIXES flag. It was already accidentally enabled by default, and there's really no good reason why it shouldn't always be enabled. diffstat: src/doveadm/doveadm-mailbox-list-iter.c | 2 -- src/lib-storage/mailbox-list.h | 3 --- 2 files changed, 0 insertions(+), 5 deletions(-) diffs (25 lines): diff -r bbe6b6c2ee99 -r d4f40b5c080c src/doveadm/doveadm-mailbox-list-iter.c --- a/src/doveadm/doveadm-mailbox-list-iter.c Sun Mar 04 14:31:23 2012 +0200 +++ b/src/doveadm/doveadm-mailbox-list-iter.c Sun Mar 04 14:33:00 2012 +0200 @@ -69,8 +69,6 @@ ARRAY_TYPE(const_string) patterns; bool have_guid = FALSE; - iter_flags |= MAILBOX_LIST_ITER_LIST_PREFIXES; - iter = i_new(struct doveadm_mailbox_list_iter, 1); iter->ctx = ctx; iter->search_args = search_args; diff -r bbe6b6c2ee99 -r d4f40b5c080c src/lib-storage/mailbox-list.h --- a/src/lib-storage/mailbox-list.h Sun Mar 04 14:31:23 2012 +0200 +++ b/src/lib-storage/mailbox-list.h Sun Mar 04 14:33:00 2012 +0200 @@ -73,9 +73,6 @@ namespace prefixes, if there exists a parent namespace whose children it matches. */ MAILBOX_LIST_ITER_STAR_WITHIN_NS = 0x000010, - /* For mailbox_list_iter_init_namespaces(): List also namespace - prefixes if they match */ - MAILBOX_LIST_ITER_LIST_PREFIXES = 0x000020, /* List only subscribed mailboxes */ MAILBOX_LIST_ITER_SELECT_SUBSCRIBED = 0x000100, From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: Makefile: Link with SSL_LIBS whenever linking libss... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/e540404debb7 changeset: 14285:e540404debb7 user: Timo Sirainen date: Sun Mar 04 15:30:31 2012 +0200 description: Makefile: Link with SSL_LIBS whenever linking libssl_iostream.la diffstat: configure.in | 9 +++++++-- dovecot-config.in.in | 2 +- src/auth/Makefile.am | 2 +- src/doveadm/Makefile.am | 5 ++++- src/doveadm/dsync/Makefile.am | 3 --- src/imap/Makefile.am | 13 ++++++++----- src/indexer/Makefile.am | 13 ++++++++----- src/lda/Makefile.am | 12 +++++++++--- src/lib-storage/Makefile.am | 1 + src/lib-storage/index/Makefile.am | 3 --- src/login-common/Makefile.am | 2 +- src/plugins/fts-squat/Makefile.am | 12 +++++++----- src/pop3/Makefile.am | 13 ++++++++----- src/util/Makefile.am | 9 +++++++-- 14 files changed, 62 insertions(+), 37 deletions(-) diffs (285 lines): diff -r a743cc250208 -r e540404debb7 configure.in --- a/configure.in Sun Mar 04 15:21:21 2012 +0200 +++ b/configure.in Sun Mar 04 15:30:31 2012 +0200 @@ -2488,10 +2488,13 @@ want_ssl_libs=yes fi done +LINKED_STORAGE_LDADD= if test "$want_ssl_libs" = yes; then LINKED_STORAGE_LIBS="$LINKED_STORAGE_LIBS \$(top_builddir)/src/lib-ssl-iostream/libssl_iostream.la" + LINKED_STORAGE_LDADD="$SSL_LIBS" fi AC_SUBST(LINKED_STORAGE_LIBS) +AC_SUBST(LINKED_STORAGE_LDADD) AC_SUBST(mailbox_list_drivers) AC_DEFINE_UNQUOTED(MAIL_STORAGES, "$mail_storages", List of compiled in mail storages) @@ -2507,7 +2510,7 @@ if test "$want_shared_libs" = "yes"; then LIBDOVECOT_DEPS='$(top_builddir)/src/lib-dovecot/libdovecot.la' LIBDOVECOT="$LIBDOVECOT_DEPS" - LIBDOVECOT_STORAGE='$(top_builddir)/src/lib-storage/libdovecot-storage.la' + LIBDOVECOT_STORAGE_DEPS='$(top_builddir)/src/lib-storage/libdovecot-storage.la' LIBDOVECOT_LOGIN='$(top_builddir)/src/login-common/libdovecot-login.la' LIBDOVECOT_LDA='$(top_builddir)/src/lib-lda/libdovecot-lda.la' else @@ -2515,14 +2518,16 @@ LIBDOVECOT="$LIBDOVECOT_DEPS \$(LIBICONV)" LIBDOVECOT_STORAGE_LAST='$(top_builddir)/src/lib-storage/list/libstorage_list.la $(top_builddir)/src/lib-storage/index/libstorage_index.la $(top_builddir)/src/lib-storage/libstorage.la $(top_builddir)/src/lib-index/libindex.la' LIBDOVECOT_STORAGE_FIRST='$(top_builddir)/src/lib-storage/libstorage_service.la $(top_builddir)/src/lib-storage/register/libstorage_register.la' - LIBDOVECOT_STORAGE="$LIBDOVECOT_STORAGE_FIRST $LINKED_STORAGE_LIBS $LIBDOVECOT_STORAGE_LAST" + LIBDOVECOT_STORAGE_DEPS="$LIBDOVECOT_STORAGE_FIRST $LINKED_STORAGE_LIBS $LIBDOVECOT_STORAGE_LAST" LIBDOVECOT_LOGIN='$(top_builddir)/src/login-common/liblogin.la $(top_builddir)/src/lib-ssl-iostream/libssl_iostream.la' LIBDOVECOT_LDA='$(top_builddir)/src/lib-lda/liblda.la' fi +LIBDOVECOT_STORAGE="$LIBDOVECOT_STORAGE_DEPS $LINKED_STORAGE_LDADD" LIBDOVECOT_SQL='$(top_builddir)/src/lib-sql/libsql.la' AC_SUBST(LIBDOVECOT) AC_SUBST(LIBDOVECOT_DEPS) AC_SUBST(LIBDOVECOT_STORAGE) +AC_SUBST(LIBDOVECOT_STORAGE_DEPS) AC_SUBST(LIBDOVECOT_LOGIN) AC_SUBST(LIBDOVECOT_SQL) AC_SUBST(LIBDOVECOT_LDA) diff -r a743cc250208 -r e540404debb7 dovecot-config.in.in --- a/dovecot-config.in.in Sun Mar 04 15:21:21 2012 +0200 +++ b/dovecot-config.in.in Sun Mar 04 15:30:31 2012 +0200 @@ -13,7 +13,7 @@ LIBDOVECOT_LOGIN_DEPS="@LIBDOVECOT_LOGIN@" LIBDOVECOT_SQL_DEPS="@LIBDOVECOT_SQL@" LIBDOVECOT_LDA_DEPS="@LIBDOVECOT_LDA@" -LIBDOVECOT_STORAGE_DEPS="@LIBDOVECOT_STORAGE@" +LIBDOVECOT_STORAGE_DEPS="@LIBDOVECOT_STORAGE_DEPS@" LIBDOVECOT_INCLUDE="-I$(incdir) -I$(incdir)/src/lib -I$(incdir)/src/lib-dict -I$(incdir)/src/lib-dns -I$(incdir)/src/lib-mail -I$(incdir)/src/lib-imap -I$(incdir)/src/lib-fs -I$(incdir)/src/lib-charset" LIBDOVECOT_LDA_INCLUDE="-I$(incdir)/src/lib-lda -I$(incdir)/src/lda" diff -r a743cc250208 -r e540404debb7 src/auth/Makefile.am --- a/src/auth/Makefile.am Sun Mar 04 15:21:21 2012 +0200 +++ b/src/auth/Makefile.am Sun Mar 04 15:30:31 2012 +0200 @@ -167,7 +167,7 @@ libauthdb_imap_la_LIBADD = \ ../lib-imap-client/libimap_client.la \ ../lib-ssl-iostream/libssl_iostream.la \ - $(LIBDOVECOT) + $(LIBDOVECOT) $(SSL_LIBS) libauthdb_imap_la_CPPFLAGS = \ $(AM_CPPFLAGS) \ -I$(top_srcdir)/src/lib-imap \ diff -r a743cc250208 -r e540404debb7 src/doveadm/Makefile.am --- a/src/doveadm/Makefile.am Sun Mar 04 15:21:21 2012 +0200 +++ b/src/doveadm/Makefile.am Sun Mar 04 15:30:31 2012 +0200 @@ -39,26 +39,29 @@ libs = \ dsync/libdsync.a \ - $(LIBDOVECOT_STORAGE) \ $(unused_objects) doveadm_LDADD = \ $(libs) \ $(cmd_pw_libs) \ $(CRYPT_LIBS) \ + $(LIBDOVECOT_STORAGE) \ $(LIBDOVECOT) \ $(MODULE_LIBS) doveadm_DEPENDENCIES = \ $(libs) \ $(cmd_pw_libs) \ + $(LIBDOVECOT_STORAGE_DEPS) \ $(LIBDOVECOT_DEPS) doveadm_server_LDADD = \ $(libs) \ + $(LIBDOVECOT_STORAGE) \ $(LIBDOVECOT) \ $(MODULE_LIBS) doveadm_server_DEPENDENCIES = \ $(libs) \ + $(LIBDOVECOT_STORAGE_DEPS) \ $(LIBDOVECOT_DEPS) common = \ diff -r a743cc250208 -r e540404debb7 src/doveadm/dsync/Makefile.am --- a/src/doveadm/dsync/Makefile.am Sun Mar 04 15:21:21 2012 +0200 +++ b/src/doveadm/dsync/Makefile.am Sun Mar 04 15:30:31 2012 +0200 @@ -11,9 +11,6 @@ -I$(top_srcdir)/src/lib-storage \ -I$(top_srcdir)/src/doveadm -libs = \ - $(LIBDOVECOT_STORAGE) - libdsync_a_SOURCES = \ doveadm-dsync.c \ dsync-brain.c \ diff -r a743cc250208 -r e540404debb7 src/imap/Makefile.am --- a/src/imap/Makefile.am Sun Mar 04 15:21:21 2012 +0200 +++ b/src/imap/Makefile.am Sun Mar 04 15:30:31 2012 +0200 @@ -19,12 +19,15 @@ ../lib/mountpoint.o endif -libs = \ +imap_LDADD = \ + $(unused_objects) \ $(LIBDOVECOT_STORAGE) \ - $(unused_objects) - -imap_LDADD = $(libs) $(LIBDOVECOT) $(MODULE_LIBS) -imap_DEPENDENCIES = $(libs) $(LIBDOVECOT_DEPS) + $(LIBDOVECOT) \ + $(MODULE_LIBS) +imap_DEPENDENCIES = \ + $(unused_objects) \ + $(LIBDOVECOT_STORAGE_DEPS) \ + $(LIBDOVECOT_DEPS) cmds = \ cmd-append.c \ diff -r a743cc250208 -r e540404debb7 src/indexer/Makefile.am --- a/src/indexer/Makefile.am Sun Mar 04 15:21:21 2012 +0200 +++ b/src/indexer/Makefile.am Sun Mar 04 15:30:31 2012 +0200 @@ -27,12 +27,15 @@ ../lib-storage/mail-search-parser-imap.o endif -libs = \ +indexer_worker_LDADD = \ + $(unused_objects) \ $(LIBDOVECOT_STORAGE) \ - $(unused_objects) - -indexer_worker_LDADD = $(libs) $(LIBDOVECOT) $(MODULE_LIBS) -indexer_worker_DEPENDENCIES = $(libs) $(LIBDOVECOT_DEPS) + $(LIBDOVECOT) \ + $(MODULE_LIBS) +indexer_worker_DEPENDENCIES = \ + $(unused_objects) \ + $(LIBDOVECOT_STORAGE_DEPS) \ + $(LIBDOVECOT_DEPS) indexer_worker_SOURCES = \ indexer-worker.c \ indexer-worker-settings.c \ diff -r a743cc250208 -r e540404debb7 src/lda/Makefile.am --- a/src/lda/Makefile.am Sun Mar 04 15:21:21 2012 +0200 +++ b/src/lda/Makefile.am Sun Mar 04 15:30:31 2012 +0200 @@ -26,11 +26,17 @@ libs = \ $(unused_objects) \ - $(LIBDOVECOT_STORAGE) \ $(LIBDOVECOT_LDA) -dovecot_lda_LDADD = $(libs) $(LIBDOVECOT) $(MODULE_LIBS) -dovecot_lda_DEPENDENCIES = $(libs) $(LIBDOVECOT_DEPS) +dovecot_lda_LDADD = \ + $(libs) \ + $(LIBDOVECOT_STORAGE) \ + $(LIBDOVECOT) \ + $(MODULE_LIBS) +dovecot_lda_DEPENDENCIES = \ + $(libs) \ + $(LIBDOVECOT_STORAGE_DEPS) \ + $(LIBDOVECOT_DEPS) dovecot_lda_SOURCES = \ main.c diff -r a743cc250208 -r e540404debb7 src/lib-storage/Makefile.am --- a/src/lib-storage/Makefile.am Sun Mar 04 15:21:21 2012 +0200 +++ b/src/lib-storage/Makefile.am Sun Mar 04 15:30:31 2012 +0200 @@ -75,6 +75,7 @@ mailbox-uidvalidity.h shlibs = \ + @LINKED_STORAGE_LIBS@ \ libstorage.la \ libstorage_service.la \ list/libstorage_list.la \ diff -r a743cc250208 -r e540404debb7 src/lib-storage/index/Makefile.am --- a/src/lib-storage/index/Makefile.am Sun Mar 04 15:21:21 2012 +0200 +++ b/src/lib-storage/index/Makefile.am Sun Mar 04 15:30:31 2012 +0200 @@ -32,9 +32,6 @@ index-thread-links.c \ index-transaction.c -libstorage_index_la_LIBADD = @LINKED_STORAGE_LIBS@ -libstorage_index_la_DEPENDENCIES = @LINKED_STORAGE_LIBS@ - headers = \ istream-attachment.h \ istream-mail.h \ diff -r a743cc250208 -r e540404debb7 src/login-common/Makefile.am --- a/src/login-common/Makefile.am Sun Mar 04 15:21:21 2012 +0200 +++ b/src/login-common/Makefile.am Sun Mar 04 15:30:31 2012 +0200 @@ -40,6 +40,6 @@ pkglib_LTLIBRARIES = libdovecot-login.la libdovecot_login_la_SOURCES = -libdovecot_login_la_LIBADD = liblogin.la ../lib-ssl-iostream/libssl_iostream.la ../lib-dovecot/libdovecot.la +libdovecot_login_la_LIBADD = liblogin.la ../lib-ssl-iostream/libssl_iostream.la ../lib-dovecot/libdovecot.la $(SSL_LIBS) libdovecot_login_la_DEPENDENCIES = liblogin.la libdovecot_login_la_LDFLAGS = -export-dynamic diff -r a743cc250208 -r e540404debb7 src/plugins/fts-squat/Makefile.am --- a/src/plugins/fts-squat/Makefile.am Sun Mar 04 15:21:21 2012 +0200 +++ b/src/plugins/fts-squat/Makefile.am Sun Mar 04 15:30:31 2012 +0200 @@ -37,9 +37,11 @@ squat-trie.lo \ squat-uidlist.lo -libs = \ +squat_test_LDADD = \ + $(common_objects) \ $(LIBDOVECOT_STORAGE) \ - $(common_objects) - -squat_test_LDADD = $(libs) $(LIBDOVECOT) -squat_test_DEPENDENCIES = $(libs) $(LIBDOVECOT_DEPS) + $(LIBDOVECOT) +squat_test_DEPENDENCIES = \ + $(common_objects) \ + $(LIBDOVECOT_STORAGE_DEPS) \ + $(LIBDOVECOT_DEPS) diff -r a743cc250208 -r e540404debb7 src/pop3/Makefile.am --- a/src/pop3/Makefile.am Sun Mar 04 15:21:21 2012 +0200 +++ b/src/pop3/Makefile.am Sun Mar 04 15:30:31 2012 +0200 @@ -19,12 +19,15 @@ ../lib-storage/mail-search-parser-imap.o endif -libs = \ +pop3_LDADD = \ + $(unused_objects) \ $(LIBDOVECOT_STORAGE) \ - $(unused_objects) - -pop3_LDADD = $(libs) $(LIBDOVECOT) $(MODULE_LIBS) -pop3_DEPENDENCIES = $(libs) $(LIBDOVECOT_DEPS) + $(LIBDOVECOT) \ + $(MODULE_LIBS) +pop3_DEPENDENCIES = \ + $(unused_objects) \ + $(LIBDOVECOT_STORAGE_DEPS) \ + $(LIBDOVECOT_DEPS) pop3_SOURCES = \ main.c \ diff -r a743cc250208 -r e540404debb7 src/util/Makefile.am --- a/src/util/Makefile.am Sun Mar 04 15:21:21 2012 +0200 +++ b/src/util/Makefile.am Sun Mar 04 15:30:31 2012 +0200 @@ -27,8 +27,13 @@ rawlog_SOURCES = \ rawlog.c -script_login_LDADD = $(LIBDOVECOT_STORAGE) $(LIBDOVECOT) $(MODULE_LIBS) -script_login_DEPENDENCIES = $(LIBDOVECOT_STORAGE) $(LIBDOVECOT_DEPS) +script_login_LDADD = \ + $(LIBDOVECOT_STORAGE) \ + $(LIBDOVECOT) \ + $(MODULE_LIBS) +script_login_DEPENDENCIES = \ + $(LIBDOVECOT_STORAGE_DEPS) \ + $(LIBDOVECOT_DEPS) script_login_SOURCES = \ script-login.c From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: Don't auto-add mountpoints under /media or /cdrom. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/a63651897eac changeset: 14286:a63651897eac user: Timo Sirainen date: Sun Mar 04 15:48:48 2012 +0200 description: Don't auto-add mountpoints under /media or /cdrom. diffstat: src/doveadm/doveadm-mount.c | 6 ++++-- src/lib-master/mountpoint-list.c | 30 ++++++++++++++++++++++++------ src/lib-master/mountpoint-list.h | 7 ++++++- src/master/main.c | 1 + 4 files changed, 35 insertions(+), 9 deletions(-) diffs (113 lines): diff -r e540404debb7 -r a63651897eac src/doveadm/doveadm-mount.c --- a/src/doveadm/doveadm-mount.c Sun Mar 04 15:30:31 2012 +0200 +++ b/src/doveadm/doveadm-mount.c Sun Mar 04 15:48:48 2012 +0200 @@ -77,8 +77,10 @@ mountpoints = mountpoint_list_get(); if (argv[1] == NULL) { - mountpoint_list_add_missing(mountpoints, MOUNTPOINT_STATE_DEFAULT, - mountpoint_list_default_ignore_types); + mountpoint_list_add_missing(mountpoints, + MOUNTPOINT_STATE_DEFAULT, + mountpoint_list_default_ignore_prefixes, + mountpoint_list_default_ignore_types); } else { memset(&rec, 0, sizeof(rec)); rec.mount_path = argv[1]; diff -r e540404debb7 -r a63651897eac src/lib-master/mountpoint-list.c --- a/src/lib-master/mountpoint-list.c Sun Mar 04 15:30:31 2012 +0200 +++ b/src/lib-master/mountpoint-list.c Sun Mar 04 15:48:48 2012 +0200 @@ -50,6 +50,12 @@ NULL }; +const char *const mountpoint_list_default_ignore_prefixes[] = { + "/cdrom", + "/media", + NULL +}; + struct mountpoint_list * mountpoint_list_init(const char *perm_path, const char *state_path) { @@ -251,8 +257,20 @@ return FALSE; } +static bool str_array_find_prefix(const char *const *prefixes, const char *str) +{ + if (prefixes == NULL) + return FALSE; + for (; *prefixes != NULL; prefixes++) { + if (strncmp(*prefixes, str, strlen(*prefixes)) == 0) + return TRUE; + } + return FALSE; +} + int mountpoint_list_add_missing(struct mountpoint_list *list, const char *default_state, + const char *const *ignore_prefixes, const char *const *ignore_types) { struct mountpoint_list_rec new_rec, *rec, *const *recp; @@ -269,15 +287,15 @@ /* get a sorted list of all current mountpoints */ iter = mountpoint_iter_init(); while ((mnt = mountpoint_iter_next(iter)) != NULL) { - if (str_array_find(ignore_types, mnt->type)) - continue; - rec = mountpoint_list_find(list, mnt->mount_path); - if (rec == NULL) { + if (rec != NULL) { + if (!rec->wildcard) + rec->mounted = TRUE; + } else if (!str_array_find(ignore_types, mnt->type) && + !str_array_find_prefix(ignore_prefixes, + mnt->mount_path)) { new_rec.mount_path = mnt->mount_path; mountpoint_list_add(list, &new_rec); - } else if (!rec->wildcard) { - rec->mounted = TRUE; } } return mountpoint_iter_deinit(&iter); diff -r e540404debb7 -r a63651897eac src/lib-master/mountpoint-list.h --- a/src/lib-master/mountpoint-list.h Sun Mar 04 15:30:31 2012 +0200 +++ b/src/lib-master/mountpoint-list.h Sun Mar 04 15:48:48 2012 +0200 @@ -21,6 +21,9 @@ /* A default known good list of mountpoint types that don't contain emails (e.g. proc, tmpfs, etc.) */ extern const char *const mountpoint_list_default_ignore_types[]; +/* A default known good list of directories which shouldn't contain emails + (e.g. /media) */ +extern const char *const mountpoint_list_default_ignore_prefixes[]; struct mountpoint_list * mountpoint_list_init(const char *perm_path, const char *state_path); @@ -43,10 +46,12 @@ const char *mount_path); /* Add all currently mounted missing mountpoints to the list and update all mountpoints' mounted state. The mountpoints that match existing wildcards - aren't added. Mountpoints with type in ignore_types list also aren't added. + aren't added. Mountpoints with paths under ignore_prefixes aren't added. + Mountpoints with type in ignore_types list also aren't added. Returns 0 if we successfully iterated through all mountpoints, -1 if not. */ int mountpoint_list_add_missing(struct mountpoint_list *list, const char *default_state, + const char *const *ignore_prefixes, const char *const *ignore_types); /* Update "mounted" status for all mountpoints. */ int mountpoint_list_update_mounted(struct mountpoint_list *list); diff -r e540404debb7 -r a63651897eac src/master/main.c --- a/src/master/main.c Sun Mar 04 15:30:31 2012 +0200 +++ b/src/master/main.c Sun Mar 04 15:48:48 2012 +0200 @@ -317,6 +317,7 @@ mountpoints = mountpoint_list_init(perm_path, state_path); if (mountpoint_list_add_missing(mountpoints, MOUNTPOINT_STATE_DEFAULT, + mountpoint_list_default_ignore_prefixes, mountpoint_list_default_ignore_types) == 0) mountpoints_warn_missing(mountpoints); (void)mountpoint_list_save(mountpoints); From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: dsync: Set user.dsyncing=TRUE also for dsync-server. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/5429dac9830c changeset: 14288:5429dac9830c user: Timo Sirainen date: Mon Mar 05 11:54:33 2012 +0200 description: dsync: Set user.dsyncing=TRUE also for dsync-server. This avoids dsync server immediately triggering another dsync when using replication. diffstat: src/doveadm/dsync/doveadm-dsync.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff -r f549cd60fec9 -r 5429dac9830c src/doveadm/dsync/doveadm-dsync.c --- a/src/doveadm/dsync/doveadm-dsync.c Mon Mar 05 11:53:45 2012 +0200 +++ b/src/doveadm/dsync/doveadm-dsync.c Mon Mar 05 11:54:33 2012 +0200 @@ -561,6 +561,7 @@ int lock_fd, ret = 0; user->admin = TRUE; + user->dsyncing = TRUE; i_set_failure_prefix(t_strdup_printf("dsync-remote(%s): ", user->username)); From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: Makefile: Fixed compiling lmtp with some SSL_LIBS o... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/32edf12d7624 changeset: 14289:32edf12d7624 user: Timo Sirainen date: Mon Mar 05 13:08:14 2012 +0200 description: Makefile: Fixed compiling lmtp with some SSL_LIBS options. diffstat: src/lmtp/Makefile.am | 12 +++++++++--- 1 files changed, 9 insertions(+), 3 deletions(-) diffs (24 lines): diff -r 5429dac9830c -r 32edf12d7624 src/lmtp/Makefile.am --- a/src/lmtp/Makefile.am Mon Mar 05 11:54:33 2012 +0200 +++ b/src/lmtp/Makefile.am Mon Mar 05 13:08:14 2012 +0200 @@ -29,11 +29,17 @@ libs = \ $(unused_objects) \ - $(LIBDOVECOT_STORAGE) \ $(LIBDOVECOT_LDA) -lmtp_LDADD = $(libs) $(LIBDOVECOT) $(MODULE_LIBS) -lmtp_DEPENDENCIES = $(libs) $(LIBDOVECOT_DEPS) +lmtp_LDADD = \ + $(libs) \ + $(LIBDOVECOT_STORAGE) \ + $(LIBDOVECOT) \ + $(MODULE_LIBS) +lmtp_DEPENDENCIES = \ + $(libs) \ + $(LIBDOVECOT_STORAGE_DEPS) \ + $(LIBDOVECOT_DEPS) lmtp_SOURCES = \ main.c \ From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: dsync: If mailbox was expunged empty, the messages ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/f549cd60fec9 changeset: 14287:f549cd60fec9 user: Timo Sirainen date: Mon Mar 05 11:53:45 2012 +0200 description: dsync: If mailbox was expunged empty, the messages may have reappeared. This dependend on what the last UID of the previous mailbox was. diffstat: src/doveadm/dsync/dsync-worker-local.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff -r a63651897eac -r f549cd60fec9 src/doveadm/dsync/dsync-worker-local.c --- a/src/doveadm/dsync/dsync-worker-local.c Sun Mar 04 15:48:48 2012 +0200 +++ b/src/doveadm/dsync/dsync-worker-local.c Mon Mar 05 11:53:45 2012 +0200 @@ -850,6 +850,7 @@ static void iter_local_mailbox_close(struct local_dsync_worker_msg_iter *iter) { + iter->prev_uid = 0; iter->expunges_set = FALSE; if (mailbox_search_deinit(&iter->search_ctx) < 0) { i_error("msg search failed: %s", From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: aggregator: Fixed leaking connections to replicator. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/7131eb54e9cf changeset: 14290:7131eb54e9cf user: Timo Sirainen date: Mon Mar 05 13:18:28 2012 +0200 description: aggregator: Fixed leaking connections to replicator. diffstat: src/replication/aggregator/replicator-connection.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diffs (21 lines): diff -r 32edf12d7624 -r 7131eb54e9cf src/replication/aggregator/replicator-connection.c --- a/src/replication/aggregator/replicator-connection.c Mon Mar 05 13:08:14 2012 +0200 +++ b/src/replication/aggregator/replicator-connection.c Mon Mar 05 13:18:28 2012 +0200 @@ -137,6 +137,9 @@ unsigned int n; int fd = -1; + if (conn->fd != -1) + return; + if (conn->port == 0) { fd = net_connect_unix(conn->path); if (fd == -1) @@ -195,6 +198,7 @@ i_stream_destroy(&conn->input); o_stream_destroy(&conn->output); net_disconnect(conn->fd); + conn->fd = -1; } static struct replicator_connection *replicator_connection_create(void) From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: replication_max_conns setting is a number, not a ti... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/005bcb8d8d02 changeset: 14291:005bcb8d8d02 user: Timo Sirainen date: Mon Mar 05 14:27:02 2012 +0200 description: replication_max_conns setting is a number, not a time interval. diffstat: src/replication/replicator/replicator-settings.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 7131eb54e9cf -r 005bcb8d8d02 src/replication/replicator/replicator-settings.c --- a/src/replication/replicator/replicator-settings.c Mon Mar 05 13:18:28 2012 +0200 +++ b/src/replication/replicator/replicator-settings.c Mon Mar 05 14:27:02 2012 +0200 @@ -53,7 +53,7 @@ DEF(SET_STR, doveadm_socket_path), DEF(SET_TIME, replication_full_sync_interval), - DEF(SET_TIME, replication_max_conns), + DEF(SET_UINT, replication_max_conns), SETTING_DEFINE_LIST_END }; From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: Added director_username_hash setting to specify wha... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/817ef4c9f1f3 changeset: 14292:817ef4c9f1f3 user: Timo Sirainen date: Mon Mar 05 17:28:06 2012 +0200 description: Added director_username_hash setting to specify what part of the username is hashed. The default is "%u" meaning the full username. Another potentially useful value is "%d" for hashing only the domain (i.e. redirect users from the same domain always to same server so they can safely access each others' shared mailboxes). diffstat: src/director/Makefile.am | 3 +- src/director/director-request.c | 3 +- src/director/director-settings.c | 2 + src/director/director-settings.h | 1 + src/director/director.c | 3 +- src/director/doveadm-connection.c | 4 +- src/director/notify-connection.c | 2 +- src/director/user-directory.c | 21 ++++++++----------- src/director/user-directory.h | 6 +++- src/lib-mail/Makefile.am | 2 + src/lib-mail/mail-user-hash.c | 41 +++++++++++++++++++++++++++++++++++++++ src/lib-mail/mail-user-hash.h | 8 +++++++ src/login-common/Makefile.am | 1 + src/login-common/login-proxy.c | 19 +++++------------ src/login-common/login-settings.c | 2 + src/login-common/login-settings.h | 1 + 16 files changed, 86 insertions(+), 33 deletions(-) diffs (truncated from 365 to 300 lines): diff -r 005bcb8d8d02 -r 817ef4c9f1f3 src/director/Makefile.am --- a/src/director/Makefile.am Mon Mar 05 14:27:02 2012 +0200 +++ b/src/director/Makefile.am Mon Mar 05 17:28:06 2012 +0200 @@ -7,7 +7,8 @@ -I$(top_srcdir)/src/lib-auth \ -I$(top_srcdir)/src/lib-imap \ -I$(top_srcdir)/src/lib-settings \ - -I$(top_srcdir)/src/lib-master + -I$(top_srcdir)/src/lib-master \ + -I$(top_srcdir)/src/lib-mail director_LDADD = $(LIBDOVECOT) director_DEPENDENCIES = $(LIBDOVECOT_DEPS) diff -r 005bcb8d8d02 -r 817ef4c9f1f3 src/director/director-request.c --- a/src/director/director-request.c Mon Mar 05 14:27:02 2012 +0200 +++ b/src/director/director-request.c Mon Mar 05 17:28:06 2012 +0200 @@ -73,7 +73,8 @@ director_request_callback *callback, void *context) { struct director_request *request; - unsigned int username_hash = user_directory_get_username_hash(username); + unsigned int username_hash = + user_directory_get_username_hash(dir->users, username); request = i_new(struct director_request, 1); request->dir = dir; diff -r 005bcb8d8d02 -r 817ef4c9f1f3 src/director/director-settings.c --- a/src/director/director-settings.c Mon Mar 05 14:27:02 2012 +0200 +++ b/src/director/director-settings.c Mon Mar 05 17:28:06 2012 +0200 @@ -68,6 +68,7 @@ DEF(SET_STR, director_servers), DEF(SET_STR, director_mail_servers), + DEF(SET_STR, director_username_hash), DEF(SET_TIME, director_user_expire), DEF(SET_UINT, director_doveadm_port), @@ -79,6 +80,7 @@ .director_servers = "", .director_mail_servers = "", + .director_username_hash = "%u", .director_user_expire = 60*15, .director_doveadm_port = 0 }; diff -r 005bcb8d8d02 -r 817ef4c9f1f3 src/director/director-settings.h --- a/src/director/director-settings.h Mon Mar 05 14:27:02 2012 +0200 +++ b/src/director/director-settings.h Mon Mar 05 17:28:06 2012 +0200 @@ -6,6 +6,7 @@ const char *director_servers; const char *director_mail_servers; + const char *director_username_hash; unsigned int director_user_expire; unsigned int director_doveadm_port; }; diff -r 005bcb8d8d02 -r 817ef4c9f1f3 src/director/director.c --- a/src/director/director.c Mon Mar 05 14:27:02 2012 +0200 +++ b/src/director/director.c Mon Mar 05 17:28:06 2012 +0200 @@ -625,7 +625,8 @@ dir->state_change_callback = callback; i_array_init(&dir->dir_hosts, 16); i_array_init(&dir->pending_requests, 16); - dir->users = user_directory_init(set->director_user_expire); + dir->users = user_directory_init(set->director_user_expire, + set->director_username_hash); dir->mail_hosts = mail_hosts_init(); dir->ipc_proxy = ipc_client_init(DIRECTOR_IPC_PROXY_PATH); diff -r 005bcb8d8d02 -r 817ef4c9f1f3 src/director/doveadm-connection.c --- a/src/director/doveadm-connection.c Mon Mar 05 14:27:02 2012 +0200 +++ b/src/director/doveadm-connection.c Mon Mar 05 17:28:06 2012 +0200 @@ -223,7 +223,7 @@ string_t *str = t_str_new(256); if (str_to_uint(line, &username_hash) < 0) - username_hash = user_directory_get_username_hash(line); + username_hash = user_directory_get_username_hash(conn->dir->users, line); /* get user's current host */ user = user_directory_lookup(conn->dir->users, username_hash); @@ -309,7 +309,7 @@ } if (str_to_uint(args[0], &username_hash) < 0) - username_hash = user_directory_get_username_hash(line); + username_hash = user_directory_get_username_hash(conn->dir->users, line); user = user_directory_lookup(conn->dir->users, username_hash); if (user != NULL && user->kill_state != USER_KILL_STATE_NONE) { o_stream_send_str(conn->output, "TRYAGAIN\n"); diff -r 005bcb8d8d02 -r 817ef4c9f1f3 src/director/notify-connection.c --- a/src/director/notify-connection.c Mon Mar 05 14:27:02 2012 +0200 +++ b/src/director/notify-connection.c Mon Mar 05 17:28:06 2012 +0200 @@ -24,7 +24,7 @@ unsigned int hash; while ((line = i_stream_read_next_line(conn->input)) != NULL) { - hash = user_directory_get_username_hash(line); + hash = user_directory_get_username_hash(conn->dir->users, line); user = user_directory_lookup(conn->dir->users, hash); if (user != NULL) { user_directory_refresh(conn->dir->users, user); diff -r 005bcb8d8d02 -r 817ef4c9f1f3 src/director/user-directory.c --- a/src/director/user-directory.c Mon Mar 05 14:27:02 2012 +0200 +++ b/src/director/user-directory.c Mon Mar 05 17:28:06 2012 +0200 @@ -3,9 +3,9 @@ #include "lib.h" #include "ioloop.h" #include "array.h" -#include "md5.h" #include "hash.h" #include "llist.h" +#include "mail-user-hash.h" #include "mail-host.h" #include "user-directory.h" @@ -24,6 +24,7 @@ ARRAY_DEFINE(iters, struct user_directory_iter *); + char *username_hash_fmt; unsigned int timeout_secs; }; @@ -120,17 +121,10 @@ } } -unsigned int user_directory_get_username_hash(const char *username) +unsigned int user_directory_get_username_hash(struct user_directory *dir, + const char *username) { - /* NOTE: If you modify this, modify also - director_username_hash() in login-common/login-proxy.c */ - unsigned char md5[MD5_RESULTLEN]; - unsigned int i, hash = 0; - - md5_get_digest(username, strlen(username), md5); - for (i = 0; i < sizeof(hash); i++) - hash = (hash << CHAR_BIT) | md5[i]; - return hash; + return mail_user_hash(username, dir->username_hash_fmt); } bool user_directory_user_has_connections(struct user_directory *dir, @@ -141,12 +135,14 @@ return expire_timestamp - MAX_CLOCK_DRIFT_SECS >= ioloop_time; } -struct user_directory *user_directory_init(unsigned int timeout_secs) +struct user_directory * +user_directory_init(unsigned int timeout_secs, const char *username_hash_fmt) { struct user_directory *dir; dir = i_new(struct user_directory, 1); dir->timeout_secs = timeout_secs; + dir->username_hash_fmt = i_strdup(username_hash_fmt); dir->hash = hash_table_create(default_pool, default_pool, 0, NULL, NULL); i_array_init(&dir->iters, 8); @@ -165,6 +161,7 @@ user_free(dir, dir->head); hash_table_destroy(&dir->hash); array_free(&dir->iters); + i_free(dir->username_hash_fmt); i_free(dir); } diff -r 005bcb8d8d02 -r 817ef4c9f1f3 src/director/user-directory.h --- a/src/director/user-directory.h Mon Mar 05 14:27:02 2012 +0200 +++ b/src/director/user-directory.h Mon Mar 05 17:28:06 2012 +0200 @@ -42,7 +42,8 @@ /* Create a new directory. Users are dropped if their time gets older than timeout_secs. */ -struct user_directory *user_directory_init(unsigned int timeout_secs); +struct user_directory * +user_directory_init(unsigned int timeout_secs, const char *username_hash_fmt); void user_directory_deinit(struct user_directory **dir); /* Look up username from directory. Returns NULL if not found. */ @@ -59,7 +60,8 @@ void user_directory_remove_host(struct user_directory *dir, struct mail_host *host); -unsigned int user_directory_get_username_hash(const char *username); +unsigned int user_directory_get_username_hash(struct user_directory *dir, + const char *username); /* Returns TRUE if user still potentially has connections. */ bool user_directory_user_has_connections(struct user_directory *dir, diff -r 005bcb8d8d02 -r 817ef4c9f1f3 src/lib-mail/Makefile.am --- a/src/lib-mail/Makefile.am Mon Mar 05 14:27:02 2012 +0200 +++ b/src/lib-mail/Makefile.am Mon Mar 05 17:28:06 2012 +0200 @@ -8,6 +8,7 @@ libmail_la_SOURCES = \ istream-dot.c \ istream-header-filter.c \ + mail-user-hash.c \ mbox-from.c \ message-address.c \ message-date.c \ @@ -28,6 +29,7 @@ headers = \ istream-dot.h \ istream-header-filter.h \ + mail-user-hash.h \ mbox-from.h \ mail-types.h \ message-address.h \ diff -r 005bcb8d8d02 -r 817ef4c9f1f3 src/lib-mail/mail-user-hash.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/lib-mail/mail-user-hash.c Mon Mar 05 17:28:06 2012 +0200 @@ -0,0 +1,41 @@ +/* Copyright (c) 2012 Dovecot authors, see the included COPYING file */ + +#include "lib.h" +#include "md5.h" +#include "str.h" +#include "var-expand.h" +#include "mail-user-hash.h" + +unsigned int mail_user_hash(const char *username, const char *format) +{ + static struct var_expand_table static_tab[] = { + { 'u', NULL, "user" }, + { 'n', NULL, "username" }, + { 'd', NULL, "domain" }, + { '\0', NULL, NULL } + }; + struct var_expand_table *tab; + unsigned char md5[MD5_RESULTLEN]; + unsigned int i, hash = 0; + + if (strcmp(format, "%u") == 0) { + /* fast path */ + md5_get_digest(username, strlen(username), md5); + } else T_BEGIN { + string_t *str = t_str_new(128); + + tab = t_malloc(sizeof(static_tab)); + memcpy(tab, static_tab, sizeof(static_tab)); + tab[0].value = username; + tab[1].value = t_strcut(username, '@'); + tab[2].value = strchr(username, '@'); + if (tab[2].value != NULL) tab[2].value++; + + var_expand(str, format, tab); + md5_get_digest(str_data(str), str_len(str), md5); + } T_END; + for (i = 0; i < sizeof(hash); i++) + hash = (hash << CHAR_BIT) | md5[i]; + return hash; +} + diff -r 005bcb8d8d02 -r 817ef4c9f1f3 src/lib-mail/mail-user-hash.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/lib-mail/mail-user-hash.h Mon Mar 05 17:28:06 2012 +0200 @@ -0,0 +1,8 @@ +#ifndef MAIL_USER_HASH +#define MAIL_USER_HASH + +/* Return a hash for username, based on given format. The format can use + %n, %d and %u variables. */ +unsigned int mail_user_hash(const char *username, const char *format); + +#endif diff -r 005bcb8d8d02 -r 817ef4c9f1f3 src/login-common/Makefile.am --- a/src/login-common/Makefile.am Mon Mar 05 14:27:02 2012 +0200 +++ b/src/login-common/Makefile.am Mon Mar 05 17:28:06 2012 +0200 @@ -7,6 +7,7 @@ -I$(top_srcdir)/src/lib-dns \ -I$(top_srcdir)/src/lib-master \ -I$(top_srcdir)/src/lib-ssl-iostream \ + -I$(top_srcdir)/src/lib-mail \ -DPKG_STATEDIR=\""$(statedir)"\" liblogin_la_SOURCES = \ diff -r 005bcb8d8d02 -r 817ef4c9f1f3 src/login-common/login-proxy.c --- a/src/login-common/login-proxy.c Mon Mar 05 14:27:02 2012 +0200 +++ b/src/login-common/login-proxy.c Mon Mar 05 17:28:06 2012 +0200 @@ -5,12 +5,12 @@ #include "istream.h" #include "ostream.h" #include "llist.h" -#include "md5.h" #include "str-sanitize.h" #include "time-util.h" #include "master-service.h" #include "ipc-server.h" #include "dns-lookup.h" +#include "mail-user-hash.h" #include "client-common.h" #include "ssl-proxy.h" #include "login-proxy-state.h" @@ -617,17 +617,10 @@ ipc_cmd_success_reply(&cmd, t_strdup_printf("%u", count)); } From dovecot at dovecot.org Sun May 20 03:26:25 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:25 +0300 Subject: dovecot-2.2: stats: Update stats once per second for long runnin... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d66568d34e40 changeset: 14293:d66568d34e40 user: Timo Sirainen date: Mon Mar 05 18:09:45 2012 +0200 description: stats: Update stats once per second for long running nonblocking searches. diffstat: src/plugins/stats/stats-plugin.c | 164 ++++++++++++++++++++++---------------- src/plugins/stats/stats-plugin.h | 1 + 2 files changed, 94 insertions(+), 71 deletions(-) diffs (199 lines): diff -r 817ef4c9f1f3 -r d66568d34e40 src/plugins/stats/stats-plugin.c --- a/src/plugins/stats/stats-plugin.c Mon Mar 05 17:28:06 2012 +0200 +++ b/src/plugins/stats/stats-plugin.c Mon Mar 05 18:09:45 2012 +0200 @@ -165,77 +165,6 @@ user_trans_stats_get(suser, &stats_r->trans_stats); } -static struct mailbox_transaction_context * -stats_transaction_begin(struct mailbox *box, - enum mailbox_transaction_flags flags) -{ - struct stats_user *suser = STATS_USER_CONTEXT(box->storage->user); - struct stats_mailbox *sbox = STATS_CONTEXT(box); - struct mailbox_transaction_context *trans; - struct stats_transaction_context *strans; - - trans = sbox->module_ctx.super.transaction_begin(box, flags); - trans->stats_track = TRUE; - - strans = i_new(struct stats_transaction_context, 1); - strans->trans = trans; - DLLIST_PREPEND(&suser->transactions, strans); - - MODULE_CONTEXT_SET(trans, stats_storage_module, strans); - return trans; -} - -static void stats_transaction_free(struct stats_user *suser, - struct stats_transaction_context *strans) -{ - DLLIST_REMOVE(&suser->transactions, strans); - - trans_stats_add(&suser->session_stats.trans_stats, - &strans->trans->stats); -} - -static int -stats_transaction_commit(struct mailbox_transaction_context *ctx, - struct mail_transaction_commit_changes *changes_r) -{ - struct stats_transaction_context *strans = STATS_CONTEXT(ctx); - struct stats_mailbox *sbox = STATS_CONTEXT(ctx->box); - struct stats_user *suser = STATS_USER_CONTEXT(ctx->box->storage->user); - - stats_transaction_free(suser, strans); - return sbox->module_ctx.super.transaction_commit(ctx, changes_r); -} - -static void -stats_transaction_rollback(struct mailbox_transaction_context *ctx) -{ - struct stats_transaction_context *strans = STATS_CONTEXT(ctx); - struct stats_mailbox *sbox = STATS_CONTEXT(ctx->box); - struct stats_user *suser = STATS_USER_CONTEXT(ctx->box->storage->user); - - stats_transaction_free(suser, strans); - sbox->module_ctx.super.transaction_rollback(ctx); -} - -static void stats_mailbox_allocated(struct mailbox *box) -{ - struct mailbox_vfuncs *v = box->vlast; - struct stats_mailbox *sbox; - struct stats_user *suser = STATS_USER_CONTEXT(box->storage->user); - - if (suser == NULL) - return; - - sbox = p_new(box->pool, struct stats_mailbox, 1); - sbox->module_ctx.super = *v; - box->vlast = &sbox->module_ctx.super; - - v->transaction_begin = stats_transaction_begin; - v->transaction_commit = stats_transaction_commit; - v->transaction_rollback = stats_transaction_rollback; - MODULE_CONTEXT_SET(box, stats_storage_module, sbox); -} - static void stats_io_activate(void *context) { struct mail_user *user = context; @@ -398,6 +327,7 @@ bool changed; if (session_stats_need_send(suser, &changed, &to_next_secs)) { + suser->last_refresh = time(NULL); suser->session_sent_duplicate = !changed; suser->last_session_update = ioloop_time; suser->last_sent_session_stats = suser->session_stats; @@ -412,6 +342,98 @@ session_stats_refresh_timeout, user); } +static struct mailbox_transaction_context * +stats_transaction_begin(struct mailbox *box, + enum mailbox_transaction_flags flags) +{ + struct stats_user *suser = STATS_USER_CONTEXT(box->storage->user); + struct stats_mailbox *sbox = STATS_CONTEXT(box); + struct mailbox_transaction_context *trans; + struct stats_transaction_context *strans; + + trans = sbox->module_ctx.super.transaction_begin(box, flags); + trans->stats_track = TRUE; + + strans = i_new(struct stats_transaction_context, 1); + strans->trans = trans; + DLLIST_PREPEND(&suser->transactions, strans); + + MODULE_CONTEXT_SET(trans, stats_storage_module, strans); + return trans; +} + +static void stats_transaction_free(struct stats_user *suser, + struct stats_transaction_context *strans) +{ + DLLIST_REMOVE(&suser->transactions, strans); + + trans_stats_add(&suser->session_stats.trans_stats, + &strans->trans->stats); +} + +static int +stats_transaction_commit(struct mailbox_transaction_context *ctx, + struct mail_transaction_commit_changes *changes_r) +{ + struct stats_transaction_context *strans = STATS_CONTEXT(ctx); + struct stats_mailbox *sbox = STATS_CONTEXT(ctx->box); + struct stats_user *suser = STATS_USER_CONTEXT(ctx->box->storage->user); + + stats_transaction_free(suser, strans); + return sbox->module_ctx.super.transaction_commit(ctx, changes_r); +} + +static void +stats_transaction_rollback(struct mailbox_transaction_context *ctx) +{ + struct stats_transaction_context *strans = STATS_CONTEXT(ctx); + struct stats_mailbox *sbox = STATS_CONTEXT(ctx->box); + struct stats_user *suser = STATS_USER_CONTEXT(ctx->box->storage->user); + + stats_transaction_free(suser, strans); + sbox->module_ctx.super.transaction_rollback(ctx); +} + +static bool stats_search_next_nonblock(struct mail_search_context *ctx, + struct mail **mail_r, bool *tryagain_r) +{ + struct stats_mailbox *sbox = STATS_CONTEXT(ctx->transaction->box); + struct mail_user *user = ctx->transaction->box->storage->user; + struct stats_user *suser = STATS_USER_CONTEXT(user); + bool ret; + + ret = sbox->module_ctx.super. + search_next_nonblock(ctx, mail_r, tryagain_r); + if (ret || !*tryagain_r) + return ret; + + /* retrying, so this is a long running search. update the stats once + a second */ + if (time(NULL) != suser->last_refresh) + session_stats_refresh(user); + return FALSE; +} + +static void stats_mailbox_allocated(struct mailbox *box) +{ + struct mailbox_vfuncs *v = box->vlast; + struct stats_mailbox *sbox; + struct stats_user *suser = STATS_USER_CONTEXT(box->storage->user); + + if (suser == NULL) + return; + + sbox = p_new(box->pool, struct stats_mailbox, 1); + sbox->module_ctx.super = *v; + box->vlast = &sbox->module_ctx.super; + + v->transaction_begin = stats_transaction_begin; + v->transaction_commit = stats_transaction_commit; + v->transaction_rollback = stats_transaction_rollback; + v->search_next_nonblock = stats_search_next_nonblock; + MODULE_CONTEXT_SET(box, stats_storage_module, sbox); +} + static void session_stats_refresh_timeout(struct mail_user *user) { if (stats_global_user != NULL) diff -r 817ef4c9f1f3 -r d66568d34e40 src/plugins/stats/stats-plugin.h --- a/src/plugins/stats/stats-plugin.h Mon Mar 05 17:28:06 2012 +0200 +++ b/src/plugins/stats/stats-plugin.h Mon Mar 05 18:09:45 2012 +0200 @@ -31,6 +31,7 @@ struct stats_connection *stats_conn; guid_128_t session_guid; + time_t last_refresh; unsigned int refresh_secs; bool track_commands; From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: pop3: Added assert. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/8a47994cd509 changeset: 14294:8a47994cd509 user: Timo Sirainen date: Mon Mar 05 18:41:53 2012 +0200 description: pop3: Added assert. diffstat: src/pop3/pop3-client.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff -r d66568d34e40 -r 8a47994cd509 src/pop3/pop3-client.c --- a/src/pop3/pop3-client.c Mon Mar 05 18:09:45 2012 +0200 +++ b/src/pop3/pop3-client.c Mon Mar 05 18:41:53 2012 +0200 @@ -188,6 +188,7 @@ array_free(&msgnum_to_seq_map); return ret; } + i_assert(msgnum == client->messages_count); client->trans = t; client->message_sizes = From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: lib-storage: When searching with a sort program, do... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/0c3cb8976e81 changeset: 14295:0c3cb8976e81 user: Timo Sirainen date: Mon Mar 05 18:43:05 2012 +0200 description: lib-storage: When searching with a sort program, don't prefetch any mails. The prefetching probably ends up being pointless. diffstat: src/lib-storage/index/index-search.c | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diffs (18 lines): diff -r 8a47994cd509 -r 0c3cb8976e81 src/lib-storage/index/index-search.c --- a/src/lib-storage/index/index-search.c Mon Mar 05 18:41:53 2012 +0200 +++ b/src/lib-storage/index/index-search.c Mon Mar 05 18:43:05 2012 +0200 @@ -1501,6 +1501,14 @@ ret = search_more_with_mail(ctx, mail); if (ret <= 0) break; + + if (ctx->mail_ctx.sort_program != NULL) { + /* don't prefetch when using a sort program, + since the mails' access order will change */ + i_assert(ctx->unused_mail_idx == 0); + *mail_r = mail; + return 1; + } if (mail_prefetch(mail) && ctx->unused_mail_idx == 0) { /* no prefetching done, return it immediately */ *mail_r = mail; From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: stats: More changes to send stats while doing long ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/721e127e107f changeset: 14296:721e127e107f user: Timo Sirainen date: Mon Mar 05 18:44:01 2012 +0200 description: stats: More changes to send stats while doing long running searches. diffstat: src/plugins/stats/stats-plugin.c | 39 +++++++++++++++++++++++---------------- src/plugins/stats/stats-plugin.h | 2 +- 2 files changed, 24 insertions(+), 17 deletions(-) diffs (116 lines): diff -r 0c3cb8976e81 -r 721e127e107f src/plugins/stats/stats-plugin.c --- a/src/plugins/stats/stats-plugin.c Mon Mar 05 18:43:05 2012 +0200 +++ b/src/plugins/stats/stats-plugin.c Mon Mar 05 18:44:01 2012 +0200 @@ -18,6 +18,7 @@ /* If session isn't refreshed every 15 minutes, it's dropped. Must be smaller than MAIL_SESSION_IDLE_TIMEOUT_MSECS in stats server */ #define SESSION_STATS_FORCE_REFRESH_SECS (5*60) +#define REFRESH_CHECK_INTERVAL 100 #define MAIL_STATS_SOCKET_NAME "stats-mail" #define USECS_PER_SEC 1000000 @@ -287,8 +288,9 @@ return FALSE; } -static bool session_stats_need_send(struct stats_user *suser, bool *changed_r, - unsigned int *to_next_secs_r) +static bool +session_stats_need_send(struct stats_user *suser, time_t now, + bool *changed_r, unsigned int *to_next_secs_r) { unsigned int diff; @@ -303,7 +305,7 @@ *changed_r = FALSE; if (!suser->session_sent_duplicate) { - if (suser->last_session_update != ioloop_time) { + if (suser->last_session_update != now) { /* send one duplicate notification so stats reader knows that this session is idle now */ return TRUE; @@ -312,7 +314,7 @@ return FALSE; } - diff = ioloop_time - suser->last_session_update; + diff = now - suser->last_session_update; if (diff < SESSION_STATS_FORCE_REFRESH_SECS) { *to_next_secs_r = SESSION_STATS_FORCE_REFRESH_SECS - diff; return FALSE; @@ -324,12 +326,12 @@ { struct stats_user *suser = STATS_USER_CONTEXT(user); unsigned int to_next_secs; + time_t now = time(NULL); bool changed; - if (session_stats_need_send(suser, &changed, &to_next_secs)) { - suser->last_refresh = time(NULL); + if (session_stats_need_send(suser, now, &changed, &to_next_secs)) { suser->session_sent_duplicate = !changed; - suser->last_session_update = ioloop_time; + suser->last_session_update = now; suser->last_sent_session_stats = suser->session_stats; stats_connection_send_session(suser->stats_conn, user, &suser->session_stats); @@ -404,14 +406,19 @@ ret = sbox->module_ctx.super. search_next_nonblock(ctx, mail_r, tryagain_r); - if (ret || !*tryagain_r) - return ret; + if (!ret && !*tryagain_r) { + /* end of search */ + return FALSE; + } - /* retrying, so this is a long running search. update the stats once - a second */ - if (time(NULL) != suser->last_refresh) - session_stats_refresh(user); - return FALSE; + if (*tryagain_r || + ++suser->refresh_check_counter % REFRESH_CHECK_INTERVAL == 0) { + /* a) retrying, so this is a long running search. + b) we've returned enough matches */ + if (time(NULL) != suser->last_session_update) + session_stats_refresh(user); + } + return ret; } static void stats_mailbox_allocated(struct mailbox *box) @@ -450,7 +457,7 @@ if (stats_global_user == NULL) stats_add_session(user); - last_update_secs = ioloop_time - suser->last_session_update; + last_update_secs = time(NULL) - suser->last_session_update; if (last_update_secs >= suser->refresh_secs) { if (stats_global_user != NULL) stats_add_session(user); @@ -552,7 +559,7 @@ suser->stats_conn = global_stats_conn; guid_128_generate(suser->session_guid); - suser->last_session_update = ioloop_time; + suser->last_session_update = time(NULL); suser->ioloop_ctx = ioloop_ctx; io_loop_context_add_callbacks(ioloop_ctx, diff -r 0c3cb8976e81 -r 721e127e107f src/plugins/stats/stats-plugin.h --- a/src/plugins/stats/stats-plugin.h Mon Mar 05 18:43:05 2012 +0200 +++ b/src/plugins/stats/stats-plugin.h Mon Mar 05 18:44:01 2012 +0200 @@ -31,9 +31,9 @@ struct stats_connection *stats_conn; guid_128_t session_guid; - time_t last_refresh; unsigned int refresh_secs; bool track_commands; + unsigned int refresh_check_counter; /* current session statistics */ struct mail_stats session_stats; From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: stats: Don't log write()=EPIPE failures. Retry open... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/315f0d8cc2b2 changeset: 14297:315f0d8cc2b2 user: Timo Sirainen date: Mon Mar 05 19:33:51 2012 +0200 description: stats: Don't log write()=EPIPE failures. Retry opening the pipe once. diffstat: src/plugins/stats/stats-connection.c | 36 +++++++++++++++++++++++++++--------- 1 files changed, 27 insertions(+), 9 deletions(-) diffs (70 lines): diff -r 721e127e107f -r 315f0d8cc2b2 src/plugins/stats/stats-connection.c --- a/src/plugins/stats/stats-connection.c Mon Mar 05 18:44:01 2012 +0200 +++ b/src/plugins/stats/stats-connection.c Mon Mar 05 19:33:51 2012 +0200 @@ -14,8 +14,24 @@ int fd; char *path; + + bool open_failed; }; +static bool stats_connection_open(struct stats_connection *conn) +{ + if (conn->open_failed) + return FALSE; + + conn->fd = open(conn->path, O_WRONLY); + if (conn->fd == -1) { + i_error("stats: open(%s) failed: %m", conn->path); + conn->open_failed = TRUE; + return FALSE; + } + return TRUE; +} + struct stats_connection * stats_connection_create(const char *path) { @@ -24,9 +40,7 @@ conn = i_new(struct stats_connection, 1); conn->refcount = 1; conn->path = i_strdup(path); - conn->fd = open(path, O_WRONLY); - if (conn->fd == -1) - i_error("stats: open(%s) failed: %m", path); + stats_connection_open(conn); return conn; } @@ -57,8 +71,10 @@ static bool pipe_warned = FALSE; ssize_t ret; - if (conn->fd == -1) - return; + if (conn->fd == -1) { + if (!stats_connection_open(conn)) + return; + } if (str_len(str) > PIPE_BUF && !pipe_warned) { i_warning("stats update sent more bytes that PIPE_BUF " @@ -69,11 +85,13 @@ ret = write(conn->fd, str_data(str), str_len(str)); if (ret != (ssize_t)str_len(str)) { - if (ret < 0) - i_error("write(%s) failed: %m", conn->path); - else if ((size_t)ret != str_len(str)) + if (ret < 0) { + /* don't log EPIPE errors. they can happen when + Dovecot is stopped. */ + if (errno != EPIPE) + i_error("write(%s) failed: %m", conn->path); + } else if ((size_t)ret != str_len(str)) i_error("write(%s) wrote partial update", conn->path); - /* this shouldn't happen, just stop sending updates */ if (close(conn->fd) < 0) i_error("close(%s) failed: %m", conn->path); conn->fd = -1; From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: decode2text: Avoid leaving temp files and hanging c... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/e540405902f1 changeset: 14298:e540405902f1 user: Timo Sirainen date: Wed Mar 07 11:23:03 2012 +0200 description: decode2text: Avoid leaving temp files and hanging child processes. diffstat: src/plugins/fts/decode2text.sh | 22 ++++++++++++++++------ 1 files changed, 16 insertions(+), 6 deletions(-) diffs (51 lines): diff -r 315f0d8cc2b2 -r e540405902f1 src/plugins/fts/decode2text.sh --- a/src/plugins/fts/decode2text.sh Mon Mar 05 19:33:51 2012 +0200 +++ b/src/plugins/fts/decode2text.sh Wed Mar 07 11:23:03 2012 +0200 @@ -52,7 +52,7 @@ # most decoders can't handle stdin directly, so write the attachment # to a temp file path=`mktemp` -trap "rm -f $path" 0 1 2 3 15 +trap "rm -f $path" 0 1 2 3 14 15 cat > $path xmlunzip() { @@ -62,23 +62,33 @@ if [ "$tempdir" = "" ]; then exit 1 fi - trap "rm -rf $tempdir" 0 1 2 3 15 + trap "rm -rf $path $tempdir" 0 1 2 3 14 15 cd $tempdir || exit 1 unzip -q "$path" 2>/dev/null || exit 0 find . -name "$name" -print0 | xargs -0 cat | /usr/local/libexec/dovecot/xml2text } +wait_timeout() { + childpid=$! + trap "kill -9 $childpid; rm -f $path" 1 2 3 14 15 + wait $childpid +} + LANG=en_US.UTF-8 export LANG if [ $fmt = "pdf" ]; then - /usr/bin/pdftotext $path - 2>/dev/null + /usr/bin/pdftotext $path - 2>/dev/null& + wait_timeout 2>/dev/null elif [ $fmt = "doc" ]; then - (/usr/bin/catdoc $path; true) 2>/dev/null + (/usr/bin/catdoc $path; true) 2>/dev/null& + wait_timeout 2>/dev/null elif [ $fmt = "ppt" ]; then - (/usr/bin/catppt $path; true) 2>/dev/null + (/usr/bin/catppt $path; true) 2>/dev/null& + wait_timeout 2>/dev/null elif [ $fmt = "xls" ]; then - (/usr/bin/xls2csv $path; true) 2>/dev/null + (/usr/bin/xls2csv $path; true) 2>/dev/null& + wait_timeout 2>/dev/null elif [ $fmt = "odt" -o $fmt = "ods" -o $fmt = "odp" ]; then xmlunzip "content.xml" elif [ $fmt = "docx" ]; then From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: login_log_format_elements: Allow using %{long_varia... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/9b78e93e5b71 changeset: 14300:9b78e93e5b71 user: Timo Sirainen date: Wed Mar 07 13:01:23 2012 +0200 description: login_log_format_elements: Allow using %{long_variables} without breaking. diffstat: src/login-common/client-common.c | 86 +++++++++++++++++++++------------------ 1 files changed, 46 insertions(+), 40 deletions(-) diffs (134 lines): diff -r 9d47d53650be -r 9b78e93e5b71 src/login-common/client-common.c --- a/src/login-common/client-common.c Wed Mar 07 11:29:42 2012 +0200 +++ b/src/login-common/client-common.c Wed Mar 07 13:01:23 2012 +0200 @@ -376,31 +376,33 @@ return clients_count; } +static struct var_expand_table login_var_expand_empty_tab[] = { + { 'u', NULL, "user" }, + { 'n', NULL, "username" }, + { 'd', NULL, "domain" }, + { 's', NULL, "service" }, + { 'h', NULL, "home" }, + { 'l', NULL, "lip" }, + { 'r', NULL, "rip" }, + { 'p', NULL, "pid" }, + { 'm', NULL, "mech" }, + { 'a', NULL, "lport" }, + { 'b', NULL, "rport" }, + { 'c', NULL, "secured" }, + { 'k', NULL, "ssl_security" }, + { 'e', NULL, "mail_pid" }, + { '\0', NULL, NULL } +}; + static const struct var_expand_table * get_var_expand_table(struct client *client) { - static struct var_expand_table static_tab[] = { - { 'u', NULL, "user" }, - { 'n', NULL, "username" }, - { 'd', NULL, "domain" }, - { 's', NULL, "service" }, - { 'h', NULL, "home" }, - { 'l', NULL, "lip" }, - { 'r', NULL, "rip" }, - { 'p', NULL, "pid" }, - { 'm', NULL, "mech" }, - { 'a', NULL, "lport" }, - { 'b', NULL, "rport" }, - { 'c', NULL, "secured" }, - { 'k', NULL, "ssl_security" }, - { 'e', NULL, "mail_pid" }, - { '\0', NULL, NULL } - }; struct var_expand_table *tab; unsigned int i; - tab = t_malloc(sizeof(static_tab)); - memcpy(tab, static_tab, sizeof(static_tab)); + tab = t_malloc(sizeof(login_var_expand_empty_tab)); + memcpy(tab, login_var_expand_empty_tab, + sizeof(login_var_expand_empty_tab)); if (client->virtual_user != NULL) { tab[0].value = client->virtual_user; @@ -440,20 +442,16 @@ return tab; } -static bool have_key(const struct var_expand_table *table, const char *str) +static bool have_username_key(const char *str) { char key; - unsigned int i; - key = var_get_key(str); - for (i = 0; table[i].key != '\0'; i++) { - if (table[i].key == key) { - if (table[i].value == NULL) - return FALSE; - if (table[i].value[0] != '\0') + for (; *str != '\0'; str++) { + if (str[0] == '%' && str[1] != '\0') { + str++; + key = var_get_key(str); + if (key == 'u' || key == 'n') return TRUE; - /* "" key - hide except in username */ - return key == 'u' || key == 'n'; } } return FALSE; @@ -469,9 +467,9 @@ }; const struct var_expand_table *var_expand_table; struct var_expand_table *tab; - const char *p; char *const *e; - string_t *str; + string_t *str, *str2; + unsigned int pos; var_expand_table = get_var_expand_table(client); @@ -479,21 +477,29 @@ memcpy(tab, static_tab, sizeof(static_tab)); str = t_str_new(256); + str2 = t_str_new(128); for (e = client->set->log_format_elements_split; *e != NULL; e++) { - for (p = *e; *p != '\0'; p++) { - if (*p != '%' || p[1] == '\0') + pos = str_len(str); + var_expand(str, *e, var_expand_table); + if (have_username_key(*e)) { + /* username is added even if it's empty */ + } else { + str_truncate(str2, 0); + var_expand(str2, *e, login_var_expand_empty_tab); + if (strcmp(str_c(str)+pos, str_c(str2)) == 0) { + /* empty %variables, don't add */ + str_truncate(str, pos); continue; - - p++; - if (have_key(var_expand_table, p)) { - if (str_len(str) > 0) - str_append(str, ", "); - var_expand(str, *e, var_expand_table); - break; } } + + if (str_len(str) > 0) + str_append(str, ", "); } + if (str_len(str) > 0) + str_truncate(str, str_len(str)-2); + tab[0].value = t_strdup(str_c(str)); tab[1].value = msg; str_truncate(str, 0); From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: doveadm: Handle -NOUSER replies from doveadm-server. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/9d47d53650be changeset: 14299:9d47d53650be user: Timo Sirainen date: Wed Mar 07 11:29:42 2012 +0200 description: doveadm: Handle -NOUSER replies from doveadm-server. diffstat: src/doveadm/doveadm-mail-server.c | 16 ++++++++++++---- src/doveadm/server-connection.c | 16 ++++++++++------ src/doveadm/server-connection.h | 1 + 3 files changed, 23 insertions(+), 10 deletions(-) diffs (78 lines): diff -r e540405902f1 -r 9d47d53650be src/doveadm/doveadm-mail-server.c --- a/src/doveadm/doveadm-mail-server.c Wed Mar 07 11:23:03 2012 +0200 +++ b/src/doveadm/doveadm-mail-server.c Wed Mar 07 11:29:42 2012 +0200 @@ -83,15 +83,23 @@ struct server_connection *conn = context; struct doveadm_server *server; - if (reply == SERVER_CMD_REPLY_INTERNAL_FAILURE) { + switch (reply) { + case SERVER_CMD_REPLY_INTERNAL_FAILURE: internal_failure = TRUE; master_service_stop(master_service); return; + case SERVER_CMD_REPLY_UNKNOWN_USER: + i_error("No such user"); + if (cmd_ctx->exit_code == 0) + cmd_ctx->exit_code = EX_NOUSER; + break; + case SERVER_CMD_REPLY_FAIL: + doveadm_mail_failed_error(cmd_ctx, MAIL_ERROR_TEMP); + break; + case SERVER_CMD_REPLY_OK: + break; } - if (reply != SERVER_CMD_REPLY_OK) - doveadm_mail_failed_error(cmd_ctx, MAIL_ERROR_TEMP); - server = server_connection_get_server(conn); if (array_count(&server->queue) > 0) { char *const *usernamep = array_idx(&server->queue, 0); diff -r e540405902f1 -r 9d47d53650be src/doveadm/server-connection.c --- a/src/doveadm/server-connection.c Wed Mar 07 11:23:03 2012 +0200 +++ b/src/doveadm/server-connection.c Wed Mar 07 11:29:42 2012 +0200 @@ -197,6 +197,7 @@ const unsigned char *data; size_t size; const char *line; + enum server_cmd_reply reply; if (!conn->handshaked) { if ((line = i_stream_read_next_line(conn->input)) == NULL) { @@ -254,15 +255,18 @@ if (conn->state != SERVER_REPLY_STATE_RET) break; /* fall through */ - data = i_stream_get_data(conn->input, &size); case SERVER_REPLY_STATE_RET: - if (size < 2) + line = i_stream_next_line(conn->input); + if (line == NULL) return; - if (data[0] == '+' && data[1] == '\n') + if (line[0] == '+') server_connection_callback(conn, SERVER_CMD_REPLY_OK); - else if (data[0] == '-' && data[1] == '\n') - server_connection_callback(conn, SERVER_CMD_REPLY_FAIL); - else + else if (line[0] == '-') { + reply = strcmp(line+1, "NOUSER") == 0 ? + SERVER_CMD_REPLY_UNKNOWN_USER : + SERVER_CMD_REPLY_FAIL; + server_connection_callback(conn, reply); + } else i_error("doveadm server sent broken input"); /* we're finished, close the connection */ server_connection_destroy(&conn); diff -r e540405902f1 -r 9d47d53650be src/doveadm/server-connection.h --- a/src/doveadm/server-connection.h Wed Mar 07 11:23:03 2012 +0200 +++ b/src/doveadm/server-connection.h Wed Mar 07 11:29:42 2012 +0200 @@ -3,6 +3,7 @@ enum server_cmd_reply { SERVER_CMD_REPLY_INTERNAL_FAILURE, + SERVER_CMD_REPLY_UNKNOWN_USER, SERVER_CMD_REPLY_FAIL, SERVER_CMD_REPLY_OK }; From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: login proxy: Log "disconnecting" message also with ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/49b832c5de0e changeset: 14301:49b832c5de0e user: Timo Sirainen date: Wed Mar 07 13:32:40 2012 +0200 description: login proxy: Log "disconnecting" message also with login_log_format_elements. diffstat: src/login-common/login-proxy.c | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diffs (19 lines): diff -r 9b78e93e5b71 -r 49b832c5de0e src/login-common/login-proxy.c --- a/src/login-common/login-proxy.c Wed Mar 07 13:01:23 2012 +0200 +++ b/src/login-common/login-proxy.c Wed Mar 07 13:32:40 2012 +0200 @@ -370,10 +370,11 @@ DLLIST_REMOVE(&login_proxies, proxy); ipstr = net_ip2addr(&proxy->client->ip); - i_info("proxy(%s): disconnecting %s%s", - proxy->client->virtual_user, - ipstr != NULL ? ipstr : "", - reason == NULL ? "" : t_strdup_printf(" (%s)", reason)); + client_log(proxy->client, t_strdup_printf( + "proxy(%s): disconnecting %s%s", + proxy->client->virtual_user, + ipstr != NULL ? ipstr : "", + reason == NULL ? "" : t_strdup_printf(" (%s)", reason))); if (proxy->client_io != NULL) io_remove(&proxy->client_io); From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: Added a "session ID" string for imap/pop3 connectio... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/5bbcf636bbeb changeset: 14302:5bbcf636bbeb user: Timo Sirainen date: Wed Mar 07 13:36:34 2012 +0200 description: Added a "session ID" string for imap/pop3 connections, available in %{session} variable. The session ID passes through Dovecot IMAP/POP3 proxying to backend server. The same session ID is can be reused after a long time (currently a bit under 9 years). diffstat: src/imap-login/client.c | 4 +++ src/imap-login/imap-proxy.c | 2 + src/imap/imap-client.c | 6 ++++- src/imap/imap-client.h | 4 ++- src/imap/main.c | 4 ++- src/lib-master/master-login.c | 16 +++++++++++++- src/lib-master/master-login.h | 3 ++ src/lib-storage/mail-storage-service.c | 2 + src/lib-storage/mail-storage-service.h | 1 + src/login-common/client-common.c | 39 ++++++++++++++++++++++++++++++++++ src/login-common/client-common.h | 6 ++++- src/login-common/sasl-server.c | 6 ++++- src/pop3-login/client.c | 3 ++ src/pop3-login/pop3-proxy.c | 5 ++- src/pop3/main.c | 4 ++- src/pop3/pop3-client.c | 7 +++++- src/pop3/pop3-client.h | 4 ++- 17 files changed, 105 insertions(+), 11 deletions(-) diffs (truncated from 445 to 300 lines): diff -r 49b832c5de0e -r 5bbcf636bbeb src/imap-login/client.c --- a/src/imap-login/client.c Wed Mar 07 13:32:40 2012 +0200 +++ b/src/imap-login/client.c Wed Mar 07 13:36:34 2012 +0200 @@ -111,6 +111,10 @@ (void)net_addr2ip(value, &client->common.local_ip); else if (strcasecmp(key, "x-connected-port") == 0) client->common.local_port = atoi(value); + else if (strcasecmp(key, "x-session-id") == 0) { + client->common.session_id = + p_strdup(client->common.pool, value); + } args += 2; } } diff -r 49b832c5de0e -r 5bbcf636bbeb src/imap-login/imap-proxy.c --- a/src/imap-login/imap-proxy.c Wed Mar 07 13:32:40 2012 +0200 +++ b/src/imap-login/imap-proxy.c Wed Mar 07 13:36:34 2012 +0200 @@ -30,10 +30,12 @@ static void proxy_write_id(struct imap_client *client, string_t *str) { str_printfa(str, "I ID (" + "\"x-session-id\" \"%s\" " "\"x-originating-ip\" \"%s\" " "\"x-originating-port\" \"%u\" " "\"x-connected-ip\" \"%s\" " "\"x-connected-port\" \"%u\")\r\n", + client_get_session_id(&client->common), net_ip2addr(&client->common.ip), client->common.remote_port, net_ip2addr(&client->common.local_ip), diff -r 49b832c5de0e -r 5bbcf636bbeb src/imap/imap-client.c --- a/src/imap/imap-client.c Wed Mar 07 13:32:40 2012 +0200 +++ b/src/imap/imap-client.c Wed Mar 07 13:36:34 2012 +0200 @@ -32,7 +32,8 @@ client_destroy(client, "Disconnected for inactivity"); } -struct client *client_create(int fd_in, int fd_out, struct mail_user *user, +struct client *client_create(int fd_in, int fd_out, const char *session_id, + struct mail_user *user, struct mail_storage_service_user *service_user, const struct imap_settings *set) { @@ -49,6 +50,7 @@ client->pool = pool; client->set = set; client->service_user = service_user; + client->session_id = p_strdup(pool, session_id); client->fd_in = fd_in; client->fd_out = fd_out; client->input = i_stream_create_fd(fd_in, @@ -143,6 +145,7 @@ static struct var_expand_table static_tab[] = { { 'i', NULL, "input" }, { 'o', NULL, "output" }, + { '\0', NULL, "session" }, { '\0', NULL, NULL } }; struct var_expand_table *tab; @@ -153,6 +156,7 @@ tab[0].value = dec2str(i_stream_get_absolute_offset(client->input)); tab[1].value = dec2str(client->output->offset); + tab[2].value = client->session_id; str = t_str_new(128); var_expand(str, client->set->imap_logout_format, tab); diff -r 49b832c5de0e -r 5bbcf636bbeb src/imap/imap-client.h --- a/src/imap/imap-client.h Wed Mar 07 13:32:40 2012 +0200 +++ b/src/imap/imap-client.h Wed Mar 07 13:36:34 2012 +0200 @@ -99,6 +99,7 @@ struct client { struct client *prev, *next; + const char *session_id; int fd_in, fd_out; struct io *io; struct istream *input; @@ -170,7 +171,8 @@ /* Create new client with specified input/output handles. socket specifies if the handle is a socket. */ -struct client *client_create(int fd_in, int fd_out, struct mail_user *user, +struct client *client_create(int fd_in, int fd_out, const char *session_id, + struct mail_user *user, struct mail_storage_service_user *service_user, const struct imap_settings *set); void client_destroy(struct client *client, const char *reason); diff -r 49b832c5de0e -r 5bbcf636bbeb src/imap/main.c --- a/src/imap/main.c Wed Mar 07 13:32:40 2012 +0200 +++ b/src/imap/main.c Wed Mar 07 13:36:34 2012 +0200 @@ -208,7 +208,8 @@ if (set->verbose_proctitle) verbose_proctitle = TRUE; - client = client_create(fd_in, fd_out, mail_user, user, set); + client = client_create(fd_in, fd_out, login_client->session_id, + mail_user, user, set); T_BEGIN { client_add_input(client, input_buf); } T_END; @@ -261,6 +262,7 @@ input.remote_ip = client->auth_req.remote_ip; input.username = username; input.userdb_fields = extra_fields; + input.session_id = client->session_id; buffer_create_const_data(&input_buf, client->data, client->auth_req.data_size); diff -r 49b832c5de0e -r 5bbcf636bbeb src/lib-master/master-login.c --- a/src/lib-master/master-login.c Wed Mar 07 13:32:40 2012 +0200 +++ b/src/lib-master/master-login.c Wed Mar 07 13:36:34 2012 +0200 @@ -392,6 +392,7 @@ struct master_login_client *client; struct master_login *login = conn->login; unsigned char data[MASTER_AUTH_MAX_DATA_SIZE]; + unsigned int i, session_len = 0; int ret, client_fd; ret = master_login_conn_read_request(conn, &req, data, &client_fd); @@ -408,12 +409,25 @@ } fd_close_on_exec(client_fd, TRUE); + /* extract the session ID from the request data */ + for (i = 0; i < req.data_size; i++) { + if (data[i] == '\0') { + session_len = i++; + break; + } + } + if (session_len >= sizeof(client->session_id)) { + i_error("login client: Session ID too long"); + session_len = 0; + } + /* @UNSAFE: we have a request. do userdb lookup for it. */ client = i_malloc(sizeof(struct master_login_client) + req.data_size); client->conn = conn; client->fd = client_fd; client->auth_req = req; - memcpy(client->data, data, req.data_size); + memcpy(client->session_id, data, session_len); + memcpy(client->data, data+i, req.data_size-i); conn->refcount++; master_login_auth_request(login->auth, &req, diff -r 49b832c5de0e -r 5bbcf636bbeb src/lib-master/master-login.h --- a/src/lib-master/master-login.h Wed Mar 07 13:32:40 2012 +0200 +++ b/src/lib-master/master-login.h Wed Mar 07 13:36:34 2012 +0200 @@ -4,12 +4,15 @@ #include "master-auth.h" #define MASTER_POSTLOGIN_TIMEOUT_DEFAULT 60 +/* base64(<48bit timestamp>) + NUL */ +#define LOGIN_MAX_SESSION_ID_LEN 33 struct master_login_client { struct master_login_connection *conn; int fd; struct master_auth_request auth_req; + char session_id[LOGIN_MAX_SESSION_ID_LEN]; unsigned char data[FLEXIBLE_ARRAY_MEMBER]; }; diff -r 49b832c5de0e -r 5bbcf636bbeb src/lib-storage/mail-storage-service.c --- a/src/lib-storage/mail-storage-service.c Wed Mar 07 13:32:40 2012 +0200 +++ b/src/lib-storage/mail-storage-service.c Wed Mar 07 13:36:34 2012 +0200 @@ -374,6 +374,7 @@ { 'p', NULL, "pid" }, { 'i', NULL, "uid" }, { '\0', NULL, "gid" }, + { '\0', NULL, "session" }, { '\0', NULL, NULL } }; struct var_expand_table *tab; @@ -391,6 +392,7 @@ tab[6].value = my_pid; tab[7].value = dec2str(priv->uid == (uid_t)-1 ? geteuid() : priv->uid); tab[8].value = dec2str(priv->gid == (gid_t)-1 ? getegid() : priv->gid); + tab[9].value = input->session_id; return tab; } diff -r 49b832c5de0e -r 5bbcf636bbeb src/lib-storage/mail-storage-service.h --- a/src/lib-storage/mail-storage-service.h Wed Mar 07 13:32:40 2012 +0200 +++ b/src/lib-storage/mail-storage-service.h Wed Mar 07 13:36:34 2012 +0200 @@ -40,6 +40,7 @@ const char *module; const char *service; const char *username; + const char *session_id; struct ip_addr local_ip, remote_ip; unsigned int local_port, remote_port; diff -r 49b832c5de0e -r 5bbcf636bbeb src/login-common/client-common.c --- a/src/login-common/client-common.c Wed Mar 07 13:32:40 2012 +0200 +++ b/src/login-common/client-common.c Wed Mar 07 13:36:34 2012 +0200 @@ -7,7 +7,9 @@ #include "ostream.h" #include "iostream-rawlog.h" #include "process-title.h" +#include "buffer.h" #include "str.h" +#include "base64.h" #include "str-sanitize.h" #include "safe-memset.h" #include "var-expand.h" @@ -376,6 +378,41 @@ return clients_count; } +const char *client_get_session_id(struct client *client) +{ + buffer_t *buf, *base64_buf; + struct timeval tv; + uint64_t timestamp; + unsigned int i; + + if (client->session_id != NULL) + return client->session_id; + + buf = buffer_create_dynamic(pool_datastack_create(), 24); + base64_buf = buffer_create_dynamic(pool_datastack_create(), 24*2); + + if (gettimeofday(&tv, NULL) < 0) + i_fatal("gettimeofday(): %m"); + timestamp = tv.tv_usec + (long long)tv.tv_sec * 1000ULL*1000ULL; + + /* add lowest 48 bits of the timestamp. this gives us a bit less than + 9 years until it wraps */ + for (i = 0; i < 48; i += 8) + buffer_append_c(buf, (timestamp >> i) & 0xff); + + buffer_append_c(buf, client->remote_port & 0xff); + buffer_append_c(buf, (client->remote_port >> 16) & 0xff); +#ifdef HAVE_IPV6 + if (IPADDR_IS_V6(&client->ip)) + buffer_append(buf, &client->ip.u.ip6, sizeof(client->ip.u.ip6)); + else +#endif + buffer_append(buf, &client->ip.u.ip4, sizeof(client->ip.u.ip4)); + base64_encode(buf->data, buf->used, base64_buf); + client->session_id = p_strdup(client->pool, str_c(base64_buf)); + return client->session_id; +} + static struct var_expand_table login_var_expand_empty_tab[] = { { 'u', NULL, "user" }, { 'n', NULL, "username" }, @@ -391,6 +428,7 @@ { 'c', NULL, "secured" }, { 'k', NULL, "ssl_security" }, { 'e', NULL, "mail_pid" }, + { '\0', NULL, "session" }, { '\0', NULL, NULL } }; @@ -439,6 +477,7 @@ } tab[13].value = client->mail_pid == 0 ? "" : dec2str(client->mail_pid); + tab[14].value = client_get_session_id(client); return tab; } diff -r 49b832c5de0e -r 5bbcf636bbeb src/login-common/client-common.h --- a/src/login-common/client-common.h Wed Mar 07 13:32:40 2012 +0200 +++ b/src/login-common/client-common.h Wed Mar 07 13:36:34 2012 +0200 @@ -4,6 +4,7 @@ #include "network.h" #include "login-proxy.h" #include "sasl-server.h" +#include "master-login.h" /* for LOGIN_MAX_SESSION_ID_LEN */ #define LOGIN_MAX_MASTER_PREFIX_LEN 128 @@ -14,7 +15,8 @@ POP3: Max. length of a command line (spec says 512 would be enough) */ #define LOGIN_MAX_INBUF_SIZE \ - (MASTER_AUTH_MAX_DATA_SIZE - LOGIN_MAX_MASTER_PREFIX_LEN) + (MASTER_AUTH_MAX_DATA_SIZE - LOGIN_MAX_MASTER_PREFIX_LEN - \ + LOGIN_MAX_SESSION_ID_LEN) /* max. size of output buffer. if it gets full, the client is disconnected. SASL authentication gives the largest output. */ #define LOGIN_MAX_OUTBUF_SIZE 4096 @@ -89,6 +91,7 @@ unsigned int local_port, remote_port; struct ssl_proxy *ssl_proxy; const struct login_settings *set; + const char *session_id; int fd; struct istream *input; @@ -163,6 +166,7 @@ const char *client_get_extra_disconnect_reason(struct client *client); bool client_is_trusted(struct client *client); void client_auth_failed(struct client *client); From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: director: Add more info to "User hash .. is being r... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d048cebc1fd4 changeset: 14303:d048cebc1fd4 user: Timo Sirainen date: Wed Mar 07 16:32:14 2012 +0200 description: director: Add more info to "User hash .. is being redirected to two hosts" error. diffstat: src/director/director-connection.c | 41 +++++++++++++++++++++++++------------ 1 files changed, 28 insertions(+), 13 deletions(-) diffs (82 lines): diff -r 5bbcf636bbeb -r d048cebc1fd4 src/director/director-connection.c --- a/src/director/director-connection.c Wed Mar 07 13:36:34 2012 +0200 +++ b/src/director/director-connection.c Wed Mar 07 16:32:14 2012 +0200 @@ -183,10 +183,11 @@ } static bool -director_user_refresh(struct director *dir, unsigned int username_hash, - struct mail_host *host, time_t timestamp, - struct user **user_r) +director_user_refresh(struct director_connection *conn, + unsigned int username_hash, struct mail_host *host, + time_t timestamp, struct user **user_r) { + struct director *dir = conn->dir; struct user *user; bool ret = FALSE; @@ -196,16 +197,26 @@ host, timestamp); return TRUE; } - if (timestamp == ioloop_time && (time_t)user->timestamp != timestamp) { - user_directory_refresh(dir->users, user); - ret = TRUE; - } if (user->host != host) { - i_error("User hash %u is being redirected to two hosts: " - "%s and %s", username_hash, - net_ip2addr(&user->host->ip), - net_ip2addr(&host->ip)); + string_t *str = t_str_new(128); + + str_printfa(str, "User hash %u " + "is being redirected to two hosts: %s and %s", + username_hash, net_ip2addr(&user->host->ip), + net_ip2addr(&host->ip)); + str_printfa(str, " (old_ts=%ld", (long)user->timestamp); + + if (!conn->handshake_received) { + str_printfa(str, ",handshaking,recv_ts=%ld", + (long)timestamp); + } + if (user->to_move != NULL) + str_append(str, ",moving"); + if (user->kill_state == USER_KILL_STATE_NONE) + str_printfa(str, ",kill_state=%d", user->kill_state); + str_append_c(str, ')'); + i_error("%s", str_c(str)); /* we want all the directors to redirect the user to same server, but we don't want two directors fighting over which @@ -220,6 +231,10 @@ } ret = TRUE; } + if (timestamp == ioloop_time && (time_t)user->timestamp != timestamp) { + user_directory_refresh(dir->users, user); + ret = TRUE; + } *user_r = user; return ret; } @@ -249,7 +264,7 @@ return FALSE; } - director_user_refresh(conn->dir, username_hash, host, timestamp, &user); + director_user_refresh(conn, username_hash, host, timestamp, &user); return TRUE; } @@ -274,7 +289,7 @@ return TRUE; } - if (director_user_refresh(conn->dir, username_hash, + if (director_user_refresh(conn, username_hash, host, ioloop_time, &user)) director_update_user(conn->dir, conn->host, user); return TRUE; From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: director: Log a warning if user is refreshed too late. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/2026af3cf87b changeset: 14304:2026af3cf87b user: Timo Sirainen date: Wed Mar 07 16:48:47 2012 +0200 description: director: Log a warning if user is refreshed too late. diffstat: src/director/notify-connection.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diffs (21 lines): diff -r d048cebc1fd4 -r 2026af3cf87b src/director/notify-connection.c --- a/src/director/notify-connection.c Wed Mar 07 16:32:14 2012 +0200 +++ b/src/director/notify-connection.c Wed Mar 07 16:48:47 2012 +0200 @@ -22,11 +22,17 @@ struct user *user; const char *line; unsigned int hash; + int diff; while ((line = i_stream_read_next_line(conn->input)) != NULL) { hash = user_directory_get_username_hash(conn->dir->users, line); user = user_directory_lookup(conn->dir->users, hash); if (user != NULL) { + diff = ioloop_time - user->timestamp; + if (diff >= (int)conn->dir->set->director_user_expire) { + i_warning("notify: User %s refreshed too late " + "(%d secs)", line, diff); + } user_directory_refresh(conn->dir->users, user); director_update_user(conn->dir, conn->dir->self_host, user); From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: login proxy: Handle proxy-notify errors better. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/94de7605f50f changeset: 14305:94de7605f50f user: Timo Sirainen date: Wed Mar 07 16:58:37 2012 +0200 description: login proxy: Handle proxy-notify errors better. If open() fails with ENOENT, it should still be logged as an error. If opening or anything else fails, retry opening the fifo 60 secs. diffstat: src/login-common/login-proxy-state.c | 56 ++++++++++++++++++++++++++++------- 1 files changed, 44 insertions(+), 12 deletions(-) diffs (126 lines): diff -r 2026af3cf87b -r 94de7605f50f src/login-common/login-proxy-state.c --- a/src/login-common/login-proxy-state.c Wed Mar 07 16:48:47 2012 +0200 +++ b/src/login-common/login-proxy-state.c Wed Mar 07 16:58:37 2012 +0200 @@ -2,6 +2,7 @@ #include "lib.h" #include "network.h" +#include "ioloop.h" #include "hash.h" #include "strescape.h" #include "fd-set-nonblock.h" @@ -10,6 +11,8 @@ #include #include +#define NOTIFY_RETRY_REOPEN_MSECS (60*1000) + struct login_proxy_state { struct hash_table *hash; pool_t pool; @@ -17,9 +20,11 @@ const char *notify_path; int notify_fd; - unsigned int notify_fd_broken:1; + struct timeout *to_reopen; }; +static int login_proxy_state_notify_open(struct login_proxy_state *state); + static unsigned int login_proxy_record_hash(const void *p) { const struct login_proxy_record *rec = p; @@ -51,16 +56,24 @@ return state; } +static void login_proxy_state_close(struct login_proxy_state *state) +{ + if (state->notify_fd != -1) { + if (close(state->notify_fd) < 0) + i_error("close(%s) failed: %m", state->notify_path); + state->notify_fd = -1; + } +} + void login_proxy_state_deinit(struct login_proxy_state **_state) { struct login_proxy_state *state = *_state; *_state = NULL; - if (state->notify_fd != -1) { - if (close(state->notify_fd) < 0) - i_error("close(%s) failed: %m", state->notify_path); - } + if (state->to_reopen != NULL) + timeout_remove(&state->to_reopen); + login_proxy_state_close(state); hash_table_destroy(&state->hash); pool_unref(&state->pool); i_free(state); @@ -86,31 +99,39 @@ return rec; } +static void login_proxy_state_reopen(struct login_proxy_state *state) +{ + timeout_remove(&state->to_reopen); + (void)login_proxy_state_notify_open(state); +} + static int login_proxy_state_notify_open(struct login_proxy_state *state) { - if (state->notify_fd_broken) + if (state->to_reopen != NULL) { + /* reopen later */ return -1; + } state->notify_fd = open(state->notify_path, O_WRONLY); if (state->notify_fd == -1) { - if (errno != ENOENT) - i_error("open(%s) failed: %m", state->notify_path); - state->notify_fd_broken = TRUE; + i_error("open(%s) failed: %m", state->notify_path); + state->to_reopen = timeout_add(NOTIFY_RETRY_REOPEN_MSECS, + login_proxy_state_reopen, state); return -1; } fd_set_nonblock(state->notify_fd, TRUE); return 0; } -void login_proxy_state_notify(struct login_proxy_state *state, - const char *user) +static bool login_proxy_state_try_notify(struct login_proxy_state *state, + const char *user) { unsigned int len; ssize_t ret; if (state->notify_fd == -1) { if (login_proxy_state_notify_open(state) < 0) - return; + return TRUE; } T_BEGIN { @@ -128,5 +149,16 @@ i_error("write(%s) wrote partial update", state->notify_path); } + login_proxy_state_close(state); + /* retry sending */ + return FALSE; } + return TRUE; } + +void login_proxy_state_notify(struct login_proxy_state *state, + const char *user) +{ + if (!login_proxy_state_try_notify(state, user)) + login_proxy_state_try_notify(state, user); +} From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: Makefile: s/MKDIR_P/mkdir_p/ to make it actually work Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/8c6ff808902c changeset: 14307:8c6ff808902c user: Timo Sirainen date: Thu Mar 08 10:21:09 2012 +0200 description: Makefile: s/MKDIR_P/mkdir_p/ to make it actually work diffstat: Makefile.am | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 334424e7465e -r 8c6ff808902c Makefile.am --- a/Makefile.am Wed Mar 07 20:16:33 2012 +0200 +++ b/Makefile.am Thu Mar 08 10:21:09 2012 +0200 @@ -59,7 +59,7 @@ endif install-exec-hook: - $(MKDIR_P) $(DESTDIR)$(pkglibdir); \ + $(mkdir_p) $(DESTDIR)$(pkglibdir); \ grep -v '^LIBDOVECOT_.*_INCLUDE' dovecot-config | \ grep -v '^LIBDOVECOT.*_DEPS' | sed \ -e "s|^\(LIBDOVECOT\)=.*$$|\1='-L$(pkglibdir) -ldovecot'|" \ From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: director: Keep track of the highest supported proto... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/334424e7465e changeset: 14306:334424e7465e user: Timo Sirainen date: Wed Mar 07 20:16:33 2012 +0200 description: director: Keep track of the highest supported protocol version in the ring. diffstat: src/director/director-connection.c | 41 +++++++++++++++++++------------------ src/director/director.c | 27 ++++++++++++++++++------ src/director/director.h | 8 +++++++ 3 files changed, 49 insertions(+), 27 deletions(-) diffs (194 lines): diff -r 94de7605f50f -r 334424e7465e src/director/director-connection.c --- a/src/director/director-connection.c Wed Mar 07 16:58:37 2012 +0200 +++ b/src/director/director-connection.c Wed Mar 07 20:16:33 2012 +0200 @@ -19,10 +19,6 @@ #include #include -#define DIRECTOR_VERSION_NAME "director" -#define DIRECTOR_VERSION_MAJOR 1 -#define DIRECTOR_VERSION_MINOR 0 - #define MAX_INBUF_SIZE 1024 #define MAX_OUTBUF_SIZE (1024*1024*10) #define OUTBUF_FLUSH_THRESHOLD (1024*128) @@ -586,10 +582,8 @@ finished by sending a SYNC. if we get it back, it's done. */ dir->sync_seq++; director_set_ring_unsynced(dir); - director_connection_send(dir->right, - t_strdup_printf("SYNC\t%s\t%u\t%u\n", - net_ip2addr(&dir->self_ip), - dir->self_port, dir->sync_seq)); + director_sync_send(dir, dir->self_host, dir->sync_seq, + DIRECTOR_VERSION_MINOR); } if (conn->to_ping != NULL) timeout_remove(&conn->to_ping); @@ -693,16 +687,22 @@ static void director_connection_sync_host(struct director_connection *conn, struct director_host *host, - uint32_t seq, const char *line) + uint32_t seq, unsigned int minor_version) { struct director *dir = conn->dir; + if (minor_version > DIRECTOR_VERSION_MINOR) { + /* we're not up to date */ + minor_version = DIRECTOR_VERSION_MINOR; + } + if (host->self) { if (dir->sync_seq != seq) { /* stale SYNC event */ return; } + dir->synced_minor_version = minor_version; if (!dir->ring_handshaked) { /* the ring is handshaked */ director_set_ring_handshaked(dir); @@ -716,35 +716,36 @@ } director_set_ring_synced(dir); } - } else { + } else if (dir->right != NULL) { /* forward it to the connection on right */ - if (dir->right != NULL) { - director_connection_send(dir->right, - t_strconcat(line, "\n", NULL)); - } + director_sync_send(dir, host, seq, minor_version); } } static bool director_connection_sync(struct director_connection *conn, - const char *const *args, const char *line) + const char *const *args) { struct director *dir = conn->dir; struct director_host *host; struct ip_addr ip; - unsigned int port, seq; + unsigned int port, seq, minor_version = 0; - if (str_array_length(args) != 3 || + if (str_array_length(args) < 3 || !director_args_parse_ip_port(conn, args, &ip, &port) || str_to_uint(args[2], &seq) < 0) { i_error("director(%s): Invalid SYNC args", conn->name); return FALSE; } + if (args[3] != NULL) + minor_version = atoi(args[3]); /* find the originating director. if we don't see it, it was already removed and we can ignore this sync. */ host = director_host_lookup(dir, &ip, port); - if (host != NULL) - director_connection_sync_host(conn, host, seq, line); + if (host != NULL) { + director_connection_sync_host(conn, host, seq, + minor_version); + } if (host == NULL || !host->self) director_resend_sync(dir); @@ -866,7 +867,7 @@ if (strcmp(cmd, "DIRECTOR") == 0) return director_cmd_director(conn, args); if (strcmp(cmd, "SYNC") == 0) - return director_connection_sync(conn, args, line); + return director_connection_sync(conn, args); if (strcmp(cmd, "CONNECT") == 0) return director_cmd_connect(conn, args); diff -r 94de7605f50f -r 334424e7465e src/director/director.c --- a/src/director/director.c Wed Mar 07 16:58:37 2012 +0200 +++ b/src/director/director.c Wed Mar 07 20:16:33 2012 +0200 @@ -155,6 +155,7 @@ it must have failed recently */ director_connection_deinit(&dir->left); } + dir->synced_minor_version = DIRECTOR_VERSION_MINOR; if (!dir->ring_handshaked) director_set_ring_handshaked(dir); else @@ -232,14 +233,27 @@ director_set_state_changed(dir); } +void director_sync_send(struct director *dir, struct director_host *host, + uint32_t seq, unsigned int minor_version) +{ + string_t *str = t_str_new(128); + + str_printfa(str, "SYNC\t%s\t%u\t%u", + net_ip2addr(&host->ip), host->port, seq); + if (minor_version > 0) { + /* only minor_version>0 supports this parameter */ + str_printfa(str, "\t%u", minor_version); + } + str_append_c(str, '\n'); + director_connection_send(dir->right, str_c(str)); +} + bool director_resend_sync(struct director *dir) { if (!dir->ring_synced && dir->left != NULL && dir->right != NULL) { /* send a new SYNC in case the previous one got dropped */ - director_connection_send(dir->right, - t_strdup_printf("SYNC\t%s\t%u\t%u\n", - net_ip2addr(&dir->self_ip), - dir->self_port, dir->sync_seq)); + director_sync_send(dir, dir->self_host, dir->sync_seq, + DIRECTOR_VERSION_MINOR); if (dir->to_sync != NULL) timeout_reset(dir->to_sync); return TRUE; @@ -295,9 +309,8 @@ if (dir->left != NULL) director_connection_wait_sync(dir->left); director_connection_wait_sync(dir->right); - director_connection_send(dir->right, t_strdup_printf( - "SYNC\t%s\t%u\t%u\n", net_ip2addr(&dir->self_ip), - dir->self_port, dir->sync_seq)); + director_sync_send(dir, dir->self_host, dir->sync_seq, + DIRECTOR_VERSION_MINOR); } void director_sync_freeze(struct director *dir) diff -r 94de7605f50f -r 334424e7465e src/director/director.h --- a/src/director/director.h Wed Mar 07 16:58:37 2012 +0200 +++ b/src/director/director.h Wed Mar 07 20:16:33 2012 +0200 @@ -4,6 +4,10 @@ #include "network.h" #include "director-settings.h" +#define DIRECTOR_VERSION_NAME "director" +#define DIRECTOR_VERSION_MAJOR 1 +#define DIRECTOR_VERSION_MINOR 0 + struct director; struct mail_host; struct user; @@ -46,6 +50,8 @@ struct ipc_client *ipc_proxy; unsigned int sync_seq; + /* the lowest minor version supported by the ring */ + unsigned int synced_minor_version; time_t ring_last_sync_time; /* director ring handshaking is complete. @@ -76,6 +82,8 @@ void director_set_ring_synced(struct director *dir); void director_set_ring_unsynced(struct director *dir); void director_set_state_changed(struct director *dir); +void director_sync_send(struct director *dir, struct director_host *host, + uint32_t seq, unsigned int minor_version); bool director_resend_sync(struct director *dir); void director_update_host(struct director *dir, struct director_host *src, From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: imap: Crashfix when running standalone. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/91438eb4fad8 changeset: 14308:91438eb4fad8 user: Timo Sirainen date: Thu Mar 08 10:39:58 2012 +0200 description: imap: Crashfix when running standalone. diffstat: src/imap/main.c | 9 ++++++--- 1 files changed, 6 insertions(+), 3 deletions(-) diffs (38 lines): diff -r 8c6ff808902c -r 91438eb4fad8 src/imap/main.c --- a/src/imap/main.c Thu Mar 08 10:21:09 2012 +0200 +++ b/src/imap/main.c Thu Mar 08 10:39:58 2012 +0200 @@ -208,13 +208,13 @@ if (set->verbose_proctitle) verbose_proctitle = TRUE; - client = client_create(fd_in, fd_out, login_client->session_id, + client = client_create(fd_in, fd_out, input->session_id, mail_user, user, set); T_BEGIN { client_add_input(client, input_buf); } T_END; - flags = login_client == NULL ? 0 : login_client->auth_req.flags; + flags = login_client->auth_req.flags; if ((flags & MAIL_AUTH_REQUEST_FLAG_TLS_COMPRESSION) != 0) client->tls_compression = TRUE; return 0; @@ -222,6 +222,7 @@ static void main_stdio_run(const char *username) { + struct master_login_client login_client; struct mail_storage_service_input input; const char *value, *error, *input_base64; buffer_t *input_buf; @@ -242,7 +243,9 @@ input_buf = input_base64 == NULL ? NULL : t_base64_decode_str(input_base64); - if (client_create_from_input(&input, NULL, STDIN_FILENO, STDOUT_FILENO, + memset(&login_client, 0, sizeof(login_client)); + if (client_create_from_input(&input, &login_client, + STDIN_FILENO, STDOUT_FILENO, input_buf, &error) < 0) i_fatal("%s", error); } From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: lib-master: Fixed passing request data from login p... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d6fda337af15 changeset: 14309:d6fda337af15 user: Timo Sirainen date: Thu Mar 08 10:48:08 2012 +0200 description: lib-master: Fixed passing request data from login process. diffstat: src/lib-master/master-login.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (18 lines): diff -r 91438eb4fad8 -r d6fda337af15 src/lib-master/master-login.c --- a/src/lib-master/master-login.c Thu Mar 08 10:39:58 2012 +0200 +++ b/src/lib-master/master-login.c Thu Mar 08 10:48:08 2012 +0200 @@ -422,12 +422,13 @@ } /* @UNSAFE: we have a request. do userdb lookup for it. */ + req.data_size -= i; client = i_malloc(sizeof(struct master_login_client) + req.data_size); client->conn = conn; client->fd = client_fd; client->auth_req = req; memcpy(client->session_id, data, session_len); - memcpy(client->data, data+i, req.data_size-i); + memcpy(client->data, data+i, req.data_size); conn->refcount++; master_login_auth_request(login->auth, &req, From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: director: Avoid user getting redirected to differen... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/7a26c427fc78 changeset: 14310:7a26c427fc78 user: Timo Sirainen date: Thu Mar 08 16:03:45 2012 +0200 description: director: Avoid user getting redirected to different servers near its expiration. Fixes a problem when user is logging in at the same time on director1 which thinks the user is expired, and on director2 which thinks the user expires only in 1 second. diffstat: src/director/director-connection.c | 142 +++++++++++++++++++++++++++++------- src/director/director-request.c | 74 ++++++++++++++++++- src/director/director.c | 27 ++++++- src/director/director.h | 11 ++- src/director/main.c | 3 +- src/director/notify-connection.c | 1 + src/director/user-directory.c | 40 +++++++++- src/director/user-directory.h | 12 ++- 8 files changed, 264 insertions(+), 46 deletions(-) diffs (truncated from 581 to 300 lines): diff -r d6fda337af15 -r 7a26c427fc78 src/director/director-connection.c --- a/src/director/director-connection.c Thu Mar 08 10:48:08 2012 +0200 +++ b/src/director/director-connection.c Thu Mar 08 16:03:45 2012 +0200 @@ -181,20 +181,37 @@ static bool director_user_refresh(struct director_connection *conn, unsigned int username_hash, struct mail_host *host, - time_t timestamp, struct user **user_r) + time_t timestamp, bool weak, struct user **user_r) { struct director *dir = conn->dir; struct user *user; - bool ret = FALSE; + bool ret = FALSE, unset_weak_user = FALSE; user = user_directory_lookup(dir->users, username_hash); if (user == NULL) { *user_r = user_directory_add(dir->users, username_hash, host, timestamp); + (*user_r)->weak = weak; return TRUE; } - if (user->host != host) { + if (user->weak) { + if (!weak) { + /* removing user's weakness */ + unset_weak_user = TRUE; + user->weak = FALSE; + ret = TRUE; + } else { + /* weak user marked again as weak */ + } + } else if (weak && + !user_directory_user_is_recently_updated(dir->users, user)) { + /* mark the user as weak */ + user->weak = TRUE; + ret = TRUE; + } else if (user->host != host) { + /* non-weak user received a non-weak update with + conflicting host. this shouldn't happen. */ string_t *str = t_str_new(128); str_printfa(str, "User hash %u " @@ -221,16 +238,29 @@ /* change the host. we'll also need to remove the user from the old host's user_count, because we can't keep track of the user for more than one host */ - user->host->user_count--; - user->host = host; - user->host->user_count++; + } else { + /* keep the host */ + host = user->host; } ret = TRUE; } + if (user->host != host) { + user->host->user_count--; + user->host = host; + user->host->user_count++; + ret = TRUE; + } if (timestamp == ioloop_time && (time_t)user->timestamp != timestamp) { user_directory_refresh(dir->users, user); ret = TRUE; } + + if (unset_weak_user) { + /* user is no longer weak. handle pending requests for + this user if there are any */ + director_set_state_changed(conn->dir); + } + *user_r = user; return ret; } @@ -243,8 +273,9 @@ struct ip_addr ip; struct mail_host *host; struct user *user; + bool weak; - if (str_array_length(args) != 3 || + if (str_array_length(args) < 3 || str_to_uint(args[0], &username_hash) < 0 || net_addr2ip(args[1], &ip) < 0 || str_to_uint(args[2], ×tamp) < 0) { @@ -252,6 +283,7 @@ conn->name); return FALSE; } + weak = args[3] != NULL && args[3][0] == 'w'; host = mail_host_lookup(conn->dir->mail_hosts, &ip); if (host == NULL) { @@ -260,12 +292,13 @@ return FALSE; } - director_user_refresh(conn, username_hash, host, timestamp, &user); + director_user_refresh(conn, username_hash, host, timestamp, weak, &user); return TRUE; } static bool -director_cmd_user(struct director_connection *conn, const char *const *args) +director_cmd_user(struct director_connection *conn, + const char *const *args) { unsigned int username_hash; struct ip_addr ip; @@ -286,8 +319,10 @@ } if (director_user_refresh(conn, username_hash, - host, ioloop_time, &user)) + host, ioloop_time, FALSE, &user)) { + i_assert(!user->weak); director_update_user(conn->dir, conn->host, user); + } return TRUE; } @@ -383,6 +418,54 @@ } static bool +director_cmd_user_weak(struct director_connection *conn, + const char *const *args) +{ + struct director_host *dir_host; + struct ip_addr ip; + unsigned int username_hash; + struct mail_host *host; + struct user *user; + struct director_host *src_host = conn->host; + bool weak = TRUE; + int ret; + + if ((ret = director_cmd_is_seen(conn, &args, &dir_host)) < 0) + return FALSE; + + if (str_array_length(args) != 2 || + str_to_uint(args[0], &username_hash) < 0 || + net_addr2ip(args[1], &ip) < 0) { + i_error("director(%s): Invalid USER-WEAK args", conn->name); + return FALSE; + } + + host = mail_host_lookup(conn->dir->mail_hosts, &ip); + if (host == NULL) { + /* we probably just removed this host. */ + return TRUE; + } + + if (ret > 0) { + /* The entire ring has seen this USER-WEAK. + make it non-weak now. */ + weak = FALSE; + src_host = conn->dir->self_host; + } + + if (director_user_refresh(conn, username_hash, + host, ioloop_time, weak, &user)) { + if (!user->weak) + director_update_user(conn->dir, src_host, user); + else { + director_update_user_weak(conn->dir, src_host, + dir_host, user); + } + } + return TRUE; +} + +static bool director_cmd_host_int(struct director_connection *conn, const char *const *args, struct director_host *dir_host) { @@ -666,11 +749,15 @@ /* only incoming connections get a full USER list, but outgoing connections can also receive USER updates during handshake and it wouldn't be safe to ignore them. */ - if (strcmp(cmd, "USER") == 0 && conn->me_received) { - if (conn->in) - return director_handshake_cmd_user(conn, args); - else + if (!conn->me_received) { + /* no USER updates until ME */ + } else if (conn->in && strcmp(cmd, "USER") == 0) { + return director_handshake_cmd_user(conn, args); + } else if (!conn->in) { + if (strcmp(cmd, "USER") == 0) return director_cmd_user(conn, args); + if (strcmp(cmd, "USER-WEAK") == 0) + return director_cmd_user_weak(conn, args); } /* both get DONE */ if (strcmp(cmd, "DONE") == 0 && !conn->handshake_received && @@ -702,7 +789,7 @@ return; } - dir->synced_minor_version = minor_version; + dir->ring_min_version = minor_version; if (!dir->ring_handshaked) { /* the ring is handshaked */ director_set_ring_handshaked(dir); @@ -852,6 +939,8 @@ if (strcmp(cmd, "USER") == 0) return director_cmd_user(conn, args); + if (strcmp(cmd, "USER-WEAK") == 0) + return director_cmd_user_weak(conn, args); if (strcmp(cmd, "HOST") == 0) return director_cmd_host(conn, args); if (strcmp(cmd, "HOST-REMOVE") == 0) @@ -951,20 +1040,17 @@ o_stream_cork(conn->output); while ((user = user_directory_iter_next(conn->user_iter)) != NULL) { - if (!user_directory_user_has_connections(conn->dir->users, - user)) { - /* user is already expired */ - continue; - } + T_BEGIN { + string_t *str = t_str_new(128); - T_BEGIN { - const char *line; - - line = t_strdup_printf("USER\t%u\t%s\t%u\n", - user->username_hash, - net_ip2addr(&user->host->ip), - user->timestamp); - director_connection_send(conn, line); + str_printfa(str, "USER\t%u\t%s\t%u", + user->username_hash, + net_ip2addr(&user->host->ip), + user->timestamp); + if (user->weak) + str_append(str, "\tw"); + str_append_c(str, '\n'); + director_connection_send(conn, str_c(str)); } T_END; if (o_stream_get_buffer_used_size(conn->output) >= OUTBUF_FLUSH_THRESHOLD) { diff -r d6fda337af15 -r 7a26c427fc78 src/director/director-request.c --- a/src/director/director-request.c Thu Mar 08 10:48:08 2012 +0200 +++ b/src/director/director-request.c Thu Mar 08 16:03:45 2012 +0200 @@ -117,6 +117,74 @@ ring_noconn_warning, dir); } +static bool director_request_existing(struct director *dir, struct user *user) +{ + struct mail_host *host; + + if (user->kill_state != USER_KILL_STATE_NONE) { + /* delay processing this user's connections until + its existing connections have been killed */ + return FALSE; + } + if (user->weak) { + /* wait for user to become non-weak */ + return FALSE; + } + if (!user_directory_user_is_near_expiring(dir->users, user)) + return TRUE; + + /* user is close to being expired. another director may have + already expired it. */ + host = mail_host_get_by_hash(dir->mail_hosts, user->username_hash); + if (!dir->ring_synced) { + /* try again later once ring is synced */ + return FALSE; + } + if (user->host == host) { + /* doesn't matter, other directors would + assign the user the same way regardless */ + return TRUE; + } + + /* We have to worry about two separate timepoints in here: + + a) some directors think the user isn't expiring, and + others think the user is near expiring + + b) some directors think the user is near expiring, and + others think the user has already expired + + What we don't have to worry about is: + + !c) some directors think the user isn't expiring, and + others think the user has already expired + + If !c) happens, the user might get redirected to different backends. + We'll use a large enough timeout between a) and b) states, so that + !c) should never happen. From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: auth: checkpassword code cleanup. Also fixed some e... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/71b64b7b2e63 changeset: 14311:71b64b7b2e63 user: Timo Sirainen date: Sat Mar 10 13:37:39 2012 +0200 description: auth: checkpassword code cleanup. Also fixed some error handling. Removes a lot of copy&pasted code between passdb and userdb. diffstat: src/auth/checkpassword-reply.c | 13 +- src/auth/db-checkpassword.c | 426 ++++++++++++++++++++++++++++++++++----- src/auth/db-checkpassword.h | 64 +---- src/auth/passdb-checkpassword.c | 248 +--------------------- src/auth/userdb-checkpassword.c | 232 ++------------------- 5 files changed, 443 insertions(+), 540 deletions(-) diffs (truncated from 1209 to 300 lines): diff -r 7a26c427fc78 -r 71b64b7b2e63 src/auth/checkpassword-reply.c --- a/src/auth/checkpassword-reply.c Thu Mar 08 16:03:45 2012 +0200 +++ b/src/auth/checkpassword-reply.c Sat Mar 10 13:37:39 2012 +0200 @@ -54,10 +54,21 @@ if (!gid_found) str_printfa(str, "userdb_gid=%s\t", dec2str(getgid())); + i_assert(str_len(str) > 0); + if (write_full(4, str_data(str), str_len(str)) < 0) { i_error("checkpassword: write_full() failed: %m"); exit(111); } authorized = getenv("AUTHORIZED"); - return authorized != NULL && strcmp(authorized, "2") == 0 ? 2 : 0; + if (authorized == NULL) { + /* authentication */ + return 0; + } else if (strcmp(authorized, "2") == 0) { + /* successful passdb/userdb lookup */ + return 2; + } else { + i_error("checkpassword: Script doesn't support passdb/userdb lookup"); + return 111; + } } diff -r 7a26c427fc78 -r 71b64b7b2e63 src/auth/db-checkpassword.c --- a/src/auth/db-checkpassword.c Thu Mar 08 16:03:45 2012 +0200 +++ b/src/auth/db-checkpassword.c Sat Mar 10 13:37:39 2012 +0200 @@ -4,9 +4,47 @@ #if defined(PASSDB_CHECKPASSWORD) || defined(USERDB_CHECKPASSWORD) +#include "lib-signals.h" +#include "buffer.h" +#include "str.h" +#include "ioloop.h" +#include "hash.h" +#include "execv-const.h" +#include "env-util.h" +#include "safe-memset.h" +#include "child-wait.h" #include "var-expand.h" #include "db-checkpassword.h" +#include +#include + +struct chkpw_auth_request { + struct db_checkpassword *db; + struct auth_request *request; + char *auth_password; + + db_checkpassword_callback_t *callback; + void *context; + + pid_t pid; + int fd_out, fd_in; + struct io *io_out, *io_in; + + string_t *input_buf; + unsigned int write_pos; + + int exit_status; + unsigned int exited:1; +}; + +struct db_checkpassword { + char *checkpassword_path, *checkpassword_reply_path; + + struct hash_table *clients; + struct child_wait *child_wait; +}; + static void env_put_extra_fields(const char *extra_fields) { const char *const *tmp; @@ -40,52 +78,143 @@ } } -void checkpassword_request_free(struct chkpw_auth_request *request) +static void checkpassword_request_free(struct chkpw_auth_request **_request) { + struct chkpw_auth_request *request = *_request; + + *_request = NULL; + + if (!request->exited) + child_wait_remove_pid(request->db->child_wait, request->pid); checkpassword_request_close(request); - if (request->input_buf != NULL) - str_free(&request->input_buf); - if (request->password != NULL) { - safe_memset(request->password, 0, strlen(request->password)); - i_free(request->password); + if (request->auth_password != NULL) { + safe_memset(request->auth_password, 0, + strlen(request->auth_password)); + i_free(request->auth_password); } + auth_request_unref(&request->request); + str_free(&request->input_buf); i_free(request); } -enum checkpassword_sigchld_handler_result -checkpassword_sigchld_handler(const struct child_wait_status *child_wait_status, - struct chkpw_auth_request *request) +static void checkpassword_finish(struct chkpw_auth_request **_request, + enum db_checkpassword_status status) { - int status = child_wait_status->status; - pid_t pid = child_wait_status->pid; + struct chkpw_auth_request *request = *_request; - if (request == NULL) { - i_error("checkpassword: sighandler called for unknown child %s", - dec2str(pid)); - return SIGCHLD_RESULT_UNKNOWN_CHILD; + *_request = NULL; + request->callback(request->request, status, request->context); + checkpassword_request_free(&request); +} + +static void checkpassword_internal_failure(struct chkpw_auth_request **request) +{ + checkpassword_finish(request, DB_CHECKPASSWORD_STATUS_INTERNAL_FAILURE); +} + +static void +checkpassword_request_finish_auth(struct chkpw_auth_request *request) +{ + switch (request->exit_status) { + /* vpopmail exit codes: */ + case 3: /* password fail / vpopmail user not found */ + case 12: /* null user name given */ + case 13: /* null password given */ + case 15: /* user has no password */ + case 20: /* invalid user/domain characters */ + case 21: /* system user not found */ + case 22: /* system user shadow entry not found */ + case 23: /* system password fail */ + + /* standard checkpassword exit codes: */ + case 1: + /* (1 is additionally defined in vpopmail for + "pop/smtp/webmal/ imap/access denied") */ + auth_request_log_info(request->request, "checkpassword", + "Login failed (status=%d)", + request->exit_status); + checkpassword_finish(&request, DB_CHECKPASSWORD_STATUS_FAILURE); + break; + case 0: + if (request->input_buf->used == 0) { + auth_request_log_error(request->request, "checkpassword", + "Received no input"); + checkpassword_internal_failure(&request); + break; + } + + auth_request_set_fields(request->request, + t_strsplit(str_c(request->input_buf), "\t"), NULL); + checkpassword_finish(&request, DB_CHECKPASSWORD_STATUS_OK); + break; + case 2: + /* checkpassword is called with wrong parameters? unlikely */ + auth_request_log_error(request->request, "checkpassword", + "Child %s exited with status 2 (tried to use " + "userdb-only checkpassword program for passdb?)", + dec2str(request->pid)); + checkpassword_internal_failure(&request); + break; + case 111: + /* temporary problem, treat as internal error */ + default: + /* whatever error.. */ + auth_request_log_error(request->request, "checkpassword", + "Child %s exited with status %d", + dec2str(request->pid), request->exit_status); + checkpassword_internal_failure(&request); + break; } +} - if (WIFSIGNALED(status)) { - i_error("checkpassword: Child %s died with signal %d", - dec2str(pid), WTERMSIG(status)); - return SIGCHLD_RESULT_DEAD_CHILD; - } else if (WIFEXITED(status)) { - request->exited = TRUE; - request->exit_status = WEXITSTATUS(status); +static void +checkpassword_request_finish_lookup(struct chkpw_auth_request *request) +{ + switch (request->exit_status) { + case 3: + /* User does not exist. */ + auth_request_log_info(request->request, "userdb-checkpassword", + "User unknown"); + checkpassword_finish(&request, DB_CHECKPASSWORD_STATUS_FAILURE); + break; + case 2: + /* This is intentionally not 0. checkpassword-reply exits with + 2 on success when AUTHORIZED is set. */ + if (request->input_buf->used == 0) { + auth_request_log_error(request->request, "checkpassword", + "Received no input"); + checkpassword_internal_failure(&request); + break; + } - auth_request_log_debug(request->request, - "checkpassword", "exit_status=%d", - request->exit_status); - return SIGCHLD_RESULT_OK; - } else { - /* shouldn't happen */ - auth_request_log_debug(request->request, "checkpassword", - "Child exited with status=%d", status); - return SIGCHLD_RESULT_UNKNOWN_ERROR; + auth_request_set_fields(request->request, + t_strsplit(str_c(request->input_buf), "\t"), NULL); + checkpassword_finish(&request, DB_CHECKPASSWORD_STATUS_OK); + break; + default: + /* whatever error... */ + auth_request_log_error(request->request, "userdb-checkpassword", + "Child %s exited with status %d", + dec2str(request->pid), request->exit_status); + checkpassword_internal_failure(&request); + break; } } +static void +checkpassword_request_half_finish(struct chkpw_auth_request *request) +{ + /* the process must have exited, and the input fd must have closed */ + if (!request->exited || request->fd_in != -1) + return; + + if (request->auth_password != NULL) + checkpassword_request_finish_auth(request); + else + checkpassword_request_finish_lookup(request); +} + static void env_put_auth_vars(struct auth_request *request) { const struct var_expand_table *tab; @@ -101,7 +230,7 @@ } } -void checkpassword_setup_env(struct auth_request *request) +static void checkpassword_setup_env(struct auth_request *request) { /* Besides passing the standard username and password in a pipe, also pass some other possibly interesting information @@ -146,7 +275,7 @@ env_put_auth_vars(request); } -const char * +static const char * checkpassword_get_cmd(struct auth_request *request, const char *args, const char *checkpassword_reply_path) { @@ -158,33 +287,34 @@ return t_strconcat(str_c(str), " ", checkpassword_reply_path, NULL); } -void checkpassword_child_input(struct chkpw_auth_request *request) +static void checkpassword_child_input(struct chkpw_auth_request *request) { unsigned char buf[1024]; ssize_t ret; ret = read(request->fd_in, buf, sizeof(buf)); - if (ret <= 0) { - if (ret < 0) { - auth_request_log_error(request->request, - "checkpassword", "read() failed: %m"); - } + if (ret > 0) { + str_append_n(request->input_buf, buf, ret); + return; + } - auth_request_log_debug(request->request, "checkpassword", - "Received no input"); - checkpassword_request_close(request); - request->half_finish_callback(request); + if (ret < 0) { + auth_request_log_error(request->request, + "checkpassword", "read() failed: %m"); + checkpassword_internal_failure(&request); + } else if (strchr(str_c(request->input_buf), '\n') != NULL) { From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: checkpassword: If username+password is too large, t... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/262ac078a1e4 changeset: 14312:262ac078a1e4 user: Timo Sirainen date: Sat Mar 10 13:47:16 2012 +0200 description: checkpassword: If username+password is too large, treat it as regular auth failure. diffstat: src/auth/db-checkpassword.c | 40 +++++++++++++++++++++++++++------------- 1 files changed, 27 insertions(+), 13 deletions(-) diffs (96 lines): diff -r 71b64b7b2e63 -r 262ac078a1e4 src/auth/db-checkpassword.c --- a/src/auth/db-checkpassword.c Sat Mar 10 13:37:39 2012 +0200 +++ b/src/auth/db-checkpassword.c Sat Mar 10 13:47:16 2012 +0200 @@ -19,6 +19,8 @@ #include #include +#define CHECKPASSWORD_MAX_REQUEST_LEN 512 + struct chkpw_auth_request { struct db_checkpassword *db; struct auth_request *request; @@ -32,7 +34,7 @@ struct io *io_out, *io_in; string_t *input_buf; - unsigned int write_pos; + unsigned int output_pos, output_len; int exit_status; unsigned int exited:1; @@ -326,7 +328,8 @@ size_t size; ssize_t ret; - buf = buffer_create_dynamic(pool_datastack_create(), 512+1); + buf = buffer_create_dynamic(pool_datastack_create(), + CHECKPASSWORD_MAX_REQUEST_LEN); buffer_append(buf, auth_request->user, strlen(auth_request->user)+1); if (request->auth_password != NULL) { buffer_append(buf, request->auth_password, @@ -337,15 +340,12 @@ buffer_append_c(buf, '\0'); data = buffer_get_data(buf, &size); - if (size > 512) { - auth_request_log_error(request->request, "checkpassword", - "output larger than 512 bytes: %"PRIuSIZE_T, size); - checkpassword_internal_failure(&request); - return; - } + i_assert(size == request->output_len); + /* already checked this */ + i_assert(size <= CHECKPASSWORD_MAX_REQUEST_LEN); - ret = write(request->fd_out, data + request->write_pos, - size - request->write_pos); + ret = write(request->fd_out, data + request->output_pos, + size - request->output_pos); if (ret <= 0) { if (ret < 0) { auth_request_log_error(request->request, @@ -358,8 +358,8 @@ return; } - request->write_pos += ret; - if (request->write_pos < size) + request->output_pos += ret; + if (request->output_pos < size) return; /* finished sending the data */ @@ -442,9 +442,22 @@ void *context) { struct chkpw_auth_request *chkpw_auth_request; + unsigned int output_len; int fd_in[2], fd_out[2]; pid_t pid; + /* \0 \0 timestamp \0 */ + output_len = strlen(request->user) + 3; + if (auth_password != NULL) + output_len += strlen(auth_password); + if (output_len > CHECKPASSWORD_MAX_REQUEST_LEN) { + auth_request_log_info(request, "checkpassword", + "Username+password combination too long (%u bytes)", + output_len); + callback(request, DB_CHECKPASSWORD_STATUS_FAILURE, context); + return; + } + fd_in[0] = -1; if (pipe(fd_in) < 0 || pipe(fd_out) < 0) { auth_request_log_error(request, "checkpassword", @@ -497,7 +510,8 @@ chkpw_auth_request->fd_out = fd_out[1]; chkpw_auth_request->auth_password = i_strdup(auth_password); chkpw_auth_request->request = request; - chkpw_auth_request->input_buf = str_new(default_pool, 512); + chkpw_auth_request->output_len = output_len; + chkpw_auth_request->input_buf = str_new(default_pool, 256); chkpw_auth_request->callback = callback; chkpw_auth_request->context = context; From dovecot at dovecot.org Sun May 20 03:26:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:26 +0300 Subject: dovecot-2.2: checkpassword: Error handling fix Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/e3c1f3f93add changeset: 14313:e3c1f3f93add user: Timo Sirainen date: Sat Mar 10 13:49:12 2012 +0200 description: checkpassword: Error handling fix diffstat: src/auth/db-checkpassword.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diffs (16 lines): diff -r 262ac078a1e4 -r e3c1f3f93add src/auth/db-checkpassword.c --- a/src/auth/db-checkpassword.c Sat Mar 10 13:47:16 2012 +0200 +++ b/src/auth/db-checkpassword.c Sat Mar 10 13:49:12 2012 +0200 @@ -86,8 +86,11 @@ *_request = NULL; - if (!request->exited) + if (!request->exited) { + hash_table_remove(request->db->clients, + POINTER_CAST(request->pid)); child_wait_remove_pid(request->db->child_wait, request->pid); + } checkpassword_request_close(request); if (request->auth_password != NULL) { From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: checkpassword: Implemented support for credentials ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/f7c42aa64381 changeset: 14315:f7c42aa64381 user: Timo Sirainen date: Sat Mar 10 14:17:37 2012 +0200 description: checkpassword: Implemented support for credentials lookup. diffstat: src/auth/db-checkpassword.c | 5 ++ src/auth/passdb-checkpassword.c | 79 ++++++++++++++++++++++++++++++++++++++-- 2 files changed, 79 insertions(+), 5 deletions(-) diffs (134 lines): diff -r e5ed29ef593e -r f7c42aa64381 src/auth/db-checkpassword.c --- a/src/auth/db-checkpassword.c Sat Mar 10 14:11:32 2012 +0200 +++ b/src/auth/db-checkpassword.c Sat Mar 10 14:17:37 2012 +0200 @@ -394,6 +394,11 @@ special checkpassword program which knows how to handle this. */ env_put("AUTHORIZED=1"); + if (request->credentials_scheme != NULL) { + /* passdb credentials lookup */ + env_put(t_strdup_printf("SCHEME=%s", + request->credentials_scheme)); + } } checkpassword_setup_env(request); cmd = checkpassword_get_cmd(request, db->checkpassword_path, diff -r e5ed29ef593e -r f7c42aa64381 src/auth/passdb-checkpassword.c --- a/src/auth/passdb-checkpassword.c Sat Mar 10 14:11:32 2012 +0200 +++ b/src/auth/passdb-checkpassword.c Sat Mar 10 14:17:37 2012 +0200 @@ -5,6 +5,7 @@ #ifdef PASSDB_CHECKPASSWORD +#include "password-scheme.h" #include "db-checkpassword.h" struct checkpassword_passdb_module { @@ -19,19 +20,39 @@ void *context) { verify_plain_callback_t *callback = context; + const char *scheme, *crypted_pass = NULL; + unsigned int i; switch (status) { case DB_CHECKPASSWORD_STATUS_INTERNAL_FAILURE: callback(PASSDB_RESULT_INTERNAL_FAILURE, request); - break; + return; case DB_CHECKPASSWORD_STATUS_FAILURE: callback(PASSDB_RESULT_PASSWORD_MISMATCH, request); - break; + return; case DB_CHECKPASSWORD_STATUS_OK: - auth_request_set_fields(request, extra_fields, NULL); - callback(PASSDB_RESULT_OK, request); break; } + for (i = 0; extra_fields[i] != NULL; i++) { + if (strncmp(extra_fields[i], "password=", 9) == 0) + crypted_pass = extra_fields[i]+9; + else if (extra_fields[i][0] != '\0') { + auth_request_set_field_keyvalue(request, + extra_fields[i], NULL); + } + } + if (crypted_pass != NULL) { + /* for cache */ + scheme = password_get_scheme(&crypted_pass); + if (scheme != NULL) { + auth_request_set_field(request, "password", + crypted_pass, scheme); + } else { + auth_request_log_error(request, "checkpassword", + "password field returned without {scheme} prefix"); + } + } + callback(PASSDB_RESULT_OK, request); } static void @@ -46,6 +67,54 @@ auth_checkpassword_callback, callback); } +static void +credentials_checkpassword_callback(struct auth_request *request, + enum db_checkpassword_status status, + const char *const *extra_fields, + void *context) +{ + lookup_credentials_callback_t *callback = context; + const char *scheme, *crypted_pass = NULL; + unsigned int i; + + switch (status) { + case DB_CHECKPASSWORD_STATUS_INTERNAL_FAILURE: + callback(PASSDB_RESULT_INTERNAL_FAILURE, NULL, 0, request); + return; + case DB_CHECKPASSWORD_STATUS_FAILURE: + callback(PASSDB_RESULT_USER_UNKNOWN, NULL, 0, request); + return; + case DB_CHECKPASSWORD_STATUS_OK: + break; + } + for (i = 0; extra_fields[i] != NULL; i++) { + if (strncmp(extra_fields[i], "password=", 9) == 0) + crypted_pass = extra_fields[i]+9; + else if (extra_fields[i][0] != '\0') { + auth_request_set_field_keyvalue(request, + extra_fields[i], NULL); + } + } + scheme = password_get_scheme(&crypted_pass); + if (scheme == NULL) + scheme = request->credentials_scheme; + + passdb_handle_credentials(PASSDB_RESULT_OK, crypted_pass, scheme, + callback, request); +} + +static void +checkpassword_lookup_credentials(struct auth_request *request, + lookup_credentials_callback_t *callback) +{ + struct passdb_module *_module = request->passdb->passdb; + struct checkpassword_passdb_module *module = + (struct checkpassword_passdb_module *)_module; + + db_checkpassword_call(module->db, request, NULL, + credentials_checkpassword_callback, callback); +} + static struct passdb_module * checkpassword_preinit(pool_t pool, const char *args) { @@ -76,7 +145,7 @@ checkpassword_deinit, checkpassword_verify_plain, - NULL, + checkpassword_lookup_credentials, NULL }; #else From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: checkpassword: Escape transferred extra fields prop... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/e5ed29ef593e changeset: 14314:e5ed29ef593e user: Timo Sirainen date: Sat Mar 10 14:11:32 2012 +0200 description: checkpassword: Escape transferred extra fields properly. diffstat: src/auth/auth-request.c | 30 ++++++++++++++++++------------ src/auth/auth-request.h | 3 +++ src/auth/checkpassword-reply.c | 14 +++++++++++--- src/auth/db-checkpassword.c | 20 ++++++++++---------- src/auth/db-checkpassword.h | 1 + src/auth/passdb-checkpassword.c | 5 ++++- src/auth/userdb-checkpassword.c | 8 ++++++++ 7 files changed, 55 insertions(+), 26 deletions(-) diffs (240 lines): diff -r e3c1f3f93add -r e5ed29ef593e src/auth/auth-request.c --- a/src/auth/auth-request.c Sat Mar 10 13:49:12 2012 +0200 +++ b/src/auth/auth-request.c Sat Mar 10 14:11:32 2012 +0200 @@ -1287,25 +1287,31 @@ } } +void auth_request_set_field_keyvalue(struct auth_request *request, + const char *field, + const char *default_scheme) +{ + const char *key, *value; + + value = strchr(field, '='); + if (value == NULL) { + key = field; + value = ""; + } else { + key = t_strdup_until(field, value); + value++; + } + auth_request_set_field(request, key, value, default_scheme); +} + void auth_request_set_fields(struct auth_request *request, const char *const *fields, const char *default_scheme) { - const char *key, *value; - for (; *fields != NULL; fields++) { if (**fields == '\0') continue; - - value = strchr(*fields, '='); - if (value == NULL) { - key = *fields; - value = ""; - } else { - key = t_strdup_until(*fields, value); - value++; - } - auth_request_set_field(request, key, value, default_scheme); + auth_request_set_field_keyvalue(request, *fields, default_scheme); } } diff -r e3c1f3f93add -r e5ed29ef593e src/auth/auth-request.h --- a/src/auth/auth-request.h Sat Mar 10 13:49:12 2012 +0200 +++ b/src/auth/auth-request.h Sat Mar 10 14:11:32 2012 +0200 @@ -171,6 +171,9 @@ void auth_request_set_field(struct auth_request *request, const char *name, const char *value, const char *default_scheme); +void auth_request_set_field_keyvalue(struct auth_request *request, + const char *field, + const char *default_scheme); void auth_request_set_fields(struct auth_request *request, const char *const *fields, const char *default_scheme); diff -r e3c1f3f93add -r e5ed29ef593e src/auth/checkpassword-reply.c --- a/src/auth/checkpassword-reply.c Sat Mar 10 13:49:12 2012 +0200 +++ b/src/auth/checkpassword-reply.c Sat Mar 10 14:11:32 2012 +0200 @@ -2,6 +2,7 @@ #include "lib.h" #include "str.h" +#include "strescape.h" #include "write-full.h" #include @@ -23,7 +24,9 @@ i_error("checkpassword: USER contains TAB"); return 1; } - str_printfa(str, "user=%s\t", user); + str_printfa(str, "user="); + str_tabescape_write(str, user); + str_append_c(str, '\t'); } home = getenv("HOME"); @@ -32,7 +35,9 @@ i_error("checkpassword: HOME contains TAB"); return 1; } - str_printfa(str, "userdb_home=%s\t", home); + str_printfa(str, "userdb_home="); + str_tabescape_write(str, home); + str_append_c(str, '\t'); } extra_env = getenv("EXTRA"); @@ -45,7 +50,10 @@ uid_found = TRUE; else if (strcmp(key, "userdb_gid") == 0) gid_found = TRUE; - str_printfa(str, "%s=%s\t", key, value); + str_tabescape_write(str, key); + str_append_c(str, '='); + str_tabescape_write(str, value); + str_append_c(str, '\t'); } } } diff -r e3c1f3f93add -r e5ed29ef593e src/auth/db-checkpassword.c --- a/src/auth/db-checkpassword.c Sat Mar 10 13:49:12 2012 +0200 +++ b/src/auth/db-checkpassword.c Sat Mar 10 14:11:32 2012 +0200 @@ -12,6 +12,7 @@ #include "execv-const.h" #include "env-util.h" #include "safe-memset.h" +#include "strescape.h" #include "child-wait.h" #include "var-expand.h" #include "db-checkpassword.h" @@ -107,9 +108,13 @@ enum db_checkpassword_status status) { struct chkpw_auth_request *request = *_request; + const char *const *extra_fields; *_request = NULL; - request->callback(request->request, status, request->context); + + extra_fields = t_strsplit_tabescaped(str_c(request->input_buf)); + request->callback(request->request, status, extra_fields, + request->context); checkpassword_request_free(&request); } @@ -148,9 +153,6 @@ checkpassword_internal_failure(&request); break; } - - auth_request_set_fields(request->request, - t_strsplit(str_c(request->input_buf), "\t"), NULL); checkpassword_finish(&request, DB_CHECKPASSWORD_STATUS_OK); break; case 2: @@ -192,9 +194,6 @@ checkpassword_internal_failure(&request); break; } - - auth_request_set_fields(request->request, - t_strsplit(str_c(request->input_buf), "\t"), NULL); checkpassword_finish(&request, DB_CHECKPASSWORD_STATUS_OK); break; default: @@ -457,7 +456,8 @@ auth_request_log_info(request, "checkpassword", "Username+password combination too long (%u bytes)", output_len); - callback(request, DB_CHECKPASSWORD_STATUS_FAILURE, context); + callback(request, DB_CHECKPASSWORD_STATUS_FAILURE, + NULL, context); return; } @@ -470,7 +470,7 @@ (void)close(fd_in[1]); } callback(request, DB_CHECKPASSWORD_STATUS_INTERNAL_FAILURE, - context); + NULL, context); return; } @@ -483,7 +483,7 @@ (void)close(fd_out[0]); (void)close(fd_out[1]); callback(request, DB_CHECKPASSWORD_STATUS_INTERNAL_FAILURE, - context); + NULL, context); return; } diff -r e3c1f3f93add -r e5ed29ef593e src/auth/db-checkpassword.h --- a/src/auth/db-checkpassword.h Sat Mar 10 13:49:12 2012 +0200 +++ b/src/auth/db-checkpassword.h Sat Mar 10 14:11:32 2012 +0200 @@ -12,6 +12,7 @@ typedef void db_checkpassword_callback_t(struct auth_request *request, enum db_checkpassword_status status, + const char *const *extra_fields, void *context); struct db_checkpassword * diff -r e3c1f3f93add -r e5ed29ef593e src/auth/passdb-checkpassword.c --- a/src/auth/passdb-checkpassword.c Sat Mar 10 13:49:12 2012 +0200 +++ b/src/auth/passdb-checkpassword.c Sat Mar 10 14:11:32 2012 +0200 @@ -14,7 +14,9 @@ static void auth_checkpassword_callback(struct auth_request *request, - enum db_checkpassword_status status, void *context) + enum db_checkpassword_status status, + const char *const *extra_fields, + void *context) { verify_plain_callback_t *callback = context; @@ -26,6 +28,7 @@ callback(PASSDB_RESULT_PASSWORD_MISMATCH, request); break; case DB_CHECKPASSWORD_STATUS_OK: + auth_request_set_fields(request, extra_fields, NULL); callback(PASSDB_RESULT_OK, request); break; } diff -r e3c1f3f93add -r e5ed29ef593e src/auth/userdb-checkpassword.c --- a/src/auth/userdb-checkpassword.c Sat Mar 10 13:49:12 2012 +0200 +++ b/src/auth/userdb-checkpassword.c Sat Mar 10 14:11:32 2012 +0200 @@ -15,9 +15,11 @@ static void userdb_checkpassword_callback(struct auth_request *request, enum db_checkpassword_status status, + const char *const *extra_fields, void *context) { userdb_callback_t *callback = context; + unsigned int i; switch (status) { case DB_CHECKPASSWORD_STATUS_INTERNAL_FAILURE: @@ -27,6 +29,12 @@ callback(USERDB_RESULT_USER_UNKNOWN, request); break; case DB_CHECKPASSWORD_STATUS_OK: + for (i = 0; extra_fields[i] != NULL; i++) { + if (strncmp(extra_fields[i], "userdb_", 7) != 0) + continue; + auth_request_set_field_keyvalue(request, + extra_fields[i], NULL); + } callback(USERDB_RESULT_OK, request); break; } From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: checkpassword: Set CREDENTIALS_LOOKUP=1 for passdb ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/495bda325391 changeset: 14316:495bda325391 user: Timo Sirainen date: Sat Mar 10 14:28:48 2012 +0200 description: checkpassword: Set CREDENTIALS_LOOKUP=1 for passdb lookups diffstat: src/auth/db-checkpassword.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff -r f7c42aa64381 -r 495bda325391 src/auth/db-checkpassword.c --- a/src/auth/db-checkpassword.c Sat Mar 10 14:17:37 2012 +0200 +++ b/src/auth/db-checkpassword.c Sat Mar 10 14:28:48 2012 +0200 @@ -396,6 +396,7 @@ env_put("AUTHORIZED=1"); if (request->credentials_scheme != NULL) { /* passdb credentials lookup */ + env_put("CREDENTIALS_LOOKUP=1"); env_put(t_strdup_printf("SCHEME=%s", request->credentials_scheme)); } From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: stats: UPDATE-CMD for an existing command wasn't ha... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/30802a608490 changeset: 14317:30802a608490 user: Timo Sirainen date: Sat Mar 10 15:03:11 2012 +0200 description: stats: UPDATE-CMD for an existing command wasn't handled properly. The command's stats themselves weren't updated. Also due to recent change updating session's stats from a command no longer works properly, because the session could also be updated in the middle of the command. diffstat: src/stats/mail-command.c | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diffs (20 lines): diff -r 495bda325391 -r 30802a608490 src/stats/mail-command.c --- a/src/stats/mail-command.c Sat Mar 10 14:28:48 2012 +0200 +++ b/src/stats/mail-command.c Sat Mar 10 15:03:11 2012 +0200 @@ -131,7 +131,6 @@ cmd = mail_command_add(session, args[3], args[4]); cmd->id = cmd_id; cmd->stats = stats; - diff_stats = stats; session->num_cmds++; session->user->num_cmds++; @@ -146,7 +145,7 @@ return -1; } cmd->last_update = ioloop_timeval; - mail_stats_add(&session->stats, &diff_stats); + mail_stats_add(&cmd->stats, &diff_stats); } if (done) { cmd->id = 0; From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: lib-storage: Update search's cost more widely and l... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/00887de4ad32 changeset: 14319:00887de4ad32 user: Timo Sirainen date: Sat Mar 10 15:44:27 2012 +0200 description: lib-storage: Update search's cost more widely and less often. diffstat: src/lib-storage/index/index-search.c | 31 ++++++++++++++++++++----------- 1 files changed, 20 insertions(+), 11 deletions(-) diffs (69 lines): diff -r 4a82ff720781 -r 00887de4ad32 src/lib-storage/index/index-search.c --- a/src/lib-storage/index/index-search.c Sat Mar 10 15:07:23 2012 +0200 +++ b/src/lib-storage/index/index-search.c Sat Mar 10 15:44:27 2012 +0200 @@ -1220,17 +1220,12 @@ static int search_match_once(struct index_search_context *ctx) { - unsigned long long cost1, cost2; int ret; - cost1 = search_get_cost(ctx->cur_mail->transaction); ret = mail_search_args_foreach(ctx->mail_ctx.args->args, search_cached_arg, ctx); if (ret < 0) ret = search_arg_match_text(ctx->mail_ctx.args->args, ctx); - - cost2 = search_get_cost(ctx->cur_mail->transaction); - ctx->cost += cost2 - cost1; return ret; } @@ -1423,7 +1418,8 @@ struct mail_search_context *_ctx = &ctx->mail_ctx; struct mailbox *box = _ctx->transaction->box; struct index_mail *imail = (struct index_mail *)mail; - int match; + unsigned long long cost1, cost2; + int match, ret; if (search_would_block(ctx)) { /* this lookup is useful when a large number of @@ -1437,6 +1433,8 @@ mail_search_args_reset(_ctx->args->args, FALSE); + cost1 = search_get_cost(mail->transaction); + ret = -1; while (box->v.search_next_update_seq(_ctx)) { mail_set_seq(mail, _ctx->seq); @@ -1458,12 +1456,23 @@ mail_search_args_reset(_ctx->args->args, FALSE); - if (match != 0) - return 1; - if (search_would_block(ctx)) - return 0; + if (match != 0) { + ret = 1; + break; + } + + cost2 = search_get_cost(mail->transaction); + ctx->cost += cost2 - cost1; + cost1 = cost2; + + if (search_would_block(ctx)) { + ret = 0; + break; + } } - return -1; + cost2 = search_get_cost(mail->transaction); + ctx->cost += cost2 - cost1; + return ret; } struct mail *index_search_get_mail(struct index_search_context *ctx) From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: doveadm config: Don't fail on invalid config (e.g. ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/4a82ff720781 changeset: 14318:4a82ff720781 user: Timo Sirainen date: Sat Mar 10 15:07:23 2012 +0200 description: doveadm config: Don't fail on invalid config (e.g. plugin not found) This command is often used in init scripts and such to stop/restart Dovecot. diffstat: src/doveadm/doveadm.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff -r 30802a608490 -r 4a82ff720781 src/doveadm/doveadm.c --- a/src/doveadm/doveadm.c Sat Mar 10 15:03:11 2012 +0200 +++ b/src/doveadm/doveadm.c Sat Mar 10 15:07:23 2012 +0200 @@ -331,6 +331,7 @@ doveadm_register_cmd(doveadm_commands[i]); if (cmd_name != NULL && (quick_init || + strcmp(cmd_name, "config") == 0 || strcmp(cmd_name, "stop") == 0 || strcmp(cmd_name, "reload") == 0)) { /* special case commands: even if there is something wrong From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: stats: Fixes to handling per-command stats updates. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/96ad6da5c902 changeset: 14320:96ad6da5c902 user: Timo Sirainen date: Sat Mar 10 15:53:39 2012 +0200 description: stats: Fixes to handling per-command stats updates. diffstat: src/plugins/imap-stats/imap-stats-plugin.c | 9 ++- src/stats/mail-command.c | 67 ++++++++++++++++++++--------- 2 files changed, 51 insertions(+), 25 deletions(-) diffs (128 lines): diff -r 00887de4ad32 -r 96ad6da5c902 src/plugins/imap-stats/imap-stats-plugin.c --- a/src/plugins/imap-stats/imap-stats-plugin.c Sat Mar 10 15:44:27 2012 +0200 +++ b/src/plugins/imap-stats/imap-stats-plugin.c Sat Mar 10 15:53:39 2012 +0200 @@ -56,7 +56,6 @@ struct mail_stats stats, pre_trans_stats, trans_stats; unsigned int args_pos = 0; string_t *str; - bool done; if (scmd == NULL) return; @@ -76,11 +75,13 @@ str_append(str, "UPDATE-CMD\t"); str_append(str, guid_128_to_string(suser->session_guid)); - done = cmd->state == CLIENT_COMMAND_STATE_DONE; - str_printfa(str, "\t%u\t%d\t", scmd->id, done); + str_printfa(str, "\t%u\t", scmd->id); + if (cmd->state == CLIENT_COMMAND_STATE_DONE) + str_append_c(str, 'd'); if (scmd->continued) + str_append_c(str, 'c'); + else { str_append_c(str, '\t'); - else { str_append(str, cmd->name); str_append_c(str, '\t'); args_pos = str_len(str); diff -r 00887de4ad32 -r 96ad6da5c902 src/stats/mail-command.c --- a/src/stats/mail-command.c Sat Mar 10 15:44:27 2012 +0200 +++ b/src/stats/mail-command.c Sat Mar 10 15:53:39 2012 +0200 @@ -100,11 +100,12 @@ struct mail_command *cmd; struct mail_stats stats, diff_stats; const char *error; - unsigned int cmd_id; - bool done; + unsigned int i, cmd_id; + bool done = FALSE, continued = FALSE; - /* [key=value ..] */ - if (str_array_length(args) < 4) { + /* [d] [key=value ..] + c[d] [key=value ..] */ + if (str_array_length(args) < 3) { *error_r = "UPDATE-CMD: Too few parameters"; return -1; } @@ -115,38 +116,62 @@ *error_r = "UPDATE-CMD: Invalid command id"; return -1; } - if (strcmp(args[2], "0") != 0 && - strcmp(args[2], "1") != 0) { - *error_r = "UPDATE-CMD: Invalid done parameter"; - return -1; - } - done = args[2][0] == '1'; - if (mail_stats_parse(args+5, &stats, error_r) < 0) { - *error_r = t_strconcat("UPDATE-CMD: ", *error_r, NULL); - return -1; + for (i = 0; args[2][i] != '\0'; i++) { + switch (args[2][i]) { + case 'd': + done = TRUE; + break; + case 'c': + continued = TRUE; + break; + default: + *error_r = "UPDATE-CMD: Invalid flags parameter"; + return -1; + } } cmd = mail_command_find(session, cmd_id); - if (cmd == NULL) { + if (!continued) { + /* new command */ + if (cmd != NULL) { + *error_r = "UPDATE-CMD: Duplicate new command id"; + return -1; + } + if (str_array_length(args) < 5) { + *error_r = "UPDATE-CMD: Too few parameters"; + return -1; + } cmd = mail_command_add(session, args[3], args[4]); cmd->id = cmd_id; - cmd->stats = stats; + session->highest_cmd_id = + I_MAX(session->highest_cmd_id, cmd_id); session->num_cmds++; session->user->num_cmds++; session->user->domain->num_cmds++; if (session->ip != NULL) session->ip->num_cmds++; + args += 5; } else { - if (!mail_stats_diff(&cmd->stats, &stats, &diff_stats, - &error)) { - *error_r = t_strconcat("UPDATE-CMD: stats shrank: ", - error, NULL); - return -1; + if (cmd == NULL) { + /* already expired command, ignore */ + i_warning("UPDATE-CMD: Already expired"); + return 0; } + args += 3; cmd->last_update = ioloop_timeval; - mail_stats_add(&cmd->stats, &diff_stats); } + if (mail_stats_parse(args, &stats, error_r) < 0) { + *error_r = t_strconcat("UPDATE-CMD: ", *error_r, NULL); + return -1; + } + if (!mail_stats_diff(&cmd->stats, &stats, &diff_stats, &error)) { + *error_r = t_strconcat("UPDATE-CMD: stats shrank: ", + error, NULL); + return -1; + } + mail_stats_add(&cmd->stats, &diff_stats); + if (done) { cmd->id = 0; mail_command_unref(&cmd); From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: mbox: Fixed accessing v1.x mbox index files without... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/1289b79241bb changeset: 14321:1289b79241bb user: Timo Sirainen date: Sat Mar 10 18:32:34 2012 +0200 description: mbox: Fixed accessing v1.x mbox index files without errors. diffstat: src/lib-storage/index/mbox/mbox-sync.c | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diffs (17 lines): diff -r 96ad6da5c902 -r 1289b79241bb src/lib-storage/index/mbox/mbox-sync.c --- a/src/lib-storage/index/mbox/mbox-sync.c Sat Mar 10 15:53:39 2012 +0200 +++ b/src/lib-storage/index/mbox/mbox-sync.c Sat Mar 10 18:32:34 2012 +0200 @@ -1394,6 +1394,13 @@ &data, &data_size); if (data_size != sizeof(mbox->mbox_hdr) || memcmp(data, &mbox->mbox_hdr, data_size) != 0) { + if (data_size != sizeof(mbox->mbox_hdr)) { + /* upgrading from v1.x */ + mail_index_ext_resize(trans, mbox->mbox_ext_idx, + sizeof(mbox->mbox_hdr), + sizeof(uint64_t), + sizeof(uint64_t)); + } mail_index_update_header_ext(trans, mbox->mbox_ext_idx, 0, &mbox->mbox_hdr, sizeof(mbox->mbox_hdr)); From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: Makefile: Fixed header filename. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/75f1f39c547f changeset: 14323:75f1f39c547f user: Timo Sirainen date: Sun Mar 11 14:50:05 2012 +0200 description: Makefile: Fixed header filename. diffstat: src/replication/aggregator/Makefile.am | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (9 lines): diff -r 2c5a7cc1023e -r 75f1f39c547f src/replication/aggregator/Makefile.am --- a/src/replication/aggregator/Makefile.am Sun Mar 11 09:08:08 2012 +0200 +++ b/src/replication/aggregator/Makefile.am Sun Mar 11 14:50:05 2012 +0200 @@ -23,4 +23,4 @@ noinst_HEADERS = \ aggregator-settings.h \ notify-connection.h \ - replicator-connection.c + replicator-connection.h From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: Makefile: Delete dovecot-config on distclean Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/e75446fc828c changeset: 14324:e75446fc828c user: Timo Sirainen date: Mon Mar 12 12:59:54 2012 +0200 description: Makefile: Delete dovecot-config on distclean diffstat: Makefile.am | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diffs (14 lines): diff -r 75f1f39c547f -r e75446fc828c Makefile.am --- a/Makefile.am Sun Mar 11 14:50:05 2012 +0200 +++ b/Makefile.am Mon Mar 12 12:59:54 2012 +0200 @@ -75,7 +75,9 @@ CLEANFILES += $systedmsystemunit_DATA endif -DISTCLEANFILES = $(top_builddir)/dovecot-version.h +DISTCLEANFILES = \ + $(top_builddir)/dovecot-version.h \ + $(top_builddir)/dovecot-config distcheck-hook: if which scan-build > /dev/null; then \ From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: director: If request times out, log more information. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/1b37d6422905 changeset: 14325:1b37d6422905 user: Timo Sirainen date: Mon Mar 12 13:15:26 2012 +0200 description: director: If request times out, log more information. diffstat: src/director/director-request.c | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-) diffs (26 lines): diff -r e75446fc828c -r 1b37d6422905 src/director/director-request.c --- a/src/director/director-request.c Mon Mar 12 12:59:54 2012 +0200 +++ b/src/director/director-request.c Mon Mar 12 13:15:26 2012 +0200 @@ -26,6 +26,7 @@ director_request_get_timeout_error(struct director_request *request) { string_t *str = t_str_new(128); + struct user *user; unsigned int secs; str_printfa(str, "Timeout - queued for %u secs (", @@ -40,6 +41,14 @@ else str_printfa(str, "Ring not synced for %u secs", secs); } + + user = user_directory_lookup(request->dir->users, + request->username_hash); + if (user != NULL) { + if (user->weak) + str_append(str, ", weak user"); + str_printfa(str, ", ts=%u", user->timestamp); + } str_append_c(str, ')'); return str_c(str); } From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: director: Improved request timeout error more. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d27b6724db32 changeset: 14326:d27b6724db32 user: Timo Sirainen date: Mon Mar 12 13:17:50 2012 +0200 description: director: Improved request timeout error more. diffstat: src/director/director-request.c | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diffs (22 lines): diff -r 1b37d6422905 -r d27b6724db32 src/director/director-request.c --- a/src/director/director-request.c Mon Mar 12 13:15:26 2012 +0200 +++ b/src/director/director-request.c Mon Mar 12 13:17:50 2012 +0200 @@ -35,7 +35,7 @@ if (request->dir->ring_last_sync_time == 0) str_append(str, "Ring has never been synced"); else { - secs =ioloop_time - request->dir->ring_last_sync_time; + secs = ioloop_time - request->dir->ring_last_sync_time; if (request->dir->ring_synced) str_printfa(str, "Ring synced for %u secs", secs); else @@ -47,7 +47,8 @@ if (user != NULL) { if (user->weak) str_append(str, ", weak user"); - str_printfa(str, ", ts=%u", user->timestamp); + str_printfa(str, ", user refreshed %u secs ago", + ioloop_time - user->timestamp); } str_append_c(str, ')'); return str_c(str); From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: fts: Added fts_index_timeout setting to abort searc... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/4dd97a92691a changeset: 14327:4dd97a92691a user: Timo Sirainen date: Mon Mar 12 13:45:19 2012 +0200 description: fts: Added fts_index_timeout setting to abort search if indexing hasn't finished by then. This timeout shows up to client as: tag NO [INUSE] Timeout while waiting for indexing to finish diffstat: src/plugins/fts/Makefile.am | 1 + src/plugins/fts/fts-indexer.c | 25 +++++++++++++++++++++++-- src/plugins/fts/fts-storage.c | 26 +++++++++++++++++++++++++- src/plugins/fts/fts-storage.h | 1 + 4 files changed, 50 insertions(+), 3 deletions(-) diffs (172 lines): diff -r d27b6724db32 -r 4dd97a92691a src/plugins/fts/Makefile.am --- a/src/plugins/fts/Makefile.am Mon Mar 12 13:17:50 2012 +0200 +++ b/src/plugins/fts/Makefile.am Mon Mar 12 13:45:19 2012 +0200 @@ -3,6 +3,7 @@ AM_CPPFLAGS = \ -I$(top_srcdir)/src/lib \ + -I$(top_srcdir)/src/lib-settings \ -I$(top_srcdir)/src/lib-mail \ -I$(top_srcdir)/src/lib-index \ -I$(top_srcdir)/src/lib-storage \ diff -r d27b6724db32 -r 4dd97a92691a src/plugins/fts/fts-indexer.c --- a/src/plugins/fts/fts-indexer.c Mon Mar 12 13:17:50 2012 +0200 +++ b/src/plugins/fts/fts-indexer.c Mon Mar 12 13:45:19 2012 +0200 @@ -7,6 +7,7 @@ #include "write-full.h" #include "strescape.h" #include "time-util.h" +#include "settings-parser.h" #include "mail-user.h" #include "mail-storage-private.h" #include "fts-api.h" @@ -23,6 +24,7 @@ struct timeval search_start_time, last_notify; unsigned int percentage; + unsigned int timeout_secs; char *path; int fd; @@ -93,7 +95,7 @@ struct fts_indexer_context *ctx; struct mailbox_status status; uint32_t last_uid, seq1, seq2; - const char *path, *cmd; + const char *path, *cmd, *value, *error; int fd; if (fts_backend_get_last_uid(backend, box, &last_uid) < 0) @@ -126,6 +128,13 @@ ctx->input = i_stream_create_fd(fd, 128, FALSE); ctx->search_start_time = ioloop_timeval; + value = mail_user_plugin_getenv(box->storage->user, "fts_index_timeout"); + if (value != NULL) { + if (settings_get_time(value, &ctx->timeout_secs, &error) < 0) + i_error("Invalid fts_index_timeout setting: %s", error); + } + + *ctx_r = ctx; return 1; } @@ -214,12 +223,24 @@ int fts_indexer_more(struct fts_indexer_context *ctx) { - int ret; + int ret, diff; if ((ret = fts_indexer_more_int(ctx)) < 0) { + mail_storage_set_internal_error(ctx->box->storage); ctx->failed = TRUE; return -1; } + + if (ctx->timeout_secs > 0) { + diff = ioloop_time - ctx->search_start_time.tv_sec; + if (diff > (int)ctx->timeout_secs) { + mail_storage_set_error(ctx->box->storage, + MAIL_ERROR_INUSE, + "Timeout while waiting for indexing to finish"); + ctx->failed = TRUE; + return -1; + } + } if (ret == 0) fts_indexer_notify(ctx); return ret; diff -r d27b6724db32 -r 4dd97a92691a src/plugins/fts/fts-storage.c --- a/src/plugins/fts/fts-storage.c Mon Mar 12 13:17:50 2012 +0200 +++ b/src/plugins/fts/fts-storage.c Mon Mar 12 13:45:19 2012 +0200 @@ -203,6 +203,7 @@ static bool fts_mailbox_build_continue(struct mail_search_context *ctx) { struct fts_search_context *fctx = FTS_CONTEXT(ctx); + enum mail_error error; int ret; if (fctx == NULL) @@ -218,6 +219,17 @@ ret = -1; if (ret > 0) fts_search_lookup(fctx); + if (ret < 0) { + /* if indexing timed out, it probably means that + the mailbox is still being indexed, but it's a large + mailbox and it takes a while. in this situation + we'll simply abort the search. + + if indexing failed for any other reason, just + fallback to searching the slow way. */ + (void)mailbox_get_last_error(fctx->box, &error); + fctx->indexing_timed_out = error == MAIL_ERROR_INUSE; + } } return TRUE; } @@ -227,11 +239,16 @@ struct mail **mail_r, bool *tryagain_r) { struct fts_mailbox *fbox = FTS_CONTEXT(ctx->transaction->box); + struct fts_search_context *fctx = FTS_CONTEXT(ctx); if (!fts_mailbox_build_continue(ctx)) { *tryagain_r = TRUE; return FALSE; } + if (fctx->indexing_timed_out) { + *tryagain_r = FALSE; + return FALSE; + } return fbox->module_ctx.super. search_next_nonblock(ctx, mail_r, tryagain_r); @@ -270,6 +287,8 @@ if (fctx == NULL || !fctx->fts_lookup_success) { /* fts lookup not done for this search */ + if (fctx->indexing_timed_out) + return FALSE; return fbox->module_ctx.super.search_next_update_seq(ctx); } @@ -295,12 +314,15 @@ struct fts_mailbox *fbox = FTS_CONTEXT(ctx->transaction->box); struct fts_transaction_context *ft = FTS_CONTEXT(ctx->transaction); struct fts_search_context *fctx = FTS_CONTEXT(ctx); + int ret = 0; if (fctx != NULL) { if (fctx->indexer_ctx != NULL) { if (fts_indexer_deinit(&fctx->indexer_ctx) < 0) ft->failed = TRUE; } + if (fctx->indexing_timed_out) + ret = -1; buffer_free(&fctx->orig_matches); array_free(&fctx->levels); @@ -308,7 +330,9 @@ fts_scores_unref(&fctx->scores); i_free(fctx); } - return fbox->module_ctx.super.search_deinit(ctx); + if (fbox->module_ctx.super.search_deinit(ctx) < 0) + ret = -1; + return ret; } static int fts_score_cmp(const uint32_t *uid, const struct fts_score_map *score) diff -r d27b6724db32 -r 4dd97a92691a src/plugins/fts/fts-storage.h --- a/src/plugins/fts/fts-storage.h Mon Mar 12 13:17:50 2012 +0200 +++ b/src/plugins/fts/fts-storage.h Mon Mar 12 13:45:19 2012 +0200 @@ -36,6 +36,7 @@ unsigned int virtual_mailbox:1; unsigned int fts_lookup_success:1; + unsigned int indexing_timed_out:1; }; /* Figure out if we want to use full text search indexes and update From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: fts: Crashfix when FTS isn't needed for performing ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/63f3bb8462b4 changeset: 14328:63f3bb8462b4 user: Timo Sirainen date: Mon Mar 12 14:32:23 2012 +0200 description: fts: Crashfix when FTS isn't needed for performing search. diffstat: src/plugins/fts/fts-storage.c | 57 +++++++++++++++++++++--------------------- 1 files changed, 28 insertions(+), 29 deletions(-) diffs (76 lines): diff -r 4dd97a92691a -r 63f3bb8462b4 src/plugins/fts/fts-storage.c --- a/src/plugins/fts/fts-storage.c Mon Mar 12 13:45:19 2012 +0200 +++ b/src/plugins/fts/fts-storage.c Mon Mar 12 14:32:23 2012 +0200 @@ -206,30 +206,26 @@ enum mail_error error; int ret; - if (fctx == NULL) - return TRUE; + ret = fts_indexer_more(fctx->indexer_ctx); + if (ret == 0) + return FALSE; - if (fctx->indexer_ctx != NULL) { - /* this command is still building the indexes */ - ret = fts_indexer_more(fctx->indexer_ctx); - if (ret == 0) - return FALSE; - ctx->progress_hidden = FALSE; - if (fts_indexer_deinit(&fctx->indexer_ctx) < 0) - ret = -1; - if (ret > 0) - fts_search_lookup(fctx); - if (ret < 0) { - /* if indexing timed out, it probably means that - the mailbox is still being indexed, but it's a large - mailbox and it takes a while. in this situation - we'll simply abort the search. + /* indexing finished */ + ctx->progress_hidden = FALSE; + if (fts_indexer_deinit(&fctx->indexer_ctx) < 0) + ret = -1; + if (ret > 0) + fts_search_lookup(fctx); + if (ret < 0) { + /* if indexing timed out, it probably means that + the mailbox is still being indexed, but it's a large + mailbox and it takes a while. in this situation + we'll simply abort the search. - if indexing failed for any other reason, just - fallback to searching the slow way. */ - (void)mailbox_get_last_error(fctx->box, &error); - fctx->indexing_timed_out = error == MAIL_ERROR_INUSE; - } + if indexing failed for any other reason, just + fallback to searching the slow way. */ + (void)mailbox_get_last_error(fctx->box, &error); + fctx->indexing_timed_out = error == MAIL_ERROR_INUSE; } return TRUE; } @@ -241,13 +237,16 @@ struct fts_mailbox *fbox = FTS_CONTEXT(ctx->transaction->box); struct fts_search_context *fctx = FTS_CONTEXT(ctx); - if (!fts_mailbox_build_continue(ctx)) { - *tryagain_r = TRUE; - return FALSE; - } - if (fctx->indexing_timed_out) { - *tryagain_r = FALSE; - return FALSE; + if (fctx != NULL && fctx->indexer_ctx != NULL) { + /* this command is still building the indexes */ + if (!fts_mailbox_build_continue(ctx)) { + *tryagain_r = TRUE; + return FALSE; + } + if (fctx->indexing_timed_out) { + *tryagain_r = FALSE; + return FALSE; + } } return fbox->module_ctx.super. From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: checkpassword: Compiling fix for some systems Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/2c5a7cc1023e changeset: 14322:2c5a7cc1023e user: Timo Sirainen date: Sun Mar 11 09:08:08 2012 +0200 description: checkpassword: Compiling fix for some systems diffstat: src/auth/db-checkpassword.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff -r 1289b79241bb -r 2c5a7cc1023e src/auth/db-checkpassword.c --- a/src/auth/db-checkpassword.c Sat Mar 10 18:32:34 2012 +0200 +++ b/src/auth/db-checkpassword.c Sun Mar 11 09:08:08 2012 +0200 @@ -19,6 +19,7 @@ #include #include +#include #define CHECKPASSWORD_MAX_REQUEST_LEN 512 From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: stats: When logging about a session crash, log also... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/74d3f14dec5a changeset: 14329:74d3f14dec5a user: Timo Sirainen date: Mon Mar 12 17:36:29 2012 +0200 description: stats: When logging about a session crash, log also its service. diffstat: src/stats/mail-session.c | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diffs (16 lines): diff -r 63f3bb8462b4 -r 74d3f14dec5a src/stats/mail-session.c --- a/src/stats/mail-session.c Mon Mar 12 14:32:23 2012 +0200 +++ b/src/stats/mail-session.c Mon Mar 12 17:36:29 2012 +0200 @@ -46,9 +46,10 @@ static void mail_session_idle_timeout(struct mail_session *session) { - i_warning("Session %s (user %s) appears to have crashed, " + i_warning("Session %s (user %s, service %s) appears to have crashed, " "disconnecting it", - guid_128_to_string(session->guid), session->user->name); + guid_128_to_string(session->guid), session->user->name, + session->service); mail_session_disconnect(session); } From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: fts: Another crashfix for recent changes. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/352dd32abb8d changeset: 14331:352dd32abb8d user: Timo Sirainen date: Tue Mar 13 16:14:13 2012 +0200 description: fts: Another crashfix for recent changes. diffstat: src/plugins/fts/fts-storage.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r b1c21492f8c9 -r 352dd32abb8d src/plugins/fts/fts-storage.c --- a/src/plugins/fts/fts-storage.c Tue Mar 13 15:11:13 2012 +0200 +++ b/src/plugins/fts/fts-storage.c Tue Mar 13 16:14:13 2012 +0200 @@ -286,7 +286,7 @@ if (fctx == NULL || !fctx->fts_lookup_success) { /* fts lookup not done for this search */ - if (fctx->indexing_timed_out) + if (fctx != NULL && fctx->indexing_timed_out) return FALSE; return fbox->module_ctx.super.search_next_update_seq(ctx); } From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: lib-storage: Don't log errors if trying to open a m... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/37d6dd0f053b changeset: 14332:37d6dd0f053b user: Timo Sirainen date: Tue Mar 13 16:32:56 2012 +0200 description: lib-storage: Don't log errors if trying to open a mailbox with too long name. Handle ENAMETOOLONG errors the same as ENOENT. diffstat: src/lib-storage/index/dbox-common/dbox-storage.c | 2 +- src/lib-storage/index/maildir/maildir-storage.c | 4 ++-- src/lib/compat.h | 3 ++- 3 files changed, 5 insertions(+), 4 deletions(-) diffs (46 lines): diff -r 352dd32abb8d -r 37d6dd0f053b src/lib-storage/index/dbox-common/dbox-storage.c --- a/src/lib-storage/index/dbox-common/dbox-storage.c Tue Mar 13 16:14:13 2012 +0200 +++ b/src/lib-storage/index/dbox-common/dbox-storage.c Tue Mar 13 16:32:56 2012 +0200 @@ -172,7 +172,7 @@ if (dbox_cleanup_if_exists(box->list, box_path)) ; - else if (errno == ENOENT) { + else if (errno == ENOENT || errno == ENAMETOOLONG) { mail_storage_set_error(box->storage, MAIL_ERROR_NOTFOUND, T_MAIL_ERR_MAILBOX_NOT_FOUND(box->name)); return -1; diff -r 352dd32abb8d -r 37d6dd0f053b src/lib-storage/index/maildir/maildir-storage.c --- a/src/lib-storage/index/maildir/maildir-storage.c Tue Mar 13 16:14:13 2012 +0200 +++ b/src/lib-storage/index/maildir/maildir-storage.c Tue Mar 13 16:32:56 2012 +0200 @@ -207,7 +207,7 @@ /* if tmp/ directory exists, we need to clean it up once in a while */ path = t_strconcat(dir, "/tmp", NULL); if (stat(path, &st) < 0) { - if (errno == ENOENT) + if (errno == ENOENT || errno == ENAMETOOLONG) return 0; if (errno == EACCES) { mail_storage_set_critical(storage, "%s", @@ -361,7 +361,7 @@ return maildir_mailbox_open_existing(box); } - if (errno == ENOENT) { + if (errno == ENOENT || errno == ENAMETOOLONG) { mail_storage_set_error(box->storage, MAIL_ERROR_NOTFOUND, T_MAIL_ERR_MAILBOX_NOT_FOUND(box->name)); return -1; diff -r 352dd32abb8d -r 37d6dd0f053b src/lib/compat.h --- a/src/lib/compat.h Tue Mar 13 16:14:13 2012 +0200 +++ b/src/lib/compat.h Tue Mar 13 16:32:56 2012 +0200 @@ -244,7 +244,8 @@ #endif #define ENOTFOUND(errno) \ - ((errno) == ENOENT || (errno) == ENOTDIR || (errno) == ELOOP) + ((errno) == ENOENT || (errno) == ENOTDIR || \ + (errno) == ELOOP || (errno) == ENAMETOOLONG) #define ECANTLINK(errno) \ ((errno) == EXDEV || (errno) == EMLINK || (errno) == EPERM) From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: director: Compiler warning fix Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/b1c21492f8c9 changeset: 14330:b1c21492f8c9 user: Timo Sirainen date: Tue Mar 13 15:11:13 2012 +0200 description: director: Compiler warning fix diffstat: src/director/director-request.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 74d3f14dec5a -r b1c21492f8c9 src/director/director-request.c --- a/src/director/director-request.c Mon Mar 12 17:36:29 2012 +0200 +++ b/src/director/director-request.c Tue Mar 13 15:11:13 2012 +0200 @@ -48,7 +48,7 @@ if (user->weak) str_append(str, ", weak user"); str_printfa(str, ", user refreshed %u secs ago", - ioloop_time - user->timestamp); + (unsigned int)(ioloop_time - user->timestamp)); } str_append_c(str, ')'); return str_c(str); From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: dbox: Avoid unnecessary fstat() call during file cr... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/a3b162331cd1 changeset: 14333:a3b162331cd1 user: Timo Sirainen date: Tue Mar 13 17:01:56 2012 +0200 description: dbox: Avoid unnecessary fstat() call during file creation. diffstat: src/lib-storage/index/dbox-common/dbox-file.c | 2 +- src/lib-storage/index/dbox-common/dbox-file.h | 1 + src/lib-storage/index/dbox-multi/mdbox-file.c | 1 + src/lib-storage/index/dbox-single/sdbox-file.c | 1 + 4 files changed, 4 insertions(+), 1 deletions(-) diffs (45 lines): diff -r 37d6dd0f053b -r a3b162331cd1 src/lib-storage/index/dbox-common/dbox-file.c --- a/src/lib-storage/index/dbox-common/dbox-file.c Tue Mar 13 16:32:56 2012 +0200 +++ b/src/lib-storage/index/dbox-common/dbox-file.c Tue Mar 13 17:01:56 2012 +0200 @@ -588,7 +588,7 @@ return 0; } - if (ctx->output->offset == 0) { + if (ctx->output->offset == 0 && !file->created) { /* first append to existing file. seek to eof first. */ if (fstat(file->fd, &st) < 0) { dbox_file_set_syscall_error(file, "fstat()"); diff -r 37d6dd0f053b -r a3b162331cd1 src/lib-storage/index/dbox-common/dbox-file.h --- a/src/lib-storage/index/dbox-common/dbox-file.h Tue Mar 13 16:32:56 2012 +0200 +++ b/src/lib-storage/index/dbox-common/dbox-file.h Tue Mar 13 17:01:56 2012 +0200 @@ -116,6 +116,7 @@ ARRAY_DEFINE(metadata, const char *); uoff_t metadata_read_offset; + unsigned int created:1; /* this file is now being created */ unsigned int appending:1; unsigned int deleted:1; unsigned int corrupted:1; diff -r 37d6dd0f053b -r a3b162331cd1 src/lib-storage/index/dbox-multi/mdbox-file.c --- a/src/lib-storage/index/dbox-multi/mdbox-file.c Tue Mar 13 16:32:56 2012 +0200 +++ b/src/lib-storage/index/dbox-multi/mdbox-file.c Tue Mar 13 17:01:56 2012 +0200 @@ -106,6 +106,7 @@ bool create_parents; int ret; + _file->created = TRUE; create_parents = dbox_file_is_in_alt(_file); _file->fd = _file->storage->v. file_create_fd(_file, _file->cur_path, create_parents); diff -r 37d6dd0f053b -r a3b162331cd1 src/lib-storage/index/dbox-single/sdbox-file.c --- a/src/lib-storage/index/dbox-single/sdbox-file.c Tue Mar 13 16:32:56 2012 +0200 +++ b/src/lib-storage/index/dbox-single/sdbox-file.c Tue Mar 13 17:01:56 2012 +0200 @@ -60,6 +60,7 @@ struct dbox_file *file; file = sdbox_file_init(mbox, 0); + file->created = TRUE; file->fd = file->storage->v. file_create_fd(file, file->primary_path, FALSE); return file; From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: mailbox_list_index=yes iteration: Don't allocate re... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/6cb102aa5d63 changeset: 14334:6cb102aa5d63 user: Timo Sirainen date: Tue Mar 13 21:18:32 2012 +0200 description: mailbox_list_index=yes iteration: Don't allocate returned mailbox names from data stack. This breaks dsync. diffstat: src/lib-storage/list/mailbox-list-index-iter.c | 5 +++++ src/lib-storage/list/mailbox-list-index.h | 2 ++ 2 files changed, 7 insertions(+), 0 deletions(-) diffs (47 lines): diff -r a3b162331cd1 -r 6cb102aa5d63 src/lib-storage/list/mailbox-list-index-iter.c --- a/src/lib-storage/list/mailbox-list-index-iter.c Tue Mar 13 17:01:56 2012 +0200 +++ b/src/lib-storage/list/mailbox-list-index-iter.c Tue Mar 13 21:18:32 2012 +0200 @@ -25,6 +25,7 @@ ctx->ctx.glob = imap_match_init_multiple(pool, patterns, TRUE, ns_sep); array_create(&ctx->ctx.module_contexts, pool, sizeof(void *), 5); ctx->sep = ns_sep; + ctx->info_pool = pool_alloconly_create("mailbox list index iter info", 128); if (mailbox_list_index_refresh(ctx->ctx.list) < 0) { /* no indexing */ @@ -46,12 +47,15 @@ struct mailbox_list_index_node *node = ctx->next_node; struct mailbox *box; + p_clear(ctx->info_pool); + str_truncate(ctx->path, ctx->parent_len); if (str_len(ctx->path) > 0) str_append_c(ctx->path, ctx->sep); str_append(ctx->path, node->name); ctx->info.name = mailbox_list_get_vname(ctx->ctx.list, str_c(ctx->path)); + ctx->info.name = p_strdup(ctx->info_pool, ctx->info.name); ctx->info.flags = 0; if ((node->flags & MAILBOX_LIST_INDEX_FLAG_NONEXISTENT) != 0) ctx->info.flags |= MAILBOX_NONEXISTENT; @@ -166,6 +170,7 @@ ilist->iter_refcount--; } + pool_unref(&ctx->info_pool); pool_unref(&_ctx->pool); return ret; } diff -r a3b162331cd1 -r 6cb102aa5d63 src/lib-storage/list/mailbox-list-index.h --- a/src/lib-storage/list/mailbox-list-index.h Tue Mar 13 17:01:56 2012 +0200 +++ b/src/lib-storage/list/mailbox-list-index.h Tue Mar 13 21:18:32 2012 +0200 @@ -109,6 +109,8 @@ struct mailbox_list_iterate_context *backend_ctx; struct mailbox_info info; + pool_t info_pool; + unsigned int parent_len; string_t *path; struct mailbox_list_index_node *next_node; From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: lib-storage: mailbox_list_iter_next() now enforces ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/1883074f9a9d changeset: 14335:1883074f9a9d user: Timo Sirainen date: Tue Mar 13 21:19:53 2012 +0200 description: lib-storage: mailbox_list_iter_next() now enforces backends not to return mailbox name from data stack. diffstat: src/lib-storage/mailbox-list-iter.c | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-) diffs (20 lines): diff -r 6cb102aa5d63 -r 1883074f9a9d src/lib-storage/mailbox-list-iter.c --- a/src/lib-storage/mailbox-list-iter.c Tue Mar 13 21:18:32 2012 +0200 +++ b/src/lib-storage/mailbox-list-iter.c Tue Mar 13 21:19:53 2012 +0200 @@ -703,10 +703,12 @@ const struct mailbox_info *info; do { - if (ctx->autocreate_ctx != NULL) - info = autocreate_iter_next(ctx); - else - info = mailbox_list_iter_next_call(ctx); + T_BEGIN { + if (ctx->autocreate_ctx != NULL) + info = autocreate_iter_next(ctx); + else + info = mailbox_list_iter_next_call(ctx); + } T_END; } while (info != NULL && !special_use_selection(ctx, info)); return info; } From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: doveconf: Dump protocol/local/remote sections even ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/b0ef6e05a15d changeset: 14336:b0ef6e05a15d user: Timo Sirainen date: Wed Mar 14 13:14:28 2012 +0200 description: doveconf: Dump protocol/local/remote sections even if there was an error earlier. This allows getting a mostly-working config file rather than a truncated one. diffstat: src/config/doveconf.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 1883074f9a9d -r b0ef6e05a15d src/config/doveconf.c --- a/src/config/doveconf.c Tue Mar 13 21:19:53 2012 +0200 +++ b/src/config/doveconf.c Wed Mar 14 13:14:28 2012 +0200 @@ -434,7 +434,7 @@ ret = config_dump_human_output(ctx, output, 0, setting_name_filter); config_dump_human_deinit(ctx); - if (ret == 0 && setting_name_filter == NULL) + if (setting_name_filter == NULL) ret = config_dump_human_sections(output, filter, module); o_stream_uncork(output); From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: test-primes: Fixed undefined behavior. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/0647b0a1b4e5 changeset: 14337:0647b0a1b4e5 user: Timo Sirainen date: Wed Mar 14 13:34:43 2012 +0200 description: test-primes: Fixed undefined behavior. diffstat: src/lib/test-primes.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r b0ef6e05a15d -r 0647b0a1b4e5 src/lib/test-primes.c --- a/src/lib/test-primes.c Wed Mar 14 13:14:28 2012 +0200 +++ b/src/lib/test-primes.c Wed Mar 14 13:34:43 2012 +0200 @@ -14,7 +14,7 @@ success = FALSE; } for (i = 10; i < 32; i++) { - num = (1 << i) - 100; + num = (1U << i) - 100; for (j = 0; j < 200; j++, num++) { if (primes_closest(num) < num) success = FALSE; From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: anvil: s/memcpy/memmove/ for updating checksum list. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/f94f289ce009 changeset: 14338:f94f289ce009 user: Timo Sirainen date: Wed Mar 14 13:39:00 2012 +0200 description: anvil: s/memcpy/memmove/ for updating checksum list. This may have caused anvil to update penalties wrong. diffstat: src/anvil/penalty.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diffs (16 lines): diff -r 0647b0a1b4e5 -r f94f289ce009 src/anvil/penalty.c --- a/src/anvil/penalty.c Wed Mar 14 13:34:43 2012 +0200 +++ b/src/anvil/penalty.c Wed Mar 14 13:39:00 2012 +0200 @@ -136,9 +136,9 @@ rec->checksum_is_pointer = TRUE; } - memcpy(rec->checksum.value_ptr + 1, rec->checksum.value_ptr, - sizeof(rec->checksum.value_ptr[0]) * - (CHECKSUM_VALUE_PTR_COUNT-1)); + memmove(rec->checksum.value_ptr + 1, rec->checksum.value_ptr, + sizeof(rec->checksum.value_ptr[0]) * + (CHECKSUM_VALUE_PTR_COUNT-1)); rec->checksum.value_ptr[0] = checksum; } From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: liblib: Added var_get_key_range() Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/13d4c4f91622 changeset: 14339:13d4c4f91622 user: Timo Sirainen date: Wed Mar 14 13:42:08 2012 +0200 description: liblib: Added var_get_key_range() diffstat: src/lib/test-var-expand.c | 31 +++++++++++++++++++++++++++++++ src/lib/var-expand.c | 40 +++++++++++++++++++++++++++++++--------- src/lib/var-expand.h | 4 ++++ 3 files changed, 66 insertions(+), 9 deletions(-) diffs (126 lines): diff -r f94f289ce009 -r 13d4c4f91622 src/lib/test-var-expand.c --- a/src/lib/test-var-expand.c Wed Mar 14 13:39:00 2012 +0200 +++ b/src/lib/test-var-expand.c Wed Mar 14 13:42:08 2012 +0200 @@ -11,6 +11,11 @@ const char *out; }; +struct var_get_key_range_test { + const char *in; + unsigned int idx, size; +}; + static void test_var_expand_builtin(void) { static struct var_expand_test tests[] = { @@ -37,7 +42,33 @@ test_end(); } +static void test_var_get_key_range(void) +{ + static struct var_get_key_range_test tests[] = { + { "", 0, 0 }, + { "{", 1, 0 }, + { "k", 0, 1 }, + { "{key}", 1, 3 }, + { "5.5Rk", 4, 1 }, + { "5.5R{key}", 5, 3 }, + { "{key", 1, 3 } + }; + unsigned int i, idx, size; + + test_begin("var_get_key_range"); + for (i = 0; i < N_ELEMENTS(tests); i++) { + var_get_key_range(tests[i].in, &idx, &size); + test_assert(tests[i].idx == idx); + test_assert(tests[i].size == size); + + if (tests[i].size == 1) + test_assert(tests[i].in[idx] == var_get_key(tests[i].in)); + } + test_end(); +} + void test_var_expand(void) { test_var_expand_builtin(); + test_var_get_key_range(); } diff -r f94f289ce009 -r 13d4c4f91622 src/lib/var-expand.c --- a/src/lib/var-expand.c Wed Mar 14 13:39:00 2012 +0200 +++ b/src/lib/var-expand.c Wed Mar 14 13:42:08 2012 +0200 @@ -348,28 +348,50 @@ char var_get_key(const char *str) { + unsigned int idx, size; + + var_get_key_range(str, &idx, &size); + return str[idx]; +} + +void var_get_key_range(const char *str, unsigned int *idx_r, + unsigned int *size_r) +{ const struct var_expand_modifier *m; + unsigned int i = 0; /* [.][] */ - while ((*str >= '0' && *str <= '9') || *str == '-') - str++; + while ((str[i] >= '0' && str[i] <= '9') || str[i] == '-') + i++; - if (*str == '.') { - str++; - while (*str >= '0' && *str <= '9') - str++; + if (str[i] == '.') { + i++; + while (str[i] >= '0' && str[i] <= '9') + i++; } do { for (m = modifiers; m->key != '\0'; m++) { - if (m->key == *str) { - str++; + if (m->key == str[i]) { + i++; break; } } } while (m->key != '\0'); - return *str; + if (str[i] != '{') { + /* short key */ + *idx_r = i; + *size_r = str[i] == '\0' ? 0 : 1; + } else { + /* long key */ + *idx_r = ++i; + for (; str[i] != '\0'; i++) { + if (str[i] == '}') + break; + } + *size_r = i - *idx_r; + } } static bool var_has_long_key(const char **str, const char *long_key) diff -r f94f289ce009 -r 13d4c4f91622 src/lib/var-expand.h --- a/src/lib/var-expand.h Wed Mar 14 13:39:00 2012 +0200 +++ b/src/lib/var-expand.h Wed Mar 14 13:42:08 2012 +0200 @@ -27,6 +27,10 @@ /* Returns the actual key character for given string, ie. skip any modifiers that are before it. The string should be the data after the '%' character. */ char var_get_key(const char *str) ATTR_PURE; +/* Similar to var_get_key(), but works for long keys as well. For single char + keys size=1, while for e.g. %{key} size=3 and idx points to 'k'. */ +void var_get_key_range(const char *str, unsigned int *idx_r, + unsigned int *size_r); /* Returns TRUE if key variable is used in the string. long_key may be NULL. */ bool var_has_key(const char *str, char key, const char *long_key) ATTR_PURE; From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: auth: Fixed auth cache key generation to support %{... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/a090cbbe3008 changeset: 14340:a090cbbe3008 user: Timo Sirainen date: Wed Mar 14 14:55:25 2012 +0200 description: auth: Fixed auth cache key generation to support %{long} variables diffstat: src/auth/Makefile.am | 20 +++++++++++++ src/auth/auth-cache.c | 70 ++++++++++++++++++++++++++++++++++++++------- src/auth/auth-request.c | 48 ++++++++++++++++--------------- src/auth/auth-request.h | 2 + src/auth/test-auth-cache.c | 57 +++++++++++++++++++++++++++++++++++++ 5 files changed, 162 insertions(+), 35 deletions(-) diffs (272 lines): diff -r 13d4c4f91622 -r a090cbbe3008 src/auth/Makefile.am --- a/src/auth/Makefile.am Wed Mar 14 13:42:08 2012 +0200 +++ b/src/auth/Makefile.am Wed Mar 14 14:55:25 2012 +0200 @@ -24,6 +24,7 @@ AM_CPPFLAGS = \ -I$(top_srcdir)/src/lib \ -I$(top_srcdir)/src/lib-auth \ + -I$(top_srcdir)/src/lib-test \ -I$(top_srcdir)/src/lib-dns \ -I$(top_srcdir)/src/lib-sql \ -I$(top_srcdir)/src/lib-settings \ @@ -182,3 +183,22 @@ checkpassword_reply_sources = \ checkpassword-reply.c + +test_programs = \ + test-auth-cache + +noinst_PROGRAMS = $(test_programs) + +test_libs = \ + ../lib-test/libtest.la \ + ../lib/liblib.la + +test_auth_cache_SOURCES = test-auth-cache.c +test_auth_cache_LDADD = auth-cache.o $(test_libs) +test_auth_cache_DEPENDENCIES = auth-cache.o $(test_libs) + +check: check-am check-test +check-test: all-am + for bin in $(test_programs); do \ + if ! $(RUN_TEST) ./$$bin; then exit 1; fi; \ + done diff -r 13d4c4f91622 -r a090cbbe3008 src/auth/auth-cache.c --- a/src/auth/auth-cache.c Wed Mar 14 13:42:08 2012 +0200 +++ b/src/auth/auth-cache.c Wed Mar 14 14:55:25 2012 +0200 @@ -23,29 +23,75 @@ unsigned long long pos_size, neg_size; }; +static const struct var_expand_table * +auth_request_var_expand_tab_find(const char *key, unsigned int size) +{ + const struct var_expand_table *tab = auth_request_var_expand_static_tab; + unsigned int i; + + for (i = 0; tab[i].key != '\0' || tab[i].long_key != NULL; i++) { + if (size == 1) { + if (key[0] == tab[i].key) + return &tab[i]; + } else if (tab[i].long_key != NULL) { + if (strncmp(key, tab[i].long_key, size) == 0 && + tab[i].long_key[size] == '\0') + return &tab[i]; + } + } + return NULL; +} + char *auth_cache_parse_key(pool_t pool, const char *query) { + const struct var_expand_table *tab; string_t *str; - char key_seen[256]; - uint8_t key; + bool key_seen[100]; + unsigned int idx, size, tab_idx; + bool add_key; memset(key_seen, 0, sizeof(key_seen)); str = str_new(pool, 32); - for (; *query != '\0'; query++) { - if (*query == '%' && query[1] != '\0') { + for (; *query != '\0'; ) { + if (*query != '%') { query++; - key = var_get_key(query); - if (key != '\0' && key != '%' && !key_seen[key]) { - if (str_len(str) != 0) - str_append_c(str, '\t'); - str_append_c(str, '%'); - str_append_c(str, key); + continue; + } - /* @UNSAFE */ - key_seen[key] = 1; + var_get_key_range(++query, &idx, &size); + if (size == 0) { + /* broken %variable ending too early */ + break; + } + query += idx; + + tab = auth_request_var_expand_tab_find(query, size); + if (tab == NULL) { + /* just add the key. it would be nice to prevent + duplicates here as well, but that's just too + much trouble and probably very rare. */ + add_key = TRUE; + } else { + tab_idx = tab - auth_request_var_expand_static_tab; + i_assert(tab_idx < N_ELEMENTS(key_seen)); + /* @UNSAFE */ + add_key = !key_seen[tab_idx]; + key_seen[tab_idx] = TRUE; + } + if (add_key) { + if (str_len(str) != 0) + str_append_c(str, '\t'); + str_append_c(str, '%'); + if (size == 1) + str_append_c(str, query[0]); + else { + str_append_c(str, '{'); + str_append_n(str, query, size); + str_append_c(str, '}'); } } + query += size; } return str_free_without_data(&str); } diff -r 13d4c4f91622 -r a090cbbe3008 src/auth/auth-request.c --- a/src/auth/auth-request.c Wed Mar 14 13:42:08 2012 +0200 +++ b/src/auth/auth-request.c Wed Mar 14 14:55:25 2012 +0200 @@ -1741,38 +1741,40 @@ return str_escape(string); } +const struct var_expand_table auth_request_var_expand_static_tab[] = { + { 'u', NULL, "user" }, + { 'n', NULL, "username" }, + { 'd', NULL, "domain" }, + { 's', NULL, "service" }, + { 'h', NULL, "home" }, + { 'l', NULL, "lip" }, + { 'r', NULL, "rip" }, + { 'p', NULL, "pid" }, + { 'w', NULL, "password" }, + { '!', NULL, NULL }, + { 'm', NULL, "mech" }, + { 'c', NULL, "secured" }, + { 'a', NULL, "lport" }, + { 'b', NULL, "rport" }, + { 'k', NULL, "cert" }, + { '\0', NULL, "login_user" }, + { '\0', NULL, "login_username" }, + { '\0', NULL, "login_domain" }, + { '\0', NULL, NULL } +}; + const struct var_expand_table * auth_request_get_var_expand_table(const struct auth_request *auth_request, auth_request_escape_func_t *escape_func) { - static struct var_expand_table static_tab[] = { - { 'u', NULL, "user" }, - { 'n', NULL, "username" }, - { 'd', NULL, "domain" }, - { 's', NULL, "service" }, - { 'h', NULL, "home" }, - { 'l', NULL, "lip" }, - { 'r', NULL, "rip" }, - { 'p', NULL, "pid" }, - { 'w', NULL, "password" }, - { '!', NULL, NULL }, - { 'm', NULL, "mech" }, - { 'c', NULL, "secured" }, - { 'a', NULL, "lport" }, - { 'b', NULL, "rport" }, - { 'k', NULL, "cert" }, - { '\0', NULL, "login_user" }, - { '\0', NULL, "login_username" }, - { '\0', NULL, "login_domain" }, - { '\0', NULL, NULL } - }; struct var_expand_table *tab; if (escape_func == NULL) escape_func = escape_none; - tab = t_malloc(sizeof(static_tab)); - memcpy(tab, static_tab, sizeof(static_tab)); + tab = t_malloc(sizeof(auth_request_var_expand_static_tab)); + memcpy(tab, auth_request_var_expand_static_tab, + sizeof(auth_request_var_expand_static_tab)); tab[0].value = escape_func(auth_request->user, auth_request); tab[1].value = escape_func(t_strcut(auth_request->user, '@'), diff -r 13d4c4f91622 -r a090cbbe3008 src/auth/auth-request.h --- a/src/auth/auth-request.h Wed Mar 14 13:42:08 2012 +0200 +++ b/src/auth/auth-request.h Wed Mar 14 14:55:25 2012 +0200 @@ -2,6 +2,7 @@ #define AUTH_REQUEST_H #include "network.h" +#include "var-expand.h" #include "mech.h" #include "userdb.h" #include "passdb.h" @@ -122,6 +123,7 @@ typedef void auth_request_proxy_cb_t(bool success, struct auth_request *); extern unsigned int auth_request_state_count[AUTH_REQUEST_STATE_MAX]; +extern const struct var_expand_table auth_request_var_expand_static_tab[]; struct auth_request * auth_request_new(const struct mech_module *mech); diff -r 13d4c4f91622 -r a090cbbe3008 src/auth/test-auth-cache.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/auth/test-auth-cache.c Wed Mar 14 14:55:25 2012 +0200 @@ -0,0 +1,57 @@ +/* Copyright (c) 2012 Dovecot authors, see the included COPYING file */ + +#include "lib.h" +#include "auth-request.h" +#include "auth-cache.h" +#include "test-common.h" + +const struct var_expand_table auth_request_var_expand_static_tab[] = { + { 'a', NULL, NULL }, + { '\0', NULL, "longb" }, + { 'c', NULL, "longc" }, + { '\0', NULL, NULL } +}; + +const struct var_expand_table * +auth_request_get_var_expand_table(const struct auth_request *auth_request ATTR_UNUSED, + auth_request_escape_func_t *escape_func ATTR_UNUSED) +{ + return auth_request_var_expand_static_tab; +} + +static void test_auth_cache_parse_key(void) +{ + struct { + const char *in, *out; + } tests[] = { + { "foo%5.5Mabar", "%a" }, + { "foo%5.5M{longb}bar", "%{longb}" }, + { "foo%5.5Mcbar", "%c" }, + { "foo%5.5M{longc}bar", "%{longc}" }, + { "%a%b", "%a\t%b" }, + { "%a%{longb}%a", "%a\t%{longb}" }, + { "%{longc}%c", "%{longc}" }, + { "%c%a%{longc}%c", "%c\t%a" }, + { "%a%{env:foo}%{env:foo}%a", "%a\t%{env:foo}\t%{env:foo}" } + }; + const char *cache_key; + unsigned int i; + + test_begin("auth cache parse key"); + + for (i = 0; i < N_ELEMENTS(tests); i++) { + cache_key = auth_cache_parse_key(pool_datastack_create(), + tests[i].in); + test_assert(strcmp(cache_key, tests[i].out) == 0); + } + test_end(); +} + +int main(void) +{ + static void (*test_functions[])(void) = { + test_auth_cache_parse_key, + NULL + }; + return test_run(test_functions); +} From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: auth: Make sure auth cache doesn't break if any cac... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/90738a7c7535 changeset: 14341:90738a7c7535 user: Timo Sirainen date: Wed Mar 14 14:59:24 2012 +0200 description: auth: Make sure auth cache doesn't break if any cache keys have TABs. diffstat: src/auth/auth-cache.c | 15 ++++++++++++--- 1 files changed, 12 insertions(+), 3 deletions(-) diffs (46 lines): diff -r a090cbbe3008 -r 90738a7c7535 src/auth/auth-cache.c --- a/src/auth/auth-cache.c Wed Mar 14 14:55:25 2012 +0200 +++ b/src/auth/auth-cache.c Wed Mar 14 14:59:24 2012 +0200 @@ -207,6 +207,15 @@ hash_table_clear(cache->hash, FALSE); } +static const char * +auth_cache_escape(const char *string, + const struct auth_request *auth_request ATTR_UNUSED) +{ + /* cache key %variables are separated by tabs, make sure that there + are no tabs in the string */ + return str_tabescape(string); +} + const char * auth_cache_lookup(struct auth_cache *cache, const struct auth_request *request, const char *key, struct auth_cache_node **node_r, @@ -225,7 +234,7 @@ str = t_str_new(256); var_expand(str, t_strconcat(request->userdb_lookup ? "U" : "P", "%!/", key, NULL), - auth_request_get_var_expand_table(request, NULL)); + auth_request_get_var_expand_table(request, auth_cache_escape)); node = hash_table_lookup(cache->hash, str_c(str)); if (node == NULL) { @@ -281,7 +290,7 @@ str = t_str_new(256); var_expand(str, t_strconcat(request->userdb_lookup ? "U" : "P", "%!/", key, NULL), - auth_request_get_var_expand_table(request, NULL)); + auth_request_get_var_expand_table(request, auth_cache_escape)); request->user = current_username; @@ -330,7 +339,7 @@ str = t_str_new(256); var_expand(str, key, - auth_request_get_var_expand_table(request, NULL)); + auth_request_get_var_expand_table(request, auth_cache_escape)); node = hash_table_lookup(cache->hash, str_c(str)); if (node == NULL) From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: mailbox_list_index=yes: Delay opening/creating the ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/0274c4132052 changeset: 14342:0274c4132052 user: Timo Sirainen date: Wed Mar 14 15:34:14 2012 +0200 description: mailbox_list_index=yes: Delay opening/creating the index until it's needed. This also fixes an error when trying to create the index before the parent director is created. diffstat: src/lib-storage/list/mailbox-list-index.c | 61 +++++++++++++++--------------- src/lib-storage/list/mailbox-list-index.h | 2 + 2 files changed, 33 insertions(+), 30 deletions(-) diffs (111 lines): diff -r 90738a7c7535 -r 0274c4132052 src/lib-storage/list/mailbox-list-index.c --- a/src/lib-storage/list/mailbox-list-index.c Wed Mar 14 14:59:24 2012 +0200 +++ b/src/lib-storage/list/mailbox-list-index.c Wed Mar 14 15:34:14 2012 +0200 @@ -30,6 +30,34 @@ ilist->sync_log_file_offset = 0; } +static void mailbox_list_index_index_open(struct mailbox_list *list) +{ + struct mailbox_list_index *ilist = INDEX_LIST_CONTEXT(list); + const struct mail_storage_settings *set = list->mail_set; + enum mail_index_open_flags index_flags; + unsigned int lock_timeout; + + if (ilist->opened) + return; + ilist->opened = TRUE; + + index_flags = mail_storage_settings_to_index_flags(set); + lock_timeout = set->mail_max_lock_timeout == 0 ? -1U : + set->mail_max_lock_timeout; + + mail_index_set_lock_method(ilist->index, set->parsed_lock_method, + lock_timeout); + if (mail_index_open_or_create(ilist->index, index_flags) < 0) { + if (mail_index_move_to_memory(ilist->index) < 0) { + /* try opening once more. it should be created + directly into memory now. */ + if (mail_index_open_or_create(ilist->index, + index_flags) < 0) + i_panic("in-memory index creation failed"); + } + } +} + struct mailbox_list_index_node * mailbox_list_index_node_find_sibling(struct mailbox_list_index_node *node, const char *name) @@ -220,6 +248,7 @@ return 0; } + mailbox_list_index_index_open(list); if (mail_index_refresh(ilist->index) < 0) { mailbox_list_index_set_index_error(list); return -1; @@ -244,6 +273,8 @@ struct mail_index_view *view; struct mail_index_transaction *trans; + mailbox_list_index_index_open(list); + view = mail_index_view_open(ilist->index); if (!mailbox_list_index_need_refresh(ilist, view)) { new_hdr.refresh_flag = 1; @@ -272,31 +303,6 @@ ilist->module_ctx.super.deinit(list); } -static int mailbox_list_index_index_open(struct mailbox_list *list) -{ - struct mailbox_list_index *ilist = INDEX_LIST_CONTEXT(list); - const struct mail_storage_settings *set = list->mail_set; - enum mail_index_open_flags index_flags; - unsigned int lock_timeout; - - index_flags = mail_storage_settings_to_index_flags(set); - lock_timeout = set->mail_max_lock_timeout == 0 ? -1U : - set->mail_max_lock_timeout; - - mail_index_set_lock_method(ilist->index, set->parsed_lock_method, - lock_timeout); - if (mail_index_open_or_create(ilist->index, index_flags) < 0) { - if (mail_index_move_to_memory(ilist->index) < 0) { - /* try opening once more. it should be created - directly into memory now. */ - if (mail_index_open_or_create(ilist->index, - index_flags) < 0) - i_panic("in-memory index creation failed"); - } - } - return 0; -} - static int mailbox_list_index_create_mailbox_dir(struct mailbox_list *list, const char *name, @@ -398,11 +404,6 @@ hash_table_create(default_pool, ilist->mailbox_pool, 0, NULL, NULL); - if (mailbox_list_index_index_open(list) < 0) { - list->v = ilist->module_ctx.super; - mail_index_free(&ilist->index); - MODULE_CONTEXT_UNSET(list, mailbox_list_index_module); - } mailbox_list_index_status_init_list(list); } diff -r 90738a7c7535 -r 0274c4132052 src/lib-storage/list/mailbox-list-index.h --- a/src/lib-storage/list/mailbox-list-index.h Wed Mar 14 14:59:24 2012 +0200 +++ b/src/lib-storage/list/mailbox-list-index.h Wed Mar 14 15:34:14 2012 +0200 @@ -102,6 +102,8 @@ /* uint32_t uid => struct mailbox_list_index_node* */ struct hash_table *mailbox_hash; struct mailbox_list_index_node *mailbox_tree; + + unsigned int opened:1; }; struct mailbox_list_index_iterate_context { From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: mailbox_list_index=yes: Don't crash at deinit (fix ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/316110bd18e0 changeset: 14343:316110bd18e0 user: Timo Sirainen date: Wed Mar 14 15:40:35 2012 +0200 description: mailbox_list_index=yes: Don't crash at deinit (fix to previous change) diffstat: src/lib-storage/list/mailbox-list-index.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (13 lines): diff -r 0274c4132052 -r 316110bd18e0 src/lib-storage/list/mailbox-list-index.c --- a/src/lib-storage/list/mailbox-list-index.c Wed Mar 14 15:34:14 2012 +0200 +++ b/src/lib-storage/list/mailbox-list-index.c Wed Mar 14 15:40:35 2012 +0200 @@ -298,7 +298,8 @@ hash_table_destroy(&ilist->mailbox_hash); hash_table_destroy(&ilist->mailbox_names); pool_unref(&ilist->mailbox_pool); - mail_index_close(ilist->index); + if (ilist->opened) + mail_index_close(ilist->index); mail_index_free(&ilist->index); ilist->module_ctx.super.deinit(list); } From dovecot at dovecot.org Sun May 20 03:26:28 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:28 +0300 Subject: dovecot-2.2: layout=fs: Rename mailbox names that aren't valid U... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/c077ca9bc306 changeset: 14345:c077ca9bc306 user: Timo Sirainen date: Wed Mar 14 16:24:05 2012 +0200 description: layout=fs: Rename mailbox names that aren't valid UTF-8 to avoid crashes later. diffstat: src/lib-storage/list/mailbox-list-fs-iter.c | 43 ++++++++++++++++++++++++++-- 1 files changed, 40 insertions(+), 3 deletions(-) diffs (78 lines): diff -r 3550dfc4af29 -r c077ca9bc306 src/lib-storage/list/mailbox-list-fs-iter.c --- a/src/lib-storage/list/mailbox-list-fs-iter.c Wed Mar 14 15:40:58 2012 +0200 +++ b/src/lib-storage/list/mailbox-list-fs-iter.c Wed Mar 14 16:24:05 2012 +0200 @@ -2,12 +2,16 @@ #include "lib.h" #include "array.h" +#include "str.h" +#include "unichar.h" #include "imap-match.h" +#include "imap-utf7.h" #include "mail-storage.h" #include "mailbox-tree.h" #include "mailbox-list-subscriptions.h" #include "mailbox-list-fs.h" +#include #include #include #include @@ -508,6 +512,33 @@ return strcmp(path, inbox_path) == 0; } +static void +fs_list_rename_invalid(struct fs_list_iterate_context *ctx, + const char *storage_name) +{ + /* the storage_name is completely invalid, rename it to + something more sensible. we could do this for all names that + aren't valid mUTF-7, but that might lead to accidents in + future when UTF-8 storage names are used */ + string_t *destname = t_str_new(128); + string_t *dest = t_str_new(128); + const char *root, *src; + + root = mailbox_list_get_path(ctx->ctx.list, NULL, + MAILBOX_LIST_PATH_TYPE_MAILBOX); + src = t_strconcat(root, "/", storage_name, NULL); + + (void)uni_utf8_get_valid_data((const void *)storage_name, + strlen(storage_name), destname); + + str_append(dest, root); + str_append_c(dest, '/'); + (void)imap_utf8_to_utf7(str_c(destname), dest); + + if (rename(src, str_c(dest)) < 0 && errno != ENOENT) + i_error("rename(%s, %s) failed: %m", src, str_c(dest)); +} + static int fs_list_entry(struct fs_list_iterate_context *ctx, const struct list_dir_entry *entry) @@ -515,14 +546,20 @@ struct mail_namespace *ns = ctx->ctx.list->ns; struct list_dir_context *dir, *subdir = NULL; enum imap_match_result match, child_dir_match; - const char *storage_name, *child_dir_name; + const char *storage_name, *vname, *child_dir_name; dir = ctx->dir; storage_name = *dir->storage_name == '\0' ? entry->fname : t_strconcat(dir->storage_name, "/", entry->fname, NULL); - ctx->info.name = mailbox_list_get_vname(ctx->ctx.list, storage_name); - ctx->info.name = p_strdup(ctx->info_pool, ctx->info.name); + vname = mailbox_list_get_vname(ctx->ctx.list, storage_name); + if (!uni_utf8_str_is_valid(vname)) { + fs_list_rename_invalid(ctx, storage_name); + /* just skip this in this iteration, we'll see it on the + next list */ + return 0; + } + ctx->info.name = p_strdup(ctx->info_pool, vname); ctx->info.flags = entry->info_flags; match = imap_match(ctx->ctx.glob, ctx->info.name); From dovecot at dovecot.org Sun May 20 03:26:28 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:28 +0300 Subject: dovecot-2.2: auth: If global passdb is missing, don't complain i... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/8fb3fd391d18 changeset: 14346:8fb3fd391d18 user: Timo Sirainen date: Thu Mar 15 12:59:41 2012 +0200 description: auth: If global passdb is missing, don't complain if protocol x and !x { passdb } exists. diffstat: src/auth/auth.c | 12 ++++++++++-- 1 files changed, 10 insertions(+), 2 deletions(-) diffs (38 lines): diff -r c077ca9bc306 -r 8fb3fd391d18 src/auth/auth.c --- a/src/auth/auth.c Wed Mar 14 16:24:05 2012 +0200 +++ b/src/auth/auth.c Thu Mar 15 12:59:41 2012 +0200 @@ -194,7 +194,6 @@ /* use a dummy userdb static. */ auth_userdb_preinit(auth, &userdb_dummy_set); } - auth_mech_list_verify_passdb(auth); return auth; } @@ -251,9 +250,10 @@ { struct master_service_settings_output set_output; const struct auth_settings *service_set; - struct auth *auth; + struct auth *auth, *const *authp; unsigned int i; const char *not_service = NULL; + bool check_default = TRUE; i_array_init(&auths, 8); @@ -274,6 +274,14 @@ auth = auth_preinit(service_set, services[i], pool, reg); array_append(&auths, &auth, 1); } + + if (not_service != NULL && str_array_find(services, not_service+1)) + check_default = FALSE; + + array_foreach(&auths, authp) { + if ((*authp)->service != NULL || check_default) + auth_mech_list_verify_passdb(*authp); + } } void auths_init(void) From dovecot at dovecot.org Sun May 20 03:26:27 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:27 +0300 Subject: dovecot-2.2: mailbox_list_index=yes: Don't add autocreated mailb... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/3550dfc4af29 changeset: 14344:3550dfc4af29 user: Timo Sirainen date: Wed Mar 14 15:40:58 2012 +0200 description: mailbox_list_index=yes: Don't add autocreated mailboxes to index before they are created. diffstat: src/lib-storage/list/mailbox-list-index-sync.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diffs (16 lines): diff -r 316110bd18e0 -r 3550dfc4af29 src/lib-storage/list/mailbox-list-index-sync.c --- a/src/lib-storage/list/mailbox-list-index-sync.c Wed Mar 14 15:40:35 2012 +0200 +++ b/src/lib-storage/list/mailbox-list-index-sync.c Wed Mar 14 15:40:58 2012 +0200 @@ -267,8 +267,11 @@ /* clear EXISTS-flags, so after sync we know what can be expunged */ mailbox_list_index_node_clear_exists(ilist->mailbox_tree); + /* don't include autocreated mailboxes in index until they're + actually created. */ patterns[0] = "*"; patterns[1] = NULL; - iter = ilist->module_ctx.super.iter_init(list, patterns, 0); + iter = ilist->module_ctx.super. + iter_init(list, patterns, MAILBOX_LIST_ITER_NO_AUTO_BOXES); while ((info = ilist->module_ctx.super.iter_next(iter)) != NULL) { flags = 0; if ((info->flags & MAILBOX_NONEXISTENT) != 0) From dovecot at dovecot.org Sun May 20 03:26:28 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:28 +0300 Subject: dovecot-2.2: layout=maildir++: Don't list INBOX if it doesn't ma... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/0b01fff98a04 changeset: 14347:0b01fff98a04 user: Timo Sirainen date: Thu Mar 15 13:25:36 2012 +0200 description: layout=maildir++: Don't list INBOX if it doesn't match the list patterns. diffstat: src/lib-storage/list/mailbox-list-maildir-iter.c | 14 +++++++++----- 1 files changed, 9 insertions(+), 5 deletions(-) diffs (29 lines): diff -r 8fb3fd391d18 -r 0b01fff98a04 src/lib-storage/list/mailbox-list-maildir-iter.c --- a/src/lib-storage/list/mailbox-list-maildir-iter.c Thu Mar 15 12:59:41 2012 +0200 +++ b/src/lib-storage/list/mailbox-list-maildir-iter.c Thu Mar 15 13:25:36 2012 +0200 @@ -135,16 +135,20 @@ node = mailbox_tree_lookup(ctx->tree_ctx, inbox_name); if (node != NULL) node->flags &= ~MAILBOX_NONEXISTENT; - } else { + return 0; + } + + /* add the INBOX only if it matches the patterns */ + match = imap_match(glob, inbox_name); + if (match == IMAP_MATCH_PARENT) + maildir_fill_parents(ctx, glob, FALSE, inbox_name); + else if (match == IMAP_MATCH_YES) { node = mailbox_tree_get(ctx->tree_ctx, inbox_name, &created); if (created) node->flags = MAILBOX_NOCHILDREN; else node->flags &= ~MAILBOX_NONEXISTENT; - - match = imap_match(glob, inbox_name); - if ((match & (IMAP_MATCH_YES | IMAP_MATCH_PARENT)) != 0) - node->flags |= MAILBOX_MATCHED; + node->flags |= MAILBOX_MATCHED; } return 0; } From dovecot at dovecot.org Sun May 20 03:26:28 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:28 +0300 Subject: dovecot-2.2: imap: Use mailbox_exists() to find out if namespace... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/65a75939ac2c changeset: 14349:65a75939ac2c user: Timo Sirainen date: Thu Mar 15 14:18:55 2012 +0200 description: imap: Use mailbox_exists() to find out if namespace prefix is selectable or not. mailbox_list_mailbox() should be treated as an internal function, since ACLs don't apply to it. diffstat: src/imap/cmd-list.c | 17 +++++++++++++---- 1 files changed, 13 insertions(+), 4 deletions(-) diffs (35 lines): diff -r 55586f4a86f1 -r 65a75939ac2c src/imap/cmd-list.c --- a/src/imap/cmd-list.c Thu Mar 15 14:18:13 2012 +0200 +++ b/src/imap/cmd-list.c Thu Mar 15 14:18:55 2012 +0200 @@ -273,6 +273,8 @@ { struct mail_namespace *const *listed; const struct mailbox_settings *mailbox_set; + struct mailbox *box; + enum mailbox_existence existence; unsigned int len; enum mailbox_info_flags flags; const char *name; @@ -307,11 +309,18 @@ ctx->inbox_found = TRUE; flags = list_get_inbox_flags(ctx); - } else if (same_ns && - mailbox_list_mailbox(ctx->ns->list, "", &flags) > 0) { - /* mailbox with namespace prefix exists */ + } else if (!same_ns) { + /* parent */ + flags = MAILBOX_NONEXISTENT; } else { - flags = MAILBOX_NONEXISTENT; + /* see if namespace prefix is selectable */ + box = mailbox_alloc(ctx->ns->list, name, 0); + if (mailbox_exists(box, TRUE, &existence) == 0 && + existence == MAILBOX_EXISTENCE_SELECT) + flags = MAILBOX_SELECT; + else + flags = MAILBOX_NONEXISTENT; + mailbox_free(&box); } if ((flags & MAILBOX_CHILDREN) == 0) { From dovecot at dovecot.org Sun May 20 03:26:28 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:28 +0300 Subject: dovecot-2.2: example-config: Added auth_proxy_self setting. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/b2643c587f5b changeset: 14350:b2643c587f5b user: Timo Sirainen date: Thu Mar 15 16:39:38 2012 +0200 description: example-config: Added auth_proxy_self setting. diffstat: doc/example-config/dovecot.conf | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diffs (15 lines): diff -r 65a75939ac2c -r b2643c587f5b doc/example-config/dovecot.conf --- a/doc/example-config/dovecot.conf Thu Mar 15 14:18:55 2012 +0200 +++ b/doc/example-config/dovecot.conf Thu Mar 15 16:39:38 2012 +0200 @@ -46,6 +46,11 @@ # Sepace separated list of login access check sockets (e.g. tcpwrap) #login_access_sockets = +# With proxy_maybe=yes if proxy destination matches any of these IPs, don't do +# proxying. This isn't necessary normally, but may be useful if the destination +# IP is e.g. a load balancer's IP. +#auth_proxy_self = + # Show more verbose process titles (in ps). Currently shows user name and # IP address. Useful for seeing who are actually using the IMAP processes # (eg. shared mailboxes or if same uid is used for multiple accounts). From dovecot at dovecot.org Sun May 20 03:26:28 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:28 +0300 Subject: dovecot-2.2: lib-storage: mailbox_exists() now returns namespace... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/55586f4a86f1 changeset: 14348:55586f4a86f1 user: Timo Sirainen date: Thu Mar 15 14:18:13 2012 +0200 description: lib-storage: mailbox_exists() now returns namespace prefix as selectable if it is. diffstat: src/lib-storage/mail-storage.c | 21 ++++++++++++--------- 1 files changed, 12 insertions(+), 9 deletions(-) diffs (36 lines): diff -r 0b01fff98a04 -r 55586f4a86f1 src/lib-storage/mail-storage.c --- a/src/lib-storage/mail-storage.c Thu Mar 15 13:25:36 2012 +0200 +++ b/src/lib-storage/mail-storage.c Thu Mar 15 14:18:13 2012 +0200 @@ -749,20 +749,23 @@ return 0; } - if (!box->inbox_user && - have_listable_namespace_prefix(box->storage->user->namespaces, - box->vname)) { - /* listable namespace prefix always exists */ - *existence_r = MAILBOX_EXISTENCE_NOSELECT; - return 0; - } - if (auto_boxes && box->set != NULL && mailbox_is_autocreated(box)) { *existence_r = MAILBOX_EXISTENCE_SELECT; return 0; } - return box->v.exists(box, auto_boxes, existence_r); + if (box->v.exists(box, auto_boxes, existence_r) < 0) + return -1; + + if (!box->inbox_user && *existence_r == MAILBOX_EXISTENCE_NOSELECT && + have_listable_namespace_prefix(box->storage->user->namespaces, + box->vname)) { + /* listable namespace prefix always exists. */ + *existence_r = MAILBOX_EXISTENCE_NOSELECT; + return 0; + } + + return 0; } static int mailbox_check_mismatching_separators(struct mailbox *box) From dovecot at dovecot.org Sun May 20 03:26:28 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:28 +0300 Subject: dovecot-2.2: Released v2.1.2. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/744e0d7f1b25 changeset: 14351:744e0d7f1b25 user: Timo Sirainen date: Thu Mar 15 16:42:18 2012 +0200 description: Released v2.1.2. diffstat: NEWS | 32 ++++++++++++++++++++++++++++++++ configure.in | 2 +- 2 files changed, 33 insertions(+), 1 deletions(-) diffs (49 lines): diff -r b2643c587f5b -r 744e0d7f1b25 NEWS --- a/NEWS Thu Mar 15 16:39:38 2012 +0200 +++ b/NEWS Thu Mar 15 16:42:18 2012 +0200 @@ -1,3 +1,35 @@ +v2.1.2 2012-03-15 Timo Sirainen + + + Initial implementation of dsync-based replication. For now this + should be used only on non-critical systems. + + Proxying: POP3 now supports sending remote IP+port from proxy to + backend server via Dovecot-specific XCLIENT extension. + + Proxying: proxy_maybe=yes with host= (instead of IP) + works now properly. + + Proxying: Added auth_proxy_self setting + + Proxying: Added proxy_always extra field (see wiki docs) + + Added director_username_hash setting to specify what part of the + username is hashed. This can be used to implement per-domain + backends (which allows safely accessing shared mailboxes within + domain). + + Added a "session ID" string for imap/pop3 connections, available + in %{session} variable. The session ID passes through Dovecot + IMAP/POP3 proxying to backend server. The same session ID is can be + reused after a long time (currently a bit under 9 years). + + passdb checkpassword: Support "credentials lookups" (for + non-plaintext auth and for lmtp_proxy lookups) + + fts: Added fts_index_timeout setting to abort search if indexing + hasn't finished by then (default is to wait forever). + - doveadm sync: If mailbox was expunged empty, messages may have + become back instead of also being expunged in the other side. + - director: If user logged into two directors while near user + expiration, the directors might have redirected the user to two + different backends. + - imap_id_* settings were ignored before login. + - Several fixes to mailbox_list_index=yes + - Previous v2.1.x didn't log all messages at shutdown. + - mbox: Fixed accessing Dovecot v1.x mbox index files without errors. + v2.1.1 2012-02-23 Timo Sirainen + dsync: If message with same GUID is saved multiple times in session, diff -r b2643c587f5b -r 744e0d7f1b25 configure.in --- a/configure.in Thu Mar 15 16:39:38 2012 +0200 +++ b/configure.in Thu Mar 15 16:42:18 2012 +0200 @@ -1,5 +1,5 @@ AC_PREREQ([2.59]) -AC_INIT([Dovecot],[2.1.1],[dovecot at dovecot.org]) +AC_INIT([Dovecot],[2.1.2],[dovecot at dovecot.org]) AC_CONFIG_SRCDIR([src]) AM_INIT_AUTOMAKE([foreign]) From dovecot at dovecot.org Sun May 20 03:26:28 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:28 +0300 Subject: dovecot-2.2: Added tag 2.1.2 for changeset 744e0d7f1b25 Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/67ef6f6d45bf changeset: 14352:67ef6f6d45bf user: Timo Sirainen date: Thu Mar 15 16:42:18 2012 +0200 description: Added tag 2.1.2 for changeset 744e0d7f1b25 diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r 744e0d7f1b25 -r 67ef6f6d45bf .hgtags --- a/.hgtags Thu Mar 15 16:42:18 2012 +0200 +++ b/.hgtags Thu Mar 15 16:42:18 2012 +0200 @@ -78,3 +78,4 @@ 736f1b7af190ea68e65719277bd8d3d4682e0844 2.1.rc7 e2cd03cc9c690c4989fb53a1871b7109c547388a 2.1.0 04b0acc03f1eaa0353888a75a452e5c8e61e4c94 2.1.1 +744e0d7f1b255a9339060f761b850303121f14df 2.1.2 From dovecot at dovecot.org Sun May 20 03:26:28 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:28 +0300 Subject: dovecot-2.2: Added signature for changeset 744e0d7f1b25 Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/914076de4c48 changeset: 14353:914076de4c48 user: Timo Sirainen date: Thu Mar 15 16:42:22 2012 +0200 description: Added signature for changeset 744e0d7f1b25 diffstat: .hgsigs | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r 67ef6f6d45bf -r 914076de4c48 .hgsigs --- a/.hgsigs Thu Mar 15 16:42:18 2012 +0200 +++ b/.hgsigs Thu Mar 15 16:42:22 2012 +0200 @@ -41,3 +41,4 @@ 736f1b7af190ea68e65719277bd8d3d4682e0844 0 iEYEABECAAYFAk87Kz4ACgkQyUhSUUBVismKXACeME7EBYoEuoLLELp0uX6B/lkgRWQAoJ2OAgz4mmluGbi0Db8grDDWSsTj e2cd03cc9c690c4989fb53a1871b7109c547388a 0 iEYEABECAAYFAk89MXsACgkQyUhSUUBVisnPmwCgiAr4OfQX1uAjhuqj5X0xbd8O1NQAn2bQW+h8QPAbqN6dQRNTm82D2hNF 04b0acc03f1eaa0353888a75a452e5c8e61e4c94 0 iEYEABECAAYFAk9F+k0ACgkQyUhSUUBViskLmwCfUt/aex6wOIEohJKnRGA4diF5WxoAn2zlMxSaPX/b0LBmV1P46GAMqZbO +744e0d7f1b255a9339060f761b850303121f14df 0 iEYEABECAAYFAk9h/8oACgkQyUhSUUBVism2OQCfWh62w8pMxJaf1oYx2A+2PxQvBocAn29RFDgZblGRLn7iMCPw6We1yiIw From dovecot at dovecot.org Sun May 20 03:26:28 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:28 +0300 Subject: dovecot-2.2: replicator: Fixed off-by-one-second "do we do a ful... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/e935ddb5a9fc changeset: 14354:e935ddb5a9fc user: Timo Sirainen date: Thu Mar 15 18:24:11 2012 +0200 description: replicator: Fixed off-by-one-second "do we do a full sync now?" check. diffstat: src/replication/replicator/replicator-brain.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diffs (22 lines): diff -r 914076de4c48 -r e935ddb5a9fc src/replication/replicator/replicator-brain.c --- a/src/replication/replicator/replicator-brain.c Thu Mar 15 16:42:22 2012 +0200 +++ b/src/replication/replicator/replicator-brain.c Thu Mar 15 18:24:11 2012 +0200 @@ -103,14 +103,16 @@ { struct replicator_sync_context *ctx; struct doveadm_connection *conn; + time_t next_full_sync; bool full; conn = get_doveadm_connection(brain); if (conn == NULL) return FALSE; - full = user->last_full_sync + - brain->set->replication_full_sync_interval < ioloop_time; + next_full_sync = user->last_full_sync + + brain->set->replication_full_sync_interval; + full = next_full_sync <= ioloop_time; /* update the sync times immediately. if the replication fails we still wouldn't want it to be retried immediately. */ user->last_fast_sync = ioloop_time; From dovecot at dovecot.org Sun May 20 03:26:28 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:28 +0300 Subject: dovecot-2.2: Compiler warning fixes. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/5b05411a81cf changeset: 14355:5b05411a81cf user: Timo Sirainen date: Thu Mar 15 18:24:15 2012 +0200 description: Compiler warning fixes. diffstat: src/director/user-directory.c | 2 +- src/lib-index/mail-cache-fields.c | 2 +- src/replication/replicator/replicator-queue.c | 7 ++++--- 3 files changed, 6 insertions(+), 5 deletions(-) diffs (50 lines): diff -r e935ddb5a9fc -r 5b05411a81cf src/director/user-directory.c --- a/src/director/user-directory.c Thu Mar 15 18:24:11 2012 +0200 +++ b/src/director/user-directory.c Thu Mar 15 18:24:15 2012 +0200 @@ -144,7 +144,7 @@ bool user_directory_user_is_recently_updated(struct user_directory *dir, struct user *user) { - return user->timestamp + dir->timeout_secs/2 >= ioloop_time; + return (time_t)(user->timestamp + dir->timeout_secs/2) >= ioloop_time; } bool user_directory_user_is_near_expiring(struct user_directory *dir, diff -r e935ddb5a9fc -r 5b05411a81cf src/lib-index/mail-cache-fields.c --- a/src/lib-index/mail-cache-fields.c Thu Mar 15 18:24:11 2012 +0200 +++ b/src/lib-index/mail-cache-fields.c Thu Mar 15 18:24:15 2012 +0200 @@ -403,7 +403,7 @@ cache->file_field_map[i] = fidx; /* update last_used if it's newer than ours */ - if (last_used[i] > cache->fields[fidx].field.last_used) + if ((time_t)last_used[i] > cache->fields[fidx].field.last_used) cache->fields[fidx].field.last_used = last_used[i]; dec = cache->fields[fidx].field.decision; diff -r e935ddb5a9fc -r 5b05411a81cf src/replication/replicator/replicator-queue.c --- a/src/replication/replicator/replicator-queue.c Thu Mar 15 18:24:11 2012 +0200 +++ b/src/replication/replicator/replicator-queue.c Thu Mar 15 18:24:15 2012 +0200 @@ -188,6 +188,7 @@ { struct priorityq_item *item; struct replicator_user *user; + time_t next_full_sync; item = priorityq_peek(queue->user_queue); if (item == NULL) { @@ -197,11 +198,11 @@ } user = (struct replicator_user *)item; + next_full_sync = user->last_full_sync + queue->full_sync_interval; if (user->priority == REPLICATION_PRIORITY_NONE && - user->last_full_sync + queue->full_sync_interval > ioloop_time) { + next_full_sync > ioloop_time) { /* we don't want to do a full sync yet */ - *next_secs_r = user->last_full_sync + - queue->full_sync_interval - ioloop_time; + *next_secs_r = next_full_sync - ioloop_time; return NULL; } priorityq_remove(queue->user_queue, &user->item); From dovecot at dovecot.org Sun May 20 03:26:28 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:28 +0300 Subject: dovecot-2.2: dbox: Reverted recent fstat() avoidance change. It ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/58732d172107 changeset: 14356:58732d172107 user: Timo Sirainen date: Fri Mar 16 18:45:50 2012 +0200 description: dbox: Reverted recent fstat() avoidance change. It didn't do that, just broke mdbox. diffstat: src/lib-storage/index/dbox-common/dbox-file.c | 2 +- src/lib-storage/index/dbox-common/dbox-file.h | 1 - src/lib-storage/index/dbox-multi/mdbox-file.c | 1 - src/lib-storage/index/dbox-single/sdbox-file.c | 1 - 4 files changed, 1 insertions(+), 4 deletions(-) diffs (45 lines): diff -r 5b05411a81cf -r 58732d172107 src/lib-storage/index/dbox-common/dbox-file.c --- a/src/lib-storage/index/dbox-common/dbox-file.c Thu Mar 15 18:24:15 2012 +0200 +++ b/src/lib-storage/index/dbox-common/dbox-file.c Fri Mar 16 18:45:50 2012 +0200 @@ -588,7 +588,7 @@ return 0; } - if (ctx->output->offset == 0 && !file->created) { + if (ctx->output->offset == 0) { /* first append to existing file. seek to eof first. */ if (fstat(file->fd, &st) < 0) { dbox_file_set_syscall_error(file, "fstat()"); diff -r 5b05411a81cf -r 58732d172107 src/lib-storage/index/dbox-common/dbox-file.h --- a/src/lib-storage/index/dbox-common/dbox-file.h Thu Mar 15 18:24:15 2012 +0200 +++ b/src/lib-storage/index/dbox-common/dbox-file.h Fri Mar 16 18:45:50 2012 +0200 @@ -116,7 +116,6 @@ ARRAY_DEFINE(metadata, const char *); uoff_t metadata_read_offset; - unsigned int created:1; /* this file is now being created */ unsigned int appending:1; unsigned int deleted:1; unsigned int corrupted:1; diff -r 5b05411a81cf -r 58732d172107 src/lib-storage/index/dbox-multi/mdbox-file.c --- a/src/lib-storage/index/dbox-multi/mdbox-file.c Thu Mar 15 18:24:15 2012 +0200 +++ b/src/lib-storage/index/dbox-multi/mdbox-file.c Fri Mar 16 18:45:50 2012 +0200 @@ -106,7 +106,6 @@ bool create_parents; int ret; - _file->created = TRUE; create_parents = dbox_file_is_in_alt(_file); _file->fd = _file->storage->v. file_create_fd(_file, _file->cur_path, create_parents); diff -r 5b05411a81cf -r 58732d172107 src/lib-storage/index/dbox-single/sdbox-file.c --- a/src/lib-storage/index/dbox-single/sdbox-file.c Thu Mar 15 18:24:15 2012 +0200 +++ b/src/lib-storage/index/dbox-single/sdbox-file.c Fri Mar 16 18:45:50 2012 +0200 @@ -60,7 +60,6 @@ struct dbox_file *file; file = sdbox_file_init(mbox, 0); - file->created = TRUE; file->fd = file->storage->v. file_create_fd(file, file->primary_path, FALSE); return file; From dovecot at dovecot.org Sun May 20 03:26:28 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:28 +0300 Subject: dovecot-2.2: Released v2.1.3. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/b9adfd52cb66 changeset: 14357:b9adfd52cb66 user: Timo Sirainen date: Fri Mar 16 18:48:40 2012 +0200 description: Released v2.1.3. diffstat: NEWS | 4 ++++ configure.in | 2 +- 2 files changed, 5 insertions(+), 1 deletions(-) diffs (21 lines): diff -r 58732d172107 -r b9adfd52cb66 NEWS --- a/NEWS Fri Mar 16 18:45:50 2012 +0200 +++ b/NEWS Fri Mar 16 18:48:40 2012 +0200 @@ -1,3 +1,7 @@ +v2.1.3 2012-03-16 Timo Sirainen + + - mdbox was broken in v2.1.2 + v2.1.2 2012-03-15 Timo Sirainen + Initial implementation of dsync-based replication. For now this diff -r 58732d172107 -r b9adfd52cb66 configure.in --- a/configure.in Fri Mar 16 18:45:50 2012 +0200 +++ b/configure.in Fri Mar 16 18:48:40 2012 +0200 @@ -1,5 +1,5 @@ AC_PREREQ([2.59]) -AC_INIT([Dovecot],[2.1.2],[dovecot at dovecot.org]) +AC_INIT([Dovecot],[2.1.3],[dovecot at dovecot.org]) AC_CONFIG_SRCDIR([src]) AM_INIT_AUTOMAKE([foreign]) From dovecot at dovecot.org Sun May 20 03:26:29 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:29 +0300 Subject: dovecot-2.2: Added tag 2.1.3 for changeset b9adfd52cb66 Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/3df9273f15b4 changeset: 14358:3df9273f15b4 user: Timo Sirainen date: Fri Mar 16 18:48:40 2012 +0200 description: Added tag 2.1.3 for changeset b9adfd52cb66 diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r b9adfd52cb66 -r 3df9273f15b4 .hgtags --- a/.hgtags Fri Mar 16 18:48:40 2012 +0200 +++ b/.hgtags Fri Mar 16 18:48:40 2012 +0200 @@ -79,3 +79,4 @@ e2cd03cc9c690c4989fb53a1871b7109c547388a 2.1.0 04b0acc03f1eaa0353888a75a452e5c8e61e4c94 2.1.1 744e0d7f1b255a9339060f761b850303121f14df 2.1.2 +b9adfd52cb66d5d89d291b76b9845d6361216d12 2.1.3 From dovecot at dovecot.org Sun May 20 03:26:29 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:29 +0300 Subject: dovecot-2.2: Added signature for changeset b9adfd52cb66 Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/063cd2a15056 changeset: 14359:063cd2a15056 user: Timo Sirainen date: Fri Mar 16 18:48:46 2012 +0200 description: Added signature for changeset b9adfd52cb66 diffstat: .hgsigs | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r 3df9273f15b4 -r 063cd2a15056 .hgsigs --- a/.hgsigs Fri Mar 16 18:48:40 2012 +0200 +++ b/.hgsigs Fri Mar 16 18:48:46 2012 +0200 @@ -42,3 +42,4 @@ e2cd03cc9c690c4989fb53a1871b7109c547388a 0 iEYEABECAAYFAk89MXsACgkQyUhSUUBVisnPmwCgiAr4OfQX1uAjhuqj5X0xbd8O1NQAn2bQW+h8QPAbqN6dQRNTm82D2hNF 04b0acc03f1eaa0353888a75a452e5c8e61e4c94 0 iEYEABECAAYFAk9F+k0ACgkQyUhSUUBViskLmwCfUt/aex6wOIEohJKnRGA4diF5WxoAn2zlMxSaPX/b0LBmV1P46GAMqZbO 744e0d7f1b255a9339060f761b850303121f14df 0 iEYEABECAAYFAk9h/8oACgkQyUhSUUBVism2OQCfWh62w8pMxJaf1oYx2A+2PxQvBocAn29RFDgZblGRLn7iMCPw6We1yiIw +b9adfd52cb66d5d89d291b76b9845d6361216d12 0 iEYEABECAAYFAk9jbugACgkQyUhSUUBVislSgwCgpo3f0bsSujItBum/M6js8SzF06YAmwftHlwaOstKeALdjLR5vtF2c5F7 From dovecot at dovecot.org Sun May 20 03:26:29 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:29 +0300 Subject: dovecot-2.2: layout=fs: Renaming non-UTF8 mailbox names wasn't d... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/c77fbfce438d changeset: 14360:c77fbfce438d user: Timo Sirainen date: Mon Mar 19 15:53:25 2012 +0200 description: layout=fs: Renaming non-UTF8 mailbox names wasn't done in all situations. diffstat: src/lib-storage/list/mailbox-list-fs-iter.c | 67 ++++++++++++++-------------- 1 files changed, 34 insertions(+), 33 deletions(-) diffs (98 lines): diff -r 063cd2a15056 -r c77fbfce438d src/lib-storage/list/mailbox-list-fs-iter.c --- a/src/lib-storage/list/mailbox-list-fs-iter.c Fri Mar 16 18:48:46 2012 +0200 +++ b/src/lib-storage/list/mailbox-list-fs-iter.c Mon Mar 19 15:53:25 2012 +0200 @@ -85,6 +85,33 @@ return 0; } +static void +fs_list_rename_invalid(struct fs_list_iterate_context *ctx, + const char *storage_name) +{ + /* the storage_name is completely invalid, rename it to + something more sensible. we could do this for all names that + aren't valid mUTF-7, but that might lead to accidents in + future when UTF-8 storage names are used */ + string_t *destname = t_str_new(128); + string_t *dest = t_str_new(128); + const char *root, *src; + + root = mailbox_list_get_path(ctx->ctx.list, NULL, + MAILBOX_LIST_PATH_TYPE_MAILBOX); + src = t_strconcat(root, "/", storage_name, NULL); + + (void)uni_utf8_get_valid_data((const void *)storage_name, + strlen(storage_name), destname); + + str_append(dest, root); + str_append_c(dest, '/'); + (void)imap_utf8_to_utf7(str_c(destname), dest); + + if (rename(src, str_c(dest)) < 0 && errno != ENOENT) + i_error("rename(%s, %s) failed: %m", src, str_c(dest)); +} + static int dir_entry_get(struct fs_list_iterate_context *ctx, const char *dir_path, struct list_dir_context *dir, const struct dirent *d) @@ -120,6 +147,13 @@ storage_name = *dir->storage_name == '\0' ? d->d_name : t_strconcat(dir->storage_name, "/", d->d_name, NULL); vname = mailbox_list_get_vname(ctx->ctx.list, storage_name); + if (!uni_utf8_str_is_valid(vname)) { + fs_list_rename_invalid(ctx, storage_name); + /* just skip this in this iteration, we'll see it on the + next list */ + return 0; + } + match = imap_match(ctx->ctx.glob, vname); if ((dir->info_flags & (MAILBOX_CHILDREN | MAILBOX_NOCHILDREN | @@ -512,33 +546,6 @@ return strcmp(path, inbox_path) == 0; } -static void -fs_list_rename_invalid(struct fs_list_iterate_context *ctx, - const char *storage_name) -{ - /* the storage_name is completely invalid, rename it to - something more sensible. we could do this for all names that - aren't valid mUTF-7, but that might lead to accidents in - future when UTF-8 storage names are used */ - string_t *destname = t_str_new(128); - string_t *dest = t_str_new(128); - const char *root, *src; - - root = mailbox_list_get_path(ctx->ctx.list, NULL, - MAILBOX_LIST_PATH_TYPE_MAILBOX); - src = t_strconcat(root, "/", storage_name, NULL); - - (void)uni_utf8_get_valid_data((const void *)storage_name, - strlen(storage_name), destname); - - str_append(dest, root); - str_append_c(dest, '/'); - (void)imap_utf8_to_utf7(str_c(destname), dest); - - if (rename(src, str_c(dest)) < 0 && errno != ENOENT) - i_error("rename(%s, %s) failed: %m", src, str_c(dest)); -} - static int fs_list_entry(struct fs_list_iterate_context *ctx, const struct list_dir_entry *entry) @@ -553,12 +560,6 @@ t_strconcat(dir->storage_name, "/", entry->fname, NULL); vname = mailbox_list_get_vname(ctx->ctx.list, storage_name); - if (!uni_utf8_str_is_valid(vname)) { - fs_list_rename_invalid(ctx, storage_name); - /* just skip this in this iteration, we'll see it on the - next list */ - return 0; - } ctx->info.name = p_strdup(ctx->info_pool, vname); ctx->info.flags = entry->info_flags; From dovecot at dovecot.org Sun May 20 03:26:29 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:29 +0300 Subject: dovecot-2.2: imap: Handle XLIST command by running LIST command. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/03499202690f changeset: 14362:03499202690f user: Timo Sirainen date: Wed Mar 21 12:08:32 2012 +0200 description: imap: Handle XLIST command by running LIST command. If the SPECIAL-USE mailboxes are enabled, this results in mostly compatible output. This change allows easily enabling the GMail XLIST extension simply by adding +XLIST to imap_capability setting. diffstat: src/imap/imap-commands.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (13 lines): diff -r b2a3775c7a87 -r 03499202690f src/imap/imap-commands.c --- a/src/imap/imap-commands.c Tue Mar 20 17:46:44 2012 +0200 +++ b/src/imap/imap-commands.c Wed Mar 21 12:08:32 2012 +0200 @@ -59,7 +59,8 @@ { "UID SORT", cmd_sort, COMMAND_FLAG_BREAKS_SEQS }, { "UID THREAD", cmd_thread, COMMAND_FLAG_BREAKS_SEQS }, { "UNSELECT", cmd_unselect, COMMAND_FLAG_BREAKS_MAILBOX }, - { "X-CANCEL", cmd_x_cancel, 0 } + { "X-CANCEL", cmd_x_cancel, 0 }, + { "XLIST", cmd_list, 0 } }; #define IMAP_EXT_COMMANDS_COUNT N_ELEMENTS(imap_ext_commands) From dovecot at dovecot.org Sun May 20 03:26:29 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:29 +0300 Subject: dovecot-2.2: doveadm import: Copy also message flags. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/b2a3775c7a87 changeset: 14361:b2a3775c7a87 user: Timo Sirainen date: Tue Mar 20 17:46:44 2012 +0200 description: doveadm import: Copy also message flags. diffstat: src/doveadm/doveadm-mail-import.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff -r c77fbfce438d -r b2a3775c7a87 src/doveadm/doveadm-mail-import.c --- a/src/doveadm/doveadm-mail-import.c Mon Mar 19 15:53:25 2012 +0200 +++ b/src/doveadm/doveadm-mail-import.c Tue Mar 20 17:46:44 2012 +0200 @@ -91,6 +91,7 @@ mailbox, src_mail->uid); } save_ctx = mailbox_save_alloc(dest_trans); + mailbox_save_copy_flags(save_ctx, src_mail); if (mailbox_copy(&save_ctx, src_mail) < 0) { i_error("Copying box=%s uid=%u failed: %s", mailbox, src_mail->uid, From dovecot at dovecot.org Sun May 20 03:26:29 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:29 +0300 Subject: dovecot-2.2: maildir: Avoid duplicate S=size and W=sizes in mail... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/3599790da3d7 changeset: 14364:3599790da3d7 user: Timo Sirainen date: Wed Mar 21 14:25:12 2012 +0200 description: maildir: Avoid duplicate S=size and W=sizes in maildir filenames. This happens with maildir_copy_with_hardlinks=no and when copying a mail with zlib compression enabled. diffstat: src/lib-storage/index/maildir/maildir-save.c | 9 ++++++++- 1 files changed, 8 insertions(+), 1 deletions(-) diffs (29 lines): diff -r 23ed09ca847a -r 3599790da3d7 src/lib-storage/index/maildir/maildir-save.c --- a/src/lib-storage/index/maildir/maildir-save.c Wed Mar 21 13:43:56 2012 +0200 +++ b/src/lib-storage/index/maildir/maildir-save.c Wed Mar 21 14:25:12 2012 +0200 @@ -516,6 +516,7 @@ struct mail_storage *storage = &ctx->mbox->storage->storage; const char *path; off_t real_size; + uoff_t size; int output_errno; ctx->last_save_finished = TRUE; @@ -575,10 +576,16 @@ if (real_size == (off_t)-1) { mail_storage_set_critical(storage, "lseek(%s) failed: %m", path); - } else if (real_size != (off_t)ctx->file_last->size) { + } else if (real_size != (off_t)ctx->file_last->size && + (!maildir_filename_get_size(ctx->file_last->dest_basename, + MAILDIR_EXTRA_FILE_SIZE, &size) || + size != ctx->file_last->size)) { /* e.g. zlib plugin was used. the "physical size" must be in the maildir filename, since stat() will return wrong size */ ctx->file_last->preserve_filename = FALSE; + /* reset the base name as well, just in case there's a + ,W=vsize */ + ctx->file_last->dest_basename = ctx->file_last->tmp_name; } if (close(ctx->fd) < 0) { if (!mail_storage_set_error_from_errno(storage)) { From dovecot at dovecot.org Sun May 20 03:26:29 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:29 +0300 Subject: dovecot-2.2: auth: Don't check client PID in non-login auth sock... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/23ed09ca847a changeset: 14363:23ed09ca847a user: Timo Sirainen date: Wed Mar 21 13:43:56 2012 +0200 description: auth: Don't check client PID in non-login auth sockets. This fixes PID conflict errors when using TCP auth sockets for e.g. MTAs. diffstat: src/auth/auth-client-connection.c | 11 ++++++++++- 1 files changed, 10 insertions(+), 1 deletions(-) diffs (21 lines): diff -r 03499202690f -r 23ed09ca847a src/auth/auth-client-connection.c --- a/src/auth/auth-client-connection.c Wed Mar 21 12:08:32 2012 +0200 +++ b/src/auth/auth-client-connection.c Wed Mar 21 13:43:56 2012 +0200 @@ -95,7 +95,16 @@ return FALSE; } - old = auth_client_connection_lookup(pid); + if (conn->login_requests) + old = auth_client_connection_lookup(pid); + else { + /* the client is only authenticating, not logging in. + the PID isn't necessary, and since we allow authentication + via TCP sockets the PIDs may conflict, so ignore them. */ + old = NULL; + pid = 0; + } + if (old != NULL) { /* already exists. it's possible that it just reconnected, see if the old connection is still there. */ From dovecot at dovecot.org Sun May 20 03:26:29 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:29 +0300 Subject: dovecot-2.2: lib-settings: settings_parser_apply_changes() now d... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/bb8387e6b18f changeset: 14365:bb8387e6b18f user: Timo Sirainen date: Wed Mar 21 18:58:37 2012 +0200 description: lib-settings: settings_parser_apply_changes() now deduplicates SET_STRLIST arrays. This fixes overriding strlist keys in config file filters, e.g.: plugin { foo = general } protocol lda { plugin { foo = lda-specific setting } } diffstat: src/lib-settings/settings-parser.c | 20 +++++++++++++++++--- src/lib-settings/settings-parser.h | 6 +++++- 2 files changed, 22 insertions(+), 4 deletions(-) diffs (55 lines): diff -r 3599790da3d7 -r bb8387e6b18f src/lib-settings/settings-parser.c --- a/src/lib-settings/settings-parser.c Wed Mar 21 14:25:12 2012 +0200 +++ b/src/lib-settings/settings-parser.c Wed Mar 21 18:58:37 2012 +0200 @@ -1379,18 +1379,32 @@ case SET_STRLIST: { const ARRAY_TYPE(const_string) *src_arr = src; ARRAY_TYPE(const_string) *dest_arr = dest; - const char *const *strings, *dup; - unsigned int i, count; + const char *const *strings, *const *dest_strings, *dup; + unsigned int i, j, count, dest_count; if (!array_is_created(src_arr)) break; strings = array_get(src_arr, &count); + i_assert(count % 2 == 0); if (!array_is_created(dest_arr)) p_array_init(dest_arr, pool, count); - for (i = 0; i < count; i++) { + dest_count = array_count(dest_arr); + i_assert(dest_count % 2 == 0); + for (i = 0; i < count; i += 2) { + if (dest_count > 0) { + dest_strings = array_idx(dest_arr, 0); + for (j = 0; j < dest_count; j += 2) { + if (strcmp(strings[i], dest_strings[j]) == 0) + break; + } + if (j < dest_count) + continue; + } dup = p_strdup(pool, strings[i]); array_append(dest_arr, &dup, 1); + dup = p_strdup(pool, strings[i+1]); + array_append(dest_arr, &dup, 1); } break; } diff -r 3599790da3d7 -r bb8387e6b18f src/lib-settings/settings-parser.h --- a/src/lib-settings/settings-parser.h Wed Mar 21 14:25:12 2012 +0200 +++ b/src/lib-settings/settings-parser.h Wed Mar 21 18:58:37 2012 +0200 @@ -208,7 +208,11 @@ /* Copy changed settings from src to dest. If conflict_key_r is not NULL and both src and dest have changed the same setting, return -1 and set the - key name. If it's NULL, the old setting is kept. */ + key name. If it's NULL, the old setting is kept. + + KLUDGE: For SET_STRLIST types if both source and destination have identical + keys, the duplicates in the source side are ignored. This is required to + make the current config code work correctly. */ int settings_parser_apply_changes(struct setting_parser_context *dest, const struct setting_parser_context *src, pool_t pool, const char **conflict_key_r); From dovecot at dovecot.org Sun May 20 03:26:29 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:29 +0300 Subject: dovecot-2.2: config: Added a comment Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/ff5c341f8838 changeset: 14366:ff5c341f8838 user: Timo Sirainen date: Wed Mar 21 19:01:44 2012 +0200 description: config: Added a comment diffstat: src/config/config-filter.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diffs (15 lines): diff -r bb8387e6b18f -r ff5c341f8838 src/config/config-filter.c --- a/src/config/config-filter.c Wed Mar 21 18:58:37 2012 +0200 +++ b/src/config/config-filter.c Wed Mar 21 19:01:44 2012 +0200 @@ -314,6 +314,11 @@ const char *error = NULL, **error_p; unsigned int i, count; + /* get the matching filters. the most specific ones are handled first, + so that if more generic filters try to override settings we'll fail + with an error. Merging SET_STRLIST types requires + settings_parser_apply_changes() to work a bit unintuitively by + letting the destination settings override the source settings. */ src = config_filter_find_all(ctx, module, filter, output_r); /* all of them should have the same number of parsers. From dovecot at dovecot.org Sun May 20 03:26:29 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:29 +0300 Subject: dovecot-2.2: *-login: Another crashfix Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/842e5124038d changeset: 14368:842e5124038d user: Timo Sirainen date: Thu Mar 22 16:03:04 2012 +0200 description: *-login: Another crashfix diffstat: src/login-common/client-common.c | 2 +- src/login-common/login-proxy.c | 2 +- src/login-common/main.c | 2 +- src/login-common/ssl-proxy-openssl.c | 18 ++++++++++++------ src/login-common/ssl-proxy.h | 4 ++-- 5 files changed, 17 insertions(+), 11 deletions(-) diffs (122 lines): diff -r 1d23440ccb89 -r 842e5124038d src/login-common/client-common.c --- a/src/login-common/client-common.c Thu Mar 22 15:32:00 2012 +0200 +++ b/src/login-common/client-common.c Thu Mar 22 16:03:04 2012 +0200 @@ -297,7 +297,7 @@ if (!client_unref(&client) || client->destroyed) return; - fd_ssl = ssl_proxy_alloc(client->fd, &client->ip, + fd_ssl = ssl_proxy_alloc(client->fd, &client->ip, client->pool, client->set, &client->ssl_proxy); if (fd_ssl == -1) { client_send_line(client, CLIENT_CMD_REPLY_BYE, diff -r 1d23440ccb89 -r 842e5124038d src/login-common/login-proxy.c --- a/src/login-common/login-proxy.c Thu Mar 22 15:32:00 2012 +0200 +++ b/src/login-common/login-proxy.c Thu Mar 22 16:03:04 2012 +0200 @@ -545,7 +545,7 @@ io_remove(&proxy->server_io); fd = ssl_proxy_client_alloc(proxy->server_fd, &proxy->client->ip, - proxy->client->set, + proxy->client->pool, proxy->client->set, login_proxy_ssl_handshaked, proxy, &proxy->ssl_server_proxy); if (fd < 0) { diff -r 1d23440ccb89 -r 842e5124038d src/login-common/main.c --- a/src/login-common/main.c Thu Mar 22 15:32:00 2012 +0200 +++ b/src/login-common/main.c Thu Mar 22 16:03:04 2012 +0200 @@ -123,7 +123,7 @@ client = client_create(conn->fd, FALSE, pool, set, other_sets, &local_ip, &conn->remote_ip); } else { - fd_ssl = ssl_proxy_alloc(conn->fd, &conn->remote_ip, set, + fd_ssl = ssl_proxy_alloc(conn->fd, &conn->remote_ip, pool, set, &proxy); if (fd_ssl == -1) { net_disconnect(conn->fd); diff -r 1d23440ccb89 -r 842e5124038d src/login-common/ssl-proxy-openssl.c --- a/src/login-common/ssl-proxy-openssl.c Thu Mar 22 15:32:00 2012 +0200 +++ b/src/login-common/ssl-proxy-openssl.c Thu Mar 22 16:03:04 2012 +0200 @@ -52,6 +52,7 @@ struct client *client; struct ip_addr ip; const struct login_settings *set; + pool_t set_pool; int fd_ssl, fd_plain; struct io *io_ssl_read, *io_ssl_write, *io_plain_read, *io_plain_write; @@ -543,7 +544,7 @@ static int ssl_proxy_alloc_common(SSL_CTX *ssl_ctx, int fd, const struct ip_addr *ip, - const struct login_settings *set, + pool_t set_pool, const struct login_settings *set, struct ssl_proxy **proxy_r) { struct ssl_proxy *proxy; @@ -590,7 +591,9 @@ proxy->fd_ssl = fd; proxy->fd_plain = sfd[0]; proxy->ip = *ip; - SSL_set_ex_data(ssl, extdata_index, proxy); + proxy->set_pool = set_pool; + pool_ref(set_pool); + SSL_set_ex_data(ssl, extdata_index, proxy); ssl_proxy_count++; DLLIST_PREPEND(&ssl_proxies, proxy); @@ -618,24 +621,26 @@ return ctx; } -int ssl_proxy_alloc(int fd, const struct ip_addr *ip, +int ssl_proxy_alloc(int fd, const struct ip_addr *ip, pool_t set_pool, const struct login_settings *set, struct ssl_proxy **proxy_r) { struct ssl_server_context *ctx; ctx = ssl_server_context_get(set); - return ssl_proxy_alloc_common(ctx->ctx, fd, ip, set, proxy_r); + return ssl_proxy_alloc_common(ctx->ctx, fd, ip, + set_pool, set, proxy_r); } -int ssl_proxy_client_alloc(int fd, struct ip_addr *ip, +int ssl_proxy_client_alloc(int fd, struct ip_addr *ip, pool_t set_pool, const struct login_settings *set, ssl_handshake_callback_t *callback, void *context, struct ssl_proxy **proxy_r) { int ret; - ret = ssl_proxy_alloc_common(ssl_client_ctx, fd, ip, set, proxy_r); + ret = ssl_proxy_alloc_common(ssl_client_ctx, fd, ip, + set_pool, set, proxy_r); if (ret < 0) return -1; @@ -767,6 +772,7 @@ SSL_free(proxy->ssl); + pool_unref(&proxy->set_pool); i_free(proxy->last_error); i_free(proxy); } diff -r 1d23440ccb89 -r 842e5124038d src/login-common/ssl-proxy.h --- a/src/login-common/ssl-proxy.h Thu Mar 22 15:32:00 2012 +0200 +++ b/src/login-common/ssl-proxy.h Thu Mar 22 16:03:04 2012 +0200 @@ -13,10 +13,10 @@ /* establish SSL connection with the given fd, returns a new fd which you must use from now on, or -1 if error occurred. Unless -1 is returned, the given fd must be simply forgotten. */ -int ssl_proxy_alloc(int fd, const struct ip_addr *ip, +int ssl_proxy_alloc(int fd, const struct ip_addr *ip, pool_t set_pool, const struct login_settings *set, struct ssl_proxy **proxy_r); -int ssl_proxy_client_alloc(int fd, struct ip_addr *ip, +int ssl_proxy_client_alloc(int fd, struct ip_addr *ip, pool_t set_pool, const struct login_settings *set, ssl_handshake_callback_t *callback, void *context, struct ssl_proxy **proxy_r); From dovecot at dovecot.org Sun May 20 03:26:29 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:29 +0300 Subject: dovecot-2.2: *-login: Fixed crashing when proxying SSL connectio... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/1d23440ccb89 changeset: 14367:1d23440ccb89 user: Timo Sirainen date: Thu Mar 22 15:32:00 2012 +0200 description: *-login: Fixed crashing when proxying SSL connections to a remote server. diffstat: src/login-common/client-common.c | 7 ++++--- src/login-common/ssl-proxy-openssl.c | 8 +++++--- src/login-common/ssl-proxy.h | 1 + 3 files changed, 10 insertions(+), 6 deletions(-) diffs (67 lines): diff -r ff5c341f8838 -r 1d23440ccb89 src/login-common/client-common.c --- a/src/login-common/client-common.c Wed Mar 21 19:01:44 2012 +0200 +++ b/src/login-common/client-common.c Thu Mar 22 15:32:00 2012 +0200 @@ -180,10 +180,10 @@ i_free_and_null(client->proxy_password); } + if (client->ssl_proxy != NULL) + ssl_proxy_unset_client(client->ssl_proxy); if (client->login_proxy != NULL) login_proxy_free(&client->login_proxy); - if (client->ssl_proxy != NULL) - ssl_proxy_free(&client->ssl_proxy); client->v.destroy(client); if (client_unref(&client) && initial_service_count == 1) { /* as soon as this connection is done with proxying @@ -229,9 +229,10 @@ *_client = NULL; i_assert(client->destroyed); - i_assert(client->ssl_proxy == NULL); i_assert(client->login_proxy == NULL); + if (client->ssl_proxy != NULL) + ssl_proxy_free(&client->ssl_proxy); if (client->input != NULL) i_stream_unref(&client->input); if (client->output != NULL) diff -r ff5c341f8838 -r 1d23440ccb89 src/login-common/ssl-proxy-openssl.c --- a/src/login-common/ssl-proxy-openssl.c Wed Mar 21 19:01:44 2012 +0200 +++ b/src/login-common/ssl-proxy-openssl.c Thu Mar 22 15:32:00 2012 +0200 @@ -654,10 +654,14 @@ { i_assert(proxy->client == NULL); - client_ref(client); proxy->client = client; } +void ssl_proxy_unset_client(struct ssl_proxy *proxy) +{ + proxy->client = NULL; +} + bool ssl_proxy_has_valid_client_cert(const struct ssl_proxy *proxy) { return proxy->cert_received && !proxy->cert_broken; @@ -763,8 +767,6 @@ SSL_free(proxy->ssl); - if (proxy->client != NULL) - client_unref(&proxy->client); i_free(proxy->last_error); i_free(proxy); } diff -r ff5c341f8838 -r 1d23440ccb89 src/login-common/ssl-proxy.h --- a/src/login-common/ssl-proxy.h Wed Mar 21 19:01:44 2012 +0200 +++ b/src/login-common/ssl-proxy.h Thu Mar 22 15:32:00 2012 +0200 @@ -22,6 +22,7 @@ struct ssl_proxy **proxy_r); void ssl_proxy_start(struct ssl_proxy *proxy); void ssl_proxy_set_client(struct ssl_proxy *proxy, struct client *client); +void ssl_proxy_unset_client(struct ssl_proxy *proxy); bool ssl_proxy_has_valid_client_cert(const struct ssl_proxy *proxy) ATTR_PURE; bool ssl_proxy_has_broken_client_cert(struct ssl_proxy *proxy); int ssl_proxy_cert_match_name(struct ssl_proxy *proxy, const char *verify_name); From dovecot at dovecot.org Sun May 20 03:26:29 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:29 +0300 Subject: dovecot-2.2: imap-login: Memory leak fix. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/5ae5c44b1943 changeset: 14369:5ae5c44b1943 user: Timo Sirainen date: Thu Mar 22 16:03:29 2012 +0200 description: imap-login: Memory leak fix. diffstat: src/login-common/client-common.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff -r 842e5124038d -r 5ae5c44b1943 src/login-common/client-common.c --- a/src/login-common/client-common.c Thu Mar 22 16:03:04 2012 +0200 +++ b/src/login-common/client-common.c Thu Mar 22 16:03:29 2012 +0200 @@ -242,6 +242,7 @@ i_free(client->proxy_master_user); i_free(client->virtual_user); i_free(client->auth_mech_name); + i_free(client->master_data_prefix); pool_unref(&client->pool); i_assert(clients_count > 0); From dovecot at dovecot.org Sun May 20 03:26:29 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:29 +0300 Subject: dovecot-2.2: lib-master: Minor memory leak fix on deinit. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/bb8d2d74a71d changeset: 14370:bb8d2d74a71d user: Timo Sirainen date: Thu Mar 22 16:03:38 2012 +0200 description: lib-master: Minor memory leak fix on deinit. diffstat: src/lib-master/master-service.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff -r 5ae5c44b1943 -r bb8d2d74a71d src/lib-master/master-service.c --- a/src/lib-master/master-service.c Thu Mar 22 16:03:29 2012 +0200 +++ b/src/lib-master/master-service.c Thu Mar 22 16:03:38 2012 +0200 @@ -734,6 +734,8 @@ lib_signals_deinit(); io_loop_destroy(&service->ioloop); + if (service->listener_names != NULL) + p_strsplit_free(default_pool, service->listener_names); i_free(service->listeners); i_free(service->getopt_str); i_free(service->name); From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: Increased initial memory pool sizes. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/3cb33e050dc4 changeset: 14372:3cb33e050dc4 user: Timo Sirainen date: Thu Mar 22 17:12:35 2012 +0200 description: Increased initial memory pool sizes. diffstat: src/lib-dict/dict-file.c | 2 +- src/lib-storage/fail-mailbox.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diffs (24 lines): diff -r 703380324b71 -r 3cb33e050dc4 src/lib-dict/dict-file.c --- a/src/lib-dict/dict-file.c Thu Mar 22 17:11:37 2012 +0200 +++ b/src/lib-dict/dict-file.c Thu Mar 22 17:12:35 2012 +0200 @@ -254,7 +254,7 @@ struct file_dict_transaction_context *ctx; pool_t pool; - pool = pool_alloconly_create("file dict transaction", 1024); + pool = pool_alloconly_create("file dict transaction", 2048); ctx = p_new(pool, struct file_dict_transaction_context, 1); ctx->ctx.dict = _dict; ctx->pool = pool; diff -r 703380324b71 -r 3cb33e050dc4 src/lib-storage/fail-mailbox.c --- a/src/lib-storage/fail-mailbox.c Thu Mar 22 17:11:37 2012 +0200 +++ b/src/lib-storage/fail-mailbox.c Thu Mar 22 17:12:35 2012 +0200 @@ -299,7 +299,7 @@ struct mailbox *box; pool_t pool; - pool = pool_alloconly_create("fail mailbox", 1024); + pool = pool_alloconly_create("fail mailbox", 1024+512); box = p_new(pool, struct mailbox, 1); *box = fail_mailbox; box->vname = p_strdup(pool, vname); From dovecot at dovecot.org Sun May 20 03:26:29 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:29 +0300 Subject: dovecot-2.2: mdbox: Fixed a long loop/crash when mdbox is broken... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/703380324b71 changeset: 14371:703380324b71 user: Timo Sirainen date: Thu Mar 22 17:11:37 2012 +0200 description: mdbox: Fixed a long loop/crash when mdbox is broken during saving. diffstat: src/lib-storage/index/dbox-multi/mdbox-sync.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diffs (16 lines): diff -r bb8d2d74a71d -r 703380324b71 src/lib-storage/index/dbox-multi/mdbox-sync.c --- a/src/lib-storage/index/dbox-multi/mdbox-sync.c Thu Mar 22 16:03:38 2012 +0200 +++ b/src/lib-storage/index/dbox-multi/mdbox-sync.c Thu Mar 22 17:11:37 2012 +0200 @@ -274,6 +274,12 @@ /* we'll need to rebuild storage. try again from the beginning. */ mdbox_storage_set_corrupted(mbox->storage); + if ((flags & MDBOX_SYNC_FLAG_NO_REBUILD) != 0) { + mail_storage_set_critical(storage, + "mdbox %s: Can't rebuild storage", + mailbox_get_path(&mbox->box)); + return -1; + } return mdbox_sync_begin(mbox, flags, atomic, ctx_r); } From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: lib-storage: Don't access mail->box->view, but mail... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/39db9e4c2283 changeset: 14373:39db9e4c2283 user: Timo Sirainen date: Thu Mar 22 17:44:51 2012 +0200 description: lib-storage: Don't access mail->box->view, but mail->transaction->view. They may not be the same. In such cases e.g. mail_set_uid() may have accessed a wrong mail. diffstat: src/lib-storage/index/index-mail.c | 6 +++--- src/plugins/virtual/virtual-mail.c | 8 ++++---- 2 files changed, 7 insertions(+), 7 deletions(-) diffs (62 lines): diff -r 3cb33e050dc4 -r 39db9e4c2283 src/lib-storage/index/index-mail.c --- a/src/lib-storage/index/index-mail.c Thu Mar 22 17:12:35 2012 +0200 +++ b/src/lib-storage/index/index-mail.c Thu Mar 22 17:44:51 2012 +0200 @@ -442,7 +442,7 @@ if (set->mail_cache_min_mail_count > 0) { /* First check if we've configured caching not to be used with low enough message count. */ - hdr = mail_index_get_header(_mail->box->view); + hdr = mail_index_get_header(_mail->transaction->view); if (hdr->messages_count < set->mail_cache_min_mail_count) return; } @@ -1357,7 +1357,7 @@ /* open the stream only if we didn't get here from mailbox_save_init() */ - hdr = mail_index_get_header(_mail->box->view); + hdr = mail_index_get_header(_mail->transaction->view); if (!_mail->saving && _mail->uid < hdr->next_uid) { if ((data->access_part & READ_BODY) != 0) (void)mail_get_stream(_mail, NULL, NULL, &input); @@ -1453,7 +1453,7 @@ struct index_mail *mail = (struct index_mail *)_mail; uint32_t seq; - if (mail_index_lookup_seq(_mail->box->view, uid, &seq)) { + if (mail_index_lookup_seq(_mail->transaction->view, uid, &seq)) { index_mail_set_seq(_mail, seq, FALSE); return TRUE; } else { diff -r 3cb33e050dc4 -r 39db9e4c2283 src/plugins/virtual/virtual-mail.c --- a/src/plugins/virtual/virtual-mail.c Thu Mar 22 17:12:35 2012 +0200 +++ b/src/plugins/virtual/virtual-mail.c Thu Mar 22 17:44:51 2012 +0200 @@ -117,8 +117,8 @@ i_assert(!saving); - mail_index_lookup_ext(mail->box->view, seq, mbox->virtual_ext_id, - &data, &expunged); + mail_index_lookup_ext(mail->transaction->view, seq, + mbox->virtual_ext_id, &data, &expunged); vrec = data; bbox = virtual_backend_box_lookup(mbox, vrec->mailbox_id); @@ -131,7 +131,7 @@ vmail->imail.data.seq = seq; mail->seq = seq; - mail_index_lookup_uid(mail->box->view, seq, &mail->uid); + mail_index_lookup_uid(mail->transaction->view, seq, &mail->uid); if (!vmail->lost) { mail->expunged = vmail->backend_mail->expunged; @@ -148,7 +148,7 @@ { uint32_t seq; - if (!mail_index_lookup_seq(mail->box->view, uid, &seq)) + if (!mail_index_lookup_seq(mail->transaction->view, uid, &seq)) return FALSE; virtual_mail_set_seq(mail, seq, FALSE); From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: sdbox: Don't log "Rebuilding index" when another pr... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/f30437ed63dc changeset: 14375:f30437ed63dc user: Timo Sirainen date: Thu Mar 22 18:28:59 2012 +0200 description: sdbox: Don't log "Rebuilding index" when another process already did it and we cancel it. diffstat: src/lib-storage/index/dbox-single/sdbox-sync-rebuild.c | 1 + src/lib-storage/index/dbox-single/sdbox-sync.c | 2 -- 2 files changed, 1 insertions(+), 2 deletions(-) diffs (23 lines): diff -r fb70195b5d47 -r f30437ed63dc src/lib-storage/index/dbox-single/sdbox-sync-rebuild.c --- a/src/lib-storage/index/dbox-single/sdbox-sync-rebuild.c Thu Mar 22 18:05:08 2012 +0200 +++ b/src/lib-storage/index/dbox-single/sdbox-sync-rebuild.c Thu Mar 22 18:28:59 2012 +0200 @@ -195,6 +195,7 @@ return 0; } } + i_warning("sdbox %s: Rebuilding index", mailbox_get_path(&mbox->box)); if (dbox_sync_rebuild_verify_alt_storage(mbox->box.list) < 0) { mail_storage_set_critical(mbox->box.storage, diff -r fb70195b5d47 -r f30437ed63dc src/lib-storage/index/dbox-single/sdbox-sync.c --- a/src/lib-storage/index/dbox-single/sdbox-sync.c Thu Mar 22 18:05:08 2012 +0200 +++ b/src/lib-storage/index/dbox-single/sdbox-sync.c Thu Mar 22 18:28:59 2012 +0200 @@ -237,8 +237,6 @@ ret = -1; } else { /* do a full resync and try again. */ - i_warning("sdbox %s: Rebuilding index", - mailbox_get_path(&ctx->mbox->box)); rebuild = FALSE; ret = sdbox_sync_index_rebuild(mbox, force_rebuild); From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: quota: Added a kludge to avoid recalculating quota. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/fb70195b5d47 changeset: 14374:fb70195b5d47 user: Timo Sirainen date: Thu Mar 22 18:05:08 2012 +0200 description: quota: Added a kludge to avoid recalculating quota. diffstat: src/lib-storage/index/dbox-multi/mdbox-sync.c | 2 ++ src/lib-storage/index/dbox-single/sdbox-sync.c | 4 ++++ src/lib-storage/index/maildir/maildir-sync-index.c | 2 ++ src/lib-storage/mail-storage-private.h | 2 ++ src/plugins/quota/quota-storage.c | 8 ++++++++ 5 files changed, 18 insertions(+), 0 deletions(-) diffs (97 lines): diff -r 39db9e4c2283 -r fb70195b5d47 src/lib-storage/index/dbox-multi/mdbox-sync.c --- a/src/lib-storage/index/dbox-multi/mdbox-sync.c Thu Mar 22 17:44:51 2012 +0200 +++ b/src/lib-storage/index/dbox-multi/mdbox-sync.c Thu Mar 22 18:05:08 2012 +0200 @@ -115,11 +115,13 @@ if (box->v.sync_notify != NULL) { /* do notifications after commit finished successfully */ + box->tmp_sync_view = ctx->sync_view; seq_range_array_iter_init(&iter, &ctx->expunged_seqs); n = 0; while (seq_range_array_iter_nth(&iter, n++, &seq)) { mail_index_lookup_uid(ctx->sync_view, seq, &uid); box->v.sync_notify(box, uid, MAILBOX_SYNC_TYPE_EXPUNGE); } + box->tmp_sync_view = NULL; } return 0; } diff -r 39db9e4c2283 -r fb70195b5d47 src/lib-storage/index/dbox-single/sdbox-sync.c --- a/src/lib-storage/index/dbox-single/sdbox-sync.c Thu Mar 22 17:44:51 2012 +0200 +++ b/src/lib-storage/index/dbox-single/sdbox-sync.c Thu Mar 22 18:05:08 2012 +0200 @@ -150,10 +150,12 @@ /* NOTE: Index is no longer locked. Multiple processes may be unlinking the files at the same time. */ + ctx->mbox->box.tmp_sync_view = ctx->sync_view; array_foreach(&ctx->expunged_uids, uidp) dbox_sync_file_expunge(ctx, *uidp); if (ctx->mbox->box.v.sync_notify != NULL) ctx->mbox->box.v.sync_notify(&ctx->mbox->box, 0, 0); + ctx->mbox->box.tmp_sync_view = NULL; } static int @@ -262,11 +264,13 @@ *_ctx = NULL; if (success) { + mail_index_view_ref(ctx->sync_view); if (mail_index_sync_commit(&ctx->index_sync_ctx) < 0) { mail_storage_set_index_error(&ctx->mbox->box); ret = -1; } else { dbox_sync_expunge_files(ctx); + mail_index_view_close(&ctx->sync_view); } } else { mail_index_sync_rollback(&ctx->index_sync_ctx); diff -r 39db9e4c2283 -r fb70195b5d47 src/lib-storage/index/maildir/maildir-sync-index.c --- a/src/lib-storage/index/maildir/maildir-sync-index.c Thu Mar 22 17:44:51 2012 +0200 +++ b/src/lib-storage/index/maildir/maildir-sync-index.c Thu Mar 22 18:05:08 2012 +0200 @@ -492,6 +492,7 @@ } hdr_next_uid = hdr->next_uid; + ctx->mbox->box.tmp_sync_view = view; private_flags_mask = mailbox_get_private_flags_mask(&mbox->box); time_before_sync = time(NULL); mbox->syncing_commit = TRUE; @@ -647,6 +648,7 @@ if (mbox->box.v.sync_notify != NULL) mbox->box.v.sync_notify(&mbox->box, 0, 0); + ctx->mbox->box.tmp_sync_view = NULL; /* check cur/ mtime later. if we came here from saving messages they could still be moved to cur/ directory. */ diff -r 39db9e4c2283 -r fb70195b5d47 src/lib-storage/mail-storage-private.h --- a/src/lib-storage/mail-storage-private.h Thu Mar 22 17:44:51 2012 +0200 +++ b/src/lib-storage/mail-storage-private.h Thu Mar 22 18:05:08 2012 +0200 @@ -240,6 +240,8 @@ unsigned int transaction_count; enum mailbox_feature enabled_features; + struct mail_index_view *tmp_sync_view; + /* Mailbox notification settings: */ unsigned int notify_min_interval; mailbox_notify_callback_t *notify_callback; diff -r 39db9e4c2283 -r fb70195b5d47 src/plugins/quota/quota-storage.c --- a/src/plugins/quota/quota-storage.c Thu Mar 22 17:44:51 2012 +0200 +++ b/src/plugins/quota/quota-storage.c Thu Mar 22 18:05:08 2012 +0200 @@ -298,7 +298,15 @@ /* try to look up the size. this works only if it's cached. */ if (qbox->expunge_qt->tmp_mail == NULL) { + /* FIXME: ugly kludge to open the transaction for sync_view. + box->view may not have all the new messages that + sync_notify() notifies about, and those messages would + cause a quota recalculation. */ + struct mail_index_view *box_view = box->view; + if (box->tmp_sync_view != NULL) + box->view = box->tmp_sync_view; qbox->expunge_trans = mailbox_transaction_begin(box, 0); + box->view = box_view; qbox->expunge_qt->tmp_mail = mail_alloc(qbox->expunge_trans, MAIL_FETCH_PHYSICAL_SIZE, NULL); From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: net_getunixcred(): Fixed Solaris to use getpeerucre... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/98fd46f8d1ab changeset: 14377:98fd46f8d1ab user: Timo Sirainen date: Fri Mar 23 13:24:34 2012 +0200 description: net_getunixcred(): Fixed Solaris to use getpeerucred() properly. diffstat: src/lib/network.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r d4cc57c8a455 -r 98fd46f8d1ab src/lib/network.c --- a/src/lib/network.c Fri Mar 23 12:26:50 2012 +0200 +++ b/src/lib/network.c Fri Mar 23 13:24:34 2012 +0200 @@ -715,7 +715,7 @@ return 0; #elif defined(HAVE_GETPEERUCRED) /* Solaris */ - ucred_t *ucred; + ucred_t *ucred = NULL; if (getpeerucred(fd, &ucred) < 0) { i_error("getpeerucred() failed: %m"); From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: lib-mail: message_date_parse() now also accepts '.'... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d4cc57c8a455 changeset: 14376:d4cc57c8a455 user: Timo Sirainen date: Fri Mar 23 12:26:50 2012 +0200 description: lib-mail: message_date_parse() now also accepts '.' separator in hh.mm.ss time. diffstat: src/lib-mail/message-date.c | 10 ++++++++-- 1 files changed, 8 insertions(+), 2 deletions(-) diffs (34 lines): diff -r f30437ed63dc -r d4cc57c8a455 src/lib-mail/message-date.c --- a/src/lib-mail/message-date.c Thu Mar 22 18:28:59 2012 +0200 +++ b/src/lib-mail/message-date.c Fri Mar 23 12:26:50 2012 +0200 @@ -9,6 +9,11 @@ #include +/* RFC specifies ':' as the only allowed separator, + but be forgiving also for some broken ones */ +#define IS_TIME_SEP(c) \ + ((c) == ':' || (c) == '.') + struct message_date_parser_context { struct rfc822_parser_context parser; string_t *str; @@ -189,7 +194,7 @@ } /* :mm (may be the last token) */ - if (*ctx->parser.data != ':') + if (!IS_TIME_SEP(*ctx->parser.data)) return FALSE; ctx->parser.data++; (void)rfc822_skip_lwsp(&ctx->parser); @@ -200,7 +205,8 @@ tm.tm_min = (value[0]-'0') * 10 + (value[1]-'0'); /* [:ss] */ - if (ctx->parser.data != ctx->parser.end && *ctx->parser.data == ':') { + if (ctx->parser.data != ctx->parser.end && + IS_TIME_SEP(*ctx->parser.data)) { ctx->parser.data++; (void)rfc822_skip_lwsp(&ctx->parser); From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: unlink_old_files(): Update atime before scan, not a... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/091b9f49f24f changeset: 14378:091b9f49f24f user: Timo Sirainen date: Fri Mar 23 13:35:05 2012 +0200 description: unlink_old_files(): Update atime before scan, not after. If Maildir/dbox scanning takes long this makes sure that a lot of processes won't be scanning it in parallel. diffstat: src/lib/unlink-old-files.c | 18 +++++------------- src/lib/unlink-old-files.h | 3 ++- 2 files changed, 7 insertions(+), 14 deletions(-) diffs (48 lines): diff -r 98fd46f8d1ab -r 091b9f49f24f src/lib/unlink-old-files.c --- a/src/lib/unlink-old-files.c Fri Mar 23 13:24:34 2012 +0200 +++ b/src/lib/unlink-old-files.c Fri Mar 23 13:35:05 2012 +0200 @@ -28,6 +28,11 @@ return -1; } + /* update atime immediately, so if this scanning is done based on + atime it won't be done by multiple processes if the scan is slow */ + if (utime(dir, NULL) < 0 && errno != ENOENT) + i_error("utime(%s) failed: %m", dir); + path = t_str_new(256); str_printfa(path, "%s/", dir); dir_len = str_len(path); @@ -54,19 +59,6 @@ } } -#ifdef HAVE_DIRFD - if (fstat(dirfd(dirp), &st) < 0) - i_error("fstat(%s) failed: %m", dir); -#else - if (stat(dir, &st) < 0) - i_error("stat(%s) failed: %m", dir); -#endif - else if (st.st_atime < ioloop_time) { - /* mounted with noatime. update it ourself. */ - if (utime(dir, NULL) < 0 && errno != ENOENT) - i_error("utime(%s) failed: %m", dir); - } - if (closedir(dirp) < 0) i_error("closedir(%s) failed: %m", dir); return 0; diff -r 98fd46f8d1ab -r 091b9f49f24f src/lib/unlink-old-files.h --- a/src/lib/unlink-old-files.h Fri Mar 23 13:24:34 2012 +0200 +++ b/src/lib/unlink-old-files.h Fri Mar 23 13:35:05 2012 +0200 @@ -2,7 +2,8 @@ #define UNLINK_OLD_FILES_H /* Unlink all files from directory beginning with given prefix and having - ctime older than min_time. Returns -1 if there were some errors. */ + ctime older than min_time. Makes sure that the directory's atime is updated. + Returns -1 if there were some errors. */ int unlink_old_files(const char *dir, const char *prefix, time_t min_time); #endif From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: Added mail_temp_scan_interval setting and changed i... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/370e1f945c01 changeset: 14379:370e1f945c01 user: Timo Sirainen date: Fri Mar 23 13:44:54 2012 +0200 description: Added mail_temp_scan_interval setting and changed its default from 8h -> 1w. diffstat: doc/example-config/conf.d/10-mail.conf | 4 ++++ src/lib-storage/index/dbox-common/dbox-storage.c | 7 +++++-- src/lib-storage/index/dbox-common/dbox-storage.h | 2 -- src/lib-storage/index/dbox-multi/mdbox-map.c | 8 ++++++-- src/lib-storage/index/maildir/maildir-storage.c | 7 +++++-- src/lib-storage/index/maildir/maildir-storage.h | 2 -- src/lib-storage/mail-storage-settings.c | 2 ++ src/lib-storage/mail-storage-settings.h | 1 + 8 files changed, 23 insertions(+), 10 deletions(-) diffs (144 lines): diff -r 091b9f49f24f -r 370e1f945c01 doc/example-config/conf.d/10-mail.conf --- a/doc/example-config/conf.d/10-mail.conf Fri Mar 23 13:35:05 2012 +0200 +++ b/doc/example-config/conf.d/10-mail.conf Fri Mar 23 13:44:54 2012 +0200 @@ -227,6 +227,10 @@ # some mailbox formats and/or operating systems. #mail_prefetch_count = 0 +# How often to scan for stale temporary files and delete them. These should +# exist only after Dovecot dies in the middle of saving mails. +#mail_temp_scan_interval = 1w + ## ## Maildir-specific settings ## diff -r 091b9f49f24f -r 370e1f945c01 src/lib-storage/index/dbox-common/dbox-storage.c --- a/src/lib-storage/index/dbox-common/dbox-storage.c Fri Mar 23 13:35:05 2012 +0200 +++ b/src/lib-storage/index/dbox-common/dbox-storage.c Fri Mar 23 13:44:54 2012 +0200 @@ -147,15 +147,18 @@ dbox_cleanup_if_exists(struct mailbox_list *list, const char *path) { struct stat st; + unsigned int interval = list->mail_set->mail_temp_scan_interval; if (stat(path, &st) < 0) return FALSE; /* check once in a while if there are temp files to clean up */ - if (st.st_atime > st.st_ctime + DBOX_TMP_DELETE_SECS) { + if (interval == 0) { + /* disabled */ + } else if (st.st_atime > st.st_ctime + DBOX_TMP_DELETE_SECS) { /* there haven't been any changes to this directory since we last checked it. */ - } else if (st.st_atime < ioloop_time - DBOX_TMP_SCAN_SECS) { + } else if (st.st_atime < ioloop_time - interval) { /* time to scan */ const char *prefix = mailbox_list_get_global_temp_prefix(list); diff -r 091b9f49f24f -r 370e1f945c01 src/lib-storage/index/dbox-common/dbox-storage.h --- a/src/lib-storage/index/dbox-common/dbox-storage.h Fri Mar 23 13:35:05 2012 +0200 +++ b/src/lib-storage/index/dbox-common/dbox-storage.h Fri Mar 23 13:44:54 2012 +0200 @@ -18,8 +18,6 @@ #define DBOX_TRASH_DIR_NAME "trash" #define DBOX_MAILDIR_NAME "dbox-Mails" -/* How often to scan for stale temp files (based on dir's atime) */ -#define DBOX_TMP_SCAN_SECS (8*60*60) /* Delete temp files having ctime older than this. */ #define DBOX_TMP_DELETE_SECS (36*60*60) diff -r 091b9f49f24f -r 370e1f945c01 src/lib-storage/index/dbox-multi/mdbox-map.c --- a/src/lib-storage/index/dbox-multi/mdbox-map.c Fri Mar 23 13:35:05 2012 +0200 +++ b/src/lib-storage/index/dbox-multi/mdbox-map.c Fri Mar 23 13:44:54 2012 +0200 @@ -133,16 +133,20 @@ static void mdbox_map_cleanup(struct mdbox_map *map) { + unsigned int interval = + MAP_STORAGE(map)->set->mail_temp_scan_interval; struct stat st; if (stat(map->path, &st) < 0) return; /* check once in a while if there are temp files to clean up */ - if (st.st_atime > st.st_ctime + DBOX_TMP_DELETE_SECS) { + if (interval == 0) { + /* disabled */ + } else if (st.st_atime > st.st_ctime + DBOX_TMP_DELETE_SECS) { /* there haven't been any changes to this directory since we last checked it. */ - } else if (st.st_atime < ioloop_time - DBOX_TMP_SCAN_SECS) { + } else if (st.st_atime < ioloop_time - interval) { /* time to scan */ (void)unlink_old_files(map->path, DBOX_TEMP_FILE_PREFIX, ioloop_time - DBOX_TMP_DELETE_SECS); diff -r 091b9f49f24f -r 370e1f945c01 src/lib-storage/index/maildir/maildir-storage.c --- a/src/lib-storage/index/maildir/maildir-storage.c Fri Mar 23 13:35:05 2012 +0200 +++ b/src/lib-storage/index/maildir/maildir-storage.c Fri Mar 23 13:44:54 2012 +0200 @@ -201,6 +201,7 @@ static int maildir_check_tmp(struct mail_storage *storage, const char *dir) { + unsigned int interval = storage->set->mail_temp_scan_interval; const char *path; struct stat st; @@ -218,10 +219,12 @@ return -1; } - if (st.st_atime > st.st_ctime + MAILDIR_TMP_DELETE_SECS) { + if (interval == 0) { + /* disabled */ + } else if (st.st_atime > st.st_ctime + MAILDIR_TMP_DELETE_SECS) { /* the directory should be empty. we won't do anything until ctime changes. */ - } else if (st.st_atime < ioloop_time - MAILDIR_TMP_SCAN_SECS) { + } else if (st.st_atime < ioloop_time - interval) { /* time to scan */ (void)unlink_old_files(path, "", ioloop_time - MAILDIR_TMP_DELETE_SECS); diff -r 091b9f49f24f -r 370e1f945c01 src/lib-storage/index/maildir/maildir-storage.h --- a/src/lib-storage/index/maildir/maildir-storage.h Fri Mar 23 13:35:05 2012 +0200 +++ b/src/lib-storage/index/maildir/maildir-storage.h Fri Mar 23 13:44:54 2012 +0200 @@ -31,8 +31,6 @@ calculating file's virtual size (added missing CRs). */ #define MAILDIR_EXTRA_VIRTUAL_SIZE 'W' -/* How often to scan tmp/ directory for old files (based on dir's atime) */ -#define MAILDIR_TMP_SCAN_SECS (8*60*60) /* Delete files having ctime older than this from tmp/. 36h is standard. */ #define MAILDIR_TMP_DELETE_SECS (36*60*60) diff -r 091b9f49f24f -r 370e1f945c01 src/lib-storage/mail-storage-settings.c --- a/src/lib-storage/mail-storage-settings.c Fri Mar 23 13:35:05 2012 +0200 +++ b/src/lib-storage/mail-storage-settings.c Fri Mar 23 13:44:54 2012 +0200 @@ -37,6 +37,7 @@ DEF(SET_TIME, mailbox_idle_check_interval), DEF(SET_UINT, mail_max_keyword_length), DEF(SET_TIME, mail_max_lock_timeout), + DEF(SET_TIME, mail_temp_scan_interval), DEF(SET_BOOL, mail_save_crlf), DEF(SET_ENUM, mail_fsync), DEF(SET_BOOL, mmap_disable), @@ -66,6 +67,7 @@ .mailbox_idle_check_interval = 30, .mail_max_keyword_length = 50, .mail_max_lock_timeout = 0, + .mail_temp_scan_interval = 7*24*60*60, .mail_save_crlf = FALSE, .mail_fsync = "optimized:never:always", .mmap_disable = FALSE, diff -r 091b9f49f24f -r 370e1f945c01 src/lib-storage/mail-storage-settings.h --- a/src/lib-storage/mail-storage-settings.h Fri Mar 23 13:35:05 2012 +0200 +++ b/src/lib-storage/mail-storage-settings.h Fri Mar 23 13:44:54 2012 +0200 @@ -22,6 +22,7 @@ unsigned int mailbox_idle_check_interval; unsigned int mail_max_keyword_length; unsigned int mail_max_lock_timeout; + unsigned int mail_temp_scan_interval; bool mail_save_crlf; const char *mail_fsync; bool mmap_disable; From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: mail_temp_scan_interval comment update. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/24b3de4952b5 changeset: 14380:24b3de4952b5 user: Timo Sirainen date: Fri Mar 23 13:46:27 2012 +0200 description: mail_temp_scan_interval comment update. diffstat: doc/example-config/conf.d/10-mail.conf | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff -r 370e1f945c01 -r 24b3de4952b5 doc/example-config/conf.d/10-mail.conf --- a/doc/example-config/conf.d/10-mail.conf Fri Mar 23 13:44:54 2012 +0200 +++ b/doc/example-config/conf.d/10-mail.conf Fri Mar 23 13:46:27 2012 +0200 @@ -227,8 +227,8 @@ # some mailbox formats and/or operating systems. #mail_prefetch_count = 0 -# How often to scan for stale temporary files and delete them. These should -# exist only after Dovecot dies in the middle of saving mails. +# How often to scan for stale temporary files and delete them (0 = never). +# These should exist only after Dovecot dies in the middle of saving mails. #mail_temp_scan_interval = 1w ## From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: Fixed compiling without SSL. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/f369e76d0867 changeset: 14381:f369e76d0867 user: Timo Sirainen date: Fri Mar 23 13:59:52 2012 +0200 description: Fixed compiling without SSL. diffstat: src/login-common/ssl-proxy.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (19 lines): diff -r 24b3de4952b5 -r f369e76d0867 src/login-common/ssl-proxy.c --- a/src/login-common/ssl-proxy.c Fri Mar 23 13:46:27 2012 +0200 +++ b/src/login-common/ssl-proxy.c Fri Mar 23 13:59:52 2012 +0200 @@ -10,6 +10,7 @@ /* no SSL support */ int ssl_proxy_alloc(int fd ATTR_UNUSED, const struct ip_addr *ip ATTR_UNUSED, + pool_t set_pool ATTR_UNUSED, const struct login_settings *set ATTR_UNUSED, struct ssl_proxy **proxy_r ATTR_UNUSED) { @@ -18,6 +19,7 @@ } int ssl_proxy_client_alloc(int fd ATTR_UNUSED, struct ip_addr *ip ATTR_UNUSED, + pool_t set_pool ATTR_UNUSED, const struct login_settings *set ATTR_UNUSED, ssl_handshake_callback_t *callback ATTR_UNUSED, void *context ATTR_UNUSED, From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: Compiler warning fixes. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/4ae85f573c93 changeset: 14382:4ae85f573c93 user: Timo Sirainen date: Fri Mar 23 14:37:22 2012 +0200 description: Compiler warning fixes. diffstat: src/auth/auth-request.h | 2 +- src/lib-imap/test-imap-utf7.c | 2 +- src/lib-storage/index/imapc/imapc-storage.h | 2 +- src/lib-test/test-common.c | 2 +- src/lib/strfuncs.h | 2 +- src/master/main.c | 4 ++-- 6 files changed, 7 insertions(+), 7 deletions(-) diffs (81 lines): diff -r f369e76d0867 -r 4ae85f573c93 src/auth/auth-request.h --- a/src/auth/auth-request.h Fri Mar 23 13:59:52 2012 +0200 +++ b/src/auth/auth-request.h Fri Mar 23 14:37:22 2012 +0200 @@ -212,7 +212,7 @@ const char *format, ...) ATTR_FORMAT(3, 4); void auth_request_log_warning(struct auth_request *auth_request, const char *subsystem, - const char *format, ...); + const char *format, ...) ATTR_FORMAT(3, 4); void auth_request_log_error(struct auth_request *auth_request, const char *subsystem, const char *format, ...) ATTR_FORMAT(3, 4); diff -r f369e76d0867 -r 4ae85f573c93 src/lib-imap/test-imap-utf7.c --- a/src/lib-imap/test-imap-utf7.c Fri Mar 23 13:59:52 2012 +0200 +++ b/src/lib-imap/test-imap-utf7.c Fri Mar 23 14:37:22 2012 +0200 @@ -12,7 +12,7 @@ "&&x&&", "&-&-x&-&-", "~peter/mail/??????/?????????", "~peter/mail/&U,BTFw-/&ZeVnLIqe-", "tiet??j??", "tiet&AOQ-j&AOQ-", - "p??", NULL, + "p\xe4\xe4", NULL, NULL }; static const char *invalid_utf7[] = { diff -r f369e76d0867 -r 4ae85f573c93 src/lib-storage/index/imapc/imapc-storage.h --- a/src/lib-storage/index/imapc/imapc-storage.h Fri Mar 23 13:59:52 2012 +0200 +++ b/src/lib-storage/index/imapc/imapc-storage.h Fri Mar 23 14:37:22 2012 +0200 @@ -124,7 +124,7 @@ bool *changes_r); void imapc_mailbox_noop(struct imapc_mailbox *mbox); void imapc_mailbox_set_corrupted(struct imapc_mailbox *mbox, - const char *reason, ...); + const char *reason, ...) ATTR_FORMAT(2, 3); void imapc_storage_register_untagged(struct imapc_storage *storage, const char *name, diff -r f369e76d0867 -r 4ae85f573c93 src/lib-test/test-common.c --- a/src/lib-test/test-common.c Fri Mar 23 13:59:52 2012 +0200 +++ b/src/lib-test/test-common.c Fri Mar 23 14:37:22 2012 +0200 @@ -185,7 +185,7 @@ total_count++; } -static void +static void ATTR_FORMAT(2, 0) test_error_handler(const struct failure_context *ctx, const char *format, va_list args) { diff -r f369e76d0867 -r 4ae85f573c93 src/lib/strfuncs.h --- a/src/lib/strfuncs.h Fri Mar 23 13:59:52 2012 +0200 +++ b/src/lib/strfuncs.h Fri Mar 23 14:37:22 2012 +0200 @@ -86,7 +86,7 @@ /* INTERNAL */ char *t_noalloc_strdup_vprintf(const char *format, va_list args, - unsigned int *size_r); + unsigned int *size_r) ATTR_FORMAT(1, 0); char *vstrconcat(const char *str1, va_list args, size_t *ret_len) ATTR_MALLOC; #endif diff -r f369e76d0867 -r 4ae85f573c93 src/master/main.c --- a/src/master/main.c Fri Mar 23 13:59:52 2012 +0200 +++ b/src/master/main.c Fri Mar 23 14:37:22 2012 +0200 @@ -164,7 +164,7 @@ abort(); /* just to silence the noreturn attribute warnings */ } -static void ATTR_NORETURN +static void ATTR_NORETURN ATTR_FORMAT(2, 0) startup_fatal_handler(const struct failure_context *ctx, const char *fmt, va_list args) { @@ -177,7 +177,7 @@ abort(); } -static void +static void ATTR_FORMAT(2, 0) startup_error_handler(const struct failure_context *ctx, const char *fmt, va_list args) { From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: Compiling fix for building without SSL. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/8699f3d3c1e4 changeset: 14383:8699f3d3c1e4 user: Timo Sirainen date: Sat Mar 24 14:21:59 2012 +0200 description: Compiling fix for building without SSL. diffstat: src/login-common/ssl-proxy.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diffs (14 lines): diff -r 4ae85f573c93 -r 8699f3d3c1e4 src/login-common/ssl-proxy.c --- a/src/login-common/ssl-proxy.c Fri Mar 23 14:37:22 2012 +0200 +++ b/src/login-common/ssl-proxy.c Sat Mar 24 14:21:59 2012 +0200 @@ -38,6 +38,10 @@ { } +void ssl_proxy_unset_client(struct ssl_proxy *proxy ATTR_UNUSED) +{ +} + bool ssl_proxy_has_valid_client_cert(const struct ssl_proxy *proxy ATTR_UNUSED) { return FALSE; From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: data-stack: Fixed calling t_push()/t_malloc() befor... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/e79496bb09f5 changeset: 14384:e79496bb09f5 user: Timo Sirainen date: Mon Mar 26 15:34:46 2012 +0300 description: data-stack: Fixed calling t_push()/t_malloc() before data_stack_init(). diffstat: src/lib/data-stack.c | 36 ++++++++++++++++++++---------------- 1 files changed, 20 insertions(+), 16 deletions(-) diffs (58 lines): diff -r 8699f3d3c1e4 -r e79496bb09f5 src/lib/data-stack.c --- a/src/lib/data-stack.c Sat Mar 24 14:21:59 2012 +0200 +++ b/src/lib/data-stack.c Mon Mar 26 15:34:46 2012 +0300 @@ -64,7 +64,11 @@ static struct stack_block *last_buffer_block; static size_t last_buffer_size; +#ifdef DEBUG +static bool clean_after_pop = TRUE; +#else static bool clean_after_pop = FALSE; +#endif static bool outofmem = FALSE; static union { @@ -490,26 +494,26 @@ void data_stack_init(void) { -#ifdef DEBUG - clean_after_pop = TRUE; -#endif - if (data_stack_frame == 0) { - data_stack_frame = 1; + if (data_stack_frame > 0) { + /* already initialized (we did auto-initialization in + t_malloc/t_push) */ + return; + } + data_stack_frame = 1; - outofmem_area.block.size = outofmem_area.block.left = - sizeof(outofmem_area) - sizeof(outofmem_area.block); + outofmem_area.block.size = outofmem_area.block.left = + sizeof(outofmem_area) - sizeof(outofmem_area.block); - current_block = mem_block_alloc(INITIAL_STACK_SIZE); - current_block->left = current_block->size; - current_block->next = NULL; + current_block = mem_block_alloc(INITIAL_STACK_SIZE); + current_block->left = current_block->size; + current_block->next = NULL; - current_frame_block = NULL; - unused_frame_blocks = NULL; - frame_pos = BLOCK_FRAME_COUNT-1; + current_frame_block = NULL; + unused_frame_blocks = NULL; + frame_pos = BLOCK_FRAME_COUNT-1; - last_buffer_block = NULL; - last_buffer_size = 0; - } + last_buffer_block = NULL; + last_buffer_size = 0; t_push(); } From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: lib-storage: If trying to copy a message into alrea... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/8897f32939b0 changeset: 14385:8897f32939b0 user: Timo Sirainen date: Mon Mar 26 16:23:18 2012 +0300 description: lib-storage: If trying to copy a message into already deleted mailbox, don't crash. diffstat: src/lib-storage/mail-storage.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r e79496bb09f5 -r 8897f32939b0 src/lib-storage/mail-storage.c --- a/src/lib-storage/mail-storage.c Mon Mar 26 15:34:46 2012 +0300 +++ b/src/lib-storage/mail-storage.c Mon Mar 26 16:23:18 2012 +0300 @@ -1667,7 +1667,7 @@ if (mail_index_is_deleted(box->index)) { mailbox_set_deleted(box); - mailbox_save_cancel(_ctx); + mailbox_save_cancel(&ctx); return -1; } From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: net_listen(): If bind(ip=any) fails with non-EADDRI... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/cd7e15b2d575 changeset: 14387:cd7e15b2d575 user: Timo Sirainen date: Mon Mar 26 17:01:21 2012 +0300 description: net_listen(): If bind(ip=any) fails with non-EADDRINUSE, don't crash. diffstat: src/lib/network.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 3980c025805c -r cd7e15b2d575 src/lib/network.c --- a/src/lib/network.c Mon Mar 26 16:57:32 2012 +0300 +++ b/src/lib/network.c Mon Mar 26 17:01:21 2012 +0300 @@ -408,7 +408,7 @@ if (ret < 0) { if (errno != EADDRINUSE) { i_error("bind(%s, %u) failed: %m", - net_ip2addr(my_ip), *port); + my_ip == NULL ? "" : net_ip2addr(my_ip), *port); } } else { /* get the actual port we started listen */ From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: stats: When freeing memory, make sure we don't cras... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/3980c025805c changeset: 14386:3980c025805c user: Timo Sirainen date: Mon Mar 26 16:57:32 2012 +0300 description: stats: When freeing memory, make sure we don't crash if some list gets empty. diffstat: src/stats/mail-command.c | 3 ++- src/stats/mail-domain.c | 3 ++- src/stats/mail-ip.c | 3 ++- src/stats/mail-session.c | 3 ++- src/stats/mail-user.c | 3 ++- 5 files changed, 10 insertions(+), 5 deletions(-) diffs (65 lines): diff -r 8897f32939b0 -r 3980c025805c src/stats/mail-command.c --- a/src/stats/mail-command.c Mon Mar 26 16:23:18 2012 +0300 +++ b/src/stats/mail-command.c Mon Mar 26 16:57:32 2012 +0300 @@ -206,7 +206,8 @@ } mail_command_free(stable_mail_commands_head); - if (global_used_memory < stats_settings->memory_limit) + if (global_used_memory < stats_settings->memory_limit || + stable_mail_commands_head == NULL) break; diff = ioloop_time - stable_mail_commands_head->last_update.tv_sec; diff -r 8897f32939b0 -r 3980c025805c src/stats/mail-domain.c --- a/src/stats/mail-domain.c Mon Mar 26 16:23:18 2012 +0300 +++ b/src/stats/mail-domain.c Mon Mar 26 16:57:32 2012 +0300 @@ -100,7 +100,8 @@ while (mail_domains_head != NULL && mail_domains_head->refcount == 0) { mail_domain_free(mail_domains_head); - if (global_used_memory < stats_settings->memory_limit) + if (global_used_memory < stats_settings->memory_limit || + mail_domains_head == NULL) break; diff = ioloop_time - mail_domains_head->last_update.tv_sec; diff -r 8897f32939b0 -r 3980c025805c src/stats/mail-ip.c --- a/src/stats/mail-ip.c Mon Mar 26 16:23:18 2012 +0300 +++ b/src/stats/mail-ip.c Mon Mar 26 16:57:32 2012 +0300 @@ -96,7 +96,8 @@ while (mail_ips_head != NULL && mail_ips_head->refcount == 0) { mail_ip_free(mail_ips_head); - if (global_used_memory < stats_settings->memory_limit) + if (global_used_memory < stats_settings->memory_limit || + mail_ips_head == NULL) break; diff = ioloop_time - mail_ips_head->last_update.tv_sec; diff -r 8897f32939b0 -r 3980c025805c src/stats/mail-session.c --- a/src/stats/mail-session.c Mon Mar 26 16:23:18 2012 +0300 +++ b/src/stats/mail-session.c Mon Mar 26 16:57:32 2012 +0300 @@ -277,7 +277,8 @@ i_assert(mail_sessions_head->disconnected); mail_session_free(mail_sessions_head); - if (global_used_memory < stats_settings->memory_limit) + if (global_used_memory < stats_settings->memory_limit || + mail_sessions_head == NULL) break; diff = ioloop_time - mail_sessions_head->last_update.tv_sec; diff -r 8897f32939b0 -r 3980c025805c src/stats/mail-user.c --- a/src/stats/mail-user.c Mon Mar 26 16:23:18 2012 +0300 +++ b/src/stats/mail-user.c Mon Mar 26 16:57:32 2012 +0300 @@ -118,7 +118,8 @@ while (mail_users_head != NULL && mail_users_head->refcount == 0) { mail_user_free(mail_users_head); - if (global_used_memory < stats_settings->memory_limit) + if (global_used_memory < stats_settings->memory_limit || + mail_users_head == NULL) break; diff = ioloop_time - mail_users_head->last_update.tv_sec; From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: director: Avoid crashing if all directors are remov... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/a71bc8dbe53d changeset: 14388:a71bc8dbe53d user: Timo Sirainen date: Mon Mar 26 17:24:59 2012 +0300 description: director: Avoid crashing if all directors are removed when reconnecting. There's no way to currently remove directors though, so this couldn't have actually happened. diffstat: src/director/director.c | 8 ++++++-- 1 files changed, 6 insertions(+), 2 deletions(-) diffs (26 lines): diff -r cd7e15b2d575 -r a71bc8dbe53d src/director/director.c --- a/src/director/director.c Mon Mar 26 17:01:21 2012 +0300 +++ b/src/director/director.c Mon Mar 26 17:24:59 2012 +0300 @@ -115,8 +115,10 @@ unsigned int count, self_idx; hosts = array_get(&dir->dir_hosts, &count); - if (count == 1) + if (count == 1) { + /* self */ return NULL; + } self_idx = director_find_self_idx(dir); return hosts[(self_idx + 1) % count]; @@ -189,7 +191,9 @@ cur_host = dir->right == NULL ? NULL : director_connection_get_host(dir->right); - if (cur_host != preferred_host) + if (preferred_host == NULL) { + /* all directors have been removed, try again later */ + } else if (cur_host != preferred_host) (void)director_connect_host(dir, preferred_host); else { /* the connection hasn't finished sync yet. From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: pgsql: Fixed a potential crash if connection got cl... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/e05be9afaed0 changeset: 14390:e05be9afaed0 user: Timo Sirainen date: Mon Mar 26 20:38:18 2012 +0300 description: pgsql: Fixed a potential crash if connection got closed during synchronous query. diffstat: src/lib-sql/driver-pgsql.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diffs (13 lines): diff -r bcc5e71650b9 -r e05be9afaed0 src/lib-sql/driver-pgsql.c --- a/src/lib-sql/driver-pgsql.c Mon Mar 26 18:17:52 2012 +0300 +++ b/src/lib-sql/driver-pgsql.c Mon Mar 26 20:38:18 2012 +0300 @@ -603,6 +603,9 @@ /* we don't end up in pgsql's free function, so sync_result won't be set to NULL if we don't do it here. */ db->sync_result = NULL; + } else if (result == NULL) { + result = &sql_not_connected_result; + result->refcount++; } i_assert(db->io == NULL); From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: fts-solr: Indexing mail bodies was broken. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/bcc5e71650b9 changeset: 14389:bcc5e71650b9 user: Timo Sirainen date: Mon Mar 26 18:17:52 2012 +0300 description: fts-solr: Indexing mail bodies was broken. diffstat: src/plugins/fts-solr/fts-backend-solr.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff -r a71bc8dbe53d -r bcc5e71650b9 src/plugins/fts-solr/fts-backend-solr.c --- a/src/plugins/fts-solr/fts-backend-solr.c Mon Mar 26 17:24:59 2012 +0300 +++ b/src/plugins/fts-solr/fts-backend-solr.c Mon Mar 26 18:17:52 2012 +0300 @@ -514,6 +514,7 @@ data += len; size -= len; } + xml_encode_data(ctx->cur_value, data, size); } else { xml_encode_data(ctx->cur_value, data, size); if (ctx->cur_value2 != NULL) From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: Changes to make static analyzer happier. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/ba1fc76e3a2c changeset: 14391:ba1fc76e3a2c user: Timo Sirainen date: Mon Mar 26 20:39:09 2012 +0300 description: Changes to make static analyzer happier. diffstat: src/config/config-filter.c | 7 +++- src/doveadm/doveadm-stats.c | 2 +- src/doveadm/dsync/doveadm-dsync.c | 4 +- src/doveadm/dsync/test-dsync-proxy-server-cmd.c | 2 + src/imap/imap-sync.c | 7 +++- src/lib-dict/dict-sql.c | 6 ++- src/lib-mail/message-parser.c | 29 ++++++++++++++------ src/lib-master/master-login.c | 6 ++- src/lib-storage/index/cydir/cydir-mail.c | 6 ++- src/lib-storage/index/maildir/maildir-mail.c | 14 +++++---- src/lib-storage/index/maildir/maildir-sync.c | 6 +++- src/lib-storage/list/subscription-file.c | 34 +++++++++++------------- src/lib/file-cache.c | 10 ++++++- src/lib/ostream.c | 3 +- src/lib/process-title.c | 2 + src/lib/strfuncs.c | 4 ++ src/lib/test-str-find.c | 3 +- src/plugins/expire/doveadm-expire.c | 2 + src/plugins/fts-squat/squat-trie.c | 1 + src/plugins/fts-squat/squat-uidlist.c | 3 ++ 20 files changed, 101 insertions(+), 50 deletions(-) diffs (truncated from 488 to 300 lines): diff -r e05be9afaed0 -r ba1fc76e3a2c src/config/config-filter.c --- a/src/config/config-filter.c Mon Mar 26 20:38:18 2012 +0300 +++ b/src/config/config-filter.c Mon Mar 26 20:39:09 2012 +0300 @@ -287,14 +287,17 @@ const struct config_filter_parser *src, pool_t pool, const char **error_r) { + const char *conflict_key; unsigned int i; for (i = 0; dest[i].root != NULL; i++) { if (settings_parser_apply_changes(dest[i].parser, src->parsers[i].parser, pool, - error_r) < 0) { + error_r == NULL ? NULL : + &conflict_key) < 0) { + i_assert(error_r != NULL); *error_r = t_strdup_printf("Conflict in setting %s " - "found from filter at %s", *error_r, + "found from filter at %s", conflict_key, src->file_and_line); return -1; } diff -r e05be9afaed0 -r ba1fc76e3a2c src/doveadm/doveadm-stats.c --- a/src/doveadm/doveadm-stats.c Mon Mar 26 20:38:18 2012 +0300 +++ b/src/doveadm/doveadm-stats.c Mon Mar 26 20:39:09 2012 +0300 @@ -102,7 +102,7 @@ do { T_BEGIN { args = read_next_line(input); - if (args[0] == NULL) + if (args != NULL && args[0] == NULL) args = NULL; if (args != NULL) { for (i = 0; args[i] != NULL; i++) diff -r e05be9afaed0 -r ba1fc76e3a2c src/doveadm/dsync/doveadm-dsync.c --- a/src/doveadm/dsync/doveadm-dsync.c Mon Mar 26 20:38:18 2012 +0300 +++ b/src/doveadm/dsync/doveadm-dsync.c Mon Mar 26 20:39:09 2012 +0300 @@ -655,7 +655,7 @@ /* @UNSAFE: this is called when the "doveadm" binary is called as "dsync" (for backwards compatibility) */ max_argc = argc + 7; - new_argv = calloc(sizeof(char *), max_argc); + new_argv = t_new(char *, max_argc); new_argv[0] = argv[0]; dest = 1; getopt_str = master_service_getopt_string(); @@ -666,7 +666,7 @@ break; flag_m = FALSE; flag_C = FALSE; has_arg = FALSE; flag_u = FALSE; - dup = strdup(argv[src]); + dup = t_strdup_noconst(argv[src]); for (i = j = 1; argv[src][i] != '\0'; i++) { switch (argv[src][i]) { case 'C': diff -r e05be9afaed0 -r ba1fc76e3a2c src/doveadm/dsync/test-dsync-proxy-server-cmd.c --- a/src/doveadm/dsync/test-dsync-proxy-server-cmd.c Mon Mar 26 20:38:18 2012 +0300 +++ b/src/doveadm/dsync/test-dsync-proxy-server-cmd.c Mon Mar 26 20:39:09 2012 +0300 @@ -30,6 +30,8 @@ { int ret; + i_assert(cur_cmd != NULL); + ret = cur_cmd->func(server, cur_cmd_args); if (ret == 0) return 0; diff -r e05be9afaed0 -r ba1fc76e3a2c src/imap/imap-sync.c --- a/src/imap/imap-sync.c Mon Mar 26 20:38:18 2012 +0300 +++ b/src/imap/imap-sync.c Mon Mar 26 20:39:09 2012 +0300 @@ -669,8 +669,11 @@ for (; cmd != NULL; cmd = prev) { prev = cmd->next; - if (cmd->state == CLIENT_COMMAND_STATE_WAIT_SYNC && - (cmd->sync->flags & MAILBOX_SYNC_FLAG_FAST) != 0) { + if (cmd->state != CLIENT_COMMAND_STATE_WAIT_SYNC) + continue; + + i_assert(cmd->sync != NULL); + if ((cmd->sync->flags & MAILBOX_SYNC_FLAG_FAST) != 0) { if (cmd_finish_sync(cmd)) { client_command_free(&cmd); ret = TRUE; diff -r e05be9afaed0 -r ba1fc76e3a2c src/lib-dict/dict-sql.c --- a/src/lib-dict/dict-sql.c Mon Mar 26 20:38:18 2012 +0300 +++ b/src/lib-dict/dict-sql.c Mon Mar 26 20:39:09 2012 +0300 @@ -164,14 +164,16 @@ return FALSE; } } + + *path_len_r = path - path_start; + *pat_len_r = pat - map->pattern; + if (*pat == '\0') return *path == '\0'; else if (!partial_ok) return FALSE; else { /* partial matches must end with '/' */ - *path_len_r = path - path_start; - *pat_len_r = pat - map->pattern; return pat == map->pattern || pat[-1] == '/'; } } diff -r e05be9afaed0 -r ba1fc76e3a2c src/lib-mail/message-parser.c --- a/src/lib-mail/message-parser.c Mon Mar 26 20:38:18 2012 +0300 +++ b/src/lib-mail/message-parser.c Mon Mar 26 20:39:09 2012 +0300 @@ -702,10 +702,10 @@ return preparsed_parse_next_header(ctx, block_r); } -struct message_parser_ctx * -message_parser_init(pool_t part_pool, struct istream *input, - enum message_header_parser_flags hdr_flags, - enum message_parser_flags flags) +static struct message_parser_ctx * +message_parser_init_int(struct istream *input, + enum message_header_parser_flags hdr_flags, + enum message_parser_flags flags) { struct message_parser_ctx *ctx; pool_t pool; @@ -713,14 +713,24 @@ pool = pool_alloconly_create("Message Parser", 1024); ctx = p_new(pool, struct message_parser_ctx, 1); ctx->parser_pool = pool; - ctx->part_pool = part_pool; ctx->hdr_flags = hdr_flags; ctx->flags = flags; ctx->input = input; - ctx->parts = ctx->part = part_pool == NULL ? NULL : - p_new(part_pool, struct message_part, 1); + i_stream_ref(input); + return ctx; +} + +struct message_parser_ctx * +message_parser_init(pool_t part_pool, struct istream *input, + enum message_header_parser_flags hdr_flags, + enum message_parser_flags flags) +{ + struct message_parser_ctx *ctx; + + ctx = message_parser_init_int(input, hdr_flags, flags); + ctx->part_pool = part_pool; + ctx->parts = ctx->part = p_new(part_pool, struct message_part, 1); ctx->parse_next_block = parse_next_header_init; - i_stream_ref(input); return ctx; } @@ -732,7 +742,7 @@ { struct message_parser_ctx *ctx; - ctx = message_parser_init(NULL, input, hdr_flags, flags); + ctx = message_parser_init_int(input, hdr_flags, flags); ctx->parts = ctx->part = parts; ctx->parse_next_block = preparsed_parse_next_header_init; return ctx; @@ -806,6 +816,7 @@ break; } i_assert(ret != 0); + i_assert(ctx->part != NULL); if (ret < 0) { /* well, can't return error so fake end of headers */ diff -r e05be9afaed0 -r ba1fc76e3a2c src/lib-master/master-login.c --- a/src/lib-master/master-login.c Mon Mar 26 20:38:18 2012 +0300 +++ b/src/lib-master/master-login.c Mon Mar 26 20:39:09 2012 +0300 @@ -187,8 +187,10 @@ /* FIXME: currently we create a separate connection for each request, so close the connection after we're done with this client */ - if (!master_login_conn_is_closed(client->conn)) - master_login_conn_unref(&client->conn); + if (!master_login_conn_is_closed(client->conn)) { + i_assert(client->conn->refcount > 1); + client->conn->refcount--; + } master_login_conn_unref(&client->conn); i_free(client); } diff -r e05be9afaed0 -r ba1fc76e3a2c src/lib-storage/index/cydir/cydir-mail.c --- a/src/lib-storage/index/cydir/cydir-mail.c Mon Mar 26 20:38:18 2012 +0300 +++ b/src/lib-storage/index/cydir/cydir-mail.c Mon Mar 26 20:39:09 2012 +0300 @@ -22,8 +22,10 @@ { const char *path; - if (mail->lookup_abort == MAIL_LOOKUP_ABORT_NOT_IN_CACHE) - return mail_set_aborted(mail); + if (mail->lookup_abort == MAIL_LOOKUP_ABORT_NOT_IN_CACHE) { + mail_set_aborted(mail); + return -1; + } mail->transaction->stats.stat_lookup_count++; path = cydir_mail_get_path(mail); diff -r e05be9afaed0 -r ba1fc76e3a2c src/lib-storage/index/maildir/maildir-mail.c --- a/src/lib-storage/index/maildir/maildir-mail.c Mon Mar 26 20:38:18 2012 +0300 +++ b/src/lib-storage/index/maildir/maildir-mail.c Mon Mar 26 20:39:09 2012 +0300 @@ -102,7 +102,7 @@ return input; } -static int maildir_mail_stat(struct mail *mail, struct stat *st) +static int maildir_mail_stat(struct mail *mail, struct stat *st_r) { struct maildir_mailbox *mbox = (struct maildir_mailbox *)mail->box; struct index_mail *imail = (struct index_mail *)mail; @@ -110,8 +110,10 @@ const char *path; int ret; - if (mail->lookup_abort == MAIL_LOOKUP_ABORT_NOT_IN_CACHE) - return mail_set_aborted(mail); + if (mail->lookup_abort == MAIL_LOOKUP_ABORT_NOT_IN_CACHE) { + mail_set_aborted(mail); + return -1; + } if (imail->data.access_part != 0 && imail->data.stream == NULL) { @@ -126,10 +128,10 @@ stp = i_stream_stat(imail->data.stream, FALSE); if (stp == NULL) return -1; - *st = *stp; + *st_r = *stp; } else if (!mail->saving) { mail->transaction->stats.stat_lookup_count++; - ret = maildir_file_do(mbox, mail->uid, do_stat, st); + ret = maildir_file_do(mbox, mail->uid, do_stat, st_r); if (ret <= 0) { if (ret == 0) mail_set_expunged(mail); @@ -138,7 +140,7 @@ } else { mail->transaction->stats.stat_lookup_count++; path = maildir_save_file_get_path(mail->transaction, mail->seq); - if (stat(path, st) < 0) { + if (stat(path, st_r) < 0) { mail_storage_set_critical(mail->box->storage, "stat(%s) failed: %m", path); return -1; diff -r e05be9afaed0 -r ba1fc76e3a2c src/lib-storage/index/maildir/maildir-sync.c --- a/src/lib-storage/index/maildir/maildir-sync.c Mon Mar 26 20:38:18 2012 +0300 +++ b/src/lib-storage/index/maildir/maildir-sync.c Mon Mar 26 20:39:09 2012 +0300 @@ -595,6 +595,8 @@ struct stat new_st, cur_st; bool refreshed = FALSE, check_new = FALSE, check_cur = FALSE; + *why_r = 0; + if (mbox->maildir_hdr.new_mtime == 0) { maildir_sync_get_header(mbox); if (mbox->maildir_hdr.new_mtime == 0) { @@ -710,6 +712,8 @@ enum mail_index_sync_flags flags = 0; bool undirty = (ctx->flags & MAILBOX_SYNC_FLAG_FULL_READ) != 0; + *why_r = 0; + if (maildir_sync_quick_check(mbox, undirty, ctx->new_dir, ctx->cur_dir, new_changed_r, cur_changed_r, why_r) < 0) return -1; @@ -752,7 +756,7 @@ enum maildir_uidlist_rec_flag flags; bool new_changed, cur_changed, lock_failure; const char *fname; - enum maildir_scan_why why = 0; + enum maildir_scan_why why; int ret; *lost_files_r = FALSE; diff -r e05be9afaed0 -r ba1fc76e3a2c src/lib-storage/list/subscription-file.c --- a/src/lib-storage/list/subscription-file.c Mon Mar 26 20:38:18 2012 +0300 +++ b/src/lib-storage/list/subscription-file.c Mon Mar 26 20:39:09 2012 +0300 @@ -57,8 +57,6 @@ const char *line; *failed_r = FALSE; - if (input == NULL) - return NULL; while ((line = i_stream_next_line(input)) == NULL) { switch (i_stream_read(input)) { @@ -138,24 +136,26 @@ return -1; } - input = fd_in == -1 ? NULL : - i_stream_create_fd(fd_in, list->mailbox_name_max_length+1, - TRUE); + found = FALSE; From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: lib-settings: Make static analyzer happier. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/94286d055872 changeset: 14392:94286d055872 user: Timo Sirainen date: Mon Mar 26 20:46:55 2012 +0300 description: lib-settings: Make static analyzer happier. diffstat: src/lib-settings/settings-parser.c | 11 +++++++---- 1 files changed, 7 insertions(+), 4 deletions(-) diffs (50 lines): diff -r ba1fc76e3a2c -r 94286d055872 src/lib-settings/settings-parser.c --- a/src/lib-settings/settings-parser.c Mon Mar 26 20:39:09 2012 +0300 +++ b/src/lib-settings/settings-parser.c Mon Mar 26 20:46:55 2012 +0300 @@ -773,7 +773,8 @@ } do { - if (link->info == &strlist_info) { + if (def == NULL) { + i_assert(link->info == &strlist_info); settings_parse_strlist(ctx, link, key, value); return 1; } @@ -804,6 +805,7 @@ return NULL; if (def == NULL) { /* strlist */ + i_assert(link->info == &strlist_info); return key; } @@ -823,7 +825,7 @@ if (!settings_find_key(ctx, key, &def, &link)) return NULL; - if (link->set_struct == NULL) + if (link->set_struct == NULL || def == NULL) return NULL; *type_r = def->type; @@ -839,7 +841,7 @@ if (!settings_find_key(ctx, key, &def, &link)) return FALSE; - if (link->change_struct == NULL) + if (link->change_struct == NULL || def == NULL) return FALSE; p = STRUCT_MEMBER_P(link->change_struct, def->offset); @@ -1159,8 +1161,9 @@ if (!settings_find_key(ctx, key, &def, &link)) return; - if (link->info == &strlist_info) { + if (def == NULL) { /* parent is strlist, no expansion needed */ + i_assert(link->info == &strlist_info); return; } From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: *-login: Previous SSL fixes were still broken with ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/339b1337aab0 changeset: 14393:339b1337aab0 user: Timo Sirainen date: Tue Mar 27 00:12:39 2012 +0300 description: *-login: Previous SSL fixes were still broken with service_count=1. diffstat: src/login-common/client-common.c | 2 -- src/login-common/ssl-proxy-openssl.c | 8 +++----- src/login-common/ssl-proxy.c | 4 ---- src/login-common/ssl-proxy.h | 1 - 4 files changed, 3 insertions(+), 12 deletions(-) diffs (65 lines): diff -r 94286d055872 -r 339b1337aab0 src/login-common/client-common.c --- a/src/login-common/client-common.c Mon Mar 26 20:46:55 2012 +0300 +++ b/src/login-common/client-common.c Tue Mar 27 00:12:39 2012 +0300 @@ -180,8 +180,6 @@ i_free_and_null(client->proxy_password); } - if (client->ssl_proxy != NULL) - ssl_proxy_unset_client(client->ssl_proxy); if (client->login_proxy != NULL) login_proxy_free(&client->login_proxy); client->v.destroy(client); diff -r 94286d055872 -r 339b1337aab0 src/login-common/ssl-proxy-openssl.c --- a/src/login-common/ssl-proxy-openssl.c Mon Mar 26 20:46:55 2012 +0300 +++ b/src/login-common/ssl-proxy-openssl.c Tue Mar 27 00:12:39 2012 +0300 @@ -659,14 +659,10 @@ { i_assert(proxy->client == NULL); + client_ref(client); proxy->client = client; } -void ssl_proxy_unset_client(struct ssl_proxy *proxy) -{ - proxy->client = NULL; -} - bool ssl_proxy_has_valid_client_cert(const struct ssl_proxy *proxy) { return proxy->cert_received && !proxy->cert_broken; @@ -800,6 +796,8 @@ (void)net_disconnect(proxy->fd_ssl); (void)net_disconnect(proxy->fd_plain); + if (proxy->client != NULL) + client_unref(&proxy->client); ssl_proxy_unref(proxy); } diff -r 94286d055872 -r 339b1337aab0 src/login-common/ssl-proxy.c --- a/src/login-common/ssl-proxy.c Mon Mar 26 20:46:55 2012 +0300 +++ b/src/login-common/ssl-proxy.c Tue Mar 27 00:12:39 2012 +0300 @@ -38,10 +38,6 @@ { } -void ssl_proxy_unset_client(struct ssl_proxy *proxy ATTR_UNUSED) -{ -} - bool ssl_proxy_has_valid_client_cert(const struct ssl_proxy *proxy ATTR_UNUSED) { return FALSE; diff -r 94286d055872 -r 339b1337aab0 src/login-common/ssl-proxy.h --- a/src/login-common/ssl-proxy.h Mon Mar 26 20:46:55 2012 +0300 +++ b/src/login-common/ssl-proxy.h Tue Mar 27 00:12:39 2012 +0300 @@ -22,7 +22,6 @@ struct ssl_proxy **proxy_r); void ssl_proxy_start(struct ssl_proxy *proxy); void ssl_proxy_set_client(struct ssl_proxy *proxy, struct client *client); -void ssl_proxy_unset_client(struct ssl_proxy *proxy); bool ssl_proxy_has_valid_client_cert(const struct ssl_proxy *proxy) ATTR_PURE; bool ssl_proxy_has_broken_client_cert(struct ssl_proxy *proxy); int ssl_proxy_cert_match_name(struct ssl_proxy *proxy, const char *verify_name); From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: fts-solr: Added assert + minor code cleanup Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/7b5b40f64439 changeset: 14394:7b5b40f64439 user: Timo Sirainen date: Thu Mar 29 01:58:35 2012 +0300 description: fts-solr: Added assert + minor code cleanup diffstat: src/plugins/fts-solr/fts-backend-solr.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (16 lines): diff -r 339b1337aab0 -r 7b5b40f64439 src/plugins/fts-solr/fts-backend-solr.c --- a/src/plugins/fts-solr/fts-backend-solr.c Tue Mar 27 00:12:39 2012 +0300 +++ b/src/plugins/fts-solr/fts-backend-solr.c Thu Mar 29 01:58:35 2012 +0300 @@ -511,10 +511,11 @@ SOLR_CMDBUF_FLUSH_SIZE - str_len(ctx->cmd)); i_assert(len > 0); + i_assert(len <= size); data += len; size -= len; } - xml_encode_data(ctx->cur_value, data, size); + xml_encode_data(ctx->cmd, data, size); } else { xml_encode_data(ctx->cur_value, data, size); if (ctx->cur_value2 != NULL) From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: dsync: Fixed handling messages without GUID. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/078697a32109 changeset: 14395:078697a32109 user: Timo Sirainen date: Fri Mar 30 03:07:12 2012 +0300 description: dsync: Fixed handling messages without GUID. diffstat: src/doveadm/dsync/dsync-brain-msgs.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff -r 7b5b40f64439 -r 078697a32109 src/doveadm/dsync/dsync-brain-msgs.c --- a/src/doveadm/dsync/dsync-brain-msgs.c Thu Mar 29 01:58:35 2012 +0300 +++ b/src/doveadm/dsync/dsync-brain-msgs.c Fri Mar 30 03:07:12 2012 +0300 @@ -58,6 +58,8 @@ if ((msg->flags & DSYNC_MAIL_FLAG_EXPUNGED) != 0) return; + if (*msg->guid == '\0') + return; inst = p_new(iter->sync->pool, struct dsync_brain_guid_instance, 1); inst->mailbox_idx = mailbox_idx; From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: layout=fs: Don't crash in iteration if there is no ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/b7dc140e6b4d changeset: 14396:b7dc140e6b4d user: Timo Sirainen date: Fri Mar 30 03:42:21 2012 +0300 description: layout=fs: Don't crash in iteration if there is no root dir (pop3c). diffstat: src/lib-storage/list/mailbox-list-fs-iter.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diffs (14 lines): diff -r 078697a32109 -r b7dc140e6b4d src/lib-storage/list/mailbox-list-fs-iter.c --- a/src/lib-storage/list/mailbox-list-fs-iter.c Fri Mar 30 03:07:12 2012 +0300 +++ b/src/lib-storage/list/mailbox-list-fs-iter.c Fri Mar 30 03:42:21 2012 +0300 @@ -234,6 +234,10 @@ if (!fs_list_get_storage_path(ctx, dir->storage_name, &path)) return 0; + if (path == NULL) { + /* no mailbox root dir */ + return 0; + } fsdir = opendir(path); if (fsdir == NULL) { From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: pop3c: Allow accessing via INBOX, regardless of wha... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/8a94981d8040 changeset: 14397:8a94981d8040 user: Timo Sirainen date: Fri Mar 30 03:43:08 2012 +0300 description: pop3c: Allow accessing via INBOX, regardless of what namespace it exists in. diffstat: src/lib-storage/index/pop3c/pop3c-storage.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r b7dc140e6b4d -r 8a94981d8040 src/lib-storage/index/pop3c/pop3c-storage.c --- a/src/lib-storage/index/pop3c/pop3c-storage.c Fri Mar 30 03:42:21 2012 +0300 +++ b/src/lib-storage/index/pop3c/pop3c-storage.c Fri Mar 30 03:43:08 2012 +0300 @@ -162,7 +162,7 @@ { struct pop3c_mailbox *mbox = (struct pop3c_mailbox *)box; - if (!box->inbox_any) { + if (strcmp(box->name, "INBOX") != 0) { mail_storage_set_error(box->storage, MAIL_ERROR_NOTFOUND, T_MAIL_ERR_MAILBOX_NOT_FOUND(box->name)); return -1; From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: Added pop3-migration plugin for getting POP3 UIDLs ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/78317179b4af changeset: 14398:78317179b4af user: Timo Sirainen date: Fri Mar 30 03:46:37 2012 +0300 description: Added pop3-migration plugin for getting POP3 UIDLs from POP3 server. The idea is to use this with dsync to migrate mails via imapc, but for getting POP3 UIDLs via pop3c. diffstat: configure.in | 1 + src/plugins/Makefile.am | 1 + src/plugins/pop3-migration/Makefile.am | 17 + src/plugins/pop3-migration/pop3-migration-plugin.c | 595 +++++++++++++++++++++ src/plugins/pop3-migration/pop3-migration-plugin.h | 7 + 5 files changed, 621 insertions(+), 0 deletions(-) diffs (truncated from 653 to 300 lines): diff -r 8a94981d8040 -r 78317179b4af configure.in --- a/configure.in Fri Mar 30 03:43:08 2012 +0300 +++ b/configure.in Fri Mar 30 03:46:37 2012 +0300 @@ -2809,6 +2809,7 @@ src/plugins/listescape/Makefile src/plugins/mail-log/Makefile src/plugins/notify/Makefile +src/plugins/pop3-migration/Makefile src/plugins/quota/Makefile src/plugins/imap-quota/Makefile src/plugins/replication/Makefile diff -r 8a94981d8040 -r 78317179b4af src/plugins/Makefile.am --- a/src/plugins/Makefile.am Fri Mar 30 03:43:08 2012 +0300 +++ b/src/plugins/Makefile.am Fri Mar 30 03:46:37 2012 +0300 @@ -23,6 +23,7 @@ mail-log \ quota \ imap-quota \ + pop3-migration \ replication \ snarf \ stats \ diff -r 8a94981d8040 -r 78317179b4af src/plugins/pop3-migration/Makefile.am --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/plugins/pop3-migration/Makefile.am Fri Mar 30 03:46:37 2012 +0300 @@ -0,0 +1,17 @@ +AM_CPPFLAGS = \ + -I$(top_srcdir)/src/lib \ + -I$(top_srcdir)/src/lib-mail \ + -I$(top_srcdir)/src/lib-index \ + -I$(top_srcdir)/src/lib-storage + +NOPLUGIN_LDFLAGS = +lib05_pop3_migration_plugin_la_LDFLAGS = -module -avoid-version + +module_LTLIBRARIES = \ + lib05_pop3_migration_plugin.la + +lib05_pop3_migration_plugin_la_SOURCES = \ + pop3-migration-plugin.c + +noinst_HEADERS = \ + pop3-migration-plugin.h diff -r 8a94981d8040 -r 78317179b4af src/plugins/pop3-migration/pop3-migration-plugin.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/plugins/pop3-migration/pop3-migration-plugin.c Fri Mar 30 03:46:37 2012 +0300 @@ -0,0 +1,595 @@ +/* Copyright (c) 2007-2012 Dovecot authors, see the included COPYING file */ + +#include "lib.h" +#include "array.h" +#include "istream.h" +#include "istream-header-filter.h" +#include "sha1.h" +#include "mail-namespace.h" +#include "mail-search-build.h" +#include "mail-storage-private.h" +#include "pop3-migration-plugin.h" + +#define POP3_MIGRATION_CONTEXT(obj) \ + MODULE_CONTEXT(obj, pop3_migration_storage_module) +#define POP3_MIGRATION_MAIL_CONTEXT(obj) \ + MODULE_CONTEXT(obj, pop3_migration_mail_module) + +struct pop3_uidl_map { + uint32_t pop3_seq; + + /* UIDL */ + const char *pop3_uidl; + /* LIST size */ + uoff_t size; + /* sha1(TOP 0) - set only when needed */ + unsigned char hdr_sha1[SHA1_RESULTLEN]; + unsigned int hdr_sha1_set:1; + unsigned int imap_uid_found:1; +}; + +struct imap_msg_map { + uint32_t uid; + uoff_t psize; + const char *pop3_uidl; + + /* sha1(header) - set only when needed */ + unsigned char hdr_sha1[SHA1_RESULTLEN]; + unsigned int hdr_sha1_set:1; +}; + +struct pop3_migration_mail_storage { + union mail_storage_module_context module_ctx; + + const char *pop3_box_vname; + struct mailbox *pop3_box; + ARRAY_DEFINE(pop3_uidl_map, struct pop3_uidl_map); + + unsigned int all_mailboxes:1; + unsigned int pop3_all_hdr_sha1_set:1; +}; + +struct pop3_migration_mailbox { + union mailbox_module_context module_ctx; + + ARRAY_DEFINE(imap_msg_map, struct imap_msg_map); + unsigned int first_unfound_idx; + + unsigned int uidl_synced:1; + unsigned int uidl_sync_failed:1; +}; + +static const char *hdr_hash_skip_headers[] = { + "Content-Length", + "Status", + "X-IMAP", + "X-IMAPbase", + "X-Keywords", + "X-Message-Flag", + "X-Status", + "X-UID", + "X-UIDL" +}; +const char *pop3_migration_plugin_version = DOVECOT_VERSION; + +static MODULE_CONTEXT_DEFINE_INIT(pop3_migration_storage_module, + &mail_storage_module_register); +static MODULE_CONTEXT_DEFINE_INIT(pop3_migration_mail_module, + &mail_module_register); + +static int imap_msg_map_uid_cmp(const struct imap_msg_map *map1, + const struct imap_msg_map *map2) +{ + if (map1->uid < map2->uid) + return -1; + if (map1->uid > map2->uid) + return 1; + return 0; +} + +static int pop3_uidl_map_pop3_seq_cmp(const struct pop3_uidl_map *map1, + const struct pop3_uidl_map *map2) +{ + if (map1->pop3_seq < map2->pop3_seq) + return -1; + if (map1->pop3_seq > map2->pop3_seq) + return 1; + return 0; +} + +static int pop3_uidl_map_hdr_cmp(const struct pop3_uidl_map *map1, + const struct pop3_uidl_map *map2) +{ + return memcmp(map1->hdr_sha1, map2->hdr_sha1, sizeof(map1->hdr_sha1)); +} + +static int imap_msg_map_hdr_cmp(const struct imap_msg_map *map1, + const struct imap_msg_map *map2) +{ + return memcmp(map1->hdr_sha1, map2->hdr_sha1, sizeof(map1->hdr_sha1)); +} + +static int get_hdr_sha1(struct mail *mail, unsigned char sha1[SHA1_RESULTLEN]) +{ + struct istream *input; + const unsigned char *data; + size_t size; + struct sha1_ctxt sha1_ctx; + + if (mail_get_hdr_stream(mail, NULL, &input) < 0) { + i_error("pop3_migration: Failed to get header for msg %u", + mail->seq); + return -1; + } + /* hide headers that might change or be different in IMAP vs. POP3 */ + input = i_stream_create_header_filter(input, + HEADER_FILTER_EXCLUDE | HEADER_FILTER_NO_CR, + hdr_hash_skip_headers, + N_ELEMENTS(hdr_hash_skip_headers), + null_header_filter_callback, NULL); + + sha1_init(&sha1_ctx); + while (i_stream_read_data(input, &data, &size, 0) > 0) { + sha1_loop(&sha1_ctx, data, size); + i_stream_skip(input, size); + } + if (input->stream_errno != 0) { + i_error("pop3_migration: Failed to read header for msg %u: %m", + mail->seq); + i_stream_unref(&input); + return -1; + } + sha1_result(&sha1_ctx, sha1); + i_stream_unref(&input); + return 0; +} + +static int pop3_mailbox_open(struct mail_storage *storage) +{ + struct pop3_migration_mail_storage *mstorage = + POP3_MIGRATION_CONTEXT(storage); + struct mail_namespace *ns; + + if (mstorage->pop3_box != NULL) + return 0; + + ns = mail_namespace_find(storage->user->namespaces, + mstorage->pop3_box_vname); + if (ns == NULL) { + i_error("pop3_migration: Namespace not found for mailbox %s", + mstorage->pop3_box_vname); + return -1; + } + mstorage->pop3_box = mailbox_alloc(ns->list, mstorage->pop3_box_vname, + MAILBOX_FLAG_READONLY | + MAILBOX_FLAG_POP3_SESSION); + mstorage->all_mailboxes = + mail_user_plugin_getenv(storage->user, + "pop3_migration_all_mailboxes") != NULL; + return 0; +} + +static int pop3_map_read(struct mail_storage *storage) +{ + struct pop3_migration_mail_storage *mstorage = + POP3_MIGRATION_CONTEXT(storage); + struct mailbox *pop3_box = mstorage->pop3_box; + struct mailbox_transaction_context *t; + struct mail_search_args *search_args; + struct mail_search_context *ctx; + struct mail *mail; + struct pop3_uidl_map *map; + const char *uidl; + uoff_t size; + int ret = 0; + + if (array_is_created(&mstorage->pop3_uidl_map)) { + /* already read these, just reset the imap_uids */ + array_foreach_modifiable(&mstorage->pop3_uidl_map, map) + map->imap_uid_found = FALSE; + return 0; + } + i_array_init(&mstorage->pop3_uidl_map, 128); + + if (mailbox_sync(pop3_box, 0) < 0) { + i_error("pop3_migration: Couldn't sync mailbox %s: %s", + pop3_box->vname, mailbox_get_last_error(pop3_box, NULL)); + return -1; + } + + t = mailbox_transaction_begin(pop3_box, 0); + search_args = mail_search_build_init(); + mail_search_build_add_all(search_args); + ctx = mailbox_search_init(t, search_args, NULL, + MAIL_FETCH_VIRTUAL_SIZE, NULL); + mail_search_args_unref(&search_args); + + while (mailbox_search_next(ctx, &mail)) { + if (mail_get_virtual_size(mail, &size) < 0) { + i_error("pop3_migration: Failed to get size for msg %u: %s", + mail->seq, + mailbox_get_last_error(pop3_box, NULL)); + ret = -1; + break; + } + if (mail_get_special(mail, MAIL_FETCH_UIDL_BACKEND, &uidl) < 0) { + i_error("pop3_migration: Failed to get UIDL for msg %u: %s", + mail->seq, + mailbox_get_last_error(pop3_box, NULL)); + ret = -1; + break; + } + if (*uidl == '\0') { + i_warning("pop3_migration: UIDL for msg %u is empty", + mail->seq); + continue; + } + + map = array_append_space(&mstorage->pop3_uidl_map); + map->pop3_seq = mail->seq; + map->pop3_uidl = p_strdup(storage->pool, uidl); + map->size = size; + } + + if (mailbox_search_deinit(&ctx) < 0) + ret = -1; + (void)mailbox_transaction_commit(&t); + return ret; +} + +static int pop3_map_read_hdr_hashes(struct mail_storage *storage, + unsigned first_seq) +{ + struct pop3_migration_mail_storage *mstorage = + POP3_MIGRATION_CONTEXT(storage); + struct mailbox_transaction_context *t; + struct mail_search_args *search_args; + struct mail_search_context *ctx; + struct mail *mail; + struct pop3_uidl_map *map; + int ret = 0; + + if (mstorage->pop3_all_hdr_sha1_set) + return 0; From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: lib-storage: Added mailbox_save_set_pop3_order() an... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/82cd7aa65faf changeset: 14399:82cd7aa65faf user: Timo Sirainen date: Fri Mar 30 04:23:59 2012 +0300 description: lib-storage: Added mailbox_save_set_pop3_order() and implemented for Maildir. diffstat: src/lib-storage/index/maildir/maildir-save.c | 7 +++++++ src/lib-storage/mail-storage-private.h | 1 + src/lib-storage/mail-storage.c | 8 ++++++++ src/lib-storage/mail-storage.h | 4 ++++ 4 files changed, 20 insertions(+), 0 deletions(-) diffs (74 lines): diff -r 78317179b4af -r 82cd7aa65faf src/lib-storage/index/maildir/maildir-save.c --- a/src/lib-storage/index/maildir/maildir-save.c Fri Mar 30 03:46:37 2012 +0300 +++ b/src/lib-storage/index/maildir/maildir-save.c Fri Mar 30 04:23:59 2012 +0300 @@ -34,6 +34,7 @@ uoff_t size, vsize; enum mail_flags flags; + unsigned int pop3_order; unsigned int preserve_filename:1; unsigned int keywords_count; /* unsigned int keywords[]; */ @@ -187,6 +188,7 @@ } if (_ctx->pop3_uidl != NULL) mf->pop3_uidl = p_strdup(ctx->pool, _ctx->pop3_uidl); + mf->pop3_order = _ctx->pop3_order; /* insert into index */ mail_index_append(ctx->trans, _ctx->uid, &ctx->seq); @@ -929,6 +931,11 @@ MAILDIR_UIDLIST_REC_EXT_POP3_UIDL, mf->pop3_uidl); } + if (mf->pop3_order > 0) { + maildir_uidlist_sync_set_ext(ctx->uidlist_sync_ctx, rec, + MAILDIR_UIDLIST_REC_EXT_POP3_ORDER, + t_strdup_printf("%u", mf->pop3_order)); + } } T_END; i_assert(!seq_range_array_iter_nth(&iter, n, &uid)); } diff -r 78317179b4af -r 82cd7aa65faf src/lib-storage/mail-storage-private.h --- a/src/lib-storage/mail-storage-private.h Fri Mar 30 03:46:37 2012 +0300 +++ b/src/lib-storage/mail-storage-private.h Fri Mar 30 04:23:59 2012 +0300 @@ -444,6 +444,7 @@ uint32_t uid; char *guid, *pop3_uidl, *from_envelope; struct ostream *output; + unsigned int pop3_order; struct mail_save_attachment *attach; diff -r 78317179b4af -r 82cd7aa65faf src/lib-storage/mail-storage.c --- a/src/lib-storage/mail-storage.c Fri Mar 30 03:46:37 2012 +0300 +++ b/src/lib-storage/mail-storage.c Fri Mar 30 04:23:59 2012 +0300 @@ -1588,6 +1588,14 @@ ctx->pop3_uidl = i_strdup(uidl); } +void mailbox_save_set_pop3_order(struct mail_save_context *ctx, + unsigned int order) +{ + i_assert(order > 0); + + ctx->pop3_order = order; +} + void mailbox_save_set_dest_mail(struct mail_save_context *ctx, struct mail *mail) { diff -r 78317179b4af -r 82cd7aa65faf src/lib-storage/mail-storage.h --- a/src/lib-storage/mail-storage.h Fri Mar 30 03:46:37 2012 +0300 +++ b/src/lib-storage/mail-storage.h Fri Mar 30 04:23:59 2012 +0300 @@ -647,6 +647,10 @@ /* Set message's POP3 UIDL, if the backend supports it. */ void mailbox_save_set_pop3_uidl(struct mail_save_context *ctx, const char *uidl); +/* Specify ordering for POP3 messages. The default is to add them to the end + of the mailbox. Not all backends support this. */ +void mailbox_save_set_pop3_order(struct mail_save_context *ctx, + unsigned int order); /* If dest_mail is set, the saved message can be accessed using it. Note that setting it may require mailbox syncing, so don't set it unless you need it. Also you shouldn't try to access it before mailbox_save_finish() is From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: dsync: Preserve pop3 ordering. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/7ae930911a74 changeset: 14400:7ae930911a74 user: Timo Sirainen date: Fri Mar 30 04:25:46 2012 +0300 description: dsync: Preserve pop3 ordering. This is currently done only when syncing locally, because it's not easy to add more fields for the current dsync proxying protocol in backwards compatible way. dsync redesign should fix this. diffstat: src/doveadm/dsync/dsync-data.h | 1 + src/doveadm/dsync/dsync-worker-local.c | 6 ++++++ 2 files changed, 7 insertions(+), 0 deletions(-) diffs (41 lines): diff -r 82cd7aa65faf -r 7ae930911a74 src/doveadm/dsync/dsync-data.h --- a/src/doveadm/dsync/dsync-data.h Fri Mar 30 04:23:59 2012 +0300 +++ b/src/doveadm/dsync/dsync-data.h Fri Mar 30 04:25:46 2012 +0300 @@ -51,6 +51,7 @@ struct dsync_msg_static_data { const char *pop3_uidl; + unsigned int pop3_order; time_t received_date; struct istream *input; }; diff -r 82cd7aa65faf -r 7ae930911a74 src/doveadm/dsync/dsync-worker-local.c --- a/src/doveadm/dsync/dsync-worker-local.c Fri Mar 30 04:23:59 2012 +0300 +++ b/src/doveadm/dsync/dsync-worker-local.c Fri Mar 30 04:25:46 2012 +0300 @@ -1717,6 +1717,8 @@ save_ctx, msg); if (*data->pop3_uidl != '\0') mailbox_save_set_pop3_uidl(save_ctx, data->pop3_uidl); + if (data->pop3_order > 0) + mailbox_save_set_pop3_order(save_ctx, data->pop3_order); mailbox_save_set_received_date(save_ctx, data->received_date, 0); @@ -1793,6 +1795,7 @@ struct dsync_msg_static_data data; struct mailbox_transaction_context *trans; struct mailbox *box; + const char *value; i_assert(!worker->reading_mail); @@ -1824,6 +1827,9 @@ data.pop3_uidl = ""; else data.pop3_uidl = t_strdup(data.pop3_uidl); + if (mail_get_special(worker->get_mail, MAIL_FETCH_POP3_ORDER, &value) < 0 || + str_to_uint(value, &data.pop3_order) < 0) + data.pop3_order = 0; if (mail_get_received_date(worker->get_mail, &data.received_date) < 0 || mail_get_stream(worker->get_mail, NULL, NULL, &data.input) < 0) { get->callback(worker->get_mail->expunged ? From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: pop3-replication: Don't request virtual size from I... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/87cd04f35f82 changeset: 14402:87cd04f35f82 user: Timo Sirainen date: Fri Mar 30 05:11:09 2012 +0300 description: pop3-replication: Don't request virtual size from IMAP mailbox, we don't need it. diffstat: src/plugins/pop3-migration/pop3-migration-plugin.c | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diffs (13 lines): diff -r 6df2235fd6af -r 87cd04f35f82 src/plugins/pop3-migration/pop3-migration-plugin.c --- a/src/plugins/pop3-migration/pop3-migration-plugin.c Fri Mar 30 04:26:00 2012 +0300 +++ b/src/plugins/pop3-migration/pop3-migration-plugin.c Fri Mar 30 05:11:09 2012 +0300 @@ -309,8 +309,7 @@ search_args = mail_search_build_init(); mail_search_build_add_all(search_args); ctx = mailbox_search_init(t, search_args, NULL, - MAIL_FETCH_PHYSICAL_SIZE | - MAIL_FETCH_VIRTUAL_SIZE, NULL); + MAIL_FETCH_PHYSICAL_SIZE, NULL); mail_search_args_unref(&search_args); while (mailbox_search_next(ctx, &mail)) { From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: pop3-migration: Migrate also POP3 ordering. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/6df2235fd6af changeset: 14401:6df2235fd6af user: Timo Sirainen date: Fri Mar 30 04:26:00 2012 +0300 description: pop3-migration: Migrate also POP3 ordering. diffstat: src/plugins/pop3-migration/pop3-migration-plugin.c | 64 +++++++++++++++++---- 1 files changed, 51 insertions(+), 13 deletions(-) diffs (195 lines): diff -r 7ae930911a74 -r 6df2235fd6af src/plugins/pop3-migration/pop3-migration-plugin.c --- a/src/plugins/pop3-migration/pop3-migration-plugin.c Fri Mar 30 04:25:46 2012 +0300 +++ b/src/plugins/pop3-migration/pop3-migration-plugin.c Fri Mar 30 04:26:00 2012 +0300 @@ -5,6 +5,7 @@ #include "istream.h" #include "istream-header-filter.h" #include "sha1.h" +#include "message-size.h" #include "mail-namespace.h" #include "mail-search-build.h" #include "mail-storage-private.h" @@ -17,6 +18,7 @@ struct pop3_uidl_map { uint32_t pop3_seq; + uint32_t imap_uid; /* UIDL */ const char *pop3_uidl; @@ -25,11 +27,10 @@ /* sha1(TOP 0) - set only when needed */ unsigned char hdr_sha1[SHA1_RESULTLEN]; unsigned int hdr_sha1_set:1; - unsigned int imap_uid_found:1; }; struct imap_msg_map { - uint32_t uid; + uint32_t uid, pop3_seq; uoff_t psize; const char *pop3_uidl; @@ -57,6 +58,7 @@ unsigned int uidl_synced:1; unsigned int uidl_sync_failed:1; + unsigned int uidl_ordered:1; }; static const char *hdr_hash_skip_headers[] = { @@ -111,22 +113,25 @@ static int get_hdr_sha1(struct mail *mail, unsigned char sha1[SHA1_RESULTLEN]) { - struct istream *input; + struct message_size hdr_size; + struct istream *input, *input2; const unsigned char *data; size_t size; struct sha1_ctxt sha1_ctx; - if (mail_get_hdr_stream(mail, NULL, &input) < 0) { + if (mail_get_hdr_stream(mail, &hdr_size, &input) < 0) { i_error("pop3_migration: Failed to get header for msg %u", mail->seq); return -1; } + input2 = i_stream_create_limit(input, hdr_size.physical_size); /* hide headers that might change or be different in IMAP vs. POP3 */ - input = i_stream_create_header_filter(input, + input = i_stream_create_header_filter(input2, HEADER_FILTER_EXCLUDE | HEADER_FILTER_NO_CR, hdr_hash_skip_headers, N_ELEMENTS(hdr_hash_skip_headers), null_header_filter_callback, NULL); + i_stream_unref(&input2); sha1_init(&sha1_ctx); while (i_stream_read_data(input, &data, &size, 0) > 0) { @@ -186,7 +191,7 @@ if (array_is_created(&mstorage->pop3_uidl_map)) { /* already read these, just reset the imap_uids */ array_foreach_modifiable(&mstorage->pop3_uidl_map, map) - map->imap_uid_found = FALSE; + map->imap_uid = 0; return 0; } i_array_init(&mstorage->pop3_uidl_map, 128); @@ -378,9 +383,14 @@ for (i = 0; i < count; i++) { if (pop3_map[i].size != imap_map[i].psize) break; + if (i+1 < count && pop3_map[i].size == pop3_map[i+1].size) { + /* two messages with same size, don't trust them */ + break; + } - pop3_map[i].imap_uid_found = TRUE; + pop3_map[i].imap_uid = imap_map[i].uid; imap_map[i].pop3_uidl = pop3_map[i].pop3_uidl; + imap_map[i].pop3_seq = pop3_map[i].pop3_seq; } mbox->first_unfound_idx = i; return i == count; @@ -411,7 +421,7 @@ pop3_idx = imap_idx = 0; while (pop3_idx < pop3_count && imap_idx < imap_count) { if (!pop3_map[pop3_idx].hdr_sha1_set || - pop3_map[pop3_idx].imap_uid_found) { + pop3_map[pop3_idx].imap_uid != 0) { pop3_idx++; continue; } @@ -428,28 +438,35 @@ else if (ret > 0) imap_idx++; else { - pop3_map[pop3_idx].imap_uid_found = TRUE; + pop3_map[pop3_idx].imap_uid = imap_map[imap_idx].uid; imap_map[imap_idx].pop3_uidl = pop3_map[pop3_idx].pop3_uidl; + imap_map[imap_idx].pop3_seq = + pop3_map[pop3_idx].pop3_seq; } } missing_uids_count = 0; for (pop3_idx = 0; pop3_idx < pop3_count; pop3_idx++) { - if (!pop3_map[pop3_idx].imap_uid_found) + if (pop3_map[pop3_idx].imap_uid == 0) missing_uids_count++; } if (missing_uids_count > 0 && !mstorage->all_mailboxes) { i_warning("pop3_migration: %u POP3 messages have no " "matching IMAP messages", missing_uids_count); } - array_sort(&mstorage->pop3_uidl_map, pop3_uidl_map_pop3_seq_cmp); + array_sort(&mbox->imap_msg_map, imap_msg_map_uid_cmp); return 0; } static int pop3_migration_uidl_sync(struct mailbox *box) { struct pop3_migration_mailbox *mbox = POP3_MIGRATION_CONTEXT(box); + struct pop3_migration_mail_storage *mstorage = + POP3_MIGRATION_CONTEXT(box->storage); + const struct pop3_uidl_map *pop3_map; + unsigned int i, count; + uint32_t prev_uid; if (mbox->uidl_synced) return 0; @@ -467,6 +484,21 @@ return -1; } + /* see if the POP3 UIDL order is the same as IMAP UID order */ + mbox->uidl_ordered = TRUE; + pop3_map = array_get(&mstorage->pop3_uidl_map, &count); + prev_uid = 0; + for (i = 0; i < count; i++) { + if (pop3_map[i].imap_uid == 0) + continue; + + if (prev_uid > pop3_map[i].imap_uid) { + mbox->uidl_ordered = FALSE; + break; + } + prev_uid = pop3_map[i].imap_uid; + } + mbox->uidl_synced = TRUE; return 0; } @@ -480,7 +512,8 @@ struct pop3_migration_mailbox *mbox = POP3_MIGRATION_CONTEXT(_mail->box); struct imap_msg_map map_key, *map; - if (field == MAIL_FETCH_UIDL_BACKEND) { + if (field == MAIL_FETCH_UIDL_BACKEND || + field == MAIL_FETCH_POP3_ORDER) { if (mbox->uidl_sync_failed || pop3_migration_uidl_sync(_mail->box) < 0) { mbox->uidl_sync_failed = TRUE; @@ -489,14 +522,19 @@ "POP3 UIDLs couldn't be synced"); return -1; } + memset(&map_key, 0, sizeof(map_key)); map_key.uid = _mail->uid; map = array_bsearch(&mbox->imap_msg_map, &map_key, imap_msg_map_uid_cmp); if (map != NULL && map->pop3_uidl != NULL) { - *value_r = map->pop3_uidl; + if (field == MAIL_FETCH_UIDL_BACKEND) + *value_r = map->pop3_uidl; + else + *value_r = t_strdup_printf("%u", map->pop3_seq); return 0; } + /* not found from POP3 server, fallback to default */ } return mmail->super.get_special(_mail, field, value_r); } From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: imapc: Added imapc_features=rfc822.size setting to ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/6a8b78450202 changeset: 14403:6a8b78450202 user: Timo Sirainen date: Fri Mar 30 05:20:08 2012 +0300 description: imapc: Added imapc_features=rfc822.size setting to use RFC822.size for physical sizes. diffstat: src/lib-storage/index/imapc/imapc-mail-fetch.c | 18 ++++++++ src/lib-storage/index/imapc/imapc-mail.c | 53 ++++++++++++++++++------- src/lib-storage/index/imapc/imapc-settings.c | 41 ++++++++++++++++++++ src/lib-storage/index/imapc/imapc-settings.h | 9 ++++ src/lib-storage/index/imapc/imapc-storage.h | 6 ++ 5 files changed, 112 insertions(+), 15 deletions(-) diffs (264 lines): diff -r 87cd04f35f82 -r 6a8b78450202 src/lib-storage/index/imapc/imapc-mail-fetch.c --- a/src/lib-storage/index/imapc/imapc-mail-fetch.c Fri Mar 30 05:11:09 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-mail-fetch.c Fri Mar 30 05:20:08 2012 +0300 @@ -95,6 +95,8 @@ str_printfa(str, "UID FETCH %u (", _mail->uid); if ((fields & MAIL_FETCH_RECEIVED_DATE) != 0) str_append(str, "INTERNALDATE "); + if ((fields & MAIL_FETCH_PHYSICAL_SIZE) != 0) + str_append(str, "RFC822.SIZE "); if ((fields & MAIL_FETCH_GUID) != 0) { str_append(str, mbox->guid_fetch_field_name); str_append_c(str, ' '); @@ -157,6 +159,10 @@ if ((data->wanted_fields & MAIL_FETCH_RECEIVED_DATE) != 0 && data->received_date == (time_t)-1) fields |= MAIL_FETCH_RECEIVED_DATE; + if ((data->wanted_fields & MAIL_FETCH_PHYSICAL_SIZE) != 0 && + data->physical_size == (uoff_t)-1 && + IMAPC_BOX_HAS_FEATURE(mbox, IMAPC_FEATURE_RFC822_SIZE)) + fields |= MAIL_FETCH_PHYSICAL_SIZE; if ((data->wanted_fields & MAIL_FETCH_GUID) != 0 && data->guid == NULL && mbox->guid_fetch_field_name != NULL) fields |= MAIL_FETCH_GUID; @@ -181,6 +187,11 @@ return FALSE; fields &= ~MAIL_FETCH_RECEIVED_DATE; } + if ((fields & MAIL_FETCH_PHYSICAL_SIZE) != 0) { + if (imail->imail.data.physical_size == (uoff_t)-1) + return FALSE; + fields &= ~MAIL_FETCH_PHYSICAL_SIZE; + } if ((fields & MAIL_FETCH_GUID) != 0) { if (imail->imail.data.guid == NULL) return FALSE; @@ -357,6 +368,7 @@ (struct imapc_mailbox *)mail->imail.mail.mail.box; const char *key, *value; unsigned int i; + uoff_t size; time_t t; int tz; bool match = FALSE; @@ -377,6 +389,12 @@ imap_parse_datetime(value, &t, &tz)) mail->imail.data.received_date = t; match = TRUE; + } else if (strcasecmp(key, "RFC822.SIZE") == 0) { + if (imap_arg_get_atom(&args[i+1], &value) && + str_to_uoff(value, &size) == 0 && + IMAPC_BOX_HAS_FEATURE(mbox, IMAPC_FEATURE_RFC822_SIZE)) + mail->imail.data.physical_size = size; + match = TRUE; } else if (strcasecmp(key, "X-GM-MSGID") == 0 || strcasecmp(key, "X-GUID") == 0) { if (imap_arg_get_astring(&args[i+1], &value)) { diff -r 87cd04f35f82 -r 6a8b78450202 src/lib-storage/index/imapc/imapc-mail.c --- a/src/lib-storage/index/imapc/imapc-mail.c Fri Mar 30 05:11:09 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-mail.c Fri Mar 30 05:20:08 2012 +0300 @@ -107,29 +107,50 @@ static int imapc_mail_get_physical_size(struct mail *_mail, uoff_t *size_r) { + struct imapc_mailbox *mbox = (struct imapc_mailbox *)_mail->box; struct index_mail *mail = (struct index_mail *)_mail; struct index_mail_data *data = &mail->data; struct istream *input; uoff_t old_offset; int ret; - if (data->physical_size == (uoff_t)-1) + if (data->physical_size == (uoff_t)-1) { (void)index_mail_get_physical_size(_mail, size_r); - if (data->physical_size == (uoff_t)-1) { - old_offset = data->stream == NULL ? 0 : data->stream->v_offset; - if (mail_get_stream(_mail, NULL, NULL, &input) < 0) + if (data->physical_size != (uoff_t)-1) { + *size_r = data->physical_size; + return 0; + } + } + + if (IMAPC_BOX_HAS_FEATURE(mbox, IMAPC_FEATURE_RFC822_SIZE) && + data->stream == NULL) { + /* trust RFC822.SIZE to be correct */ + if (imapc_mail_fetch(_mail, MAIL_FETCH_PHYSICAL_SIZE) < 0) return -1; - i_stream_seek(data->stream, old_offset); + if (data->physical_size == (uoff_t)-1) { + if (imapc_mail_failed(_mail, "RFC822.SIZE") < 0) + return -1; + /* assume that the server never returns RFC822.SIZE + for this mail (see BODY[] failure handling) */ + data->physical_size = 0; + } + *size_r = data->physical_size; + return 0; + } - ret = i_stream_get_size(data->stream, TRUE, - &data->physical_size); - if (ret <= 0) { - i_assert(ret != 0); - mail_storage_set_critical(_mail->box->storage, - "imapc: stat(%s) failed: %m", - i_stream_get_name(data->stream)); - return -1; - } + old_offset = data->stream == NULL ? 0 : data->stream->v_offset; + if (mail_get_stream(_mail, NULL, NULL, &input) < 0) + return -1; + i_stream_seek(data->stream, old_offset); + + ret = i_stream_get_size(data->stream, TRUE, + &data->physical_size); + if (ret <= 0) { + i_assert(ret != 0); + mail_storage_set_critical(_mail->box->storage, + "imapc: stat(%s) failed: %m", + i_stream_get_name(data->stream)); + return -1; } *size_r = data->physical_size; return 0; @@ -200,6 +221,7 @@ static void index_mail_update_access_parts(struct index_mail *mail) { struct mail *_mail = &mail->mail.mail; + struct imapc_mailbox *mbox = (struct imapc_mailbox *)_mail->box; struct index_mail_data *data = &mail->data; struct mailbox_header_lookup_ctx *header_ctx; time_t date; @@ -208,7 +230,8 @@ if ((data->wanted_fields & MAIL_FETCH_RECEIVED_DATE) != 0) (void)index_mail_get_received_date(_mail, &date); if ((data->wanted_fields & MAIL_FETCH_PHYSICAL_SIZE) != 0) { - if (index_mail_get_physical_size(_mail, &size) < 0) + if (index_mail_get_physical_size(_mail, &size) < 0 && + !IMAPC_BOX_HAS_FEATURE(mbox, IMAPC_FEATURE_RFC822_SIZE)) data->access_part |= READ_HDR | READ_BODY; } diff -r 87cd04f35f82 -r 6a8b78450202 src/lib-storage/index/imapc/imapc-settings.c --- a/src/lib-storage/index/imapc/imapc-settings.c Fri Mar 30 05:11:09 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-settings.c Fri Mar 30 05:20:08 2012 +0300 @@ -25,6 +25,7 @@ DEF(SET_STR, imapc_ssl_ca_dir), DEF(SET_BOOL, imapc_ssl_verify), + DEF(SET_STR, imapc_features), DEF(SET_STR, imapc_rawlog_dir), DEF(SET_STR, ssl_crypto_device), @@ -43,6 +44,7 @@ .imapc_ssl_ca_dir = "", .imapc_ssl_verify = TRUE, + .imapc_features = "", .imapc_rawlog_dir = "", .ssl_crypto_device = "" }; @@ -67,6 +69,43 @@ } /* */ +struct imapc_feature_list { + const char *name; + enum imapc_features num; +}; + +static const struct imapc_feature_list imapc_feature_list[] = { + { "rfc822.size", IMAPC_FEATURE_RFC822_SIZE }, + { NULL, 0 } +}; + +static int +imapc_settings_parse_features(struct imapc_settings *set, + const char **error_r) +{ + enum imapc_features features = 0; + const struct imapc_feature_list *list; + const char *const *str; + + str = t_strsplit_spaces(set->imapc_features, " ,"); + for (; *str != NULL; str++) { + list = imapc_feature_list; + for (; list->name != NULL; list++) { + if (strcasecmp(*str, list->name) == 0) { + features |= list->num; + break; + } + } + if (list->name == NULL) { + *error_r = t_strdup_printf("imapc_features: " + "Unknown feature: %s", *str); + return -1; + } + } + set->parsed_features = features; + return 0; +} + static bool imapc_settings_check(void *_set, pool_t pool ATTR_UNUSED, const char **error_r) { @@ -85,5 +124,7 @@ return FALSE; } #endif + if (imapc_settings_parse_features(set, error_r) < 0) + return FALSE; return TRUE; } diff -r 87cd04f35f82 -r 6a8b78450202 src/lib-storage/index/imapc/imapc-settings.h --- a/src/lib-storage/index/imapc/imapc-settings.h Fri Mar 30 05:11:09 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-settings.h Fri Mar 30 05:20:08 2012 +0300 @@ -1,6 +1,12 @@ #ifndef IMAPC_SETTINGS_H #define IMAPC_SETTINGS_H +/* */ +enum imapc_features { + IMAPC_FEATURE_RFC822_SIZE = 0x01 +}; +/* */ + struct imapc_settings { const char *imapc_host; unsigned int imapc_port; @@ -13,8 +19,11 @@ const char *imapc_ssl_ca_dir; bool imapc_ssl_verify; + const char *imapc_features; const char *imapc_rawlog_dir; const char *ssl_crypto_device; + + enum imapc_features parsed_features; }; const struct setting_parser_info *imapc_get_setting_parser_info(void); diff -r 87cd04f35f82 -r 6a8b78450202 src/lib-storage/index/imapc/imapc-storage.h --- a/src/lib-storage/index/imapc/imapc-storage.h Fri Mar 30 05:11:09 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-storage.h Fri Mar 30 05:20:08 2012 +0300 @@ -2,6 +2,7 @@ #define IMAPC_STORAGE_H #include "index-storage.h" +#include "imapc-settings.h" #define IMAPC_STORAGE_NAME "imapc" #define IMAPC_INDEX_PREFIX "dovecot.index" @@ -28,6 +29,11 @@ imapc_mailbox_callback_t *callback; }; +#define IMAPC_HAS_FEATURE(mstorage, feature) \ + (((mstorage)->set->parsed_features & feature) != 0) +#define IMAPC_BOX_HAS_FEATURE(mbox, feature) \ + (((mbox)->storage->set->parsed_features & feature) != 0) + struct imapc_storage { struct mail_storage storage; const struct imapc_settings *set; From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: director: Changes to PING handling. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/168e3b5cb6e8 changeset: 14405:168e3b5cb6e8 user: Timo Sirainen date: Mon Apr 02 23:28:22 2012 +0300 description: director: Changes to PING handling. Use larger ping timeouts. While waiting for sync keep doing rapid pings until the sync is finished, not just once. diffstat: src/director/director-connection.c | 63 +++++++++++++++++++++---------------- src/director/director-connection.h | 3 +- src/director/director.c | 11 +++++- 3 files changed, 46 insertions(+), 31 deletions(-) diffs (203 lines): diff -r 7f4c35fcae80 -r 168e3b5cb6e8 src/director/director-connection.c --- a/src/director/director-connection.c Mon Apr 02 21:49:05 2012 +0300 +++ b/src/director/director-connection.c Mon Apr 02 23:28:22 2012 +0300 @@ -23,13 +23,13 @@ #define MAX_OUTBUF_SIZE (1024*1024*10) #define OUTBUF_FLUSH_THRESHOLD (1024*128) /* Max idling time while connecting/handshaking before disconnecting */ -#define DIRECTOR_CONNECTION_INIT_TIMEOUT_MSECS (2*1000) +#define DIRECTOR_CONNECTION_INIT_TIMEOUT_MSECS (10*1000) /* How long to wait for PONG after PING request */ -#define DIRECTOR_CONNECTION_PING_TIMEOUT_MSECS (2*1000) +#define DIRECTOR_CONNECTION_PING_TIMEOUT_MSECS (10*1000) /* How long to wait to send PING when connection is idle */ #define DIRECTOR_CONNECTION_PING_INTERVAL_MSECS (15*1000) /* How long to wait before sending PING while waiting for SYNC reply */ -#define DIRECTOR_CONNECTION_SYNC_TIMEOUT_MSECS 1000 +#define DIRECTOR_CONNECTION_PING_SYNC_INTERVAL_MSECS 1000 /* If outgoing director connection exists for less than this many seconds, mark the host as failed so we won't try to reconnect to it immediately */ #define DIRECTOR_SUCCESS_MIN_CONNECT_SECS 10 @@ -61,7 +61,7 @@ unsigned int ignore_host_events:1; unsigned int handshake_sending_hosts:1; unsigned int ping_waiting:1; - unsigned int sync_ping:1; + unsigned int synced:1; }; static void director_connection_ping(struct director_connection *conn); @@ -636,6 +636,19 @@ return TRUE; } +static void +director_connection_set_ping_timeout(struct director_connection *conn) +{ + unsigned int msecs; + + msecs = conn->synced || !conn->handshake_received ? + DIRECTOR_CONNECTION_PING_INTERVAL_MSECS : + DIRECTOR_CONNECTION_PING_SYNC_INTERVAL_MSECS; + + timeout_remove(&conn->to_ping); + conn->to_ping = timeout_add(msecs, director_connection_ping, conn); +} + static void director_handshake_cmd_done(struct director_connection *conn) { struct director *dir = conn->dir; @@ -668,10 +681,7 @@ director_sync_send(dir, dir->self_host, dir->sync_seq, DIRECTOR_VERSION_MINOR); } - if (conn->to_ping != NULL) - timeout_remove(&conn->to_ping); - conn->to_ping = timeout_add(DIRECTOR_CONNECTION_PING_INTERVAL_MSECS, - director_connection_ping, conn); + director_connection_set_ping_timeout(conn); } static bool @@ -893,11 +903,9 @@ { if (!conn->ping_waiting) return TRUE; + conn->ping_waiting = FALSE; - conn->ping_waiting = FALSE; - timeout_remove(&conn->to_ping); - conn->to_ping = timeout_add(DIRECTOR_CONNECTION_PING_INTERVAL_MSECS, - director_connection_ping, conn); + director_connection_set_ping_timeout(conn); return TRUE; } @@ -971,8 +979,6 @@ char *line; bool ret; - if (conn->to_ping != NULL) - timeout_reset(conn->to_ping); switch (i_stream_read(conn->input)) { case 0: return; @@ -1007,6 +1013,8 @@ } } director_sync_thaw(dir); + if (conn != NULL) + timeout_reset(conn->to_ping); } static void director_connection_send_directors(struct director_connection *conn, @@ -1056,6 +1064,7 @@ if (o_stream_get_buffer_used_size(conn->output) >= OUTBUF_FLUSH_THRESHOLD) { if ((ret = o_stream_flush(conn->output)) <= 0) { /* continue later */ + timeout_reset(conn->to_ping); return ret; } } @@ -1068,6 +1077,7 @@ ret = o_stream_flush(conn->output); o_stream_uncork(conn->output); + timeout_reset(conn->to_ping); return ret; } @@ -1224,8 +1234,7 @@ user_directory_iter_deinit(&conn->user_iter); if (conn->to != NULL) timeout_remove(&conn->to); - if (conn->to_ping != NULL) - timeout_remove(&conn->to_ping); + timeout_remove(&conn->to_ping); if (conn->io != NULL) io_remove(&conn->io); i_stream_unref(&conn->input); @@ -1298,12 +1307,10 @@ static void director_connection_ping(struct director_connection *conn) { - conn->sync_ping = FALSE; if (conn->ping_waiting) return; - if (conn->to_ping != NULL) - timeout_remove(&conn->to_ping); + timeout_remove(&conn->to_ping); conn->to_ping = timeout_add(DIRECTOR_CONNECTION_PING_TIMEOUT_MSECS, director_connection_ping_timeout, conn); director_connection_send(conn, "PING\n"); @@ -1344,18 +1351,18 @@ o_stream_uncork(conn->output); } -void director_connection_wait_sync(struct director_connection *conn) +void director_connection_set_synced(struct director_connection *conn, + bool synced) { - /* switch to faster ping timeout. avoid reseting the timeout if it's - already fast. */ - if (conn->ping_waiting || conn->sync_ping) + if (conn->synced == synced) + return; + conn->synced = synced; + + /* switch ping timeout, unless we're already waiting for PONG */ + if (conn->ping_waiting) return; - if (conn->to_ping != NULL) - timeout_remove(&conn->to_ping); - conn->to_ping = timeout_add(DIRECTOR_CONNECTION_SYNC_TIMEOUT_MSECS, - director_connection_ping, conn); - conn->sync_ping = TRUE; + director_connection_set_ping_timeout(conn); } void director_connections_deinit(struct director *dir) diff -r 7f4c35fcae80 -r 168e3b5cb6e8 src/director/director-connection.h --- a/src/director/director-connection.h Mon Apr 02 21:49:05 2012 +0300 +++ b/src/director/director-connection.h Mon Apr 02 23:28:22 2012 +0300 @@ -14,7 +14,8 @@ void director_connection_send(struct director_connection *conn, const char *data); -void director_connection_wait_sync(struct director_connection *conn); +void director_connection_set_synced(struct director_connection *conn, + bool synced); void director_connection_send_except(struct director_connection *conn, struct director_host *skip_host, const char *data); diff -r 7f4c35fcae80 -r 168e3b5cb6e8 src/director/director.c --- a/src/director/director.c Mon Apr 02 21:49:05 2012 +0300 +++ b/src/director/director.c Mon Apr 02 23:28:22 2012 +0300 @@ -230,6 +230,10 @@ timeout_remove(&dir->to_reconnect); } + if (dir->left != NULL) + director_connection_set_synced(dir->left, TRUE); + if (dir->right != NULL) + director_connection_set_synced(dir->right, TRUE); if (dir->to_sync != NULL) timeout_remove(&dir->to_sync); dir->ring_synced = TRUE; @@ -310,9 +314,12 @@ director_connection_get_name(dir->right)); } + /* send PINGs to our connections more rapidly until we've synced again. + if the connection has actually died, we don't need to wait (and + delay requests) for as long to detect it */ if (dir->left != NULL) - director_connection_wait_sync(dir->left); - director_connection_wait_sync(dir->right); + director_connection_set_synced(dir->left, FALSE); + director_connection_set_synced(dir->right, FALSE); director_sync_send(dir, dir->self_host, dir->sync_seq, DIRECTOR_VERSION_MINOR); } From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: director: List of director connections belongs to s... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/c760ac046203 changeset: 14406:c760ac046203 user: Timo Sirainen date: Mon Apr 02 23:39:48 2012 +0300 description: director: List of director connections belongs to struct director. diffstat: src/director/director-connection.c | 40 ++++++++++++------------------------- src/director/director-connection.h | 6 +---- src/director/director.c | 26 ++++++++++++++++++++++- src/director/director.h | 2 +- 4 files changed, 39 insertions(+), 35 deletions(-) diffs (185 lines): diff -r 168e3b5cb6e8 -r c760ac046203 src/director/director-connection.c --- a/src/director/director-connection.c Mon Apr 02 23:28:22 2012 +0300 +++ b/src/director/director-connection.c Mon Apr 02 23:39:48 2012 +0300 @@ -7,7 +7,6 @@ #include "istream.h" #include "ostream.h" #include "str.h" -#include "llist.h" #include "master-service.h" #include "mail-host.h" #include "director.h" @@ -35,8 +34,6 @@ #define DIRECTOR_SUCCESS_MIN_CONNECT_SECS 10 struct director_connection { - struct director_connection *prev, *next; - struct director *dir; char *name; time_t created; @@ -1116,7 +1113,7 @@ director_connection_output, conn); conn->to_ping = timeout_add(DIRECTOR_CONNECTION_INIT_TIMEOUT_MSECS, director_connection_init_timeout, conn); - DLLIST_PREPEND(&dir->connections, conn); + array_append(&dir->connections, &conn, 1); return conn; } @@ -1212,8 +1209,9 @@ void director_connection_deinit(struct director_connection **_conn) { - struct director_connection *conn = *_conn; + struct director_connection *const *conns, *conn = *_conn; struct director *dir = conn->dir; + unsigned int i, count; *_conn = NULL; @@ -1224,7 +1222,14 @@ conn->created + DIRECTOR_SUCCESS_MIN_CONNECT_SECS > ioloop_time) conn->host->last_failed = ioloop_time; - DLLIST_REMOVE(&dir->connections, conn); + conns = array_get(&dir->connections, &count); + for (i = 0; i < count; i++) { + if (conns[i] == conn) { + array_delete(&dir->connections, i, 1); + break; + } + } + i_assert(i < count); if (dir->left == conn) dir->left = NULL; if (dir->right == conn) @@ -1328,17 +1333,9 @@ return conn->host; } -struct director_connection * -director_connection_find_outgoing(struct director *dir, - struct director_host *host) +bool director_connection_is_incoming(struct director_connection *conn) { - struct director_connection *conn; - - for (conn = dir->connections; conn != NULL; conn = conn->next) { - if (conn->host == host && !conn->in) - return conn; - } - return NULL; + return conn->in; } void director_connection_cork(struct director_connection *conn) @@ -1364,14 +1361,3 @@ director_connection_set_ping_timeout(conn); } - -void director_connections_deinit(struct director *dir) -{ - struct director_connection *conn; - - while (dir->connections != NULL) { - conn = dir->connections; - dir->connections = conn->next; - director_connection_deinit(&conn); - } -} diff -r 168e3b5cb6e8 -r c760ac046203 src/director/director-connection.h --- a/src/director/director-connection.h Mon Apr 02 23:28:22 2012 +0300 +++ b/src/director/director-connection.h Mon Apr 02 23:39:48 2012 +0300 @@ -23,13 +23,9 @@ const char *director_connection_get_name(struct director_connection *conn); struct director_host * director_connection_get_host(struct director_connection *conn); -struct director_connection * -director_connection_find_outgoing(struct director *dir, - struct director_host *host); +bool director_connection_is_incoming(struct director_connection *conn); void director_connection_cork(struct director_connection *conn); void director_connection_uncork(struct director_connection *conn); -void director_connections_deinit(struct director *dir); - #endif diff -r 168e3b5cb6e8 -r c760ac046203 src/director/director.c --- a/src/director/director.c Mon Apr 02 23:28:22 2012 +0300 +++ b/src/director/director.c Mon Apr 02 23:39:48 2012 +0300 @@ -81,12 +81,26 @@ i_unreached(); } +static bool +director_has_outgoing_connection(struct director *dir, + struct director_host *host) +{ + struct director_connection *const *connp; + + array_foreach(&dir->connections, connp) { + if (director_connection_get_host(*connp) == host && + !director_connection_is_incoming(*connp)) + return TRUE; + } + return FALSE; +} + int director_connect_host(struct director *dir, struct director_host *host) { unsigned int port; int fd; - if (director_connection_find_outgoing(dir, host) != NULL) + if (director_has_outgoing_connection(dir, host)) return 0; if (dir->debug) { @@ -669,6 +683,7 @@ dir->state_change_callback = callback; i_array_init(&dir->dir_hosts, 16); i_array_init(&dir->pending_requests, 16); + i_array_init(&dir->connections, 8); dir->users = user_directory_init(set->director_user_expire, set->director_username_hash); dir->mail_hosts = mail_hosts_init(); @@ -682,10 +697,16 @@ { struct director *dir = *_dir; struct director_host *const *hostp; + struct director_connection *conn, *const *connp; *_dir = NULL; - director_connections_deinit(dir); + while (array_count(&dir->connections) > 0) { + connp = array_idx(&dir->connections, 0); + conn = *connp; + director_connection_deinit(&conn); + } + user_directory_deinit(&dir->users); mail_hosts_deinit(&dir->mail_hosts); mail_hosts_deinit(&dir->orig_config_hosts); @@ -703,5 +724,6 @@ director_host_free(*hostp); array_free(&dir->pending_requests); array_free(&dir->dir_hosts); + array_free(&dir->connections); i_free(dir); } diff -r 168e3b5cb6e8 -r c760ac046203 src/director/director.h --- a/src/director/director.h Mon Apr 02 23:28:22 2012 +0300 +++ b/src/director/director.h Mon Apr 02 23:39:48 2012 +0300 @@ -29,7 +29,7 @@ struct director_host *self_host; struct director_connection *left, *right; /* all director connections */ - struct director_connection *connections; + ARRAY_DEFINE(connections, struct director_connection *); struct timeout *to_reconnect; struct timeout *to_sync; From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: director: Improved error logging for handling inval... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d6cd93e32b37 changeset: 14407:d6cd93e32b37 user: Timo Sirainen date: Tue Apr 03 00:23:02 2012 +0300 description: director: Improved error logging for handling invalid commands/parameters. diffstat: src/director/director-connection.c | 103 ++++++++++++++++++++++-------------- 1 files changed, 63 insertions(+), 40 deletions(-) diffs (251 lines): diff -r c760ac046203 -r d6cd93e32b37 src/director/director-connection.c --- a/src/director/director-connection.c Mon Apr 02 23:39:48 2012 +0300 +++ b/src/director/director-connection.c Tue Apr 03 00:23:02 2012 +0300 @@ -50,6 +50,9 @@ struct user_directory_iter *user_iter; + /* set during command execution */ + const char *cur_cmd, *cur_line; + unsigned int in:1; unsigned int connected:1; unsigned int version_received:1; @@ -64,19 +67,32 @@ static void director_connection_ping(struct director_connection *conn); static void director_connection_disconnected(struct director_connection **conn); +static void ATTR_FORMAT(2, 3) +director_cmd_error(struct director_connection *conn, const char *fmt, ...) +{ + va_list args; + + va_start(args, fmt); + i_error("director(%s): Command %s: %s (input: %s)", conn->name, + conn->cur_cmd, t_strdup_vprintf(fmt, args), conn->cur_line); + va_end(args); +} + static bool director_args_parse_ip_port(struct director_connection *conn, const char *const *args, struct ip_addr *ip_r, unsigned int *port_r) { + if (args[0] == NULL || args[1] == NULL) { + director_cmd_error(conn, "Missing IP+port parameters"); + return FALSE; + } if (net_addr2ip(args[0], ip_r) < 0) { - i_error("director(%s): Command has invalid IP address: %s", - conn->name, args[0]); + director_cmd_error(conn, "Invalid IP address: %s", args[0]); return FALSE; } if (str_to_uint(args[1], port_r) < 0) { - i_error("director(%s): Command has invalid port: %s", - conn->name, args[1]); + director_cmd_error(conn, "Invalid port: %s", args[1]); return FALSE; } return TRUE; @@ -276,8 +292,7 @@ str_to_uint(args[0], &username_hash) < 0 || net_addr2ip(args[1], &ip) < 0 || str_to_uint(args[2], ×tamp) < 0) { - i_error("director(%s): Invalid USER handshake args", - conn->name); + director_cmd_error(conn, "Invalid parameters"); return FALSE; } weak = args[3] != NULL && args[3][0] == 'w'; @@ -305,7 +320,7 @@ if (str_array_length(args) != 2 || str_to_uint(args[0], &username_hash) < 0 || net_addr2ip(args[1], &ip) < 0) { - i_error("director(%s): Invalid USER args", conn->name); + director_cmd_error(conn, "Invalid parameters"); return FALSE; } @@ -356,9 +371,9 @@ struct mail_host *const *hostp; unsigned int remote_ring_completed; - if (args == NULL || str_to_uint(args[0], &remote_ring_completed) < 0) { - i_error("director(%s): Invalid HOST-HAND-START args", - conn->name); + if (args[0] == NULL || + str_to_uint(args[0], &remote_ring_completed) < 0) { + director_cmd_error(conn, "Invalid parameters"); return FALSE; } @@ -391,8 +406,7 @@ net_addr2ip(args[0], &ip) < 0 || str_to_uint(args[1], &port) < 0 || str_to_uint(args[2], &seq) < 0) { - i_error("director(%s): Command is missing parameters: %s", - conn->name, t_strarray_join(args, " ")); + director_cmd_error(conn, "Invalid parameters"); return -1; } *_args = args + 3; @@ -433,7 +447,7 @@ if (str_array_length(args) != 2 || str_to_uint(args[0], &username_hash) < 0 || net_addr2ip(args[1], &ip) < 0) { - i_error("director(%s): Invalid USER-WEAK args", conn->name); + director_cmd_error(conn, "Invalid parameters"); return FALSE; } @@ -474,7 +488,7 @@ if (str_array_length(args) != 2 || net_addr2ip(args[0], &ip) < 0 || str_to_uint(args[1], &vhost_count) < 0) { - i_error("director(%s): Invalid HOST args", conn->name); + director_cmd_error(conn, "Invalid parameters"); return FALSE; } if (conn->ignore_host_events) { @@ -532,7 +546,7 @@ if (str_array_length(args) != 1 || net_addr2ip(args[0], &ip) < 0) { - i_error("director(%s): Invalid HOST-REMOVE args", conn->name); + director_cmd_error(conn, "Invalid parameters"); return FALSE; } @@ -556,7 +570,7 @@ if (str_array_length(args) != 1 || net_addr2ip(args[0], &ip) < 0) { - i_error("director(%s): Invalid HOST-FLUSH args", conn->name); + director_cmd_error(conn, "Invalid parameters"); return FALSE; } @@ -582,7 +596,7 @@ if (str_array_length(args) != 2 || str_to_uint(args[0], &username_hash) < 0 || net_addr2ip(args[1], &ip) < 0) { - i_error("director(%s): Invalid USER-MOVE args", conn->name); + director_cmd_error(conn, "Invalid parameters"); return FALSE; } @@ -602,7 +616,7 @@ if (str_array_length(args) != 1 || str_to_uint(args[0], &username_hash) < 0) { - i_error("director(%s): Invalid USER-KILLED args", conn->name); + director_cmd_error(conn, "Invalid parameters"); return FALSE; } @@ -623,8 +637,7 @@ if (str_array_length(args) != 1 || str_to_uint(args[0], &username_hash) < 0) { - i_error("director(%s): Invalid USER-KILLED-EVERYWHERE args", - conn->name); + director_cmd_error(conn, "Invalid parameters"); return FALSE; } @@ -706,17 +719,15 @@ return TRUE; } if (!conn->version_received) { - i_error("director(%s): Incompatible protocol", conn->name); + director_cmd_error(conn, "Incompatible protocol"); return FALSE; } - if (strcmp(cmd, "ME") == 0 && !conn->me_received && - str_array_length(args) == 2) + if (strcmp(cmd, "ME") == 0 && !conn->me_received) return director_cmd_me(conn, args); /* only outgoing connections get a CONNECT reference */ - if (!conn->in && strcmp(cmd, "CONNECT") == 0 && - str_array_length(args) == 2) { + if (!conn->in && strcmp(cmd, "CONNECT") == 0) { /* remote wants us to connect elsewhere */ if (!director_args_parse_ip_port(conn, args, &ip, &port)) return FALSE; @@ -827,7 +838,7 @@ if (str_array_length(args) < 3 || !director_args_parse_ip_port(conn, args, &ip, &port) || str_to_uint(args[2], &seq) < 0) { - i_error("director(%s): Invalid SYNC args", conn->name); + director_cmd_error(conn, "Invalid parameters"); return FALSE; } if (args[3] != NULL) @@ -856,7 +867,7 @@ if (str_array_length(args) != 2 || !director_args_parse_ip_port(conn, args, &ip, &port)) { - i_error("director(%s): Invalid CONNECT args", conn->name); + director_cmd_error(conn, "Invalid parameters"); return FALSE; } @@ -907,18 +918,9 @@ } static bool -director_connection_handle_line(struct director_connection *conn, - const char *line) +director_connection_handle_cmd(struct director_connection *conn, + const char *cmd, const char *const *args) { - const char *cmd, *const *args; - - args = t_strsplit(line, "\t"); - cmd = args[0]; args++; - if (cmd == NULL) { - i_error("director(%s): Received empty line", conn->name); - return FALSE; - } - /* ping/pong is always handled */ if (strcmp(cmd, "PING") == 0) { director_connection_send(conn, "PONG\n"); @@ -965,11 +967,32 @@ if (strcmp(cmd, "CONNECT") == 0) return director_cmd_connect(conn, args); - i_error("director(%s): Unknown command (in this state): %s", - conn->name, cmd); + director_cmd_error(conn, "Unknown command %s", cmd); return FALSE; } +static bool +director_connection_handle_line(struct director_connection *conn, + const char *line) +{ + const char *cmd, *const *args; + bool ret; + + args = t_strsplit(line, "\t"); + cmd = args[0]; args++; + if (cmd == NULL) { + i_error("director(%s): Received empty line", conn->name); + return FALSE; + } + + conn->cur_cmd = cmd; + conn->cur_line = line; + ret = director_connection_handle_cmd(conn, cmd, args); + conn->cur_cmd = NULL; + conn->cur_line = NULL; + return ret; +} + static void director_connection_input(struct director_connection *conn) { struct director *dir = conn->dir; From dovecot at dovecot.org Sun May 20 03:26:30 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:30 +0300 Subject: dovecot-2.2: imap: Enabling SPECIAL-USE selection option should ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/7f4c35fcae80 changeset: 14404:7f4c35fcae80 user: Timo Sirainen date: Mon Apr 02 21:49:05 2012 +0300 description: imap: Enabling SPECIAL-USE selection option should also enable the same return option. diffstat: src/imap/cmd-list.c | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diffs (17 lines): diff -r 6a8b78450202 -r 7f4c35fcae80 src/imap/cmd-list.c --- a/src/imap/cmd-list.c Fri Mar 30 05:20:08 2012 +0300 +++ b/src/imap/cmd-list.c Mon Apr 02 21:49:05 2012 +0300 @@ -115,9 +115,10 @@ MAILBOX_LIST_ITER_RETURN_SUBSCRIBED; } else if (strcasecmp(str, "RECURSIVEMATCH") == 0) list_flags |= MAILBOX_LIST_ITER_SELECT_RECURSIVEMATCH; - else if (strcasecmp(str, "SPECIAL-USE") == 0) - list_flags |= MAILBOX_LIST_ITER_SELECT_SPECIALUSE; - else if (strcasecmp(str, "REMOTE") == 0) { + else if (strcasecmp(str, "SPECIAL-USE") == 0) { + list_flags |= MAILBOX_LIST_ITER_SELECT_SPECIALUSE | + MAILBOX_LIST_ITER_RETURN_SPECIALUSE; + } else if (strcasecmp(str, "REMOTE") == 0) { /* not supported, ignore */ } else { /* skip also optional list value */ From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: director: Redesigned connection handling and error ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/b43ae3805f5f changeset: 14409:b43ae3805f5f user: Timo Sirainen date: Tue Apr 03 05:58:29 2012 +0300 description: director: Redesigned connection handling and error handling. Director now accepts all connections from everywhere and syncs them until the handshaking is finished. At that point it finally decides if this is a connection that should be used as our left/right connection, or if it should be disconnected. This should make connecting more reliable, especially if one of the directors sends broken handshake or has other trouble. diffstat: src/director/director-connection.c | 539 ++++++++++++++++++++++-------------- src/director/director-connection.h | 4 +- src/director/director.c | 40 +- src/director/director.h | 4 + 4 files changed, 357 insertions(+), 230 deletions(-) diffs (truncated from 929 to 300 lines): diff -r 544a8c4705e5 -r b43ae3805f5f src/director/director-connection.c --- a/src/director/director-connection.c Tue Apr 03 00:50:12 2012 +0300 +++ b/src/director/director-connection.c Tue Apr 03 05:58:29 2012 +0300 @@ -7,8 +7,10 @@ VERSION ME - + DONE + Outgoing director connections send: @@ -21,6 +23,9 @@ [0..n] USER DONE + + */ #include "lib.h" @@ -44,10 +49,16 @@ #define MAX_INBUF_SIZE 1024 #define MAX_OUTBUF_SIZE (1024*1024*10) #define OUTBUF_FLUSH_THRESHOLD (1024*128) -/* Max idling time while connecting/handshaking before disconnecting */ -#define DIRECTOR_CONNECTION_INIT_TIMEOUT_MSECS (10*1000) -/* How long to wait for PONG after PING request */ -#define DIRECTOR_CONNECTION_PING_TIMEOUT_MSECS (10*1000) +/* Max idling time before "ME" command must have been received, + or we'll disconnect. */ +#define DIRECTOR_CONNECTION_ME_TIMEOUT_MSECS (2*1000) +/* Max idling time before "DONE" command must have been received, + or we'll disconnect. */ +#define DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS (30*1000) +/* How long to wait for PONG for an idling connection */ +#define DIRECTOR_CONNECTION_PING_IDLE_TIMEOUT_MSECS (2*1000) +/* Maximum time to wait for PONG reply */ +#define DIRECTOR_CONNECTION_PONG_TIMEOUT_MSECS (60*1000) /* How long to wait to send PING when connection is idle */ #define DIRECTOR_CONNECTION_PING_INTERVAL_MSECS (15*1000) /* How long to wait before sending PING while waiting for SYNC reply */ @@ -56,6 +67,14 @@ mark the host as failed so we won't try to reconnect to it immediately */ #define DIRECTOR_SUCCESS_MIN_CONNECT_SECS 10 +#if DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS <= DIRECTOR_CONNECTION_PING_TIMEOUT_MSECS +# error DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS is too low +#endif + +#if DIRECTOR_CONNECTION_PONG_TIMEOUT_MSECS <= DIRECTOR_CONNECTION_PING_IDLE_TIMEOUT_MSECS +# error DIRECTOR_CONNECTION_PONG_TIMEOUT_MSECS is too low +#endif + #define CMD_IS_USER_HANDHAKE(args) \ (str_array_length(args) > 2) @@ -72,7 +91,7 @@ struct io *io; struct istream *input; struct ostream *output; - struct timeout *to, *to_ping; + struct timeout *to, *to_ping, *to_pong; struct user_directory_iter *user_iter; @@ -88,9 +107,10 @@ unsigned int handshake_sending_hosts:1; unsigned int ping_waiting:1; unsigned int synced:1; + unsigned int wrong_host:1; + unsigned int verifying_left:1; }; -static void director_connection_ping(struct director_connection *conn); static void director_connection_disconnected(struct director_connection **conn); static void ATTR_FORMAT(2, 3) @@ -104,6 +124,172 @@ va_end(args); } +static void +director_connection_init_timeout(struct director_connection *conn) +{ + unsigned int secs = ioloop_time - conn->created; + + if (!conn->connected) { + i_error("director(%s): Connect timed out (%u secs)", + conn->name, secs); + } else if (!conn->me_received) { + i_error("director(%s): Handshaking ME timed out (%u secs)", + conn->name, secs); + } else { + i_error("director(%s): Handshaking DONE timed out (%u secs)", + conn->name, secs); + } + director_connection_disconnected(&conn); +} + +static void +director_connection_set_ping_timeout(struct director_connection *conn) +{ + unsigned int msecs; + + msecs = conn->synced || !conn->handshake_received ? + DIRECTOR_CONNECTION_PING_INTERVAL_MSECS : + DIRECTOR_CONNECTION_PING_SYNC_INTERVAL_MSECS; + + timeout_remove(&conn->to_ping); + conn->to_ping = timeout_add(msecs, director_connection_ping, conn); +} + +static void director_connection_send_connect(struct director_connection *conn, + struct director_host *host) +{ + const char *connect_str; + + connect_str = t_strdup_printf("CONNECT\t%s\t%u\n", + net_ip2addr(&host->ip), host->port); + director_connection_send(conn, connect_str); + (void)o_stream_flush(conn->output); + o_stream_uncork(conn->output); +} + +static void director_connection_assigned(struct director_connection *conn) +{ + struct director *dir = conn->dir; + + if (dir->left != NULL && dir->right != NULL) { + /* we're connected to both directors. see if the ring is + finished by sending a SYNC. if we get it back, it's done. */ + dir->sync_seq++; + director_set_ring_unsynced(dir); + director_sync_send(dir, dir->self_host, dir->sync_seq, + DIRECTOR_VERSION_MINOR); + } + director_connection_set_ping_timeout(conn); +} + +static bool director_connection_assign_left(struct director_connection *conn) +{ + struct director *dir = conn->dir; + + i_assert(conn->in); + i_assert(dir->left != conn); + + /* make sure this is the correct incoming connection */ + if (conn->host->self) { + i_error("Connection from self, dropping"); + return FALSE; + } else if (dir->left == NULL) { + /* no conflicts yet */ + } else if (dir->left->host == conn->host) { + i_info("Dropping existing connection %s " + "in favor of its new connection %s", + dir->left->host->name, conn->host->name); + director_connection_deinit(&dir->left); + } else if (dir->left->verifying_left) { + /* we're waiting to verify if our current left is still + working. if we don't receive a PONG, the current left + gets disconnected and a new left gets assigned. if we do + receive a PONG, we'll wait until the current left + disconnects us and then reassign the new left. */ + return TRUE; + } else if (director_host_cmp_to_self(dir->left->host, conn->host, + dir->self_host) < 0) { + /* the old connection is the correct one. + refer the client there (FIXME: do we ever get here?) */ + i_warning("Director connection %s tried to connect to " + "us, should use %s instead", + conn->name, dir->left->host->name); + director_connection_send_connect(conn, dir->left->host); + return FALSE; + } else { + /* this new connection is the correct one, but wait until the + old connection gets disconnected before using this one. + that guarantees that the director inserting itself into + the ring has finished handshaking its left side, so the + switch will be fast. */ + return TRUE; + } + dir->left = conn; + i_free(conn->name); + conn->name = i_strdup_printf("%s/left", conn->host->name); + director_connection_assigned(conn); + return TRUE; +} + +static void director_assign_left(struct director *dir) +{ + struct director_connection *conn, *const *connp; + + array_foreach(&dir->connections, connp) { + conn = *connp; + + if (conn->in && conn->handshake_received && conn != dir->left) { + /* either use this or disconnect it */ + if (!director_connection_assign_left(conn)) { + /* we don't want this */ + director_connection_deinit(&dir->left); + director_assign_left(dir); + break; + } + } + } +} + +static bool director_has_outgoing_connections(struct director *dir) +{ + struct director_connection *const *connp; + + array_foreach(&dir->connections, connp) { + if (!(*connp)->in) + return TRUE; + } + return FALSE; +} + +static bool director_connection_assign_right(struct director_connection *conn) +{ + struct director *dir = conn->dir; + + i_assert(!conn->in); + + if (dir->right != NULL) { + /* see if we should disconnect or keep the existing + connection. */ + if (director_host_cmp_to_self(conn->host, dir->right->host, + dir->self_host) <= 0) { + /* the old connection is the correct one */ + i_warning("Aborting incorrect outgoing connection to %s " + "(already connected to correct one: %s)", + conn->host->name, dir->right->host->name); + conn->wrong_host = TRUE; + return FALSE; + } + i_info("Replacing right director connection %s with %s", + dir->right->host->name, conn->host->name); + director_connection_deinit(&dir->right); + } + dir->right = conn; + i_free(conn->name); + conn->name = i_strdup_printf("%s/right", conn->host->name); + director_connection_assigned(conn); + return TRUE; +} + static bool director_args_parse_ip_port(struct director_connection *conn, const char *const *args, @@ -128,7 +314,6 @@ const char *const *args) { struct director *dir = conn->dir; - struct director_host *host; const char *connect_str; struct ip_addr ip; unsigned int port; @@ -148,75 +333,55 @@ net_ip2addr(&ip), port); return FALSE; } - host = director_host_get(dir, &ip, port); - /* the host is up now, make sure we can connect to it immediately - if needed */ - host->last_failed = 0; conn->me_received = TRUE; + timeout_remove(&conn->to_ping); + conn->to_ping = timeout_add(DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS, + director_connection_init_timeout, conn); + if (!conn->in) return TRUE; - i_free(conn->name); - conn->name = i_strdup_printf("%s/left", host->name); - conn->host = host; + /* Incoming connection: + + a) we don't have an established ring yet. make sure we're connecting + to our right side (which might become our left side). + + b) it's our current "left" connection. the previous connection + is most likely dead. + + c) we have an existing ring. tell our current "left" to connect to + it with CONNECT command. + + d) the incoming connection doesn't belong to us at all, refer it + elsewhere with CONNECT. however, before disconnecting it verify + first that our left side is actually still functional. + */ + conn->host = director_host_get(dir, &ip, port); /* make sure we don't keep old sequence values across restarts */ From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: lib-index: Don't assert-crash with broken extension... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/47526bf86c57 changeset: 14410:47526bf86c57 user: Timo Sirainen date: Wed Apr 04 03:44:04 2012 +0300 description: lib-index: Don't assert-crash with broken extension record intros in log. diffstat: src/lib-index/mail-index-map.c | 10 +++++++--- 1 files changed, 7 insertions(+), 3 deletions(-) diffs (27 lines): diff -r b43ae3805f5f -r 47526bf86c57 src/lib-index/mail-index-map.c --- a/src/lib-index/mail-index-map.c Tue Apr 03 05:58:29 2012 +0300 +++ b/src/lib-index/mail-index-map.c Wed Apr 04 03:44:04 2012 +0300 @@ -174,6 +174,12 @@ return -1; } + if (ext_hdr->record_offset == 0) { + /* if we get here from extension introduction, record_offset=0 + and hdr->record_size hasn't been updated yet */ + return 0; + } + if (ext_hdr->record_offset + ext_hdr->record_size > hdr->record_size) { *error_r = t_strdup_printf("Record field points " "outside record size (%u+%u > %u)", @@ -210,9 +216,7 @@ return -1; } - /* if we get here from extension introduction, record_offset=0 and - hdr->record_size hasn't been updated yet */ - if (ext_hdr->record_offset != 0 && ext_hdr->record_size != 0) { + if (ext_hdr->record_size != 0) { if (mail_index_map_ext_hdr_check_record(hdr, ext_hdr, error_r) < 0) return -1; From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: doveadm mailbox delete: -s parameter wasn't working. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/411344f9daf3 changeset: 14411:411344f9daf3 user: Timo Sirainen date: Wed Apr 04 03:47:52 2012 +0300 description: doveadm mailbox delete: -s parameter wasn't working. diffstat: src/doveadm/doveadm-mail-mailbox.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diffs (22 lines): diff -r 47526bf86c57 -r 411344f9daf3 src/doveadm/doveadm-mail-mailbox.c --- a/src/doveadm/doveadm-mail-mailbox.c Wed Apr 04 03:44:04 2012 +0300 +++ b/src/doveadm/doveadm-mail-mailbox.c Wed Apr 04 03:47:52 2012 +0300 @@ -371,6 +371,9 @@ case 'r': ctx->recursive = TRUE; break; + case 's': + ctx->ctx.subscriptions = TRUE; + break; default: return FALSE; } @@ -385,7 +388,7 @@ ctx->ctx.ctx.v.init = cmd_mailbox_delete_init; ctx->ctx.ctx.v.run = cmd_mailbox_delete_run; ctx->ctx.ctx.v.parse_arg = cmd_mailbox_delete_parse_arg; - ctx->ctx.ctx.getopt_args = "r"; + ctx->ctx.ctx.getopt_args = "rs"; p_array_init(&ctx->mailboxes, ctx->ctx.ctx.pool, 16); return &ctx->ctx.ctx; } From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: lib-storage: Fixed setting usable/unusable flags fo... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/4fe357d34faf changeset: 14412:4fe357d34faf user: Timo Sirainen date: Wed Apr 04 04:31:09 2012 +0300 description: lib-storage: Fixed setting usable/unusable flags for shared namespaces. If a namespace doesn't have a root that exists, mark it as unusable. This avoids ACL plugin from autocreating that directory. diffstat: src/lib-storage/index/shared/shared-storage.c | 19 ++++++++++++++++++- 1 files changed, 18 insertions(+), 1 deletions(-) diffs (36 lines): diff -r 411344f9daf3 -r 4fe357d34faf src/lib-storage/index/shared/shared-storage.c --- a/src/lib-storage/index/shared/shared-storage.c Wed Apr 04 03:47:52 2012 +0300 +++ b/src/lib-storage/index/shared/shared-storage.c Wed Apr 04 04:31:09 2012 +0300 @@ -120,6 +120,19 @@ str_append(location, username); } +static bool shared_namespace_exists(struct mail_namespace *ns) +{ + const char *path; + struct stat st; + + path = mailbox_list_get_path(ns->list, NULL, MAILBOX_LIST_PATH_TYPE_DIR); + if (path == NULL) { + /* we can't know if this exists */ + return TRUE; + } + return stat(path, &st) == 0; +} + int shared_storage_get_namespace(struct mail_namespace **_ns, const char **_name) { @@ -310,7 +323,11 @@ mail_namespace_destroy(new_ns); return -1; } - ns->flags |= NAMESPACE_FLAG_USABLE; + if ((new_ns->flags & NAMESPACE_FLAG_UNUSABLE) == 0 && + !shared_namespace_exists(new_ns)) { + /* this user doesn't have a usable storage */ + new_ns->flags |= NAMESPACE_FLAG_UNUSABLE; + } *_name = mailbox_list_get_storage_name(new_ns->list, t_strconcat(new_ns->prefix, name, NULL)); *_ns = new_ns; From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: director: Handle all commands during handshake. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/544a8c4705e5 changeset: 14408:544a8c4705e5 user: Timo Sirainen date: Tue Apr 03 00:50:12 2012 +0300 description: director: Handle all commands during handshake. Previously the sender might have sent these commands, but the receiver wouldn't have handled them and instead just disconnected. diffstat: src/director/director-connection.c | 166 +++++++++++++++++++++++------------- src/director/director-connection.h | 1 + src/director/director.c | 8 +- 3 files changed, 114 insertions(+), 61 deletions(-) diffs (truncated from 317 to 300 lines): diff -r d6cd93e32b37 -r 544a8c4705e5 src/director/director-connection.c --- a/src/director/director-connection.c Tue Apr 03 00:23:02 2012 +0300 +++ b/src/director/director-connection.c Tue Apr 03 00:50:12 2012 +0300 @@ -1,5 +1,28 @@ /* Copyright (c) 2010-2012 Dovecot authors, see the included COPYING file */ +/* + Handshaking: + + Incoming director connections send: + + VERSION + ME + + DONE + + Outgoing director connections send: + + VERSION + ME + [0..n] DIRECTOR + HOST-HAND-START + [0..n] HOST + HOST-HAND-END + [0..n] USER + + DONE +*/ + #include "lib.h" #include "ioloop.h" #include "array.h" @@ -33,6 +56,9 @@ mark the host as failed so we won't try to reconnect to it immediately */ #define DIRECTOR_SUCCESS_MIN_CONNECT_SECS 10 +#define CMD_IS_USER_HANDHAKE(args) \ + (str_array_length(args) > 2) + struct director_connection { struct director *dir; char *name; @@ -109,6 +135,10 @@ if (!director_args_parse_ip_port(conn, args, &ip, &port)) return FALSE; + if (conn->me_received) { + director_cmd_error(conn, "Duplicate ME"); + return FALSE; + } if (!conn->in && (!net_ip_compare(&conn->host->ip, &ip) || conn->host->port != port)) { @@ -317,6 +347,8 @@ struct mail_host *host; struct user *user; + /* NOTE: if more parameters are added, update also + CMD_IS_USER_HANDHAKE() macro */ if (str_array_length(args) != 2 || str_to_uint(args[0], &username_hash) < 0 || net_addr2ip(args[1], &ip) < 0) { @@ -694,7 +726,7 @@ director_connection_set_ping_timeout(conn); } -static bool +static int director_connection_handle_handshake(struct director_connection *conn, const char *cmd, const char *const *args) { @@ -708,29 +740,33 @@ i_error("director(%s): Wrong protocol in socket " "(%s vs %s)", conn->name, args[0], DIRECTOR_VERSION_NAME); - return FALSE; + return -1; } else if (atoi(args[1]) != DIRECTOR_VERSION_MAJOR) { i_error("director(%s): Incompatible protocol version: " "%u vs %u", conn->name, atoi(args[1]), DIRECTOR_VERSION_MAJOR); - return FALSE; + return -1; } conn->version_received = TRUE; - return TRUE; + return 1; } if (!conn->version_received) { director_cmd_error(conn, "Incompatible protocol"); - return FALSE; + return -1; } - if (strcmp(cmd, "ME") == 0 && !conn->me_received) - return director_cmd_me(conn, args); + if (strcmp(cmd, "ME") == 0) + return director_cmd_me(conn, args) ? 1 : -1; - /* only outgoing connections get a CONNECT reference */ - if (!conn->in && strcmp(cmd, "CONNECT") == 0) { + if (strcmp(cmd, "CONNECT") == 0) { /* remote wants us to connect elsewhere */ if (!director_args_parse_ip_port(conn, args, &ip, &port)) - return FALSE; + return -1; + if (conn->in) { + director_cmd_error(conn, + "Incoming connections can't request CONNECT"); + return -1; + } conn->dir->right = NULL; host = director_host_get(conn->dir, &ip, port); @@ -740,53 +776,44 @@ if (conn->dir->debug) i_debug("Received CONNECT reference to %s", host->name); (void)director_connect_host(conn->dir, host); - return FALSE; + return 1; } - /* only incoming connections get DIRECTOR and HOST lists */ - if (conn->in && strcmp(cmd, "DIRECTOR") == 0 && conn->me_received) - return director_cmd_director(conn, args); + /* Only VERSION and CONNECT commands are allowed before ME */ + if (!conn->me_received) { + director_cmd_error(conn, "Expecting ME command first"); + return -1; + } - if (strcmp(cmd, "HOST") == 0) { - /* allow hosts from all connections always, - this could be an host update */ - if (conn->handshake_sending_hosts) - return director_cmd_host_handshake(conn, args); - else - return director_cmd_host(conn, args); + /* incoming connections get a HOST list */ + if (conn->handshake_sending_hosts) { + if (strcmp(cmd, "HOST") == 0) + return director_cmd_host_handshake(conn, args) ? 1 : -1; + if (strcmp(cmd, "HOST-HAND-END") == 0) { + conn->ignore_host_events = FALSE; + conn->handshake_sending_hosts = FALSE; + return 1; + } + director_cmd_error(conn, "Unexpected command during host list"); + return -1; } - if (conn->handshake_sending_hosts && - strcmp(cmd, "HOST-HAND-END") == 0) { - conn->ignore_host_events = FALSE; - conn->handshake_sending_hosts = FALSE; - return TRUE; + if (strcmp(cmd, "HOST-HAND-START") == 0) { + if (!conn->in) { + director_cmd_error(conn, + "Host list is only for incoming connections"); + return -1; + } + return director_cmd_host_hand_start(conn, args) ? 1 : -1; } - if (conn->in && strcmp(cmd, "HOST-HAND-START") == 0 && - conn->me_received) - return director_cmd_host_hand_start(conn, args); - /* only incoming connections get a full USER list, but outgoing - connections can also receive USER updates during handshake and - it wouldn't be safe to ignore them. */ - if (!conn->me_received) { - /* no USER updates until ME */ - } else if (conn->in && strcmp(cmd, "USER") == 0) { - return director_handshake_cmd_user(conn, args); - } else if (!conn->in) { - if (strcmp(cmd, "USER") == 0) - return director_cmd_user(conn, args); - if (strcmp(cmd, "USER-WEAK") == 0) - return director_cmd_user_weak(conn, args); + if (conn->in && strcmp(cmd, "USER") == 0 && CMD_IS_USER_HANDHAKE(args)) + return director_handshake_cmd_user(conn, args) ? 1 : -1; + + /* both get DONE */ + if (strcmp(cmd, "DONE") == 0) { + director_handshake_cmd_done(conn); + return 1; } - /* both get DONE */ - if (strcmp(cmd, "DONE") == 0 && !conn->handshake_received && - !conn->handshake_sending_hosts) { - director_handshake_cmd_done(conn); - return TRUE; - } - i_error("director(%s): Invalid handshake command: %s " - "(in=%d me_received=%d)", conn->name, cmd, - conn->in, conn->me_received); - return FALSE; + return 0; } static void @@ -806,6 +833,9 @@ /* stale SYNC event */ return; } + /* sync_seq increases when we get disconnected, so we must be + successfully connected to both directions */ + i_assert(dir->left != NULL && dir->right != NULL); dir->ring_min_version = minor_version; if (!dir->ring_handshaked) { @@ -921,6 +951,8 @@ director_connection_handle_cmd(struct director_connection *conn, const char *cmd, const char *const *args) { + int ret; + /* ping/pong is always handled */ if (strcmp(cmd, "PING") == 0) { director_connection_send(conn, "PONG\n"); @@ -930,7 +962,10 @@ return director_cmd_pong(conn); if (!conn->handshake_received) { - if (!director_connection_handle_handshake(conn, cmd, args)) { + ret = director_connection_handle_handshake(conn, cmd, args); + if (ret > 0) + return TRUE; + if (ret < 0) { /* invalid commands during handshake, we probably don't want to reconnect here */ if (conn->dir->debug) { @@ -941,7 +976,7 @@ conn->host->last_failed = ioloop_time; return FALSE; } - return TRUE; + /* allow also other commands during handshake */ } if (strcmp(cmd, "USER") == 0) @@ -1103,21 +1138,27 @@ static int director_connection_output(struct director_connection *conn) { - if (conn->user_iter != NULL) + if (conn->user_iter != NULL) { + /* still handshaking USER list */ return director_connection_send_users(conn); - else - return o_stream_flush(conn->output); + } + return o_stream_flush(conn->output); } static void director_connection_init_timeout(struct director_connection *conn) { + unsigned int secs = ioloop_time - conn->created; + if (conn->host != NULL) conn->host->last_failed = ioloop_time; - if (!conn->connected) - i_error("director(%s): Connect timed out", conn->name); - else - i_error("director(%s): Handshaking timed out", conn->name); + if (!conn->connected) { + i_error("director(%s): Connect timed out (%u secs)", + conn->name, secs); + } else { + i_error("director(%s): Handshaking timed out (%u secs)", + conn->name, secs); + } director_connection_disconnected(&conn); } @@ -1356,6 +1397,11 @@ return conn->host; } +bool director_connection_is_handshaked(struct director_connection *conn) +{ + return conn->handshake_received; +} + bool director_connection_is_incoming(struct director_connection *conn) { return conn->in; diff -r d6cd93e32b37 -r 544a8c4705e5 src/director/director-connection.h --- a/src/director/director-connection.h Tue Apr 03 00:23:02 2012 +0300 +++ b/src/director/director-connection.h Tue Apr 03 00:50:12 2012 +0300 @@ -23,6 +23,7 @@ const char *director_connection_get_name(struct director_connection *conn); struct director_host * director_connection_get_host(struct director_connection *conn); +bool director_connection_is_handshaked(struct director_connection *conn); bool director_connection_is_incoming(struct director_connection *conn); void director_connection_cork(struct director_connection *conn); diff -r d6cd93e32b37 -r 544a8c4705e5 src/director/director.c --- a/src/director/director.c Tue Apr 03 00:23:02 2012 +0300 From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: lib-storage: Previous change incorrectly reverted s... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/2bcab08dc994 changeset: 14413:2bcab08dc994 user: Timo Sirainen date: Wed Apr 04 04:33:41 2012 +0300 description: lib-storage: Previous change incorrectly reverted setting parent shared namespace usable. diffstat: src/lib-storage/index/shared/shared-storage.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diffs (13 lines): diff -r 4fe357d34faf -r 2bcab08dc994 src/lib-storage/index/shared/shared-storage.c --- a/src/lib-storage/index/shared/shared-storage.c Wed Apr 04 04:31:09 2012 +0300 +++ b/src/lib-storage/index/shared/shared-storage.c Wed Apr 04 04:33:41 2012 +0300 @@ -328,6 +328,9 @@ /* this user doesn't have a usable storage */ new_ns->flags |= NAMESPACE_FLAG_UNUSABLE; } + /* mark the shared namespace root as usable, since it now has + child namespaces */ + ns->flags |= NAMESPACE_FLAG_USABLE; *_name = mailbox_list_get_storage_name(new_ns->list, t_strconcat(new_ns->prefix, name, NULL)); *_ns = new_ns; From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: sdbox: If copying with link() fails with ENOENT, fa... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/cc734436c140 changeset: 14414:cc734436c140 user: Timo Sirainen date: Wed Apr 04 05:15:49 2012 +0300 description: sdbox: If copying with link() fails with ENOENT, fallback to trying regular copying. This also fixes missing error reporting for that case. diffstat: src/lib-storage/index/dbox-single/sdbox-copy.c | 9 ++++++--- 1 files changed, 6 insertions(+), 3 deletions(-) diffs (19 lines): diff -r 2bcab08dc994 -r cc734436c140 src/lib-storage/index/dbox-single/sdbox-copy.c --- a/src/lib-storage/index/dbox-single/sdbox-copy.c Wed Apr 04 04:33:41 2012 +0300 +++ b/src/lib-storage/index/dbox-single/sdbox-copy.c Wed Apr 04 05:15:49 2012 +0300 @@ -104,9 +104,12 @@ if (ret < 0) { if (ECANTLINK(errno)) ret = 0; - else if (errno == ENOENT) - mail_set_expunged(mail); - else { + else if (errno == ENOENT) { + /* try if the fallback copying code can still + read the file (the mail could still have the + stream open) */ + ret = 0; + } else { mail_storage_set_critical( _ctx->transaction->box->storage, "link(%s, %s) failed: %m", From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: maildir: Remember the mail's GUID/filename in memor... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/96ce9c0c6f05 changeset: 14415:96ce9c0c6f05 user: Timo Sirainen date: Wed Apr 04 05:34:11 2012 +0300 description: maildir: Remember the mail's GUID/filename in memory once it's fetched once. This allows retrieving it later even if the message gets expunged. diffstat: src/lib-storage/index/index-mail.h | 2 +- src/lib-storage/index/maildir/maildir-mail.c | 21 ++++++++++++++++----- 2 files changed, 17 insertions(+), 6 deletions(-) diffs (67 lines): diff -r cc734436c140 -r 96ce9c0c6f05 src/lib-storage/index/index-mail.h --- a/src/lib-storage/index/index-mail.h Wed Apr 04 05:15:49 2012 +0300 +++ b/src/lib-storage/index/index-mail.h Wed Apr 04 05:34:11 2012 +0300 @@ -77,7 +77,7 @@ uint32_t parse_line_num; struct message_part *parts; - const char *envelope, *body, *bodystructure, *uid_string, *guid; + const char *envelope, *body, *bodystructure, *guid, *filename; const char *from_envelope; struct message_part_envelope_data *envelope_data; diff -r cc734436c140 -r 96ce9c0c6f05 src/lib-storage/index/maildir/maildir-mail.c --- a/src/lib-storage/index/maildir/maildir-mail.c Wed Apr 04 05:15:49 2012 +0300 +++ b/src/lib-storage/index/maildir/maildir-mail.c Wed Apr 04 05:34:11 2012 +0300 @@ -482,6 +482,11 @@ /* use GUID from uidlist if it exists */ i_assert(!_mail->saving); + if (mail->data.guid != NULL) { + *value_r = mail->data.guid; + return 0; + } + /* first make sure that we have a refreshed uidlist */ if (maildir_mail_get_fname(mbox, _mail, &fname) <= 0) return -1; @@ -490,7 +495,8 @@ MAILDIR_UIDLIST_REC_EXT_GUID); if (guid != NULL) { if (*guid != '\0') { - *value_r = p_strdup(mail->data_pool, guid); + *value_r = mail->data.guid = + p_strdup(mail->data_pool, guid); return 0; } @@ -503,9 +509,14 @@ } /* default to base filename: */ + if (maildir_mail_get_special(_mail, MAIL_FETCH_UIDL_FILE_NAME, + value_r) < 0) + return -1; + mail->data.guid = mail->data.filename; + return 0; case MAIL_FETCH_UIDL_FILE_NAME: - if (mail->data.guid != NULL) { - *value_r = mail->data.guid; + if (mail->data.filename != NULL) { + *value_r = mail->data.filename; return 0; } if (fname != NULL) { @@ -521,10 +532,10 @@ fname = fname != NULL ? fname + 1 : path; } end = strchr(fname, MAILDIR_INFO_SEP); - mail->data.guid = end == NULL ? + mail->data.filename = end == NULL ? p_strdup(mail->data_pool, fname) : p_strdup_until(mail->data_pool, fname, end); - *value_r = mail->data.guid; + *value_r = mail->data.filename; return 0; case MAIL_FETCH_UIDL_BACKEND: uidl = maildir_uidlist_lookup_ext(mbox->uidlist, _mail->uid, From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: lib-lda: mail_deliver()'s dest_mail now prefetches ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/c93601fa444e changeset: 14416:c93601fa444e user: Timo Sirainen date: Wed Apr 04 05:34:51 2012 +0300 description: lib-lda: mail_deliver()'s dest_mail now prefetches message's GUID. This fixes copying an already expunged mail with maildir. diffstat: src/lib-lda/mail-deliver.c | 8 ++++++-- 1 files changed, 6 insertions(+), 2 deletions(-) diffs (25 lines): diff -r 96ce9c0c6f05 -r c93601fa444e src/lib-lda/mail-deliver.c --- a/src/lib-lda/mail-deliver.c Wed Apr 04 05:34:11 2012 +0300 +++ b/src/lib-lda/mail-deliver.c Wed Apr 04 05:34:51 2012 +0300 @@ -268,7 +268,7 @@ struct mailbox_header_lookup_ctx *headers_ctx; struct mail_keywords *kw; enum mail_error error; - const char *mailbox_name, *errstr; + const char *mailbox_name, *errstr, *guid; struct mail_transaction_commit_changes changes; const struct seq_range *range; bool default_save; @@ -338,7 +338,11 @@ t = mailbox_transaction_begin(box, 0); ctx->dest_mail = mail_alloc(t, MAIL_FETCH_STREAM_BODY, NULL); - if (!mail_set_uid(ctx->dest_mail, range[0].seq1)) { + /* copying needs the message body. with maildir we also + need to get the GUID in case the message gets + expunged */ + if (!mail_set_uid(ctx->dest_mail, range[0].seq1) || + mail_get_special(ctx->dest_mail, MAIL_FETCH_GUID, &guid) < 0) { mail_free(&ctx->dest_mail); mailbox_transaction_rollback(&t); } From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: ldap: Treat =key the same as =key=, just like v2.0. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/9930c75b989a changeset: 14417:9930c75b989a user: Timo Sirainen date: Wed Apr 04 06:10:33 2012 +0300 description: ldap: Treat =key the same as =key=, just like v2.0. diffstat: src/auth/db-ldap.c | 11 +++++++---- src/auth/db-ldap.h | 2 +- 2 files changed, 8 insertions(+), 5 deletions(-) diffs (40 lines): diff -r c93601fa444e -r 9930c75b989a src/auth/db-ldap.c --- a/src/auth/db-ldap.c Wed Apr 04 05:34:51 2012 +0300 +++ b/src/auth/db-ldap.c Wed Apr 04 06:10:33 2012 +0300 @@ -1032,9 +1032,12 @@ } templ = strchr(name, '='); - if (templ == NULL) - templ = ""; - else { + if (templ == NULL) { + if (*ldap_attr == '\0') { + /* =foo static value */ + templ = ""; + } + } else { *templ++ = '\0'; str_truncate(tmp_str, 0); var_expand_with_funcs(tmp_str, templ, NULL, @@ -1245,7 +1248,7 @@ values = ctx->val_1_arr; } - if (*field->value == '\0') { + if (field->value == NULL) { /* use the LDAP attribute's value */ } else { /* template */ diff -r c93601fa444e -r 9930c75b989a src/auth/db-ldap.h --- a/src/auth/db-ldap.h Wed Apr 04 05:34:51 2012 +0300 +++ b/src/auth/db-ldap.h Wed Apr 04 06:10:33 2012 +0300 @@ -122,7 +122,7 @@ struct ldap_field { /* Dovecot field name. */ const char *name; - /* Field value template with %vars. "" = same as LDAP value. */ + /* Field value template with %vars. NULL = same as LDAP value. */ const char *value; /* LDAP attribute name, or "" if this is a static field. */ const char *ldap_attr_name; From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: lib-storage: When doing a userdb lookup, use the us... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/2250f11695bc changeset: 14418:2250f11695bc user: Timo Sirainen date: Wed Apr 04 06:12:24 2012 +0300 description: lib-storage: When doing a userdb lookup, use the user's service instead of "lib-storage". diffstat: src/lib-storage/mail-user.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 9930c75b989a -r 2250f11695bc src/lib-storage/mail-user.c --- a/src/lib-storage/mail-user.c Wed Apr 04 06:10:33 2012 +0300 +++ b/src/lib-storage/mail-user.c Wed Apr 04 06:12:24 2012 +0300 @@ -272,7 +272,7 @@ int ret; memset(&info, 0, sizeof(info)); - info.service = "lib-storage"; + info.service = user->service; if (user->local_ip != NULL) info.local_ip = *user->local_ip; if (user->remote_ip != NULL) From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: ldap: Removed warning about changed user_* settings. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/324df4134049 changeset: 14419:324df4134049 user: Timo Sirainen date: Wed Apr 04 06:16:18 2012 +0300 description: ldap: Removed warning about changed user_* settings. This warning can happen even with recommended settings when symlinking the passdb's ldap config to userdb ldap config. diffstat: src/auth/db-ldap.c | 27 --------------------------- src/auth/db-ldap.h | 3 +-- src/auth/passdb-ldap.c | 1 - 3 files changed, 1 insertions(+), 30 deletions(-) diffs (68 lines): diff -r 2250f11695bc -r 324df4134049 src/auth/db-ldap.c --- a/src/auth/db-ldap.c Wed Apr 04 06:12:24 2012 +0300 +++ b/src/auth/db-ldap.c Wed Apr 04 06:16:18 2012 +0300 @@ -1464,33 +1464,6 @@ pool_unref(&conn->pool); } -void db_ldap_check_userdb_warning(struct ldap_connection *conn) -{ - const struct ldap_settings *def = &default_ldap_settings; - const char *set_name; - - if (worker || conn->userdb_used || conn->set.userdb_warning_disable) - return; - - if (strcmp(conn->set.user_attrs, def->user_attrs) != 0) - set_name = "user_attrs"; - else if (strcmp(conn->set.user_filter, def->user_filter) != 0) - set_name = "user_filter"; - else if (strcmp(conn->set.iterate_attrs, def->iterate_attrs) != 0) - set_name = "iterate_attrs"; - else if (strcmp(conn->set.iterate_filter, def->iterate_filter) != 0) - set_name = "iterate_filter"; - else - set_name = NULL; - - if (set_name != NULL) { - i_warning("ldap: Ignoring changed %s in %s, " - "because userdb ldap not used. " - "(If this is intentional, set userdb_warning_disable=yes)", - set_name, conn->config_path); - } -} - #ifndef BUILTIN_LDAP /* Building a plugin */ extern struct passdb_module_interface passdb_ldap_plugin; diff -r 2250f11695bc -r 324df4134049 src/auth/db-ldap.h --- a/src/auth/db-ldap.h Wed Apr 04 06:12:24 2012 +0300 +++ b/src/auth/db-ldap.h Wed Apr 04 06:16:18 2012 +0300 @@ -63,7 +63,7 @@ const char *iterate_filter; const char *default_pass_scheme; - bool userdb_warning_disable; + bool userdb_warning_disable; /* deprecated for now at least */ /* ... */ int ldap_deref, ldap_scope; @@ -172,7 +172,6 @@ struct ldap_connection *db_ldap_init(const char *config_path, bool userdb); void db_ldap_unref(struct ldap_connection **conn); -void db_ldap_check_userdb_warning(struct ldap_connection *conn); int db_ldap_connect(struct ldap_connection *conn); void db_ldap_enable_input(struct ldap_connection *conn, bool enable); diff -r 2250f11695bc -r 324df4134049 src/auth/passdb-ldap.c --- a/src/auth/passdb-ldap.c Wed Apr 04 06:12:24 2012 +0300 +++ b/src/auth/passdb-ldap.c Wed Apr 04 06:16:18 2012 +0300 @@ -432,7 +432,6 @@ /* Credential lookups can't be done with authentication binds */ _module->iface.lookup_credentials = NULL; } - db_ldap_check_userdb_warning(module->conn); } static void passdb_ldap_deinit(struct passdb_module *_module) From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: Makefile: Link libdovecot-storage.so with LINKED_ST... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/8b91367bc3e1 changeset: 14420:8b91367bc3e1 user: Timo Sirainen date: Wed Apr 04 06:25:00 2012 +0300 description: Makefile: Link libdovecot-storage.so with LINKED_STORAGE_LDADD. diffstat: src/lib-storage/Makefile.am | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 324df4134049 -r 8b91367bc3e1 src/lib-storage/Makefile.am --- a/src/lib-storage/Makefile.am Wed Apr 04 06:16:18 2012 +0300 +++ b/src/lib-storage/Makefile.am Wed Apr 04 06:25:00 2012 +0300 @@ -86,7 +86,7 @@ pkglib_LTLIBRARIES = libdovecot-storage.la libdovecot_storage_la_SOURCES = -libdovecot_storage_la_LIBADD = $(shlibs) $(MODULE_LIBS) +libdovecot_storage_la_LIBADD = $(shlibs) $(LINKED_STORAGE_LDADD) $(MODULE_LIBS) libdovecot_storage_la_DEPENDENCIES = $(shlibs) libdovecot_storage_la_LDFLAGS = -export-dynamic From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: doveadm user: Added -m parameter to show some of th... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/601014feade4 changeset: 14421:601014feade4 user: Timo Sirainen date: Wed Apr 04 07:26:37 2012 +0300 description: doveadm user: Added -m parameter to show some of the mail settings. diffstat: src/doveadm/doveadm-auth.c | 114 ++++++++++++++++++++++++++++++++++++-------- 1 files changed, 93 insertions(+), 21 deletions(-) diffs (168 lines): diff -r 8b91367bc3e1 -r 601014feade4 src/doveadm/doveadm-auth.c --- a/src/doveadm/doveadm-auth.c Wed Apr 04 06:25:00 2012 +0300 +++ b/src/doveadm/doveadm-auth.c Wed Apr 04 07:26:37 2012 +0300 @@ -7,10 +7,14 @@ #include "base64.h" #include "str.h" #include "wildcard-match.h" +#include "master-service.h" #include "auth-client.h" #include "auth-master.h" #include "auth-server-connection.h" +#include "mail-storage-service.h" +#include "mail-user.h" #include "doveadm.h" +#include "doveadm-print.h" #include #include @@ -245,19 +249,74 @@ doveadm_exit_code = EX_NOPERM; } +static void cmd_user_mail_input_field(const char *key, const char *value, + const char *show_field) +{ + if (show_field == NULL) { + doveadm_print(key); + doveadm_print(value); + } else if (strcmp(show_field, key) == 0) { + printf("%s\n", value); + } +} + +static int cmd_user_mail_input(struct mail_storage_service_ctx *storage_service, + const struct authtest_input *input, + const char *show_field) +{ + struct mail_storage_service_input service_input; + struct mail_storage_service_user *service_user; + struct mail_user *user; + const struct mail_storage_settings *mail_set; + const char *error; + int ret; + + memset(&service_input, 0, sizeof(service_input)); + service_input.module = "mail"; + service_input.service = input->info.service; + service_input.username = input->username; + service_input.local_ip = input->info.local_ip; + service_input.local_port = input->info.local_port; + service_input.remote_ip = input->info.remote_ip; + service_input.remote_port = input->info.remote_port; + + if ((ret = mail_storage_service_lookup_next(storage_service, &service_input, + &service_user, &user, + &error)) <= 0) + return ret == 0 ? 0 : -1; + + if (show_field == NULL) { + doveadm_print_init(DOVEADM_PRINT_TYPE_TAB); + doveadm_print_header_simple("field"); + doveadm_print_header_simple("value"); + } + + cmd_user_mail_input_field("uid", user->set->mail_uid, show_field); + cmd_user_mail_input_field("gid", user->set->mail_gid, show_field); + cmd_user_mail_input_field("home", user->set->mail_home, show_field); + + mail_set = mail_user_set_get_storage_set(user); + cmd_user_mail_input_field("mail", mail_set->mail_location, show_field); + + mail_user_unref(&user); + mail_storage_service_user_free(&service_user); + return 1; +} + static void cmd_user(int argc, char *argv[]) { const char *auth_socket_path = NULL; struct authtest_input input; const char *show_field = NULL; + struct mail_storage_service_ctx *storage_service = NULL; unsigned int i; - bool have_wildcards; - int c; + bool have_wildcards, mail_fields = FALSE, first = TRUE; + int c, ret; memset(&input, 0, sizeof(input)); input.info.service = "doveadm"; - while ((c = getopt(argc, argv, "a:f:x:")) > 0) { + while ((c = getopt(argc, argv, "a:f:mx:")) > 0) { switch (c) { case 'a': auth_socket_path = optarg; @@ -265,6 +324,9 @@ case 'f': show_field = optarg; break; + case 'm': + mail_fields = TRUE; + break; case 'x': auth_user_info_parse(&input.info, optarg); break; @@ -285,27 +347,37 @@ } } - if (have_wildcards) + if (have_wildcards) { cmd_user_list(auth_socket_path, &input, argv + optind); - else { - bool first = TRUE; + return; + } - while ((input.username = argv[optind++]) != NULL) { - if (first) - first = FALSE; - else - putchar('\n'); - switch (cmd_user_input(auth_socket_path, &input, - show_field)) { - case -1: - doveadm_exit_code = EX_TEMPFAIL; - break; - case 0: - doveadm_exit_code = EX_NOUSER; - break; - } + if (mail_fields) { + storage_service = mail_storage_service_init(master_service, NULL, + MAIL_STORAGE_SERVICE_FLAG_NO_LOG_INIT | + MAIL_STORAGE_SERVICE_FLAG_USERDB_LOOKUP); + } + + while ((input.username = argv[optind++]) != NULL) { + if (first) + first = FALSE; + else + putchar('\n'); + + ret = mail_fields ? + cmd_user_mail_input(storage_service, &input, show_field) : + cmd_user_input(auth_socket_path, &input, show_field); + switch (ret) { + case -1: + doveadm_exit_code = EX_TEMPFAIL; + break; + case 0: + doveadm_exit_code = EX_NOUSER; + break; } } + if (storage_service != NULL) + mail_storage_service_deinit(&storage_service); } struct doveadm_cmd doveadm_cmd_auth = { @@ -315,5 +387,5 @@ struct doveadm_cmd doveadm_cmd_user = { cmd_user, "user", - "[-a ] [-x ] [-f field] [...]" + "[-a ] [-x ] [-f field] [-m] [...]" }; From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: lib-storage: Require at least one namespace to be n... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/1a33cc98666a changeset: 14422:1a33cc98666a user: Timo Sirainen date: Wed Apr 04 07:46:09 2012 +0300 description: lib-storage: Require at least one namespace to be non-hidden diffstat: src/lib-storage/mail-namespace.c | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diffs (31 lines): diff -r 601014feade4 -r 1a33cc98666a src/lib-storage/mail-namespace.c --- a/src/lib-storage/mail-namespace.c Wed Apr 04 07:26:37 2012 +0300 +++ b/src/lib-storage/mail-namespace.c Wed Apr 04 07:46:09 2012 +0300 @@ -189,6 +189,7 @@ { struct mail_namespace *ns, *inbox_ns = NULL; unsigned int subscriptions_count = 0; + bool visible_namespaces = FALSE; char ns_sep, list_sep = '\0'; for (ns = namespaces; ns != NULL; ns = ns->next) { @@ -201,6 +202,8 @@ } if (namespace_set_alias_for(ns, namespaces, error_r) < 0) return FALSE; + if ((ns->flags & NAMESPACE_FLAG_HIDDEN) == 0) + visible_namespaces = TRUE; if ((ns->flags & NAMESPACE_FLAG_INBOX_USER) != 0) { if (inbox_ns != NULL) { *error_r = "There can be only one namespace with " @@ -249,6 +252,10 @@ *error_r = "no list=yes namespaces"; return FALSE; } + if (!visible_namespaces) { + *error_r = "no hidden=no namespaces"; + return FALSE; + } if (subscriptions_count == 0) { *error_r = "no subscriptions=yes namespaces"; return FALSE; From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: doveadm status: Don't include the fields parameter ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/19e9a83ca8dc changeset: 14424:19e9a83ca8dc user: Timo Sirainen date: Wed Apr 04 09:40:24 2012 +0300 description: doveadm status: Don't include the fields parameter as a mailbox in search query. diffstat: src/doveadm/doveadm-mail-mailbox-status.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 52ad5a66ee58 -r 19e9a83ca8dc src/doveadm/doveadm-mail-mailbox-status.c --- a/src/doveadm/doveadm-mail-mailbox-status.c Wed Apr 04 09:12:29 2012 +0300 +++ b/src/doveadm/doveadm-mail-mailbox-status.c Wed Apr 04 09:40:24 2012 +0300 @@ -182,7 +182,7 @@ doveadm_mail_help_name("mailbox status"); status_parse_fields(ctx, t_strsplit_spaces(fields, " ")); - ctx->search_args = doveadm_mail_mailbox_search_args_build(args); + ctx->search_args = doveadm_mail_mailbox_search_args_build(args+1); if (!ctx->total_sum) { doveadm_print_header("mailbox", "mailbox", From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: pop3-migration: Error message improvement. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/52ad5a66ee58 changeset: 14423:52ad5a66ee58 user: Timo Sirainen date: Wed Apr 04 09:12:29 2012 +0300 description: pop3-migration: Error message improvement. diffstat: src/plugins/pop3-migration/pop3-migration-plugin.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff -r 1a33cc98666a -r 52ad5a66ee58 src/plugins/pop3-migration/pop3-migration-plugin.c --- a/src/plugins/pop3-migration/pop3-migration-plugin.c Wed Apr 04 07:46:09 2012 +0300 +++ b/src/plugins/pop3-migration/pop3-migration-plugin.c Wed Apr 04 09:12:29 2012 +0300 @@ -120,8 +120,8 @@ struct sha1_ctxt sha1_ctx; if (mail_get_hdr_stream(mail, &hdr_size, &input) < 0) { - i_error("pop3_migration: Failed to get header for msg %u", - mail->seq); + i_error("pop3_migration: Failed to get header for msg %u: %s", + mail->seq, mailbox_get_last_error(mail->box, NULL)); return -1; } input2 = i_stream_create_limit(input, hdr_size.physical_size); From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: dovadm mail commands: When mailbox parameter has no... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/200af5e4c44f changeset: 14425:200af5e4c44f user: Timo Sirainen date: Wed Apr 04 09:41:15 2012 +0300 description: dovadm mail commands: When mailbox parameter has no wildcard, use SEARCH_MAILBOX. diffstat: src/doveadm/doveadm-mail-mailbox.c | 8 +++++++- 1 files changed, 7 insertions(+), 1 deletions(-) diffs (23 lines): diff -r 19e9a83ca8dc -r 200af5e4c44f src/doveadm/doveadm-mail-mailbox.c --- a/src/doveadm/doveadm-mail-mailbox.c Wed Apr 04 09:40:24 2012 +0300 +++ b/src/doveadm/doveadm-mail-mailbox.c Wed Apr 04 09:41:15 2012 +0300 @@ -139,12 +139,18 @@ { struct mail_search_args *search_args; struct mail_search_arg *arg; + enum mail_search_arg_type type; unsigned int i; doveadm_mailbox_args_check(args); search_args = mail_search_build_init(); for (i = 0; args[i] != NULL; i++) { - arg = mail_search_build_add(search_args, SEARCH_MAILBOX_GLOB); + if (strchr(args[i], '*') != NULL || + strchr(args[i], '%') != NULL) + type = SEARCH_MAILBOX_GLOB; + else + type = SEARCH_MAILBOX; + arg = mail_search_build_add(search_args, type); arg->value.str = p_strdup(search_args->pool, args[i]); } if (i > 1) { From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: doveadm: When AND-search includes mailbox names, do... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/1b6fb6363e7f changeset: 14426:1b6fb6363e7f user: Timo Sirainen date: Wed Apr 04 09:42:34 2012 +0300 description: doveadm: When AND-search includes mailbox names, don't bother going through them by list iteration. diffstat: src/doveadm/doveadm-mailbox-list-iter.c | 63 +++++++++++++++++++++++++------- 1 files changed, 48 insertions(+), 15 deletions(-) diffs (141 lines): diff -r 200af5e4c44f -r 1b6fb6363e7f src/doveadm/doveadm-mailbox-list-iter.c --- a/src/doveadm/doveadm-mailbox-list-iter.c Wed Apr 04 09:41:15 2012 +0300 +++ b/src/doveadm/doveadm-mailbox-list-iter.c Wed Apr 04 09:42:34 2012 +0300 @@ -10,18 +10,24 @@ #include "doveadm-mailbox-list-iter.h" struct doveadm_mailbox_list_iter { + struct mail_user *user; struct doveadm_mail_cmd_context *ctx; struct mail_search_args *search_args; enum mailbox_list_iter_flags iter_flags; struct mailbox_list_iterate_context *iter; + + struct mailbox_info info; + ARRAY_TYPE(const_string) patterns; + unsigned int pattern_idx; + bool only_selectable; }; static int search_args_get_mailbox_patterns(const struct mail_search_arg *args, ARRAY_TYPE(const_string) *patterns, - bool *have_guid_r) + bool *have_guid, bool *have_wildcards) { const struct mail_search_arg *subargs; @@ -35,12 +41,15 @@ subargs = args->value.subargs; for (; subargs != NULL; subargs = subargs->next) { if (!search_args_get_mailbox_patterns(subargs, - patterns, have_guid_r)) + patterns, have_guid, + have_wildcards)) return 0; } break; + case SEARCH_MAILBOX_GLOB: + *have_wildcards = TRUE; + /* fall through */ case SEARCH_MAILBOX: - case SEARCH_MAILBOX_GLOB: if (args->match_not) { array_clear(patterns); return 0; @@ -48,7 +57,7 @@ array_append(patterns, &args->value.str, 1); break; case SEARCH_MAILBOX_GUID: - *have_guid_r = TRUE; + *have_guid = TRUE; break; default: break; @@ -66,31 +75,36 @@ { static const char *all_pattern = "*"; struct doveadm_mailbox_list_iter *iter; - ARRAY_TYPE(const_string) patterns; - bool have_guid = FALSE; + bool have_guid = FALSE, have_wildcards = FALSE; iter = i_new(struct doveadm_mailbox_list_iter, 1); iter->ctx = ctx; iter->search_args = search_args; + iter->user = user; + i_array_init(&iter->patterns, 16); + search_args_get_mailbox_patterns(search_args->args, &iter->patterns, + &have_guid, &have_wildcards); - t_array_init(&patterns, 16); - search_args_get_mailbox_patterns(search_args->args, &patterns, - &have_guid); - if (array_count(&patterns) == 0) { + if (array_count(&iter->patterns) == 0) { iter_flags |= MAILBOX_LIST_ITER_SKIP_ALIASES; if (have_guid) ns_mask |= NAMESPACE_SHARED | NAMESPACE_PUBLIC; - array_append(&patterns, &all_pattern, 1); - } else { + array_append(&iter->patterns, &all_pattern, 1); + } else if (have_wildcards) { iter_flags |= MAILBOX_LIST_ITER_STAR_WITHIN_NS; ns_mask |= NAMESPACE_SHARED | NAMESPACE_PUBLIC; + } else { + /* just return the listed mailboxes without actually + iterating through. this also allows accessing mailboxes + without lookup ACL right */ + return iter; } - (void)array_append_space(&patterns); + (void)array_append_space(&iter->patterns); iter->only_selectable = TRUE; iter->iter_flags = iter_flags; iter->iter = mailbox_list_iter_init_namespaces(user->namespaces, - array_idx(&patterns, 0), + array_idx(&iter->patterns, 0), ns_mask, iter_flags); return iter; } @@ -130,10 +144,13 @@ *_iter = NULL; - if ((ret = mailbox_list_iter_deinit(&iter->iter)) < 0) { + if (iter->iter == NULL) + ret = 0; + else if ((ret = mailbox_list_iter_deinit(&iter->iter)) < 0) { i_error("Listing mailboxes failed"); doveadm_mail_failed_error(iter->ctx, MAIL_ERROR_TEMP); } + array_free(&iter->patterns); i_free(iter); return ret; } @@ -142,6 +159,22 @@ doveadm_mailbox_list_iter_next(struct doveadm_mailbox_list_iter *iter) { const struct mailbox_info *info; + const char *const *patterns; + unsigned int count; + + while (iter->iter == NULL) { + patterns = array_get(&iter->patterns, &count); + if (iter->pattern_idx == count) + return NULL; + + iter->info.name = patterns[iter->pattern_idx++]; + iter->info.ns = mail_namespace_find(iter->user->namespaces, + iter->info.name); + if (iter->info.ns != NULL) + return &iter->info; + /* FIXME: maybe fail?.. or just wait for v2.2 to get rid of + this error condition */ + } while ((info = mailbox_list_iter_next(iter->iter)) != NULL) { char sep = mail_namespace_get_sep(info->ns); From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: lib-storage: Human search builder now uses SEARCH_M... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/7a414d055b3c changeset: 14427:7a414d055b3c user: Timo Sirainen date: Wed Apr 04 09:48:46 2012 +0300 description: lib-storage: Human search builder now uses SEARCH_MAILBOX_GLOB only when wildcards are used. diffstat: src/lib-storage/mail-search-register-human.c | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-) diffs (19 lines): diff -r 1b6fb6363e7f -r 7a414d055b3c src/lib-storage/mail-search-register-human.c --- a/src/lib-storage/mail-search-register-human.c Wed Apr 04 09:42:34 2012 +0300 +++ b/src/lib-storage/mail-search-register-human.c Wed Apr 04 09:48:46 2012 +0300 @@ -136,10 +136,14 @@ { struct mail_search_arg *sarg; - sarg = mail_search_build_str(ctx, SEARCH_MAILBOX_GLOB); + sarg = mail_search_build_str(ctx, SEARCH_MAILBOX); if (sarg == NULL) return NULL; + if (strchr(sarg->value.str, '*') != NULL || + strchr(sarg->value.str, '%') != NULL) + sarg->type = SEARCH_MAILBOX_GLOB; + if (!uni_utf8_str_is_valid(sarg->value.str)) { ctx->_error = p_strconcat(ctx->pool, "Mailbox name not valid UTF-8: ", From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: Maildir++: Return nonexistent subscribed mailbox wi... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/f03e81b702a8 changeset: 14428:f03e81b702a8 user: Timo Sirainen date: Wed Apr 04 10:07:41 2012 +0300 description: Maildir++: Return nonexistent subscribed mailbox with nonexistent flag. e.g. LIST (SUBSCRIBED) "" * diffstat: src/lib-storage/list/mailbox-list-maildir-iter.c | 6 +++++- src/lib-storage/list/mailbox-list-subscriptions.c | 7 +++++-- src/lib-storage/list/mailbox-list-subscriptions.h | 3 ++- 3 files changed, 12 insertions(+), 4 deletions(-) diffs (60 lines): diff -r 7a414d055b3c -r f03e81b702a8 src/lib-storage/list/mailbox-list-maildir-iter.c --- a/src/lib-storage/list/mailbox-list-maildir-iter.c Wed Apr 04 09:48:46 2012 +0300 +++ b/src/lib-storage/list/mailbox-list-maildir-iter.c Wed Apr 04 10:07:41 2012 +0300 @@ -440,7 +440,11 @@ if ((flags & MAILBOX_LIST_ITER_SELECT_SUBSCRIBED) != 0) { /* Listing only subscribed mailboxes. Flags are set later if needed. */ - mailbox_list_subscriptions_fill(&ctx->ctx, ctx->tree_ctx); + bool default_nonexistent = + (flags & MAILBOX_LIST_ITER_RETURN_NO_FLAGS) == 0; + + mailbox_list_subscriptions_fill(&ctx->ctx, ctx->tree_ctx, + default_nonexistent); } if ((flags & MAILBOX_LIST_ITER_SELECT_SUBSCRIBED) == 0 || diff -r 7a414d055b3c -r f03e81b702a8 src/lib-storage/list/mailbox-list-subscriptions.c --- a/src/lib-storage/list/mailbox-list-subscriptions.c Wed Apr 04 09:48:46 2012 +0300 +++ b/src/lib-storage/list/mailbox-list-subscriptions.c Wed Apr 04 10:07:41 2012 +0300 @@ -185,7 +185,8 @@ } void mailbox_list_subscriptions_fill(struct mailbox_list_iterate_context *ctx, - struct mailbox_tree_context *tree) + struct mailbox_tree_context *tree, + bool default_nonexistent) { struct mailbox_list_iter_update_context update_ctx; struct mailbox_tree_iterate_context *iter; @@ -197,6 +198,8 @@ update_ctx.tree_ctx = tree; update_ctx.glob = ctx->glob; update_ctx.leaf_flags = MAILBOX_SUBSCRIBED; + if (default_nonexistent) + update_ctx.leaf_flags |= MAILBOX_NONEXISTENT; update_ctx.parent_flags = MAILBOX_CHILD_SUBSCRIBED; update_ctx.match_parents = (ctx->flags & MAILBOX_LIST_ITER_SELECT_RECURSIVEMATCH) != 0; @@ -226,7 +229,7 @@ array_create(&ctx->ctx.module_contexts, pool, sizeof(void *), 5); ctx->tree = mailbox_tree_init(sep); - mailbox_list_subscriptions_fill(&ctx->ctx, ctx->tree); + mailbox_list_subscriptions_fill(&ctx->ctx, ctx->tree, FALSE); ctx->info.ns = list->ns; /* the tree usually has only those entries we want to iterate through, diff -r 7a414d055b3c -r f03e81b702a8 src/lib-storage/list/mailbox-list-subscriptions.h --- a/src/lib-storage/list/mailbox-list-subscriptions.h Wed Apr 04 09:48:46 2012 +0300 +++ b/src/lib-storage/list/mailbox-list-subscriptions.h Wed Apr 04 10:07:41 2012 +0300 @@ -17,7 +17,8 @@ /* Add subscriptions matching the iteration to the given tree */ void mailbox_list_subscriptions_fill(struct mailbox_list_iterate_context *ctx, - struct mailbox_tree_context *tree); + struct mailbox_tree_context *tree, + bool default_nonexistent); /* Iterate through subscriptions, call mailbox_list.get_mailbox_flags() if necessary for mailboxes to get their flags. */ From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: lib-storage: Minor code cleanup. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/f267a2b3f424 changeset: 14430:f267a2b3f424 user: Timo Sirainen date: Wed Apr 04 11:08:32 2012 +0300 description: lib-storage: Minor code cleanup. diffstat: src/lib-storage/mail-user.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 86e6dc46a80e -r f267a2b3f424 src/lib-storage/mail-user.c --- a/src/lib-storage/mail-user.c Wed Apr 04 11:07:05 2012 +0300 +++ b/src/lib-storage/mail-user.c Wed Apr 04 11:08:32 2012 +0300 @@ -341,7 +341,7 @@ { const char *home, *path = *pathp; - if (mail_user_get_home(user, &home) < 0) + if (mail_user_get_home(user, &home) <= 0) return -1; path = home_expand_tilde(path, home); From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: layout=fs: Fixed iteration with mail_full_filesyste... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/86e6dc46a80e changeset: 14429:86e6dc46a80e user: Timo Sirainen date: Wed Apr 04 11:07:05 2012 +0300 description: layout=fs: Fixed iteration with mail_full_filesystem_access=yes diffstat: src/lib-storage/list/mailbox-list-fs-iter.c | 28 +++++++++++++++++++++------- 1 files changed, 21 insertions(+), 7 deletions(-) diffs (66 lines): diff -r f03e81b702a8 -r 86e6dc46a80e src/lib-storage/list/mailbox-list-fs-iter.c --- a/src/lib-storage/list/mailbox-list-fs-iter.c Wed Apr 04 10:07:41 2012 +0300 +++ b/src/lib-storage/list/mailbox-list-fs-iter.c Wed Apr 04 11:07:05 2012 +0300 @@ -112,6 +112,21 @@ i_error("rename(%s, %s) failed: %m", src, str_c(dest)); } +static const char * +dir_get_storage_name(struct list_dir_context *dir, const char *fname) +{ + if (*dir->storage_name == '\0') { + /* regular root */ + return fname; + } else if (strcmp(dir->storage_name, "/") == 0) { + /* full_filesystem_access=yes "/" root */ + return t_strconcat("/", fname, NULL); + } else { + /* child */ + return t_strconcat(dir->storage_name, "/", fname, NULL); + } +} + static int dir_entry_get(struct fs_list_iterate_context *ctx, const char *dir_path, struct list_dir_context *dir, const struct dirent *d) @@ -144,8 +159,7 @@ } /* check the pattern */ - storage_name = *dir->storage_name == '\0' ? d->d_name : - t_strconcat(dir->storage_name, "/", d->d_name, NULL); + storage_name = dir_get_storage_name(dir, d->d_name); vname = mailbox_list_get_vname(ctx->ctx.list, storage_name); if (!uni_utf8_str_is_valid(vname)) { fs_list_rename_invalid(ctx, storage_name); @@ -205,8 +219,9 @@ if (*path == '~') { if (!mailbox_list_try_get_absolute_path(ctx->ctx.list, &path)) { - /* couldn't expand ~user/ */ - return FALSE; + /* a) couldn't expand ~user/ + b) mailbox is under our mail root, we changed + path to storage_name */ } /* NOTE: the path may have been translated to a storage_name instead of path */ @@ -374,7 +389,7 @@ if (*p == '/') last = p; } - if (p == last && *pattern == '/') + if (p == last+1 && *pattern == '/') root = "/"; else { root = mailbox_list_get_storage_name(ctx->ctx.list, @@ -560,8 +575,7 @@ const char *storage_name, *vname, *child_dir_name; dir = ctx->dir; - storage_name = *dir->storage_name == '\0' ? entry->fname : - t_strconcat(dir->storage_name, "/", entry->fname, NULL); + storage_name = dir_get_storage_name(dir, entry->fname); vname = mailbox_list_get_vname(ctx->ctx.list, storage_name); ctx->info.name = p_strdup(ctx->info_pool, vname); From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: director: Don't try to send the new SYNC parameter ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/084064444f89 changeset: 14431:084064444f89 user: Timo Sirainen date: Mon Apr 09 07:52:25 2012 +0300 description: director: Don't try to send the new SYNC parameter to old director versions. This should allow adding new directors to an old director ring without anything breaking. Once all directors have upgraded they start using the new features. diffstat: src/director/director-connection.c | 8 ++++++++ src/director/director-connection.h | 2 ++ src/director/director.c | 3 ++- 3 files changed, 12 insertions(+), 1 deletions(-) diffs (57 lines): diff -r f267a2b3f424 -r 084064444f89 src/director/director-connection.c --- a/src/director/director-connection.c Wed Apr 04 11:08:32 2012 +0300 +++ b/src/director/director-connection.c Mon Apr 09 07:52:25 2012 +0300 @@ -82,6 +82,7 @@ struct director *dir; char *name; time_t created; + unsigned int minor_version; /* for incoming connections the director host isn't known until ME-line is received */ @@ -890,6 +891,7 @@ DIRECTOR_VERSION_MAJOR); return -1; } + conn->minor_version = atoi(args[2]); conn->version_received = TRUE; return 1; } @@ -1528,6 +1530,12 @@ return conn->in; } +unsigned int +director_connection_get_minor_version(struct director_connection *conn) +{ + return conn->minor_version; +} + void director_connection_cork(struct director_connection *conn) { o_stream_cork(conn->output); diff -r f267a2b3f424 -r 084064444f89 src/director/director-connection.h --- a/src/director/director-connection.h Wed Apr 04 11:08:32 2012 +0300 +++ b/src/director/director-connection.h Mon Apr 09 07:52:25 2012 +0300 @@ -23,6 +23,8 @@ director_connection_get_host(struct director_connection *conn); bool director_connection_is_handshaked(struct director_connection *conn); bool director_connection_is_incoming(struct director_connection *conn); +unsigned int +director_connection_get_minor_version(struct director_connection *conn); void director_connection_cork(struct director_connection *conn); void director_connection_uncork(struct director_connection *conn); diff -r f267a2b3f424 -r 084064444f89 src/director/director.c --- a/src/director/director.c Wed Apr 04 11:08:32 2012 +0300 +++ b/src/director/director.c Mon Apr 09 07:52:25 2012 +0300 @@ -263,7 +263,8 @@ str = t_str_new(128); str_printfa(str, "SYNC\t%s\t%u\t%u", net_ip2addr(&host->ip), host->port, seq); - if (minor_version > 0) { + if (minor_version > 0 && + director_connection_get_minor_version(dir->right) > 0) { /* only minor_version>0 supports this parameter */ str_printfa(str, "\t%u", minor_version); } From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: director: Don't communicate with directors that rec... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/366b9e5fc85c changeset: 14432:366b9e5fc85c user: Timo Sirainen date: Mon Apr 09 08:17:52 2012 +0300 description: director: Don't communicate with directors that recently sent invalid input. Track network and protocol failures separately. If a director had sent invalid protocol data within last 60 seconds, don't try to connect to it and don't allow it to connect to us. diffstat: src/director/director-connection.c | 56 ++++++++++++++++++++++++++----------- src/director/director-host.h | 5 ++- src/director/director.c | 14 ++++++-- src/director/director.h | 4 ++ src/director/doveadm-connection.c | 6 +++- 5 files changed, 61 insertions(+), 24 deletions(-) diffs (229 lines): diff -r 084064444f89 -r 366b9e5fc85c src/director/director-connection.c --- a/src/director/director-connection.c Mon Apr 09 07:52:25 2012 +0300 +++ b/src/director/director-connection.c Mon Apr 09 08:17:52 2012 +0300 @@ -65,7 +65,7 @@ #define DIRECTOR_CONNECTION_PING_SYNC_INTERVAL_MSECS 1000 /* If outgoing director connection exists for less than this many seconds, mark the host as failed so we won't try to reconnect to it immediately */ -#define DIRECTOR_SUCCESS_MIN_CONNECT_SECS 10 +#define DIRECTOR_SUCCESS_MIN_CONNECT_SECS 40 #if DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS <= DIRECTOR_CONNECTION_PING_TIMEOUT_MSECS # error DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS is too low @@ -113,6 +113,7 @@ }; static void director_connection_disconnected(struct director_connection **conn); +static void director_connection_protocol_error(struct director_connection **conn); static void ATTR_FORMAT(2, 3) director_cmd_error(struct director_connection *conn, const char *fmt, ...) @@ -318,6 +319,7 @@ const char *connect_str; struct ip_addr ip; unsigned int port; + time_t next_comm_attempt; if (!director_args_parse_ip_port(conn, args, &ip, &port)) return FALSE; @@ -362,10 +364,19 @@ /* make sure we don't keep old sequence values across restarts */ conn->host->last_seq = 0; - if (dir->left == NULL) { + next_comm_attempt = conn->host->last_protocol_failure + + DIRECTOR_PROTOCOL_FAILURE_RETRY_SECS; + if (next_comm_attempt > ioloop_time) { + /* the director recently sent invalid protocol data, + don't try retrying yet */ + i_error("director(%s): Remote sent invalid protocol data recently, " + "waiting %u secs before allowing further communication", + conn->name, (unsigned int)(next_comm_attempt-ioloop_time)); + return FALSE; + } else if (dir->left == NULL) { /* a) - just in case the left is also our right side reset its failed state, so we can connect to it */ - conn->host->last_failed = 0; + conn->host->last_network_failure = 0; if (!director_has_outgoing_connections(dir)) director_connect(dir); } else if (dir->left->host == conn->host) { @@ -548,9 +559,9 @@ host = director_host_lookup(conn->dir, &ip, port); if (host != NULL) { - /* already have this. just reset its last_failed timestamp, - since it might be up now. */ - host->last_failed = 0; + /* already have this. just reset its last_network_failure + timestamp, since it might be up now. */ + host->last_network_failure = 0; return TRUE; } @@ -853,7 +864,7 @@ /* the host is up now, make sure we can connect to it immediately if needed */ - conn->host->last_failed = 0; + conn->host->last_network_failure = 0; conn->handshake_received = TRUE; if (conn->in) { @@ -1024,7 +1035,7 @@ host = director_host_get(conn->dir, &ip, port); /* reset failure timestamp so we'll actually try to connect there. */ - host->last_failed = 0; + host->last_network_failure = 0; /* remote suggests us to connect elsewhere */ if (dir->right != NULL && @@ -1193,7 +1204,7 @@ /* buffer full */ i_error("BUG: Director %s sent us more than %d bytes", conn->name, MAX_INBUF_SIZE); - director_connection_disconnected(&conn); + director_connection_protocol_error(&conn); return; } @@ -1208,7 +1219,7 @@ i_debug("director(%s): Invalid input, disconnecting", conn->name); } - director_connection_disconnected(&conn); + director_connection_protocol_error(&conn); break; } } @@ -1394,13 +1405,6 @@ if (dir->debug && conn->host != NULL) i_debug("Disconnecting from %s", conn->host->name); - if (conn->host != NULL && !conn->wrong_host && - (!conn->handshake_received || - conn->created + DIRECTOR_SUCCESS_MIN_CONNECT_SECS > ioloop_time)) { - /* avoid reconnecting back here immediately */ - conn->host->last_failed = ioloop_time; - } - conns = array_get(&dir->connections, &count); for (i = 0; i < count; i++) { if (conns[i] == conn) { @@ -1449,6 +1453,24 @@ struct director_connection *conn = *_conn; struct director *dir = conn->dir; + if (conn->created + DIRECTOR_SUCCESS_MIN_CONNECT_SECS > ioloop_time) { + /* connection didn't exist for very long, assume it has a + network problem */ + conn->host->last_network_failure = ioloop_time; + } + + director_connection_deinit(_conn); + if (dir->right == NULL) + director_connect(dir); +} + +void director_connection_protocol_error(struct director_connection **_conn) +{ + struct director_connection *conn = *_conn; + struct director *dir = conn->dir; + + conn->host->last_protocol_failure = ioloop_time; + director_connection_deinit(_conn); if (dir->right == NULL) director_connect(dir); diff -r 084064444f89 -r 366b9e5fc85c src/director/director-host.h --- a/src/director/director-host.h Mon Apr 09 07:52:25 2012 +0300 +++ b/src/director/director-host.h Mon Apr 09 08:17:52 2012 +0300 @@ -17,8 +17,9 @@ it can be ignored (or: it must be ignored to avoid potential command loops) */ unsigned int last_seq; - /* Last time host was detected to be down/broken */ - time_t last_failed; + /* Last time host was detected to be down */ + time_t last_network_failure; + time_t last_protocol_failure; /* we are this director */ unsigned int self:1; }; diff -r 084064444f89 -r 366b9e5fc85c src/director/director.c --- a/src/director/director.c Mon Apr 09 07:52:25 2012 +0300 +++ b/src/director/director.c Mon Apr 09 08:17:52 2012 +0300 @@ -110,13 +110,13 @@ port = dir->test_port != 0 ? dir->test_port : host->port; fd = net_connect_ip(&host->ip, port, &dir->self_ip); if (fd == -1) { - host->last_failed = ioloop_time; + host->last_network_failure = ioloop_time; i_error("connect(%s) failed: %m", host->name); return -1; } /* Reset timestamp so that director_connect() won't skip this host while we're still trying to connect to it */ - host->last_failed = 0; + host->last_network_failure = 0; director_connection_init_out(dir, fd, host); return 0; @@ -151,9 +151,15 @@ for (i = 1; i < count; i++) { unsigned int idx = (self_idx + i) % count; - if (hosts[idx]->last_failed + + if (hosts[idx]->last_network_failure + DIRECTOR_RECONNECT_RETRY_SECS > ioloop_time) { - /* failed recently, don't try retrying here */ + /* connection failed recently, don't try retrying here */ + continue; + } + if (hosts[idx]->last_protocol_failure + + DIRECTOR_PROTOCOL_FAILURE_RETRY_SECS > ioloop_time) { + /* the director recently sent invalid protocol data, + don't try retrying yet */ continue; } diff -r 084064444f89 -r 366b9e5fc85c src/director/director.h --- a/src/director/director.h Mon Apr 09 07:52:25 2012 +0300 +++ b/src/director/director.h Mon Apr 09 08:17:52 2012 +0300 @@ -11,6 +11,10 @@ /* weak users supported in protocol v1.1+ */ #define DIRECTOR_VERSION_WEAK_USERS 1 +/* Minimum time between even attempting to communicate with a director that + failed due to a protocol error. */ +#define DIRECTOR_PROTOCOL_FAILURE_RETRY_SECS 60 + struct director; struct mail_host; struct user; diff -r 084064444f89 -r 366b9e5fc85c src/director/doveadm-connection.c --- a/src/director/doveadm-connection.c Mon Apr 09 07:52:25 2012 +0300 +++ b/src/director/doveadm-connection.c Mon Apr 09 08:17:52 2012 +0300 @@ -99,6 +99,7 @@ string_t *str = t_str_new(1024); const char *type; bool left, right; + time_t last_failed; array_foreach(&dir->dir_hosts, hostp) { const struct director_host *host = *hostp; @@ -116,9 +117,12 @@ type = "right"; else type = ""; + + last_failed = I_MAX(host->last_network_failure, + host->last_protocol_failure); str_printfa(str, "%s\t%u\t%s\t%lu\n", net_ip2addr(&host->ip), host->port, type, - (unsigned long)host->last_failed); + (unsigned long)last_failed); } str_append_c(str, '\n'); o_stream_send(conn->output, str_data(str), str_len(str)); From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: imap: Don't allow FETCH items list to be empty. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/19e09ab09383 changeset: 14433:19e09ab09383 user: Timo Sirainen date: Mon Apr 09 09:53:13 2012 +0300 description: imap: Don't allow FETCH items list to be empty. diffstat: src/imap/cmd-fetch.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diffs (15 lines): diff -r 366b9e5fc85c -r 19e09ab09383 src/imap/cmd-fetch.c --- a/src/imap/cmd-fetch.c Mon Apr 09 08:17:52 2012 +0300 +++ b/src/imap/cmd-fetch.c Mon Apr 09 09:53:13 2012 +0300 @@ -57,6 +57,11 @@ } else { *next_arg_r = arg + 1; arg = imap_arg_as_list(arg); + if (IMAP_ARG_IS_EOL(arg)) { + client_send_command_error(ctx->cmd, + "FETCH list is empty."); + return FALSE; + } while (imap_arg_get_atom(arg, &str)) { str = t_str_ucase(str); arg++; From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: lib-storage: Set mail_user.service earlier. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/2bf504b33d64 changeset: 14434:2bf504b33d64 user: Timo Sirainen date: Mon Apr 09 11:53:46 2012 +0300 description: lib-storage: Set mail_user.service earlier. diffstat: src/lib-storage/mail-user.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diffs (28 lines): diff -r 19e09ab09383 -r 2bf504b33d64 src/lib-storage/mail-user.c --- a/src/lib-storage/mail-user.c Mon Apr 09 09:53:13 2012 +0300 +++ b/src/lib-storage/mail-user.c Mon Apr 09 11:53:46 2012 +0300 @@ -50,6 +50,7 @@ user->set_info = set_info; user->unexpanded_set = settings_dup(set_info, set, pool); user->set = settings_dup(set_info, set, pool); + user->service = master_service_get_name(master_service); /* check settings so that the duplicated structure will again contain the parsed fields */ @@ -113,7 +114,6 @@ mail_set = mail_user_set_get_storage_set(user); user->mail_debug = mail_set->mail_debug; - user->service = master_service_get_name(master_service); user->initialized = TRUE; hook_mail_user_created(user); @@ -166,6 +166,8 @@ const struct ip_addr *local_ip, const struct ip_addr *remote_ip) { + i_assert(service != NULL); + user->service = p_strdup(user->pool, service); if (local_ip != NULL && local_ip->family != 0) { user->local_ip = p_new(user->pool, struct ip_addr, 1); From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: lib-index: Minor code cleanup. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/30f8ef8696f0 changeset: 14435:30f8ef8696f0 user: Timo Sirainen date: Tue Apr 03 14:45:10 2012 +0300 description: lib-index: Minor code cleanup. diffstat: src/lib-index/mail-transaction-log-file.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (13 lines): diff -r 2bf504b33d64 -r 30f8ef8696f0 src/lib-index/mail-transaction-log-file.c --- a/src/lib-index/mail-transaction-log-file.c Mon Apr 09 11:53:46 2012 +0300 +++ b/src/lib-index/mail-transaction-log-file.c Tue Apr 03 14:45:10 2012 +0300 @@ -945,8 +945,8 @@ modseq_ext_len) == 0) { /* modseq tracking started */ *cur_modseq += 1; - return; } + return; } else { /* not tracking modseqs */ return; From dovecot at dovecot.org Sun May 20 03:26:31 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:31 +0300 Subject: dovecot-2.2: lib-storage: If mailbox deletion crashed, undelete ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/a109580dea25 changeset: 14436:a109580dea25 user: Timo Sirainen date: Mon Apr 09 13:04:19 2012 +0300 description: lib-storage: If mailbox deletion crashed, undelete the mailbox when opening it. The deletion is assumed to be crashed if it was started over 5 minutes ago. diffstat: src/lib-storage/mail-storage.c | 59 +++++++++++++++++++++-------------------- 1 files changed, 30 insertions(+), 29 deletions(-) diffs (85 lines): diff -r 30f8ef8696f0 -r a109580dea25 src/lib-storage/mail-storage.c --- a/src/lib-storage/mail-storage.c Tue Apr 03 14:45:10 2012 +0300 +++ b/src/lib-storage/mail-storage.c Mon Apr 09 13:04:19 2012 +0300 @@ -914,9 +914,37 @@ return 0; } +static bool mailbox_try_undelete(struct mailbox *box) +{ + time_t mtime; + + if (mail_index_get_modification_time(box->index, &mtime) < 0) + return FALSE; + if (mtime + MAILBOX_DELETE_RETRY_SECS > time(NULL)) + return FALSE; + + if (mailbox_mark_index_deleted(box, FALSE) < 0) + return FALSE; + box->mailbox_deleted = FALSE; + return TRUE; +} + int mailbox_open(struct mailbox *box) { - return mailbox_open_full(box, NULL); + if (mailbox_open_full(box, NULL) < 0) { + if (!box->mailbox_deleted) + return -1; + + /* mailbox has been marked as deleted. if this deletion + started (and crashed) a long time ago, it can be confusing + to user that the mailbox can't be opened. so we'll just + undelete it and reopen. */ + if(!mailbox_try_undelete(box)) + return -1; + if (mailbox_open_full(box, NULL) < 0) + return -1; + } + return 0; } int mailbox_open_stream(struct mailbox *box, struct istream *input) @@ -1043,21 +1071,6 @@ return 0; } -static bool mailbox_try_undelete(struct mailbox *box) -{ - time_t mtime; - - if (mail_index_get_modification_time(box->index, &mtime) < 0) - return FALSE; - if (mtime + MAILBOX_DELETE_RETRY_SECS > time(NULL)) - return FALSE; - - if (mailbox_mark_index_deleted(box, FALSE) < 0) - return FALSE; - box->mailbox_deleted = FALSE; - return TRUE; -} - int mailbox_delete(struct mailbox *box) { int ret; @@ -1077,19 +1090,7 @@ if (mailbox_open(box) < 0) { if (mailbox_get_last_mail_error(box) != MAIL_ERROR_NOTFOUND) return -1; - if (!box->mailbox_deleted) { - /* \noselect mailbox */ - } else { - /* if deletion happened a long time ago, it means it - crashed while doing it. undelete the mailbox in - that case. */ - if (!mailbox_try_undelete(box)) - return -1; - - /* retry */ - if (mailbox_open(box) < 0) - return -1; - } + /* \noselect mailbox */ } ret = box->v.delete(box); From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: director: Fixed disconnecting unwanted connection. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/3771ff9c04d2 changeset: 14437:3771ff9c04d2 user: Timo Sirainen date: Mon Apr 09 13:12:29 2012 +0300 description: director: Fixed disconnecting unwanted connection. diffstat: src/director/director-connection.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r a109580dea25 -r 3771ff9c04d2 src/director/director-connection.c --- a/src/director/director-connection.c Mon Apr 09 13:04:19 2012 +0300 +++ b/src/director/director-connection.c Mon Apr 09 13:12:29 2012 +0300 @@ -244,7 +244,7 @@ /* either use this or disconnect it */ if (!director_connection_assign_left(conn)) { /* we don't want this */ - director_connection_deinit(&dir->left); + director_connection_deinit(&conn); director_assign_left(dir); break; } From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: Added tag 2.1.4 for changeset 2c21c940e19d Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/1fe529f50259 changeset: 14439:1fe529f50259 user: Timo Sirainen date: Mon Apr 09 13:13:26 2012 +0300 description: Added tag 2.1.4 for changeset 2c21c940e19d diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r 2c21c940e19d -r 1fe529f50259 .hgtags --- a/.hgtags Mon Apr 09 13:13:24 2012 +0300 +++ b/.hgtags Mon Apr 09 13:13:26 2012 +0300 @@ -80,3 +80,4 @@ 04b0acc03f1eaa0353888a75a452e5c8e61e4c94 2.1.1 744e0d7f1b255a9339060f761b850303121f14df 2.1.2 b9adfd52cb66d5d89d291b76b9845d6361216d12 2.1.3 +2c21c940e19d97a772128a7f281cea302e157d73 2.1.4 From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: Released v2.1.4. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/2c21c940e19d changeset: 14438:2c21c940e19d user: Timo Sirainen date: Mon Apr 09 13:13:24 2012 +0300 description: Released v2.1.4. diffstat: NEWS | 17 +++++++++++++++++ configure.in | 2 +- 2 files changed, 18 insertions(+), 1 deletions(-) diffs (34 lines): diff -r 3771ff9c04d2 -r 2c21c940e19d NEWS --- a/NEWS Mon Apr 09 13:12:29 2012 +0300 +++ b/NEWS Mon Apr 09 13:13:24 2012 +0300 @@ -1,3 +1,20 @@ +v2.1.4 2012-04-09 Timo Sirainen + + + Added mail_temp_scan_interval setting and changed its default value + from 8 hours to 1 week. + + Added pop3-migration plugin for easily doing a transparent IMAP+POP3 + migration to Dovecot: http://wiki2.dovecot.org/Migration/Dsync + + doveadm user: Added -m parameter to show some of the mail settings. + - Proxying SSL connections crashed in v2.1.[23] + - fts-solr: Indexing mail bodies was broken. + - director: Several changes to significantly improve error handling + - doveadm import didn't import messages' flags + - mail_full_filesystem_access=yes was broken + - Make sure IMAP clients can't create directories when accessing + nonexistent users' mailboxes via shared namespace. + - Dovecot auth clients authenticating via TCP socket could have failed + with bogus "PID already in use" errors. + v2.1.3 2012-03-16 Timo Sirainen - mdbox was broken in v2.1.2 diff -r 3771ff9c04d2 -r 2c21c940e19d configure.in --- a/configure.in Mon Apr 09 13:12:29 2012 +0300 +++ b/configure.in Mon Apr 09 13:13:24 2012 +0300 @@ -1,5 +1,5 @@ AC_PREREQ([2.59]) -AC_INIT([Dovecot],[2.1.3],[dovecot at dovecot.org]) +AC_INIT([Dovecot],[2.1.4],[dovecot at dovecot.org]) AC_CONFIG_SRCDIR([src]) AM_INIT_AUTOMAKE([foreign]) From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: pop3c: Removed extra "R " prefix from POP3 server e... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/35e232e961d3 changeset: 14441:35e232e961d3 user: Timo Sirainen date: Tue Apr 10 09:52:00 2012 +0300 description: pop3c: Removed extra "R " prefix from POP3 server error messages. diffstat: src/lib-storage/index/pop3c/pop3c-client.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 97cba128896f -r 35e232e961d3 src/lib-storage/index/pop3c/pop3c-client.c --- a/src/lib-storage/index/pop3c/pop3c-client.c Mon Apr 09 13:13:32 2012 +0300 +++ b/src/lib-storage/index/pop3c/pop3c-client.c Tue Apr 10 09:52:00 2012 +0300 @@ -663,7 +663,7 @@ *reply_r = line + 3; ret = 0; } else if (strncasecmp(line, "-ERR", 4) == 0) { - *reply_r = line + 3; + *reply_r = line + 4; ret = -1; } else { *reply_r = line; From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: Added signature for changeset 2c21c940e19d Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/97cba128896f changeset: 14440:97cba128896f user: Timo Sirainen date: Mon Apr 09 13:13:32 2012 +0300 description: Added signature for changeset 2c21c940e19d diffstat: .hgsigs | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r 1fe529f50259 -r 97cba128896f .hgsigs --- a/.hgsigs Mon Apr 09 13:13:26 2012 +0300 +++ b/.hgsigs Mon Apr 09 13:13:32 2012 +0300 @@ -43,3 +43,4 @@ 04b0acc03f1eaa0353888a75a452e5c8e61e4c94 0 iEYEABECAAYFAk9F+k0ACgkQyUhSUUBViskLmwCfUt/aex6wOIEohJKnRGA4diF5WxoAn2zlMxSaPX/b0LBmV1P46GAMqZbO 744e0d7f1b255a9339060f761b850303121f14df 0 iEYEABECAAYFAk9h/8oACgkQyUhSUUBVism2OQCfWh62w8pMxJaf1oYx2A+2PxQvBocAn29RFDgZblGRLn7iMCPw6We1yiIw b9adfd52cb66d5d89d291b76b9845d6361216d12 0 iEYEABECAAYFAk9jbugACgkQyUhSUUBVislSgwCgpo3f0bsSujItBum/M6js8SzF06YAmwftHlwaOstKeALdjLR5vtF2c5F7 +2c21c940e19d97a772128a7f281cea302e157d73 0 iEYEABECAAYFAk+CtkYACgkQyUhSUUBVisknxgCfTJw2YPGJ17HbHRGmbwmCyLqepbMAn17j7IYzUfEU0xkXhCgwEAmk7XO4 From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: lib-storage: mail_user_try_home_expand() now looks ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/20e1aa322b1e changeset: 14443:20e1aa322b1e user: Timo Sirainen date: Tue Apr 10 10:09:39 2012 +0300 description: lib-storage: mail_user_try_home_expand() now looks up home only when needed. Before a recent code cleanup the lookup was always done, but if the home didn't exist it still just happened to return success. The cleanup changed it to return failure, which broke the original behavior. diffstat: src/lib-storage/mail-user.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diffs (15 lines): diff -r 0b59dbb1a288 -r 20e1aa322b1e src/lib-storage/mail-user.c --- a/src/lib-storage/mail-user.c Tue Apr 10 09:52:35 2012 +0300 +++ b/src/lib-storage/mail-user.c Tue Apr 10 10:09:39 2012 +0300 @@ -343,6 +343,11 @@ { const char *home, *path = *pathp; + if (strncmp(path, "~/", 2) != 0) { + /* no need to expand home */ + return 0; + } + if (mail_user_get_home(user, &home) <= 0) return -1; From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: pop3c: "TOP msgnum 0" was missing the 0 parameter. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/0b59dbb1a288 changeset: 14442:0b59dbb1a288 user: Timo Sirainen date: Tue Apr 10 09:52:35 2012 +0300 description: pop3c: "TOP msgnum 0" was missing the 0 parameter. diffstat: src/lib-storage/index/pop3c/pop3c-mail.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 35e232e961d3 -r 0b59dbb1a288 src/lib-storage/index/pop3c/pop3c-mail.c --- a/src/lib-storage/index/pop3c/pop3c-mail.c Tue Apr 10 09:52:00 2012 +0300 +++ b/src/lib-storage/index/pop3c/pop3c-mail.c Tue Apr 10 09:52:35 2012 +0300 @@ -113,7 +113,7 @@ if (get_body || (capa & POP3C_CAPABILITY_TOP) == 0) cmd = t_strdup_printf("RETR %u\r\n", _mail->seq); else - cmd = t_strdup_printf("TOP %u\r\n", _mail->seq); + cmd = t_strdup_printf("TOP %u 0\r\n", _mail->seq); if (pop3c_client_cmd_stream(mbox->client, cmd, &input, &error) < 0) { mail_storage_set_error(mbox->box.storage, From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: dict file: Optionally use fcntl/flock locking, inst... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/a1c0e4046d81 changeset: 14444:a1c0e4046d81 user: Timo Sirainen date: Tue Apr 10 17:57:09 2012 +0300 description: dict file: Optionally use fcntl/flock locking, instead of dotlocks. diffstat: src/lib-dict/dict-file.c | 142 +++++++++++++++++++++++++++++++++++++++------- 1 files changed, 120 insertions(+), 22 deletions(-) diffs (235 lines): diff -r 20e1aa322b1e -r a1c0e4046d81 src/lib-dict/dict-file.c --- a/src/lib-dict/dict-file.c Tue Apr 10 10:09:39 2012 +0300 +++ b/src/lib-dict/dict-file.c Tue Apr 10 17:57:09 2012 +0300 @@ -3,12 +3,14 @@ #include "lib.h" #include "array.h" #include "hash.h" +#include "file-lock.h" #include "file-dotlock.h" #include "nfs-workarounds.h" #include "istream.h" #include "ostream.h" #include "dict-private.h" +#include #include #include #include @@ -17,10 +19,13 @@ struct file_dict { struct dict dict; pool_t hash_pool; + enum file_lock_method lock_method; char *path; struct hash_table *hash; int fd; + + bool refreshed; }; struct file_dict_iterate_path { @@ -75,8 +80,17 @@ const char *base_dir ATTR_UNUSED) { struct file_dict *dict; - + dict = i_new(struct file_dict, 1); + if (strncmp(uri, "lock=fcntl ", 11) == 0) { + dict->lock_method = FILE_LOCK_METHOD_FCNTL; + uri += 11; + } else if (strncmp(uri, "lock=flock ", 11) == 0) { + dict->lock_method = FILE_LOCK_METHOD_FLOCK; + uri += 11; + } else { + dict->lock_method = FILE_LOCK_METHOD_DOTLOCK; + } dict->dict = *driver; dict->path = i_strdup(uri); dict->hash_pool = pool_alloconly_create("file dict", 1024); @@ -126,10 +140,9 @@ return FALSE; } -static int file_dict_refresh(struct file_dict *dict) +static int file_dict_open_latest(struct file_dict *dict) { - struct istream *input; - char *key, *value; + int open_type; if (!file_dict_need_refresh(dict)) return 0; @@ -138,13 +151,29 @@ if (close(dict->fd) < 0) i_error("close(%s) failed: %m", dict->path); } - dict->fd = open(dict->path, O_RDONLY); + + open_type = dict->lock_method == FILE_LOCK_METHOD_DOTLOCK ? + O_RDONLY : O_RDWR; + dict->fd = open(dict->path, open_type); if (dict->fd == -1) { if (errno == ENOENT) return 0; i_error("open(%s) failed: %m", dict->path); return -1; } + dict->refreshed = FALSE; + return 1; +} + +static int file_dict_refresh(struct file_dict *dict) +{ + struct istream *input; + char *key, *value; + + if (file_dict_open_latest(dict) < 0) + return -1; + if (dict->refreshed) + return 0; hash_table_clear(dict->hash, TRUE); p_clear(dict->hash_pool); @@ -157,6 +186,7 @@ hash_table_insert(dict->hash, key, value); } i_stream_destroy(&input); + dict->refreshed = TRUE; return 0; } @@ -381,35 +411,92 @@ return fd_copy_stat_permissions(&src_st, dest_fd, dest_path); } +static int +file_dict_lock(struct file_dict *dict, struct file_lock **lock_r) +{ + int fd, ret; + + if (file_dict_open_latest(dict) < 0) + return -1; + + if (dict->fd == -1) { + /* quota file doesn't exist yet, we need to create it */ + fd = open(dict->path, O_CREAT | O_RDWR, 0600); + if (fd == -1) { + i_error("creat(%s) failed: %m", dict->path); + return -1; + } + (void)fd_copy_parent_dir_permissions(dict->path, fd, dict->path); + (void)close(fd); + } + + do { + if (file_wait_lock(dict->fd, dict->path, F_WRLCK, + dict->lock_method, + file_dict_dotlock_settings.timeout, + lock_r) <= 0) { + i_error("file_wait_lock(%s) failed: %m", dict->path); + return -1; + } + /* check again if we need to reopen the file because it was + just replaced */ + } while ((ret = file_dict_open_latest(dict)) > 0); + + return ret < 0 ? -1 : 0; +} + static int file_dict_write_changes(struct file_dict_transaction_context *ctx) { struct file_dict *dict = (struct file_dict *)ctx->ctx.dict; - struct dotlock *dotlock; + struct dotlock *dotlock = NULL; + struct file_lock *lock = NULL; + const char *temp_path = NULL; struct hash_iterate_context *iter; struct ostream *output; void *key, *value; - int fd; + int fd = -1; - fd = file_dotlock_open(&file_dict_dotlock_settings, dict->path, 0, - &dotlock); - if (fd == -1) { - i_error("file dict commit: file_dotlock_open(%s) failed: %m", - dict->path); - return -1; + switch (dict->lock_method) { + case FILE_LOCK_METHOD_FCNTL: + case FILE_LOCK_METHOD_FLOCK: + if (file_dict_lock(dict, &lock) < 0) + return -1; + temp_path = t_strdup_printf("%s.tmp", dict->path); + fd = creat(temp_path, 0600); + if (fd == -1) { + i_error("file dict commit: creat(%s) failed: %m", + temp_path); + return -1; + } + break; + case FILE_LOCK_METHOD_DOTLOCK: + fd = file_dotlock_open(&file_dict_dotlock_settings, dict->path, 0, + &dotlock); + if (fd == -1) { + i_error("file dict commit: file_dotlock_open(%s) failed: %m", + dict->path); + return -1; + } + temp_path = file_dotlock_get_lock_path(dotlock); + break; } + /* refresh once more now that we're locked */ if (file_dict_refresh(dict) < 0) { - file_dotlock_delete(&dotlock); + if (dotlock != NULL) + file_dotlock_delete(&dotlock); + else { + (void)close(fd); + file_unlock(&lock); + } return -1; } if (dict->fd != -1) { /* preserve the permissions */ - (void)fd_copy_permissions(dict->fd, dict->path, fd, - file_dotlock_get_lock_path(dotlock)); + (void)fd_copy_permissions(dict->fd, dict->path, fd, temp_path); } else { /* get initial permissions from parent directory */ - (void)fd_copy_parent_dir_permissions(dict->path, fd, - file_dotlock_get_lock_path(dotlock)); + (void)fd_copy_parent_dir_permissions(dict->path, fd, temp_path); } file_dict_apply_changes(ctx); @@ -425,10 +512,21 @@ hash_table_iterate_deinit(&iter); o_stream_destroy(&output); - if (file_dotlock_replace(&dotlock, - DOTLOCK_REPLACE_FLAG_DONT_CLOSE_FD) < 0) { - (void)close(fd); - return -1; + if (dotlock != NULL) { + if (file_dotlock_replace(&dotlock, + DOTLOCK_REPLACE_FLAG_DONT_CLOSE_FD) < 0) { + (void)close(fd); + return -1; + } + } else { + if (rename(temp_path, dict->path) < 0) { + i_error("rename(%s, %s) failed: %m", + temp_path, dict->path); + file_unlock(&lock); + (void)close(fd); + return -1; + } + file_lock_free(&lock); } if (dict->fd != -1) From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: dict file: Bugfix for previous lock change and chan... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/bf5ae73e9475 changeset: 14445:bf5ae73e9475 user: Timo Sirainen date: Tue Apr 10 18:05:36 2012 +0300 description: dict file: Bugfix for previous lock change and changed its configuration syntax. diffstat: src/lib-dict/dict-file.c | 38 +++++++++++++++++++++++--------------- 1 files changed, 23 insertions(+), 15 deletions(-) diffs (61 lines): diff -r a1c0e4046d81 -r bf5ae73e9475 src/lib-dict/dict-file.c --- a/src/lib-dict/dict-file.c Tue Apr 10 17:57:09 2012 +0300 +++ b/src/lib-dict/dict-file.c Tue Apr 10 18:05:36 2012 +0300 @@ -80,19 +80,25 @@ const char *base_dir ATTR_UNUSED) { struct file_dict *dict; + const char *p; dict = i_new(struct file_dict, 1); - if (strncmp(uri, "lock=fcntl ", 11) == 0) { - dict->lock_method = FILE_LOCK_METHOD_FCNTL; - uri += 11; - } else if (strncmp(uri, "lock=flock ", 11) == 0) { - dict->lock_method = FILE_LOCK_METHOD_FLOCK; - uri += 11; + dict->lock_method = FILE_LOCK_METHOD_DOTLOCK; + + p = strchr(uri, ':'); + if (p == NULL) { + /* no parameters */ + dict->path = i_strdup(uri); } else { - dict->lock_method = FILE_LOCK_METHOD_DOTLOCK; + dict->path = i_strdup_until(uri, p++); + if (strcmp(p, "lock=fcntl") == 0) + dict->lock_method = FILE_LOCK_METHOD_FCNTL; + else if (strcmp(p, "lock=flock") == 0) + dict->lock_method = FILE_LOCK_METHOD_FLOCK; + else + i_error("dict file: Invalid parameter: %s", p+1); } dict->dict = *driver; - dict->path = i_strdup(uri); dict->hash_pool = pool_alloconly_create("file dict", 1024); dict->hash = hash_table_create(default_pool, dict->hash_pool, 0, str_hash, (hash_cmp_callback_t *)strcmp); @@ -178,14 +184,16 @@ hash_table_clear(dict->hash, TRUE); p_clear(dict->hash_pool); - input = i_stream_create_fd(dict->fd, (size_t)-1, FALSE); - while ((key = i_stream_read_next_line(input)) != NULL && - (value = i_stream_read_next_line(input)) != NULL) { - key = p_strdup(dict->hash_pool, key); - value = p_strdup(dict->hash_pool, value); - hash_table_insert(dict->hash, key, value); + if (dict->fd != -1) { + input = i_stream_create_fd(dict->fd, (size_t)-1, FALSE); + while ((key = i_stream_read_next_line(input)) != NULL && + (value = i_stream_read_next_line(input)) != NULL) { + key = p_strdup(dict->hash_pool, key); + value = p_strdup(dict->hash_pool, value); + hash_table_insert(dict->hash, key, value); + } + i_stream_destroy(&input); } - i_stream_destroy(&input); dict->refreshed = TRUE; return 0; } From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: --enable-devel-checks: If DEBUG_SILENT environment ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/7ec8f6515b0d changeset: 14446:7ec8f6515b0d user: Timo Sirainen date: Wed Apr 11 12:35:19 2012 +0300 description: --enable-devel-checks: If DEBUG_SILENT environment is set, hide "Growing pool" warnings. diffstat: src/lib/data-stack.c | 2 +- src/lib/mempool-alloconly.c | 3 ++- src/master/master-settings.c | 2 +- 3 files changed, 4 insertions(+), 3 deletions(-) diffs (37 lines): diff -r bf5ae73e9475 -r 7ec8f6515b0d src/lib/data-stack.c --- a/src/lib/data-stack.c Tue Apr 10 18:05:36 2012 +0300 +++ b/src/lib/data-stack.c Wed Apr 11 12:35:19 2012 +0300 @@ -371,7 +371,7 @@ ret = STACK_BLOCK_DATA(current_block); #ifdef DEBUG - if (warn) { + if (warn && getenv("DEBUG_SILENT") == NULL) { /* warn after allocation, so if i_warning() wants to allocate more memory we don't go to infinite loop */ i_warning("Growing data stack with: %"PRIuSIZE_T, diff -r bf5ae73e9475 -r 7ec8f6515b0d src/lib/mempool-alloconly.c --- a/src/lib/mempool-alloconly.c Tue Apr 10 18:05:36 2012 +0300 +++ b/src/lib/mempool-alloconly.c Wed Apr 11 12:35:19 2012 +0300 @@ -143,7 +143,8 @@ new_apool = p_new(&apool.pool, struct alloconly_pool, 1); *new_apool = apool; #ifdef DEBUG - if (strncmp(name, MEMPOOL_GROWING, strlen(MEMPOOL_GROWING)) == 0) { + if (strncmp(name, MEMPOOL_GROWING, strlen(MEMPOOL_GROWING)) == 0 || + getenv("DEBUG_SILENT") != NULL) { name += strlen(MEMPOOL_GROWING); new_apool->disable_warning = TRUE; } diff -r bf5ae73e9475 -r 7ec8f6515b0d src/master/master-settings.c --- a/src/master/master-settings.c Tue Apr 10 18:05:36 2012 +0300 +++ b/src/master/master-settings.c Wed Apr 11 12:35:19 2012 +0300 @@ -203,7 +203,7 @@ # define ENV_SYSTEMD "" #endif #ifdef DEBUG -# define ENV_GDB " GDB" +# define ENV_GDB " GDB DEBUG_SILENT" #else # define ENV_GDB "" #endif From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: layout=fs: LIST may have shown mailbox as \HasNoChi... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/b016e771d8ab changeset: 14447:b016e771d8ab user: Timo Sirainen date: Wed Apr 11 14:29:37 2012 +0300 description: layout=fs: LIST may have shown mailbox as \HasNoChildren when it did. This probably only happened with mbox. diffstat: src/lib-storage/list/mailbox-list-fs-iter.c | 11 ++++++----- 1 files changed, 6 insertions(+), 5 deletions(-) diffs (23 lines): diff -r 7ec8f6515b0d -r b016e771d8ab src/lib-storage/list/mailbox-list-fs-iter.c --- a/src/lib-storage/list/mailbox-list-fs-iter.c Wed Apr 11 12:35:19 2012 +0300 +++ b/src/lib-storage/list/mailbox-list-fs-iter.c Wed Apr 11 14:29:37 2012 +0300 @@ -323,13 +323,14 @@ dir->info_flags = info_flags; p_array_init(&dir->entries, pool, 16); - if ((dir->info_flags & MAILBOX_CHILDREN) == 0) { - /* start with the assumption of not having children */ + if (fs_list_dir_read(ctx, dir) < 0) + ctx->ctx.failed = TRUE; + + if ((dir->info_flags & (MAILBOX_CHILDREN | MAILBOX_NOCHILDREN | + MAILBOX_NOINFERIORS)) == 0) { + /* assume this directory has no children */ dir->info_flags |= MAILBOX_NOCHILDREN; } - - if (fs_list_dir_read(ctx, dir) < 0) - ctx->ctx.failed = TRUE; return dir; } From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: mbox: Deleting a mailbox didn't delete its index fi... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/c67ea0d68b77 changeset: 14448:c67ea0d68b77 user: Timo Sirainen date: Wed Apr 11 14:44:45 2012 +0300 description: mbox: Deleting a mailbox didn't delete its index files. diffstat: src/lib-storage/index/index-storage.c | 11 +++++++---- 1 files changed, 7 insertions(+), 4 deletions(-) diffs (21 lines): diff -r b016e771d8ab -r c67ea0d68b77 src/lib-storage/index/index-storage.c --- a/src/lib-storage/index/index-storage.c Wed Apr 11 14:29:37 2012 +0300 +++ b/src/lib-storage/index/index-storage.c Wed Apr 11 14:44:45 2012 +0300 @@ -519,10 +519,13 @@ return index_storage_mailbox_delete_dir(box, FALSE); } - /* specifically support symlinked shared mailboxes. a deletion will - simply remove the symlink, not actually expunge any mails */ - if (mailbox_list_delete_symlink(box->list, box->name) == 0) - return 0; + if ((box->list->flags & MAILBOX_LIST_FLAG_MAILBOX_FILES) == 0) { + /* specifically support symlinked shared mailboxes. a deletion + will simply remove the symlink, not actually expunge any + mails */ + if (mailbox_list_delete_symlink(box->list, box->name) == 0) + return 0; + } /* we can't easily atomically delete all mails and the mailbox. so: 1) expunge all mails From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: mbox: Make sure each mailbox creation generates a d... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/85966c47f172 changeset: 14449:85966c47f172 user: Timo Sirainen date: Wed Apr 11 15:02:53 2012 +0300 description: mbox: Make sure each mailbox creation generates a different UIDVALIDITY. diffstat: src/lib-storage/index/mbox/mbox-storage.h | 1 + src/lib-storage/index/mbox/mbox-sync.c | 13 ++++++++++++- 2 files changed, 13 insertions(+), 1 deletions(-) diffs (48 lines): diff -r c67ea0d68b77 -r 85966c47f172 src/lib-storage/index/mbox/mbox-storage.h --- a/src/lib-storage/index/mbox/mbox-storage.h Wed Apr 11 14:44:45 2012 +0300 +++ b/src/lib-storage/index/mbox/mbox-storage.h Wed Apr 11 15:02:53 2012 +0300 @@ -14,6 +14,7 @@ #define MBOX_SUBSCRIPTION_FILE_NAME ".subscriptions" #define MBOX_INDEX_PREFIX "dovecot.index" #define MBOX_INDEX_DIR_NAME ".imap" +#define MBOX_UIDVALIDITY_FNAME "dovecot-uidvalidity" struct mbox_index_header { uint64_t sync_size; diff -r c67ea0d68b77 -r 85966c47f172 src/lib-storage/index/mbox/mbox-sync.c --- a/src/lib-storage/index/mbox/mbox-sync.c Wed Apr 11 14:44:45 2012 +0300 +++ b/src/lib-storage/index/mbox/mbox-sync.c Wed Apr 11 15:02:53 2012 +0300 @@ -45,6 +45,7 @@ #include "istream-raw-mbox.h" #include "mbox-storage.h" #include "index-sync-changes.h" +#include "mailbox-uidvalidity.h" #include "mbox-from.h" #include "mbox-file.h" #include "mbox-lock.h" @@ -1407,6 +1408,16 @@ } } +static uint32_t mbox_get_uidvalidity_next(struct mailbox_list *list) +{ + const char *path; + + path = mailbox_list_get_path(list, NULL, + MAILBOX_LIST_PATH_TYPE_CONTROL); + path = t_strconcat(path, "/"MBOX_UIDVALIDITY_FNAME, NULL); + return mailbox_uidvalidity_next(list, path); +} + static int mbox_sync_update_index_header(struct mbox_sync_context *sync_ctx) { struct mail_index_view *view; @@ -1462,7 +1473,7 @@ if (sync_ctx->base_uid_validity == 0) { sync_ctx->base_uid_validity = sync_ctx->hdr->uid_validity != 0 ? sync_ctx->hdr->uid_validity : - I_MAX((unsigned int)ioloop_time, 1); + mbox_get_uidvalidity_next(sync_ctx->mbox->box.list); } if (sync_ctx->base_uid_validity != sync_ctx->hdr->uid_validity) { mail_index_update_header(sync_ctx->t, From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: dict file: Fixed assert-crash with fcntl/flock lock... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/4fb683720a06 changeset: 14450:4fb683720a06 user: Timo Sirainen date: Wed Apr 11 17:21:01 2012 +0300 description: dict file: Fixed assert-crash with fcntl/flock lock method. diffstat: src/lib-dict/dict-file.c | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diffs (29 lines): diff -r 85966c47f172 -r 4fb683720a06 src/lib-dict/dict-file.c --- a/src/lib-dict/dict-file.c Wed Apr 11 15:02:53 2012 +0300 +++ b/src/lib-dict/dict-file.c Wed Apr 11 17:21:01 2012 +0300 @@ -422,20 +422,20 @@ static int file_dict_lock(struct file_dict *dict, struct file_lock **lock_r) { - int fd, ret; + int ret; if (file_dict_open_latest(dict) < 0) return -1; if (dict->fd == -1) { /* quota file doesn't exist yet, we need to create it */ - fd = open(dict->path, O_CREAT | O_RDWR, 0600); - if (fd == -1) { + dict->fd = open(dict->path, O_CREAT | O_RDWR, 0600); + if (dict->fd == -1) { i_error("creat(%s) failed: %m", dict->path); return -1; } - (void)fd_copy_parent_dir_permissions(dict->path, fd, dict->path); - (void)close(fd); + (void)fd_copy_parent_dir_permissions(dict->path, dict->fd, + dict->path); } do { From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: lib-storage: LIST with mailbox {} settings incorrec... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/9cc178c7afb3 changeset: 14451:9cc178c7afb3 user: Timo Sirainen date: Wed Apr 11 18:23:34 2012 +0300 description: lib-storage: LIST with mailbox {} settings incorrectly set subscribed-flags. diffstat: src/lib-storage/mailbox-list-iter.c | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diffs (11 lines): diff -r 4fb683720a06 -r 9cc178c7afb3 src/lib-storage/mailbox-list-iter.c --- a/src/lib-storage/mailbox-list-iter.c Wed Apr 11 17:21:01 2012 +0300 +++ b/src/lib-storage/mailbox-list-iter.c Wed Apr 11 18:23:34 2012 +0300 @@ -533,7 +533,6 @@ if ((ctx->flags & MAILBOX_LIST_ITER_SELECT_SUBSCRIBED) == 0) match2 = match; else { - info->flags |= MAILBOX_SUBSCRIBED; match2 = autocreate_box_match(&actx->all_ns_box_sets, ctx->list->ns, info->name, FALSE, &idx); From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: Memory leak fixes. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/584bd77c38fd changeset: 14452:584bd77c38fd user: Timo Sirainen date: Wed Apr 11 19:06:44 2012 +0300 description: Memory leak fixes. diffstat: src/lib-storage/index/index-thread.c | 3 +++ src/login-common/ssl-proxy-openssl.c | 9 +++++---- src/plugins/quota/quota.c | 2 +- 3 files changed, 9 insertions(+), 5 deletions(-) diffs (72 lines): diff -r 9cc178c7afb3 -r 584bd77c38fd src/lib-storage/index/index-thread.c --- a/src/lib-storage/index/index-thread.c Wed Apr 11 18:23:34 2012 +0300 +++ b/src/lib-storage/index/index-thread.c Wed Apr 11 19:06:44 2012 +0300 @@ -325,6 +325,7 @@ if (seq1 == 0) { /* nothing is missing */ mail_index_strmap_view_sync_commit(&ctx->strmap_sync); + mailbox_header_lookup_unref(&headers_ctx); return 0; } @@ -332,6 +333,8 @@ mail_search_build_add_seqset(search_args, seq1, seq2); search_ctx = mailbox_search_init(ctx->t, search_args, NULL, 0, headers_ctx); + mailbox_header_lookup_unref(&headers_ctx); + mail_search_args_unref(&search_args); while (mailbox_search_next(search_ctx, &mail)) { if (mail_thread_map_add_mail(ctx, mail) < 0) { diff -r 9cc178c7afb3 -r 584bd77c38fd src/login-common/ssl-proxy-openssl.c --- a/src/login-common/ssl-proxy-openssl.c Wed Apr 11 18:23:34 2012 +0300 +++ b/src/login-common/ssl-proxy-openssl.c Wed Apr 11 19:06:44 2012 +0300 @@ -942,7 +942,8 @@ } static STACK_OF(X509_NAME) * -ssl_proxy_ctx_init(SSL_CTX *ssl_ctx, const struct login_settings *set) +ssl_proxy_ctx_init(SSL_CTX *ssl_ctx, const struct login_settings *set, + bool load_xnames) { X509_STORE *store; STACK_OF(X509_NAME) *xnames = NULL; @@ -956,7 +957,7 @@ SSL_CTX_set_mode(ssl_ctx, SSL_MODE_RELEASE_BUFFERS); #endif - if (*set->ssl_ca != '\0') { + if (*set->ssl_ca != '\0' && load_xnames) { /* set trusted CA certs */ store = SSL_CTX_get_cert_store(ssl_ctx); xnames = load_ca(store, set->ssl_ca); @@ -1198,7 +1199,7 @@ ctx->ctx = ssl_ctx = SSL_CTX_new(SSLv23_server_method()); if (ssl_ctx == NULL) i_fatal("SSL_CTX_new() failed"); - xnames = ssl_proxy_ctx_init(ssl_ctx, set); + xnames = ssl_proxy_ctx_init(ssl_ctx, set, ctx->verify_client_cert); if (SSL_CTX_set_cipher_list(ssl_ctx, ctx->cipher_list) != 1) { i_fatal("Can't set cipher list to '%s': %s", @@ -1265,7 +1266,7 @@ if ((ssl_client_ctx = SSL_CTX_new(SSLv23_client_method())) == NULL) i_fatal("SSL_CTX_new() failed"); - xnames = ssl_proxy_ctx_init(ssl_client_ctx, set); + xnames = ssl_proxy_ctx_init(ssl_client_ctx, set, TRUE); ssl_proxy_ctx_verify_client(ssl_client_ctx, xnames); ssl_proxy_client_ctx_set_client_cert(ssl_client_ctx, set); diff -r 9cc178c7afb3 -r 584bd77c38fd src/plugins/quota/quota.c --- a/src/plugins/quota/quota.c Wed Apr 11 18:23:34 2012 +0300 +++ b/src/plugins/quota/quota.c Wed Apr 11 19:06:44 2012 +0300 @@ -723,7 +723,7 @@ return -1; warning = array_append_space(&root_set->warning_rules); - warning->command = i_strdup(p+1); + warning->command = p_strdup(root_set->set->pool, p+1); warning->rule = rule; warning->reverse = reverse; From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: login: Reverted memory leak fix, because it broke s... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/f80f18d0ffa3 changeset: 14453:f80f18d0ffa3 user: Timo Sirainen date: Thu Apr 12 10:41:44 2012 +0300 description: login: Reverted memory leak fix, because it broke some SSL setups? diffstat: src/login-common/ssl-proxy-openssl.c | 9 ++++----- 1 files changed, 4 insertions(+), 5 deletions(-) diffs (40 lines): diff -r 584bd77c38fd -r f80f18d0ffa3 src/login-common/ssl-proxy-openssl.c --- a/src/login-common/ssl-proxy-openssl.c Wed Apr 11 19:06:44 2012 +0300 +++ b/src/login-common/ssl-proxy-openssl.c Thu Apr 12 10:41:44 2012 +0300 @@ -942,8 +942,7 @@ } static STACK_OF(X509_NAME) * -ssl_proxy_ctx_init(SSL_CTX *ssl_ctx, const struct login_settings *set, - bool load_xnames) +ssl_proxy_ctx_init(SSL_CTX *ssl_ctx, const struct login_settings *set) { X509_STORE *store; STACK_OF(X509_NAME) *xnames = NULL; @@ -957,7 +956,7 @@ SSL_CTX_set_mode(ssl_ctx, SSL_MODE_RELEASE_BUFFERS); #endif - if (*set->ssl_ca != '\0' && load_xnames) { + if (*set->ssl_ca != '\0') { /* set trusted CA certs */ store = SSL_CTX_get_cert_store(ssl_ctx); xnames = load_ca(store, set->ssl_ca); @@ -1199,7 +1198,7 @@ ctx->ctx = ssl_ctx = SSL_CTX_new(SSLv23_server_method()); if (ssl_ctx == NULL) i_fatal("SSL_CTX_new() failed"); - xnames = ssl_proxy_ctx_init(ssl_ctx, set, ctx->verify_client_cert); + xnames = ssl_proxy_ctx_init(ssl_ctx, set); if (SSL_CTX_set_cipher_list(ssl_ctx, ctx->cipher_list) != 1) { i_fatal("Can't set cipher list to '%s': %s", @@ -1266,7 +1265,7 @@ if ((ssl_client_ctx = SSL_CTX_new(SSLv23_client_method())) == NULL) i_fatal("SSL_CTX_new() failed"); - xnames = ssl_proxy_ctx_init(ssl_client_ctx, set, TRUE); + xnames = ssl_proxy_ctx_init(ssl_client_ctx, set); ssl_proxy_ctx_verify_client(ssl_client_ctx, xnames); ssl_proxy_client_ctx_set_client_cert(ssl_client_ctx, set); From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: login: Fixed per-IP ssl_ca setting inside local/rem... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/b60701dd4751 changeset: 14455:b60701dd4751 user: Timo Sirainen date: Mon Apr 16 22:05:47 2012 +0300 description: login: Fixed per-IP ssl_ca setting inside local/remote block. diffstat: src/login-common/ssl-proxy-openssl.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff -r 85ad4baedd43 -r b60701dd4751 src/login-common/ssl-proxy-openssl.c --- a/src/login-common/ssl-proxy-openssl.c Thu Apr 12 10:48:55 2012 +0300 +++ b/src/login-common/ssl-proxy-openssl.c Mon Apr 16 22:05:47 2012 +0300 @@ -139,6 +139,8 @@ return 1; if (strcmp(ctx1->key, ctx2->key) != 0) return 1; + if (null_strcmp(ctx1->ca, ctx2->ca) != 0) + return 1; if (null_strcmp(ctx1->cipher_list, ctx2->cipher_list) != 0) return 1; if (null_strcmp(ctx1->protocols, ctx2->protocols) != 0) From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: login: Another attempt at fixing SSL memory leak. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/85ad4baedd43 changeset: 14454:85ad4baedd43 user: Timo Sirainen date: Thu Apr 12 10:48:55 2012 +0300 description: login: Another attempt at fixing SSL memory leak. diffstat: src/login-common/ssl-proxy-openssl.c | 30 +++++++++++++++++------------- 1 files changed, 17 insertions(+), 13 deletions(-) diffs (85 lines): diff -r f80f18d0ffa3 -r 85ad4baedd43 src/login-common/ssl-proxy-openssl.c --- a/src/login-common/ssl-proxy-openssl.c Thu Apr 12 10:41:44 2012 +0300 +++ b/src/login-common/ssl-proxy-openssl.c Thu Apr 12 10:48:55 2012 +0300 @@ -903,11 +903,11 @@ return strstr(cert, "PRIVATE KEY---") != NULL; } -static STACK_OF(X509_NAME) *load_ca(X509_STORE *store, const char *ca) +static void load_ca(X509_STORE *store, const char *ca, + STACK_OF(X509_NAME) **xnames_r) { /* mostly just copy&pasted from X509_load_cert_crl_file() */ STACK_OF(X509_INFO) *inf; - STACK_OF(X509_NAME) *xnames; X509_INFO *itmp; X509_NAME *xname; BIO *bio; @@ -921,28 +921,32 @@ i_fatal("Couldn't parse ssl_ca: %s", ssl_last_error()); BIO_free(bio); - xnames = sk_X509_NAME_new_null(); - if (xnames == NULL) - i_fatal("sk_X509_NAME_new_null() failed"); + if (xnames_r != NULL) { + *xnames_r = sk_X509_NAME_new_null(); + if (*xnames_r == NULL) + i_fatal_status(FATAL_OUTOFMEM, "sk_X509_NAME_new_null() failed"); + } for(i = 0; i < sk_X509_INFO_num(inf); i++) { itmp = sk_X509_INFO_value(inf, i); if(itmp->x509) { X509_STORE_add_cert(store, itmp->x509); xname = X509_get_subject_name(itmp->x509); - if (xname != NULL) + if (xname != NULL && xnames_r != NULL) { xname = X509_NAME_dup(xname); - if (xname != NULL) - sk_X509_NAME_push(xnames, xname); + if (xname == NULL) + i_fatal_status(FATAL_OUTOFMEM, "X509_NAME_dup() failed"); + sk_X509_NAME_push(*xnames_r, xname); + } } if(itmp->crl) X509_STORE_add_crl(store, itmp->crl); } sk_X509_INFO_pop_free(inf, X509_INFO_free); - return xnames; } static STACK_OF(X509_NAME) * -ssl_proxy_ctx_init(SSL_CTX *ssl_ctx, const struct login_settings *set) +ssl_proxy_ctx_init(SSL_CTX *ssl_ctx, const struct login_settings *set, + bool load_xnames) { X509_STORE *store; STACK_OF(X509_NAME) *xnames = NULL; @@ -959,7 +963,7 @@ if (*set->ssl_ca != '\0') { /* set trusted CA certs */ store = SSL_CTX_get_cert_store(ssl_ctx); - xnames = load_ca(store, set->ssl_ca); + load_ca(store, set->ssl_ca, load_xnames ? &xnames : NULL); } SSL_CTX_set_info_callback(ssl_ctx, ssl_info_callback); if (SSL_CTX_need_tmp_RSA(ssl_ctx)) @@ -1198,7 +1202,7 @@ ctx->ctx = ssl_ctx = SSL_CTX_new(SSLv23_server_method()); if (ssl_ctx == NULL) i_fatal("SSL_CTX_new() failed"); - xnames = ssl_proxy_ctx_init(ssl_ctx, set); + xnames = ssl_proxy_ctx_init(ssl_ctx, set, ctx->verify_client_cert); if (SSL_CTX_set_cipher_list(ssl_ctx, ctx->cipher_list) != 1) { i_fatal("Can't set cipher list to '%s': %s", @@ -1265,7 +1269,7 @@ if ((ssl_client_ctx = SSL_CTX_new(SSLv23_client_method())) == NULL) i_fatal("SSL_CTX_new() failed"); - xnames = ssl_proxy_ctx_init(ssl_client_ctx, set); + xnames = ssl_proxy_ctx_init(ssl_client_ctx, set, TRUE); ssl_proxy_ctx_verify_client(ssl_client_ctx, xnames); ssl_proxy_client_ctx_set_client_cert(ssl_client_ctx, set); From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: login: Log message change: Tried to use s/disabled/... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d6376942d9a6 changeset: 14456:d6376942d9a6 user: Timo Sirainen date: Tue Apr 17 13:56:00 2012 +0300 description: login: Log message change: Tried to use s/disabled/disallowed/ plaintext auth diffstat: src/login-common/client-common.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r b60701dd4751 -r d6376942d9a6 src/login-common/client-common.c --- a/src/login-common/client-common.c Mon Apr 16 22:05:47 2012 +0300 +++ b/src/login-common/client-common.c Tue Apr 17 13:56:00 2012 +0300 @@ -616,7 +616,7 @@ /* some auth attempts without SSL/TLS */ if (client->auth_tried_disabled_plaintext) - return "(tried to use disabled plaintext auth)"; + return "(tried to use disallowed plaintext auth)"; if (client->set->auth_ssl_require_client_cert && client->ssl_proxy == NULL) return "(cert required, client didn't start TLS)"; From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: imapc: Use SHA1 of message header for guid-forced i... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/ea7ca6330491 changeset: 14458:ea7ca6330491 user: Timo Sirainen date: Wed Apr 18 18:33:02 2012 +0300 description: imapc: Use SHA1 of message header for guid-forced instead of the entire body. diffstat: src/lib-storage/index/imapc/imapc-mail.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diffs (32 lines): diff -r 16ed300484c9 -r ea7ca6330491 src/lib-storage/index/imapc/imapc-mail.c --- a/src/lib-storage/index/imapc/imapc-mail.c Wed Apr 18 18:09:14 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-mail.c Wed Apr 18 18:33:02 2012 +0300 @@ -309,7 +309,7 @@ buffer_free(&mail->body); } -static int imapc_mail_get_body_hash(struct index_mail *imail) +static int imapc_mail_get_hdr_hash(struct index_mail *imail) { struct istream *input; const unsigned char *data; @@ -322,7 +322,7 @@ sha1_init(&sha1_ctx); old_offset = imail->data.stream == NULL ? 0 : imail->data.stream->v_offset; - if (mail_get_stream(&imail->mail.mail, NULL, NULL, &input) < 0) + if (mail_get_hdr_stream(&imail->mail.mail, NULL, &input) < 0) return -1; while (i_stream_read_data(input, &data, &size, 0) > 0) { sha1_loop(&sha1_ctx, data, size); @@ -365,8 +365,8 @@ return -1; } } else { - /* use hash of message body as the GUID */ - if (imapc_mail_get_body_hash(imail) < 0) + /* use hash of message headers as the GUID */ + if (imapc_mail_get_hdr_hash(imail) < 0) return -1; } From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: imapc: Added guid-forced feature to use SHA1(messag... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/16ed300484c9 changeset: 14457:16ed300484c9 user: Timo Sirainen date: Wed Apr 18 18:09:14 2012 +0300 description: imapc: Added guid-forced feature to use SHA1(message) as GUID. diffstat: src/lib-storage/index/imapc/imapc-mail.c | 48 ++++++++++++++++++++++++--- src/lib-storage/index/imapc/imapc-settings.c | 1 + src/lib-storage/index/imapc/imapc-settings.h | 3 +- 3 files changed, 45 insertions(+), 7 deletions(-) diffs (103 lines): diff -r d6376942d9a6 -r 16ed300484c9 src/lib-storage/index/imapc/imapc-mail.c --- a/src/lib-storage/index/imapc/imapc-mail.c Tue Apr 17 13:56:00 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-mail.c Wed Apr 18 18:09:14 2012 +0300 @@ -2,6 +2,8 @@ #include "lib.h" #include "str.h" +#include "hex-binary.h" +#include "sha1.h" #include "istream.h" #include "imap-envelope.h" #include "imapc-msgmap.h" @@ -307,6 +309,33 @@ buffer_free(&mail->body); } +static int imapc_mail_get_body_hash(struct index_mail *imail) +{ + struct istream *input; + const unsigned char *data; + size_t size; + uoff_t old_offset; + struct sha1_ctxt sha1_ctx; + unsigned char sha1_output[SHA1_RESULTLEN]; + const char *sha1_str; + + sha1_init(&sha1_ctx); + old_offset = imail->data.stream == NULL ? 0 : + imail->data.stream->v_offset; + if (mail_get_stream(&imail->mail.mail, NULL, NULL, &input) < 0) + return -1; + while (i_stream_read_data(input, &data, &size, 0) > 0) { + sha1_loop(&sha1_ctx, data, size); + i_stream_skip(input, size); + } + i_stream_seek(imail->data.stream, old_offset); + sha1_result(&sha1_ctx, sha1_output); + + sha1_str = binary_to_hex(sha1_output, sizeof(sha1_output)); + imail->data.guid = p_strdup(imail->data_pool, sha1_str); + return 0; +} + static int imapc_mail_get_guid(struct mail *_mail, const char **value_r) { struct index_mail *imail = (struct index_mail *)_mail; @@ -328,11 +357,17 @@ } /* GUID not in cache, fetch it */ - if (imapc_mail_fetch(_mail, MAIL_FETCH_GUID) < 0) - return -1; - if (imail->data.guid == NULL) { - (void)imapc_mail_failed(_mail, mbox->guid_fetch_field_name); - return -1; + if (mbox->guid_fetch_field_name != NULL) { + if (imapc_mail_fetch(_mail, MAIL_FETCH_GUID) < 0) + return -1; + if (imail->data.guid == NULL) { + (void)imapc_mail_failed(_mail, mbox->guid_fetch_field_name); + return -1; + } + } else { + /* use hash of message body as the GUID */ + if (imapc_mail_get_body_hash(imail) < 0) + return -1; } index_mail_cache_add_idx(imail, cache_idx, @@ -349,7 +384,8 @@ switch (field) { case MAIL_FETCH_GUID: - if (mbox->guid_fetch_field_name == NULL) { + if (!IMAPC_BOX_HAS_FEATURE(mbox, IMAPC_FEATURE_GUID_FORCED) && + mbox->guid_fetch_field_name == NULL) { /* GUIDs not supported by server */ break; } diff -r d6376942d9a6 -r 16ed300484c9 src/lib-storage/index/imapc/imapc-settings.c --- a/src/lib-storage/index/imapc/imapc-settings.c Tue Apr 17 13:56:00 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-settings.c Wed Apr 18 18:09:14 2012 +0300 @@ -76,6 +76,7 @@ static const struct imapc_feature_list imapc_feature_list[] = { { "rfc822.size", IMAPC_FEATURE_RFC822_SIZE }, + { "guid-forced", IMAPC_FEATURE_GUID_FORCED }, { NULL, 0 } }; diff -r d6376942d9a6 -r 16ed300484c9 src/lib-storage/index/imapc/imapc-settings.h --- a/src/lib-storage/index/imapc/imapc-settings.h Tue Apr 17 13:56:00 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-settings.h Wed Apr 18 18:09:14 2012 +0300 @@ -3,7 +3,8 @@ /* */ enum imapc_features { - IMAPC_FEATURE_RFC822_SIZE = 0x01 + IMAPC_FEATURE_RFC822_SIZE = 0x01, + IMAPC_FEATURE_GUID_FORCED = 0x02 }; /* */ From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: fts-solr: doveadm fts rescan now resets the last-in... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d145ea6f7061 changeset: 14460:d145ea6f7061 user: Timo Sirainen date: Wed Apr 18 19:56:29 2012 +0300 description: fts-solr: doveadm fts rescan now resets the last-indexed-uid fields to allow reindexing. diffstat: src/plugins/fts-solr/fts-backend-solr.c | 30 +++++++++++++++++++++++++++++- 1 files changed, 29 insertions(+), 1 deletions(-) diffs (47 lines): diff -r 51b40cffb628 -r d145ea6f7061 src/plugins/fts-solr/fts-backend-solr.c --- a/src/plugins/fts-solr/fts-backend-solr.c Wed Apr 18 19:29:22 2012 +0300 +++ b/src/plugins/fts-solr/fts-backend-solr.c Wed Apr 18 19:56:29 2012 +0300 @@ -545,6 +545,34 @@ return 0; } +static int fts_backend_solr_rescan(struct fts_backend *backend) +{ + struct mailbox_list_iterate_context *iter; + const struct mailbox_info *info; + struct mailbox *box; + int ret = 0; + + /* FIXME: proper rescan needed. for now we'll just reset the + last-uids */ + iter = mailbox_list_iter_init(backend->ns->list, "*", + MAILBOX_LIST_ITER_NO_AUTO_BOXES); + while ((info = mailbox_list_iter_next(iter)) != NULL) { + if ((info->flags & + (MAILBOX_NONEXISTENT | MAILBOX_NOSELECT)) != 0) + continue; + + box = mailbox_alloc(info->ns->list, info->name, 0); + if (mailbox_open(box) == 0) { + if (fts_index_set_last_uid(box, 0) < 0) + ret = -1; + } + mailbox_free(&box); + } + if (mailbox_list_iter_deinit(&iter) < 0) + ret = -1; + return ret; +} + static int fts_backend_solr_optimize(struct fts_backend *backend ATTR_UNUSED) { return 0; @@ -850,7 +878,7 @@ fts_backend_solr_update_unset_build_key, fts_backend_solr_update_build_more, fts_backend_solr_refresh, - NULL, + fts_backend_solr_rescan, fts_backend_solr_optimize, fts_backend_default_can_lookup, fts_backend_solr_lookup, From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: trash plugin fixes Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/51b40cffb628 changeset: 14459:51b40cffb628 user: Timo Sirainen date: Wed Apr 18 19:29:22 2012 +0300 description: trash plugin fixes diffstat: src/plugins/trash/trash-plugin.c | 16 +++++++--------- 1 files changed, 7 insertions(+), 9 deletions(-) diffs (43 lines): diff -r ea7ca6330491 -r 51b40cffb628 src/plugins/trash/trash-plugin.c --- a/src/plugins/trash/trash-plugin.c Wed Apr 18 18:33:02 2012 +0300 +++ b/src/plugins/trash/trash-plugin.c Wed Apr 18 19:29:22 2012 +0300 @@ -158,9 +158,10 @@ trash->mail = NULL; (void)mailbox_search_deinit(&trash->search_ctx); - if (size_expunged >= size_needed) + if (size_expunged >= size_needed) { (void)mailbox_transaction_commit(&trash->trans); - else { + (void)mailbox_sync(trash->box, 0); + } else { /* couldn't get enough space, don't expunge anything */ mailbox_transaction_rollback(&trash->trans); } @@ -175,14 +176,12 @@ (unsigned long long)size_needed, (unsigned long long)size_expunged); } - return FALSE; + return 0; } - ctx->bytes_used = ctx->bytes_used > (int64_t)size_expunged ? - ctx->bytes_used - size_expunged : 0; - ctx->count_used = ctx->count_used > (int64_t)expunged_count ? - ctx->count_used - expunged_count : 0; - return TRUE; + ctx->bytes_ceil += size_expunged; + ctx->count_ceil += expunged_count; + return 1; } static int @@ -214,7 +213,6 @@ if (ret <= 0) return 0; } - return 0; } From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: maildir_very_dirty_syncs=yes: Try harder to avoid r... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/ff07827db69a changeset: 14462:ff07827db69a user: Timo Sirainen date: Thu Apr 19 14:39:26 2012 +0300 description: maildir_very_dirty_syncs=yes: Try harder to avoid readdir()ing. diffstat: src/lib-storage/index/maildir/maildir-sync.c | 51 +++++++++++++++------------ src/lib-storage/index/maildir/maildir-sync.h | 1 + src/lib-storage/index/maildir/maildir-util.c | 11 ++++++ 3 files changed, 41 insertions(+), 22 deletions(-) diffs (108 lines): diff -r bda92bf286a0 -r ff07827db69a src/lib-storage/index/maildir/maildir-sync.c --- a/src/lib-storage/index/maildir/maildir-sync.c Wed Apr 18 20:09:30 2012 +0300 +++ b/src/lib-storage/index/maildir/maildir-sync.c Thu Apr 19 14:39:26 2012 +0300 @@ -993,6 +993,34 @@ return ret; } +int maildir_sync_refresh_flags_view(struct maildir_mailbox *mbox) +{ + struct mail_index_view_sync_ctx *sync_ctx; + bool delayed_expunges; + + (void)mail_index_refresh(mbox->box.index); + if (mbox->flags_view == NULL) + mbox->flags_view = mail_index_view_open(mbox->box.index); + + sync_ctx = mail_index_view_sync_begin(mbox->flags_view, + MAIL_INDEX_VIEW_SYNC_FLAG_FIX_INCONSISTENT); + if (mail_index_view_sync_commit(&sync_ctx, &delayed_expunges) < 0) { + mail_storage_set_index_error(&mbox->box); + return -1; + } + /* make sure the map stays in private memory */ + if (mbox->flags_view->map->refcount > 1) { + struct mail_index_map *map; + + map = mail_index_map_clone(mbox->flags_view->map); + mail_index_unmap(&mbox->flags_view->map); + mbox->flags_view->map = map; + } + mail_index_record_map_move_to_private(mbox->flags_view->map); + mail_index_map_move_to_memory(mbox->flags_view->map); + return 0; +} + struct mailbox_sync_context * maildir_storage_sync_init(struct mailbox *box, enum mailbox_sync_flags flags) { @@ -1025,29 +1053,8 @@ } if (mbox->storage->set->maildir_very_dirty_syncs) { - struct mail_index_view_sync_ctx *sync_ctx; - bool b; - - if (mbox->flags_view == NULL) { - mbox->flags_view = - mail_index_view_open(mbox->box.index); - } - sync_ctx = mail_index_view_sync_begin(mbox->flags_view, - MAIL_INDEX_VIEW_SYNC_FLAG_FIX_INCONSISTENT); - if (mail_index_view_sync_commit(&sync_ctx, &b) < 0) { - mail_storage_set_index_error(&mbox->box); + if (maildir_sync_refresh_flags_view(mbox) < 0) ret = -1; - } - /* make sure the map stays in private memory */ - if (mbox->flags_view->map->refcount > 1) { - struct mail_index_map *map; - - map = mail_index_map_clone(mbox->flags_view->map); - mail_index_unmap(&mbox->flags_view->map); - mbox->flags_view->map = map; - } - mail_index_record_map_move_to_private(mbox->flags_view->map); - mail_index_map_move_to_memory(mbox->flags_view->map); maildir_uidlist_set_all_nonsynced(mbox->uidlist); } mbox->synced = TRUE; diff -r bda92bf286a0 -r ff07827db69a src/lib-storage/index/maildir/maildir-sync.h --- a/src/lib-storage/index/maildir/maildir-sync.h Wed Apr 18 20:09:30 2012 +0300 +++ b/src/lib-storage/index/maildir/maildir-sync.h Thu Apr 19 14:39:26 2012 +0300 @@ -42,6 +42,7 @@ void maildir_sync_notify(struct maildir_sync_context *ctx); void maildir_sync_set_new_msgs_count(struct maildir_index_sync_context *ctx, unsigned int count); +int maildir_sync_refresh_flags_view(struct maildir_mailbox *mbox); int maildir_sync_lookup(struct maildir_mailbox *mbox, uint32_t uid, enum maildir_uidlist_rec_flag *flags_r, diff -r bda92bf286a0 -r ff07827db69a src/lib-storage/index/maildir/maildir-util.c --- a/src/lib-storage/index/maildir/maildir-util.c Wed Apr 18 20:09:30 2012 +0300 +++ b/src/lib-storage/index/maildir/maildir-util.c Thu Apr 19 14:39:26 2012 +0300 @@ -103,6 +103,12 @@ if (ret > 0 && (flags & MAILDIR_UIDLIST_REC_FLAG_NONSYNCED) != 0) { /* file was found. make sure we remember its latest name. */ maildir_uidlist_update_fname(mbox->uidlist, fname); + } else if (ret == 0 && + (flags & MAILDIR_UIDLIST_REC_FLAG_NONSYNCED) == 0) { + /* file wasn't found. mark this message nonsynced, so we can + retry the lookup by guessing the flags */ + maildir_uidlist_add_flags(mbox->uidlist, fname, + MAILDIR_UIDLIST_REC_FLAG_NONSYNCED); } return ret; } @@ -133,6 +139,11 @@ T_BEGIN { ret = maildir_file_do_try(mbox, uid, callback, context); } T_END; + if (ret == 0 && mbox->storage->set->maildir_very_dirty_syncs) T_BEGIN { + /* try guessing again with refreshed flags */ + if (maildir_sync_refresh_flags_view(mbox) == 0) + ret = maildir_file_do_try(mbox, uid, callback, context); + } T_END; for (i = 0; i < MAILDIR_RESYNC_RETRY_COUNT && ret == 0; i++) { /* file is either renamed or deleted. sync the maildir and see which one. if file appears to be renamed constantly, From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: auth: Changed auth-userdb socket's default user to ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/bda92bf286a0 changeset: 14461:bda92bf286a0 user: Timo Sirainen date: Wed Apr 18 20:09:30 2012 +0300 description: auth: Changed auth-userdb socket's default user to $default_internal_user. This makes the defaults more usable for director setups. diffstat: src/auth/auth-settings.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r d145ea6f7061 -r bda92bf286a0 src/auth/auth-settings.c --- a/src/auth/auth-settings.c Wed Apr 18 19:56:29 2012 +0300 +++ b/src/auth/auth-settings.c Wed Apr 18 20:09:30 2012 +0300 @@ -19,7 +19,7 @@ { "login/login", 0666, "", "" }, { "auth-login", 0600, "$default_internal_user", "" }, { "auth-client", 0600, "", "" }, - { "auth-userdb", 0666, "", "" }, + { "auth-userdb", 0666, "$default_internal_user", "" }, { "auth-master", 0600, "", "" } }; static struct file_listener_settings *auth_unix_listeners[] = { From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: lib-storage: Added mailbox_delete_empty(). Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/2d8bafd11569 changeset: 14463:2d8bafd11569 user: Timo Sirainen date: Thu Apr 19 18:15:37 2012 +0300 description: lib-storage: Added mailbox_delete_empty(). diffstat: src/lib-storage/index/index-storage.c | 6 ++++-- src/lib-storage/mail-storage-private.h | 2 ++ src/lib-storage/mail-storage.c | 12 ++++++++++++ src/lib-storage/mail-storage.h | 3 +++ 4 files changed, 21 insertions(+), 2 deletions(-) diffs (63 lines): diff -r ff07827db69a -r 2d8bafd11569 src/lib-storage/index/index-storage.c --- a/src/lib-storage/index/index-storage.c Thu Apr 19 14:39:26 2012 +0300 +++ b/src/lib-storage/index/index-storage.c Thu Apr 19 18:15:37 2012 +0300 @@ -535,8 +535,10 @@ no) finish deleting the mailbox */ - if (mailbox_expunge_all_mails(box) < 0) - return -1; + if (!box->deleting_must_be_empty) { + if (mailbox_expunge_all_mails(box) < 0) + return -1; + } if (mailbox_mark_index_deleted(box, TRUE) < 0) return -1; diff -r ff07827db69a -r 2d8bafd11569 src/lib-storage/mail-storage-private.h --- a/src/lib-storage/mail-storage-private.h Thu Apr 19 14:39:26 2012 +0300 +++ b/src/lib-storage/mail-storage-private.h Thu Apr 19 18:15:37 2012 +0300 @@ -265,6 +265,8 @@ unsigned int creating:1; /* Mailbox is being deleted */ unsigned int deleting:1; + /* Delete mailbox only if it's empty */ + unsigned int deleting_must_be_empty:1; /* Mailbox was already marked as deleted within this allocation. */ unsigned int marked_deleted:1; /* TRUE if this is an INBOX for this user */ diff -r ff07827db69a -r 2d8bafd11569 src/lib-storage/mail-storage.c --- a/src/lib-storage/mail-storage.c Thu Apr 19 14:39:26 2012 +0300 +++ b/src/lib-storage/mail-storage.c Thu Apr 19 18:15:37 2012 +0300 @@ -1106,6 +1106,18 @@ return ret; } +int mailbox_delete_empty(struct mailbox *box) +{ + int ret; + + /* FIXME: should be a parameter to delete(), but since it changes API + don't do it for now */ + box->deleting_must_be_empty = TRUE; + ret = mailbox_delete(box); + box->deleting_must_be_empty = FALSE; + return ret; +} + static bool mail_storages_rename_compatible(struct mail_storage *storage1, struct mail_storage *storage2, diff -r ff07827db69a -r 2d8bafd11569 src/lib-storage/mail-storage.h --- a/src/lib-storage/mail-storage.h Thu Apr 19 14:39:26 2012 +0300 +++ b/src/lib-storage/mail-storage.h Thu Apr 19 18:15:37 2012 +0300 @@ -417,6 +417,9 @@ int mailbox_update(struct mailbox *box, const struct mailbox_update *update); /* Delete mailbox (and its parent directory, if it has no siblings) */ int mailbox_delete(struct mailbox *box); +/* Delete mailbox, but only if it's empty. If it's not, fails with + MAIL_ERROR_EXISTS. */ +int mailbox_delete_empty(struct mailbox *box); /* Rename mailbox. Renaming across different mailbox lists is possible only between private namespaces and storages of the same type. If the rename fails, the error is set to src's storage. */ From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: doveadm expunge: Added -d parameter to delete mailb... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/782570f644f7 changeset: 14464:782570f644f7 user: Timo Sirainen date: Thu Apr 19 18:15:56 2012 +0300 description: doveadm expunge: Added -d parameter to delete mailbox if it's empty after expunging. diffstat: src/doveadm/doveadm-mail-expunge.c | 64 ++++++++++++++++++++++++++++++++----- src/doveadm/doveadm-mail-iter.c | 18 +++++++-- src/doveadm/doveadm-mail-iter.h | 2 + 3 files changed, 70 insertions(+), 14 deletions(-) diffs (161 lines): diff -r 2d8bafd11569 -r 782570f644f7 src/doveadm/doveadm-mail-expunge.c --- a/src/doveadm/doveadm-mail-expunge.c Thu Apr 19 18:15:37 2012 +0300 +++ b/src/doveadm/doveadm-mail-expunge.c Thu Apr 19 18:15:56 2012 +0300 @@ -9,16 +9,24 @@ #include "doveadm-mail-iter.h" #include "doveadm-mail.h" +struct expunge_cmd_context { + struct doveadm_mail_cmd_context ctx; + bool delete_empty_mailbox; +}; + static int -cmd_expunge_box(struct doveadm_mail_cmd_context *ctx, +cmd_expunge_box(struct doveadm_mail_cmd_context *_ctx, const struct mailbox_info *info, struct mail_search_args *search_args) { + struct expunge_cmd_context *ctx = (struct expunge_cmd_context *)_ctx; struct doveadm_mail_iter *iter; + struct mailbox *box; struct mailbox_transaction_context *trans; struct mail *mail; + enum mail_error error; - if (doveadm_mail_iter_init(ctx, info, search_args, 0, NULL, + if (doveadm_mail_iter_init(_ctx, info, search_args, 0, NULL, &trans, &iter) < 0) return -1; @@ -29,7 +37,29 @@ } mail_expunge(mail); } - return doveadm_mail_iter_deinit_sync(&iter); + + if (doveadm_mail_iter_deinit_keep_box(&iter, &box) < 0) { + mailbox_free(&box); + return -1; + } + if (mailbox_sync(box, 0) < 0) { + doveadm_mail_failed_mailbox(_ctx, box); + mailbox_free(&box); + return -1; + } + + if (ctx->delete_empty_mailbox) { + if (mailbox_delete_empty(box) < 0) { + (void)mailbox_get_last_error(box, &error); + if (error != MAIL_ERROR_EXISTS) { + doveadm_mail_failed_mailbox(_ctx, box); + mailbox_free(&box); + return -1; + } + } + } + mailbox_free(&box); + return 0; } static bool @@ -208,16 +238,32 @@ expunge_search_args_check(ctx->search_args, "expunge"); } +static bool cmd_expunge_parse_arg(struct doveadm_mail_cmd_context *_ctx, int c) +{ + struct expunge_cmd_context *ctx = (struct expunge_cmd_context *)_ctx; + + switch (c) { + case 'd': + ctx->delete_empty_mailbox = TRUE; + break; + default: + return FALSE; + } + return TRUE; +} + static struct doveadm_mail_cmd_context *cmd_expunge_alloc(void) { - struct doveadm_mail_cmd_context *ctx; + struct expunge_cmd_context *ctx; - ctx = doveadm_mail_cmd_alloc(struct doveadm_mail_cmd_context); - ctx->v.init = cmd_expunge_init; - ctx->v.run = cmd_expunge_run; - return ctx; + ctx = doveadm_mail_cmd_alloc(struct expunge_cmd_context); + ctx->ctx.getopt_args = "d"; + ctx->ctx.v.parse_arg = cmd_expunge_parse_arg; + ctx->ctx.v.init = cmd_expunge_init; + ctx->ctx.v.run = cmd_expunge_run; + return &ctx->ctx; } struct doveadm_mail_cmd cmd_expunge = { - cmd_expunge_alloc, "expunge", "" + cmd_expunge_alloc, "expunge", "[-d] " }; diff -r 2d8bafd11569 -r 782570f644f7 src/doveadm/doveadm-mail-iter.c --- a/src/doveadm/doveadm-mail-iter.c Thu Apr 19 18:15:37 2012 +0300 +++ b/src/doveadm/doveadm-mail-iter.c Thu Apr 19 18:15:56 2012 +0300 @@ -83,7 +83,7 @@ static int doveadm_mail_iter_deinit_full(struct doveadm_mail_iter **_iter, - bool sync, bool commit) + bool sync, bool commit, bool keep_box) { struct doveadm_mail_iter *iter = *_iter; int ret; @@ -95,24 +95,32 @@ ret = mailbox_sync(iter->box, 0); if (ret < 0) doveadm_mail_failed_mailbox(iter->ctx, iter->box); - mailbox_free(&iter->box); + if (!keep_box) + mailbox_free(&iter->box); i_free(iter); return ret; } int doveadm_mail_iter_deinit(struct doveadm_mail_iter **_iter) { - return doveadm_mail_iter_deinit_full(_iter, FALSE, TRUE); + return doveadm_mail_iter_deinit_full(_iter, FALSE, TRUE, FALSE); } int doveadm_mail_iter_deinit_sync(struct doveadm_mail_iter **_iter) { - return doveadm_mail_iter_deinit_full(_iter, TRUE, TRUE); + return doveadm_mail_iter_deinit_full(_iter, TRUE, TRUE, FALSE); +} + +int doveadm_mail_iter_deinit_keep_box(struct doveadm_mail_iter **iter, + struct mailbox **box_r) +{ + *box_r = (*iter)->box; + return doveadm_mail_iter_deinit_full(iter, FALSE, TRUE, TRUE); } void doveadm_mail_iter_deinit_rollback(struct doveadm_mail_iter **_iter) { - (void)doveadm_mail_iter_deinit_full(_iter, FALSE, FALSE); + (void)doveadm_mail_iter_deinit_full(_iter, FALSE, FALSE, FALSE); } bool doveadm_mail_iter_next(struct doveadm_mail_iter *iter, diff -r 2d8bafd11569 -r 782570f644f7 src/doveadm/doveadm-mail-iter.h --- a/src/doveadm/doveadm-mail-iter.h Thu Apr 19 18:15:37 2012 +0300 +++ b/src/doveadm/doveadm-mail-iter.h Thu Apr 19 18:15:56 2012 +0300 @@ -13,6 +13,8 @@ struct doveadm_mail_iter **iter_r); int doveadm_mail_iter_deinit(struct doveadm_mail_iter **iter); int doveadm_mail_iter_deinit_sync(struct doveadm_mail_iter **iter); +int doveadm_mail_iter_deinit_keep_box(struct doveadm_mail_iter **iter, + struct mailbox **box_r); void doveadm_mail_iter_deinit_rollback(struct doveadm_mail_iter **iter); bool doveadm_mail_iter_next(struct doveadm_mail_iter *iter, From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: LMTP: Don't add Return-Path: header when proxying. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/85b3975c9008 changeset: 14465:85b3975c9008 user: Timo Sirainen date: Thu Apr 19 18:22:39 2012 +0300 description: LMTP: Don't add Return-Path: header when proxying. The backend also adds it and we don't want duplicates. diffstat: src/lmtp/commands.c | 13 ++++++++++--- 1 files changed, 10 insertions(+), 3 deletions(-) diffs (30 lines): diff -r 782570f644f7 -r 85b3975c9008 src/lmtp/commands.c --- a/src/lmtp/commands.c Thu Apr 19 18:15:56 2012 +0300 +++ b/src/lmtp/commands.c Thu Apr 19 18:22:39 2012 +0300 @@ -416,6 +416,9 @@ } if (client->proxy != NULL) { + /* NOTE: if this restriction is ever removed, we'll also need + to send different message bodies to local and proxy + (with and without Return-Path: header) */ client_send_line(client, "451 4.3.0 <%s> " "Can't handle mixed proxy/non-proxy destinations", address); @@ -742,9 +745,13 @@ rcpt_to = rcpt->address; } - str_printfa(str, "Return-Path: <%s>\r\n", client->state.mail_from); - if (rcpt_to != NULL) - str_printfa(str, "Delivered-To: <%s>\r\n", rcpt_to); + /* don't set Return-Path when proxying so it won't get added twice */ + if (array_count(&client->state.rcpt_to) > 0) { + str_printfa(str, "Return-Path: <%s>\r\n", + client->state.mail_from); + if (rcpt_to != NULL) + str_printfa(str, "Delivered-To: <%s>\r\n", rcpt_to); + } str_printfa(str, "Received: from %s", client->lhlo); if ((host = net_ip2addr(&client->remote_ip)) != NULL) From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: raw storage: Name autocreated namespace as "raw-sto... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/90d7364b8c9d changeset: 14466:90d7364b8c9d user: Timo Sirainen date: Thu Apr 19 18:59:33 2012 +0300 description: raw storage: Name autocreated namespace as "raw-storage". diffstat: src/lib-storage/index/raw/raw-storage.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff -r 85b3975c9008 -r 90d7364b8c9d src/lib-storage/index/raw/raw-storage.c --- a/src/lib-storage/index/raw/raw-storage.c Thu Apr 19 18:22:39 2012 +0300 +++ b/src/lib-storage/index/raw/raw-storage.c Thu Apr 19 18:59:33 2012 +0300 @@ -28,6 +28,7 @@ i_fatal("Raw user initialization failed: %s", error); ns_set = p_new(user->pool, struct mail_namespace_settings, 1); + ns_set->name = "raw-storage"; ns_set->location = ":LAYOUT=none"; ns_set->separator = "/"; From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: director: Increased timeouts. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/110673e82868 changeset: 14467:110673e82868 user: Timo Sirainen date: Thu Apr 19 20:29:25 2012 +0300 description: director: Increased timeouts. diffstat: src/director/director-connection.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (18 lines): diff -r 90d7364b8c9d -r 110673e82868 src/director/director-connection.c --- a/src/director/director-connection.c Thu Apr 19 18:59:33 2012 +0300 +++ b/src/director/director-connection.c Thu Apr 19 20:29:25 2012 +0300 @@ -51,12 +51,12 @@ #define OUTBUF_FLUSH_THRESHOLD (1024*128) /* Max idling time before "ME" command must have been received, or we'll disconnect. */ -#define DIRECTOR_CONNECTION_ME_TIMEOUT_MSECS (2*1000) +#define DIRECTOR_CONNECTION_ME_TIMEOUT_MSECS (10*1000) /* Max idling time before "DONE" command must have been received, or we'll disconnect. */ #define DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS (30*1000) /* How long to wait for PONG for an idling connection */ -#define DIRECTOR_CONNECTION_PING_IDLE_TIMEOUT_MSECS (2*1000) +#define DIRECTOR_CONNECTION_PING_IDLE_TIMEOUT_MSECS (10*1000) /* Maximum time to wait for PONG reply */ #define DIRECTOR_CONNECTION_PONG_TIMEOUT_MSECS (60*1000) /* How long to wait to send PING when connection is idle */ From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: director: Never set user's timestamp higher than ou... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/9104a9074a5a changeset: 14469:9104a9074a5a user: Timo Sirainen date: Thu Apr 19 22:17:34 2012 +0300 description: director: Never set user's timestamp higher than our ioloop_time. diffstat: src/director/user-directory.c | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diffs (20 lines): diff -r 0b4295b48941 -r 9104a9074a5a src/director/user-directory.c --- a/src/director/user-directory.c Thu Apr 19 21:51:48 2012 +0300 +++ b/src/director/user-directory.c Thu Apr 19 22:17:34 2012 +0300 @@ -132,11 +132,12 @@ struct user *user; if (timestamp == (time_t)-1) { - /* make sure we add it at the end */ + /* add it at the end */ timestamp = ioloop_time; - if (dir->tail != NULL && - timestamp < (time_t)dir->tail->timestamp) - timestamp = (time_t)dir->tail->timestamp; + } else { + /* make sure we don't add timestamps higher than ioloop time */ + if (timestamp > ioloop_time) + timestamp = ioloop_time; } user = i_new(struct user, 1); From dovecot at dovecot.org Sun May 20 03:26:32 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:32 +0300 Subject: dovecot-2.2: director: Optimized adding users to linked list dur... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/0b4295b48941 changeset: 14468:0b4295b48941 user: Timo Sirainen date: Thu Apr 19 21:51:48 2012 +0300 description: director: Optimized adding users to linked list during handshake. diffstat: src/director/Makefile.am | 20 ++++++- src/director/director-connection.c | 4 +- src/director/director-request.c | 2 +- src/director/director.c | 2 +- src/director/test-user-directory.c | 104 +++++++++++++++++++++++++++++++++++++ src/director/user-directory.c | 83 ++++++++++++++++++++++++---- src/director/user-directory.h | 3 +- 7 files changed, 199 insertions(+), 19 deletions(-) diffs (truncated from 327 to 300 lines): diff -r 110673e82868 -r 0b4295b48941 src/director/Makefile.am --- a/src/director/Makefile.am Thu Apr 19 20:29:25 2012 +0300 +++ b/src/director/Makefile.am Thu Apr 19 21:51:48 2012 +0300 @@ -4,6 +4,7 @@ AM_CPPFLAGS = \ -I$(top_srcdir)/src/lib \ + -I$(top_srcdir)/src/lib-test \ -I$(top_srcdir)/src/lib-auth \ -I$(top_srcdir)/src/lib-imap \ -I$(top_srcdir)/src/lib-settings \ @@ -40,10 +41,27 @@ notify-connection.h \ user-directory.h -noinst_PROGRAMS = director-test +noinst_PROGRAMS = director-test $(test_programs) director_test_LDADD = $(LIBDOVECOT) director_test_DEPENDENCIES = $(LIBDOVECOT_DEPS) director_test_SOURCES = \ director-test.c + +test_programs = \ + test-user-directory + +test_libs = \ + ../lib-test/libtest.la \ + ../lib/liblib.la + +test_user_directory_SOURCES = test-user-directory.c +test_user_directory_LDADD = user-directory.o $(test_libs) +test_user_directory_DEPENDENCIES = user-directory.o $(test_libs) + +check: check-am check-test +check-test: all-am + for bin in $(test_programs); do \ + if ! $(RUN_TEST) ./$$bin; then exit 1; fi; \ + done diff -r 110673e82868 -r 0b4295b48941 src/director/director-connection.c --- a/src/director/director-connection.c Thu Apr 19 20:29:25 2012 +0300 +++ b/src/director/director-connection.c Thu Apr 19 21:51:48 2012 +0300 @@ -540,7 +540,7 @@ } if (director_user_refresh(conn, username_hash, - host, ioloop_time, FALSE, &user)) { + host, (time_t)-1, FALSE, &user)) { i_assert(!user->weak); director_update_user(conn->dir, conn->host, user); } @@ -674,7 +674,7 @@ } if (director_user_refresh(conn, username_hash, - host, ioloop_time, weak, &user)) { + host, (time_t)-1, weak, &user)) { if (!user->weak) director_update_user(conn->dir, src_host, user); else { diff -r 110673e82868 -r 0b4295b48941 src/director/director-request.c --- a/src/director/director-request.c Thu Apr 19 20:29:25 2012 +0300 +++ b/src/director/director-request.c Thu Apr 19 21:51:48 2012 +0300 @@ -225,7 +225,7 @@ return FALSE; } user = user_directory_add(dir->users, request->username_hash, - host, ioloop_time); + host, (time_t)-1); } i_assert(!user->weak); diff -r 110673e82868 -r 0b4295b48941 src/director/director.c --- a/src/director/director.c Thu Apr 19 20:29:25 2012 +0300 +++ b/src/director/director.c Thu Apr 19 21:51:48 2012 +0300 @@ -582,7 +582,7 @@ user = user_directory_lookup(dir->users, username_hash); if (user == NULL) { user = user_directory_add(dir->users, username_hash, - host, ioloop_time); + host, (time_t)-1); } else { if (user->host == host) { /* user is already in this host */ diff -r 110673e82868 -r 0b4295b48941 src/director/test-user-directory.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/director/test-user-directory.c Thu Apr 19 21:51:48 2012 +0300 @@ -0,0 +1,104 @@ +/* Copyright (c) 2012 Dovecot authors, see the included COPYING file */ + +#include "lib.h" +#include "ioloop.h" +#include "mail-user-hash.h" +#include "mail-host.h" +#include "user-directory.h" +#include "test-common.h" + +#include + +#define USER_DIR_TIMEOUT 1000000 + +unsigned int mail_user_hash(const char *username ATTR_UNUSED, + const char *format ATTR_UNUSED) { return 0; } + +static void +verify_user_directory(struct user_directory *dir, unsigned int user_count) +{ + struct user_directory_iter *iter; + struct user *user, *prev = NULL; + unsigned int prev_stamp = 0, iter_count = 0; + + iter = user_directory_iter_init(dir); + while ((user = user_directory_iter_next(iter)) != NULL) { + test_assert(prev_stamp <= user->timestamp); + test_assert(user->prev == prev); + test_assert(prev == NULL || user->prev->next == user); + + iter_count++; + prev = user; + } + test_assert(prev == NULL || prev->next == NULL); + user_directory_iter_deinit(&iter); + test_assert(iter_count == user_count); +} + +static void test_user_directory_ascending(void) +{ + const unsigned int count = 100000; + struct user_directory *dir; + struct mail_host *host = t_new(struct mail_host, 1); + unsigned int i; + + test_begin("user directory ascending"); + dir = user_directory_init(USER_DIR_TIMEOUT, "%u"); + user_directory_add(dir, 1, host, ioloop_time + count+1); + + for (i = 0; i < count; i++) + user_directory_add(dir, i+2, host, ioloop_time + i); + verify_user_directory(dir, count+1); + user_directory_deinit(&dir); + test_end(); +} + +static void test_user_directory_descending(void) +{ + const unsigned int count = 1000; + struct user_directory *dir; + struct mail_host *host = t_new(struct mail_host, 1); + unsigned int i; + + test_begin("user directory descending"); + dir = user_directory_init(USER_DIR_TIMEOUT, "%u"); + + for (i = 0; i < count; i++) + user_directory_add(dir, i+1, host, ioloop_time - i); + verify_user_directory(dir, count); + user_directory_deinit(&dir); + test_end(); +} + +static void test_user_directory_random(void) +{ + struct user_directory *dir; + struct mail_host *host = t_new(struct mail_host, 1); + time_t timestamp; + unsigned int i, count = 10000 + rand()%10000; + + test_begin("user directory random"); + dir = user_directory_init(USER_DIR_TIMEOUT, "%u"); + for (i = 0; i < count; i++) { + if (rand() % 10 == 0) + timestamp = (time_t)-1; + else + timestamp = ioloop_time-rand()%100; + user_directory_add(dir, i+1, host, timestamp); + } + verify_user_directory(dir, count); + user_directory_deinit(&dir); + test_end(); +} + +int main(void) +{ + static void (*test_functions[])(void) = { + test_user_directory_ascending, + test_user_directory_descending, + test_user_directory_random, + NULL + }; + ioloop_time = 1234567890; + return test_run(test_functions); +} diff -r 110673e82868 -r 0b4295b48941 src/director/user-directory.c --- a/src/director/user-directory.c Thu Apr 19 20:29:25 2012 +0300 +++ b/src/director/user-directory.c Thu Apr 19 21:51:48 2012 +0300 @@ -24,6 +24,7 @@ struct hash_table *hash; /* sorted by time */ struct user *head, *tail; + struct user *prev_insert_pos; ARRAY_DEFINE(iters, struct user_directory_iter *); @@ -42,6 +43,9 @@ if ((*iterp)->pos == user) (*iterp)->pos = user->next; } + + if (dir->prev_insert_pos == user) + dir->prev_insert_pos = user->next; } static void user_free(struct user_directory *dir, struct user *user) @@ -79,11 +83,61 @@ return hash_table_lookup(dir->hash, POINTER_CAST(username_hash)); } +static void +user_directory_insert_backwards(struct user_directory *dir, + struct user *pos, struct user *user) +{ + for (; pos != NULL; pos = pos->prev) { + if ((time_t)pos->timestamp <= user->timestamp) + break; + } + if (pos == NULL) + DLLIST2_PREPEND(&dir->head, &dir->tail, user); + else { + user->prev = pos; + user->next = pos->next; + user->prev->next = user; + if (user->next != NULL) + user->next->prev = user; + else + dir->tail = user; + } +} + +static void +user_directory_insert_forwards(struct user_directory *dir, + struct user *pos, struct user *user) +{ + for (; pos != NULL; pos = pos->next) { + if ((time_t)pos->timestamp >= user->timestamp) + break; + } + if (pos == NULL) + DLLIST2_APPEND(&dir->head, &dir->tail, user); + else { + user->prev = pos->prev; + user->next = pos; + if (user->prev != NULL) + user->prev->next = user; + else + dir->head = user; + user->next->prev = user; + } +} + struct user * user_directory_add(struct user_directory *dir, unsigned int username_hash, struct mail_host *host, time_t timestamp) { - struct user *user, *pos; + struct user *user; + + if (timestamp == (time_t)-1) { + /* make sure we add it at the end */ + timestamp = ioloop_time; + if (dir->tail != NULL && + timestamp < (time_t)dir->tail->timestamp) + timestamp = (time_t)dir->tail->timestamp; + } user = i_new(struct user, 1); user->username_hash = username_hash; @@ -94,21 +148,24 @@ if (dir->tail == NULL || (time_t)dir->tail->timestamp <= timestamp) DLLIST2_APPEND(&dir->head, &dir->tail, user); else { - /* need to insert to correct position */ - for (pos = dir->tail; pos != NULL; pos = pos->prev) { - if ((time_t)pos->timestamp <= timestamp) - break; - } - if (pos == NULL) - DLLIST2_PREPEND(&dir->head, &dir->tail, user); - else { - user->prev = pos; - user->next = pos->next; - user->prev->next = user; - user->next->prev = user; + /* need to insert to correct position. we should get here + only when handshaking. the handshaking USER requests should + come sorted by timestamp. so keep track of the previous + insert position, the next USER should be inserted after + it. */ + if (dir->prev_insert_pos == NULL) { + /* find the position starting from tail */ From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: Use t_strsplit_tab() wherever possible Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/fc8031c5e691 changeset: 14471:fc8031c5e691 user: Timo Sirainen date: Thu Apr 19 23:06:55 2012 +0300 description: Use t_strsplit_tab() wherever possible diffstat: src/anvil/anvil-connection.c | 2 +- src/auth/auth-master-connection.c | 6 +++--- src/auth/auth-request-handler.c | 4 ++-- src/auth/auth-stream.c | 2 +- src/auth/auth-worker-client.c | 2 +- src/auth/db-checkpassword.c | 2 +- src/auth/passdb-blocking.c | 2 +- src/auth/passdb-cache.c | 4 ++-- src/config/config-connection.c | 2 +- src/dict/dict-commands.c | 8 ++++---- src/director/director-connection.c | 2 +- src/director/director-test.c | 2 +- src/director/doveadm-connection.c | 4 ++-- src/director/login-connection.c | 2 +- src/doveadm/doveadm-director.c | 10 +++++----- src/doveadm/doveadm-penalty.c | 2 +- src/doveadm/doveadm-proxy.c | 2 +- src/doveadm/doveadm-who.c | 2 +- src/ipc/ipc-connection.c | 2 +- src/lib-auth/auth-master.c | 4 ++-- src/lib-auth/auth-server-connection.c | 2 +- src/lib-master/master-login-auth.c | 4 ++-- src/lib-master/master-login.c | 2 +- src/lib-master/master-service-settings.c | 2 +- src/login-common/login-proxy.c | 2 +- src/util/script-login.c | 2 +- 26 files changed, 40 insertions(+), 40 deletions(-) diffs (truncated from 438 to 300 lines): diff -r 7eb9688c266f -r fc8031c5e691 src/anvil/anvil-connection.c --- a/src/anvil/anvil-connection.c Thu Apr 19 23:00:16 2012 +0300 +++ b/src/anvil/anvil-connection.c Thu Apr 19 23:06:55 2012 +0300 @@ -40,7 +40,7 @@ const char *line; line = i_stream_next_line(conn->input); - return line == NULL ? NULL : t_strsplit(line, "\t"); + return line == NULL ? NULL : t_strsplit_tab(line); } static int diff -r 7eb9688c266f -r fc8031c5e691 src/auth/auth-master-connection.c --- a/src/auth/auth-master-connection.c Thu Apr 19 23:00:16 2012 +0300 +++ b/src/auth/auth-master-connection.c Thu Apr 19 23:06:55 2012 +0300 @@ -100,7 +100,7 @@ buffer_t buf; /* */ - list = t_strsplit(args, "\t"); + list = t_strsplit_tab(args); if (str_array_length(list) < 4 || str_to_uint(list[0], &id) < 0 || str_to_uint(list[1], &client_pid) < 0 || @@ -146,7 +146,7 @@ unsigned int id; /* [] */ - list = t_strsplit(args, "\t"); + list = t_strsplit_tab(args); if (list[0] == NULL || list[1] == NULL || str_to_uint(list[0], &id) < 0) { i_error("BUG: Master sent broken %s", cmd); @@ -485,7 +485,7 @@ unsigned int id; /* [] */ - list = t_strsplit(args, "\t"); + list = t_strsplit_tab(args); if (list[0] == NULL || str_to_uint(list[0], &id) < 0) { i_error("BUG: Master sent broken LIST"); return -1; diff -r 7eb9688c266f -r fc8031c5e691 src/auth/auth-request-handler.c --- a/src/auth/auth-request-handler.c Thu Apr 19 23:00:16 2012 +0300 +++ b/src/auth/auth-request-handler.c Thu Apr 19 23:06:55 2012 +0300 @@ -170,7 +170,7 @@ } } - fields = t_strsplit(extra_fields, "\t"); + fields = t_strsplit_tab(extra_fields); for (src = 0; fields[src] != NULL; src++) { if (strncmp(fields[src], "userdb_", 7) != 0) { if (!seen_pass && strncmp(fields[src], "pass=", 5) == 0) @@ -439,7 +439,7 @@ i_assert(!handler->destroyed); /* [...] */ - list = t_strsplit(args, "\t"); + list = t_strsplit_tab(args); if (list[0] == NULL || list[1] == NULL || str_to_uint(list[0], &id) < 0) { i_error("BUG: Authentication client %u " diff -r 7eb9688c266f -r fc8031c5e691 src/auth/auth-stream.c --- a/src/auth/auth-stream.c Thu Apr 19 23:00:16 2012 +0300 +++ b/src/auth/auth-stream.c Thu Apr 19 23:06:55 2012 +0300 @@ -140,7 +140,7 @@ const char *const *auth_stream_split(struct auth_stream_reply *reply) { - return t_strsplit(str_c(reply->str), "\t"); + return t_strsplit_tab(str_c(reply->str)); } string_t *auth_stream_reply_get_str(struct auth_stream_reply *reply) diff -r 7eb9688c266f -r fc8031c5e691 src/auth/auth-worker-client.c --- a/src/auth/auth-worker-client.c Thu Apr 19 23:00:16 2012 +0300 +++ b/src/auth/auth-worker-client.c Thu Apr 19 23:06:55 2012 +0300 @@ -547,7 +547,7 @@ unsigned int id; bool ret = FALSE; - args = t_strsplit(line, "\t"); + args = t_strsplit_tab(line); if (args[0] == NULL || args[1] == NULL || args[2] == NULL || str_to_uint(args[0], &id) < 0) { i_error("BUG: Invalid input: %s", line); diff -r 7eb9688c266f -r fc8031c5e691 src/auth/db-checkpassword.c --- a/src/auth/db-checkpassword.c Thu Apr 19 23:00:16 2012 +0300 +++ b/src/auth/db-checkpassword.c Thu Apr 19 23:06:55 2012 +0300 @@ -54,7 +54,7 @@ const char *const *tmp; const char *key, *p; - for (tmp = t_strsplit(extra_fields, "\t"); *tmp != NULL; tmp++) { + for (tmp = t_strsplit_tab(extra_fields); *tmp != NULL; tmp++) { key = t_str_ucase(t_strcut(*tmp, '=')); p = strchr(*tmp, '='); if (p == NULL) diff -r 7eb9688c266f -r fc8031c5e691 src/auth/passdb-blocking.c --- a/src/auth/passdb-blocking.c Thu Apr 19 23:00:16 2012 +0300 +++ b/src/auth/passdb-blocking.c Thu Apr 19 23:06:55 2012 +0300 @@ -30,7 +30,7 @@ enum passdb_result ret; const char *const *args; - args = t_strsplit(reply, "\t"); + args = t_strsplit_tab(reply); if (strcmp(*args, "OK") == 0 && args[1] != NULL && args[2] != NULL) { /* OK \t user \t password [\t extra] */ diff -r 7eb9688c266f -r fc8031c5e691 src/auth/passdb-cache.c --- a/src/auth/passdb-cache.c Thu Apr 19 23:00:16 2012 +0300 +++ b/src/auth/passdb-cache.c Thu Apr 19 23:06:55 2012 +0300 @@ -53,7 +53,7 @@ return TRUE; } - list = t_strsplit(value, "\t"); + list = t_strsplit_tab(value); cached_pw = list[0]; if (*cached_pw == '\0') { @@ -117,7 +117,7 @@ return TRUE; } - list = t_strsplit(value, "\t"); + list = t_strsplit_tab(value); auth_request_set_fields(request, list + 1, NULL); *result_r = PASSDB_RESULT_OK; diff -r 7eb9688c266f -r fc8031c5e691 src/config/config-connection.c --- a/src/config/config-connection.c Thu Apr 19 23:00:16 2012 +0300 +++ b/src/config/config-connection.c Thu Apr 19 23:06:55 2012 +0300 @@ -42,7 +42,7 @@ if (line == NULL) return NULL; - return t_strsplit(line, "\t"); + return t_strsplit_tab(line); } static void diff -r 7eb9688c266f -r fc8031c5e691 src/dict/dict-commands.c --- a/src/dict/dict-commands.c Thu Apr 19 23:00:16 2012 +0300 +++ b/src/dict/dict-commands.c Thu Apr 19 23:06:55 2012 +0300 @@ -90,7 +90,7 @@ return -1; } - args = t_strsplit(line, "\t"); + args = t_strsplit_tab(line); if (str_array_length(args) < 2 || str_to_uint(args[0], &flags) < 0) { i_error("dict client: ITERATE: broken input"); @@ -272,7 +272,7 @@ const char *const *args; /* */ - args = t_strsplit(line, "\t"); + args = t_strsplit_tab(line); if (str_array_length(args) != 3) { i_error("dict client: SET: broken input"); return -1; @@ -291,7 +291,7 @@ const char *const *args; /* */ - args = t_strsplit(line, "\t"); + args = t_strsplit_tab(line); if (str_array_length(args) != 2) { i_error("dict client: UNSET: broken input"); return -1; @@ -311,7 +311,7 @@ long long diff; /* */ - args = t_strsplit(line, "\t"); + args = t_strsplit_tab(line); if (str_array_length(args) != 3 || str_to_llong(args[2], &diff) < 0) { i_error("dict client: ATOMIC_INC: broken input"); diff -r 7eb9688c266f -r fc8031c5e691 src/director/director-connection.c --- a/src/director/director-connection.c Thu Apr 19 23:00:16 2012 +0300 +++ b/src/director/director-connection.c Thu Apr 19 23:06:55 2012 +0300 @@ -1169,7 +1169,7 @@ const char *cmd, *const *args; bool ret; - args = t_strsplit(line, "\t"); + args = t_strsplit_tab(line); cmd = args[0]; args++; if (cmd == NULL) { i_error("director(%s): Received empty line", conn->name); diff -r 7eb9688c266f -r fc8031c5e691 src/director/director-test.c --- a/src/director/director-test.c Thu Apr 19 23:00:16 2012 +0300 +++ b/src/director/director-test.c Thu Apr 19 23:06:55 2012 +0300 @@ -492,7 +492,7 @@ break; /* ip vhost-count user-count */ T_BEGIN { - const char *const *args = t_strsplit(line, "\t"); + const char *const *args = t_strsplit_tab(line); struct host *host; host = i_new(struct host, 1); diff -r 7eb9688c266f -r fc8031c5e691 src/director/doveadm-connection.c --- a/src/director/doveadm-connection.c Thu Apr 19 23:00:16 2012 +0300 +++ b/src/director/doveadm-connection.c Thu Apr 19 23:06:55 2012 +0300 @@ -137,7 +137,7 @@ struct ip_addr ip; unsigned int vhost_count = -1U; - args = t_strsplit(line, "\t"); + args = t_strsplit_tab(line); if (args[0] == NULL || net_addr2ip(args[0], &ip) < 0 || (args[1] != NULL && str_to_uint(args[1], &vhost_count) < 0)) { @@ -300,7 +300,7 @@ struct mail_host *host; struct ip_addr ip; - args = t_strsplit(line, "\t"); + args = t_strsplit_tab(line); if (args[0] == NULL || args[1] == NULL || net_addr2ip(args[1], &ip) < 0) { i_error("doveadm sent invalid USER-MOVE parameters: %s", line); diff -r 7eb9688c266f -r fc8031c5e691 src/director/login-connection.c --- a/src/director/login-connection.c Thu Apr 19 23:00:16 2012 +0300 +++ b/src/director/login-connection.c Thu Apr 19 23:06:55 2012 +0300 @@ -121,7 +121,7 @@ } /* OK [] */ - args = t_strsplit(line_params, "\t"); + args = t_strsplit_tab(line_params); if (*args != NULL) { /* we should always get here, but in case we don't just forward as-is and let login process handle the error. */ diff -r 7eb9688c266f -r fc8031c5e691 src/doveadm/doveadm-director.c --- a/src/doveadm/doveadm-director.c Thu Apr 19 23:00:16 2012 +0300 +++ b/src/doveadm/doveadm-director.c Thu Apr 19 23:06:55 2012 +0300 @@ -120,7 +120,7 @@ return; } - args = t_strsplit(line, "\t"); + args = t_strsplit_tab(line); if (str_array_length(args) != 4 || str_to_uint(args[1], &expires) < 0) { i_error("Invalid reply from director"); @@ -162,7 +162,7 @@ if (*line == '\0') break; T_BEGIN { - args = t_strsplit(line, "\t"); + args = t_strsplit_tab(line); if (str_array_length(args) >= 3) { doveadm_print(args[0]); doveadm_print(args[1]); @@ -315,7 +315,7 @@ if (*line == '\0') break; T_BEGIN { - args = t_strsplit(line, "\t"); + args = t_strsplit_tab(line); if (str_array_length(args) < 3 || str_to_uint(args[0], &user_hash) < 0 || str_to_uint(args[1], &expires) < 0 || @@ -559,7 +559,7 @@ if (*line == '\0') break; T_BEGIN { - args = t_strsplit(line, "\t"); + args = t_strsplit_tab(line); if (str_array_length(args) >= 2) { director_dump_cmd(ctx, "add", "%s %s", args[0], args[1]); @@ -599,7 +599,7 @@ if (*line == '\0') break; T_BEGIN { - args = t_strsplit(line, "\t"); + args = t_strsplit_tab(line); if (str_array_length(args) >= 4 && str_to_ulong(args[3], &l) == 0) { doveadm_print(args[0]); diff -r 7eb9688c266f -r fc8031c5e691 src/doveadm/doveadm-penalty.c --- a/src/doveadm/doveadm-penalty.c Thu Apr 19 23:00:16 2012 +0300 +++ b/src/doveadm/doveadm-penalty.c Thu Apr 19 23:06:55 2012 +0300 @@ -27,7 +27,7 @@ static void penalty_parse_line(const char *line, struct penalty_line *line_r) { - const char *const *args = t_strsplit(line, "\t"); + const char *const *args = t_strsplit_tab(line); const char *ident = args[0]; const char *penalty_str = args[1]; const char *last_penalty_str = args[2]; From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: Added t_strsplit_tab() Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/7eb9688c266f changeset: 14470:7eb9688c266f user: Timo Sirainen date: Thu Apr 19 23:00:16 2012 +0300 description: Added t_strsplit_tab() diffstat: src/lib/strfuncs.c | 41 ++++++++++++++++++++++++++++++++++++ src/lib/strfuncs.h | 2 + src/lib/test-strfuncs.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 98 insertions(+), 0 deletions(-) diffs (136 lines): diff -r 9104a9074a5a -r 7eb9688c266f src/lib/strfuncs.c --- a/src/lib/strfuncs.c Thu Apr 19 22:17:34 2012 +0300 +++ b/src/lib/strfuncs.c Thu Apr 19 23:00:16 2012 +0300 @@ -463,6 +463,47 @@ return split_str(pool, data, separators, TRUE); } +const char **t_strsplit_tab(const char *data) +{ + const char **array; + char *dest; + unsigned int i, array_idx, array_size, dest_size; + + if (*data == '\0') + return t_new(const char *, 1); + + array_size = 1; + dest_size = 128; + dest = t_buffer_get(dest_size+1); + for (i = 0; data[i] != '\0'; i++) { + if (i >= dest_size) { + dest_size = nearest_power(dest_size+1); + dest = t_buffer_reget(dest, dest_size+1); + } + if (data[i] != '\t') + dest[i] = data[i]; + else { + dest[i] = '\0'; + array_size++; + } + } + i_assert(i <= dest_size); + dest[i] = '\0'; + t_buffer_alloc(i+1); + dest_size = i; + + array = t_new(const char *, array_size + 1); + array[0] = dest; array_idx = 1; + + for (i = 0; i < dest_size; i++) { + if (dest[i] == '\0') + array[array_idx++] = dest+i+1; + } + i_assert(array_idx == array_size); + array[array_idx] = NULL; + return array; +} + void p_strsplit_free(pool_t pool, char **arr) { p_free(pool, arr[0]); diff -r 9104a9074a5a -r 7eb9688c266f src/lib/strfuncs.h --- a/src/lib/strfuncs.h Thu Apr 19 22:17:34 2012 +0300 +++ b/src/lib/strfuncs.h Thu Apr 19 23:00:16 2012 +0300 @@ -66,6 +66,8 @@ const char **t_strsplit_spaces(const char *data, const char *separators) ATTR_MALLOC; void p_strsplit_free(pool_t pool, char **arr); +/* Optimized version of t_strsplit(data, "\t") */ +const char **t_strsplit_tab(const char *data); const char *dec2str(uintmax_t number); diff -r 9104a9074a5a -r 7eb9688c266f src/lib/test-strfuncs.c --- a/src/lib/test-strfuncs.c Thu Apr 19 22:17:34 2012 +0300 +++ b/src/lib/test-strfuncs.c Thu Apr 19 23:00:16 2012 +0300 @@ -2,6 +2,8 @@ #include "test-lib.h" +#include + static void test_p_strarray_dup(void) { const char *input[][3] = { @@ -27,7 +29,60 @@ test_end(); } +static void strsplit_verify(const char *str) +{ + T_BEGIN { + const char **s1, **s2; + unsigned int i; + + s1 = t_strsplit_tab(str); + s2 = t_strsplit(str, "\t"); + for (i = 0; s1[i] != NULL; i++) { + test_assert(s2[i] != NULL); + test_assert(strcmp(s1[i], s2[i]) == 0); + } + test_assert(s2[i] == NULL); + } T_END; +} + +static void test_t_strsplit_tab(void) +{ + char buf[4096]; + unsigned int i, j, max; + + test_begin("t_strsplit_tab"); + strsplit_verify(""); + strsplit_verify("\t"); + strsplit_verify("\t\t"); + strsplit_verify("foo"); + strsplit_verify("foo\tbar"); + strsplit_verify("foo\tbar\tbaz"); + strsplit_verify("foo\t\tbaz"); + buf[sizeof(buf)-1] = '\0'; + for (i = 0; i < sizeof(buf)-1; i++) + buf[i] = '\t'; + strsplit_verify(buf); + for (j = 0; j < 256; j++) { + memset(buf, '\t', j); + buf[j+1] = '\0'; + strsplit_verify(buf); + } + for (j = 0; j < 100; j++) { + max = (rand() % sizeof(buf)) + 1; + buf[--max] = '\0'; + for (i = 0; i < max; i++) { + if (rand() % 10 == 0) + buf[i] = '\t'; + else + buf[i] = 'x'; + } + strsplit_verify(buf); + } + test_end(); +} + void test_strfuncs(void) { test_p_strarray_dup(); + test_t_strsplit_tab(); } From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: Decreased timeout for resending lost SYNC... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/3acc1451ab56 changeset: 14472:3acc1451ab56 user: Timo Sirainen date: Thu Apr 19 23:34:41 2012 +0300 description: director: Decreased timeout for resending lost SYNC request. diffstat: src/director/director.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r fc8031c5e691 -r 3acc1451ab56 src/director/director.c --- a/src/director/director.c Thu Apr 19 23:06:55 2012 +0300 +++ b/src/director/director.c Thu Apr 19 23:34:41 2012 +0300 @@ -17,7 +17,7 @@ #define DIRECTOR_RECONNECT_TIMEOUT_MSECS (30*1000) #define DIRECTOR_USER_MOVE_TIMEOUT_MSECS (30*1000) #define DIRECTOR_USER_MOVE_FINISH_DELAY_MSECS (2*1000) -#define DIRECTOR_SYNC_TIMEOUT_MSECS (15*1000) +#define DIRECTOR_SYNC_TIMEOUT_MSECS (5*1000) static bool director_is_self_ip_set(struct director *dir) { From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: timeout_reset(): Don't use cached ioloop_timeval. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/266521d59e2c changeset: 14473:266521d59e2c user: Timo Sirainen date: Thu Apr 19 23:41:59 2012 +0300 description: timeout_reset(): Don't use cached ioloop_timeval. Timeouts are sometimes reset after a long running callback. In those cases a cached timeval may be too far behind. diffstat: src/lib/ioloop.c | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diffs (13 lines): diff -r 3acc1451ab56 -r 266521d59e2c src/lib/ioloop.c --- a/src/lib/ioloop.c Thu Apr 19 23:34:41 2012 +0300 +++ b/src/lib/ioloop.c Thu Apr 19 23:41:59 2012 +0300 @@ -206,8 +206,7 @@ void timeout_reset(struct timeout *timeout) { - timeout_reset_timeval(timeout, timeout->ioloop->running ? NULL : - &ioloop_timeval); + timeout_reset_timeval(timeout, NULL); } static int timeout_get_wait_time(struct timeout *timeout, struct timeval *tv_r, From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: Reverted previous change, it didn't work ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/9ef4dee0792f changeset: 14475:9ef4dee0792f user: Timo Sirainen date: Thu Apr 19 23:51:46 2012 +0300 description: director: Reverted previous change, it didn't work properly. diffstat: src/director/director-connection.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diffs (21 lines): diff -r 25c941423e42 -r 9ef4dee0792f src/director/director-connection.c --- a/src/director/director-connection.c Thu Apr 19 23:45:31 2012 +0300 +++ b/src/director/director-connection.c Thu Apr 19 23:51:46 2012 +0300 @@ -1282,6 +1282,9 @@ user_directory_iter_deinit(&conn->user_iter); director_connection_send(conn, "DONE\n"); + i_assert(conn->io == NULL); + conn->io = io_add(conn->fd, IO_READ, director_connection_input, conn); + ret = o_stream_flush(conn->output); timeout_reset(conn->to_ping); return ret; @@ -1362,7 +1365,6 @@ director_connection_output, conn); io_remove(&conn->io); - conn->io = io_add(conn->fd, IO_READ, director_connection_input, conn); o_stream_cork(conn->output); director_connection_send_handshake(conn); From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: Don't delay reading input from remote con... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/25c941423e42 changeset: 14474:25c941423e42 user: Timo Sirainen date: Thu Apr 19 23:45:31 2012 +0300 description: director: Don't delay reading input from remote connection during handshake. diffstat: src/director/director-connection.c | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) diffs (21 lines): diff -r 266521d59e2c -r 25c941423e42 src/director/director-connection.c --- a/src/director/director-connection.c Thu Apr 19 23:41:59 2012 +0300 +++ b/src/director/director-connection.c Thu Apr 19 23:45:31 2012 +0300 @@ -1282,9 +1282,6 @@ user_directory_iter_deinit(&conn->user_iter); director_connection_send(conn, "DONE\n"); - i_assert(conn->io == NULL); - conn->io = io_add(conn->fd, IO_READ, director_connection_input, conn); - ret = o_stream_flush(conn->output); timeout_reset(conn->to_ping); return ret; @@ -1365,6 +1362,7 @@ director_connection_output, conn); io_remove(&conn->io); + conn->io = io_add(conn->fd, IO_READ, director_connection_input, conn); o_stream_cork(conn->output); director_connection_send_handshake(conn); From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: Improved error message Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/8245a97bf36c changeset: 14476:8245a97bf36c user: Timo Sirainen date: Fri Apr 20 00:03:47 2012 +0300 description: director: Improved error message diffstat: src/director/director-connection.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diffs (13 lines): diff -r 9ef4dee0792f -r 8245a97bf36c src/director/director-connection.c --- a/src/director/director-connection.c Thu Apr 19 23:51:46 2012 +0300 +++ b/src/director/director-connection.c Fri Apr 20 00:03:47 2012 +0300 @@ -134,6 +134,9 @@ if (!conn->connected) { i_error("director(%s): Connect timed out (%u secs)", conn->name, secs); + } else if (conn->io == NULL) { + i_error("director(%s): Sending handshake (%u secs)", + conn->name, secs); } else if (!conn->me_received) { i_error("director(%s): Handshaking ME timed out (%u secs)", conn->name, secs); From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: Increased timeout for sending USER data i... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/ea5b949a623b changeset: 14477:ea5b949a623b user: Timo Sirainen date: Fri Apr 20 00:13:55 2012 +0300 description: director: Increased timeout for sending USER data in handshake. diffstat: src/director/director-connection.c | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diffs (24 lines): diff -r 8245a97bf36c -r ea5b949a623b src/director/director-connection.c --- a/src/director/director-connection.c Fri Apr 20 00:03:47 2012 +0300 +++ b/src/director/director-connection.c Fri Apr 20 00:13:55 2012 +0300 @@ -52,6 +52,10 @@ /* Max idling time before "ME" command must have been received, or we'll disconnect. */ #define DIRECTOR_CONNECTION_ME_TIMEOUT_MSECS (10*1000) +/* Max time to wait for USERs in handshake to be sent. With a lot of users the + kernel may quickly eat up everything we send, while the receiver is busy + parsing the data. */ +#define DIRECTOR_CONNECTION_SEND_USERS_TIMEOUT_MSECS (120*1000) /* Max idling time before "DONE" command must have been received, or we'll disconnect. */ #define DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS (30*1000) @@ -1368,6 +1372,9 @@ director_connection_output, conn); io_remove(&conn->io); + timeout_remove(&conn->to_ping); + conn->to_ping = timeout_add(DIRECTOR_CONNECTION_SEND_USERS_TIMEOUT_MSECS, + director_connection_init_timeout, conn); o_stream_cork(conn->output); director_connection_send_handshake(conn); From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: Improved debug handshake message. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/ee169584bc44 changeset: 14478:ee169584bc44 user: Timo Sirainen date: Fri Apr 20 00:32:34 2012 +0300 description: director: Improved debug handshake message. diffstat: src/director/director-connection.c | 10 ++++++++-- 1 files changed, 8 insertions(+), 2 deletions(-) diffs (20 lines): diff -r ea5b949a623b -r ee169584bc44 src/director/director-connection.c --- a/src/director/director-connection.c Fri Apr 20 00:13:55 2012 +0300 +++ b/src/director/director-connection.c Fri Apr 20 00:32:34 2012 +0300 @@ -866,8 +866,14 @@ { struct director *dir = conn->dir; - if (dir->debug) - i_debug("Handshaked to %s", conn->host->name); + if (dir->debug) { + unsigned int secs = time(NULL)-conn->created; + + i_debug("director(%s): Handshake took %u secs, " + "bytes in=%"PRIuUOFF_T" out=%"PRIuUOFF_T, + conn->name, secs, conn->input->v_offset, + conn->output->offset); + } /* the host is up now, make sure we can connect to it immediately if needed */ From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: Fixed protocol error detection/handling. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/293ada796ae6 changeset: 14479:293ada796ae6 user: Timo Sirainen date: Fri Apr 20 00:33:13 2012 +0300 description: director: Fixed protocol error detection/handling. diffstat: src/director/director-connection.c | 30 +++++++++++++----------------- 1 files changed, 13 insertions(+), 17 deletions(-) diffs (86 lines): diff -r ee169584bc44 -r 293ada796ae6 src/director/director-connection.c --- a/src/director/director-connection.c Fri Apr 20 00:32:34 2012 +0300 +++ b/src/director/director-connection.c Fri Apr 20 00:33:13 2012 +0300 @@ -117,7 +117,7 @@ }; static void director_connection_disconnected(struct director_connection **conn); -static void director_connection_protocol_error(struct director_connection **conn); +static void director_connection_reconnect(struct director_connection **conn); static void ATTR_FORMAT(2, 3) director_cmd_error(struct director_connection *conn, const char *fmt, ...) @@ -128,6 +128,8 @@ i_error("director(%s): Command %s: %s (input: %s)", conn->name, conn->cur_cmd, t_strdup_vprintf(fmt, args), conn->cur_line); va_end(args); + + conn->host->last_protocol_failure = ioloop_time; } static void @@ -1133,10 +1135,8 @@ if (ret < 0) { /* invalid commands during handshake, we probably don't want to reconnect here */ - if (conn->dir->debug) { - i_debug("director(%s): Handshaking failed", - conn->host->name); - } + i_error("director(%s): Handshaking failed", + conn->name); return FALSE; } /* allow also other commands during handshake */ @@ -1185,7 +1185,7 @@ args = t_strsplit_tab(line); cmd = args[0]; args++; if (cmd == NULL) { - i_error("director(%s): Received empty line", conn->name); + director_cmd_error(conn, "Received empty line"); return FALSE; } @@ -1215,9 +1215,9 @@ return; case -2: /* buffer full */ - i_error("BUG: Director %s sent us more than %d bytes", - conn->name, MAX_INBUF_SIZE); - director_connection_protocol_error(&conn); + director_cmd_error(conn, "Director sent us more than %d bytes", + MAX_INBUF_SIZE); + director_connection_reconnect(&conn); return; } @@ -1228,11 +1228,9 @@ } T_END; if (!ret) { - if (dir->debug) { - i_debug("director(%s): Invalid input, disconnecting", - conn->name); - } - director_connection_protocol_error(&conn); + i_error("director(%s): Invalid input, disconnecting", + conn->name); + director_connection_reconnect(&conn); break; } } @@ -1480,13 +1478,11 @@ director_connect(dir); } -void director_connection_protocol_error(struct director_connection **_conn) +void director_connection_reconnect(struct director_connection **_conn) { struct director_connection *conn = *_conn; struct director *dir = conn->dir; - conn->host->last_protocol_failure = ioloop_time; - director_connection_deinit(_conn); if (dir->right == NULL) director_connect(dir); From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: Make sure handshaking doesn't get stuck s... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d9f33d78fa3d changeset: 14480:d9f33d78fa3d user: Timo Sirainen date: Fri Apr 20 00:42:10 2012 +0300 description: director: Make sure handshaking doesn't get stuck sending USERs. diffstat: src/director/director-connection.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff -r 293ada796ae6 -r d9f33d78fa3d src/director/director-connection.c --- a/src/director/director-connection.c Fri Apr 20 00:33:13 2012 +0300 +++ b/src/director/director-connection.c Fri Apr 20 00:42:10 2012 +0300 @@ -1312,6 +1312,8 @@ o_stream_uncork(conn->output); if (ret < 0) director_connection_disconnected(&conn); + else + o_stream_set_flush_pending(conn->output, TRUE); return ret; } return o_stream_flush(conn->output); From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: Another fix for stuck handshake. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/ce4e1bf7262d changeset: 14481:ce4e1bf7262d user: Timo Sirainen date: Fri Apr 20 00:49:29 2012 +0300 description: director: Another fix for stuck handshake. diffstat: src/director/director-connection.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (13 lines): diff -r d9f33d78fa3d -r ce4e1bf7262d src/director/director-connection.c --- a/src/director/director-connection.c Fri Apr 20 00:42:10 2012 +0300 +++ b/src/director/director-connection.c Fri Apr 20 00:49:29 2012 +0300 @@ -1389,7 +1389,8 @@ director_connection_send(conn, str_c(str)); conn->user_iter = user_directory_iter_init(dir->users); - (void)director_connection_send_users(conn); + if (director_connection_send_users(conn) == 0) + o_stream_set_flush_pending(conn->output, TRUE); o_stream_uncork(conn->output); } From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: Removed unnecessary error messages. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/0c3c4d098842 changeset: 14482:0c3c4d098842 user: Timo Sirainen date: Fri Apr 20 00:59:01 2012 +0300 description: director: Removed unnecessary error messages. diffstat: src/director/director-connection.c | 4 ---- 1 files changed, 0 insertions(+), 4 deletions(-) diffs (21 lines): diff -r ce4e1bf7262d -r 0c3c4d098842 src/director/director-connection.c --- a/src/director/director-connection.c Fri Apr 20 00:49:29 2012 +0300 +++ b/src/director/director-connection.c Fri Apr 20 00:59:01 2012 +0300 @@ -1135,8 +1135,6 @@ if (ret < 0) { /* invalid commands during handshake, we probably don't want to reconnect here */ - i_error("director(%s): Handshaking failed", - conn->name); return FALSE; } /* allow also other commands during handshake */ @@ -1228,8 +1226,6 @@ } T_END; if (!ret) { - i_error("director(%s): Invalid input, disconnecting", - conn->name); director_connection_reconnect(&conn); break; } From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: USER sending timeout was higher than it n... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/10f5456704a7 changeset: 14483:10f5456704a7 user: Timo Sirainen date: Fri Apr 20 01:14:41 2012 +0300 description: director: USER sending timeout was higher than it needed to be. The problem was USER sending getting stuck, not the timeout itself.. diffstat: src/director/director-connection.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 0c3c4d098842 -r 10f5456704a7 src/director/director-connection.c --- a/src/director/director-connection.c Fri Apr 20 00:59:01 2012 +0300 +++ b/src/director/director-connection.c Fri Apr 20 01:14:41 2012 +0300 @@ -55,7 +55,7 @@ /* Max time to wait for USERs in handshake to be sent. With a lot of users the kernel may quickly eat up everything we send, while the receiver is busy parsing the data. */ -#define DIRECTOR_CONNECTION_SEND_USERS_TIMEOUT_MSECS (120*1000) +#define DIRECTOR_CONNECTION_SEND_USERS_TIMEOUT_MSECS (30*1000) /* Max idling time before "DONE" command must have been received, or we'll disconnect. */ #define DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS (30*1000) From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: Adding 25c941423e42 patch yet again, now ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/25ca6b6d754a changeset: 14484:25ca6b6d754a user: Timo Sirainen date: Fri Apr 20 01:32:01 2012 +0300 description: director: Adding 25c941423e42 patch yet again, now it seems to work right. And also fixes detection of outgoing connections that die during handshake sending. diffstat: src/director/director-connection.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff -r 10f5456704a7 -r 25ca6b6d754a src/director/director-connection.c --- a/src/director/director-connection.c Fri Apr 20 01:14:41 2012 +0300 +++ b/src/director/director-connection.c Fri Apr 20 01:32:01 2012 +0300 @@ -1374,6 +1374,8 @@ director_connection_output, conn); io_remove(&conn->io); + conn->io = io_add(conn->fd, IO_READ, director_connection_input, conn); + timeout_remove(&conn->to_ping); conn->to_ping = timeout_add(DIRECTOR_CONNECTION_SEND_USERS_TIMEOUT_MSECS, director_connection_init_timeout, conn); From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: And crashfix for previous change.. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/5d169948b59d changeset: 14485:5d169948b59d user: Timo Sirainen date: Fri Apr 20 01:35:06 2012 +0300 description: director: And crashfix for previous change.. diffstat: src/director/director-connection.c | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) diffs (13 lines): diff -r 25ca6b6d754a -r 5d169948b59d src/director/director-connection.c --- a/src/director/director-connection.c Fri Apr 20 01:32:01 2012 +0300 +++ b/src/director/director-connection.c Fri Apr 20 01:35:06 2012 +0300 @@ -1289,9 +1289,6 @@ user_directory_iter_deinit(&conn->user_iter); director_connection_send(conn, "DONE\n"); - i_assert(conn->io == NULL); - conn->io = io_add(conn->fd, IO_READ, director_connection_input, conn); - ret = o_stream_flush(conn->output); timeout_reset(conn->to_ping); return ret; From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: Minor error message fix Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/58286c1e272d changeset: 14486:58286c1e272d user: Timo Sirainen date: Fri Apr 20 11:09:36 2012 +0300 description: director: Minor error message fix diffstat: src/director/director-connection.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 5d169948b59d -r 58286c1e272d src/director/director-connection.c --- a/src/director/director-connection.c Fri Apr 20 01:35:06 2012 +0300 +++ b/src/director/director-connection.c Fri Apr 20 11:09:36 2012 +0300 @@ -455,7 +455,7 @@ } if (user->to_move != NULL) str_append(str, ",moving"); - if (user->kill_state == USER_KILL_STATE_NONE) + if (user->kill_state != USER_KILL_STATE_NONE) str_printfa(str, ",kill_state=%d", user->kill_state); str_append_c(str, ')'); i_error("%s", str_c(str)); From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: Removed special (time_t)-1 code, it's not... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/45be27ff3414 changeset: 14487:45be27ff3414 user: Timo Sirainen date: Fri Apr 20 11:22:26 2012 +0300 description: director: Removed special (time_t)-1 code, it's not actually needed. Also this fixes a user refreshing issue. diffstat: src/director/director-connection.c | 4 ++-- src/director/director-request.c | 2 +- src/director/director.c | 2 +- src/director/test-user-directory.c | 2 +- src/director/user-directory.c | 9 ++------- src/director/user-directory.h | 3 +-- 6 files changed, 8 insertions(+), 14 deletions(-) diffs (90 lines): diff -r 58286c1e272d -r 45be27ff3414 src/director/director-connection.c --- a/src/director/director-connection.c Fri Apr 20 11:09:36 2012 +0300 +++ b/src/director/director-connection.c Fri Apr 20 11:22:26 2012 +0300 @@ -549,7 +549,7 @@ } if (director_user_refresh(conn, username_hash, - host, (time_t)-1, FALSE, &user)) { + host, ioloop_time, FALSE, &user)) { i_assert(!user->weak); director_update_user(conn->dir, conn->host, user); } @@ -683,7 +683,7 @@ } if (director_user_refresh(conn, username_hash, - host, (time_t)-1, weak, &user)) { + host, ioloop_time, weak, &user)) { if (!user->weak) director_update_user(conn->dir, src_host, user); else { diff -r 58286c1e272d -r 45be27ff3414 src/director/director-request.c --- a/src/director/director-request.c Fri Apr 20 11:09:36 2012 +0300 +++ b/src/director/director-request.c Fri Apr 20 11:22:26 2012 +0300 @@ -225,7 +225,7 @@ return FALSE; } user = user_directory_add(dir->users, request->username_hash, - host, (time_t)-1); + host, ioloop_time); } i_assert(!user->weak); diff -r 58286c1e272d -r 45be27ff3414 src/director/director.c --- a/src/director/director.c Fri Apr 20 11:09:36 2012 +0300 +++ b/src/director/director.c Fri Apr 20 11:22:26 2012 +0300 @@ -582,7 +582,7 @@ user = user_directory_lookup(dir->users, username_hash); if (user == NULL) { user = user_directory_add(dir->users, username_hash, - host, (time_t)-1); + host, ioloop_time); } else { if (user->host == host) { /* user is already in this host */ diff -r 58286c1e272d -r 45be27ff3414 src/director/test-user-directory.c --- a/src/director/test-user-directory.c Fri Apr 20 11:09:36 2012 +0300 +++ b/src/director/test-user-directory.c Fri Apr 20 11:22:26 2012 +0300 @@ -81,7 +81,7 @@ dir = user_directory_init(USER_DIR_TIMEOUT, "%u"); for (i = 0; i < count; i++) { if (rand() % 10 == 0) - timestamp = (time_t)-1; + timestamp = ioloop_time; else timestamp = ioloop_time-rand()%100; user_directory_add(dir, i+1, host, timestamp); diff -r 58286c1e272d -r 45be27ff3414 src/director/user-directory.c --- a/src/director/user-directory.c Fri Apr 20 11:09:36 2012 +0300 +++ b/src/director/user-directory.c Fri Apr 20 11:22:26 2012 +0300 @@ -131,14 +131,9 @@ { struct user *user; - if (timestamp == (time_t)-1) { - /* add it at the end */ + /* make sure we don't add timestamps higher than ioloop time */ + if (timestamp > ioloop_time) timestamp = ioloop_time; - } else { - /* make sure we don't add timestamps higher than ioloop time */ - if (timestamp > ioloop_time) - timestamp = ioloop_time; - } user = i_new(struct user, 1); user->username_hash = username_hash; diff -r 58286c1e272d -r 45be27ff3414 src/director/user-directory.h --- a/src/director/user-directory.h Fri Apr 20 11:09:36 2012 +0300 +++ b/src/director/user-directory.h Fri Apr 20 11:22:26 2012 +0300 @@ -54,8 +54,7 @@ /* Look up username from directory. Returns NULL if not found. */ struct user *user_directory_lookup(struct user_directory *dir, unsigned int username_hash); -/* Add a user to directory and return it. If timestamp is (time_t)-1, - the current time is used. */ +/* Add a user to directory and return it. */ struct user * user_directory_add(struct user_directory *dir, unsigned int username_hash, struct mail_host *host, time_t timestamp); From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: layout=fs: Fixed listing mailboxes with prefix=INBOX/ Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/8d0b71fb3e49 changeset: 14490:8d0b71fb3e49 user: Timo Sirainen date: Fri Apr 20 15:26:43 2012 +0300 description: layout=fs: Fixed listing mailboxes with prefix=INBOX/ diffstat: src/lib-storage/list/mailbox-list-fs-iter.c | 18 ++++++++++++++---- 1 files changed, 14 insertions(+), 4 deletions(-) diffs (41 lines): diff -r a41f64348d0d -r 8d0b71fb3e49 src/lib-storage/list/mailbox-list-fs-iter.c --- a/src/lib-storage/list/mailbox-list-fs-iter.c Fri Apr 20 15:18:14 2012 +0300 +++ b/src/lib-storage/list/mailbox-list-fs-iter.c Fri Apr 20 15:26:43 2012 +0300 @@ -371,11 +371,12 @@ static void fs_list_get_roots(struct fs_list_iterate_context *ctx) { + struct mail_namespace *ns = ctx->ctx.list->ns; + char ns_sep = mail_namespace_get_sep(ns); bool full_fs_access = ctx->ctx.list->mail_set->mail_full_filesystem_access; - char ns_sep = mail_namespace_get_sep(ctx->ctx.list->ns); const char *const *patterns, *pattern, *const *parentp, *const *childp; - const char *p, *last, *root; + const char *p, *last, *root, *prefix_vname; unsigned int i, parentlen; i_assert(*ctx->valid_patterns != NULL); @@ -391,11 +392,20 @@ if (*p == ns_sep) last = p; } + prefix_vname = t_strdup_until(pattern, last); + if (p == last+1 && *pattern == ns_sep) root = "/"; - else { + else if ((ns->flags & NAMESPACE_FLAG_INBOX_USER) != 0 && + strcasecmp(prefix_vname, "INBOX") == 0 && + strncasecmp(ns->prefix, pattern, ns->prefix_len) == 0) { + /* special case: Namespace prefix is INBOX/ and + we just want to see its contents (not the + INBOX's children). */ + root = ""; + } else { root = mailbox_list_get_storage_name(ctx->ctx.list, - t_strdup_until(pattern, last)); + prefix_vname); } if (*root == '/') { From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: layout=fs: Don't assume '/' hierarchy separator whe... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/a41f64348d0d changeset: 14489:a41f64348d0d user: Timo Sirainen date: Fri Apr 20 15:18:14 2012 +0300 description: layout=fs: Don't assume '/' hierarchy separator when finding mailbox roots. diffstat: src/lib-storage/list/mailbox-list-fs-iter.c | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diffs (24 lines): diff -r 798b5e1609e5 -r a41f64348d0d src/lib-storage/list/mailbox-list-fs-iter.c --- a/src/lib-storage/list/mailbox-list-fs-iter.c Fri Apr 20 15:04:58 2012 +0300 +++ b/src/lib-storage/list/mailbox-list-fs-iter.c Fri Apr 20 15:18:14 2012 +0300 @@ -373,6 +373,7 @@ { bool full_fs_access = ctx->ctx.list->mail_set->mail_full_filesystem_access; + char ns_sep = mail_namespace_get_sep(ctx->ctx.list->ns); const char *const *patterns, *pattern, *const *parentp, *const *childp; const char *p, *last, *root; unsigned int i, parentlen; @@ -387,10 +388,10 @@ for (p = last = pattern; *p != '\0'; p++) { if (*p == '%' || *p == '*') break; - if (*p == '/') + if (*p == ns_sep) last = p; } - if (p == last+1 && *pattern == '/') + if (p == last+1 && *pattern == ns_sep) root = "/"; else { root = mailbox_list_get_storage_name(ctx->ctx.list, From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: When another director reconnects, reset i... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/798b5e1609e5 changeset: 14488:798b5e1609e5 user: Timo Sirainen date: Fri Apr 20 15:04:58 2012 +0300 description: director: When another director reconnects, reset its "last received seq". This fixes some problems where HOST updates and others were thought as already being received after the originating director was restarted. diffstat: src/director/director-connection.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff -r 45be27ff3414 -r 798b5e1609e5 src/director/director-connection.c --- a/src/director/director-connection.c Fri Apr 20 11:22:26 2012 +0300 +++ b/src/director/director-connection.c Fri Apr 20 15:04:58 2012 +0300 @@ -571,6 +571,8 @@ /* already have this. just reset its last_network_failure timestamp, since it might be up now. */ host->last_network_failure = 0; + /* it also may have been restarted, so reset last_seq */ + host->last_seq = 0; return TRUE; } From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: Director ring needs to be set unsynced im... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/a5b7dda1db6d changeset: 14491:a5b7dda1db6d user: Timo Sirainen date: Fri Apr 20 19:03:26 2012 +0300 description: director: Director ring needs to be set unsynced immediately even when sycning is frozen. diffstat: src/director/director.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diffs (25 lines): diff -r 8d0b71fb3e49 -r a5b7dda1db6d src/director/director.c --- a/src/director/director.c Fri Apr 20 15:26:43 2012 +0300 +++ b/src/director/director.c Fri Apr 20 19:03:26 2012 +0300 @@ -323,6 +323,10 @@ static void director_sync(struct director *dir) { + /* we're synced again when we receive this SYNC back */ + dir->sync_seq++; + director_set_ring_unsynced(dir); + if (dir->sync_frozen) { dir->sync_pending = TRUE; return; @@ -333,10 +337,6 @@ return; } - /* we're synced again when we receive this SYNC back */ - dir->sync_seq++; - director_set_ring_unsynced(dir); - if (dir->debug) { i_debug("Ring is desynced (seq=%u, sending SYNC to %s)", dir->sync_seq, dir->right == NULL ? "(nowhere)" : From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: Try harder to connect to ring before thin... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/f880cf0efa0a changeset: 14492:f880cf0efa0a user: Timo Sirainen date: Fri Apr 20 19:05:38 2012 +0300 description: director: Try harder to connect to ring before thinking we're alone. diffstat: src/director/director.c | 80 +++++++++++++++++++++++++++++++++--------------- src/director/director.h | 2 + 2 files changed, 56 insertions(+), 26 deletions(-) diffs (127 lines): diff -r a5b7dda1db6d -r f880cf0efa0a src/director/director.c --- a/src/director/director.c Fri Apr 20 19:03:26 2012 +0300 +++ b/src/director/director.c Fri Apr 20 19:05:38 2012 +0300 @@ -18,6 +18,8 @@ #define DIRECTOR_USER_MOVE_TIMEOUT_MSECS (30*1000) #define DIRECTOR_USER_MOVE_FINISH_DELAY_MSECS (2*1000) #define DIRECTOR_SYNC_TIMEOUT_MSECS (5*1000) +#define DIRECTOR_RING_MIN_WAIT_SECS 20 +#define DIRECTOR_QUICK_RECONNECT_TIMEOUT_MSECS 1000 static bool director_is_self_ip_set(struct director *dir) { @@ -138,6 +140,30 @@ return hosts[(self_idx + 1) % count]; } +static bool director_wait_for_others(struct director *dir) +{ + struct director_host *const *hostp; + + /* don't assume we're alone until we've attempted to connect + to others for a while */ + if (dir->ring_first_alone != 0 && + ioloop_time - dir->ring_first_alone > DIRECTOR_RING_MIN_WAIT_SECS) + return FALSE; + + if (dir->ring_first_alone == 0) + dir->ring_first_alone = ioloop_time; + /* reset all failures and try again */ + array_foreach(&dir->dir_hosts, hostp) { + (*hostp)->last_network_failure = 0; + (*hostp)->last_protocol_failure = 0; + } + if (dir->to_reconnect != NULL) + timeout_remove(&dir->to_reconnect); + dir->to_reconnect = timeout_add(DIRECTOR_QUICK_RECONNECT_TIMEOUT_MSECS, + director_connect, dir); + return TRUE; +} + void director_connect(struct director *dir) { struct director_host *const *hosts; @@ -163,26 +189,30 @@ continue; } - if (director_connect_host(dir, hosts[idx]) == 0) - break; + if (director_connect_host(dir, hosts[idx]) == 0) { + /* success */ + return; + } } - if (i == count) { - /* we're the only one */ - if (dir->debug) { - i_debug("director: Couldn't connect to right side, " - "we must be the only director left"); - } - if (dir->left != NULL) { - /* since we couldn't connect to it, - it must have failed recently */ - director_connection_deinit(&dir->left); - } - dir->ring_min_version = DIRECTOR_VERSION_MINOR; - if (!dir->ring_handshaked) - director_set_ring_handshaked(dir); - else - director_set_ring_synced(dir); + + if (count > 1 && director_wait_for_others(dir)) + return; + + /* we're the only one */ + if (count > 1) { + i_warning("director: Couldn't connect to right side, " + "we must be the only director left"); } + if (dir->left != NULL) { + /* since we couldn't connect to it, + it must have failed recently */ + director_connection_deinit(&dir->left); + } + dir->ring_min_version = DIRECTOR_VERSION_MINOR; + if (!dir->ring_handshaked) + director_set_ring_handshaked(dir); + else + director_set_ring_synced(dir); } void director_set_ring_handshaked(struct director *dir) @@ -238,16 +268,14 @@ host = dir->right == NULL ? NULL : director_connection_get_host(dir->right); + + if (dir->to_reconnect != NULL) + timeout_remove(&dir->to_reconnect); if (host != director_get_preferred_right_host(dir)) { /* try to reconnect to preferred host later */ - if (dir->to_reconnect == NULL) { - dir->to_reconnect = - timeout_add(DIRECTOR_RECONNECT_TIMEOUT_MSECS, - director_reconnect_timeout, dir); - } - } else { - if (dir->to_reconnect != NULL) - timeout_remove(&dir->to_reconnect); + dir->to_reconnect = + timeout_add(DIRECTOR_RECONNECT_TIMEOUT_MSECS, + director_reconnect_timeout, dir); } if (dir->left != NULL) diff -r a5b7dda1db6d -r f880cf0efa0a src/director/director.h --- a/src/director/director.h Fri Apr 20 19:03:26 2012 +0300 +++ b/src/director/director.h Fri Apr 20 19:05:38 2012 +0300 @@ -65,6 +65,8 @@ unsigned int ring_min_version; time_t ring_last_sync_time; + time_t ring_first_alone; + /* director ring handshaking is complete. director can start serving clients. */ unsigned int ring_handshaked:1; From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: Forward DIRECTOR commands to other connec... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/1856e9085f94 changeset: 14493:1856e9085f94 user: Timo Sirainen date: Fri Apr 20 19:08:25 2012 +0300 description: director: Forward DIRECTOR commands to other connected directors. They can use it to realize that the director was restarted and its sequences start from 1. diffstat: src/director/director-connection.c | 29 +++++++++++++++++++++-------- 1 files changed, 21 insertions(+), 8 deletions(-) diffs (49 lines): diff -r f880cf0efa0a -r 1856e9085f94 src/director/director-connection.c --- a/src/director/director-connection.c Fri Apr 20 19:05:38 2012 +0300 +++ b/src/director/director-connection.c Fri Apr 20 19:08:25 2012 +0300 @@ -562,24 +562,37 @@ struct director_host *host; struct ip_addr ip; unsigned int port; + bool forward = FALSE; if (!director_args_parse_ip_port(conn, args, &ip, &port)) return FALSE; host = director_host_lookup(conn->dir, &ip, port); if (host != NULL) { + if (host == conn->dir->self_host) { + /* ignore updates to ourself */ + return TRUE; + } + /* already have this. just reset its last_network_failure timestamp, since it might be up now. */ host->last_network_failure = 0; - /* it also may have been restarted, so reset last_seq */ - host->last_seq = 0; - return TRUE; + if (host->last_seq != 0) { + /* it also may have been restarted, reset last_seq */ + host->last_seq = 0; + forward = TRUE; + } + } else { + /* save the director and forward it */ + director_host_add(conn->dir, &ip, port); + forward = TRUE; } - - /* save the director and forward it */ - director_host_add(conn->dir, &ip, port); - director_update_send(conn->dir, director_connection_get_host(conn), - t_strdup_printf("DIRECTOR\t%s\t%u\n", net_ip2addr(&ip), port)); + if (forward) { + director_update_send(conn->dir, + director_connection_get_host(conn), + t_strdup_printf("DIRECTOR\t%s\t%u\n", + net_ip2addr(&ip), port)); + } return TRUE; } From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: director: Delay disconnecting director after sendin... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d27b743c9921 changeset: 14494:d27b743c9921 user: Timo Sirainen date: Fri Apr 20 19:09:55 2012 +0300 description: director: Delay disconnecting director after sending CONNECT command. The director may not otherwise read the CONNECT. diffstat: src/director/director-connection.c | 47 +++++++++++++++++++++++++------------ 1 files changed, 32 insertions(+), 15 deletions(-) diffs (143 lines): diff -r 1856e9085f94 -r d27b743c9921 src/director/director-connection.c --- a/src/director/director-connection.c Fri Apr 20 19:08:25 2012 +0300 +++ b/src/director/director-connection.c Fri Apr 20 19:09:55 2012 +0300 @@ -70,6 +70,7 @@ /* If outgoing director connection exists for less than this many seconds, mark the host as failed so we won't try to reconnect to it immediately */ #define DIRECTOR_SUCCESS_MIN_CONNECT_SECS 40 +#define DIRECTOR_WAIT_DISCONNECT_MSECS 10 #if DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS <= DIRECTOR_CONNECTION_PING_TIMEOUT_MSECS # error DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS is too low @@ -96,7 +97,7 @@ struct io *io; struct istream *input; struct ostream *output; - struct timeout *to, *to_ping, *to_pong; + struct timeout *to_disconnect, *to_ping, *to_pong; struct user_directory_iter *user_iter; @@ -166,16 +167,28 @@ conn->to_ping = timeout_add(msecs, director_connection_ping, conn); } +static void director_connection_wait_timeout(struct director_connection *conn) +{ + director_connection_deinit(&conn); +} + static void director_connection_send_connect(struct director_connection *conn, struct director_host *host) { const char *connect_str; + if (conn->to_disconnect != NULL) + return; + connect_str = t_strdup_printf("CONNECT\t%s\t%u\n", net_ip2addr(&host->ip), host->port); director_connection_send(conn, connect_str); (void)o_stream_flush(conn->output); o_stream_uncork(conn->output); + + conn->to_disconnect = + timeout_add(DIRECTOR_WAIT_DISCONNECT_MSECS, + director_connection_wait_timeout, conn); } static void director_connection_assigned(struct director_connection *conn) @@ -226,7 +239,7 @@ "us, should use %s instead", conn->name, dir->left->host->name); director_connection_send_connect(conn, dir->left->host); - return FALSE; + return TRUE; } else { /* this new connection is the correct one, but wait until the old connection gets disconnected before using this one. @@ -249,7 +262,8 @@ array_foreach(&dir->connections, connp) { conn = *connp; - if (conn->in && conn->handshake_received && conn != dir->left) { + if (conn->in && conn->handshake_received && + conn->to_disconnect == NULL && conn != dir->left) { /* either use this or disconnect it */ if (!director_connection_assign_left(conn)) { /* we don't want this */ @@ -266,7 +280,7 @@ struct director_connection *const *connp; array_foreach(&dir->connections, connp) { - if (!(*connp)->in) + if (!(*connp)->in && (*connp)->to_disconnect == NULL) return TRUE; } return FALSE; @@ -1104,15 +1118,13 @@ conn = *connp; if (conn->in && conn != dir->left && conn->me_received && + conn->to_disconnect == NULL && director_host_cmp_to_self(dir->left->host, conn->host, dir->self_host) < 0) { i_warning("Director connection %s tried to connect to " "us, should use %s instead", conn->name, dir->left->host->name); director_connection_send_connect(conn, dir->left->host); - director_connection_deinit(&conn); - director_disconnect_wrong_lefts(dir); - return; } } } @@ -1234,6 +1246,17 @@ return; } + if (conn->to_disconnect != NULL) { + /* just read everything the remote sends, and wait for it + to disconnect. we mainly just want the remote to read the + CONNECT we sent it. */ + size_t size; + + (void)i_stream_get_data(conn->input, &size); + i_stream_skip(conn->input, size); + return; + } + director_sync_freeze(dir); while ((line = i_stream_next_line(conn->input)) != NULL) { T_BEGIN { @@ -1451,8 +1474,8 @@ if (conn->user_iter != NULL) user_directory_iter_deinit(&conn->user_iter); - if (conn->to != NULL) - timeout_remove(&conn->to); + if (conn->to_disconnect != NULL) + timeout_remove(&conn->to_disconnect); if (conn->to_pong != NULL) timeout_remove(&conn->to_pong); timeout_remove(&conn->to_ping); @@ -1501,11 +1524,6 @@ director_connect(dir); } -static void director_connection_timeout(struct director_connection *conn) -{ - director_connection_disconnected(&conn); -} - void director_connection_send(struct director_connection *conn, const char *data) { @@ -1524,7 +1542,6 @@ "disconnecting", conn->name); } o_stream_close(conn->output); - conn->to = timeout_add(0, director_connection_timeout, conn); } } From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: imapc: Added imapc_list_prefix setting to limit wha... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/03f44d61d582 changeset: 14495:03f44d61d582 user: Timo Sirainen date: Fri Apr 20 22:27:01 2012 +0300 description: imapc: Added imapc_list_prefix setting to limit what mailboxes are accessed. diffstat: src/lib-storage/index/imapc/imapc-list.c | 49 +++++++++++++++++++++++++-- src/lib-storage/index/imapc/imapc-settings.c | 2 + src/lib-storage/index/imapc/imapc-settings.h | 1 + 3 files changed, 47 insertions(+), 5 deletions(-) diffs (124 lines): diff -r d27b743c9921 -r 03f44d61d582 src/lib-storage/index/imapc/imapc-list.c --- a/src/lib-storage/index/imapc/imapc-list.c Fri Apr 20 19:09:55 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-list.c Fri Apr 20 22:27:01 2012 +0300 @@ -92,7 +92,7 @@ T_BEGIN { const char *vname = - mailbox_list_default_get_vname(&list->list, name); + mailbox_list_get_vname(&list->list, name); if ((info_flags & MAILBOX_NONEXISTENT) != 0) node = mailbox_tree_lookup(tree, vname); @@ -191,6 +191,37 @@ return list->sep; } +static const char * +imapc_list_get_storage_name(struct mailbox_list *_list, const char *vname) +{ + struct imapc_mailbox_list *list = (struct imapc_mailbox_list *)_list; + const char *prefix = list->storage->set->imapc_list_prefix; + const char *storage_name; + + storage_name = mailbox_list_default_get_storage_name(_list, vname); + if (*prefix != '\0') { + storage_name = t_strdup_printf("%s%c%s", prefix, list->sep, + storage_name); + } + return storage_name; +} + +static const char * +imapc_list_get_vname(struct mailbox_list *_list, const char *storage_name) +{ + struct imapc_mailbox_list *list = (struct imapc_mailbox_list *)_list; + const char *prefix = list->storage->set->imapc_list_prefix; + unsigned int prefix_len; + + if (*prefix != '\0') { + prefix_len = strlen(prefix); + i_assert(strncmp(prefix, storage_name, prefix_len) == 0 && + storage_name[prefix_len] == list->sep); + storage_name += prefix_len+1; + } + return mailbox_list_default_get_vname(_list, storage_name); +} + static struct mailbox_list *imapc_list_get_fs(struct imapc_mailbox_list *list) { struct mailbox_list_settings list_set; @@ -313,14 +344,22 @@ { struct imapc_command *cmd; struct imapc_simple_context ctx; + const char *pattern; i_assert(list->sep != '\0'); if (list->refreshed_mailboxes) return 0; + if (*list->storage->set->imapc_list_prefix == '\0') + pattern = "*"; + else { + pattern = t_strdup_printf("%s%c*", + list->storage->set->imapc_list_prefix, list->sep); + } + cmd = imapc_list_simple_context_init(&ctx, list); - imapc_command_send(cmd, "LIST \"\" *"); + imapc_command_sendf(cmd, "LIST \"\" %s", pattern); mailbox_tree_deinit(&list->mailboxes); list->mailboxes = mailbox_tree_init(list->sep); @@ -611,7 +650,7 @@ i_assert(list->sep != '\0'); - vname = mailbox_list_default_get_vname(_list, name); + vname = mailbox_list_get_vname(_list, name); if (!list->refreshed_mailboxes) { node = mailbox_tree_lookup(list->mailboxes, vname); if (node != NULL) @@ -646,8 +685,8 @@ imapc_is_valid_existing_name, imapc_is_valid_create_name, imapc_list_get_hierarchy_sep, - mailbox_list_default_get_vname, - mailbox_list_default_get_storage_name, + imapc_list_get_vname, + imapc_list_get_storage_name, imapc_list_get_path, imapc_list_get_temp_prefix, imapc_list_join_refpattern, diff -r d27b743c9921 -r 03f44d61d582 src/lib-storage/index/imapc/imapc-settings.c --- a/src/lib-storage/index/imapc/imapc-settings.c Fri Apr 20 19:09:55 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-settings.c Fri Apr 20 22:27:01 2012 +0300 @@ -27,6 +27,7 @@ DEF(SET_STR, imapc_features), DEF(SET_STR, imapc_rawlog_dir), + DEF(SET_STR, imapc_list_prefix), DEF(SET_STR, ssl_crypto_device), SETTING_DEFINE_LIST_END @@ -46,6 +47,7 @@ .imapc_features = "", .imapc_rawlog_dir = "", + .imapc_list_prefix = "", .ssl_crypto_device = "" }; diff -r d27b743c9921 -r 03f44d61d582 src/lib-storage/index/imapc/imapc-settings.h --- a/src/lib-storage/index/imapc/imapc-settings.h Fri Apr 20 19:09:55 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-settings.h Fri Apr 20 22:27:01 2012 +0300 @@ -22,6 +22,7 @@ const char *imapc_features; const char *imapc_rawlog_dir; + const char *imapc_list_prefix; const char *ssl_crypto_device; enum imapc_features parsed_features; From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: imapc: Nonexistent mailboxes were returned as selec... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/4cea936d57f2 changeset: 14496:4cea936d57f2 user: Timo Sirainen date: Mon Apr 23 12:52:59 2012 +0300 description: imapc: Nonexistent mailboxes were returned as selectable mailboxes. diffstat: src/lib-storage/index/imapc/imapc-list.c | 3 +++ src/lib-storage/mailbox-tree.c | 30 ++++++++++++++++++++++-------- src/lib-storage/mailbox-tree.h | 3 ++- 3 files changed, 27 insertions(+), 9 deletions(-) diffs (132 lines): diff -r 03f44d61d582 -r 4cea936d57f2 src/lib-storage/index/imapc/imapc-list.c --- a/src/lib-storage/index/imapc/imapc-list.c Fri Apr 20 22:27:01 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-list.c Mon Apr 23 12:52:59 2012 +0300 @@ -33,6 +33,7 @@ list->list.pool = pool; /* separator is set when storage is created */ list->mailboxes = mailbox_tree_init('\0'); + mailbox_tree_set_parents_nonexistent(list->mailboxes); return &list->list; } @@ -362,6 +363,7 @@ imapc_command_sendf(cmd, "LIST \"\" %s", pattern); mailbox_tree_deinit(&list->mailboxes); list->mailboxes = mailbox_tree_init(list->sep); + mailbox_tree_set_parents_nonexistent(list->mailboxes); imapc_simple_run(&ctx); if (ctx.ret == 0) { @@ -436,6 +438,7 @@ ctx->info.ns = _list->ns; ctx->tree = mailbox_tree_init(sep); + mailbox_tree_set_parents_nonexistent(ctx->tree); imapc_list_build_match_tree(ctx); if (list->list.ns->prefix_len > 0) { diff -r 03f44d61d582 -r 4cea936d57f2 src/lib-storage/mailbox-tree.c --- a/src/lib-storage/mailbox-tree.c Fri Apr 20 22:27:01 2012 +0300 +++ b/src/lib-storage/mailbox-tree.c Mon Apr 23 12:52:59 2012 +0300 @@ -8,6 +8,8 @@ struct mailbox_tree_context { pool_t pool; char separator; + bool parents_nonexistent; + struct mailbox_node *nodes; }; @@ -49,6 +51,11 @@ tree->separator = separator; } +void mailbox_tree_set_parents_nonexistent(struct mailbox_tree_context *tree) +{ + tree->parents_nonexistent = TRUE; +} + void mailbox_tree_clear(struct mailbox_tree_context *tree) { p_clear(tree->pool); @@ -57,14 +64,13 @@ static struct mailbox_node * mailbox_tree_traverse(struct mailbox_tree_context *tree, const char *path, - bool create, bool *created) + bool create, bool *created_r) { struct mailbox_node **node, *parent; const char *name; string_t *str; - if (created != NULL) - *created = FALSE; + *created_r = FALSE; if (path == NULL) return tree->nodes; @@ -101,9 +107,10 @@ *node = p_new(tree->pool, struct mailbox_node, 1); (*node)->parent = parent; (*node)->name = p_strdup(tree->pool, name); + if (tree->parents_nonexistent) + (*node)->flags = MAILBOX_NONEXISTENT; - if (created != NULL) - *created = TRUE; + *created_r = TRUE; } if (*path == '\0') @@ -120,13 +127,19 @@ struct mailbox_node * mailbox_tree_get(struct mailbox_tree_context *tree, const char *path, - bool *created) + bool *created_r) { struct mailbox_node *node; + bool created; T_BEGIN { - node = mailbox_tree_traverse(tree, path, TRUE, created); + node = mailbox_tree_traverse(tree, path, TRUE, &created); } T_END; + if (created && tree->parents_nonexistent) + node->flags = 0; + + if (created_r != NULL) + *created_r = created; return node; } @@ -134,9 +147,10 @@ mailbox_tree_lookup(struct mailbox_tree_context *tree, const char *path) { struct mailbox_node *node; + bool created; T_BEGIN { - node = mailbox_tree_traverse(tree, path, FALSE, NULL); + node = mailbox_tree_traverse(tree, path, FALSE, &created); } T_END; return node; } diff -r 03f44d61d582 -r 4cea936d57f2 src/lib-storage/mailbox-tree.h --- a/src/lib-storage/mailbox-tree.h Fri Apr 20 22:27:01 2012 +0300 +++ b/src/lib-storage/mailbox-tree.h Mon Apr 23 12:52:59 2012 +0300 @@ -17,11 +17,12 @@ void mailbox_tree_set_separator(struct mailbox_tree_context *tree, char separator); +void mailbox_tree_set_parents_nonexistent(struct mailbox_tree_context *tree); void mailbox_tree_clear(struct mailbox_tree_context *tree); struct mailbox_node * mailbox_tree_get(struct mailbox_tree_context *tree, const char *path, - bool *created); + bool *created_r); struct mailbox_node * mailbox_tree_lookup(struct mailbox_tree_context *tree, const char *path); From dovecot at dovecot.org Sun May 20 03:26:33 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:33 +0300 Subject: dovecot-2.2: imapc: Don't list mailboxes that don't match list p... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/f24cdf8b0624 changeset: 14497:f24cdf8b0624 user: Timo Sirainen date: Mon Apr 23 12:58:42 2012 +0300 description: imapc: Don't list mailboxes that don't match list patterns. diffstat: src/lib-storage/index/imapc/imapc-list.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 4cea936d57f2 -r f24cdf8b0624 src/lib-storage/index/imapc/imapc-list.c --- a/src/lib-storage/index/imapc/imapc-list.c Mon Apr 23 12:52:59 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-list.c Mon Apr 23 12:58:42 2012 +0300 @@ -471,7 +471,7 @@ node = mailbox_tree_iterate_next(ctx->iter, &name); if (node == NULL) return NULL; - } while (node == ctx->ns_root); + } while (node == ctx->ns_root || (node->flags & MAILBOX_MATCHED) == 0); ctx->info.name = name; ctx->info.flags = node->flags; From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: quota: Recalculation now also counts the namespace ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/4c8f79d1f9f1 changeset: 14498:4c8f79d1f9f1 user: Timo Sirainen date: Mon Apr 23 13:19:44 2012 +0300 description: quota: Recalculation now also counts the namespace prefix mailbox's quota if it exists. diffstat: src/plugins/quota/quota-count.c | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diffs (17 lines): diff -r f24cdf8b0624 -r 4c8f79d1f9f1 src/plugins/quota/quota-count.c --- a/src/plugins/quota/quota-count.c Mon Apr 23 12:58:42 2012 +0300 +++ b/src/plugins/quota/quota-count.c Mon Apr 23 13:19:44 2012 +0300 @@ -83,7 +83,12 @@ } if (mailbox_list_iter_deinit(&ctx) < 0) ret = -1; - + if (ns->prefix_len > 0 && ret == 0 && + (ns->prefix_len != 6 || strncasecmp(ns->prefix, "INBOX", 5) != 0)) { + /* if the namespace prefix itself exists, count it also */ + const char *name = t_strndup(ns->prefix, ns->prefix_len-1); + ret = quota_count_mailbox(root, ns, name, bytes, count); + } return ret; } From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: listescape: Don't unescape namespace prefix. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/63af3274fb6f changeset: 14499:63af3274fb6f user: Timo Sirainen date: Mon Apr 23 14:11:37 2012 +0300 description: listescape: Don't unescape namespace prefix. diffstat: src/lib-storage/mailbox-list.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diffs (15 lines): diff -r 4c8f79d1f9f1 -r 63af3274fb6f src/lib-storage/mailbox-list.c --- a/src/lib-storage/mailbox-list.c Mon Apr 23 13:19:44 2012 +0300 +++ b/src/lib-storage/mailbox-list.c Mon Apr 23 14:11:37 2012 +0300 @@ -458,6 +458,11 @@ string_t *dest = t_str_new(strlen(src)); unsigned int num; + if (strncmp(src, list->ns->prefix, list->ns->prefix_len) == 0) { + str_append_n(dest, src, list->ns->prefix_len); + src += list->ns->prefix_len; + } + for (; *src != '\0'; src++) { if (*src == list->set.escape_char && i_isxdigit(src[1]) && i_isxdigit(src[2])) { From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: imap: While modseq tracking hasn't been enabled, re... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/1222b7e38bf3 changeset: 14500:1222b7e38bf3 user: Timo Sirainen date: Mon Apr 23 14:47:13 2012 +0300 description: imap: While modseq tracking hasn't been enabled, return mailbox as NOMODSEQ. It would be possible to avoid this by always keeping track of highestmodseq, but that requires changing the index format a bit. diffstat: src/lib-storage/index/index-status.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diffs (14 lines): diff -r 63af3274fb6f -r 1222b7e38bf3 src/lib-storage/index/index-status.c --- a/src/lib-storage/index/index-status.c Mon Apr 23 14:11:37 2012 +0300 +++ b/src/lib-storage/index/index-status.c Mon Apr 23 14:47:13 2012 +0300 @@ -58,7 +58,9 @@ status_r->uidvalidity = hdr->uid_validity; status_r->uidnext = hdr->next_uid; status_r->first_recent_uid = hdr->first_recent_uid; - status_r->nonpermanent_modseqs = mail_index_is_in_memory(box->index); + status_r->nonpermanent_modseqs = + mail_index_is_in_memory(box->index) || + !mail_index_have_modseq_tracking(box->index); if ((items & STATUS_HIGHESTMODSEQ) != 0) { status_r->highest_modseq = mail_index_modseq_get_highest(box->view); From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: lib-index: Added mail_index_have_modseq_tracking() ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/972b296db20a changeset: 14501:972b296db20a user: Timo Sirainen date: Mon Apr 23 14:50:30 2012 +0300 description: lib-index: Added mail_index_have_modseq_tracking() needed for previous commit. diffstat: src/lib-index/mail-index-modseq.c | 5 +++++ src/lib-index/mail-index-modseq.h | 1 + 2 files changed, 6 insertions(+), 0 deletions(-) diffs (26 lines): diff -r 1222b7e38bf3 -r 972b296db20a src/lib-index/mail-index-modseq.c --- a/src/lib-index/mail-index-modseq.c Mon Apr 23 14:47:13 2012 +0300 +++ b/src/lib-index/mail-index-modseq.c Mon Apr 23 14:50:30 2012 +0300 @@ -88,6 +88,11 @@ index->modseqs_enabled = TRUE; } +bool mail_index_have_modseq_tracking(struct mail_index *index) +{ + return mail_index_map_get_modseq_header(index->map) != NULL; +} + const struct mail_index_modseq_header * mail_index_map_get_modseq_header(struct mail_index_map *map) { diff -r 1222b7e38bf3 -r 972b296db20a src/lib-index/mail-index-modseq.h --- a/src/lib-index/mail-index-modseq.h Mon Apr 23 14:47:13 2012 +0300 +++ b/src/lib-index/mail-index-modseq.h Mon Apr 23 14:50:30 2012 +0300 @@ -26,6 +26,7 @@ mail_index_map_get_modseq_header(struct mail_index_map *map); uint64_t mail_index_map_modseq_get_highest(struct mail_index_map *map); void mail_index_modseq_enable(struct mail_index *index); +bool mail_index_have_modseq_tracking(struct mail_index *index); uint64_t mail_index_modseq_get_highest(struct mail_index_view *view); uint64_t mail_index_modseq_lookup(struct mail_index_view *view, uint32_t seq); From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: Make static analyzer happier. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/27415a4568b3 changeset: 14502:27415a4568b3 user: Timo Sirainen date: Mon Apr 23 16:18:36 2012 +0300 description: Make static analyzer happier. diffstat: src/lib/test-strfuncs.c | 6 ++---- 1 files changed, 2 insertions(+), 4 deletions(-) diffs (16 lines): diff -r 972b296db20a -r 27415a4568b3 src/lib/test-strfuncs.c --- a/src/lib/test-strfuncs.c Mon Apr 23 14:50:30 2012 +0300 +++ b/src/lib/test-strfuncs.c Mon Apr 23 16:18:36 2012 +0300 @@ -37,10 +37,8 @@ s1 = t_strsplit_tab(str); s2 = t_strsplit(str, "\t"); - for (i = 0; s1[i] != NULL; i++) { - test_assert(s2[i] != NULL); - test_assert(strcmp(s1[i], s2[i]) == 0); - } + for (i = 0; s1[i] != NULL; i++) + test_assert(null_strcmp(s1[i], s2[i]) == 0); test_assert(s2[i] == NULL); } T_END; } From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: Released v2.1.5. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/469cee314d9c changeset: 14503:469cee314d9c user: Timo Sirainen date: Mon Apr 23 16:35:37 2012 +0300 description: Released v2.1.5. diffstat: NEWS | 22 ++++++++++++++++++++++ TODO | 1 - configure.in | 2 +- 3 files changed, 23 insertions(+), 2 deletions(-) diffs (50 lines): diff -r 27415a4568b3 -r 469cee314d9c NEWS --- a/NEWS Mon Apr 23 16:18:36 2012 +0300 +++ b/NEWS Mon Apr 23 16:35:37 2012 +0300 @@ -1,3 +1,25 @@ +v2.1.5 2012-04-23 Timo Sirainen + + * IMAP: When neither the session nor the mailbox has modseq tracking + enabled, return the mailbox as having NOMODSEQ in SELECT/EXAMINE + reply. Old versions in this situation always simply returned + HIGHESTMODSEQ as 1, which could have broken some clients. + + + dict file: Added optional fcntl/flock locking (default is dotlock) + + fts-solr: doveadm fts rescan now resets indexes, which allows + reindexing mails. (This isn't a full rescan implementation like + fts-lucene has.) + + doveadm expunge: Added -d parameter to delete mailbox if it's + empty after expunging. + - IMAP: Several fixes related to mailbox listing in some configs + - director: A lot of fixes and performance improvements + - v2.1.4 didn't work without a mail home directory set + - mbox: Deleting a mailbox didn't delete its index files. + - pop3c: TOP command was sent incorrectly + - trash plugin didn't work properly + - LMTP: Don't add a duplicate Return-Path: header when proxying. + - listescape: Don't unescape namespace prefixes. + v2.1.4 2012-04-09 Timo Sirainen + Added mail_temp_scan_interval setting and changed its default value diff -r 27415a4568b3 -r 469cee314d9c TODO --- a/TODO Mon Apr 23 16:18:36 2012 +0300 +++ b/TODO Mon Apr 23 16:35:37 2012 +0300 @@ -46,7 +46,6 @@ non-userdb_* extra fields too? - imap, pop3: if client init fails, wait a second or two before disconnecting client. - - doveadm expunge parameter to delete empty mailboxes (for lazy-expunge) - doveadm search savedbefore 7d could be optimized in large mailboxes.. - mdbox: storage rebuilding could log about changes it does - mdbox: broken extrefs header keeps causing index rebuilds diff -r 27415a4568b3 -r 469cee314d9c configure.in --- a/configure.in Mon Apr 23 16:18:36 2012 +0300 +++ b/configure.in Mon Apr 23 16:35:37 2012 +0300 @@ -1,5 +1,5 @@ AC_PREREQ([2.59]) -AC_INIT([Dovecot],[2.1.4],[dovecot at dovecot.org]) +AC_INIT([Dovecot],[2.1.5],[dovecot at dovecot.org]) AC_CONFIG_SRCDIR([src]) AM_INIT_AUTOMAKE([foreign]) From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: Added tag 2.1.5 for changeset 469cee314d9c Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/9fff96c9cfcd changeset: 14504:9fff96c9cfcd user: Timo Sirainen date: Mon Apr 23 16:35:37 2012 +0300 description: Added tag 2.1.5 for changeset 469cee314d9c diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r 469cee314d9c -r 9fff96c9cfcd .hgtags --- a/.hgtags Mon Apr 23 16:35:37 2012 +0300 +++ b/.hgtags Mon Apr 23 16:35:37 2012 +0300 @@ -81,3 +81,4 @@ 744e0d7f1b255a9339060f761b850303121f14df 2.1.2 b9adfd52cb66d5d89d291b76b9845d6361216d12 2.1.3 2c21c940e19d97a772128a7f281cea302e157d73 2.1.4 +469cee314d9c54d2d7101ec9e38579fdc9610eaf 2.1.5 From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: Added signature for changeset 469cee314d9c Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/6a2ce99e39d4 changeset: 14505:6a2ce99e39d4 user: Timo Sirainen date: Mon Apr 23 16:35:41 2012 +0300 description: Added signature for changeset 469cee314d9c diffstat: .hgsigs | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r 9fff96c9cfcd -r 6a2ce99e39d4 .hgsigs --- a/.hgsigs Mon Apr 23 16:35:37 2012 +0300 +++ b/.hgsigs Mon Apr 23 16:35:41 2012 +0300 @@ -44,3 +44,4 @@ 744e0d7f1b255a9339060f761b850303121f14df 0 iEYEABECAAYFAk9h/8oACgkQyUhSUUBVism2OQCfWh62w8pMxJaf1oYx2A+2PxQvBocAn29RFDgZblGRLn7iMCPw6We1yiIw b9adfd52cb66d5d89d291b76b9845d6361216d12 0 iEYEABECAAYFAk9jbugACgkQyUhSUUBVislSgwCgpo3f0bsSujItBum/M6js8SzF06YAmwftHlwaOstKeALdjLR5vtF2c5F7 2c21c940e19d97a772128a7f281cea302e157d73 0 iEYEABECAAYFAk+CtkYACgkQyUhSUUBVisknxgCfTJw2YPGJ17HbHRGmbwmCyLqepbMAn17j7IYzUfEU0xkXhCgwEAmk7XO4 +469cee314d9c54d2d7101ec9e38579fdc9610eaf 0 iEYEABECAAYFAk+VWqkACgkQyUhSUUBVislnXACfVjPqMmPUvYtXQXwqff0h7N76mZUAn02lPeUCyuyr1TF9e1hGM/sKgmko From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: imapc: Fixed imapc_list_prefix setting to work with... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/6607e35dd8d2 changeset: 14506:6607e35dd8d2 user: Timo Sirainen date: Mon Apr 23 17:59:39 2012 +0300 description: imapc: Fixed imapc_list_prefix setting to work with INBOX. diffstat: src/lib-storage/index/imapc/imapc-list.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (21 lines): diff -r 6a2ce99e39d4 -r 6607e35dd8d2 src/lib-storage/index/imapc/imapc-list.c --- a/src/lib-storage/index/imapc/imapc-list.c Mon Apr 23 16:35:41 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-list.c Mon Apr 23 17:59:39 2012 +0300 @@ -200,7 +200,7 @@ const char *storage_name; storage_name = mailbox_list_default_get_storage_name(_list, vname); - if (*prefix != '\0') { + if (*prefix != '\0' && strcasecmp(storage_name, "INBOX") != 0) { storage_name = t_strdup_printf("%s%c%s", prefix, list->sep, storage_name); } @@ -214,7 +214,7 @@ const char *prefix = list->storage->set->imapc_list_prefix; unsigned int prefix_len; - if (*prefix != '\0') { + if (*prefix != '\0' && strcasecmp(storage_name, "INBOX") != 0) { prefix_len = strlen(prefix); i_assert(strncmp(prefix, storage_name, prefix_len) == 0 && storage_name[prefix_len] == list->sep); From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: quota: Set to quota transaction how many bytes/mess... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/ec8564741aa8 changeset: 14507:ec8564741aa8 user: Timo Sirainen date: Tue Apr 24 21:03:19 2012 +0300 description: quota: Set to quota transaction how many bytes/messages we're over quota. diffstat: src/plugins/quota/quota-private.h | 3 +++ src/plugins/quota/quota.c | 28 +++++++++++++++++++++------- 2 files changed, 24 insertions(+), 7 deletions(-) diffs (65 lines): diff -r 6607e35dd8d2 -r ec8564741aa8 src/plugins/quota/quota-private.h --- a/src/plugins/quota/quota-private.h Mon Apr 23 17:59:39 2012 +0300 +++ b/src/plugins/quota/quota-private.h Tue Apr 24 21:03:19 2012 +0300 @@ -133,6 +133,9 @@ /* how many bytes/mails can be saved until limit is reached. (set once, not updated by bytes_used/count_used) */ uint64_t bytes_ceil, count_ceil; + /* how many bytes/mails we are over quota (either *_ceil or *_over + is always zero) */ + uint64_t bytes_over, count_over; struct mail *tmp_mail; diff -r 6607e35dd8d2 -r ec8564741aa8 src/plugins/quota/quota.c --- a/src/plugins/quota/quota.c Mon Apr 23 17:59:39 2012 +0300 +++ b/src/plugins/quota/quota.c Tue Apr 24 21:03:19 2012 +0300 @@ -940,7 +940,7 @@ struct quota_root *const *roots; const char *mailbox_name; unsigned int i, count; - uint64_t bytes_limit, count_limit, current, limit, ceil; + uint64_t bytes_limit, count_limit, current, limit, diff; int ret; ctx->limits_set = TRUE; @@ -964,9 +964,16 @@ QUOTA_NAME_STORAGE_BYTES, ¤t, &limit); if (ret > 0) { - ceil = limit < current ? 0 : limit - current; - if (ctx->bytes_ceil > ceil) - ctx->bytes_ceil = ceil; + if (limit < current) { + ctx->bytes_ceil = 0; + diff = current - limit; + if (ctx->bytes_over < diff) + ctx->bytes_over = diff; + } else { + diff = limit - current; + if (ctx->bytes_ceil > diff) + ctx->bytes_ceil = diff; + } } else if (ret < 0) { ctx->failed = TRUE; return -1; @@ -978,9 +985,16 @@ QUOTA_NAME_MESSAGES, ¤t, &limit); if (ret > 0) { - ceil = limit < current ? 0 : limit - current; - if (ctx->count_ceil > ceil) - ctx->count_ceil = ceil; + if (limit < current) { + ctx->count_ceil = 0; + diff = current - limit; + if (ctx->count_over < diff) + ctx->count_over = diff; + } else { + diff = limit - current; + if (ctx->count_ceil > diff) + ctx->count_ceil = diff; + } } else if (ret < 0) { ctx->failed = TRUE; return -1; From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: trash plugin: Fixed behavior when quota is already ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/dd3798681283 changeset: 14508:dd3798681283 user: Timo Sirainen date: Tue Apr 24 21:09:25 2012 +0300 description: trash plugin: Fixed behavior when quota is already over limit. diffstat: src/plugins/trash/trash-plugin.c | 36 +++++++++++++++++++++++++++++++----- 1 files changed, 31 insertions(+), 5 deletions(-) diffs (79 lines): diff -r ec8564741aa8 -r dd3798681283 src/plugins/trash/trash-plugin.c --- a/src/plugins/trash/trash-plugin.c Tue Apr 24 21:03:19 2012 +0300 +++ b/src/plugins/trash/trash-plugin.c Tue Apr 24 21:09:25 2012 +0300 @@ -97,13 +97,15 @@ } static int trash_try_clean_mails(struct quota_transaction_context *ctx, - uint64_t size_needed) + uint64_t size_needed, + unsigned int count_needed) { struct trash_user *tuser = TRASH_USER_CONTEXT(ctx->quota->user); struct trash_mailbox *trashes; unsigned int i, j, count, oldest_idx; time_t oldest, received = 0; - uint64_t size, size_expunged = 0, expunged_count = 0; + uint64_t size, size_expunged = 0; + unsigned int expunged_count = 0; int ret = 0; trashes = array_get_modifiable(&tuser->trash_boxes, &count); @@ -139,7 +141,8 @@ mail_expunge(trashes[oldest_idx].mail); expunged_count++; size_expunged += size; - if (size_expunged >= size_needed) + if (size_expunged >= size_needed && + expunged_count >= count_needed) break; trashes[oldest_idx].mail = NULL; } else { @@ -158,7 +161,8 @@ trash->mail = NULL; (void)mailbox_search_deinit(&trash->search_ctx); - if (size_expunged >= size_needed) { + if (size_expunged >= size_needed && + expunged_count >= count_needed) { (void)mailbox_transaction_commit(&trash->trans); (void)mailbox_sync(trash->box, 0); } else { @@ -178,6 +182,27 @@ } return 0; } + if (expunged_count < count_needed) { + if (ctx->quota->user->mail_debug) { + i_debug("trash plugin: Failed to remove enough messages " + "(needed %u messages, expunged only %u messages)", + count_needed, expunged_count); + } + return 0; + } + + if (ctx->bytes_over > 0) { + /* user is over quota. drop the over-bytes first. */ + i_assert(ctx->bytes_over <= size_expunged); + size_expunged -= ctx->bytes_over; + ctx->bytes_over = 0; + } + if (ctx->count_over > 0) { + /* user is over quota. drop the over-count first. */ + i_assert(ctx->count_over <= expunged_count); + expunged_count -= ctx->count_over; + ctx->count_over = 0; + } ctx->bytes_ceil += size_expunged; ctx->count_ceil += expunged_count; @@ -209,7 +234,8 @@ } /* not enough space. try deleting some from mailbox. */ - ret = trash_try_clean_mails(ctx, size); + ret = trash_try_clean_mails(ctx, size + ctx->bytes_over, + 1 + ctx->count_over); if (ret <= 0) return 0; } From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: doveadm expunge -d: If mailbox is deleted, unsubscr... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/b1653b7bd165 changeset: 14509:b1653b7bd165 user: Timo Sirainen date: Tue Apr 24 21:12:36 2012 +0300 description: doveadm expunge -d: If mailbox is deleted, unsubscribe it also diffstat: src/doveadm/doveadm-mail-expunge.c | 24 +++++++++++++----------- 1 files changed, 13 insertions(+), 11 deletions(-) diffs (52 lines): diff -r dd3798681283 -r b1653b7bd165 src/doveadm/doveadm-mail-expunge.c --- a/src/doveadm/doveadm-mail-expunge.c Tue Apr 24 21:09:25 2012 +0300 +++ b/src/doveadm/doveadm-mail-expunge.c Tue Apr 24 21:12:36 2012 +0300 @@ -25,6 +25,7 @@ struct mailbox_transaction_context *trans; struct mail *mail; enum mail_error error; + int ret = 0; if (doveadm_mail_iter_init(_ctx, info, search_args, 0, NULL, &trans, &iter) < 0) @@ -38,28 +39,29 @@ mail_expunge(mail); } - if (doveadm_mail_iter_deinit_keep_box(&iter, &box) < 0) { - mailbox_free(&box); - return -1; - } - if (mailbox_sync(box, 0) < 0) { + if (doveadm_mail_iter_deinit_keep_box(&iter, &box) < 0) + ret = -1; + else if (mailbox_sync(box, 0) < 0) { doveadm_mail_failed_mailbox(_ctx, box); - mailbox_free(&box); - return -1; + ret = -1; } - if (ctx->delete_empty_mailbox) { + if (ctx->delete_empty_mailbox && ret == 0) { if (mailbox_delete_empty(box) < 0) { (void)mailbox_get_last_error(box, &error); if (error != MAIL_ERROR_EXISTS) { doveadm_mail_failed_mailbox(_ctx, box); - mailbox_free(&box); - return -1; + ret = -1; + } + } else { + if (mailbox_set_subscribed(box, FALSE) < 0) { + doveadm_mail_failed_mailbox(_ctx, box); + ret = -1; } } } mailbox_free(&box); - return 0; + return ret; } static bool From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: lib-ssl-iostream: Fixed a memory leak when verifyin... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d2b01dcce79f changeset: 14510:d2b01dcce79f user: Timo Sirainen date: Wed Apr 25 02:48:36 2012 +0300 description: lib-ssl-iostream: Fixed a memory leak when verifying certificate name. This mainly mattered when proxying with SSL to remote server. diffstat: src/lib-ssl-iostream/iostream-openssl.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff -r b1653b7bd165 -r d2b01dcce79f src/lib-ssl-iostream/iostream-openssl.c --- a/src/lib-ssl-iostream/iostream-openssl.c Tue Apr 24 21:12:36 2012 +0300 +++ b/src/lib-ssl-iostream/iostream-openssl.c Wed Apr 25 02:48:36 2012 +0300 @@ -517,6 +517,8 @@ } } sk_GENERAL_NAME_pop_free(gnames, GENERAL_NAME_free); + X509_free(cert); + /* verify against CommonName only when there wasn't any DNS SubjectAltNames */ if (dns_names) From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: dsync: Treat \Nonexistent mailboxes the same as \No... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/cb78154a9c0c changeset: 14511:cb78154a9c0c user: Timo Sirainen date: Wed Apr 25 16:29:22 2012 +0300 description: dsync: Treat \Nonexistent mailboxes the same as \Noselect mailboxes. diffstat: src/doveadm/dsync/dsync-worker-local.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r d2b01dcce79f -r cb78154a9c0c src/doveadm/dsync/dsync-worker-local.c --- a/src/doveadm/dsync/dsync-worker-local.c Wed Apr 25 02:48:36 2012 +0300 +++ b/src/doveadm/dsync/dsync-worker-local.c Wed Apr 25 16:29:22 2012 +0300 @@ -543,7 +543,7 @@ dsync_box_r->last_change = dir_change->last_rename; } - if ((info->flags & MAILBOX_NOSELECT) != 0) { + if ((info->flags & (MAILBOX_NOSELECT | MAILBOX_NONEXISTENT)) != 0) { dsync_box_r->flags |= DSYNC_MAILBOX_FLAG_NOSELECT; local_dsync_worker_add_mailbox(worker, info->ns, info->name, &dsync_box_r->name_sha1); From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: lib-storage: Make sure both MAILBOX_CHILDREN and MA... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/e928c1575689 changeset: 14512:e928c1575689 user: Timo Sirainen date: Wed Apr 25 16:44:00 2012 +0300 description: lib-storage: Make sure both MAILBOX_CHILDREN and MAILBOX_NOCHILDREN flags aren't set. diffstat: src/lib-storage/mailbox-list-iter.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff -r cb78154a9c0c -r e928c1575689 src/lib-storage/mailbox-list-iter.c --- a/src/lib-storage/mailbox-list-iter.c Wed Apr 25 16:29:22 2012 +0300 +++ b/src/lib-storage/mailbox-list-iter.c Wed Apr 25 16:44:00 2012 +0300 @@ -762,6 +762,8 @@ if (node != NULL) { if (!ctx->update_only && add_matched) node->flags |= MAILBOX_MATCHED; + if ((always_flags & MAILBOX_CHILDREN) != 0) + node->flags &= ~MAILBOX_NOCHILDREN; node->flags |= always_flags; } /* We don't want to show the parent mailboxes unless From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: auth: Minor code cleanup. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/26bf68448638 changeset: 14514:26bf68448638 user: Timo Sirainen date: Wed Apr 25 21:23:24 2012 +0300 description: auth: Minor code cleanup. diffstat: src/auth/auth-request-handler.c | 11 +++++------ 1 files changed, 5 insertions(+), 6 deletions(-) diffs (21 lines): diff -r aba344bb85f3 -r 26bf68448638 src/auth/auth-request-handler.c --- a/src/auth/auth-request-handler.c Wed Apr 25 16:44:22 2012 +0300 +++ b/src/auth/auth-request-handler.c Wed Apr 25 21:23:24 2012 +0300 @@ -162,12 +162,11 @@ extra_fields = auth_stream_reply_export(request->extra_fields); - if (!request->proxy) { - /* we only wish to remove all fields prefixed with "userdb_" */ - if (strstr(extra_fields, "userdb_") == NULL) { - auth_stream_reply_import(reply, extra_fields); - return; - } + if (!request->proxy && strstr(extra_fields, "userdb_") == NULL) { + /* optimization: there are no userdb_* fields, we can just + import */ + auth_stream_reply_import(reply, extra_fields); + return; } fields = t_strsplit_tab(extra_fields); From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: imapc: List INBOX for inbox=yes namespace even imap... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/aba344bb85f3 changeset: 14513:aba344bb85f3 user: Timo Sirainen date: Wed Apr 25 16:44:22 2012 +0300 description: imapc: List INBOX for inbox=yes namespace even imapc_list_prefix is set. diffstat: src/lib-storage/index/imapc/imapc-list.c | 10 +++++++++- 1 files changed, 9 insertions(+), 1 deletions(-) diffs (27 lines): diff -r e928c1575689 -r aba344bb85f3 src/lib-storage/index/imapc/imapc-list.c --- a/src/lib-storage/index/imapc/imapc-list.c Wed Apr 25 16:44:00 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-list.c Wed Apr 25 16:44:22 2012 +0300 @@ -365,6 +365,14 @@ list->mailboxes = mailbox_tree_init(list->sep); mailbox_tree_set_parents_nonexistent(list->mailboxes); + if ((list->list.ns->flags & NAMESPACE_FLAG_INBOX_USER) != 0) { + /* INBOX always exists in IMAP server. since this namespace is + marked with inbox=yes, show the INBOX even if + imapc_list_prefix doesn't match it */ + bool created; + (void)mailbox_tree_get(list->mailboxes, "INBOX", &created); + } + imapc_simple_run(&ctx); if (ctx.ret == 0) { list->refreshed_mailboxes = TRUE; @@ -471,7 +479,7 @@ node = mailbox_tree_iterate_next(ctx->iter, &name); if (node == NULL) return NULL; - } while (node == ctx->ns_root || (node->flags & MAILBOX_MATCHED) == 0); + } while ((node->flags & MAILBOX_MATCHED) == 0); ctx->info.name = name; ctx->info.flags = node->flags; From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: *-login: Don't fail client's certificate if CRL is ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/c2b7767afc38 changeset: 14517:c2b7767afc38 user: Timo Sirainen date: Wed Apr 25 21:29:14 2012 +0300 description: *-login: Don't fail client's certificate if CRL is expired. This seems to be returned also when there is no CRL.. diffstat: src/login-common/ssl-proxy-openssl.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diffs (14 lines): diff -r 36cde186aec6 -r c2b7767afc38 src/login-common/ssl-proxy-openssl.c --- a/src/login-common/ssl-proxy-openssl.c Wed Apr 25 21:28:16 2012 +0300 +++ b/src/login-common/ssl-proxy-openssl.c Wed Apr 25 21:29:14 2012 +0300 @@ -864,7 +864,9 @@ proxy = SSL_get_ex_data(ssl, extdata_index); proxy->cert_received = TRUE; - if (proxy->client_proxy && ctx->error == X509_V_ERR_UNABLE_TO_GET_CRL) { + if (proxy->client_proxy && + (ctx->error == X509_V_ERR_UNABLE_TO_GET_CRL || + ctx->error == X509_V_ERR_CRL_HAS_EXPIRED)) { /* no CRL given with the CA list. don't worry about it. */ preverify_ok = 1; } From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: *-login: If client certificate isn't valid, log the... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/36cde186aec6 changeset: 14516:36cde186aec6 user: Timo Sirainen date: Wed Apr 25 21:28:16 2012 +0300 description: *-login: If client certificate isn't valid, log the reason why. diffstat: src/login-common/login-proxy.c | 5 ++- src/login-common/ssl-proxy-openssl.c | 43 ++++++++++++++++++++++++----------- src/login-common/ssl-proxy.c | 5 ++++ src/login-common/ssl-proxy.h | 1 + 4 files changed, 38 insertions(+), 16 deletions(-) diffs (120 lines): diff -r 3893f2b7e4ab -r 36cde186aec6 src/login-common/login-proxy.c --- a/src/login-common/login-proxy.c Wed Apr 25 21:26:25 2012 +0300 +++ b/src/login-common/login-proxy.c Wed Apr 25 21:28:16 2012 +0300 @@ -516,8 +516,9 @@ if (ssl_proxy_has_broken_client_cert(proxy->ssl_server_proxy)) { client_log_err(proxy->client, t_strdup_printf( - "proxy: Received invalid SSL certificate from %s:%u", - proxy->host, proxy->port)); + "proxy: Received invalid SSL certificate from %s:%u: %s", + proxy->host, proxy->port, + ssl_proxy_get_cert_error(proxy->ssl_server_proxy))); } else if (!ssl_proxy_has_valid_client_cert(proxy->ssl_server_proxy)) { client_log_err(proxy->client, t_strdup_printf( "proxy: SSL certificate not received from %s:%u", diff -r 3893f2b7e4ab -r 36cde186aec6 src/login-common/ssl-proxy-openssl.c --- a/src/login-common/ssl-proxy-openssl.c Wed Apr 25 21:26:25 2012 +0300 +++ b/src/login-common/ssl-proxy-openssl.c Wed Apr 25 21:28:16 2012 +0300 @@ -66,6 +66,7 @@ ssl_handshake_callback_t *handshake_callback; void *handshake_context; + const char *cert_error; char *last_error; unsigned int handshaked:1; unsigned int destroyed:1; @@ -754,6 +755,12 @@ #endif } +const char *ssl_proxy_get_cert_error(struct ssl_proxy *proxy) +{ + return proxy->cert_error != NULL ? proxy->cert_error : + "(Unknown error)"; +} + void ssl_proxy_free(struct ssl_proxy **_proxy) { struct ssl_proxy *proxy = *_proxy; @@ -849,32 +856,40 @@ { SSL *ssl; struct ssl_proxy *proxy; + char buf[1024]; + X509_NAME *subject; ssl = X509_STORE_CTX_get_ex_data(ctx, SSL_get_ex_data_X509_STORE_CTX_idx()); proxy = SSL_get_ex_data(ssl, extdata_index); proxy->cert_received = TRUE; - if (proxy->set->verbose_ssl || - (proxy->set->auth_verbose && !preverify_ok)) { - char buf[1024]; - X509_NAME *subject; - - subject = X509_get_subject_name(ctx->current_cert); - (void)X509_NAME_oneline(subject, buf, sizeof(buf)); - buf[sizeof(buf)-1] = '\0'; /* just in case.. */ - if (!preverify_ok) - i_info("Invalid certificate: %s: %s", X509_verify_cert_error_string(ctx->error),buf); - else - i_info("Valid certificate: %s", buf); - } - if (ctx->error == X509_V_ERR_UNABLE_TO_GET_CRL && proxy->client_proxy) { + if (proxy->client_proxy && ctx->error == X509_V_ERR_UNABLE_TO_GET_CRL) { /* no CRL given with the CA list. don't worry about it. */ preverify_ok = 1; } if (!preverify_ok) proxy->cert_broken = TRUE; + subject = X509_get_subject_name(ctx->current_cert); + (void)X509_NAME_oneline(subject, buf, sizeof(buf)); + buf[sizeof(buf)-1] = '\0'; /* just in case.. */ + + if (proxy->cert_error == NULL) { + proxy->cert_error = p_strdup_printf(proxy->client->pool, "%s: %s", + X509_verify_cert_error_string(ctx->error), buf); + } + + if (proxy->set->verbose_ssl || + (proxy->set->auth_verbose && !preverify_ok)) { + if (preverify_ok) + i_info("Valid certificate: %s", buf); + else { + i_info("Invalid certificate: %s: %s", + X509_verify_cert_error_string(ctx->error), buf); + } + } + /* Return success anyway, because if ssl_require_client_cert=no we could still allow authentication. */ return 1; diff -r 3893f2b7e4ab -r 36cde186aec6 src/login-common/ssl-proxy.c --- a/src/login-common/ssl-proxy.c Wed Apr 25 21:26:25 2012 +0300 +++ b/src/login-common/ssl-proxy.c Wed Apr 25 21:28:16 2012 +0300 @@ -79,6 +79,11 @@ return NULL; } +const char *ssl_proxy_get_cert_error(struct ssl_proxy *proxy ATTR_UNUSED) +{ + return ""; +} + void ssl_proxy_free(struct ssl_proxy **proxy ATTR_UNUSED) {} unsigned int ssl_proxy_get_count(void) diff -r 3893f2b7e4ab -r 36cde186aec6 src/login-common/ssl-proxy.h --- a/src/login-common/ssl-proxy.h Wed Apr 25 21:26:25 2012 +0300 +++ b/src/login-common/ssl-proxy.h Wed Apr 25 21:28:16 2012 +0300 @@ -30,6 +30,7 @@ const char *ssl_proxy_get_last_error(const struct ssl_proxy *proxy) ATTR_PURE; const char *ssl_proxy_get_security_string(struct ssl_proxy *proxy); const char *ssl_proxy_get_compression(struct ssl_proxy *proxy); +const char *ssl_proxy_get_cert_error(struct ssl_proxy *proxy); void ssl_proxy_free(struct ssl_proxy **proxy); /* Return number of active SSL proxies */ From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: SSL proxying: Remote's host never matched cert, bec... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/773ca397d799 changeset: 14518:773ca397d799 user: Timo Sirainen date: Wed Apr 25 21:32:00 2012 +0300 description: SSL proxying: Remote's host never matched cert, because auth process changed it to IP. Now the "host" parameter isn't changed, but a new optional "hostip" parameter contains the IP address where to connect to. diffstat: src/auth/auth-request.c | 4 ++-- src/login-common/client-common-auth.c | 5 +++++ src/login-common/client-common.h | 2 +- src/login-common/login-proxy.c | 4 +++- src/login-common/login-proxy.h | 3 +++ 5 files changed, 14 insertions(+), 4 deletions(-) diffs (89 lines): diff -r c2b7767afc38 -r 773ca397d799 src/auth/auth-request.c --- a/src/auth/auth-request.c Wed Apr 25 21:29:14 2012 +0300 +++ b/src/auth/auth-request.c Wed Apr 25 21:32:00 2012 +0300 @@ -1521,8 +1521,8 @@ "DNS lookup for %s took %u.%03u s", host, result->msecs/1000, result->msecs % 1000); } - auth_stream_reply_remove(request->extra_fields, "host"); - auth_stream_reply_add(request->extra_fields, "host", + auth_stream_reply_remove(request->extra_fields, "hostip"); + auth_stream_reply_add(request->extra_fields, "hostip", net_ip2addr(&result->ips[0])); for (i = 0; i < result->ips_count; i++) { if (auth_request_proxy_ip_is_self(request, diff -r c2b7767afc38 -r 773ca397d799 src/login-common/client-common-auth.c --- a/src/login-common/client-common-auth.c Wed Apr 25 21:29:14 2012 +0300 +++ b/src/login-common/client-common-auth.c Wed Apr 25 21:32:00 2012 +0300 @@ -86,6 +86,8 @@ reply_r->reason = value; else if (strcmp(key, "host") == 0) reply_r->host = value; + else if (strcmp(key, "hostip") == 0) + reply_r->hostip = value; else if (strcmp(key, "port") == 0) reply_r->port = atoi(value); else if (strcmp(key, "destuser") == 0) @@ -294,6 +296,9 @@ memset(&proxy_set, 0, sizeof(proxy_set)); proxy_set.host = reply->host; + if (reply->hostip != NULL && + net_addr2ip(reply->hostip, &proxy_set.ip) < 0) + proxy_set.ip.family = 0; proxy_set.port = reply->port; proxy_set.dns_client_socket_path = LOGIN_DNS_CLIENT_SOCKET_PATH; proxy_set.connect_timeout_msecs = reply->proxy_timeout_msecs; diff -r c2b7767afc38 -r 773ca397d799 src/login-common/client-common.h --- a/src/login-common/client-common.h Wed Apr 25 21:29:14 2012 +0300 +++ b/src/login-common/client-common.h Wed Apr 25 21:32:00 2012 +0300 @@ -49,7 +49,7 @@ struct client_auth_reply { const char *master_user, *reason; /* for proxying */ - const char *host, *destuser, *password; + const char *host, *hostip, *destuser, *password; unsigned int port; unsigned int proxy_timeout_msecs; unsigned int proxy_refresh_secs; diff -r c2b7767afc38 -r 773ca397d799 src/login-common/login-proxy.c --- a/src/login-common/login-proxy.c Wed Apr 25 21:29:14 2012 +0300 +++ b/src/login-common/login-proxy.c Wed Apr 25 21:32:00 2012 +0300 @@ -308,6 +308,7 @@ proxy->client_fd = -1; proxy->server_fd = -1; proxy->created = ioloop_timeval; + proxy->ip = set->ip; proxy->host = i_strdup(set->host); proxy->port = set->port; proxy->connect_timeout_msecs = set->connect_timeout_msecs; @@ -319,7 +320,8 @@ dns_lookup_set.dns_client_socket_path = set->dns_client_socket_path; dns_lookup_set.timeout_msecs = set->connect_timeout_msecs; - if (net_addr2ip(set->host, &proxy->ip) < 0) { + if (set->ip.family == 0 && + net_addr2ip(set->host, &proxy->ip) < 0) { if (dns_lookup(set->host, &dns_lookup_set, login_proxy_dns_done, proxy) < 0) return -1; diff -r c2b7767afc38 -r 773ca397d799 src/login-common/login-proxy.h --- a/src/login-common/login-proxy.h Wed Apr 25 21:29:14 2012 +0300 +++ b/src/login-common/login-proxy.h Wed Apr 25 21:32:00 2012 +0300 @@ -1,6 +1,8 @@ #ifndef LOGIN_PROXY_H #define LOGIN_PROXY_H +#include "network.h" + struct client; struct login_proxy; @@ -15,6 +17,7 @@ struct login_proxy_settings { const char *host; + struct ip_addr ip; const char *dns_client_socket_path; unsigned int port; unsigned int connect_timeout_msecs; From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: stats plugin: Open /proc/self/io only once, reopeni... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/96800058f29b changeset: 14519:96800058f29b user: Timo Sirainen date: Wed Apr 25 22:12:26 2012 +0300 description: stats plugin: Open /proc/self/io only once, reopening is unnecessary. diffstat: src/plugins/stats/stats-plugin.c | 48 ++++++++++++++++++++++++--------------- 1 files changed, 29 insertions(+), 19 deletions(-) diffs (91 lines): diff -r 773ca397d799 -r 96800058f29b src/plugins/stats/stats-plugin.c --- a/src/plugins/stats/stats-plugin.c Wed Apr 25 21:32:00 2012 +0300 +++ b/src/plugins/stats/stats-plugin.c Wed Apr 25 22:12:26 2012 +0300 @@ -20,6 +20,7 @@ #define SESSION_STATS_FORCE_REFRESH_SECS (5*60) #define REFRESH_CHECK_INTERVAL 100 #define MAIL_STATS_SOCKET_NAME "stats-mail" +#define PROC_IO_PATH "/proc/self/io" #define USECS_PER_SEC 1000000 @@ -43,6 +44,9 @@ static MODULE_CONTEXT_DEFINE_INIT(stats_storage_module, &mail_storage_module_register); +static bool proc_io_disabled = FALSE; +static int proc_io_fd = -1; + static struct stats_connection *global_stats_conn = NULL; static struct mail_user *stats_global_user = NULL; static unsigned int stats_user_count = 0; @@ -106,44 +110,50 @@ return 0; } +static int process_io_open(void) +{ + if (proc_io_fd == -1) { + if (proc_io_disabled) + return -1; + proc_io_fd = open(PROC_IO_PATH, O_RDONLY); + if (proc_io_fd == -1) { + if (errno != ENOENT) + i_error("open(%s) failed: %m", PROC_IO_PATH); + proc_io_disabled = TRUE; + return -1; + } + } + return proc_io_fd; +} + static void process_read_io_stats(struct mail_stats *stats) { - const char *path = "/proc/self/io"; - static bool io_disabled = FALSE; char buf[1024]; int fd, ret; - if (io_disabled) + if ((fd = process_io_open()) == -1) return; - fd = open(path, O_RDONLY); - if (fd == -1) { - if (errno != ENOENT) - i_error("open(%s) failed: %m", path); - io_disabled = TRUE; - return; - } - ret = read(fd, buf, sizeof(buf)); + ret = pread(fd, buf, sizeof(buf), 0); if (ret <= 0) { if (ret == -1) - i_error("read(%s) failed: %m", path); + i_error("read(%s) failed: %m", PROC_IO_PATH); else - i_error("read(%s) returned EOF", path); + i_error("read(%s) returned EOF", PROC_IO_PATH); } else if (ret == sizeof(buf)) { /* just shouldn't happen.. */ - i_error("%s is larger than expected", path); - io_disabled = TRUE; + i_error("%s is larger than expected", PROC_IO_PATH); + proc_io_disabled = TRUE; } else { buf[ret] = '\0'; T_BEGIN { if (process_io_buffer_parse(buf, stats) < 0) { - i_error("Invalid input in file %s", path); - io_disabled = TRUE; + i_error("Invalid input in file %s", + PROC_IO_PATH); + proc_io_disabled = TRUE; } } T_END; } - if (close(fd) < 0) - i_error("close(%s) failed: %m", path); } void mail_stats_get(struct stats_user *suser, struct mail_stats *stats_r) From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: auth: Fixed handling multiple passdbs where some us... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/3893f2b7e4ab changeset: 14515:3893f2b7e4ab user: Timo Sirainen date: Wed Apr 25 21:26:25 2012 +0300 description: auth: Fixed handling multiple passdbs where some use proxying and others don't. diffstat: src/auth/auth-request.c | 13 ++++++++++++- 1 files changed, 12 insertions(+), 1 deletions(-) diffs (41 lines): diff -r 26bf68448638 -r 3893f2b7e4ab src/auth/auth-request.c --- a/src/auth/auth-request.c Wed Apr 25 21:23:24 2012 +0300 +++ b/src/auth/auth-request.c Wed Apr 25 21:26:25 2012 +0300 @@ -517,6 +517,10 @@ request->passdb = request->passdb->next; request->passdb_password = NULL; + request->proxy = FALSE; + request->proxy_maybe = FALSE; + request->proxy_always = FALSE; + if (*result == PASSDB_RESULT_USER_UNKNOWN) { /* remember that we did at least one successful passdb lookup */ @@ -1477,11 +1481,14 @@ } else { /* proxying to ourself - log in without proxying by dropping all the proxying fields. */ + bool proxy_always = request->proxy_always; + auth_request_proxy_finish_failure(request); - if (request->proxy_always) { + if (proxy_always) { /* director adds the host */ auth_stream_reply_add(request->extra_fields, "proxy", NULL); + request->proxy = TRUE; } } } @@ -1602,6 +1609,10 @@ auth_stream_reply_remove(request->extra_fields, "host"); auth_stream_reply_remove(request->extra_fields, "port"); auth_stream_reply_remove(request->extra_fields, "destuser"); + + request->proxy = FALSE; + request->proxy_maybe = FALSE; + request->proxy_always = FALSE; } static void log_password_failure(struct auth_request *request, From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: *-login: Added ssl_require_crl setting. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/008c1afeba3c changeset: 14520:008c1afeba3c user: Timo Sirainen date: Wed Apr 25 22:28:03 2012 +0300 description: *-login: Added ssl_require_crl setting. diffstat: doc/example-config/conf.d/10-ssl.conf | 3 +++ src/login-common/login-settings.c | 2 ++ src/login-common/login-settings.h | 1 + src/login-common/ssl-proxy-openssl.c | 2 +- 4 files changed, 7 insertions(+), 1 deletions(-) diffs (55 lines): diff -r 96800058f29b -r 008c1afeba3c doc/example-config/conf.d/10-ssl.conf --- a/doc/example-config/conf.d/10-ssl.conf Wed Apr 25 22:12:26 2012 +0300 +++ b/doc/example-config/conf.d/10-ssl.conf Wed Apr 25 22:28:03 2012 +0300 @@ -23,6 +23,9 @@ # followed by the matching CRL(s). (e.g. ssl_ca = cert_received = TRUE; - if (proxy->client_proxy && + if (proxy->client_proxy && !proxy->set->ssl_require_crl && (ctx->error == X509_V_ERR_UNABLE_TO_GET_CRL || ctx->error == X509_V_ERR_CRL_HAS_EXPIRED)) { /* no CRL given with the CA list. don't worry about it. */ From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: acl: Fixed looking up parent mailbox name with diff... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/e8b80e0767ac changeset: 14523:e8b80e0767ac user: Timo Sirainen date: Thu Apr 26 00:26:58 2012 +0300 description: acl: Fixed looking up parent mailbox name with different namespace vs. layout separators. diffstat: src/plugins/acl/acl-backend-vfile.c | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diffs (15 lines): diff -r 249c1c89d9d3 -r e8b80e0767ac src/plugins/acl/acl-backend-vfile.c --- a/src/plugins/acl/acl-backend-vfile.c Wed Apr 25 23:44:32 2012 +0300 +++ b/src/plugins/acl/acl-backend-vfile.c Thu Apr 26 00:26:58 2012 +0300 @@ -184,10 +184,9 @@ static const char * get_parent_mailbox(struct acl_backend *backend, const char *name) { - struct mail_namespace *ns = mailbox_list_get_namespace(backend->list); const char *p; - p = strrchr(name, mail_namespace_get_sep(ns)); + p = strrchr(name, mailbox_list_get_hierarchy_sep(backend->list)); return p == NULL ? NULL : t_strdup_until(name, p); } From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: passdb static: Don't crash if password/nopassword i... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/c2038fb33633 changeset: 14521:c2038fb33633 user: Timo Sirainen date: Wed Apr 25 23:34:29 2012 +0300 description: passdb static: Don't crash if password/nopassword isn't set. diffstat: src/auth/passdb-static.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 008c1afeba3c -r c2038fb33633 src/auth/passdb-static.c --- a/src/auth/passdb-static.c Wed Apr 25 22:28:03 2012 +0300 +++ b/src/auth/passdb-static.c Wed Apr 25 23:34:29 2012 +0300 @@ -25,7 +25,7 @@ passdb_template_export(module->tmpl, request); if (module->static_password_tmpl == NULL) - *password_r = NULL; + *password_r = ""; else { table = auth_request_get_var_expand_table(request, NULL); var_expand(str, module->static_password_tmpl, table); From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: lib-storage: Fixed expanding path that ends with "~/" Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/249c1c89d9d3 changeset: 14522:249c1c89d9d3 user: Timo Sirainen date: Wed Apr 25 23:44:32 2012 +0300 description: lib-storage: Fixed expanding path that ends with "~/" For example mail_location=maildir:~/ would actually try to create '~' directory. diffstat: src/lib-storage/mail-user.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r c2038fb33633 -r 249c1c89d9d3 src/lib-storage/mail-user.c --- a/src/lib-storage/mail-user.c Wed Apr 25 23:34:29 2012 +0300 +++ b/src/lib-storage/mail-user.c Wed Apr 25 23:44:32 2012 +0300 @@ -343,7 +343,7 @@ { const char *home, *path = *pathp; - if (strncmp(path, "~/", 2) != 0) { + if (*path != '~') { /* no need to expand home */ return 0; } From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: lib-storage: Allocate MAILBOX_METADATA_CACHE_FIELDS... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/3b4c336ed01b changeset: 14524:3b4c336ed01b user: Timo Sirainen date: Sat Apr 28 17:33:34 2012 +0300 description: lib-storage: Allocate MAILBOX_METADATA_CACHE_FIELDS from a new metadata mempool. diffstat: src/lib-storage/index/index-status.c | 12 +++++++++--- src/lib-storage/mail-storage-private.h | 2 +- src/lib-storage/mail-storage.c | 5 +++++ 3 files changed, 15 insertions(+), 4 deletions(-) diffs (58 lines): diff -r e8b80e0767ac -r 3b4c336ed01b src/lib-storage/index/index-status.c --- a/src/lib-storage/index/index-status.c Thu Apr 26 00:26:58 2012 +0300 +++ b/src/lib-storage/index/index-status.c Sat Apr 28 17:33:34 2012 +0300 @@ -100,11 +100,17 @@ struct mailbox_cache_field *cf; unsigned int i, count; + if (box->metadata_pool == NULL) { + box->metadata_pool = + pool_alloconly_create("mailbox metadata", 256); + } + fields = mail_cache_register_get_list(box->cache, - pool_datastack_create(), &count); + box->metadata_pool, &count); - cache_fields = t_new(ARRAY_TYPE(mailbox_cache_field), 1); - t_array_init(cache_fields, count); + cache_fields = p_new(box->metadata_pool, + ARRAY_TYPE(mailbox_cache_field), 1); + p_array_init(cache_fields, box->metadata_pool, count); for (i = 0; i < count; i++) { dec = fields[i].decision & ~MAIL_CACHE_DECISION_FORCED; if (dec != MAIL_CACHE_DECISION_NO) { diff -r e8b80e0767ac -r 3b4c336ed01b src/lib-storage/mail-storage-private.h --- a/src/lib-storage/mail-storage-private.h Thu Apr 26 00:26:58 2012 +0300 +++ b/src/lib-storage/mail-storage-private.h Sat Apr 28 17:33:34 2012 +0300 @@ -212,7 +212,7 @@ struct mailbox_vfuncs v, *vlast; /* private: */ - pool_t pool; + pool_t pool, metadata_pool; /* Linked list of all mailboxes in this storage */ struct mailbox *prev, *next; diff -r e8b80e0767ac -r 3b4c336ed01b src/lib-storage/mail-storage.c --- a/src/lib-storage/mail-storage.c Thu Apr 26 00:26:58 2012 +0300 +++ b/src/lib-storage/mail-storage.c Sat Apr 28 17:33:34 2012 +0300 @@ -989,6 +989,8 @@ DLLIST_REMOVE(&box->storage->mailboxes, box); mail_storage_obj_unref(box->storage); + if (box->metadata_pool != NULL) + pool_unref(&box->metadata_pool); pool_unref(&box->pool); } @@ -1282,6 +1284,9 @@ { memset(metadata_r, 0, sizeof(*metadata_r)); + if (box->metadata_pool != NULL) + p_clear(box->metadata_pool); + if (box->v.get_metadata(box, items, metadata_r) < 0) return -1; From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: auth: Include session ID in log line prefix. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/629afda8e29d changeset: 14525:629afda8e29d user: Timo Sirainen date: Sat Apr 28 17:41:14 2012 +0300 description: auth: Include session ID in log line prefix. diffstat: src/auth/auth-request.c | 4 ++++ src/auth/auth-request.h | 2 +- src/lib-auth/auth-client-request.c | 6 ++++++ src/lib-auth/auth-client.h | 1 + src/login-common/login-settings.c | 2 +- src/login-common/sasl-server.c | 1 + 6 files changed, 14 insertions(+), 2 deletions(-) diffs (90 lines): diff -r 3b4c336ed01b -r 629afda8e29d src/auth/auth-request.c --- a/src/auth/auth-request.c Sat Apr 28 17:33:34 2012 +0300 +++ b/src/auth/auth-request.c Sat Apr 28 17:41:14 2012 +0300 @@ -237,6 +237,8 @@ request->local_port = atoi(value); else if (strcmp(key, "rport") == 0) request->remote_port = atoi(value); + else if (strcmp(key, "session") == 0) + request->session_id = p_strdup(request->pool, value); else return FALSE; return TRUE; @@ -1856,6 +1858,8 @@ } if (auth_request->requested_login_user != NULL) str_append(str, ",master"); + if (auth_request->session_id != NULL) + str_printfa(str, ",<%s>", auth_request->session_id); str_append(str, "): "); } diff -r 3b4c336ed01b -r 629afda8e29d src/auth/auth-request.h --- a/src/auth/auth-request.h Sat Apr 28 17:33:34 2012 +0300 +++ b/src/auth/auth-request.h Sat Apr 28 17:41:14 2012 +0300 @@ -71,7 +71,7 @@ unsigned int id; time_t last_access; - const char *service, *mech_name; + const char *service, *mech_name, *session_id; struct ip_addr local_ip, remote_ip; unsigned int local_port, remote_port; diff -r 3b4c336ed01b -r 629afda8e29d src/lib-auth/auth-client-request.c --- a/src/lib-auth/auth-client-request.c Sat Apr 28 17:33:34 2012 +0300 +++ b/src/lib-auth/auth-client-request.c Sat Apr 28 17:41:14 2012 +0300 @@ -44,6 +44,10 @@ if ((info->flags & AUTH_REQUEST_FLAG_VALID_CLIENT_CERT) != 0) str_append(str, "\tvalid-client-cert"); + if (info->session_id != NULL) { + str_append(str, "\tsession="); + str_tabescape_write(str, info->session_id); + } if (info->cert_username != NULL) { str_append(str, "\tcert_username="); str_tabescape_write(str, info->cert_username); @@ -82,6 +86,8 @@ request->request_info = *request_info; request->request_info.mech = p_strdup(pool, request_info->mech); request->request_info.service = p_strdup(pool, request_info->service); + request->request_info.session_id = + p_strdup_empty(pool, request_info->session_id); request->request_info.cert_username = p_strdup_empty(pool, request_info->cert_username); request->request_info.initial_resp_base64 = diff -r 3b4c336ed01b -r 629afda8e29d src/lib-auth/auth-client.h --- a/src/lib-auth/auth-client.h Sat Apr 28 17:33:34 2012 +0300 +++ b/src/lib-auth/auth-client.h Sat Apr 28 17:41:14 2012 +0300 @@ -37,6 +37,7 @@ struct auth_request_info { const char *mech; const char *service; + const char *session_id; const char *cert_username; enum auth_request_flags flags; diff -r 3b4c336ed01b -r 629afda8e29d src/login-common/login-settings.c --- a/src/login-common/login-settings.c Sat Apr 28 17:33:34 2012 +0300 +++ b/src/login-common/login-settings.c Sat Apr 28 17:41:14 2012 +0300 @@ -56,7 +56,7 @@ static const struct login_settings login_default_settings = { .login_trusted_networks = "", .login_greeting = PACKAGE_NAME" ready.", - .login_log_format_elements = "user=<%u> method=%m rip=%r lip=%l mpid=%e %c", + .login_log_format_elements = "user=<%u> method=%m rip=%r lip=%l mpid=%e %c session=<%{session}>", .login_log_format = "%$: %s", .login_access_sockets = "", .director_username_hash = "%u", diff -r 3b4c336ed01b -r 629afda8e29d src/login-common/sasl-server.c --- a/src/login-common/sasl-server.c Sat Apr 28 17:33:34 2012 +0300 +++ b/src/login-common/sasl-server.c Sat Apr 28 17:41:14 2012 +0300 @@ -318,6 +318,7 @@ memset(&info, 0, sizeof(info)); info.mech = mech->name; info.service = service; + info.session_id = client_get_session_id(client); info.cert_username = client->ssl_proxy == NULL ? NULL : ssl_proxy_get_peer_name(client->ssl_proxy); info.flags = client_get_auth_flags(client); From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: Code cleanup: Use array_count_i() to calculate arra... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/2f776ca73cef changeset: 14526:2f776ca73cef user: Timo Sirainen date: Sat Apr 28 18:24:24 2012 +0300 description: Code cleanup: Use array_count_i() to calculate array's size internally. diffstat: src/lib/aqueue.c | 6 ++---- src/lib/array.c | 6 +++--- src/lib/array.h | 20 ++++++++++---------- 3 files changed, 15 insertions(+), 17 deletions(-) diffs (104 lines): diff -r 629afda8e29d -r 2f776ca73cef src/lib/aqueue.c --- a/src/lib/aqueue.c Sat Apr 28 17:41:14 2012 +0300 +++ b/src/lib/aqueue.c Sat Apr 28 18:24:24 2012 +0300 @@ -10,8 +10,7 @@ aqueue = i_new(struct aqueue, 1); aqueue->arr = array; - aqueue->area_size = buffer_get_size(aqueue->arr->buffer) / - aqueue->arr->element_size; + aqueue->area_size = array_count_i(array); i_assert(aqueue->area_size > 0); return aqueue; } @@ -32,8 +31,7 @@ orig_area_size = aqueue->area_size; (void)array_append_space_i(aqueue->arr); - aqueue->area_size = buffer_get_size(aqueue->arr->buffer) / - aqueue->arr->element_size; + aqueue->area_size = array_count_i(aqueue->arr); i_assert(orig_area_size < aqueue->area_size); count = I_MIN(aqueue->area_size - orig_area_size, aqueue->head); diff -r 629afda8e29d -r 2f776ca73cef src/lib/array.c --- a/src/lib/array.c Sat Apr 28 17:41:14 2012 +0300 +++ b/src/lib/array.c Sat Apr 28 18:24:24 2012 +0300 @@ -71,7 +71,7 @@ void array_reverse_i(struct array *array) { - const unsigned int element_size = array->element_size; + const size_t element_size = array->element_size; unsigned int i, count = array_count_i(array); size_t size; void *data, *tmp; @@ -92,7 +92,7 @@ { unsigned int count; - count = array->buffer->used / array->element_size; + count = array_count_i(array); qsort(buffer_get_modifiable_data(array->buffer, NULL), count, array->element_size, cmp); } @@ -102,7 +102,7 @@ { unsigned int count; - count = array->buffer->used / array->element_size; + count = array_count_i(array); return bsearch(key, array->buffer->data, count, array->element_size, cmp); } diff -r 629afda8e29d -r 2f776ca73cef src/lib/array.h --- a/src/lib/array.h Sat Apr 28 17:41:14 2012 +0300 +++ b/src/lib/array.h Sat Apr 28 18:24:24 2012 +0300 @@ -134,6 +134,14 @@ #define array_clear(array) \ array_clear_i(&(array)->arr) +static inline unsigned int ATTR_PURE +array_count_i(const struct array *array) +{ + return array->buffer->used / array->element_size; +} +#define array_count(array) \ + array_count_i(&(array)->arr) + static inline void array_append_i(struct array *array, const void *data, unsigned int count) { @@ -177,7 +185,7 @@ static inline const void * array_get_i(const struct array *array, unsigned int *count_r) { - *count_r = array->buffer->used / array->element_size; + *count_r = array_count_i(array); return array->buffer->data; } #define array_get(array, count) \ @@ -195,7 +203,7 @@ static inline void * array_get_modifiable_i(struct array *array, unsigned int *count_r) { - *count_r = array->buffer->used / array->element_size; + *count_r = array_count_i(array); return buffer_get_modifiable_data(array->buffer, NULL); } #define array_get_modifiable(array, count) \ @@ -244,14 +252,6 @@ count * dest->element_size); } -static inline unsigned int ATTR_PURE -array_count_i(const struct array *array) -{ - return array->buffer->used / array->element_size; -} -#define array_count(array) \ - array_count_i(&(array)->arr) - bool array_cmp_i(const struct array *array1, const struct array *array2) ATTR_PURE; #define array_cmp(array1, array2) \ From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: seq_range_array_add(): Ranges weren't always merged... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/bac5e1dae4ed changeset: 14527:bac5e1dae4ed user: Timo Sirainen date: Mon Apr 30 22:53:19 2012 +0300 description: seq_range_array_add(): Ranges weren't always merged when they could have. This didn't break anything, but it used up more memory than necessary. diffstat: src/lib/seq-range-array.c | 22 +++++++++++++--------- src/lib/test-seq-range-array.c | 16 +++++++++++++++- 2 files changed, 28 insertions(+), 10 deletions(-) diffs (73 lines): diff -r 2f776ca73cef -r bac5e1dae4ed src/lib/seq-range-array.c --- a/src/lib/seq-range-array.c Sat Apr 28 18:24:24 2012 +0300 +++ b/src/lib/seq-range-array.c Mon Apr 30 22:53:19 2012 +0300 @@ -86,16 +86,20 @@ data[idx-1].seq2 = data[idx].seq2; array_delete(array, idx, 1); } - } else if (data[idx].seq2 == seq-1) { - i_assert(idx+1 < count); /* already handled above */ - data[idx].seq2 = seq; - if (data[idx+1].seq1 == seq+1) { - /* merge */ - data[idx+1].seq1 = data[idx].seq1; - array_delete(array, idx, 1); + } else { + if (idx > 0 && data[idx-1].seq2 == seq-1) + idx--; + if (data[idx].seq2 == seq-1) { + i_assert(idx+1 < count); /* already handled above */ + data[idx].seq2 = seq; + if (data[idx+1].seq1 == seq+1) { + /* merge */ + data[idx+1].seq1 = data[idx].seq1; + array_delete(array, idx, 1); + } + } else { + array_insert(array, idx, &value, 1); } - } else { - array_insert(array, idx, &value, 1); } return FALSE; } diff -r 2f776ca73cef -r bac5e1dae4ed src/lib/test-seq-range-array.c --- a/src/lib/test-seq-range-array.c Sat Apr 28 18:24:24 2012 +0300 +++ b/src/lib/test-seq-range-array.c Mon Apr 30 22:53:19 2012 +0300 @@ -6,6 +6,19 @@ #include +static void test_seq_range_array_add_merge(void) +{ + ARRAY_TYPE(seq_range) range; + + test_begin("seq_range_array_add() merging"); + t_array_init(&range, 8); + seq_range_array_add(&range, 0, 4); + seq_range_array_add(&range, 0, 1); + seq_range_array_add(&range, 0, 2); + test_assert(array_count(&range) == 2); + test_end(); +} + static void test_seq_range_array_random(void) { #define SEQ_RANGE_TEST_BUFSIZE 20 @@ -53,7 +66,7 @@ seqs = array_get(&range, &count); for (j = 0, seq1 = 0; j < count; j++) { - if (j > 0 && seqs[j-1].seq2 >= seqs[j].seq1) + if (j > 0 && seqs[j-1].seq2+1 >= seqs[j].seq1) goto fail; for (; seq1 < seqs[j].seq1; seq1++) { if (shadowbuf[seq1] != 0) @@ -156,6 +169,7 @@ void test_seq_range_array(void) { + test_seq_range_array_add_merge(); test_seq_range_array_invert(); test_seq_range_array_have_common(); test_seq_range_array_random(); From dovecot at dovecot.org Sun May 20 03:26:34 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:34 +0300 Subject: dovecot-2.2: Increased initial memory pool size. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/6c381b3e0a3d changeset: 14528:6c381b3e0a3d user: Timo Sirainen date: Mon Apr 30 23:04:08 2012 +0300 description: Increased initial memory pool size. diffstat: src/lib-storage/index/index-status.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r bac5e1dae4ed -r 6c381b3e0a3d src/lib-storage/index/index-status.c --- a/src/lib-storage/index/index-status.c Mon Apr 30 22:53:19 2012 +0300 +++ b/src/lib-storage/index/index-status.c Mon Apr 30 23:04:08 2012 +0300 @@ -102,7 +102,7 @@ if (box->metadata_pool == NULL) { box->metadata_pool = - pool_alloconly_create("mailbox metadata", 256); + pool_alloconly_create("mailbox metadata", 2048); } fields = mail_cache_register_get_list(box->cache, From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: lib-storage: shared/$user/INBOX is now visible only... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/3bb6d153c9c3 changeset: 14530:3bb6d153c9c3 user: Timo Sirainen date: Tue May 01 03:15:46 2012 +0300 description: lib-storage: shared/$user/INBOX is now visible only as shared/$user diffstat: src/lib-storage/mailbox-list.c | 12 ++++++++++++ 1 files changed, 12 insertions(+), 0 deletions(-) diffs (29 lines): diff -r 99a30e40dce9 -r 3bb6d153c9c3 src/lib-storage/mailbox-list.c --- a/src/lib-storage/mailbox-list.c Tue May 01 03:01:49 2012 +0300 +++ b/src/lib-storage/mailbox-list.c Tue May 01 03:15:46 2012 +0300 @@ -427,6 +427,12 @@ list_sep = mailbox_list_get_hierarchy_sep(list); ns_sep = mail_namespace_get_sep(ns); + if (*storage_name == '\0' && ns->type == NAMESPACE_SHARED && + (ns->flags & NAMESPACE_FLAG_INBOX_ANY) != 0) { + /* opening shared/$user. it's the same as INBOX. */ + storage_name = "INBOX"; + } + if (list_sep == ns_sep) return storage_name; if (ns->type == NAMESPACE_SHARED && @@ -501,6 +507,12 @@ and /inBox. */ return vname; } + if (strcmp(vname, "INBOX") == 0 && + (list->ns->flags & NAMESPACE_FLAG_INBOX_ANY) != 0) { + /* convert to shared/$user, we don't really care about the + INBOX suffix here. */ + vname = ""; + } if (*vname == '\0') { /* return namespace prefix without the separator */ if (list->ns->prefix_len == 0) From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: aqueue: Reversed recent changes, they didn't work a... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/99a30e40dce9 changeset: 14529:99a30e40dce9 user: Timo Sirainen date: Tue May 01 03:01:49 2012 +0300 description: aqueue: Reversed recent changes, they didn't work as intended. diffstat: src/lib/aqueue.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diffs (23 lines): diff -r 6c381b3e0a3d -r 99a30e40dce9 src/lib/aqueue.c --- a/src/lib/aqueue.c Mon Apr 30 23:04:08 2012 +0300 +++ b/src/lib/aqueue.c Tue May 01 03:01:49 2012 +0300 @@ -10,7 +10,8 @@ aqueue = i_new(struct aqueue, 1); aqueue->arr = array; - aqueue->area_size = array_count_i(array); + aqueue->area_size = buffer_get_size(aqueue->arr->buffer) / + aqueue->arr->element_size; i_assert(aqueue->area_size > 0); return aqueue; } @@ -31,7 +32,8 @@ orig_area_size = aqueue->area_size; (void)array_append_space_i(aqueue->arr); - aqueue->area_size = array_count_i(aqueue->arr); + aqueue->area_size = buffer_get_size(aqueue->arr->buffer) / + aqueue->arr->element_size; i_assert(orig_area_size < aqueue->area_size); count = I_MIN(aqueue->area_size - orig_area_size, aqueue->head); From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: lib-index: Fixed MAIL_INDEX_SYNC_FLAG_AVOID_FLAG_UP... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/5d45870e2e4a changeset: 14532:5d45870e2e4a user: Timo Sirainen date: Thu May 03 23:26:44 2012 +0300 description: lib-index: Fixed MAIL_INDEX_SYNC_FLAG_AVOID_FLAG_UPDATES with concurrent keyword changes. If session 1 saw keywords="a", session 2 added keyword "b" and session 1 (without the client syncing session) replaced flags with "a", the "b" keyword wasn't removed. diffstat: src/lib-index/mail-index-transaction-private.h | 4 ++++ src/lib-index/mail-index-transaction-update.c | 2 +- src/lib-index/mail-index-transaction.c | 17 +++++++++++++++++ src/lib-index/test-mail-index-transaction-update.c | 6 ++++++ 4 files changed, 28 insertions(+), 1 deletions(-) diffs (83 lines): diff -r d5bbb1d203c5 -r 5d45870e2e4a src/lib-index/mail-index-transaction-private.h --- a/src/lib-index/mail-index-transaction-private.h Fri Apr 27 16:47:30 2012 +0900 +++ b/src/lib-index/mail-index-transaction-private.h Thu May 03 23:26:44 2012 +0300 @@ -35,6 +35,7 @@ enum mail_index_transaction_flags flags; struct mail_index_transaction_vfuncs v; struct mail_index_view *view; + struct mail_index_view *latest_view; /* NOTE: If you add anything new, remember to update mail_index_transaction_reset_v() to reset it. */ @@ -121,6 +122,9 @@ unsigned int left_idx, unsigned int right_idx, uint32_t seq); +void mail_index_transaction_lookup_latest_keywords(struct mail_index_transaction *t, + uint32_t seq, + ARRAY_TYPE(keyword_indexes) *keywords); bool mail_index_cancel_flag_updates(struct mail_index_transaction *t, uint32_t seq); diff -r d5bbb1d203c5 -r 5d45870e2e4a src/lib-index/mail-index-transaction-update.c --- a/src/lib-index/mail-index-transaction-update.c Fri Apr 27 16:47:30 2012 +0900 +++ b/src/lib-index/mail-index-transaction-update.c Thu May 03 23:26:44 2012 +0300 @@ -977,7 +977,7 @@ t_array_init(&existing, 32); if (seq < t->first_new_seq) - mail_index_lookup_keywords(t->view, seq, &existing); + mail_index_transaction_lookup_latest_keywords(t, seq, &existing); existing_idx = array_get(&existing, &existing_count); if (modify_type == MODIFY_REPLACE && existing_count != keywords->count) diff -r d5bbb1d203c5 -r 5d45870e2e4a src/lib-index/mail-index-transaction.c --- a/src/lib-index/mail-index-transaction.c Fri Apr 27 16:47:30 2012 +0900 +++ b/src/lib-index/mail-index-transaction.c Thu May 03 23:26:44 2012 +0300 @@ -52,6 +52,8 @@ array_free(&t->module_contexts); mail_index_view_transaction_unref(t->view); + if (t->latest_view != NULL) + mail_index_view_close(&t->latest_view); mail_index_view_close(&t->view); i_free(t); } @@ -89,6 +91,21 @@ return next_uid; } +void mail_index_transaction_lookup_latest_keywords(struct mail_index_transaction *t, + uint32_t seq, + ARRAY_TYPE(keyword_indexes) *keywords) +{ + uint32_t uid, latest_seq; + + if (t->latest_view == NULL) { + (void)mail_index_refresh(t->view->index); + t->latest_view = mail_index_view_open(t->view->index); + } + mail_index_lookup_uid(t->view, seq, &uid); + if (mail_index_lookup_seq(t->view, uid, &latest_seq)) + mail_index_lookup_keywords(t->view, latest_seq, keywords); +} + static int mail_transaction_log_file_refresh(struct mail_index_transaction *t, struct mail_transaction_log_append_ctx *ctx) diff -r d5bbb1d203c5 -r 5d45870e2e4a src/lib-index/test-mail-index-transaction-update.c --- a/src/lib-index/test-mail-index-transaction-update.c Fri Apr 27 16:47:30 2012 +0900 +++ b/src/lib-index/test-mail-index-transaction-update.c Thu May 03 23:26:44 2012 +0300 @@ -43,6 +43,12 @@ return hdr.messages_count; } +void mail_index_transaction_lookup_latest_keywords(struct mail_index_transaction *t ATTR_UNUSED, + uint32_t seq ATTR_UNUSED, + ARRAY_TYPE(keyword_indexes) *keywords ATTR_UNUSED) +{ +} + static struct mail_index_transaction * mail_index_transaction_new(void) { From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: Fix the build failure on AIX Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d5bbb1d203c5 changeset: 14531:d5bbb1d203c5 user: SATOH Fumiyasu date: Fri Apr 27 16:47:30 2012 +0900 description: Fix the build failure on AIX diffstat: src/lib-storage/index/index-search-private.h | 2 ++ src/plugins/stats/stats-plugin.h | 2 ++ 2 files changed, 4 insertions(+), 0 deletions(-) diffs (24 lines): diff -r 3bb6d153c9c3 -r d5bbb1d203c5 src/lib-storage/index/index-search-private.h --- a/src/lib-storage/index/index-search-private.h Tue May 01 03:15:46 2012 +0300 +++ b/src/lib-storage/index/index-search-private.h Fri Apr 27 16:47:30 2012 +0900 @@ -3,6 +3,8 @@ #include "mail-storage-private.h" +#include + struct index_search_context { struct mail_search_context mail_ctx; struct mail_index_view *view; diff -r 3bb6d153c9c3 -r d5bbb1d203c5 src/plugins/stats/stats-plugin.h --- a/src/plugins/stats/stats-plugin.h Tue May 01 03:15:46 2012 +0300 +++ b/src/plugins/stats/stats-plugin.h Fri Apr 27 16:47:30 2012 +0900 @@ -6,6 +6,8 @@ #include "mail-user.h" #include "mail-storage-private.h" +#include + #define STATS_USER_CONTEXT(obj) \ MODULE_CONTEXT(obj, stats_user_module) From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: lib-storage: Don't set mail_save_context->saving=TR... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/5d9a5011c1bf changeset: 14533:5d9a5011c1bf user: Timo Sirainen date: Fri May 04 19:00:04 2012 +0300 description: lib-storage: Don't set mail_save_context->saving=TRUE when copying via save. diffstat: src/lib-storage/mail-storage.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (13 lines): diff -r 5d45870e2e4a -r 5d9a5011c1bf src/lib-storage/mail-storage.c --- a/src/lib-storage/mail-storage.c Thu May 03 23:26:44 2012 +0300 +++ b/src/lib-storage/mail-storage.c Fri May 04 19:00:04 2012 +0300 @@ -1630,7 +1630,8 @@ return -1; } - (*ctx)->saving = TRUE; + if (!(*ctx)->copying_via_save) + (*ctx)->saving = TRUE; if (box->v.save_begin == NULL) { mail_storage_set_error(box->storage, MAIL_ERROR_NOTPOSSIBLE, "Saving messages not supported"); From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: notify plugin: mailbox_save_using_mail() now sends ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/8a4d314ef7d0 changeset: 14534:8a4d314ef7d0 user: Timo Sirainen date: Fri May 04 19:01:19 2012 +0300 description: notify plugin: mailbox_save_using_mail() now sends "save" events, not "copy". diffstat: src/plugins/notify/notify-storage.c | 9 ++++++++- 1 files changed, 8 insertions(+), 1 deletions(-) diffs (20 lines): diff -r 5d9a5011c1bf -r 8a4d314ef7d0 src/plugins/notify/notify-storage.c --- a/src/plugins/notify/notify-storage.c Fri May 04 19:00:04 2012 +0300 +++ b/src/plugins/notify/notify-storage.c Fri May 04 19:01:19 2012 +0300 @@ -102,8 +102,15 @@ ctx->dest_mail = lt->tmp_mail; } - if ((ret = lbox->super.copy(ctx, mail)) == 0) + if ((ret = lbox->super.copy(ctx, mail)) < 0) + return -1; + + if (ctx->saving) { + /* we came from mailbox_save_using_mail() */ + notify_contexts_mail_save(ctx->dest_mail); + } else { notify_contexts_mail_copy(mail, ctx->dest_mail); + } return ret; } From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: mail-log: Removed broken "are we saving?" check fro... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/b89b98ebe52b changeset: 14535:b89b98ebe52b user: Timo Sirainen date: Fri May 04 19:01:46 2012 +0300 description: mail-log: Removed broken "are we saving?" check from copy event handling. diffstat: src/plugins/mail-log/mail-log-plugin.c | 16 +++++----------- 1 files changed, 5 insertions(+), 11 deletions(-) diffs (26 lines): diff -r 8a4d314ef7d0 -r b89b98ebe52b src/plugins/mail-log/mail-log-plugin.c --- a/src/plugins/mail-log/mail-log-plugin.c Fri May 04 19:01:19 2012 +0300 +++ b/src/plugins/mail-log/mail-log-plugin.c Fri May 04 19:01:46 2012 +0300 @@ -360,17 +360,11 @@ (struct mail_log_mail_txn_context *)txn; const char *desc; - if (dst->saving) { - /* we came from mailbox_save_using_mail() */ - mail_log_append_mail_message(ctx, dst, - MAIL_LOG_EVENT_SAVE, "save"); - } else { - desc = t_strdup_printf("copy from %s", - str_sanitize(mailbox_get_name(src->box), - MAILBOX_NAME_LOG_LEN)); - mail_log_append_mail_message(ctx, dst, - MAIL_LOG_EVENT_COPY, desc); - } + desc = t_strdup_printf("copy from %s", + str_sanitize(mailbox_get_name(src->box), + MAILBOX_NAME_LOG_LEN)); + mail_log_append_mail_message(ctx, dst, + MAIL_LOG_EVENT_COPY, desc); } static void mail_log_mail_expunge(void *txn, struct mail *mail) From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: doveadm mailbox mutf7: If input data is invalid, ex... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/3689eced9381 changeset: 14536:3689eced9381 user: Timo Sirainen date: Fri May 04 20:07:05 2012 +0300 description: doveadm mailbox mutf7: If input data is invalid, exit with EX_DATAERR. diffstat: src/doveadm/doveadm-mutf7.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (17 lines): diff -r b89b98ebe52b -r 3689eced9381 src/doveadm/doveadm-mutf7.c --- a/src/doveadm/doveadm-mutf7.c Fri May 04 19:01:46 2012 +0300 +++ b/src/doveadm/doveadm-mutf7.c Fri May 04 20:07:05 2012 +0300 @@ -40,11 +40,13 @@ if (imap_utf8_to_utf7(argv[i], str) < 0) { i_error("Mailbox name not valid UTF-8: %s", argv[i]); + doveadm_exit_code = EX_DATAERR; } } else { if (imap_utf7_to_utf8(argv[i], str) < 0) { i_error("Mailbox name not valid mUTF-7: %s", argv[i]); + doveadm_exit_code = EX_DATAERR; } } printf("%s\n", str_c(str)); From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: dbox: Metadata read buffer's size was supposed to b... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/fa6662ab4df3 changeset: 14537:fa6662ab4df3 user: Timo Sirainen date: Fri May 04 20:14:20 2012 +0300 description: dbox: Metadata read buffer's size was supposed to be unlimited, not zero. This was broken by 59f5238c6e61, which itself had fixed another bug. diffstat: src/lib-storage/index/dbox-common/dbox-file.c | 5 +++-- src/lib-storage/index/dbox-multi/mdbox-purge.c | 6 ++++-- 2 files changed, 7 insertions(+), 4 deletions(-) diffs (45 lines): diff -r 3689eced9381 -r fa6662ab4df3 src/lib-storage/index/dbox-common/dbox-file.c --- a/src/lib-storage/index/dbox-common/dbox-file.c Fri May 04 20:07:05 2012 +0300 +++ b/src/lib-storage/index/dbox-common/dbox-file.c Fri May 04 20:14:20 2012 +0300 @@ -407,7 +407,7 @@ /* skip over the actual metadata */ buf_size = i_stream_get_max_buffer_size(file->input); - i_stream_set_max_buffer_size(file->input, 0); + i_stream_set_max_buffer_size(file->input, (size_t)-1); while ((line = i_stream_read_next_line(file->input)) != NULL) { if (*line == DBOX_METADATA_OLDV1_SPACE || *line == '\0') { /* end of metadata */ @@ -657,7 +657,8 @@ ret = 0; buf_size = i_stream_get_max_buffer_size(file->input); - i_stream_set_max_buffer_size(file->input, 0); + /* use unlimited line length for metadata */ + i_stream_set_max_buffer_size(file->input, (size_t)-1); while ((line = i_stream_read_next_line(file->input)) != NULL) { if (*line == DBOX_METADATA_OLDV1_SPACE || *line == '\0') { /* end of metadata */ diff -r 3689eced9381 -r fa6662ab4df3 src/lib-storage/index/dbox-multi/mdbox-purge.c --- a/src/lib-storage/index/dbox-multi/mdbox-purge.c Fri May 04 20:07:05 2012 +0300 +++ b/src/lib-storage/index/dbox-multi/mdbox-purge.c Fri May 04 20:14:20 2012 +0300 @@ -107,7 +107,8 @@ o_stream_send(output, &meta_hdr, sizeof(meta_hdr)); buf_size = i_stream_get_max_buffer_size(file->input); - i_stream_set_max_buffer_size(file->input, 0); + /* use unlimited line length for metadata */ + i_stream_set_max_buffer_size(file->input, (size_t)-1); while ((line = i_stream_read_next_line(file->input)) != NULL) { if (*line == '\0') { /* end of metadata */ @@ -140,7 +141,8 @@ return ret; buf_size = i_stream_get_max_buffer_size(file->input); - i_stream_set_max_buffer_size(file->input, 0); + /* use unlimited line length for metadata */ + i_stream_set_max_buffer_size(file->input, (size_t)-1); while ((line = i_stream_read_next_line(file->input)) != NULL) { if (*line == '\0') { /* end of metadata */ From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: Added mail_shared_explicit_inbox setting. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/40a544fc4778 changeset: 14538:40a544fc4778 user: Timo Sirainen date: Fri May 04 21:40:57 2012 +0300 description: Added mail_shared_explicit_inbox setting. diffstat: doc/example-config/conf.d/10-mail.conf | 2 ++ src/lib-storage/mail-storage-settings.c | 2 ++ src/lib-storage/mail-storage-settings.h | 1 + src/lib-storage/mailbox-list.c | 6 ++++-- 4 files changed, 9 insertions(+), 2 deletions(-) diffs (65 lines): diff -r fa6662ab4df3 -r 40a544fc4778 doc/example-config/conf.d/10-mail.conf --- a/doc/example-config/conf.d/10-mail.conf Fri May 04 20:14:20 2012 +0300 +++ b/doc/example-config/conf.d/10-mail.conf Fri May 04 21:40:57 2012 +0300 @@ -97,6 +97,8 @@ # List the shared/ namespace only if there are visible shared mailboxes. #list = children #} +# Should shared INBOX be visible as "shared/user" or "shared/user/INBOX"? +#mail_shared_explicit_inbox = yes # System user and group used to access mails. If you use multiple, userdb # can override these by returning uid or gid fields. You can use either numbers diff -r fa6662ab4df3 -r 40a544fc4778 src/lib-storage/mail-storage-settings.c --- a/src/lib-storage/mail-storage-settings.c Fri May 04 20:14:20 2012 +0300 +++ b/src/lib-storage/mail-storage-settings.c Fri May 04 21:40:57 2012 +0300 @@ -48,6 +48,7 @@ DEF(SET_BOOL, mail_debug), DEF(SET_BOOL, mail_full_filesystem_access), DEF(SET_BOOL, maildir_stat_dirs), + DEF(SET_BOOL, mail_shared_explicit_inbox), DEF(SET_ENUM, lock_method), DEF(SET_STR, pop3_uidl_format), @@ -78,6 +79,7 @@ .mail_debug = FALSE, .mail_full_filesystem_access = FALSE, .maildir_stat_dirs = FALSE, + .mail_shared_explicit_inbox = TRUE, .lock_method = "fcntl:flock:dotlock", .pop3_uidl_format = "%08Xu%08Xv" }; diff -r fa6662ab4df3 -r 40a544fc4778 src/lib-storage/mail-storage-settings.h --- a/src/lib-storage/mail-storage-settings.h Fri May 04 20:14:20 2012 +0300 +++ b/src/lib-storage/mail-storage-settings.h Fri May 04 21:40:57 2012 +0300 @@ -33,6 +33,7 @@ bool mail_debug; bool mail_full_filesystem_access; bool maildir_stat_dirs; + bool mail_shared_explicit_inbox; const char *lock_method; const char *pop3_uidl_format; diff -r fa6662ab4df3 -r 40a544fc4778 src/lib-storage/mailbox-list.c --- a/src/lib-storage/mailbox-list.c Fri May 04 20:14:20 2012 +0300 +++ b/src/lib-storage/mailbox-list.c Fri May 04 21:40:57 2012 +0300 @@ -428,7 +428,8 @@ ns_sep = mail_namespace_get_sep(ns); if (*storage_name == '\0' && ns->type == NAMESPACE_SHARED && - (ns->flags & NAMESPACE_FLAG_INBOX_ANY) != 0) { + (ns->flags & NAMESPACE_FLAG_INBOX_ANY) != 0 && + !list->mail_set->mail_shared_explicit_inbox) { /* opening shared/$user. it's the same as INBOX. */ storage_name = "INBOX"; } @@ -508,7 +509,8 @@ return vname; } if (strcmp(vname, "INBOX") == 0 && - (list->ns->flags & NAMESPACE_FLAG_INBOX_ANY) != 0) { + (list->ns->flags & NAMESPACE_FLAG_INBOX_ANY) != 0 && + !list->mail_set->mail_shared_explicit_inbox) { /* convert to shared/$user, we don't really care about the INBOX suffix here. */ vname = ""; From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: systemd: If a socket is enabled in systemd but not ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/4a3bf567da54 changeset: 14539:4a3bf567da54 user: Timo Sirainen date: Fri May 04 21:52:05 2012 +0300 description: systemd: If a socket is enabled in systemd but not in Dovecot config, close it. Based on patch by Michal Hlavinka. diffstat: src/master/service-listen.c | 90 +++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 90 insertions(+), 0 deletions(-) diffs (114 lines): diff -r 40a544fc4778 -r 4a3bf567da54 src/master/service-listen.c --- a/src/master/service-listen.c Fri May 04 21:40:57 2012 +0300 +++ b/src/master/service-listen.c Fri May 04 21:52:05 2012 +0300 @@ -14,6 +14,7 @@ #include #include #include +#include #define MIN_BACKLOG 4 #define MAX_BACKLOG 511 @@ -242,6 +243,90 @@ return ret; } +#ifdef HAVE_SYSTEMD +static int get_socket_info(int fd, unsigned int *family, unsigned int *port) +{ + union sockaddr_union { + struct sockaddr sa; + struct sockaddr_in in4; + struct sockaddr_in6 in6; + } sockaddr; + socklen_t l; + + if (port) *port = -1; + if (family) *family = -1; + + memset(&sockaddr, 0, sizeof(sockaddr)); + l = sizeof(sockaddr); + + if (getsockname(fd, &sockaddr.sa, &l) < 0) + return -errno; + + if (family) *family = sockaddr.sa.sa_family; + if (port) { + if (sockaddr.sa.sa_family == AF_INET) { + if (l < sizeof(struct sockaddr_in)) + return -EINVAL; + + *port = ntohs(sockaddr.in4.sin_port); + } else { + if (l < sizeof(struct sockaddr_in6)) + return -EINVAL; + + *port = ntohs(sockaddr.in6.sin6_port); + } + } + return 0; +} + +static int services_verify_systemd(struct service_list *service_list) +{ + struct service *const *services; + static int sd_fds = -1; + int fd, fd_max; + + if (sd_fds < 0) { + sd_fds = sd_listen_fds(0); + if (sd_fds == -1) { + i_error("sd_listen_fds() failed: %m"); + return -1; + } + } + + fd_max = SD_LISTEN_FDS_START + sd_fds - 1; + for (fd = SD_LISTEN_FDS_START; fd <= fd_max; fd++) { + if (sd_is_socket_inet(fd, 0, SOCK_STREAM, 1, 0) > 0) { + int found = FALSE; + unsigned int port, family; + get_socket_info(fd, &family, &port); + + array_foreach(&service_list->services, services) { + struct service_listener *const *listeners; + + array_foreach(&(*services)->listeners, listeners) { + struct service_listener *l = *listeners; + if (l->type != SERVICE_LISTENER_INET) + continue; + if (l->set.inetset.set->port == port && + l->set.inetset.ip.family == family) { + found = TRUE; + break; + } + } + if (found) break; + } + if (!found) { + i_error("systemd listens on port %d, but it's not configured in Dovecot. Closing.",port); + if (shutdown(fd, SHUT_RDWR) < 0 && errno != ENOTCONN) + i_error("shutdown() failed: %m"); + if (dup2(null_fd, fd) < 0) + i_error("dup2() failed: %m"); + } + } + } +} +#endif + int services_listen(struct service_list *service_list) { struct service *const *services; @@ -252,6 +337,11 @@ if (ret2 < ret) ret = ret2; } + +#ifdef HAVE_SYSTEMD + if (ret == 0) + services_verify_systemd(service_list); +#endif return ret; } From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: shared mailboxes: If successful userdb lookup doesn... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/6dabd9545d48 changeset: 14540:6dabd9545d48 user: Timo Sirainen date: Fri May 04 22:55:29 2012 +0300 description: shared mailboxes: If successful userdb lookup doesn't return home, fallback to mail_home setting for %%h. diffstat: src/lib-storage/index/shared/shared-storage.c | 14 +++--- src/lib-storage/mail-user.c | 57 ++++++++++++++++++--------- 2 files changed, 45 insertions(+), 26 deletions(-) diffs (139 lines): diff -r 4a3bf567da54 -r 6dabd9545d48 src/lib-storage/index/shared/shared-storage.c --- a/src/lib-storage/index/shared/shared-storage.c Fri May 04 21:52:05 2012 +0300 +++ b/src/lib-storage/index/shared/shared-storage.c Fri May 04 22:55:29 2012 +0300 @@ -256,6 +256,13 @@ owner = mail_user_alloc(userdomain, user->set_info, user->unexpanded_set); owner->autocreated = TRUE; + if (mail_user_init(owner, &error) < 0) { + mailbox_list_set_critical(list, + "Couldn't create namespace '%s' for user %s: %s", + ns->prefix, userdomain, error); + mail_user_unref(&owner); + return -1; + } if (!var_has_key(storage->location, 'h', "home")) ret = 1; else { @@ -268,13 +275,6 @@ return -1; } } - if (mail_user_init(owner, &error) < 0) { - mailbox_list_set_critical(list, - "Couldn't create namespace '%s' for user %s: %s", - ns->prefix, userdomain, error); - mail_user_unref(&owner); - return -1; - } /* create the new namespace */ new_ns = i_new(struct mail_namespace, 1); diff -r 4a3bf567da54 -r 6dabd9545d48 src/lib-storage/mail-user.c --- a/src/lib-storage/mail-user.c Fri May 04 21:52:05 2012 +0300 +++ b/src/lib-storage/mail-user.c Fri May 04 22:55:29 2012 +0300 @@ -96,19 +96,23 @@ { const struct mail_storage_settings *mail_set; const char *home, *key, *value; + bool need_home_dir; - if (user->_home == NULL && - settings_vars_have_key(user->set_info, user->set, - 'h', "home", &key, &value) && - mail_user_get_home(user, &home) <= 0) { + need_home_dir = user->_home == NULL && + settings_vars_have_key(user->set_info, user->set, + 'h', "home", &key, &value); + + /* expand mail_home setting before calling mail_user_get_home() */ + settings_var_expand(user->set_info, user->set, + user->pool, mail_user_var_expand_table(user)); + + if (need_home_dir && mail_user_get_home(user, &home) <= 0) { *error_r = t_strdup_printf( "userdb didn't return a home directory, " "but %s used it (%%h): %s", key, value); return -1; } - settings_var_expand(user->set_info, user->set, - user->pool, mail_user_var_expand_table(user)); if (mail_user_expand_plugins_envs(user, error_r) < 0) return -1; @@ -265,7 +269,7 @@ return path; } -int mail_user_get_home(struct mail_user *user, const char **home_r) +static int mail_user_userdb_lookup_home(struct mail_user *user) { struct auth_user_info info; struct auth_user_reply reply; @@ -273,6 +277,8 @@ const char *username, *const *fields; int ret; + i_assert(!user->home_looked_up); + memset(&info, 0, sizeof(info)); info.service = user->service; if (user->local_ip != NULL) @@ -280,12 +286,6 @@ if (user->remote_ip != NULL) info.remote_ip = *user->remote_ip; - if (user->home_looked_up) { - *home_r = user->_home; - return user->_home != NULL ? 1 : 0; - } - *home_r = NULL; - if (mail_user_auth_master_conn == NULL) return 0; @@ -293,18 +293,37 @@ ret = auth_master_user_lookup(mail_user_auth_master_conn, user->username, &info, userdb_pool, &username, &fields); - if (ret >= 0) { + if (ret > 0) { auth_user_fields_parse(fields, userdb_pool, &reply); - user->_home = ret == 0 ? NULL : - p_strdup(user->pool, reply.home); - user->home_looked_up = TRUE; - ret = user->_home != NULL ? 1 : 0; - *home_r = user->_home; + user->_home = p_strdup(user->pool, reply.home); } pool_unref(&userdb_pool); return ret; } +int mail_user_get_home(struct mail_user *user, const char **home_r) +{ + int ret; + + if (user->home_looked_up) { + *home_r = user->_home; + return user->_home != NULL ? 1 : 0; + } + + ret = mail_user_userdb_lookup_home(user); + if (ret < 0) + return -1; + + if (ret > 0 && user->_home == NULL && *user->set->mail_home != '\0') { + /* no home in userdb, fallback to mail_home setting */ + user->_home = user->set->mail_home; + } + user->home_looked_up = TRUE; + + *home_r = user->_home; + return user->_home != NULL ? 1 : 0; +} + bool mail_user_is_plugin_loaded(struct mail_user *user, struct module *module) { const char *const *plugins; From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: mail-log: Memory leak fix Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/2a42fe2d42f3 changeset: 14541:2a42fe2d42f3 user: Timo Sirainen date: Fri May 04 23:50:22 2012 +0300 description: mail-log: Memory leak fix diffstat: src/plugins/mail-log/mail-log-plugin.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff -r 6dabd9545d48 -r 2a42fe2d42f3 src/plugins/mail-log/mail-log-plugin.c --- a/src/plugins/mail-log/mail-log-plugin.c Fri May 04 22:55:29 2012 +0300 +++ b/src/plugins/mail-log/mail-log-plugin.c Fri May 04 23:50:22 2012 +0300 @@ -228,6 +228,8 @@ wanted_fields |= MAIL_FETCH_VIRTUAL_SIZE; mail_add_temp_wanted_fields(mail, wanted_fields, wanted_headers); + if (wanted_headers != NULL) + mailbox_header_lookup_unref(&wanted_headers); } static void From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: acl: Memory leak fixes Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/5df8bac9a0c6 changeset: 14542:5df8bac9a0c6 user: Timo Sirainen date: Fri May 04 23:56:51 2012 +0300 description: acl: Memory leak fixes diffstat: src/plugins/acl/acl-backend-vfile-acllist.c | 2 +- src/plugins/acl/acl-cache.c | 4 ++++ 2 files changed, 5 insertions(+), 1 deletions(-) diffs (26 lines): diff -r 2a42fe2d42f3 -r 5df8bac9a0c6 src/plugins/acl/acl-backend-vfile-acllist.c --- a/src/plugins/acl/acl-backend-vfile-acllist.c Fri May 04 23:50:22 2012 +0300 +++ b/src/plugins/acl/acl-backend-vfile-acllist.c Fri May 04 23:56:51 2012 +0300 @@ -81,7 +81,7 @@ path = acl_list_get_path(backend); if (path == NULL) { /* we're never going to build acllist for this namespace. */ - i_array_init(&backend->acllist, 1); + acllist_clear(backend, 0); return 0; } diff -r 2a42fe2d42f3 -r 5df8bac9a0c6 src/plugins/acl/acl-cache.c --- a/src/plugins/acl/acl-cache.c Fri May 04 23:50:22 2012 +0300 +++ b/src/plugins/acl/acl-cache.c Fri May 04 23:56:51 2012 +0300 @@ -75,6 +75,10 @@ if (obj_cache->my_current_rights != NULL && obj_cache->my_current_rights != &negative_cache_entry) acl_cache_mask_deinit(&obj_cache->my_current_rights); + if (obj_cache->my_rights != NULL) + acl_cache_mask_deinit(&obj_cache->my_rights); + if (obj_cache->my_neg_rights != NULL) + acl_cache_mask_deinit(&obj_cache->my_neg_rights); i_free(obj_cache->name); i_free(obj_cache); } From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: auth: Added %{session} variable for passdb/userdb q... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/3d8a25a4394d changeset: 14543:3d8a25a4394d user: Timo Sirainen date: Sat May 05 00:01:32 2012 +0300 description: auth: Added %{session} variable for passdb/userdb queries. diffstat: src/auth/auth-request.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (19 lines): diff -r 5df8bac9a0c6 -r 3d8a25a4394d src/auth/auth-request.c --- a/src/auth/auth-request.c Fri May 04 23:56:51 2012 +0300 +++ b/src/auth/auth-request.c Sat May 05 00:01:32 2012 +0300 @@ -1773,6 +1773,7 @@ { '\0', NULL, "login_user" }, { '\0', NULL, "login_username" }, { '\0', NULL, "login_domain" }, + { '\0', NULL, "session" }, { '\0', NULL, NULL } }; @@ -1832,6 +1833,7 @@ auth_request); } } + tab[18].value = escape_func(auth_request->session_id, auth_request); return tab; } From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: auth: Previous change caused crashing with userdb l... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/9da556b9a902 changeset: 14544:9da556b9a902 user: Timo Sirainen date: Sun May 06 06:44:04 2012 +0300 description: auth: Previous change caused crashing with userdb lookups diffstat: src/auth/auth-request.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (13 lines): diff -r 3d8a25a4394d -r 9da556b9a902 src/auth/auth-request.c --- a/src/auth/auth-request.c Sat May 05 00:01:32 2012 +0300 +++ b/src/auth/auth-request.c Sun May 06 06:44:04 2012 +0300 @@ -1833,7 +1833,8 @@ auth_request); } } - tab[18].value = escape_func(auth_request->session_id, auth_request); + tab[18].value = auth_request->session_id == NULL ? NULL : + escape_func(auth_request->session_id, auth_request); return tab; } From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: Makefile: Remove dovecot-config on make uninstall Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/f7bb8d105710 changeset: 14545:f7bb8d105710 user: Timo Sirainen date: Mon May 07 04:06:49 2012 +0300 description: Makefile: Remove dovecot-config on make uninstall diffstat: Makefile.am | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diffs (13 lines): diff -r 9da556b9a902 -r f7bb8d105710 Makefile.am --- a/Makefile.am Sun May 06 06:44:04 2012 +0300 +++ b/Makefile.am Mon May 07 04:06:49 2012 +0300 @@ -70,6 +70,9 @@ -e "s|^\(LIBDOVECOT_INCLUDE\)=.*$$|\1=-I$(pkgincludedir)|" \ > $(DESTDIR)$(pkglibdir)/dovecot-config +uninstall-hook: + rm $(DESTDIR)$(pkglibdir)/dovecot-config + CLEANFILES = $(datafiles) if HAVE_SYSTEMD CLEANFILES += $systedmsystemunit_DATA From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: Released v2.1.6. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/7c249e2a82a9 changeset: 14546:7c249e2a82a9 user: Timo Sirainen date: Mon May 07 08:36:43 2012 +0300 description: Released v2.1.6. diffstat: NEWS | 18 ++++++++++++++++++ configure.in | 2 +- 2 files changed, 19 insertions(+), 1 deletions(-) diffs (35 lines): diff -r f7bb8d105710 -r 7c249e2a82a9 NEWS --- a/NEWS Mon May 07 04:06:49 2012 +0300 +++ b/NEWS Mon May 07 08:36:43 2012 +0300 @@ -1,3 +1,21 @@ +v2.1.6 2012-05-07 Timo Sirainen + + * Session ID is now included by default in auth and login process + log lines. It can be added to mail processes also by adding + %{session} to mail_log_prefix. + + + Added ssl_require_crl setting, which specifies if CRL check must + be successful when verifying client certificates. + + Added mail_shared_explicit_inbox setting to specify if a shared INBOX + should be accessible as "shared/$user" or "shared/$user/INBOX". + - v2.1.5: Using "~/" as mail_location or elsewhere failed to actually + expand it to home directory. + - dbox: Fixed potential assert-crash when reading dbox files. + - trash plugin: Fixed behavior when quota is already over limit. + - mail_log plugin: Logging "copy" event didn't work. + - Proxying to backend server with SSL: Verifying server certificate + name always failed, because it was compared to an IP address. + v2.1.5 2012-04-23 Timo Sirainen * IMAP: When neither the session nor the mailbox has modseq tracking diff -r f7bb8d105710 -r 7c249e2a82a9 configure.in --- a/configure.in Mon May 07 04:06:49 2012 +0300 +++ b/configure.in Mon May 07 08:36:43 2012 +0300 @@ -1,5 +1,5 @@ AC_PREREQ([2.59]) -AC_INIT([Dovecot],[2.1.5],[dovecot at dovecot.org]) +AC_INIT([Dovecot],[2.1.6],[dovecot at dovecot.org]) AC_CONFIG_SRCDIR([src]) AM_INIT_AUTOMAKE([foreign]) From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: Added tag 2.1.6 for changeset 7c249e2a82a9 Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/4fc570461e88 changeset: 14547:4fc570461e88 user: Timo Sirainen date: Mon May 07 08:36:43 2012 +0300 description: Added tag 2.1.6 for changeset 7c249e2a82a9 diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r 7c249e2a82a9 -r 4fc570461e88 .hgtags --- a/.hgtags Mon May 07 08:36:43 2012 +0300 +++ b/.hgtags Mon May 07 08:36:43 2012 +0300 @@ -82,3 +82,4 @@ b9adfd52cb66d5d89d291b76b9845d6361216d12 2.1.3 2c21c940e19d97a772128a7f281cea302e157d73 2.1.4 469cee314d9c54d2d7101ec9e38579fdc9610eaf 2.1.5 +7c249e2a82a9cd33ae15ead6443c3499e16da623 2.1.6 From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: acl: Fixed listing users who have only INBOX shared... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/20c9446e537e changeset: 14549:20c9446e537e user: Timo Sirainen date: Mon May 07 23:39:01 2012 +0300 description: acl: Fixed listing users who have only INBOX shared and mail_shared_explicit_inbox=no diffstat: src/plugins/acl/acl-shared-storage.c | 20 +++++++++++++++++++- 1 files changed, 19 insertions(+), 1 deletions(-) diffs (37 lines): diff -r d3acee47cee4 -r 20c9446e537e src/plugins/acl/acl-shared-storage.c --- a/src/plugins/acl/acl-shared-storage.c Mon May 07 08:36:54 2012 +0300 +++ b/src/plugins/acl/acl-shared-storage.c Mon May 07 23:39:01 2012 +0300 @@ -12,6 +12,24 @@ #define SHARED_NS_RETRY_SECS (60*60) +static bool acl_ns_prefix_exists(struct mail_namespace *ns) +{ + struct mailbox *box; + const char *vname; + enum mailbox_existence existence; + bool ret; + + if (ns->list->mail_set->mail_shared_explicit_inbox) + return FALSE; + + vname = t_strndup(ns->prefix, ns->prefix_len-1); + box = mailbox_alloc(ns->list, vname, 0); + ret = mailbox_exists(box, FALSE, &existence) == 0 && + existence == MAILBOX_EXISTENCE_SELECT; + mailbox_free(&box); + return ret; +} + static void acl_shared_namespace_add(struct mail_namespace *ns, struct mail_storage *storage, const char *userdomain) @@ -56,7 +74,7 @@ break; (void)mailbox_list_iter_deinit(&iter); - if (info == NULL) { + if (info == NULL && !acl_ns_prefix_exists(new_ns)) { /* no visible mailboxes, remove the namespace */ mail_namespace_destroy(new_ns); } From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: Added signature for changeset 7c249e2a82a9 Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d3acee47cee4 changeset: 14548:d3acee47cee4 user: Timo Sirainen date: Mon May 07 08:36:54 2012 +0300 description: Added signature for changeset 7c249e2a82a9 diffstat: .hgsigs | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r 4fc570461e88 -r d3acee47cee4 .hgsigs --- a/.hgsigs Mon May 07 08:36:43 2012 +0300 +++ b/.hgsigs Mon May 07 08:36:54 2012 +0300 @@ -45,3 +45,4 @@ b9adfd52cb66d5d89d291b76b9845d6361216d12 0 iEYEABECAAYFAk9jbugACgkQyUhSUUBVislSgwCgpo3f0bsSujItBum/M6js8SzF06YAmwftHlwaOstKeALdjLR5vtF2c5F7 2c21c940e19d97a772128a7f281cea302e157d73 0 iEYEABECAAYFAk+CtkYACgkQyUhSUUBVisknxgCfTJw2YPGJ17HbHRGmbwmCyLqepbMAn17j7IYzUfEU0xkXhCgwEAmk7XO4 469cee314d9c54d2d7101ec9e38579fdc9610eaf 0 iEYEABECAAYFAk+VWqkACgkQyUhSUUBVislnXACfVjPqMmPUvYtXQXwqff0h7N76mZUAn02lPeUCyuyr1TF9e1hGM/sKgmko +7c249e2a82a9cd33ae15ead6443c3499e16da623 0 iEYEABECAAYFAk+nX2sACgkQyUhSUUBVisn7uwCbBD3boxBOGEJ8OYsIJ57n5Cr09FAAoIvhxL6EHRB15AMOw4sPaALJ3/bB From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: lib-storage: Don't auto-drop namespaces with only I... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/41f2bcb43dad changeset: 14550:41f2bcb43dad user: Timo Sirainen date: Mon May 07 23:39:34 2012 +0300 description: lib-storage: Don't auto-drop namespaces with only INBOX shared and mail_shared_explicit_inbox=no diffstat: src/lib-storage/mail-storage.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diffs (14 lines): diff -r 20c9446e537e -r 41f2bcb43dad src/lib-storage/mail-storage.c --- a/src/lib-storage/mail-storage.c Mon May 07 23:39:01 2012 +0300 +++ b/src/lib-storage/mail-storage.c Mon May 07 23:39:34 2012 +0300 @@ -765,6 +765,10 @@ return 0; } + /* if this is a shared namespace with only INBOX and + mail_shared_explicit_inbox=no, we'll need to mark the namespace as + usable here since nothing else will. */ + box->list->ns->flags |= NAMESPACE_FLAG_USABLE; return 0; } From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: layout=fs: Fixes to mail_shared_explicit_inbox=no Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/8cdc7c13d6f2 changeset: 14551:8cdc7c13d6f2 user: Timo Sirainen date: Tue May 08 01:22:32 2012 +0300 description: layout=fs: Fixes to mail_shared_explicit_inbox=no diffstat: src/lib-storage/list/mailbox-list-fs-iter.c | 8 +++++++- 1 files changed, 7 insertions(+), 1 deletions(-) diffs (25 lines): diff -r 41f2bcb43dad -r 8cdc7c13d6f2 src/lib-storage/list/mailbox-list-fs-iter.c --- a/src/lib-storage/list/mailbox-list-fs-iter.c Mon May 07 23:39:34 2012 +0300 +++ b/src/lib-storage/list/mailbox-list-fs-iter.c Tue May 08 01:22:32 2012 +0300 @@ -123,7 +123,8 @@ return t_strconcat("/", fname, NULL); } else { /* child */ - return t_strconcat(dir->storage_name, "/", fname, NULL); + return *fname == '\0' ? dir->storage_name : + t_strconcat(dir->storage_name, "/", fname, NULL); } } @@ -403,6 +404,11 @@ we just want to see its contents (not the INBOX's children). */ root = ""; + } else if (*prefix_vname == '\0') { + /* we need to handle "" explicitly here, because getting + storage name with mail_shared_explicit_inbox=no + would return root=INBOX. */ + root = ""; } else { root = mailbox_list_get_storage_name(ctx->ctx.list, prefix_vname); From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: lib-storage: Extra check to mail_shared_explicit_in... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/17a635cbb63d changeset: 14552:17a635cbb63d user: Timo Sirainen date: Tue May 08 01:24:01 2012 +0300 description: lib-storage: Extra check to mail_shared_explicit_inbox=no diffstat: src/lib-storage/mailbox-list.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 8cdc7c13d6f2 -r 17a635cbb63d src/lib-storage/mailbox-list.c --- a/src/lib-storage/mailbox-list.c Tue May 08 01:22:32 2012 +0300 +++ b/src/lib-storage/mailbox-list.c Tue May 08 01:24:01 2012 +0300 @@ -508,7 +508,7 @@ and /inBox. */ return vname; } - if (strcmp(vname, "INBOX") == 0 && + if (strcmp(vname, "INBOX") == 0 && list->ns->type == NAMESPACE_SHARED && (list->ns->flags & NAMESPACE_FLAG_INBOX_ANY) != 0 && !list->mail_set->mail_shared_explicit_inbox) { /* convert to shared/$user, we don't really care about the From dovecot at dovecot.org Sun May 20 03:26:35 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:35 +0300 Subject: dovecot-2.2: Compiler warning fixes Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/cb80c575b00c changeset: 14553:cb80c575b00c user: Timo Sirainen date: Tue May 08 08:31:27 2012 +0300 description: Compiler warning fixes diffstat: src/auth/db-checkpassword.c | 14 +++--- src/auth/db-checkpassword.h | 4 +- src/auth/passdb-checkpassword.c | 8 +- src/auth/userdb-checkpassword.c | 4 +- src/lib-storage/mail-search-register-human.c | 18 ++++---- src/lib-storage/mail-search-register-imap.c | 60 ++++++++++++++-------------- src/lib/test-array.c | 2 +- src/lib/test-base64.c | 8 +- 8 files changed, 59 insertions(+), 59 deletions(-) diffs (truncated from 302 to 300 lines): diff -r 17a635cbb63d -r cb80c575b00c src/auth/db-checkpassword.c --- a/src/auth/db-checkpassword.c Tue May 08 01:24:01 2012 +0300 +++ b/src/auth/db-checkpassword.c Tue May 08 08:31:27 2012 +0300 @@ -29,7 +29,7 @@ char *auth_password; db_checkpassword_callback_t *callback; - void *context; + void (*request_callback)(); pid_t pid; int fd_out, fd_in; @@ -115,7 +115,7 @@ extra_fields = t_strsplit_tabescaped(str_c(request->input_buf)); request->callback(request->request, status, extra_fields, - request->context); + request->request_callback); checkpassword_request_free(&request); } @@ -448,7 +448,7 @@ struct auth_request *request, const char *auth_password, db_checkpassword_callback_t *callback, - void *context) + void (*request_callback)()) { struct chkpw_auth_request *chkpw_auth_request; unsigned int output_len; @@ -464,7 +464,7 @@ "Username+password combination too long (%u bytes)", output_len); callback(request, DB_CHECKPASSWORD_STATUS_FAILURE, - NULL, context); + NULL, request_callback); return; } @@ -477,7 +477,7 @@ (void)close(fd_in[1]); } callback(request, DB_CHECKPASSWORD_STATUS_INTERNAL_FAILURE, - NULL, context); + NULL, request_callback); return; } @@ -490,7 +490,7 @@ (void)close(fd_out[0]); (void)close(fd_out[1]); callback(request, DB_CHECKPASSWORD_STATUS_INTERNAL_FAILURE, - NULL, context); + NULL, request_callback); return; } @@ -523,7 +523,7 @@ chkpw_auth_request->output_len = output_len; chkpw_auth_request->input_buf = str_new(default_pool, 256); chkpw_auth_request->callback = callback; - chkpw_auth_request->context = context; + chkpw_auth_request->request_callback = request_callback; chkpw_auth_request->io_in = io_add(fd_in[0], IO_READ, checkpassword_child_input, diff -r 17a635cbb63d -r cb80c575b00c src/auth/db-checkpassword.h --- a/src/auth/db-checkpassword.h Tue May 08 01:24:01 2012 +0300 +++ b/src/auth/db-checkpassword.h Tue May 08 08:31:27 2012 +0300 @@ -13,7 +13,7 @@ typedef void db_checkpassword_callback_t(struct auth_request *request, enum db_checkpassword_status status, const char *const *extra_fields, - void *context); + void (*request_callback)()); struct db_checkpassword * db_checkpassword_init(const char *checkpassword_path, @@ -24,6 +24,6 @@ struct auth_request *request, const char *auth_password, db_checkpassword_callback_t *callback, - void *context); + void (*request_callback)()); #endif diff -r 17a635cbb63d -r cb80c575b00c src/auth/passdb-checkpassword.c --- a/src/auth/passdb-checkpassword.c Tue May 08 01:24:01 2012 +0300 +++ b/src/auth/passdb-checkpassword.c Tue May 08 08:31:27 2012 +0300 @@ -17,9 +17,9 @@ auth_checkpassword_callback(struct auth_request *request, enum db_checkpassword_status status, const char *const *extra_fields, - void *context) + void (*request_callback)()) { - verify_plain_callback_t *callback = context; + verify_plain_callback_t *callback = request_callback; const char *scheme, *crypted_pass = NULL; unsigned int i; @@ -71,9 +71,9 @@ credentials_checkpassword_callback(struct auth_request *request, enum db_checkpassword_status status, const char *const *extra_fields, - void *context) + void (*request_callback)()) { - lookup_credentials_callback_t *callback = context; + lookup_credentials_callback_t *callback = request_callback; const char *scheme, *crypted_pass = NULL; unsigned int i; diff -r 17a635cbb63d -r cb80c575b00c src/auth/userdb-checkpassword.c --- a/src/auth/userdb-checkpassword.c Tue May 08 01:24:01 2012 +0300 +++ b/src/auth/userdb-checkpassword.c Tue May 08 08:31:27 2012 +0300 @@ -16,9 +16,9 @@ userdb_checkpassword_callback(struct auth_request *request, enum db_checkpassword_status status, const char *const *extra_fields, - void *context) + void (*request_callback)()) { - userdb_callback_t *callback = context; + userdb_callback_t *callback = request_callback; unsigned int i; switch (status) { diff -r 17a635cbb63d -r cb80c575b00c src/lib-storage/mail-search-register-human.c --- a/src/lib-storage/mail-search-register-human.c Tue May 08 01:24:01 2012 +0300 +++ b/src/lib-storage/mail-search-register-human.c Tue May 08 08:31:27 2012 +0300 @@ -83,17 +83,17 @@ { \ return arg_new_human_date(ctx, _type, _date_type); \ } -CALLBACK_DATE(before, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_RECEIVED); -CALLBACK_DATE(on, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_RECEIVED); -CALLBACK_DATE(since, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_RECEIVED); +CALLBACK_DATE(before, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_RECEIVED) +CALLBACK_DATE(on, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_RECEIVED) +CALLBACK_DATE(since, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_RECEIVED) -CALLBACK_DATE(sentbefore, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_SENT); -CALLBACK_DATE(senton, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_SENT); -CALLBACK_DATE(sentsince, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_SENT); +CALLBACK_DATE(sentbefore, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_SENT) +CALLBACK_DATE(senton, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_SENT) +CALLBACK_DATE(sentsince, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_SENT) -CALLBACK_DATE(savedbefore, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_SAVED); -CALLBACK_DATE(savedon, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_SAVED); -CALLBACK_DATE(savedsince, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_SAVED); +CALLBACK_DATE(savedbefore, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_SAVED) +CALLBACK_DATE(savedon, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_SAVED) +CALLBACK_DATE(savedsince, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_SAVED) static struct mail_search_arg * arg_new_human_size(struct mail_search_build_context *ctx, diff -r 17a635cbb63d -r cb80c575b00c src/lib-storage/mail-search-register-imap.c --- a/src/lib-storage/mail-search-register-imap.c Tue May 08 01:24:01 2012 +0300 +++ b/src/lib-storage/mail-search-register-imap.c Tue May 08 08:31:27 2012 +0300 @@ -116,18 +116,18 @@ sarg->match_not = _not; \ return sarg; \ } -CALLBACK_FLAG(answered, MAIL_ANSWERED, FALSE); -CALLBACK_FLAG(unanswered, MAIL_ANSWERED, TRUE); -CALLBACK_FLAG(deleted, MAIL_DELETED, FALSE); -CALLBACK_FLAG(undeleted, MAIL_DELETED, TRUE); -CALLBACK_FLAG(draft, MAIL_DRAFT, FALSE); -CALLBACK_FLAG(undraft, MAIL_DRAFT, TRUE); -CALLBACK_FLAG(flagged, MAIL_FLAGGED, FALSE); -CALLBACK_FLAG(unflagged, MAIL_FLAGGED, TRUE); -CALLBACK_FLAG(seen, MAIL_SEEN, FALSE); -CALLBACK_FLAG(unseen, MAIL_SEEN, TRUE); -CALLBACK_FLAG(recent, MAIL_RECENT, FALSE); -CALLBACK_FLAG(old, MAIL_RECENT, TRUE); +CALLBACK_FLAG(answered, MAIL_ANSWERED, FALSE) +CALLBACK_FLAG(unanswered, MAIL_ANSWERED, TRUE) +CALLBACK_FLAG(deleted, MAIL_DELETED, FALSE) +CALLBACK_FLAG(undeleted, MAIL_DELETED, TRUE) +CALLBACK_FLAG(draft, MAIL_DRAFT, FALSE) +CALLBACK_FLAG(undraft, MAIL_DRAFT, TRUE) +CALLBACK_FLAG(flagged, MAIL_FLAGGED, FALSE) +CALLBACK_FLAG(unflagged, MAIL_FLAGGED, TRUE) +CALLBACK_FLAG(seen, MAIL_SEEN, FALSE) +CALLBACK_FLAG(unseen, MAIL_SEEN, TRUE) +CALLBACK_FLAG(recent, MAIL_RECENT, FALSE) +CALLBACK_FLAG(old, MAIL_RECENT, TRUE) static struct mail_search_arg * imap_search_new(struct mail_search_build_context *ctx) @@ -141,7 +141,7 @@ return sarg; } -CALLBACK_STR(keyword, SEARCH_KEYWORDS); +CALLBACK_STR(keyword, SEARCH_KEYWORDS) static struct mail_search_arg * imap_search_unkeyword(struct mail_search_build_context *ctx) @@ -179,17 +179,17 @@ { \ return arg_new_date(ctx, _type, _date_type); \ } -CALLBACK_DATE(before, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_RECEIVED); -CALLBACK_DATE(on, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_RECEIVED); -CALLBACK_DATE(since, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_RECEIVED); +CALLBACK_DATE(before, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_RECEIVED) +CALLBACK_DATE(on, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_RECEIVED) +CALLBACK_DATE(since, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_RECEIVED) -CALLBACK_DATE(sentbefore, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_SENT); -CALLBACK_DATE(senton, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_SENT); -CALLBACK_DATE(sentsince, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_SENT); +CALLBACK_DATE(sentbefore, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_SENT) +CALLBACK_DATE(senton, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_SENT) +CALLBACK_DATE(sentsince, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_SENT) -CALLBACK_DATE(x_savedbefore, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_SAVED); -CALLBACK_DATE(x_savedon, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_SAVED); -CALLBACK_DATE(x_savedsince, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_SAVED); +CALLBACK_DATE(x_savedbefore, SEARCH_BEFORE, MAIL_SEARCH_DATE_TYPE_SAVED) +CALLBACK_DATE(x_savedon, SEARCH_ON, MAIL_SEARCH_DATE_TYPE_SAVED) +CALLBACK_DATE(x_savedsince, SEARCH_SINCE, MAIL_SEARCH_DATE_TYPE_SAVED) static struct mail_search_arg * arg_new_size(struct mail_search_build_context *ctx, @@ -245,11 +245,11 @@ { \ return arg_new_header(ctx, _type, #_name); \ } -CALLBACK_HDR(bcc, SEARCH_HEADER_ADDRESS); -CALLBACK_HDR(cc, SEARCH_HEADER_ADDRESS); -CALLBACK_HDR(from, SEARCH_HEADER_ADDRESS); -CALLBACK_HDR(to, SEARCH_HEADER_ADDRESS); -CALLBACK_HDR(subject, SEARCH_HEADER_COMPRESS_LWSP); +CALLBACK_HDR(bcc, SEARCH_HEADER_ADDRESS) +CALLBACK_HDR(cc, SEARCH_HEADER_ADDRESS) +CALLBACK_HDR(from, SEARCH_HEADER_ADDRESS) +CALLBACK_HDR(to, SEARCH_HEADER_ADDRESS) +CALLBACK_HDR(subject, SEARCH_HEADER_COMPRESS_LWSP) static struct mail_search_arg * imap_search_header(struct mail_search_build_context *ctx) @@ -291,8 +291,8 @@ { \ return arg_new_body(ctx, _type); \ } -CALLBACK_BODY(body, SEARCH_BODY); -CALLBACK_BODY(text, SEARCH_TEXT); +CALLBACK_BODY(body, SEARCH_BODY) +CALLBACK_BODY(text, SEARCH_TEXT) static struct mail_search_arg * arg_new_interval(struct mail_search_build_context *ctx, @@ -478,7 +478,7 @@ return sarg; } -CALLBACK_STR(x_guid, SEARCH_GUID); +CALLBACK_STR(x_guid, SEARCH_GUID) static struct mail_search_arg * imap_search_x_mailbox(struct mail_search_build_context *ctx) diff -r 17a635cbb63d -r cb80c575b00c src/lib/test-array.c --- a/src/lib/test-array.c Tue May 08 01:24:01 2012 +0300 +++ b/src/lib/test-array.c Tue May 08 08:31:27 2012 +0300 @@ -33,7 +33,7 @@ static void test_array_reverse(void) { ARRAY_DEFINE(intarr, int); - int input[] = { -1234567890, -272585721, 2724859223U, 824725652 }; + int input[] = { -1234567890, -272585721, 272485922, 824725652 }; const int *output; unsigned int i, j; diff -r 17a635cbb63d -r cb80c575b00c src/lib/test-base64.c --- a/src/lib/test-base64.c Tue May 08 01:24:01 2012 +0300 +++ b/src/lib/test-base64.c Tue May 08 08:31:27 2012 +0300 @@ -48,9 +48,9 @@ "aGVs!!!!!" }; static const struct test_base64_decode_output output[] = { - { "hello world", 0, -1 }, - { "foo barits", 0, -1 }, - { "just niin", 1, -1 }, + { "hello world", 0, -1U }, + { "foo barits", 0, -1U }, + { "just niin", 1, -1U }, { "hel", 1, 4 }, { "hel", -1, 4 }, { "hel", -1, 4 } @@ -71,7 +71,7 @@ test_assert(output[i].ret == ret && strcmp(output[i].text, str_c(str)) == 0 && (src_pos == output[i].src_pos || - (output[i].src_pos == (unsigned int)-1 && + (output[i].src_pos == -1U && src_pos == strlen(input[i])))); From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: lib-storage: mailbox_list_get_storage_name/_get_vna... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/5a4eb9955d29 changeset: 14555:5a4eb9955d29 user: Timo Sirainen date: Wed May 09 04:41:47 2012 +0300 description: lib-storage: mailbox_list_get_storage_name/_get_vname() are now public. diffstat: src/lib-storage/mailbox-list-private.h | 3 --- src/lib-storage/mailbox-list.h | 4 ++++ 2 files changed, 4 insertions(+), 3 deletions(-) diffs (27 lines): diff -r 328a3cd837f8 -r 5a4eb9955d29 src/lib-storage/mailbox-list-private.h --- a/src/lib-storage/mailbox-list-private.h Tue May 08 08:49:47 2012 +0300 +++ b/src/lib-storage/mailbox-list-private.h Wed May 09 04:41:47 2012 +0300 @@ -180,9 +180,6 @@ const char *storage_name); const char *mailbox_list_get_unexpanded_path(struct mailbox_list *list, enum mailbox_list_path_type type); -const char *mailbox_list_get_storage_name(struct mailbox_list *list, - const char *vname); -const char *mailbox_list_get_vname(struct mailbox_list *list, const char *name); const char * mailbox_list_get_root_path(const struct mailbox_list_settings *set, enum mailbox_list_path_type type); diff -r 328a3cd837f8 -r 5a4eb9955d29 src/lib-storage/mailbox-list.h --- a/src/lib-storage/mailbox-list.h Tue May 08 08:49:47 2012 +0300 +++ b/src/lib-storage/mailbox-list.h Wed May 09 04:41:47 2012 +0300 @@ -230,6 +230,10 @@ bool mailbox_list_is_valid_create_name(struct mailbox_list *list, const char *name); +const char *mailbox_list_get_storage_name(struct mailbox_list *list, + const char *vname); +const char *mailbox_list_get_vname(struct mailbox_list *list, const char *name); + /* Return full path for the given mailbox name. The name must be a valid existing mailbox name, or NULL to get the root directory. For INDEX=MEMORY it returns "" as the path. */ From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: Compiler warning fixes Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/328a3cd837f8 changeset: 14554:328a3cd837f8 user: Timo Sirainen date: Tue May 08 08:49:47 2012 +0300 description: Compiler warning fixes diffstat: src/lib/test-bsearch-insert-pos.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diffs (16 lines): diff -r cb80c575b00c -r 328a3cd837f8 src/lib/test-bsearch-insert-pos.c --- a/src/lib/test-bsearch-insert-pos.c Tue May 08 08:31:27 2012 +0300 +++ b/src/lib/test-bsearch-insert-pos.c Tue May 08 08:49:47 2012 +0300 @@ -13,9 +13,9 @@ void test_bsearch_insert_pos(void) { static const unsigned int input[] = { - 1, 5, 9, 15, 16, -1, - 1, 5, 9, 15, 16, 17, -1, - -1 + 1, 5, 9, 15, 16, -1U, + 1, 5, 9, 15, 16, 17, -1U, + -1U }; static const unsigned int max_key = 18; const unsigned int *cur; From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: lib-storage: Added a mailbox.synced flag, which is ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/eab6ce7a5ad6 changeset: 14556:eab6ce7a5ad6 user: Timo Sirainen date: Wed May 09 06:11:58 2012 +0300 description: lib-storage: Added a mailbox.synced flag, which is set when mailbox sync has succeeded. diffstat: src/lib-storage/mail-storage-private.h | 2 ++ src/lib-storage/mail-storage.c | 2 ++ 2 files changed, 4 insertions(+), 0 deletions(-) diffs (24 lines): diff -r 5a4eb9955d29 -r eab6ce7a5ad6 src/lib-storage/mail-storage-private.h --- a/src/lib-storage/mail-storage-private.h Wed May 09 04:41:47 2012 +0300 +++ b/src/lib-storage/mail-storage-private.h Wed May 09 06:11:58 2012 +0300 @@ -279,6 +279,8 @@ unsigned int disable_reflink_copy_to:1; /* Don't allow creating any new keywords */ unsigned int disallow_new_keywords:1; + /* Mailbox has been synced at least once */ + unsigned int synced:1; }; struct mail_vfuncs { diff -r 5a4eb9955d29 -r eab6ce7a5ad6 src/lib-storage/mail-storage.c --- a/src/lib-storage/mail-storage.c Wed May 09 04:41:47 2012 +0300 +++ b/src/lib-storage/mail-storage.c Wed May 09 06:11:58 2012 +0300 @@ -1349,6 +1349,8 @@ i_error("Syncing INBOX failed: %s", errormsg); } } + if (ret == 0) + box->synced = TRUE; return ret; } From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: master: systemd socket verification wasn't actually... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/17b20880453c changeset: 14557:17b20880453c user: Timo Sirainen date: Wed May 09 15:01:20 2012 +0300 description: master: systemd socket verification wasn't actually being called normally. diffstat: src/master/service-listen.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r eab6ce7a5ad6 -r 17b20880453c src/master/service-listen.c --- a/src/master/service-listen.c Wed May 09 06:11:58 2012 +0300 +++ b/src/master/service-listen.c Wed May 09 15:01:20 2012 +0300 @@ -339,7 +339,7 @@ } #ifdef HAVE_SYSTEMD - if (ret == 0) + if (ret > 0) services_verify_systemd(service_list); #endif return ret; From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: imapc: Use imapc_list_prefix also for listing subsc... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/2f7406e838b0 changeset: 14558:2f7406e838b0 user: Timo Sirainen date: Thu May 10 11:26:53 2012 +0300 description: imapc: Use imapc_list_prefix also for listing subscriptions diffstat: src/lib-storage/index/imapc/imapc-list.c | 10 +++++++++- 1 files changed, 9 insertions(+), 1 deletions(-) diffs (27 lines): diff -r 17b20880453c -r 2f7406e838b0 src/lib-storage/index/imapc/imapc-list.c --- a/src/lib-storage/index/imapc/imapc-list.c Wed May 09 15:01:20 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-list.c Thu May 10 11:26:53 2012 +0300 @@ -518,6 +518,7 @@ (struct imapc_mailbox_list *)_src_list; struct imapc_simple_context ctx; struct imapc_command *cmd; + const char *pattern; char sep; i_assert(src_list->tmp_subscriptions == NULL); @@ -537,7 +538,14 @@ src_list->tmp_subscriptions = mailbox_tree_init(src_list->sep); cmd = imapc_list_simple_context_init(&ctx, src_list); - imapc_command_send(cmd, "LSUB \"\" *"); + if (*src_list->storage->set->imapc_list_prefix == '\0') + pattern = "*"; + else { + pattern = t_strdup_printf("%s%c*", + src_list->storage->set->imapc_list_prefix, + src_list->sep); + } + imapc_command_sendf(cmd, "LSUB \"\" %s", pattern); imapc_simple_run(&ctx); /* replace subscriptions tree in destination */ From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: lib-master: Added more default ignore prefixes Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/fac49069993a changeset: 14559:fac49069993a user: Timo Sirainen date: Mon May 14 18:09:10 2012 +0300 description: lib-master: Added more default ignore prefixes diffstat: src/lib-master/mountpoint-list.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff -r 2f7406e838b0 -r fac49069993a src/lib-master/mountpoint-list.c --- a/src/lib-master/mountpoint-list.c Thu May 10 11:26:53 2012 +0300 +++ b/src/lib-master/mountpoint-list.c Mon May 14 18:09:10 2012 +0300 @@ -53,6 +53,8 @@ const char *const mountpoint_list_default_ignore_prefixes[] = { "/cdrom", "/media", + "/sys", + "/proc", NULL }; From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: master: Fix to systemd support Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/20d98b848f37 changeset: 14560:20d98b848f37 user: Timo Sirainen date: Mon May 14 18:42:58 2012 +0300 description: master: Fix to systemd support Patch by Joop Boonen diffstat: src/master/service-listen.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff -r fac49069993a -r 20d98b848f37 src/master/service-listen.c --- a/src/master/service-listen.c Mon May 14 18:09:10 2012 +0300 +++ b/src/master/service-listen.c Mon May 14 18:42:58 2012 +0300 @@ -324,6 +324,7 @@ } } } + return 0; } #endif From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: maildir: If dovecot-uidlist can't be opened as read... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/7f238016db81 changeset: 14561:7f238016db81 user: Timo Sirainen date: Mon May 14 19:01:46 2012 +0300 description: maildir: If dovecot-uidlist can't be opened as read-write, try opening as read-only. diffstat: src/lib-storage/index/maildir/maildir-uidlist.c | 14 ++++++++++++++ 1 files changed, 14 insertions(+), 0 deletions(-) diffs (57 lines): diff -r 20d98b848f37 -r 7f238016db81 src/lib-storage/index/maildir/maildir-uidlist.c --- a/src/lib-storage/index/maildir/maildir-uidlist.c Mon May 14 18:42:58 2012 +0300 +++ b/src/lib-storage/index/maildir/maildir-uidlist.c Mon May 14 19:01:46 2012 +0300 @@ -97,6 +97,7 @@ unsigned int locked_refresh:1; unsigned int unsorted:1; unsigned int have_mailbox_guid:1; + unsigned int opened_readonly:1; }; struct maildir_uidlist_sync_ctx { @@ -702,11 +703,16 @@ struct stat st; uoff_t last_read_offset; int fd, ret; + bool readonly = FALSE; *retry_r = FALSE; if (uidlist->fd == -1) { fd = nfs_safe_open(uidlist->path, O_RDWR); + if (fd == -1 && errno == EACCES) { + fd = nfs_safe_open(uidlist->path, O_RDONLY); + readonly = TRUE; + } if (fd == -1) { if (errno != ENOENT) { mail_storage_set_critical(storage, @@ -807,6 +813,8 @@ (void)unlink(uidlist->path); } else if (ret > 0) { /* success */ + if (readonly) + uidlist->recreate_on_change = TRUE; uidlist->fd = fd; uidlist->fd_dev = st.st_dev; uidlist->fd_ino = st.st_ino; @@ -917,6 +925,10 @@ } uidlist->fd = nfs_safe_open(uidlist->path, O_RDWR); + if (uidlist->fd == -1 && errno == EACCES) { + uidlist->fd = nfs_safe_open(uidlist->path, O_RDONLY); + uidlist->recreate_on_change = TRUE; + } if (uidlist->fd == -1 && errno != ENOENT) { mail_storage_set_critical(uidlist->box->storage, "open(%s) failed: %m", uidlist->path); @@ -1510,6 +1522,8 @@ i_assert(uidlist->initial_hdr_read); if (maildir_uidlist_open_latest(uidlist) < 0) return -1; + if (uidlist->recreate_on_change) + return maildir_uidlist_recreate(uidlist); } i_assert(ctx->first_unwritten_pos != (unsigned int)-1); From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: ldap: If attributes contain ldapAttr=key=template%$... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/4bbc12a87a29 changeset: 14562:4bbc12a87a29 user: Timo Sirainen date: Mon May 14 19:30:03 2012 +0300 description: ldap: If attributes contain ldapAttr=key=template%$ and ldapAttr doesn't exist, skip the field. This makes it work again the same as with v2.0. diffstat: src/auth/db-ldap.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diffs (15 lines): diff -r 7f238016db81 -r 4bbc12a87a29 src/auth/db-ldap.c --- a/src/auth/db-ldap.c Mon May 14 19:01:46 2012 +0300 +++ b/src/auth/db-ldap.c Mon May 14 19:30:03 2012 +0300 @@ -1252,6 +1252,11 @@ /* use the LDAP attribute's value */ } else { /* template */ + if (values[0] == NULL && *field->ldap_attr_name != '\0') { + /* ldapAttr=key=template%$, but ldapAttr doesn't + exist. */ + return values; + } if (values[0] != NULL && values[1] != NULL) { auth_request_log_warning(ctx->auth_request, "ldap", "Multiple values found for '%s', " From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: pop3: Added pop3_uidl_duplicates setting. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/3a095892242b changeset: 14563:3a095892242b user: Timo Sirainen date: Mon May 14 21:07:43 2012 +0300 description: pop3: Added pop3_uidl_duplicates setting. diffstat: doc/example-config/conf.d/20-pop3.conf | 5 + src/pop3/pop3-client.c | 37 ++- src/pop3/pop3-client.h | 5 +- src/pop3/pop3-commands.c | 272 ++++++++++++++++++++++---------- src/pop3/pop3-settings.c | 4 +- src/pop3/pop3-settings.h | 1 + 6 files changed, 220 insertions(+), 104 deletions(-) diffs (truncated from 509 to 300 lines): diff -r 4bbc12a87a29 -r 3a095892242b doc/example-config/conf.d/20-pop3.conf --- a/doc/example-config/conf.d/20-pop3.conf Mon May 14 19:30:03 2012 +0300 +++ b/doc/example-config/conf.d/20-pop3.conf Mon May 14 21:07:43 2012 +0300 @@ -54,6 +54,11 @@ # won't change those UIDLs. Currently this works only with Maildir. #pop3_save_uidl = no + # What to do about duplicate UIDLs if they exist? + # allow: Show duplicates to clients. + # rename: Append a temporary -2, -3, etc. counter after the UIDL. + #pop3_uidl_duplicates = allow + # POP3 logout format string: # %i - total number of bytes read from client # %o - total number of bytes sent to client diff -r 4bbc12a87a29 -r 3a095892242b src/pop3/pop3-client.c --- a/src/pop3/pop3-client.c Mon May 14 19:30:03 2012 +0300 +++ b/src/pop3/pop3-client.c Mon May 14 21:07:43 2012 +0300 @@ -6,6 +6,7 @@ #include "network.h" #include "istream.h" #include "ostream.h" +#include "crc32.h" #include "str.h" #include "llist.h" #include "hostpid.h" @@ -344,15 +345,20 @@ return NULL; } - if (var_has_key(set->pop3_logout_format, 'u', "uidl_change") && - client->messages_count > 0) - client->message_uidl_hashes_save = TRUE; - client->uidl_keymask = parse_uidl_keymask(client->mail_set->pop3_uidl_format); if (client->uidl_keymask == 0) i_fatal("Invalid pop3_uidl_format"); + if (var_has_key(set->pop3_logout_format, 'u', "uidl_change")) { + /* logging uidl_change. we need hashes of the UIDLs */ + client->message_uidls_save = TRUE; + } else if (strcmp(set->pop3_uidl_duplicates, "allow") != 0) { + /* UIDL duplicates aren't allowed, so we'll need to + keep track of them */ + client->message_uidls_save = TRUE; + } + if (!set->pop3_no_flag_updates && client->messages_count > 0) client->seen_bitmask = i_malloc(MSGS_BITMASK_SIZE(client)); @@ -375,12 +381,8 @@ uint32_t i, old_hash, new_hash; unsigned int old_msg_count, new_msg_count; - if (client->message_uidl_hashes == NULL) { - /* UIDL command not given or %u not actually used in format */ - return ""; - } - if (client->message_uidl_hashes_save) { - /* UIDL command not finished */ + if (client->message_uidls == NULL) { + /* UIDL command not given */ return ""; } @@ -388,18 +390,18 @@ old_msg_count = client->lowest_retr_pop3_msn > 0 ? client->lowest_retr_pop3_msn - 1 : client->messages_count; for (i = 0, old_hash = 0; i < old_msg_count; i++) - old_hash ^= client->message_uidl_hashes[i]; + old_hash ^= crc32_str(client->message_uidls[i]); /* assume all except deleted messages were sent to POP3 client */ if (!client->deleted) { for (i = 0, new_hash = 0; i < client->messages_count; i++) - new_hash ^= client->message_uidl_hashes[i]; + new_hash ^= crc32_str(client->message_uidls[i]); } else { for (i = 0, new_hash = 0; i < client->messages_count; i++) { if (client->deleted_bitmask[i / CHAR_BIT] & (1 << (i % CHAR_BIT))) continue; - new_hash ^= client->message_uidl_hashes[i]; + new_hash ^= crc32_str(client->message_uidls[i]); } } @@ -444,7 +446,11 @@ tab[6].value = dec2str(client->total_size); tab[7].value = dec2str(client->input->v_offset); tab[8].value = dec2str(client->output->offset); - tab[9].value = client_build_uidl_change_string(client); + if (var_has_key(client->set->pop3_logout_format, + tab[9].key, tab[9].long_key)) + tab[9].value = client_build_uidl_change_string(client); + else + tab[9].value = ""; tab[10].value = client->session_id; str = t_str_new(128); @@ -498,8 +504,9 @@ } mail_user_unref(&client->user); + if (client->uidl_pool != NULL) + pool_unref(&client->uidl_pool); i_free(client->message_sizes); - i_free(client->message_uidl_hashes); i_free(client->deleted_bitmask); i_free(client->seen_bitmask); i_free(client->msgnum_to_seq_map); diff -r 4bbc12a87a29 -r 3a095892242b src/pop3/pop3-client.h --- a/src/pop3/pop3-client.h Mon May 14 19:30:03 2012 +0300 +++ b/src/pop3/pop3-client.h Mon May 14 21:07:43 2012 +0300 @@ -55,7 +55,7 @@ unsigned int retr_count; /* [msgnum] */ - uint32_t *message_uidl_hashes; + const char **message_uidls; uoff_t *message_sizes; /* [msgnum/8] & msgnum%8 */ unsigned char *deleted_bitmask; @@ -64,13 +64,14 @@ /* settings: */ const struct pop3_settings *set; const struct mail_storage_settings *mail_set; + pool_t uidl_pool; enum uidl_keys uidl_keymask; unsigned int disconnected:1; unsigned int deleted:1; unsigned int waiting_input:1; unsigned int anvil_sent:1; - unsigned int message_uidl_hashes_save:1; + unsigned int message_uidls_save:1; }; extern struct client *pop3_clients; diff -r 4bbc12a87a29 -r 3a095892242b src/pop3/pop3-commands.c --- a/src/pop3/pop3-commands.c Mon May 14 19:30:03 2012 +0300 +++ b/src/pop3/pop3-commands.c Mon May 14 21:07:43 2012 +0300 @@ -4,8 +4,8 @@ #include "array.h" #include "istream.h" #include "ostream.h" +#include "hash.h" #include "str.h" -#include "crc32.h" #include "var-expand.h" #include "message-size.h" #include "mail-storage.h" @@ -550,62 +550,9 @@ bool list_all; }; -static bool pop3_get_uid(struct client *client, struct cmd_uidl_context *ctx, - struct var_expand_table *tab, string_t *str) -{ - char uid_str[MAX_INT_STRLEN]; - const char *uidl; - - if (mail_get_special(ctx->mail, MAIL_FETCH_UIDL_BACKEND, &uidl) == 0 && - *uidl != '\0') { - str_append(str, uidl); - return TRUE; - } - - if (client->set->pop3_reuse_xuidl && - mail_get_first_header(ctx->mail, "X-UIDL", &uidl) > 0) { - str_append(str, uidl); - return FALSE; - } - - if ((client->uidl_keymask & UIDL_UID) != 0) { - i_snprintf(uid_str, sizeof(uid_str), "%u", - ctx->mail->uid); - tab[1].value = uid_str; - } - if ((client->uidl_keymask & UIDL_MD5) != 0) { - if (mail_get_special(ctx->mail, MAIL_FETCH_HEADER_MD5, - &tab[2].value) < 0 || - *tab[2].value == '\0') { - /* broken */ - i_fatal("UIDL: Header MD5 not found " - "(pop3_uidl_format=%%m not supported by storage?)"); - } - } - if ((client->uidl_keymask & UIDL_FILE_NAME) != 0) { - if (mail_get_special(ctx->mail, - MAIL_FETCH_UIDL_FILE_NAME, - &tab[3].value) < 0 || - *tab[3].value == '\0') { - /* broken */ - i_fatal("UIDL: File name not found " - "(pop3_uidl_format=%%f not supported by storage?)"); - } - } - if ((client->uidl_keymask & UIDL_GUID) != 0) { - if (mail_get_special(ctx->mail, MAIL_FETCH_GUID, - &tab[4].value) < 0 || - *tab[4].value == '\0') { - /* broken */ - i_fatal("UIDL: Message GUID not found " - "(pop3_uidl_format=%%g not supported by storage?)"); - } - } - var_expand(str, client->mail_set->pop3_uidl_format, tab); - return FALSE; -} - -static bool list_uids_iter(struct client *client, struct cmd_uidl_context *ctx) +static void +pop3_get_uid(struct client *client, struct mail *mail, string_t *str, + bool *permanent_uidl_r) { static struct var_expand_table static_tab[] = { { 'v', NULL, "uidvalidity" }, @@ -616,20 +563,108 @@ { '\0', NULL, NULL } }; struct var_expand_table *tab; - string_t *str; - int ret; - unsigned int uidl_pos; - bool save_hashes, found = FALSE; + char uid_str[MAX_INT_STRLEN]; + const char *uidl; + + if (mail_get_special(mail, MAIL_FETCH_UIDL_BACKEND, &uidl) == 0 && + *uidl != '\0') { + str_append(str, uidl); + /* UIDL is already permanent */ + *permanent_uidl_r = TRUE; + return; + } + + *permanent_uidl_r = FALSE; + + if (client->set->pop3_reuse_xuidl && + mail_get_first_header(mail, "X-UIDL", &uidl) > 0) { + str_append(str, uidl); + return; + } tab = t_malloc(sizeof(static_tab)); memcpy(tab, static_tab, sizeof(static_tab)); tab[0].value = t_strdup_printf("%u", client->uid_validity); - save_hashes = client->message_uidl_hashes_save && ctx->list_all; - if (save_hashes && client->message_uidl_hashes == NULL) { - client->message_uidl_hashes = - i_new(uint32_t, client->messages_count); + if ((client->uidl_keymask & UIDL_UID) != 0) { + i_snprintf(uid_str, sizeof(uid_str), "%u", + mail->uid); + tab[1].value = uid_str; } + if ((client->uidl_keymask & UIDL_MD5) != 0) { + if (mail_get_special(mail, MAIL_FETCH_HEADER_MD5, + &tab[2].value) < 0 || + *tab[2].value == '\0') { + /* broken */ + i_fatal("UIDL: Header MD5 not found " + "(pop3_uidl_format=%%m not supported by storage?)"); + } + } + if ((client->uidl_keymask & UIDL_FILE_NAME) != 0) { + if (mail_get_special(mail, MAIL_FETCH_UIDL_FILE_NAME, + &tab[3].value) < 0 || + *tab[3].value == '\0') { + /* broken */ + i_fatal("UIDL: File name not found " + "(pop3_uidl_format=%%f not supported by storage?)"); + } + } + if ((client->uidl_keymask & UIDL_GUID) != 0) { + if (mail_get_special(mail, MAIL_FETCH_GUID, + &tab[4].value) < 0 || + *tab[4].value == '\0') { + /* broken */ + i_fatal("UIDL: Message GUID not found " + "(pop3_uidl_format=%%g not supported by storage?)"); + } + } + var_expand(str, client->mail_set->pop3_uidl_format, tab); +} + +static bool +list_uidls_saved_iter(struct client *client, struct cmd_uidl_context *ctx) +{ + bool found = FALSE; + int ret; + + while (ctx->msgnum < client->messages_count) { + uint32_t msgnum = ctx->msgnum++; + + if (client->deleted) { + if (client->deleted_bitmask[msgnum / CHAR_BIT] & From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: mdbox: mailbox_update() didn't look at the latest i... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/7dd1dd742825 changeset: 14564:7dd1dd742825 user: Timo Sirainen date: Tue May 15 18:52:41 2012 +0300 description: mdbox: mailbox_update() didn't look at the latest index data. So setting min_next_uid, min_first_recent_uid or min_highest_modseq may have actually shrank them. diffstat: src/lib-storage/index/dbox-multi/mdbox-storage.c | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diffs (36 lines): diff -r 3a095892242b -r 7dd1dd742825 src/lib-storage/index/dbox-multi/mdbox-storage.c --- a/src/lib-storage/index/dbox-multi/mdbox-storage.c Mon May 14 21:07:43 2012 +0300 +++ b/src/lib-storage/index/dbox-multi/mdbox-storage.c Tue May 15 18:52:41 2012 +0300 @@ -249,6 +249,7 @@ { struct mdbox_mailbox *mbox = (struct mdbox_mailbox *)box; struct mail_index_transaction *new_trans = NULL; + struct mail_index_view *view; const struct mail_index_header *hdr; uint32_t uid_validity, uid_next; @@ -260,7 +261,8 @@ trans = new_trans; } - hdr = mail_index_get_header(box->view); + view = mail_index_view_open(box->index); + hdr = mail_index_get_header(view); uid_validity = hdr->uid_validity; if (update != NULL && update->uid_validity != 0) uid_validity = update->uid_validity; @@ -293,12 +295,12 @@ &first_recent_uid, sizeof(first_recent_uid), FALSE); } if (update != NULL && update->min_highest_modseq != 0 && - mail_index_modseq_get_highest(box->view) < - update->min_highest_modseq) { + mail_index_modseq_get_highest(view) < update->min_highest_modseq) { mail_index_modseq_enable(box->index); mail_index_update_highest_modseq(trans, update->min_highest_modseq); } + mail_index_view_close(&view); mdbox_update_header(mbox, trans, update); if (new_trans != NULL) { From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: auth: If user is disabled or password expired, tell... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d6f06ce44b0b changeset: 14565:d6f06ce44b0b user: Timo Sirainen date: Wed May 16 16:14:01 2012 +0300 description: auth: If user is disabled or password expired, tell about it to auth-client. diffstat: src/auth/auth-request-handler.c | 15 +++++++++++++++ src/auth/auth-request.c | 2 ++ src/auth/auth-request.h | 2 ++ 3 files changed, 19 insertions(+), 0 deletions(-) diffs (56 lines): diff -r 7dd1dd742825 -r d6f06ce44b0b src/auth/auth-request-handler.c --- a/src/auth/auth-request-handler.c Tue May 15 18:52:41 2012 +0300 +++ b/src/auth/auth-request-handler.c Wed May 16 16:14:01 2012 +0300 @@ -287,6 +287,21 @@ auth_stream_reply_add(reply, "nodelay", NULL); get_client_extra_fields(request, reply); + switch (request->passdb_result) { + case PASSDB_RESULT_INTERNAL_FAILURE: + case PASSDB_RESULT_SCHEME_NOT_AVAILABLE: + case PASSDB_RESULT_USER_UNKNOWN: + case PASSDB_RESULT_PASSWORD_MISMATCH: + case PASSDB_RESULT_OK: + break; + case PASSDB_RESULT_USER_DISABLED: + auth_stream_reply_add(reply, "user_disabled", NULL); + break; + case PASSDB_RESULT_PASS_EXPIRED: + auth_stream_reply_add(reply, "pass_expired", NULL); + break; + } + auth_request_handle_failure(request, reply); } diff -r 7dd1dd742825 -r d6f06ce44b0b src/auth/auth-request.c --- a/src/auth/auth-request.c Tue May 15 18:52:41 2012 +0300 +++ b/src/auth/auth-request.c Wed May 16 16:14:01 2012 +0300 @@ -557,6 +557,7 @@ request->private_callback.verify_plain); } else { auth_request_ref(request); + request->passdb_result = result; request->private_callback.verify_plain(result, request); auth_request_unref(&request); } @@ -691,6 +692,7 @@ but the user was unknown there */ result = PASSDB_RESULT_USER_UNKNOWN; } + request->passdb_result = result; request->private_callback. lookup_credentials(result, credentials, size, request); } diff -r 7dd1dd742825 -r d6f06ce44b0b src/auth/auth-request.h --- a/src/auth/auth-request.h Tue May 15 18:52:41 2012 +0300 +++ b/src/auth/auth-request.h Wed May 16 16:14:01 2012 +0300 @@ -56,6 +56,8 @@ struct auth_stream_reply *extra_cache_fields; /* the whole userdb result reply */ struct auth_stream_reply *userdb_reply; + /* Result of passdb lookup */ + enum passdb_result passdb_result; const struct mech_module *mech; const struct auth_settings *set; From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: sdbox: Avoid using too many fds when copying messages. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/6d11513cdd0d changeset: 14567:6d11513cdd0d user: Timo Sirainen date: Wed May 16 18:57:05 2012 +0300 description: sdbox: Avoid using too many fds when copying messages. This happened at least with quota plugin. diffstat: src/lib-storage/index/dbox-single/sdbox-save.c | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-) diffs (22 lines): diff -r ca0fef559d43 -r 6d11513cdd0d src/lib-storage/index/dbox-single/sdbox-save.c --- a/src/lib-storage/index/dbox-single/sdbox-save.c Wed May 16 16:14:31 2012 +0300 +++ b/src/lib-storage/index/dbox-single/sdbox-save.c Wed May 16 18:57:05 2012 +0300 @@ -78,9 +78,18 @@ void sdbox_save_add_file(struct mail_save_context *_ctx, struct dbox_file *file) { struct sdbox_save_context *ctx = (struct sdbox_save_context *)_ctx; + struct dbox_file *const *files; + unsigned int count; if (ctx->first_saved_seq == 0) ctx->first_saved_seq = ctx->ctx.seq; + + files = array_get(&ctx->files, &count); + if (count > 0) { + /* a plugin may leave a previously saved file open. + we'll close it here to avoid eating too many fds. */ + dbox_file_close(files[count-1]); + } array_append(&ctx->files, &file, 1); } From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: lib-storage: If chroot is used and mail_debug=yes, ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/a7dd1edee318 changeset: 14569:a7dd1edee318 user: Timo Sirainen date: Sat May 19 18:43:13 2012 +0300 description: lib-storage: If chroot is used and mail_debug=yes, log the chroot path. diffstat: src/lib-storage/mail-storage-service.c | 25 +++++++++++++++++++++++-- 1 files changed, 23 insertions(+), 2 deletions(-) diffs (49 lines): diff -r bb572accd1af -r a7dd1edee318 src/lib-storage/mail-storage-service.c --- a/src/lib-storage/mail-storage-service.c Thu May 17 18:08:58 2012 +0300 +++ b/src/lib-storage/mail-storage-service.c Sat May 19 18:43:13 2012 +0300 @@ -604,8 +604,13 @@ mail_set = mail_user_set_get_storage_set(mail_user); if (mail_set->mail_debug) { - i_debug("Effective uid=%s, gid=%s, home=%s", - dec2str(geteuid()), dec2str(getegid()), home); + string_t *str = t_str_new(64); + + str_printfa(str, "Effective uid=%s, gid=%s, home=%s", + dec2str(geteuid()), dec2str(getegid()), home); + if (*priv->chroot != '\0') + str_printfa(str, ", chroot=%s", priv->chroot); + i_debug("%s", str_c(str)); } if ((user->flags & MAIL_STORAGE_SERVICE_FLAG_TEMP_PRIV_DROP) != 0 && @@ -909,6 +914,21 @@ user_set->mail_plugins, &mod_set); } +static int extra_field_key_cmp_p(const char *const *s1, const char *const *s2) +{ + const char *p1 = *s1, *p2 = *s2; + + for (; *p1 == *p2; p1++, p2++) { + if (*p1 == '\0') + return 0; + } + if (*p1 == '=') + return -1; + if (*p2 == '=') + return 1; + return *p1 - *p2; +} + int mail_storage_service_lookup(struct mail_storage_service_ctx *ctx, const struct mail_storage_service_input *input, struct mail_storage_service_user **user_r, @@ -994,6 +1014,7 @@ if (userdb_fields != NULL) { auth_user_fields_parse(userdb_fields, temp_pool, &reply); + array_sort(&reply.extra_fields, extra_field_key_cmp_p); if (user_reply_handle(ctx, user, &reply, &error) < 0) { i_error("user %s: Invalid settings in userdb: %s", username, error); From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: login: If user is disabled or password expired, say... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/ca0fef559d43 changeset: 14566:ca0fef559d43 user: Timo Sirainen date: Wed May 16 16:14:31 2012 +0300 description: login: If user is disabled or password expired, say it in logout reason. diffstat: src/login-common/client-common-auth.c | 4 ++++ src/login-common/client-common.c | 4 ++++ src/login-common/client-common.h | 2 ++ 3 files changed, 10 insertions(+), 0 deletions(-) diffs (40 lines): diff -r d6f06ce44b0b -r ca0fef559d43 src/login-common/client-common-auth.c --- a/src/login-common/client-common-auth.c Wed May 16 16:14:01 2012 +0300 +++ b/src/login-common/client-common-auth.c Wed May 16 16:14:31 2012 +0300 @@ -82,6 +82,10 @@ reply_r->temp = TRUE; else if (strcmp(key, "authz") == 0) reply_r->authz_failure = TRUE; + else if (strcmp(key, "user_disabled") == 0) + client->auth_user_disabled = TRUE; + else if (strcmp(key, "pass_expired") == 0) + client->auth_pass_expired = TRUE; else if (strcmp(key, "reason") == 0) reply_r->reason = value; else if (strcmp(key, "host") == 0) diff -r d6f06ce44b0b -r ca0fef559d43 src/login-common/client-common.c --- a/src/login-common/client-common.c Wed May 16 16:14:01 2012 +0300 +++ b/src/login-common/client-common.c Wed May 16 16:14:31 2012 +0300 @@ -645,6 +645,10 @@ return t_strdup_printf("(internal failure, %u succesful auths)", client->auth_successes); } + if (client->auth_user_disabled) + return "(user disabled)"; + if (client->auth_pass_expired) + return "(password expired)"; return t_strdup_printf("(auth failed, %u attempts in %u secs)", client->auth_attempts, auth_secs); } diff -r d6f06ce44b0b -r ca0fef559d43 src/login-common/client-common.h --- a/src/login-common/client-common.h Wed May 16 16:14:01 2012 +0300 +++ b/src/login-common/client-common.h Wed May 16 16:14:31 2012 +0300 @@ -139,6 +139,8 @@ unsigned int auth_process_comm_fail:1; unsigned int proxy_auth_failed:1; unsigned int auth_waiting:1; + unsigned int auth_user_disabled:1; + unsigned int auth_pass_expired:1; /* ... */ }; From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: update-version.sh: Use mv -f to avoid questions in ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/bb572accd1af changeset: 14568:bb572accd1af user: Timo Sirainen date: Thu May 17 18:08:58 2012 +0300 description: update-version.sh: Use mv -f to avoid questions in some systems. diffstat: update-version.sh | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (9 lines): diff -r 6d11513cdd0d -r bb572accd1af update-version.sh --- a/update-version.sh Wed May 16 18:57:05 2012 +0300 +++ b/update-version.sh Thu May 17 18:08:58 2012 +0300 @@ -64,4 +64,4 @@ cmp -s "${BUILDDIR}/${VERSION_H}" "${BUILDDIR}/${VERSION_HT}" && \ rm -f "${BUILDDIR}/${VERSION_HT}" || \ - mv "${BUILDDIR}/${VERSION_HT}" "${BUILDDIR}/${VERSION_H}" + mv -f "${BUILDDIR}/${VERSION_HT}" "${BUILDDIR}/${VERSION_H}" From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: director: Don't crash with quickly disconnecting in... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/40f958c7643b changeset: 14570:40f958c7643b user: Timo Sirainen date: Sat May 19 21:16:42 2012 +0300 description: director: Don't crash with quickly disconnecting incoming director connections. diffstat: src/director/director-connection.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (13 lines): diff -r a7dd1edee318 -r 40f958c7643b src/director/director-connection.c --- a/src/director/director-connection.c Sat May 19 18:43:13 2012 +0300 +++ b/src/director/director-connection.c Sat May 19 21:16:42 2012 +0300 @@ -1503,7 +1503,8 @@ struct director_connection *conn = *_conn; struct director *dir = conn->dir; - if (conn->created + DIRECTOR_SUCCESS_MIN_CONNECT_SECS > ioloop_time) { + if (conn->created + DIRECTOR_SUCCESS_MIN_CONNECT_SECS > ioloop_time && + conn->host != NULL) { /* connection didn't exist for very long, assume it has a network problem */ conn->host->last_network_failure = ioloop_time; From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: director: Implemented ability to remove directors f... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/42cca8a1d179 changeset: 14571:42cca8a1d179 user: Timo Sirainen date: Sat May 19 21:18:04 2012 +0300 description: director: Implemented ability to remove directors from a running ring. Also added doveadm command for adding a new director to a running ring. diffstat: src/director/director-connection.c | 44 ++++++++++++-- src/director/director-host.c | 35 +++++++++++- src/director/director-host.h | 9 ++- src/director/director.c | 110 ++++++++++++++++++++++++++++++++++-- src/director/director.h | 15 ++++- src/director/doveadm-connection.c | 60 +++++++++++++++++++- src/director/main.c | 5 +- src/doveadm/doveadm-director.c | 67 ++++++++++++++++++++++ 8 files changed, 326 insertions(+), 19 deletions(-) diffs (truncated from 614 to 300 lines): diff -r 40f958c7643b -r 42cca8a1d179 src/director/director-connection.c --- a/src/director/director-connection.c Sat May 19 21:16:42 2012 +0300 +++ b/src/director/director-connection.c Sat May 19 21:18:04 2012 +0300 @@ -383,7 +383,12 @@ elsewhere with CONNECT. however, before disconnecting it verify first that our left side is actually still functional. */ + i_assert(conn->host == NULL); conn->host = director_host_get(dir, &ip, port); + /* the host shouldn't be removed at this point, but if for some + reason it is we don't want to crash */ + conn->host->removed = FALSE; + director_host_ref(conn->host); /* make sure we don't keep old sequence values across restarts */ conn->host->last_seq = 0; @@ -587,6 +592,10 @@ /* ignore updates to ourself */ return TRUE; } + if (host->removed) { + /* ignore re-adds of removed directors */ + return TRUE; + } /* already have this. just reset its last_network_failure timestamp, since it might be up now. */ @@ -598,18 +607,32 @@ } } else { /* save the director and forward it */ - director_host_add(conn->dir, &ip, port); + host = director_host_add(conn->dir, &ip, port); forward = TRUE; } if (forward) { - director_update_send(conn->dir, - director_connection_get_host(conn), - t_strdup_printf("DIRECTOR\t%s\t%u\n", - net_ip2addr(&ip), port)); + director_notify_ring_added(host, + director_connection_get_host(conn)); } return TRUE; } +static bool director_cmd_director_remove(struct director_connection *conn, + const char *const *args) +{ + struct director_host *host; + struct ip_addr ip; + unsigned int port; + + if (!director_args_parse_ip_port(conn, args, &ip, &port)) + return FALSE; + + host = director_host_lookup(conn->dir, &ip, port); + if (host != NULL && !host->removed) + director_ring_remove(host, director_connection_get_host(conn)); + return TRUE; +} + static bool director_cmd_host_hand_start(struct director_connection *conn, const char *const *args) @@ -659,7 +682,7 @@ *_args = args + 3; host = director_host_lookup(conn->dir, &ip, port); - if (host == NULL) { + if (host == NULL || host->removed) { /* director is already gone, but we can't be sure if this command was sent everywhere. re-send it as if it was from ourself. */ @@ -1191,6 +1214,8 @@ return director_cmd_user_killed_everywhere(conn, args); if (strcmp(cmd, "DIRECTOR") == 0) return director_cmd_director(conn, args); + if (strcmp(cmd, "DIRECTOR-REMOVE") == 0) + return director_cmd_director_remove(conn, args); if (strcmp(cmd, "SYNC") == 0) return director_connection_sync(conn, args); if (strcmp(cmd, "CONNECT") == 0) @@ -1279,6 +1304,8 @@ struct director_host *const *hostp; array_foreach(&conn->dir->dir_hosts, hostp) { + if ((*hostp)->removed) + continue; str_printfa(str, "DIRECTOR\t%s\t%u\n", net_ip2addr(&(*hostp)->ip), (*hostp)->port); } @@ -1433,12 +1460,15 @@ { struct director_connection *conn; + i_assert(!host->removed); + /* make sure we don't keep old sequence values across restarts */ host->last_seq = 0; conn = director_connection_init_common(dir, fd); conn->name = i_strdup_printf("%s/out", host->name); conn->host = host; + director_host_ref(host); conn->io = io_add(conn->fd, IO_WRITE, director_connection_connected, conn); return conn; @@ -1471,6 +1501,8 @@ } if (dir->right == conn) dir->right = NULL; + if (conn->host != NULL) + director_host_unref(conn->host); if (conn->user_iter != NULL) user_directory_iter_deinit(&conn->user_iter); diff -r 40f958c7643b -r 42cca8a1d179 src/director/director-host.c --- a/src/director/director-host.c Sat May 19 21:16:42 2012 +0300 +++ b/src/director/director-host.c Sat May 19 21:18:04 2012 +0300 @@ -29,6 +29,8 @@ struct director_host *host; host = i_new(struct director_host, 1); + host->dir = dir; + host->refcount = 1; host->ip = *ip; host->port = port; host->name = i_strdup_printf("%s:%u", net_ip2addr(ip), port); @@ -41,8 +43,39 @@ return host; } -void director_host_free(struct director_host *host) +void director_host_free(struct director_host **_host) { + struct director_host *host = *_host; + + i_assert(host->refcount == 1); + + *_host = NULL; + director_host_unref(host); +} + +void director_host_ref(struct director_host *host) +{ + i_assert(host->refcount > 0); + host->refcount++; +} + +void director_host_unref(struct director_host *host) +{ + struct director_host *const *hosts; + unsigned int i, count; + + i_assert(host->refcount > 0); + + if (--host->refcount > 0) + return; + + hosts = array_get(&host->dir->dir_hosts, &count); + for (i = 0; i < count; i++) { + if (hosts[i] == host) { + array_delete(&host->dir->dir_hosts, i, 1); + break; + } + } i_free(host->name); i_free(host); } diff -r 40f958c7643b -r 42cca8a1d179 src/director/director-host.h --- a/src/director/director-host.h Sat May 19 21:16:42 2012 +0300 +++ b/src/director/director-host.h Sat May 19 21:18:04 2012 +0300 @@ -6,6 +6,9 @@ struct director; struct director_host { + struct director *dir; + int refcount; + struct ip_addr ip; unsigned int port; @@ -22,12 +25,16 @@ time_t last_protocol_failure; /* we are this director */ unsigned int self:1; + unsigned int removed:1; }; struct director_host * director_host_add(struct director *dir, const struct ip_addr *ip, unsigned int port); -void director_host_free(struct director_host *host); +void director_host_free(struct director_host **host); + +void director_host_ref(struct director_host *host); +void director_host_unref(struct director_host *host); struct director_host * director_host_get(struct director *dir, const struct ip_addr *ip, diff -r 40f958c7643b -r 42cca8a1d179 src/director/director.c --- a/src/director/director.c Sat May 19 21:16:42 2012 +0300 +++ b/src/director/director.c Sat May 19 21:18:04 2012 +0300 @@ -20,6 +20,7 @@ #define DIRECTOR_SYNC_TIMEOUT_MSECS (5*1000) #define DIRECTOR_RING_MIN_WAIT_SECS 20 #define DIRECTOR_QUICK_RECONNECT_TIMEOUT_MSECS 1000 +#define DIRECTOR_DELAYED_DIR_REMOVE_MSECS (1000*30) static bool director_is_self_ip_set(struct director *dir) { @@ -127,8 +128,8 @@ static struct director_host * director_get_preferred_right_host(struct director *dir) { - struct director_host *const *hosts; - unsigned int count, self_idx; + struct director_host *const *hosts, *host; + unsigned int i, count, self_idx; hosts = array_get(&dir->dir_hosts, &count); if (count == 1) { @@ -137,7 +138,13 @@ } self_idx = director_find_self_idx(dir); - return hosts[(self_idx + 1) % count]; + for (i = 0; i < count; i++) { + host = hosts[(self_idx + i + 1) % count]; + if (!host->removed) + return host; + } + /* self, with some removed hosts */ + return NULL; } static bool director_wait_for_others(struct director *dir) @@ -177,6 +184,9 @@ for (i = 1; i < count; i++) { unsigned int idx = (self_idx + i) % count; + if (hosts[idx]->removed) + continue; + if (hosts[idx]->last_network_failure + DIRECTOR_RECONNECT_RETRY_SECS > ioloop_time) { /* connection failed recently, don't try retrying here */ @@ -408,6 +418,79 @@ director_connection_uncork(*connp); } +void director_notify_ring_added(struct director_host *added_host, + struct director_host *src) +{ + const char *cmd; + + cmd = t_strdup_printf("DIRECTOR\t%s\t%u\n", + net_ip2addr(&added_host->ip), added_host->port); + director_update_send(added_host->dir, src, cmd); +} + +static void director_delayed_dir_remove_timeout(struct director *dir) +{ + struct director_host *const *hosts, *host; + unsigned int i, count; + + timeout_remove(&dir->to_remove_dirs); + + hosts = array_get(&dir->dir_hosts, &count); + for (i = 0; i < count; ) { + if (hosts[i]->removed) { + host = hosts[i]; + director_host_free(&host); + hosts = array_get(&dir->dir_hosts, &count); + } else { + i++; + } + } +} + +void director_ring_remove(struct director_host *removed_host, + struct director_host *src) +{ + struct director *dir = removed_host->dir; + struct director_connection *const *conns, *conn; + unsigned int i, count; + const char *cmd; + + if (removed_host->self) { + /* others will just disconnect us */ + return; + } + + /* mark the host as removed and fully remove it later. this delay is + needed, because the removal may trigger director reconnections, + which may send the director back and we don't want to re-add it */ + removed_host->removed = TRUE; + if (dir->to_remove_dirs == NULL) { + dir->to_remove_dirs = From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: expire: Handle dict errors. Don't crash if lookup f... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/8f72002cb394 changeset: 14572:8f72002cb394 user: Timo Sirainen date: Sat May 19 22:11:41 2012 +0300 description: expire: Handle dict errors. Don't crash if lookup fails. Based on patch by Michal Grzedzicki diffstat: src/plugins/expire/expire-plugin.c | 13 ++++++++++--- 1 files changed, 10 insertions(+), 3 deletions(-) diffs (30 lines): diff -r 42cca8a1d179 -r 8f72002cb394 src/plugins/expire/expire-plugin.c --- a/src/plugins/expire/expire-plugin.c Sat May 19 21:18:04 2012 +0300 +++ b/src/plugins/expire/expire-plugin.c Sat May 19 22:11:41 2012 +0300 @@ -162,8 +162,14 @@ this is the first mail in the database */ ret = dict_lookup(euser->db, pool_datastack_create(), key, &value); - if (ret == 0) { - /* first time saving here with expire enabled */ + if (ret <= 0) { + /* first time saving here with expire enabled. + also handle lookup errors by just assuming + it didn't exist */ + if (ret < 0) { + i_warning("expire: dict lookup failed, " + "assuming update is needed"); + } first_save_timestamp(box, &new_stamp); update_dict = TRUE; } else if (strcmp(value, "0") == 0) { @@ -188,7 +194,8 @@ dctx = dict_transaction_begin(euser->db); dict_set(dctx, key, dec2str(new_stamp)); - dict_transaction_commit(&dctx); + if (dict_transaction_commit(&dctx) < 0) + i_error("expire: dict commit failed"); } } T_END; i_free(xt); From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: auth: Fixed crash with DIGEST-MD5 when attempting t... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/23543c407e81 changeset: 14573:23543c407e81 user: Timo Sirainen date: Sat May 19 22:25:27 2012 +0300 description: auth: Fixed crash with DIGEST-MD5 when attempting to do master user login without master passdbs. diffstat: src/auth/auth-request.c | 27 ++++++++++++++++++++------- 1 files changed, 20 insertions(+), 7 deletions(-) diffs (51 lines): diff -r 8f72002cb394 -r 23543c407e81 src/auth/auth-request.c --- a/src/auth/auth-request.c Sat May 19 22:11:41 2012 +0300 +++ b/src/auth/auth-request.c Sat May 19 22:25:27 2012 +0300 @@ -609,6 +609,20 @@ return FALSE; } +static bool auth_request_is_disabled_master_user(struct auth_request *request) +{ + if (request->passdb != NULL) + return FALSE; + + /* no masterdbs, master logins not supported */ + i_assert(request->requested_login_user != NULL); + auth_request_log_info(request, "passdb", + "Attempted master login with no master passdbs " + "(trying to log in as user: %s)", + request->requested_login_user); + return TRUE; +} + void auth_request_verify_plain(struct auth_request *request, const char *password, verify_plain_callback_t *callback) @@ -619,13 +633,7 @@ i_assert(request->state == AUTH_REQUEST_STATE_MECH_CONTINUE); - if (request->passdb == NULL) { - /* no masterdbs, master logins not supported */ - i_assert(request->requested_login_user != NULL); - auth_request_log_info(request, "passdb", - "Attempted master login with no master passdbs " - "(trying to log in as user: %s)", - request->requested_login_user); + if (auth_request_is_disabled_master_user(request)) { callback(PASSDB_RESULT_USER_UNKNOWN, request); return; } @@ -746,6 +754,11 @@ i_assert(request->state == AUTH_REQUEST_STATE_MECH_CONTINUE); + if (auth_request_is_disabled_master_user(request)) { + callback(PASSDB_RESULT_USER_UNKNOWN, NULL, 0, request); + return; + } + request->credentials_scheme = p_strdup(request->pool, scheme); request->private_callback.lookup_credentials = callback; From dovecot at dovecot.org Sun May 20 03:26:36 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:36 +0300 Subject: dovecot-2.2: lib-master: Anvil reconnect timeout should be 5 sec... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/1c2c1d75d3b2 changeset: 14574:1c2c1d75d3b2 user: Timo Sirainen date: Sun May 20 02:41:08 2012 +0300 description: lib-master: Anvil reconnect timeout should be 5 seconds, not 5 milliseconds. diffstat: src/lib-master/anvil-client.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 23543c407e81 -r 1c2c1d75d3b2 src/lib-master/anvil-client.c --- a/src/lib-master/anvil-client.c Sat May 19 22:25:27 2012 +0300 +++ b/src/lib-master/anvil-client.c Sun May 20 02:41:08 2012 +0300 @@ -80,7 +80,7 @@ if (ioloop_time - client->last_reconnect < ANVIL_RECONNECT_MIN_SECS) { if (client->to_reconnect == NULL) { client->to_reconnect = - timeout_add(ANVIL_RECONNECT_MIN_SECS, + timeout_add(ANVIL_RECONNECT_MIN_SECS*1000, anvil_reconnect, client); } } else { From dovecot at dovecot.org Sun May 20 03:26:37 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:26:37 +0300 Subject: dovecot-2.2: Merged changes from v2.1 tree. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/fbb1ecb9b888 changeset: 14576:fbb1ecb9b888 user: Timo Sirainen date: Sun May 20 03:25:04 2012 +0300 description: Merged changes from v2.1 tree. diffstat: .hgignore | 2 + .hgsigs | 5 + .hgtags | 5 + Makefile.am | 7 +- NEWS | 93 + TODO | 1 - configure.in | 16 +- doc/example-config/conf.d/10-mail.conf | 6 + doc/example-config/conf.d/10-ssl.conf | 3 + doc/example-config/conf.d/20-pop3.conf | 5 + doc/example-config/dovecot.conf | 5 + dovecot-config.in.in | 2 +- src/Makefile.am | 1 + src/anvil/anvil-connection.c | 2 +- src/anvil/penalty.c | 6 +- src/auth/Makefile.am | 22 +- src/auth/auth-cache.c | 85 +- src/auth/auth-client-connection.c | 11 +- src/auth/auth-master-connection.c | 6 +- src/auth/auth-request-handler.c | 30 +- src/auth/auth-request.c | 131 +- src/auth/auth-request.h | 11 +- src/auth/auth-settings.c | 2 +- src/auth/auth-stream.c | 2 +- src/auth/auth-worker-client.c | 9 +- src/auth/auth.c | 12 +- src/auth/checkpassword-reply.c | 27 +- src/auth/db-checkpassword.c | 474 +++++- src/auth/db-checkpassword.h | 65 +- src/auth/db-ldap.c | 43 +- src/auth/db-ldap.h | 5 +- src/auth/passdb-blocking.c | 2 +- src/auth/passdb-cache.c | 4 +- src/auth/passdb-checkpassword.c | 312 +--- src/auth/passdb-ldap.c | 1 - src/auth/passdb-static.c | 2 +- src/auth/test-auth-cache.c | 57 + src/auth/userdb-checkpassword.c | 240 +-- src/auth/userdb-passwd.c | 25 +- src/config/config-connection.c | 2 +- src/config/config-filter.c | 12 +- src/config/config-request.c | 2 +- src/config/doveconf.c | 2 +- src/dict/dict-commands.c | 8 +- src/director/Makefile.am | 23 +- src/director/director-connection.c | 1188 ++++++++++----- src/director/director-connection.h | 16 +- src/director/director-host.c | 35 +- src/director/director-host.h | 14 +- src/director/director-request.c | 89 +- src/director/director-settings.c | 2 + src/director/director-settings.h | 1 + src/director/director-test.c | 2 +- src/director/director.c | 347 +++- src/director/director.h | 40 +- src/director/doveadm-connection.c | 74 +- src/director/login-connection.c | 2 +- src/director/main.c | 8 +- src/director/notify-connection.c | 9 +- src/director/test-user-directory.c | 104 + src/director/user-directory.c | 142 +- src/director/user-directory.h | 18 +- src/doveadm/Makefile.am | 5 +- src/doveadm/client-connection.c | 59 +- src/doveadm/doveadm-auth.c | 114 +- src/doveadm/doveadm-director.c | 77 +- src/doveadm/doveadm-mail-expunge.c | 66 +- src/doveadm/doveadm-mail-import.c | 1 + src/doveadm/doveadm-mail-iter.c | 18 +- src/doveadm/doveadm-mail-iter.h | 2 + src/doveadm/doveadm-mail-mailbox-status.c | 2 +- src/doveadm/doveadm-mail-mailbox.c | 13 +- src/doveadm/doveadm-mail-server.c | 16 +- src/doveadm/doveadm-mail.c | 37 +- src/doveadm/doveadm-mail.h | 8 +- src/doveadm/doveadm-mailbox-list-iter.c | 65 +- src/doveadm/doveadm-mount.c | 6 +- src/doveadm/doveadm-mutf7.c | 2 + src/doveadm/doveadm-penalty.c | 2 +- src/doveadm/doveadm-proxy.c | 2 +- src/doveadm/doveadm-settings.c | 2 + src/doveadm/doveadm-settings.h | 1 + src/doveadm/doveadm-stats.c | 2 +- src/doveadm/doveadm-who.c | 2 +- src/doveadm/doveadm.c | 1 + src/doveadm/dsync/Makefile.am | 3 - src/doveadm/dsync/doveadm-dsync.c | 463 ++++- src/doveadm/dsync/dsync-brain-msgs.c | 2 + src/doveadm/dsync/dsync-brain.c | 6 +- src/doveadm/dsync/dsync-data.h | 1 + src/doveadm/dsync/dsync-proxy-server.c | 2 +- src/doveadm/dsync/dsync-worker-local.c | 46 +- src/doveadm/dsync/dsync-worker.h | 3 +- src/doveadm/dsync/test-dsync-proxy-server-cmd.c | 2 + src/doveadm/server-connection.c | 16 +- src/doveadm/server-connection.h | 1 + src/imap-login/client.c | 4 + src/imap-login/imap-proxy.c | 2 + src/imap/Makefile.am | 13 +- src/imap/cmd-append.c | 13 +- src/imap/cmd-fetch.c | 5 + src/imap/cmd-list.c | 24 +- src/imap/imap-client.c | 11 +- src/imap/imap-client.h | 4 +- src/imap/imap-commands.c | 3 +- src/imap/imap-sync.c | 7 +- src/imap/main.c | 11 +- src/indexer/Makefile.am | 13 +- src/ipc/ipc-connection.c | 2 +- src/lda/Makefile.am | 12 +- src/lib-auth/auth-client-request.c | 6 + src/lib-auth/auth-client.h | 1 + src/lib-auth/auth-master.c | 4 +- src/lib-auth/auth-server-connection.c | 2 +- src/lib-dict/dict-file.c | 168 +- src/lib-dict/dict-sql.c | 6 +- src/lib-imap/test-imap-utf7.c | 2 +- src/lib-index/mail-cache-fields.c | 2 +- src/lib-index/mail-index-map.c | 10 +- src/lib-index/mail-index-modseq.c | 5 + src/lib-index/mail-index-modseq.h | 1 + src/lib-index/mail-index-transaction-private.h | 4 + src/lib-index/mail-index-transaction-update.c | 2 +- src/lib-index/mail-index-transaction.c | 17 + src/lib-index/mail-transaction-log-file.c | 2 +- src/lib-index/test-mail-index-transaction-update.c | 6 + src/lib-lda/mail-deliver.c | 8 +- src/lib-mail/Makefile.am | 2 + src/lib-mail/mail-user-hash.c | 41 + src/lib-mail/mail-user-hash.h | 8 + src/lib-mail/message-date.c | 10 +- src/lib-mail/message-parser.c | 29 +- src/lib-mail/rfc822-parser.c | 2 +- src/lib-master/anvil-client.c | 2 +- src/lib-master/master-login-auth.c | 4 +- src/lib-master/master-login.c | 25 +- src/lib-master/master-login.h | 3 + src/lib-master/master-service-settings.c | 2 +- src/lib-master/master-service.c | 2 + src/lib-master/mountpoint-list.c | 32 +- src/lib-master/mountpoint-list.h | 7 +- src/lib-settings/settings-parser.c | 37 +- src/lib-settings/settings-parser.h | 6 +- src/lib-sql/driver-mysql.c | 4 +- src/lib-sql/driver-pgsql.c | 3 + src/lib-sql/driver-sqlpool.c | 4 +- src/lib-ssl-iostream/Makefile.am | 2 +- src/lib-ssl-iostream/iostream-openssl.c | 2 + src/lib-storage/Makefile.am | 3 +- src/lib-storage/fail-mailbox.c | 2 +- src/lib-storage/index/Makefile.am | 3 - src/lib-storage/index/cydir/cydir-mail.c | 6 +- src/lib-storage/index/dbox-common/dbox-file.c | 5 +- src/lib-storage/index/dbox-common/dbox-storage.c | 9 +- src/lib-storage/index/dbox-common/dbox-storage.h | 2 - src/lib-storage/index/dbox-multi/mdbox-map.c | 8 +- src/lib-storage/index/dbox-multi/mdbox-purge.c | 6 +- src/lib-storage/index/dbox-multi/mdbox-storage.c | 8 +- src/lib-storage/index/dbox-multi/mdbox-sync.c | 8 + src/lib-storage/index/dbox-single/sdbox-copy.c | 9 +- src/lib-storage/index/dbox-single/sdbox-save.c | 9 + src/lib-storage/index/dbox-single/sdbox-sync-rebuild.c | 1 + src/lib-storage/index/dbox-single/sdbox-sync.c | 6 +- src/lib-storage/index/imapc/imapc-list.c | 72 +- src/lib-storage/index/imapc/imapc-mail-fetch.c | 18 + src/lib-storage/index/imapc/imapc-mail.c | 101 +- src/lib-storage/index/imapc/imapc-settings.c | 44 + src/lib-storage/index/imapc/imapc-settings.h | 11 + src/lib-storage/index/imapc/imapc-storage.h | 8 +- src/lib-storage/index/index-mail.c | 6 +- src/lib-storage/index/index-mail.h | 2 +- src/lib-storage/index/index-search-private.h | 2 + src/lib-storage/index/index-search.c | 39 +- src/lib-storage/index/index-status.c | 16 +- src/lib-storage/index/index-storage.c | 25 +- src/lib-storage/index/index-thread.c | 3 + src/lib-storage/index/index-transaction.c | 8 + src/lib-storage/index/maildir/maildir-mail.c | 36 +- src/lib-storage/index/maildir/maildir-save.c | 16 +- src/lib-storage/index/maildir/maildir-storage.c | 11 +- src/lib-storage/index/maildir/maildir-storage.h | 2 - src/lib-storage/index/maildir/maildir-sync-index.c | 2 + src/lib-storage/index/maildir/maildir-sync.c | 57 +- src/lib-storage/index/maildir/maildir-sync.h | 1 + src/lib-storage/index/maildir/maildir-uidlist.c | 14 + src/lib-storage/index/maildir/maildir-util.c | 11 + src/lib-storage/index/mbox/mbox-storage.h | 1 + src/lib-storage/index/mbox/mbox-sync.c | 20 +- src/lib-storage/index/pop3c/pop3c-client.c | 2 +- src/lib-storage/index/pop3c/pop3c-mail.c | 2 +- src/lib-storage/index/pop3c/pop3c-storage.c | 2 +- src/lib-storage/index/raw/raw-storage.c | 1 + src/lib-storage/index/shared/shared-storage.c | 34 +- src/lib-storage/list/mailbox-list-fs-iter.c | 115 +- src/lib-storage/list/mailbox-list-index-iter.c | 7 +- src/lib-storage/list/mailbox-list-index-status.c | 2 + src/lib-storage/list/mailbox-list-index-sync.c | 7 +- src/lib-storage/list/mailbox-list-index.c | 69 +- src/lib-storage/list/mailbox-list-index.h | 6 + src/lib-storage/list/mailbox-list-maildir-iter.c | 20 +- src/lib-storage/list/mailbox-list-subscriptions.c | 7 +- src/lib-storage/list/mailbox-list-subscriptions.h | 3 +- src/lib-storage/list/subscription-file.c | 34 +- src/lib-storage/mail-namespace.c | 7 + src/lib-storage/mail-search-register-human.c | 24 +- src/lib-storage/mail-search-register-imap.c | 60 +- src/lib-storage/mail-storage-private.h | 11 +- src/lib-storage/mail-storage-service.c | 11 +- src/lib-storage/mail-storage-service.h | 1 + src/lib-storage/mail-storage-settings.c | 4 + src/lib-storage/mail-storage-settings.h | 2 + src/lib-storage/mail-storage.c | 116 +- src/lib-storage/mail-storage.h | 10 + src/lib-storage/mail-user.c | 70 +- src/lib-storage/mail-user.h | 4 + src/lib-storage/mailbox-list-iter.c | 19 +- src/lib-storage/mailbox-list-private.h | 3 - src/lib-storage/mailbox-list.c | 19 + src/lib-storage/mailbox-list.h | 7 +- src/lib-storage/mailbox-tree.c | 30 +- src/lib-storage/mailbox-tree.h | 3 +- src/lib-test/test-common.c | 2 +- src/lib/array.c | 6 +- src/lib/array.h | 20 +- src/lib/compat.h | 3 +- src/lib/data-stack.c | 38 +- src/lib/eacces-error.c | 12 +- src/lib/file-cache.c | 10 +- src/lib/ioloop.c | 3 +- src/lib/mempool-alloconly.c | 3 +- src/lib/network.c | 4 +- src/lib/ostream.c | 3 +- src/lib/process-title.c | 2 + src/lib/seq-range-array.c | 22 +- src/lib/strfuncs.c | 45 + src/lib/strfuncs.h | 4 +- src/lib/test-array.c | 2 +- src/lib/test-base64.c | 8 +- src/lib/test-bsearch-insert-pos.c | 6 +- src/lib/test-primes.c | 2 +- src/lib/test-seq-range-array.c | 16 +- src/lib/test-str-find.c | 3 +- src/lib/test-strfuncs.c | 53 + src/lib/test-var-expand.c | 31 + src/lib/unlink-old-files.c | 18 +- src/lib/unlink-old-files.h | 3 +- src/lib/var-expand.c | 40 +- src/lib/var-expand.h | 4 + src/lmtp/Makefile.am | 12 +- src/lmtp/commands.c | 13 +- src/log/log-connection.c | 23 +- src/log/main.c | 8 +- src/login-common/Makefile.am | 3 +- src/login-common/client-common-auth.c | 9 + src/login-common/client-common.c | 172 +- src/login-common/client-common.h | 12 +- src/login-common/login-proxy-state.c | 56 +- src/login-common/login-proxy.c | 43 +- src/login-common/login-proxy.h | 3 + src/login-common/login-settings.c | 6 +- src/login-common/login-settings.h | 2 + src/login-common/main.c | 2 +- src/login-common/sasl-server.c | 8 +- src/login-common/ssl-proxy-openssl.c | 99 +- src/login-common/ssl-proxy.c | 7 + src/login-common/ssl-proxy.h | 5 +- src/master/main.c | 5 +- src/master/master-settings.c | 2 +- src/master/service-listen.c | 91 + src/plugins/Makefile.am | 2 + src/plugins/acl/acl-backend-vfile-acllist.c | 2 +- src/plugins/acl/acl-backend-vfile.c | 3 +- src/plugins/acl/acl-cache.c | 4 + src/plugins/acl/acl-shared-storage.c | 20 +- src/plugins/expire/doveadm-expire.c | 2 + src/plugins/expire/expire-plugin.c | 13 +- src/plugins/fts-lucene/fts-backend-lucene.c | 3 +- src/plugins/fts-solr/fts-backend-solr.c | 34 +- src/plugins/fts-squat/Makefile.am | 12 +- src/plugins/fts-squat/fts-backend-squat.c | 5 +- src/plugins/fts-squat/squat-trie.c | 1 + src/plugins/fts-squat/squat-uidlist.c | 3 + src/plugins/fts/Makefile.am | 1 + src/plugins/fts/decode2text.sh | 22 +- src/plugins/fts/fts-api-private.h | 4 +- src/plugins/fts/fts-indexer.c | 25 +- src/plugins/fts/fts-storage.c | 58 +- src/plugins/fts/fts-storage.h | 1 + src/plugins/imap-stats/imap-stats-plugin.c | 9 +- src/plugins/mail-log/mail-log-plugin.c | 18 +- src/plugins/notify/notify-storage.c | 9 +- src/plugins/pop3-migration/Makefile.am | 17 + src/plugins/pop3-migration/pop3-migration-plugin.c | 632 ++++++++ src/plugins/pop3-migration/pop3-migration-plugin.h | 7 + src/plugins/quota/quota-count.c | 7 +- src/plugins/quota/quota-private.h | 3 + src/plugins/quota/quota-storage.c | 8 + src/plugins/quota/quota.c | 30 +- src/plugins/replication/Makefile.am | 25 + src/plugins/replication/replication-plugin.c | 371 ++++ src/plugins/replication/replication-plugin.h | 9 + src/plugins/stats/stats-connection.c | 36 +- src/plugins/stats/stats-plugin.c | 235 +- src/plugins/stats/stats-plugin.h | 3 + src/plugins/trash/trash-plugin.c | 50 +- src/plugins/virtual/virtual-mail.c | 8 +- src/pop3-login/client.c | 3 + src/pop3-login/pop3-proxy.c | 3 +- src/pop3/Makefile.am | 13 +- src/pop3/main.c | 4 +- src/pop3/pop3-client.c | 44 +- src/pop3/pop3-client.h | 9 +- src/pop3/pop3-commands.c | 272 ++- src/pop3/pop3-settings.c | 4 +- src/pop3/pop3-settings.h | 1 + src/replication/Makefile.am | 4 + src/replication/aggregator/Makefile.am | 26 + src/replication/aggregator/aggregator-settings.c | 85 + src/replication/aggregator/aggregator-settings.h | 12 + src/replication/aggregator/aggregator.c | 75 + src/replication/aggregator/notify-connection.c | 154 ++ src/replication/aggregator/notify-connection.h | 9 + src/replication/aggregator/replicator-connection.c | 325 ++++ src/replication/aggregator/replicator-connection.h | 25 + src/replication/replication-common.h | 30 + src/replication/replicator/Makefile.am | 30 + src/replication/replicator/doveadm-connection.c | 195 ++ src/replication/replicator/doveadm-connection.h | 20 + src/replication/replicator/notify-connection.c | 197 ++ src/replication/replicator/notify-connection.h | 13 + src/replication/replicator/replicator-brain.c | 166 ++ src/replication/replicator/replicator-brain.h | 11 + src/replication/replicator/replicator-queue.c | 371 ++++ src/replication/replicator/replicator-queue.h | 60 + src/replication/replicator/replicator-settings.c | 80 + src/replication/replicator/replicator-settings.h | 15 + src/replication/replicator/replicator.c | 117 + src/stats/mail-command.c | 71 +- src/stats/mail-domain.c | 3 +- src/stats/mail-ip.c | 3 +- src/stats/mail-session.c | 8 +- src/stats/mail-user.c | 3 +- src/util/Makefile.am | 9 +- src/util/script-login.c | 2 +- update-version.sh | 2 +- 345 files changed, 9448 insertions(+), 2507 deletions(-) diffs (truncated from 20512 to 300 lines): diff -r 80688ab1ea3d -r fbb1ecb9b888 .hgignore --- a/.hgignore Sat May 19 22:40:08 2012 +0300 +++ b/.hgignore Sun May 20 03:25:04 2012 +0300 @@ -85,6 +85,8 @@ src/plugins/fts-squat/squat-test src/pop3-login/pop3-login src/pop3/pop3 +src/replication/replicator/replicator +src/replication/aggregator/aggregator src/util/gdbhelper src/util/listview src/util/maildirlock diff -r 80688ab1ea3d -r fbb1ecb9b888 .hgsigs --- a/.hgsigs Sat May 19 22:40:08 2012 +0300 +++ b/.hgsigs Sun May 20 03:25:04 2012 +0300 @@ -41,3 +41,8 @@ 736f1b7af190ea68e65719277bd8d3d4682e0844 0 iEYEABECAAYFAk87Kz4ACgkQyUhSUUBVismKXACeME7EBYoEuoLLELp0uX6B/lkgRWQAoJ2OAgz4mmluGbi0Db8grDDWSsTj e2cd03cc9c690c4989fb53a1871b7109c547388a 0 iEYEABECAAYFAk89MXsACgkQyUhSUUBVisnPmwCgiAr4OfQX1uAjhuqj5X0xbd8O1NQAn2bQW+h8QPAbqN6dQRNTm82D2hNF 04b0acc03f1eaa0353888a75a452e5c8e61e4c94 0 iEYEABECAAYFAk9F+k0ACgkQyUhSUUBViskLmwCfUt/aex6wOIEohJKnRGA4diF5WxoAn2zlMxSaPX/b0LBmV1P46GAMqZbO +744e0d7f1b255a9339060f761b850303121f14df 0 iEYEABECAAYFAk9h/8oACgkQyUhSUUBVism2OQCfWh62w8pMxJaf1oYx2A+2PxQvBocAn29RFDgZblGRLn7iMCPw6We1yiIw +b9adfd52cb66d5d89d291b76b9845d6361216d12 0 iEYEABECAAYFAk9jbugACgkQyUhSUUBVislSgwCgpo3f0bsSujItBum/M6js8SzF06YAmwftHlwaOstKeALdjLR5vtF2c5F7 +2c21c940e19d97a772128a7f281cea302e157d73 0 iEYEABECAAYFAk+CtkYACgkQyUhSUUBVisknxgCfTJw2YPGJ17HbHRGmbwmCyLqepbMAn17j7IYzUfEU0xkXhCgwEAmk7XO4 +469cee314d9c54d2d7101ec9e38579fdc9610eaf 0 iEYEABECAAYFAk+VWqkACgkQyUhSUUBVislnXACfVjPqMmPUvYtXQXwqff0h7N76mZUAn02lPeUCyuyr1TF9e1hGM/sKgmko +7c249e2a82a9cd33ae15ead6443c3499e16da623 0 iEYEABECAAYFAk+nX2sACgkQyUhSUUBVisn7uwCbBD3boxBOGEJ8OYsIJ57n5Cr09FAAoIvhxL6EHRB15AMOw4sPaALJ3/bB diff -r 80688ab1ea3d -r fbb1ecb9b888 .hgtags --- a/.hgtags Sat May 19 22:40:08 2012 +0300 +++ b/.hgtags Sun May 20 03:25:04 2012 +0300 @@ -78,3 +78,8 @@ 736f1b7af190ea68e65719277bd8d3d4682e0844 2.1.rc7 e2cd03cc9c690c4989fb53a1871b7109c547388a 2.1.0 04b0acc03f1eaa0353888a75a452e5c8e61e4c94 2.1.1 +744e0d7f1b255a9339060f761b850303121f14df 2.1.2 +b9adfd52cb66d5d89d291b76b9845d6361216d12 2.1.3 +2c21c940e19d97a772128a7f281cea302e157d73 2.1.4 +469cee314d9c54d2d7101ec9e38579fdc9610eaf 2.1.5 +7c249e2a82a9cd33ae15ead6443c3499e16da623 2.1.6 diff -r 80688ab1ea3d -r fbb1ecb9b888 Makefile.am --- a/Makefile.am Sat May 19 22:40:08 2012 +0300 +++ b/Makefile.am Sun May 20 03:25:04 2012 +0300 @@ -70,12 +70,17 @@ -e "s|^\(LIBDOVECOT_INCLUDE\)=.*$$|\1=-I$(pkgincludedir)|" \ > $(DESTDIR)$(pkglibdir)/dovecot-config +uninstall-hook: + rm $(DESTDIR)$(pkglibdir)/dovecot-config + CLEANFILES = $(datafiles) if HAVE_SYSTEMD CLEANFILES += $systedmsystemunit_DATA endif -DISTCLEANFILES = $(top_builddir)/dovecot-version.h +DISTCLEANFILES = \ + $(top_builddir)/dovecot-version.h \ + $(top_builddir)/dovecot-config distcheck-hook: if which scan-build > /dev/null; then \ diff -r 80688ab1ea3d -r fbb1ecb9b888 NEWS --- a/NEWS Sat May 19 22:40:08 2012 +0300 +++ b/NEWS Sun May 20 03:25:04 2012 +0300 @@ -1,3 +1,96 @@ +v2.1.6 2012-05-07 Timo Sirainen + + * Session ID is now included by default in auth and login process + log lines. It can be added to mail processes also by adding + %{session} to mail_log_prefix. + + + Added ssl_require_crl setting, which specifies if CRL check must + be successful when verifying client certificates. + + Added mail_shared_explicit_inbox setting to specify if a shared INBOX + should be accessible as "shared/$user" or "shared/$user/INBOX". + - v2.1.5: Using "~/" as mail_location or elsewhere failed to actually + expand it to home directory. + - dbox: Fixed potential assert-crash when reading dbox files. + - trash plugin: Fixed behavior when quota is already over limit. + - mail_log plugin: Logging "copy" event didn't work. + - Proxying to backend server with SSL: Verifying server certificate + name always failed, because it was compared to an IP address. + +v2.1.5 2012-04-23 Timo Sirainen + + * IMAP: When neither the session nor the mailbox has modseq tracking + enabled, return the mailbox as having NOMODSEQ in SELECT/EXAMINE + reply. Old versions in this situation always simply returned + HIGHESTMODSEQ as 1, which could have broken some clients. + + + dict file: Added optional fcntl/flock locking (default is dotlock) + + fts-solr: doveadm fts rescan now resets indexes, which allows + reindexing mails. (This isn't a full rescan implementation like + fts-lucene has.) + + doveadm expunge: Added -d parameter to delete mailbox if it's + empty after expunging. + - IMAP: Several fixes related to mailbox listing in some configs + - director: A lot of fixes and performance improvements + - v2.1.4 didn't work without a mail home directory set + - mbox: Deleting a mailbox didn't delete its index files. + - pop3c: TOP command was sent incorrectly + - trash plugin didn't work properly + - LMTP: Don't add a duplicate Return-Path: header when proxying. + - listescape: Don't unescape namespace prefixes. + +v2.1.4 2012-04-09 Timo Sirainen + + + Added mail_temp_scan_interval setting and changed its default value + from 8 hours to 1 week. + + Added pop3-migration plugin for easily doing a transparent IMAP+POP3 + migration to Dovecot: http://wiki2.dovecot.org/Migration/Dsync + + doveadm user: Added -m parameter to show some of the mail settings. + - Proxying SSL connections crashed in v2.1.[23] + - fts-solr: Indexing mail bodies was broken. + - director: Several changes to significantly improve error handling + - doveadm import didn't import messages' flags + - mail_full_filesystem_access=yes was broken + - Make sure IMAP clients can't create directories when accessing + nonexistent users' mailboxes via shared namespace. + - Dovecot auth clients authenticating via TCP socket could have failed + with bogus "PID already in use" errors. + +v2.1.3 2012-03-16 Timo Sirainen + + - mdbox was broken in v2.1.2 + +v2.1.2 2012-03-15 Timo Sirainen + + + Initial implementation of dsync-based replication. For now this + should be used only on non-critical systems. + + Proxying: POP3 now supports sending remote IP+port from proxy to + backend server via Dovecot-specific XCLIENT extension. + + Proxying: proxy_maybe=yes with host= (instead of IP) + works now properly. + + Proxying: Added auth_proxy_self setting + + Proxying: Added proxy_always extra field (see wiki docs) + + Added director_username_hash setting to specify what part of the + username is hashed. This can be used to implement per-domain + backends (which allows safely accessing shared mailboxes within + domain). + + Added a "session ID" string for imap/pop3 connections, available + in %{session} variable. The session ID passes through Dovecot + IMAP/POP3 proxying to backend server. The same session ID is can be + reused after a long time (currently a bit under 9 years). + + passdb checkpassword: Support "credentials lookups" (for + non-plaintext auth and for lmtp_proxy lookups) + + fts: Added fts_index_timeout setting to abort search if indexing + hasn't finished by then (default is to wait forever). + - doveadm sync: If mailbox was expunged empty, messages may have + become back instead of also being expunged in the other side. + - director: If user logged into two directors while near user + expiration, the directors might have redirected the user to two + different backends. + - imap_id_* settings were ignored before login. + - Several fixes to mailbox_list_index=yes + - Previous v2.1.x didn't log all messages at shutdown. + - mbox: Fixed accessing Dovecot v1.x mbox index files without errors. + v2.1.1 2012-02-23 Timo Sirainen + dsync: If message with same GUID is saved multiple times in session, diff -r 80688ab1ea3d -r fbb1ecb9b888 TODO --- a/TODO Sat May 19 22:40:08 2012 +0300 +++ b/TODO Sun May 20 03:25:04 2012 +0300 @@ -46,7 +46,6 @@ non-userdb_* extra fields too? - imap, pop3: if client init fails, wait a second or two before disconnecting client. - - doveadm expunge parameter to delete empty mailboxes (for lazy-expunge) - doveadm search savedbefore 7d could be optimized in large mailboxes.. - mdbox: storage rebuilding could log about changes it does - mdbox: broken extrefs header keeps causing index rebuilds diff -r 80688ab1ea3d -r fbb1ecb9b888 configure.in --- a/configure.in Sat May 19 22:40:08 2012 +0300 +++ b/configure.in Sun May 20 03:25:04 2012 +0300 @@ -2490,10 +2490,13 @@ want_ssl_libs=yes fi done +LINKED_STORAGE_LDADD= if test "$want_ssl_libs" = yes; then LINKED_STORAGE_LIBS="$LINKED_STORAGE_LIBS \$(top_builddir)/src/lib-ssl-iostream/libssl_iostream.la" + LINKED_STORAGE_LDADD="$SSL_LIBS" fi AC_SUBST(LINKED_STORAGE_LIBS) +AC_SUBST(LINKED_STORAGE_LDADD) AC_SUBST(mailbox_list_drivers) AC_DEFINE_UNQUOTED(MAIL_STORAGES, "$mail_storages", List of compiled in mail storages) @@ -2509,7 +2512,7 @@ if test "$want_shared_libs" = "yes"; then LIBDOVECOT_DEPS='$(top_builddir)/src/lib-dovecot/libdovecot.la' LIBDOVECOT="$LIBDOVECOT_DEPS" - LIBDOVECOT_STORAGE='$(top_builddir)/src/lib-storage/libdovecot-storage.la' + LIBDOVECOT_STORAGE_DEPS='$(top_builddir)/src/lib-storage/libdovecot-storage.la' LIBDOVECOT_LOGIN='$(top_builddir)/src/login-common/libdovecot-login.la' LIBDOVECOT_LDA='$(top_builddir)/src/lib-lda/libdovecot-lda.la' else @@ -2517,14 +2520,16 @@ LIBDOVECOT="$LIBDOVECOT_DEPS \$(LIBICONV)" LIBDOVECOT_STORAGE_LAST='$(top_builddir)/src/lib-storage/list/libstorage_list.la $(top_builddir)/src/lib-storage/index/libstorage_index.la $(top_builddir)/src/lib-storage/libstorage.la $(top_builddir)/src/lib-index/libindex.la' LIBDOVECOT_STORAGE_FIRST='$(top_builddir)/src/lib-storage/libstorage_service.la $(top_builddir)/src/lib-storage/register/libstorage_register.la' - LIBDOVECOT_STORAGE="$LIBDOVECOT_STORAGE_FIRST $LINKED_STORAGE_LIBS $LIBDOVECOT_STORAGE_LAST" + LIBDOVECOT_STORAGE_DEPS="$LIBDOVECOT_STORAGE_FIRST $LINKED_STORAGE_LIBS $LIBDOVECOT_STORAGE_LAST" LIBDOVECOT_LOGIN='$(top_builddir)/src/login-common/liblogin.la $(top_builddir)/src/lib-ssl-iostream/libssl_iostream.la' LIBDOVECOT_LDA='$(top_builddir)/src/lib-lda/liblda.la' fi +LIBDOVECOT_STORAGE="$LIBDOVECOT_STORAGE_DEPS $LINKED_STORAGE_LDADD" LIBDOVECOT_SQL='$(top_builddir)/src/lib-sql/libsql.la' AC_SUBST(LIBDOVECOT) AC_SUBST(LIBDOVECOT_DEPS) AC_SUBST(LIBDOVECOT_STORAGE) +AC_SUBST(LIBDOVECOT_STORAGE_DEPS) AC_SUBST(LIBDOVECOT_LOGIN) AC_SUBST(LIBDOVECOT_SQL) AC_SUBST(LIBDOVECOT_LDA) @@ -2699,7 +2704,7 @@ dnl IDLE doesn't really belong to banner. It's there just to make Blackberries dnl happy, because otherwise BIS server disables push email. capability_banner="IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE" -capability="$capability_banner SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SEARCH=FUZZY SPECIAL-USE" +capability="$capability_banner SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE" AC_DEFINE_UNQUOTED(CAPABILITY_STRING, "$capability", IMAP capabilities) AC_DEFINE_UNQUOTED(CAPABILITY_BANNER_STRING, "$capability_banner", IMAP capabilities advertised in banner) @@ -2787,6 +2792,9 @@ src/master/Makefile src/pop3/Makefile src/pop3-login/Makefile +src/replication/Makefile +src/replication/aggregator/Makefile +src/replication/replicator/Makefile src/ssl-params/Makefile src/stats/Makefile src/util/Makefile @@ -2803,8 +2811,10 @@ src/plugins/listescape/Makefile src/plugins/mail-log/Makefile src/plugins/notify/Makefile +src/plugins/pop3-migration/Makefile src/plugins/quota/Makefile src/plugins/imap-quota/Makefile +src/plugins/replication/Makefile src/plugins/snarf/Makefile src/plugins/stats/Makefile src/plugins/imap-stats/Makefile diff -r 80688ab1ea3d -r fbb1ecb9b888 doc/example-config/conf.d/10-mail.conf --- a/doc/example-config/conf.d/10-mail.conf Sat May 19 22:40:08 2012 +0300 +++ b/doc/example-config/conf.d/10-mail.conf Sun May 20 03:25:04 2012 +0300 @@ -97,6 +97,8 @@ # List the shared/ namespace only if there are visible shared mailboxes. #list = children #} +# Should shared INBOX be visible as "shared/user" or "shared/user/INBOX"? +#mail_shared_explicit_inbox = yes # System user and group used to access mails. If you use multiple, userdb # can override these by returning uid or gid fields. You can use either numbers @@ -227,6 +229,10 @@ # some mailbox formats and/or operating systems. #mail_prefetch_count = 0 +# How often to scan for stale temporary files and delete them (0 = never). +# These should exist only after Dovecot dies in the middle of saving mails. +#mail_temp_scan_interval = 1w + ## ## Maildir-specific settings ## diff -r 80688ab1ea3d -r fbb1ecb9b888 doc/example-config/conf.d/10-ssl.conf --- a/doc/example-config/conf.d/10-ssl.conf Sat May 19 22:40:08 2012 +0300 +++ b/doc/example-config/conf.d/10-ssl.conf Sun May 20 03:25:04 2012 +0300 @@ -23,6 +23,9 @@ # followed by the matching CRL(s). (e.g. ssl_ca = details: http://hg.dovecot.org/dovecot-2.2/rev/dbe6d05fd595 changeset: 14575:dbe6d05fd595 user: Timo Sirainen date: Sun May 20 03:08:01 2012 +0300 description: director: When we find unwanted connection, wait for 10s for it to disconnect us, not 10ms. diffstat: src/director/director-connection.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (21 lines): diff -r 1c2c1d75d3b2 -r dbe6d05fd595 src/director/director-connection.c --- a/src/director/director-connection.c Sun May 20 02:41:08 2012 +0300 +++ b/src/director/director-connection.c Sun May 20 03:08:01 2012 +0300 @@ -70,7 +70,7 @@ /* If outgoing director connection exists for less than this many seconds, mark the host as failed so we won't try to reconnect to it immediately */ #define DIRECTOR_SUCCESS_MIN_CONNECT_SECS 40 -#define DIRECTOR_WAIT_DISCONNECT_MSECS 10 +#define DIRECTOR_WAIT_DISCONNECT_SECS 10 #if DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS <= DIRECTOR_CONNECTION_PING_TIMEOUT_MSECS # error DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS is too low @@ -187,7 +187,7 @@ o_stream_uncork(conn->output); conn->to_disconnect = - timeout_add(DIRECTOR_WAIT_DISCONNECT_MSECS, + timeout_add(DIRECTOR_WAIT_DISCONNECT_SECS*1000, director_connection_wait_timeout, conn); } From dovecot at dovecot.org Sun May 20 03:33:26 2012 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Sun, 20 May 2012 03:33:26 +0300 Subject: dovecot-2.2: Use timeout_add_short() for sub-second timeouts. Fa... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/a47c95872745 changeset: 14577:a47c95872745 user: Timo Sirainen date: Sun May 20 03:32:55 2012 +0300 description: Use timeout_add_short() for sub-second timeouts. Fail at compile time if timeout_add() is <1s. In future timeout_add() could perhaps also be made less precise, so that it would try to group timeouts to run around at the same time. diffstat: src/auth/auth-request-handler.c | 4 ++-- src/director/director-test.c | 4 ++-- src/lib-storage/index/imapc/imapc-mailbox.c | 4 ++-- src/lib-storage/index/index-mailbox-check.c | 4 ++-- src/lib/ioloop.c | 8 ++++++++ src/lib/ioloop.h | 7 +++++++ src/plugins/fts/fts-indexer.c | 2 +- src/plugins/replication/replication-plugin.c | 4 ++-- 8 files changed, 26 insertions(+), 11 deletions(-) diffs (117 lines): diff -r fbb1ecb9b888 -r a47c95872745 src/auth/auth-request-handler.c --- a/src/auth/auth-request-handler.c Sun May 20 03:25:04 2012 +0300 +++ b/src/auth/auth-request-handler.c Sun May 20 03:32:55 2012 +0300 @@ -231,8 +231,8 @@ aqueue_append(auth_failures, &request); if (to_auth_failures == NULL) { to_auth_failures = - timeout_add(AUTH_FAILURE_DELAY_CHECK_MSECS, - auth_failure_timeout, NULL); + timeout_add_short(AUTH_FAILURE_DELAY_CHECK_MSECS, + auth_failure_timeout, NULL); } } diff -r fbb1ecb9b888 -r a47c95872745 src/director/director-test.c --- a/src/director/director-test.c Sun May 20 03:25:04 2012 +0300 +++ b/src/director/director-test.c Sun May 20 03:32:55 2012 +0300 @@ -450,8 +450,8 @@ if (conn->fd == -1) i_fatal("net_connect_unix(%s) failed: %m", path); conn->io = io_add(conn->fd, IO_READ, admin_input, conn); - conn->to_random = timeout_add(ADMIN_RANDOM_TIMEOUT_MSECS, - admin_random_action, conn); + conn->to_random = timeout_add_short(ADMIN_RANDOM_TIMEOUT_MSECS, + admin_random_action, conn); net_set_nonblock(conn->fd, FALSE); conn->input = i_stream_create_fd(conn->fd, (size_t)-1, TRUE); diff -r fbb1ecb9b888 -r a47c95872745 src/lib-storage/index/imapc/imapc-mailbox.c --- a/src/lib-storage/index/imapc/imapc-mailbox.c Sun May 20 03:25:04 2012 +0300 +++ b/src/lib-storage/index/imapc/imapc-mailbox.c Sun May 20 03:32:55 2012 +0300 @@ -117,8 +117,8 @@ mbox->to_idle_delay == NULL) { io_loop_set_current(mbox->storage->root_ioloop); mbox->to_idle_delay = - timeout_add(NOTIFY_DELAY_MSECS, - imapc_mailbox_idle_timeout, mbox); + timeout_add_short(NOTIFY_DELAY_MSECS, + imapc_mailbox_idle_timeout, mbox); io_loop_set_current(old_ioloop); } } diff -r fbb1ecb9b888 -r a47c95872745 src/lib-storage/index/index-mailbox-check.c --- a/src/lib-storage/index/index-mailbox-check.c Sun May 20 03:25:04 2012 +0300 +++ b/src/lib-storage/index/index-mailbox-check.c Sun May 20 03:32:55 2012 +0300 @@ -59,8 +59,8 @@ if (ibox->notify_delay_to == NULL) { ibox->notify_delay_to = - timeout_add(NOTIFY_DELAY_MSECS, - notify_delay_callback, box); + timeout_add_short(NOTIFY_DELAY_MSECS, + notify_delay_callback, box); } } diff -r fbb1ecb9b888 -r a47c95872745 src/lib/ioloop.c --- a/src/lib/ioloop.c Sun May 20 03:25:04 2012 +0300 +++ b/src/lib/ioloop.c Sun May 20 03:32:55 2012 +0300 @@ -165,6 +165,14 @@ return timeout; } +#undef timeout_add_short +struct timeout * +timeout_add_short(unsigned int msecs, unsigned int source_linenum, + timeout_callback_t *callback, void *context) +{ + return timeout_add(msecs, source_linenum, callback, context); +} + static void timeout_free(struct timeout *timeout) { if (timeout->ctx != NULL) diff -r fbb1ecb9b888 -r a47c95872745 src/lib/ioloop.h --- a/src/lib/ioloop.h Sun May 20 03:25:04 2012 +0300 +++ b/src/lib/ioloop.h Sun May 20 03:32:55 2012 +0300 @@ -73,6 +73,13 @@ timeout_callback_t *callback, void *context); #define timeout_add(msecs, callback, context) \ CONTEXT_CALLBACK(timeout_add, timeout_callback_t, \ + callback, context, msecs, __LINE__), \ + (void)COMPILE_ERROR_IF_TRUE(__builtin_constant_p(msecs) && (msecs > 0 && msecs < 1000)) +struct timeout * +timeout_add_short(unsigned int msecs, unsigned int source_linenum, + timeout_callback_t *callback, void *context); +#define timeout_add_short(msecs, callback, context) \ + CONTEXT_CALLBACK(timeout_add_short, timeout_callback_t, \ callback, context, msecs, __LINE__) /* Remove timeout handler, and set timeout pointer to NULL. */ void timeout_remove(struct timeout **timeout); diff -r fbb1ecb9b888 -r a47c95872745 src/plugins/fts/fts-indexer.c --- a/src/plugins/fts/fts-indexer.c Sun May 20 03:25:04 2012 +0300 +++ b/src/plugins/fts/fts-indexer.c Sun May 20 03:32:55 2012 +0300 @@ -212,7 +212,7 @@ asynchronous waits, get rid of this wait and use the mail IO loop */ ioloop = io_loop_create(); io = io_add(ctx->fd, IO_READ, io_loop_stop, ioloop); - to = timeout_add(INDEXER_WAIT_MSECS, io_loop_stop, ioloop); + to = timeout_add_short(INDEXER_WAIT_MSECS, io_loop_stop, ioloop); io_loop_run(ioloop); io_remove(&io); timeout_remove(&to); diff -r fbb1ecb9b888 -r a47c95872745 src/plugins/replication/replication-plugin.c --- a/src/plugins/replication/replication-plugin.c Sun May 20 03:25:04 2012 +0300 +++ b/src/plugins/replication/replication-plugin.c Sun May 20 03:32:55 2012 +0300 @@ -206,8 +206,8 @@ if (ruser->priority < priority) ruser->priority = priority; if (ruser->to == NULL) { - ruser->to = timeout_add(REPLICATION_NOTIFY_DELAY_MSECS, - replication_notify_now, ns->owner); + ruser->to = timeout_add_short(REPLICATION_NOTIFY_DELAY_MSECS, + replication_notify_now, ns->owner); } } From pigeonhole at rename-it.nl Sun May 20 13:17:08 2012 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Sun, 20 May 2012 12:17:08 +0200 Subject: dovecot-2.2-pigeonhole: lib-sieve: made error handler part of pu... Message-ID: details: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/a4ea7e3f8d9e changeset: 1609:a4ea7e3f8d9e user: Stephan Bosch date: Wed Feb 22 21:43:35 2012 +0100 description: lib-sieve: made error handler part of public runtime environment. diffstat: src/lib-sieve/plugins/enotify/ext-enotify-common.c | 9 +++------ src/lib-sieve/plugins/include/ext-include-common.c | 5 ++--- src/lib-sieve/plugins/vnd.dovecot/debug/ext-debug.c | 7 ++----- src/lib-sieve/sieve-interpreter.c | 12 +++++------- src/lib-sieve/sieve-runtime.h | 1 + 5 files changed, 13 insertions(+), 21 deletions(-) diffs (134 lines): diff -r 6cca5f460f86 -r a4ea7e3f8d9e src/lib-sieve/plugins/enotify/ext-enotify-common.c --- a/src/lib-sieve/plugins/enotify/ext-enotify-common.c Thu Feb 16 22:07:32 2012 +0100 +++ b/src/lib-sieve/plugins/enotify/ext-enotify-common.c Wed Feb 22 21:43:35 2012 +0100 @@ -502,8 +502,7 @@ nenv.svinst = renv->svinst; nenv.method = method; nenv.ehandler = sieve_prefix_ehandler_create - (sieve_interpreter_get_error_handler(renv->interp), - sieve_runtime_get_full_command_location(renv), + (renv->ehandler, sieve_runtime_get_full_command_location(renv), "valid_notify_method test"); /* Use the method check function to validate the URI */ @@ -567,8 +566,7 @@ nenv.svinst = renv->svinst; nenv.method = method; nenv.ehandler = sieve_prefix_ehandler_create - (sieve_interpreter_get_error_handler(renv->interp), - sieve_runtime_get_full_command_location(renv), + (renv->ehandler, sieve_runtime_get_full_command_location(renv), "notify_method_capability test"); /* Execute method function to acquire capability value */ @@ -602,8 +600,7 @@ nenv.svinst = renv->svinst; nenv.method = method; nenv.ehandler = sieve_prefix_ehandler_create - (sieve_interpreter_get_error_handler(renv->interp), - sieve_runtime_get_full_command_location(renv), + (renv->ehandler, sieve_runtime_get_full_command_location(renv), "notify action"); /* Execute check function */ diff -r 6cca5f460f86 -r a4ea7e3f8d9e src/lib-sieve/plugins/include/ext-include-common.c --- a/src/lib-sieve/plugins/include/ext-include-common.c Thu Feb 16 22:07:32 2012 +0100 +++ b/src/lib-sieve/plugins/include/ext-include-common.c Wed Feb 22 21:43:35 2012 +0100 @@ -688,10 +688,9 @@ if ( ctx->parent == NULL ) { struct ext_include_interpreter_context *curctx = NULL; - struct sieve_error_handler *ehandler = - sieve_interpreter_get_error_handler(renv->interp); + struct sieve_error_handler *ehandler = renv->ehandler; struct sieve_interpreter *subinterp; - bool interrupted = FALSE; + bool interrupted = FALSE; /* We are the top-level interpreter instance */ diff -r 6cca5f460f86 -r a4ea7e3f8d9e src/lib-sieve/plugins/vnd.dovecot/debug/ext-debug.c --- a/src/lib-sieve/plugins/vnd.dovecot/debug/ext-debug.c Thu Feb 16 22:07:32 2012 +0100 +++ b/src/lib-sieve/plugins/vnd.dovecot/debug/ext-debug.c Wed Feb 22 21:43:35 2012 +0100 @@ -63,11 +63,8 @@ (const struct sieve_extension *ext ATTR_UNUSED, const struct sieve_runtime_env *renv, sieve_size_t *address ATTR_UNUSED) { - struct sieve_error_handler *ehandler = - sieve_interpreter_get_error_handler(renv->interp); - - if ( ehandler != NULL ) { - sieve_error_handler_accept_infolog(ehandler, TRUE); + if ( renv->ehandler != NULL ) { + sieve_error_handler_accept_infolog(renv->ehandler, TRUE); } return TRUE; diff -r 6cca5f460f86 -r a4ea7e3f8d9e src/lib-sieve/sieve-interpreter.c --- a/src/lib-sieve/sieve-interpreter.c Thu Feb 16 22:07:32 2012 +0100 +++ b/src/lib-sieve/sieve-interpreter.c Wed Feb 22 21:43:35 2012 +0100 @@ -43,8 +43,6 @@ struct sieve_interpreter { pool_t pool; - - struct sieve_error_handler *ehandler; /* Runtime data for extensions */ ARRAY_DEFINE(extensions, struct sieve_interpreter_extension_reg); @@ -88,7 +86,7 @@ interp = p_new(pool, struct sieve_interpreter, 1); interp->pool = pool; - interp->ehandler = ehandler; + interp->runenv.ehandler = ehandler; sieve_error_handler_ref(ehandler); interp->runenv.interp = interp; @@ -238,7 +236,7 @@ sieve_binary_debug_reader_deinit(&(*interp)->dreader); sieve_binary_unref(&(*interp)->runenv.sbin); - sieve_error_handler_unref(&(*interp)->ehandler); + sieve_error_handler_unref(&(*interp)->runenv.ehandler); pool_unref(&((*interp)->pool)); *interp = NULL; @@ -262,7 +260,7 @@ struct sieve_error_handler *sieve_interpreter_get_error_handler (struct sieve_interpreter *interp) { - return interp->ehandler; + return interp->runenv.ehandler; } struct sieve_instance *sieve_interpreter_svinst @@ -295,7 +293,7 @@ if ( location == NULL ) location = sieve_runtime_get_full_command_location(renv); - msg_func(renv->interp->ehandler, location, fmt, args); + msg_func(renv->ehandler, location, fmt, args); } T_END; } @@ -345,7 +343,7 @@ location = sieve_runtime_get_full_command_location(renv); sieve_vcritical - (renv->svinst, renv->interp->ehandler, location, user_prefix, fmt, args); + (renv->svinst, renv->ehandler, location, user_prefix, fmt, args); } T_END; va_end(args); diff -r 6cca5f460f86 -r a4ea7e3f8d9e src/lib-sieve/sieve-runtime.h --- a/src/lib-sieve/sieve-runtime.h Thu Feb 16 22:07:32 2012 +0100 +++ b/src/lib-sieve/sieve-runtime.h Wed Feb 22 21:43:35 2012 +0100 @@ -15,6 +15,7 @@ struct sieve_instance *svinst; struct sieve_interpreter *interp; enum sieve_runtime_flags flags; + struct sieve_error_handler *ehandler; /* Executing script */ struct sieve_script *script; From pigeonhole at rename-it.nl Sun May 20 13:17:09 2012 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Sun, 20 May 2012 12:17:09 +0200 Subject: dovecot-2.2-pigeonhole: sieve-tools: forgot to enable debug in e... Message-ID: details: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/916f09fc6a5e changeset: 1614:916f09fc6a5e user: Stephan Bosch date: Mon Mar 19 22:33:12 2012 +0100 description: sieve-tools: forgot to enable debug in error handlers. diffstat: src/sieve-tools/sieve-filter.c | 1 + src/sieve-tools/sieve-test.c | 1 + 2 files changed, 2 insertions(+), 0 deletions(-) diffs (22 lines): diff -r 75c1a2fd9b26 -r 916f09fc6a5e src/sieve-tools/sieve-filter.c --- a/src/sieve-tools/sieve-filter.c Thu Mar 15 21:29:33 2012 +0100 +++ b/src/sieve-tools/sieve-filter.c Mon Mar 19 22:33:12 2012 +0100 @@ -483,6 +483,7 @@ ehandler = sieve_stderr_ehandler_create(svinst, 0); sieve_system_ehandler_set(ehandler); sieve_error_handler_accept_infolog(ehandler, verbose); + sieve_error_handler_accept_debuglog(ehandler, svinst->debug); /* Compile main sieve script */ if ( force_compile ) { diff -r 75c1a2fd9b26 -r 916f09fc6a5e src/sieve-tools/sieve-test.c --- a/src/sieve-tools/sieve-test.c Thu Mar 15 21:29:33 2012 +0100 +++ b/src/sieve-tools/sieve-test.c Mon Mar 19 22:33:12 2012 +0100 @@ -217,6 +217,7 @@ ehandler = sieve_stderr_ehandler_create(svinst, 0); sieve_system_ehandler_set(ehandler); sieve_error_handler_accept_infolog(ehandler, TRUE); + sieve_error_handler_accept_debuglog(ehandler, svinst->debug); /* Compile main sieve script */ if ( force_compile ) { From pigeonhole at rename-it.nl Sun May 20 13:17:09 2012 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Sun, 20 May 2012 12:17:09 +0200 Subject: dovecot-2.2-pigeonhole: Merged minor changes from Pigeonhole v0.... Message-ID: details: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/39730d5f39e4 changeset: 1616:39730d5f39e4 user: Stephan Bosch date: Sun May 20 11:33:49 2012 +0200 description: Merged minor changes from Pigeonhole v0.2 tree (causes a few duplicate commit messages). diffstat: .hgsigs | 1 + .hgtags | 1 + NEWS | 51 ++++++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 52 insertions(+), 1 deletions(-) diffs (76 lines): diff -r b88a6d76839b -r 39730d5f39e4 .hgsigs --- a/.hgsigs Wed Feb 29 22:51:56 2012 +0100 +++ b/.hgsigs Sun May 20 11:33:49 2012 +0200 @@ -3,3 +3,4 @@ 3ab2a125e1e2d478382c07853e99a5973d06afd6 0 iQEcBAABAgAGBQJNp1ztAAoJEATWKx49+7T0CJcH/24Txa1ynS5hBUhOuWTpUTGtm+9cMpWoQ33exiMR0pm8ycxsUQcKfRtO/cRHQX1CW3PqQs3DGZ31QdEEg0CyX8OsBbP/dwdEcnLRYF5BsJMyfy+Qnbhxn+wV0k9s9AUgZTdvPKrg1hFa6XS+6SE3N33AA4Y2eYYZGzFuDiSoN7fGx7PATCrobMsmp5WtBiKoy4WyP2SwDv/VgKy0PQTF+6+0t0MMCBSurLzpHk8dDuBonWIBgbJRM/sk9f+cYbU/ESRMcryZbbau9EwMQIQJfprGH6WP/gwysF0pu47zQERMuVt3fFzXUzrfxVpMOI7EkLgnF+Tes0vA7dKh1x+vvec= 0d071eaa6d5e2a9f524c94ddf1686f1b3091f604 0 iQEcBAABAgAGBQJOb8BjAAoJEATWKx49+7T0cAcH/3coc1MhQj8zUdC+NB3N8eUkQ3AF3QQgSfP9uXs9BhvPw70Ts9MLJiO54RhhYf/k9VxptzWk7MPJF47v4NEEKHkjDDMXtPbVOxHjNa2Ny8EAuWe4dv5X0faAlH4Ks58enDchCmunX1DgQtC1f+gHqVtvTpGAROFPqkBe5RGOJ0jQd+2hTTlf1BpLl44fiBdYd6350haX0KjDGNthX9ETVc3bnbdIiXSy7DPnn0ELhvTbgkl4Zu1tA778IJy/JjsCPb2YueX7LsksvxcSZHqv80Zd3JJhs5a3ZeHijN6twpe7VZD9FO+jPOKA1rr/HYwCv0KweKgmwVHCdaT+Mq4OLPc= 873baa85e2202f45a9c14fa21cccedc60f3715bc 0 iQEcBAABAgAGBQJOx9MCAAoJEATWKx49+7T09aUIANIKsuzM3bGhtGJ/UPIwzpOu39lEGCmHah6dMa+bDOoCZhuhASDdTuvRKXTfGC57GMu+NzBK6I7heFiPD3E4VTI4xOCK1azJ9G4SsiDEkQThucXqWBKDjPB0RgOEf6iefAkslXIU3cprJgattwpeXbUKiHjBhoYJFJ5j/GTx1B62ndvaTfMu1zF5UppiyRG1rQD7FLY4f6kANzSI2jOOCBs4UFH7ZKhafO1AeQfLNDvxdDczZafPZxrCIF+5JCNvQ6Xue/JrvRZQ0V9sxLQat7clUJ6I6Ejl5u5l1LF+VscWldfaQKwDdOktCVux84YGH8+XqXaukMiEg6j4hceAYIM= +fe7bd7ee6c2e33e38515cbeca7642135db8dea4b 0 iQEcBAABAgAGBQJPPXAPAAoJEATWKx49+7T0iqMH/3e+RKKmryOz5pak0cvdPcS/D9O9xl2l6SuoE2okTq/WOrDtZ1xDg0afg7t27D9mDfUY1hiSFS4ekN3WP620Gcb9wlL3FC+rLEYmiE8iSfZvsH+FeLa7n8NB+XdnAsXE1WdLQp5CSKEh3sXIod7Q04PL0uv/rimGS9jOGcAufW3y9QAYd+DVorPS4lV5Kz8qIqY9r/0lLqhJN1ukIJtClVkFanRljd+SfoHFFOSWbQjCKNxlOSWFhwJji7Mp091A1+N6JoZe4IMIlajMsM2Ypp726Y2LA/du+uRVFjKgta65eP9tfdrmVCJtrjIjvikowD5Zl80GuVRI5j44aQ7rJ3A= diff -r b88a6d76839b -r 39730d5f39e4 .hgtags --- a/.hgtags Wed Feb 29 22:51:56 2012 +0100 +++ b/.hgtags Sun May 20 11:33:49 2012 +0200 @@ -9,3 +9,4 @@ 3ab2a125e1e2d478382c07853e99a5973d06afd6 0.2.3 0d071eaa6d5e2a9f524c94ddf1686f1b3091f604 0.2.4 873baa85e2202f45a9c14fa21cccedc60f3715bc 0.2.5 +fe7bd7ee6c2e33e38515cbeca7642135db8dea4b 0.3.0 diff -r b88a6d76839b -r 39730d5f39e4 NEWS --- a/NEWS Wed Feb 29 22:51:56 2012 +0100 +++ b/NEWS Sun May 20 11:33:49 2012 +0200 @@ -1,6 +1,55 @@ -v0.3.0 [TO BE RELEASED] Stephan Bosch +v0.3.0 16-02-2012 Stephan Bosch + * Renamed sieve_global_path setting to sieve_default for clarity. Old name is + still recognized for backwards compatibility. Support for the ancient (pre + v1.1) name for this setting "global_script_path" is now dropped. + * Added means to prohibit use of redirect action. Setting sieve_max_redirects + to 0 now means that redirect is disallowed in stead of unlimited. Default + value remains four. + * Fixed interaction of Sieve include extension with ManageSieve. It is updated + to match new requirements in the draft include specification. Missing + included scripts are no longer an error at upload time. + * Updated RFC2822 header field body verification to exclude non-printing + characters (RFC5322). Only Sieve actions that can create unstructured header + values (currently enotify/mailto and editheader) are affected by this + change. + + Completed sieve-filter tool to a useful state. The sieve-filter tool + provides a means to (re)filter messages in a mailbox through a Sieve script. + + Implemented the Sieve editheader extension. It is now possible to add and + remove message headers from within Sieve. + + ManageSieve: added support for reading quoted and literal strings as a + stream. Fixes support for handing large SASL responses (analogous to similar + changes in Dovecot). It is now also allowed to use a quoted string for the + PUTSCRIPT script argument. + + Added code to cleanup tmp directory in Sieve storage directory (sieve_dir) + every once in a while. + + Added support for substituting the entire message during Sieve processing. + This is used for the filter action provided by the new sieve_extprograms + plugin (provided separately for now). The filter action allows passing the + message through an external program. + + Added support for restricting certain Sieve language extensions to + (admin-controled) global scripts. Restricted extensions can be configured + using the new sieve_global_extensions setting. This is particularly useful + for some of the Dovecot-specific (plugin-based) Sieve extensions, that can + be somewhat hazardous when under direct control of users (e.g. + sieve_extprograms). +v0.2.6 13-02-2012 Stephan Bosch + + * This release fixes unintentional behavior of the include extension. Included + scriptnames with a name like "name.sieve" would implicitly map to a script + file called "name.sieve" and not "name.sieve.sieve". Keep in mind that the + .sieve file extension has no meaning from within the Sieve language. A Sieve + script is always stored with an appended .sieve file extension, also when + the name already ends with a .sieve suffix. + IMPORTANT: Some installations have relied on this unintentional feature, so + check your script includes for issues before upgrading. + * Matched changes regarding auth_verbose setting in Dovecot. This means that + this release will only compile against Dovecot v2.0.18. + - Fixed problem in ManageSieve that caused it to omit a WARNINGS response code + when the uploaded script compiled with warnings. + - Made sure that locations of Sieve error never report `line 0'. + - Fixed potential segfault occurring when interpreter initialization fails. v0.2.5 19-11-2011 Stephan Bosch From pigeonhole at rename-it.nl Sun May 20 13:17:09 2012 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Sun, 20 May 2012 12:17:09 +0200 Subject: dovecot-2.2-pigeonhole: managesieve-login: fixed x86 compile war... Message-ID: details: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/75c1a2fd9b26 changeset: 1613:75c1a2fd9b26 user: Stephan Bosch date: Thu Mar 15 21:29:33 2012 +0100 description: managesieve-login: fixed x86 compile warning. diffstat: src/managesieve-login/client-authenticate.c | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diffs (35 lines): diff -r e55350fb6786 -r 75c1a2fd9b26 src/managesieve-login/client-authenticate.c --- a/src/managesieve-login/client-authenticate.c Thu Mar 08 11:02:05 2012 +0100 +++ b/src/managesieve-login/client-authenticate.c Thu Mar 15 21:29:33 2012 +0100 @@ -149,6 +149,7 @@ bool fatal; const unsigned char *data; size_t size; + uoff_t resp_size; int ret; if ( i_stream_read(client->input) == -1 ) { @@ -200,7 +201,7 @@ } if ( !managesieve_arg_get_string_stream - (&args[0], &msieve_client->auth_response_input) + (&args[0], &msieve_client->auth_response_input) || !MANAGESIEVE_ARG_IS_EOL(&args[1]) ) { if ( !initial ) *error_r = "Invalid AUTHENTICATE client response."; @@ -211,11 +212,11 @@ } if ( i_stream_get_size - (msieve_client->auth_response_input, FALSE, &size) <= 0 ) - size = 0; + (msieve_client->auth_response_input, FALSE, &resp_size) <= 0 ) + resp_size = 0; if (client->auth_response == NULL) - client->auth_response = str_new(default_pool, I_MAX(size+1, 256)); + client->auth_response = str_new(default_pool, I_MAX(resp_size+1, 256)); } while ( (ret=i_stream_read_data From pigeonhole at rename-it.nl Sun May 20 13:17:09 2012 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Sun, 20 May 2012 12:17:09 +0200 Subject: dovecot-2.2-pigeonhole: Updated dovecot.m4. Message-ID: details: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/a88c63b2d0b0 changeset: 1615:a88c63b2d0b0 user: Stephan Bosch date: Mon Mar 26 21:07:14 2012 +0200 description: Updated dovecot.m4. diffstat: m4/dovecot.m4 | 22 +++++++++++----------- 1 files changed, 11 insertions(+), 11 deletions(-) diffs (53 lines): diff -r 916f09fc6a5e -r a88c63b2d0b0 m4/dovecot.m4 --- a/m4/dovecot.m4 Mon Mar 19 22:33:12 2012 +0100 +++ b/m4/dovecot.m4 Mon Mar 26 21:07:14 2012 +0200 @@ -6,7 +6,7 @@ # unlimited permission to copy and/or distribute it, with or without # modifications, as long as this notice is preserved. -# serial 4 +# serial 5 AC_DEFUN([DC_DOVECOT_MODULEDIR],[ AC_ARG_WITH(moduledir, @@ -48,13 +48,13 @@ AC_ARG_WITH(dovecot-install-dirs, [AC_HELP_STRING([--with-dovecot-install-dirs], - [Use install directories configured for Dovecot (default)])], - if test x$withval = xno; then - use_install_dirs=no - else - use_install_dirs=yes - fi, - use_install_dirs=yes) + [Use install directories configured for Dovecot (default)])], + if test x$withval = xno; then + use_install_dirs=no + else + use_install_dirs=yes + fi, + use_install_dirs=yes) AC_MSG_CHECKING([for dovecot-config in "$dovecotdir"]) if test -f "$dovecotdir/dovecot-config"; then @@ -72,7 +72,7 @@ cd $old DISTCHECK_CONFIGURE_FLAGS="--with-dovecot=$abs_dovecotdir --without-dovecot-install-dirs" - eval `grep -i '^dovecot_[[a-z]]*=' "$dovecotdir"/dovecot-config` + eval `grep -i '^dovecot_[[a-z_]]*=' "$dovecotdir"/dovecot-config` eval `grep '^LIBDOVECOT[[A-Z_]]*=' "$dovecotdir"/dovecot-config` if test "$use_install_dirs" = "no"; then @@ -83,8 +83,8 @@ dovecot_moduledir='$(moduledir)' fi - AX_SUBST_L([DISTCHECK_CONFIGURE_FLAGS], [dovecot_moduledir], [dovecot_pkgincludedir], [dovecot_pkglibexecdir], [dovecot_pkglibdir], [dovecot_docdir]) - AX_SUBST_L([DOVECOT_CFLAGS], [DOVECOT_LIBS], [DOVECOT_SSL_LIBS]) + AX_SUBST_L([DISTCHECK_CONFIGURE_FLAGS], [dovecotdir], [dovecot_moduledir], [dovecot_pkgincludedir], [dovecot_pkglibexecdir], [dovecot_pkglibdir], [dovecot_docdir]) + AX_SUBST_L([DOVECOT_CFLAGS], [DOVECOT_LIBS], [DOVECOT_SSL_LIBS], [DOVECOT_SQL_LIBS]) AX_SUBST_L([LIBDOVECOT], [LIBDOVECOT_LOGIN], [LIBDOVECOT_SQL], [LIBDOVECOT_LDA], [LIBDOVECOT_STORAGE]) AX_SUBST_L([LIBDOVECOT_DEPS], [LIBDOVECOT_LOGIN_DEPS], [LIBDOVECOT_SQL_DEPS], [LIBDOVECOT_LDA_DEPS], [LIBDOVECOT_STORAGE_DEPS]) AX_SUBST_L([LIBDOVECOT_INCLUDE], [LIBDOVECOT_LDA_INCLUDE], [LIBDOVECOT_SERVICE_INCLUDE], [LIBDOVECOT_STORAGE_INCLUDE], [LIBDOVECOT_LOGIN_INCLUDE], [LIBDOVECOT_CONFIG_INCLUDE]) From pigeonhole at rename-it.nl Sun May 20 13:17:08 2012 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Sun, 20 May 2012 12:17:08 +0200 Subject: dovecot-2.2-pigeonhole: Added tag 0.3.0 for changeset fe7bd7ee6c2e Message-ID: details: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/abf95290dc4d changeset: 1607:abf95290dc4d user: Stephan Bosch date: Thu Feb 16 22:02:56 2012 +0100 description: Added tag 0.3.0 for changeset fe7bd7ee6c2e diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r fe7bd7ee6c2e -r abf95290dc4d .hgtags --- a/.hgtags Thu Feb 16 22:02:35 2012 +0100 +++ b/.hgtags Thu Feb 16 22:02:56 2012 +0100 @@ -9,3 +9,4 @@ 3ab2a125e1e2d478382c07853e99a5973d06afd6 0.2.3 0d071eaa6d5e2a9f524c94ddf1686f1b3091f604 0.2.4 873baa85e2202f45a9c14fa21cccedc60f3715bc 0.2.5 +fe7bd7ee6c2e33e38515cbeca7642135db8dea4b 0.3.0 From pigeonhole at rename-it.nl Sun May 20 13:17:08 2012 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Sun, 20 May 2012 12:17:08 +0200 Subject: dovecot-2.2-pigeonhole: Added signature for changeset fe7bd7ee6c2e Message-ID: details: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/6cca5f460f86 changeset: 1608:6cca5f460f86 user: Stephan Bosch date: Thu Feb 16 22:07:32 2012 +0100 description: Added signature for changeset fe7bd7ee6c2e diffstat: .hgsigs | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r abf95290dc4d -r 6cca5f460f86 .hgsigs --- a/.hgsigs Thu Feb 16 22:02:56 2012 +0100 +++ b/.hgsigs Thu Feb 16 22:07:32 2012 +0100 @@ -3,3 +3,4 @@ 3ab2a125e1e2d478382c07853e99a5973d06afd6 0 iQEcBAABAgAGBQJNp1ztAAoJEATWKx49+7T0CJcH/24Txa1ynS5hBUhOuWTpUTGtm+9cMpWoQ33exiMR0pm8ycxsUQcKfRtO/cRHQX1CW3PqQs3DGZ31QdEEg0CyX8OsBbP/dwdEcnLRYF5BsJMyfy+Qnbhxn+wV0k9s9AUgZTdvPKrg1hFa6XS+6SE3N33AA4Y2eYYZGzFuDiSoN7fGx7PATCrobMsmp5WtBiKoy4WyP2SwDv/VgKy0PQTF+6+0t0MMCBSurLzpHk8dDuBonWIBgbJRM/sk9f+cYbU/ESRMcryZbbau9EwMQIQJfprGH6WP/gwysF0pu47zQERMuVt3fFzXUzrfxVpMOI7EkLgnF+Tes0vA7dKh1x+vvec= 0d071eaa6d5e2a9f524c94ddf1686f1b3091f604 0 iQEcBAABAgAGBQJOb8BjAAoJEATWKx49+7T0cAcH/3coc1MhQj8zUdC+NB3N8eUkQ3AF3QQgSfP9uXs9BhvPw70Ts9MLJiO54RhhYf/k9VxptzWk7MPJF47v4NEEKHkjDDMXtPbVOxHjNa2Ny8EAuWe4dv5X0faAlH4Ks58enDchCmunX1DgQtC1f+gHqVtvTpGAROFPqkBe5RGOJ0jQd+2hTTlf1BpLl44fiBdYd6350haX0KjDGNthX9ETVc3bnbdIiXSy7DPnn0ELhvTbgkl4Zu1tA778IJy/JjsCPb2YueX7LsksvxcSZHqv80Zd3JJhs5a3ZeHijN6twpe7VZD9FO+jPOKA1rr/HYwCv0KweKgmwVHCdaT+Mq4OLPc= 873baa85e2202f45a9c14fa21cccedc60f3715bc 0 iQEcBAABAgAGBQJOx9MCAAoJEATWKx49+7T09aUIANIKsuzM3bGhtGJ/UPIwzpOu39lEGCmHah6dMa+bDOoCZhuhASDdTuvRKXTfGC57GMu+NzBK6I7heFiPD3E4VTI4xOCK1azJ9G4SsiDEkQThucXqWBKDjPB0RgOEf6iefAkslXIU3cprJgattwpeXbUKiHjBhoYJFJ5j/GTx1B62ndvaTfMu1zF5UppiyRG1rQD7FLY4f6kANzSI2jOOCBs4UFH7ZKhafO1AeQfLNDvxdDczZafPZxrCIF+5JCNvQ6Xue/JrvRZQ0V9sxLQat7clUJ6I6Ejl5u5l1LF+VscWldfaQKwDdOktCVux84YGH8+XqXaukMiEg6j4hceAYIM= +fe7bd7ee6c2e33e38515cbeca7642135db8dea4b 0 iQEcBAABAgAGBQJPPXAPAAoJEATWKx49+7T0iqMH/3e+RKKmryOz5pak0cvdPcS/D9O9xl2l6SuoE2okTq/WOrDtZ1xDg0afg7t27D9mDfUY1hiSFS4ekN3WP620Gcb9wlL3FC+rLEYmiE8iSfZvsH+FeLa7n8NB+XdnAsXE1WdLQp5CSKEh3sXIod7Q04PL0uv/rimGS9jOGcAufW3y9QAYd+DVorPS4lV5Kz8qIqY9r/0lLqhJN1ukIJtClVkFanRljd+SfoHFFOSWbQjCKNxlOSWFhwJji7Mp091A1+N6JoZe4IMIlajMsM2Ypp726Y2LA/du+uRVFjKgta65eP9tfdrmVCJtrjIjvikowD5Zl80GuVRI5j44aQ7rJ3A= From pigeonhole at rename-it.nl Sun May 20 13:17:09 2012 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Sun, 20 May 2012 12:17:09 +0200 Subject: dovecot-2.2-pigeonhole: Finished documentation example for SQL d... Message-ID: details: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/d07dad1ad41c changeset: 1619:d07dad1ad41c user: Stephan Bosch date: Sun Apr 22 15:36:06 2012 +0200 description: Finished documentation example for SQL dict script location type. diffstat: doc/script-location-dict.txt | 63 +++++++++++++++++++++++++++++++++++++++++-- 1 files changed, 60 insertions(+), 3 deletions(-) diffs (75 lines): diff -r 3161dcf7cd8e -r d07dad1ad41c doc/script-location-dict.txt --- a/doc/script-location-dict.txt Sun Apr 22 15:11:14 2012 +0200 +++ b/doc/script-location-dict.txt Sun Apr 22 15:36:06 2012 +0200 @@ -78,11 +78,68 @@ sieve = dict:file:/etc/dovecot/sieve.dict;name=keep;bindir=~/.sieve-bin } -Binaries are stored in the ~/.sieve-bin directory. +The Sieve script named "keep" is retrieved from the file dict as the main +script. Binaries are stored in the ~/.sieve-bin directory. Example 2 --------- -This example uses a sqlite database. +This example uses a PostgreSQL database. Our database contains the following +table: -FIXME: +CREATE TABLE user_sieve_scripts ( + id integer, + username varchar(40), + script_name varchar(256), + script_data varchar(10240), + + PRIMARY KEY (id), + UNIQUE(username, script_name) +); + +We create a file /etc/dovecot/dict-sieve-sql.conf with the following content: + +connect = host=localhost dbname=dovecot user=dovecot password=password +map { + pattern = priv/sieve/name/$script_name + table = user_sieve_scripts + username_field = username + value_field = id + fields { + script_name = $script_name + } +} +map { + pattern = priv/sieve/data/$id + table = user_sieve_scripts + username_field = username + value_field = script_data + fields { + id = $id + } +} + +These are the mappings used by the SQL dict. The first mapping is the name query +that yields the id of the Sieve script. The second mapping is the query used to +retrieve the Sieve script itself. + +Much like the dict configuration for mailbox quota, it is often not possible to +directly use an SQL dict because the SQL drivers are not linked to binaries such +as dovecot-lda and lmtp. You need to use the dict proxy service. Add the dict +URI to the dict section (typically located in your main dovecot.conf): + +dict { + sieve = pgsql:/etc/dovecot/dict-sieve-sql.conf.ext +} + +To use this SQL dict for the main active script, you can change the +configuration as follows (e.g. in /etc/dovecot/conf.d/90-sieve.conf): + +plugin { + sieve = dict:proxy::sieve;name=active;bindir=~/.sieve-bin +} + +This uses the proxy dict uri `proxy::sieve'. This refers to the `sieve =' entry +in the dict {...} section above. With this configuration, a Sieve script called +"main" is retrieved from the SQL dict. Binaries are stored in the ~/.sieve-bin +directory. From pigeonhole at rename-it.nl Sun May 20 13:17:09 2012 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Sun, 20 May 2012 12:17:09 +0200 Subject: dovecot-2.2-pigeonhole: LDA Sieve: fixed bug in debug message. Message-ID: details: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/3161dcf7cd8e changeset: 1618:3161dcf7cd8e user: Stephan Bosch date: Sun Apr 22 15:11:14 2012 +0200 description: LDA Sieve: fixed bug in debug message. diffstat: src/plugins/lda-sieve/lda-sieve-plugin.c | 14 ++++++++------ 1 files changed, 8 insertions(+), 6 deletions(-) diffs (25 lines): diff -r f7ea71241032 -r 3161dcf7cd8e src/plugins/lda-sieve/lda-sieve-plugin.c --- a/src/plugins/lda-sieve/lda-sieve-plugin.c Sun Apr 22 11:30:28 2012 +0200 +++ b/src/plugins/lda-sieve/lda-sieve-plugin.c Sun Apr 22 15:11:14 2012 +0200 @@ -631,13 +631,15 @@ if ( default_location != NULL ) { srctx.main_script = sieve_script_create_as (svinst, default_location, "main script", master_ehandler, &error); + + if ( srctx.main_script == NULL && error == SIEVE_ERROR_NOT_FOUND && + debug ) { + sieve_sys_debug(svinst, "default user script %s doesn't exist", + default_location); + } + } else { + sieve_sys_debug(svinst, "no default script configured for user"); } - - if ( srctx.main_script == NULL && error == SIEVE_ERROR_NOT_FOUND && - debug ) { - sieve_sys_debug(svinst, "default user script %s doesn't exist", - default_location); - } } if ( debug && srctx.main_script == NULL ) { From pigeonhole at rename-it.nl Sun May 20 13:17:09 2012 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Sun, 20 May 2012 12:17:09 +0200 Subject: dovecot-2.2-pigeonhole: lib-managesieve: changed EPROTO error to... Message-ID: details: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/ef5045159e6b changeset: 1622:ef5045159e6b user: Stephan Bosch date: Tue Apr 24 13:08:36 2012 +0200 description: lib-managesieve: changed EPROTO error to EIO in ManageSieve string stream because it is apparently not known in BSD. diffstat: src/lib-managesieve/managesieve-parser.c | 6 +++--- src/managesieve-login/client-authenticate.c | 2 +- src/managesieve/cmd-putscript.c | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diffs (54 lines): diff -r 2ff0a5c5c892 -r ef5045159e6b src/lib-managesieve/managesieve-parser.c --- a/src/lib-managesieve/managesieve-parser.c Sun Apr 22 17:16:08 2012 +0200 +++ b/src/lib-managesieve/managesieve-parser.c Tue Apr 24 13:08:36 2012 +0200 @@ -636,7 +636,7 @@ if (ret <= 0 && (ret != -2 || stream->skip == 0)) { if ( stream->istream.eof && stream->istream.stream_errno == 0 ) { stream->istream.eof = 0; - stream->istream.stream_errno = EPROTO; + stream->istream.stream_errno = EIO; } else { stream->istream.stream_errno = stream->parent->stream_errno; stream->istream.eof = stream->parent->eof; @@ -673,7 +673,7 @@ if ( !IS_QUOTED_SPECIAL(data[i]) ) { qsstream->parser->error = "Escaped quoted-string character is not a QUOTED-SPECIAL."; - stream->istream.stream_errno = EPROTO; + stream->istream.stream_errno = EIO; ret = -1; break; } @@ -682,7 +682,7 @@ if ( (data[i] & 0x80) == 0 && ( data[i] == '\r' || data[i] == '\n' ) ) { qsstream->parser->error = "String contains invalid character."; - stream->istream.stream_errno = EPROTO; + stream->istream.stream_errno = EIO; ret = -1; break; } diff -r 2ff0a5c5c892 -r ef5045159e6b src/managesieve-login/client-authenticate.c --- a/src/managesieve-login/client-authenticate.c Sun Apr 22 17:16:08 2012 +0200 +++ b/src/managesieve-login/client-authenticate.c Tue Apr 24 13:08:36 2012 +0200 @@ -235,7 +235,7 @@ if ( ret == 0 ) return 0; if ( msieve_client->auth_response_input->stream_errno != 0 ) { - if ( msieve_client->auth_response_input->stream_errno == EPROTO ) { + if ( msieve_client->auth_response_input->stream_errno == EIO ) { error = managesieve_parser_get_error(msieve_client->parser, &fatal); if (error != NULL ) { if (fatal) { diff -r 2ff0a5c5c892 -r ef5045159e6b src/managesieve/cmd-putscript.c --- a/src/managesieve/cmd-putscript.c Sun Apr 22 17:16:08 2012 +0200 +++ b/src/managesieve/cmd-putscript.c Tue Apr 24 13:08:36 2012 +0200 @@ -371,7 +371,7 @@ bool all_written = FALSE; if ( ctx->script_size == 0 ) { - if ( ctx->input->stream_errno == EPROTO ) { + if ( ctx->input->stream_errno == EIO ) { bool fatal; const char *parse_error; From pigeonhole at rename-it.nl Sun May 20 13:17:09 2012 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Sun, 20 May 2012 12:17:09 +0200 Subject: dovecot-2.2-pigeonhole: Added support for retrieving Sieve scrip... Message-ID: details: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/f7ea71241032 changeset: 1617:f7ea71241032 user: Stephan Bosch date: Sun Apr 22 11:30:28 2012 +0200 description: Added support for retrieving Sieve scripts from dict lookup. - Built generic interface for alternative script sources. - Implemented dict script location. NOTE: ManageSieve will not work with this yet, nor will sieve_before/sieve_after. diffstat: INSTALL | 75 +- TODO | 16 +- doc/script-location-dict.txt | 88 + src/lib-sieve-tool/sieve-tool.c | 15 +- src/lib-sieve/Makefile.am | 3 + src/lib-sieve/plugins/enotify/mailto/ntfy-mailto.c | 6 +- src/lib-sieve/plugins/environment/ext-environment-common.c | 12 +- src/lib-sieve/plugins/environment/sieve-ext-environment.h | 3 +- src/lib-sieve/plugins/include/cmd-include.c | 19 +- src/lib-sieve/plugins/include/ext-include-binary.c | 53 +- src/lib-sieve/plugins/include/ext-include-common.c | 22 +- src/lib-sieve/plugins/include/ext-include-common.h | 2 +- src/lib-sieve/plugins/notify/cmd-notify.c | 2 +- src/lib-sieve/plugins/vacation/cmd-vacation.c | 2 +- src/lib-sieve/sieve-actions.c | 15 +- src/lib-sieve/sieve-binary-file.c | 29 +- src/lib-sieve/sieve-binary.c | 51 +- src/lib-sieve/sieve-binary.h | 18 +- src/lib-sieve/sieve-common.h | 13 +- src/lib-sieve/sieve-message.c | 5 +- src/lib-sieve/sieve-message.h | 3 +- src/lib-sieve/sieve-script-dict.c | 283 +++++ src/lib-sieve/sieve-script-file.c | 397 +++++++ src/lib-sieve/sieve-script-file.h | 31 + src/lib-sieve/sieve-script-private.h | 89 +- src/lib-sieve/sieve-script.c | 629 +++++++----- src/lib-sieve/sieve-script.h | 59 +- src/lib-sieve/sieve-settings.h | 15 +- src/lib-sieve/sieve-types.h | 37 +- src/lib-sieve/sieve.c | 91 +- src/lib-sieve/sieve.h | 37 +- src/lib-sievestorage/sieve-storage-list.c | 5 + src/lib-sievestorage/sieve-storage-quota.c | 5 + src/lib-sievestorage/sieve-storage-save.c | 1 + src/lib-sievestorage/sieve-storage-script.c | 40 +- src/lib-sievestorage/sieve-storage.c | 35 + src/managesieve/cmd-getscript.c | 23 +- src/managesieve/managesieve-capabilities.c | 16 +- src/managesieve/managesieve-client.c | 37 +- src/plugins/lda-sieve/lda-sieve-plugin.c | 673 ++++++------ src/sieve-tools/sieve-filter.c | 7 +- src/sieve-tools/sieve-test.c | 23 +- src/sieve-tools/sievec.c | 5 +- src/testsuite/testsuite-binary.c | 11 +- src/testsuite/testsuite-script.c | 9 +- src/testsuite/testsuite-smtp.c | 4 +- src/testsuite/testsuite-smtp.h | 6 +- src/testsuite/testsuite.c | 2 - tests/extensions/environment/basic.svtest | 2 +- 49 files changed, 2086 insertions(+), 938 deletions(-) diffs (truncated from 4714 to 300 lines): diff -r a88c63b2d0b0 -r f7ea71241032 INSTALL --- a/INSTALL Mon Mar 26 21:07:14 2012 +0200 +++ b/INSTALL Sun Apr 22 11:30:28 2012 +0200 @@ -85,23 +85,21 @@ plugin section of the config file (default values are shown if applicable): sieve = ~/.dovecot.sieve - The path to the user's main active script. + The location of the user's main active script. sieve_default = - The path to the default personal sieve script file, which gets executed ONLY - if user's private Sieve script does no exist, e.g. - /var/lib/dovecot/default.sieve. This is usually a global script, so sure to - pre-compile this script manually using the sievec command line tool, as - explained in the README file. - - sieve_global_path = - This is the DEPRECATED former name of the sieve_default setting. + The location of the default personal sieve script file, which gets executed + ONLY if user's private Sieve script does no exist, e.g. + /var/lib/dovecot/default.sieve. This is usually a global script, so be sure + to pre-compile this script manually using the sievec command line tool, as + explained in the README file. This setting used to be called + `sieve_global_path', but that name is now deprecated. sieve_global_dir = - Directory for :global include scripts for the Sieve include extension. + Location for :global include scripts for the Sieve include extension. - sieve_dir = ~/ - Directory for :personal include scripts for the Sieve include extension. + sieve_dir = ~/sieve + Location for :personal include scripts for the Sieve include extension. sieve_extensions = Which Sieve language extensions are available to users. By default, all @@ -131,6 +129,12 @@ (wiki2.dovecot.org) or the pigeonhole website (http://pigeonhole.dovecot.org) for available plugins. + sieve_user_log = + The path to the file where the user log file is written. If not configured, a + default location is used. If the main user's personal Sieve (as configured + with sieve=) is a file, the logfile is set to .log by default. If + it is not a file, the default user log file is ~/.dovecot.sieve.log. + recipient_delimiter = + The separator that is expected between the :user and :detail address parts introduced by the subaddress extension. This may also be a sequence of @@ -176,7 +180,48 @@ The maximum number of redirect actions that can be performed during a single script execution. If set to 0, no redirect actions are allowed. -Sieve Interpreter - Per-user Sieve script location +Sieve Interpreter - Script Locations +------------------------------------ + +The location of Sieve scripts is not limited to the file system. The Sieve +interpreter can be extended to retrieve Sieve scripts from other sources as +well, such as a database. Currently, all settings that are used to obtain the +location of a single Sieve script, such as sieve=, sieve_default=, sieve_dir= +and sieve_global_dir= accept the following extended syntax: + +location = [:]path[;