dovecot-2.2-pigeonhole: Merged changes from Pigeonhole v0.3 tree.
pigeonhole at rename-it.nl
pigeonhole at rename-it.nl
Mon Jan 14 13:58:05 EET 2013
details: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/99eec511aa2c
changeset: 1712:99eec511aa2c
user: Stephan Bosch <stephan at rename-it.nl>
date: Mon Jan 14 12:57:58 2013 +0100
description:
Merged changes from Pigeonhole v0.3 tree.
diffstat:
doc/extensions/vnd.dovecot.duplicate.txt | 28 +-
doc/rfc/spec-bosch-sieve-duplicate.txt | 356 +++++++--
doc/rfc/xml/reference.IMAP4FLAGS.xml | 15 +
doc/rfc/xml/reference.INCLUDE.xml | 17 +
doc/rfc/xml/reference.MAILBOX.xml | 15 +
doc/rfc/xml/reference.VACATION.xml | 17 +
doc/rfc/xml/spec-bosch-sieve-duplicate.xml | 185 ++++-
src/lib-sieve/Makefile.am | 4 +-
src/lib-sieve/plugins/vacation/cmd-vacation.c | 76 +-
src/lib-sieve/plugins/vnd.dovecot/duplicate/ext-duplicate-common.c | 201 ++++-
src/lib-sieve/plugins/vnd.dovecot/duplicate/ext-duplicate-common.h | 13 +-
src/lib-sieve/plugins/vnd.dovecot/duplicate/ext-duplicate.c | 3 +-
src/lib-sieve/plugins/vnd.dovecot/duplicate/tst-duplicate.c | 275 ++++++-
src/lib-sieve/sieve-config.h | 2 +
src/lib-sieve/sieve-message.c | 5 +-
tests/extensions/vnd.dovecot/duplicate/errors.svtest | 2 +-
tests/extensions/vnd.dovecot/duplicate/errors/syntax.sieve | 5 +-
tests/extensions/vnd.dovecot/duplicate/execute.svtest | 10 +-
18 files changed, 957 insertions(+), 272 deletions(-)
diffs (truncated from 1695 to 300 lines):
diff -r c937a4060ee2 -r 99eec511aa2c doc/extensions/vnd.dovecot.duplicate.txt
--- a/doc/extensions/vnd.dovecot.duplicate.txt Wed Jan 09 00:55:49 2013 +0100
+++ b/doc/extensions/vnd.dovecot.duplicate.txt Mon Jan 14 12:57:58 2013 +0100
@@ -8,13 +8,11 @@
Description
===========
-Sieve (RFC 5228) is a highly extensible machine language specifically tailored
-for internet message filtering. For the Dovecot Secure IMAP server, Sieve
-support is provided by the Pigeonhole Sieve plugin. The vnd.dovecot.duplicate
-extension augments the Sieve filtering implementation with a test to verify
-whether a message was received earlier already based on its Message-ID. This can
-be used to prevent duplicate deliveries, e.g. caused by mailinglists when people
-reply both to the mailinglist and the user directly.
+The vnd.dovecot.duplicate extension augments the Sieve filtering implementation
+with a test to verify whether the evaluated string value was seen before in an
+earlier execution of the Sieve script. The main application for this new test is
+detecting and handling duplicate message deliveries, e.g. as caused by
+mailinglists when people reply both to the mailinglist and the user directly.
This extension is specific to the Pigeonhole Sieve implementation for the
Dovecot Secure IMAP server. It will therefore most likely not be supported by
@@ -35,14 +33,17 @@
=============
The "vnd.dovecot.duplicate" extension is not enabled by default and thus it
-needs to be enabled explicitly.
+needs to be enabled explicitly by adding it to the `sieve_extensions' or the
+`sieve_global_extensions' setting.
The following configuration settings are used:
-sieve_duplicate_period = 1d
- This option specifies after what period of time Message-IDs are purged from
- the duplicate database. The period is specified in s(econds), unless followed
- by a d(ay), h(our) or m(inute) specifier character.
+sieve_duplicate_default_period = 12h
+sieve_duplicate_max_period = 2d
+ These options respectively specify the default and the maximum value for the
+ period after which tracked values are purged from the duplicate tracking
+ database. The period is specified in s(econds), unless followed by a d(ay),
+ h(our) or m(inute) specifier character.
Example
=======
@@ -52,5 +53,6 @@
sieve_extensions = +vnd.dovecot.duplicate
- sieve_duplicate_period = 6h
+ sieve_duplicate_default_period = 1h
+ sieve_duplicate_max_period = 1d
}
diff -r c937a4060ee2 -r 99eec511aa2c doc/rfc/spec-bosch-sieve-duplicate.txt
--- a/doc/rfc/spec-bosch-sieve-duplicate.txt Wed Jan 09 00:55:49 2013 +0100
+++ b/doc/rfc/spec-bosch-sieve-duplicate.txt Mon Jan 14 12:57:58 2013 +0100
@@ -2,7 +2,7 @@
Pigeonhole Project S. Bosch
- February 25, 2012
+ October 17, 2012
Sieve Email Filtering: Detecting Duplicate Deliveries
@@ -10,23 +10,65 @@
Abstract
This document defines a new vendor-defined test command "duplicate"
- for the "Sieve" email filtering language that tests whether an e-mail
- message is a duplicate, i.e. whether it was seen before by the
- delivery agent. Users can use this new test to remove duplicate
- deliveries commonly caused by mailing list subscriptions or mail
- account aliases.
+ for the "Sieve" email filtering language. It can be used to test
+ whether a particular string value is a duplicate, i.e. whether it was
+ seen before by the delivery agent that is executing the Sieve script.
+ The main application for this new test is detecting duplicate message
+ deliveries commonly caused by mailing list subscriptions or
+ redirected mail addresses.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Bosch [Page 1]
+
+ Sieve: Detecting Duplicate Deliveries October 2012
Table of Contents
- 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 2
- 2. Conventions Used in This Document . . . . . . . . . . . . . . . 2
- 3. Test "duplicate" . . . . . . . . . . . . . . . . . . . . . . . 2
- 4. Sieve Capability Strings . . . . . . . . . . . . . . . . . . . 3
- 5. Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
- 6. Security Considerations . . . . . . . . . . . . . . . . . . . . 3
- 7. Normative References . . . . . . . . . . . . . . . . . . . . . 3
- Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 3
+ 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
+ 2. Conventions Used in This Document . . . . . . . . . . . . . . . 3
+ 3. Test "duplicate" . . . . . . . . . . . . . . . . . . . . . . . 4
+ 4. Sieve Capability Strings . . . . . . . . . . . . . . . . . . . 5
+ 5. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
+ 6. Security Considerations . . . . . . . . . . . . . . . . . . . . 6
+ 7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 7
+ 7.1. Normative References . . . . . . . . . . . . . . . . . . . 7
+ 7.2. Informative References . . . . . . . . . . . . . . . . . . 7
+ Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 7
@@ -52,26 +94,52 @@
-Bosch [Page 1]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Bosch [Page 2]
- Sieve: Detecting Duplicate Deliveries February 2012
+ Sieve: Detecting Duplicate Deliveries October 2012
1. Introduction
This is an extension to the Sieve filtering language defined by RFC
- 5228 [SIEVE]. It adds a test to determine whether a message was seen
- before by the delivery agent based on the Message-ID header.
+ 5228 [SIEVE]. It adds a test to determine whether a certain string
+ value was seen before by the delivery agent in an earlier execution
+ of the Sieve script. This can be used to detect and handle duplicate
+ message deliveries.
Duplicate deliveries are a common side-effect of being subscribed to
- a mailing list. If a member of the list decides to reply to both the
- user and the mailing list itself, the user will get a copy of the
- message directly and through mailing list. In another scenario, the
- user has several aliases for his mail account and one of his contacts
- sends the message to multiple addresses that eventually map to the
- same account. Using the vnd.dovecot.duplicate extension, users have
- the means to detect such duplicates and deal with these
- appropriately, e.g. by discarding them.
+ a mailing list. For example, if a member of the list decides to
+ reply to both the user and the mailing list itself, the user will get
+ a copy of the message directly and through mailing list. Also, if
+ someone cross-posts over several mailing lists to which the user is
+ subscribed, the user will receive a copy from each of those lists.
+ In another scenario, the user has several redirected mail addresses
+ all pointing to his main mail account. If one of the user's contacts
+ sends the message to more than one of those addresses, the user will
+ receive more than a single copy. Using the "vnd.dovecot.duplicate"
+ extension, users have the means to detect and handle such duplicates,
+ e.g. by discarding them or putting them in a special folder.
+
+ Duplicate messages are normally detected using the Message-ID header
+ field, which is required to be unique for each message. However, the
+ "duplicate" test is flexible enough to use different (weaker)
+ criteria for defining what makes a message a duplicate, for example
+ based on the subject line. Also, other applications of this new test
+ command are possible, as long as the tracked value is a string.
This extension is specific to the Pigeonhole Sieve implementation for
the Dovecot Secure IMAP server. It will therefore most likely not be
@@ -89,31 +157,102 @@
arguments syntax.
+
+
+
+
+
+
+
+Bosch [Page 3]
+
+ Sieve: Detecting Duplicate Deliveries October 2012
+
+
3. Test "duplicate"
- Usage: "duplicate" [<name: string>]
+ Usage: "duplicate" [":seconds" <timeout: number>]
+ [":header" <header-name: string> /
+ ":value" <value: string>]
+ [":handle" <handle: string>]
- The "duplicate" test keeps track of which Message-ID values were seen
- before by this test in an earlier delivery operation. It evaluates
- to "true" when the Message-ID header of the current message was seen
- before. If it is not known, the test evaluates to "false" and the
- Message-ID is added to a persistent internal tracking list.
- Implementations SHOULD limit the number of messages that are tracked
- and SHOULD let Message-ID entries expire after some short period of
- time.
+ The "duplicate" test keeps track of which values were seen before by
+ this test in an earlier execution of this Sieve script. In its basic
+ form, the tested value is the content of the Message-ID header of the
+ message. This way, this test can be used to detect duplicate
+ deliveries of the same message. It can also detect duplicate
+ deliveries based on other message header fields if requested and it
+ can even use a user-provided string value, e.g. as composed from text
+ extracted from the message using the "variables" [VARIABLES]
+ extension.
- Using the "name" argument, the duplicate test can be employed for
- multiple independent purposes. Only when the Message-ID was seen
- before in an earlier script execution by a duplicate test with the
+ The "duplicate" test evaluates to "true" when the provided value was
+ seen before in an earlier Sieve execution for a previous message
+ delivery. If the value was not seen earlier, the test evaluates to
+ "false".
+ As a side-effect, the "duplicate" test adds the evaluated value to an
+ internal duplicate tracking list, so that the test will evaluate to
+ "true" the next time the Sieve script is executed and the same value
+ is encountered. Note that the "duplicate" test MUST only check for
+ duplicates amongst values encountered in previous executions of the
+ Sieve script; it MUST NOT consider values encountered earlier in the
+ current Sieve script execution as potential duplicates. This means
+ that all "duplicate" tests in a Sieve script execution, including
+ those located in scripts included using the "include" [INCLUDE]
+ extension, MUST yield the same result if the arguments are identical.
+ Implementations MUST prevent adding values to the internal duplicate
+ tracking list when the Sieve script execution fails. For example,
+ this can be implemented by deferring the definitive modification of
+ the tracking list to the end of the Sieve script execution. If
+ failed script executions would add values to the duplicate tracking
+ list, all "duplicate" tests would erroneously yield "true" for the
+ next delivery attempt of the same message, which can -- depending on
+ the action taken for a duplicate -- easily lead to discarding the
+ message without further notice.
-Bosch [Page 2]
+ Implementations SHOULD limit the number of values (and thereby
+ messages) that are tracked. Also, implementations SHOULD let entries
+ in the value tracking list expire after a short period of time. The
+ user can explicitly control the length of this expiration time by
+ means of the ":seconds" argument. If the ":seconds" argument is
+
+
+
+Bosch [Page 4]
- Sieve: Detecting Duplicate Deliveries February 2012
+ Sieve: Detecting Duplicate Deliveries October 2012
- same "name" argument, it is recognized as a duplicate.
+ omitted, an appropriate default MUST be used. Sites SHOULD impose a
+ maximum limit on the expiration time. If that limit is exceeded, the
+ maximum value MUST silently be substituted; exceeding the limit MUST
+ NOT produce an error.
More information about the dovecot-cvs
mailing list