[Dovecot] Multiple copies of a message

[Bill Cole]

At 12:07 PM +0000 1/31/08, Anne Wilson wrote:
>On Wednesday 30 January 2008 23:16, Benjamin R. Haskell wrote:
>>  >
>>  > X-UID: 2041
>>  > X-KMail-Filtered: 65016
>>  > Status: RO
>>  > X-Status: OAC
>>  >
>>  > and
>>  >
>>  > X-UID: 2042
>>  > X-KMail-Filtered: 65017
>>  > Status: RO
>>  > X-Status: OAC
>>
>>  Have you changed anything recently with your kmail filters?
>
>KMail filters generally don't work on imap.

That's a tellingly vague statement. Do you mean that your KMail 
filters don't do what you expect with IMAP, that you believe you have 
filtering turned off for IMAP, that they are widely acknowledged to 
be broken for IMAP, that KMail claims to not be able to filter IMAP 
mail, or something else?

>  All my filtering is done in
>procmail.

Do you have procmail adding a X-KMail-Filtered header? That seems unlikely...

Nothing other than KMail will add that header to a message unless 
told to do so explicitly. Of course you can make procmail do anything 
you want it to do to a piece of mail, but having any other program 
add a header that is only meaningful to KMail seems like a bad idea.

>>  Are you
>>  running more than one mailserver? Dovecot doesn't add the X-UID field, so
>>  I suspect it wouldn't be changing it, either.
>>
>Temporarily, yes.  I have just set up a new server, which is now carrying all
>the traffic.  I have been leaving the old server readable while I check
>everything out.  I will probably turn dovecot off on it this weekend.

I think the question might have been aimed at the concept of KMail 
(or whatever is adding the X-UID header) playing ping-pong with a 
message between mailboxes on different machines, but maybe I'm 
misunderstanding that...

>As for the headers, yes, I would suspect that the X-UID could well be added by
>kmail.  However, multiple copies of the messages are appearing in
>~/Maildir/cur, and surely only dovecot could be putting them there?

Only Dovecot should be doing the specific task of moving message 
files between the tmp/new/cur subdirectories, i.e. of actually 
creating new files in ~/Maildir/cur, but Dovecot will do that because 
of what an IMAP client tells it to do. For a client to add a header 
to an existing message for its own use, it constructs a new version 
of the message with the added header and tell Dovecot to delete the 
old message after adding the new version as a new message. 
Implementing this incorrectly in an IMAP client could easily result 
in  multiple copies of a message in a mailbox, and even implementing 
it correctly can leave multiple versions of the same message in the 
maildir, depending on how the client and server handle the concepts 
of deleting and expunging.

My guess is that you have something broken in KMail which is 
filtering messages in a loop. Whether that's an inherent  KMail 
problem or a configuration problem (i.e. a pathological filter) I 
can't say.  You may find it useful to set up raw IMAP logging on 
Dovecot to see what exactly is happening, or if you are using 
unencrypted connections you could use tcpdump or wireshark to capture 
the IMAP sessions. The goal of either would be to figure out what 
KMail is actually doing, since it seems certain that an IMAP client 
is responsible for the existence of messages whose only differences 
are in headers that only an IMAP client would manipulate, including 
one that is clearly KMail's doing.

Incidentally, a very small amount of Google-work turns up multiple 
descriptions of KMail's involvement with the creation of duplicate 
messages on IMAP servers. It seems from a fast skim that it may be 
problematic to have anything other than KMail filtering delivered 
mail....


-- 
Bill Cole
bill at scconsult.com