[Dovecot] dovecot-1.0.rc2 problems with Thunderbird

[Pete Slagle]

Timo Sirainen wrote:

> On Sun, 2006-07-30 at 18:48 -0700, Pete Slagle wrote:
>> I posted about this previously, but for whatever reason there hasn't
>> been much of a response.
>>
>> I upgraded from beta8 to rc2, and dovecot IMAP went from being fast
>> and responsive to being painfully slow and often hanging Thunderbird. (I
>> made no configuration changes to my dovecot configuration file which is
>> here: http:/x15.us/dovecot.conf)
>>
>> The main issue seems to be about deleting messages (or moving them).
>> Delete operations often, but not always, hang for many seconds or
>> minutes.
> 
> Could you start tracing the imap process when it happens, stop it when
> it's over and show me the results? Also is the process at the time
> eating all the CPU? Is the memory usage growing?
> 
> See http://wiki.dovecot.org/Debugging/ProcessTracing
> 
>> I did a little experimentation and found a sure work around: I stop the
>> dovecot server, run 'rm ~/Maildir/dov*' on my server home directory, and
>> restart dovecot. The problems are immediately gone, for a while, then
>> they gradually return. This seems to indicate there is some kind of
>> index problem or corruption on the server side, but I'm not certain.
> 
> Well, that removed both the index files and dovecot-uidlist file. If you
> only remove dovecot.index* does it fix it? What if you remove only
> dovecot.index.cache?
> 
> How large are those dovecot* files when it's running slow?

I'll do some testing as suggested over the next few days and report
back. Here are the answers to your questions that I can tell you now:

During hangs the there is no significant CPU utilization (less than 1%)
by dovecot processes on the server. On the client, usually Thunderbird
uses no CPU, but sometimes it uses all available.

I recently cleared the dovecot files in my top Maildir directory, so I
don't know how indicative the files sizes are. The whole Maildir
directory tree contains about 70,000 messages in about 1000 nested
subfolders. The dovecot files in the top directory are:

     65 Jul 31 09:34 Maildir/dovecot-keywords
   1622 Jul 31 15:13 Maildir/dovecot-uidlist
    896 Jul 31 15:13 Maildir/dovecot.index
  58368 Jul 31 15:08 Maildir/dovecot.index.cache
  15752 Jul 31 15:13 Maildir/dovecot.index.log

Just now I got the first error message (ever!) during a hang. A
Thunderbird message box reported a server error. The server log, full of
connect and disconnect messages, showed only these two errors, in this
order, at the end:

dovecot: 2006-07-31 15:07:38 PDT Error: IMAP(pete): Fixed index file
/home/pete/Maildir/.In - lists.dovecot/dovecot.index:
first_recent_uid_lowwater 1516 -> 1515
dovecot: 2006-07-31 15:07:38 PDT Error: IMAP(pete): Corrupted index file
/home/pete/Maildir/.In - lists.dovecot/dovecot.index: Duplicate header
extension keywords

The sizes of the files in the directory that had the error are as so:

      17 May 25 09:04 dovecot-keywords
     202 Jul 31 12:44 dovecot-uidlist
    1200 Jul 31 15:07 dovecot.index
   53248 Jul 31 12:23 dovecot.index.cache
   12972 Jul 31 15:07 dovecot.index.log
  131088 Jul 22 13:36 dovecot.index.log.2

Pete