[Dovecot] Some benchmarking

[Timo Sirainen]

So I finally got the new mail caching code fully working in normal 
conditions. Index rebuilding and some error correction still needs a 
bit work. mbox is also still broken. Anyway, I was mostly interested 
how fast Dovecot now works with my 367000 LKML mails, but thought I'd 
have to get something to compare the results against, so here's Dovecot 
vs. Courier.

Results against older Dovecot would have been pretty boring, the older 
Dovecot could never perform threading without  opening all the files 
and reading References-headers from them. Sorting would could have been 
done from cache if IMAP ENVELOPEs were cached, I'd estimate maybe 2-3x 
larger user CPU usage with it and a _lot_ larger cache file since it 
would have stored from, to, etc. fields in it.

Results against Cyrus would be interesting. I'll probably do those 
sometimes later too.. Last I checked, Cyrus was very slow at inserting 
the mails into it's database, it took .. was it 1-3 hours to insert all 
those mails, while a simple copying took maybe 20 mins. Anyway, Cyrus's 
caching is pretty dummy as well so I'd expect Dovecot to be many times 
faster in threading and likely somewhat faster in sorting.

-------------- next part --------------
Maildir: Linux Kernel Mailing List with 367918 mails, total of 1,9GB.

NOTE: The total times are heavily affected by disk I/O. In these tests the
index files are cached in memory by OS. The files themselves are mostly
uncached (I flushed the cache by reading another 2GB file 3 times).

System: 1GHz AMD Athlon CPU, 1GB RAM, 120GB IDE disk,
Linux 2.4.21, XFS filesystem


DOVECOT 0.99.11
---------------

 - NOTE: Debugging enabled, disabling it might give slightly lower user CPU
   times

1) select mailbox for the first time

   2.85s user 2.20s system 55% cpu 9.035 total

2) select mailbox for second time

   0.01s user 0.02s system 127% cpu 0.024 total

3) resync cur/ directory (touch cur, select mailbox)

   2.91s user 0.55s system 77% cpu 4.483 total

4) "uid thread references us-ascii all" for the first time

   67.27s user 41.25s system 27% cpu 6:30.85 total

5) "uid thread references us-ascii all" for the second time (uses indexes)

   30.39s user 0.25s system 95% cpu 32.055 total

6) "uid sort (date) us-ascii all" (uses indexes that threading created)

   22.88s user 0.13s system 99% cpu 23.048 total

7) "uid sort (subject) us-ascii all" (uses indexes that threading created)

   38.41s user 0.15s system 99% cpu 38.708 total

size of created index file:

       4636252 .imap.index
     121513984 .imap.index.cache

8) cache file compression

   4.21s user 1.42s system 26% cpu 21.270 total

     113536000 .imap.index.cache

   Compression had only minimal effect on performance. Compression isn't
   usually done until 30% of file consists of old data for expunged messages.


COURIER 1.72
------------

1) select mailbox for the first time

   4.93s user 0.51s system 80% cpu 6.762 total
   - slightly faster since it doesn't have to write 21MB of index files

2) select mailbox for second time

   - courier doesn't have a indexes so this is equilevant to 3)

3) resync cur/ directory (touch cur, select mailbox)

   12.91s user 0.42s system 100% cpu 13.324 total
   - oops, slower than generating the uids first time. slow hashes or
     something..

4) "uid thread references us-ascii all" for the first time

   606.94s user 69.56s system 64% cpu 17:22.02 total
   - that's 10x slower in user CPU. Over 2.5x slower in total time.

5) "uid thread references us-ascii all" for the second time

   - courier doesn't have indexes, so this is equilevant to 4)

6) "uid sort (date) us-ascii all"

   442.58s user 55.11s system 61% cpu 13:28.79 total
   - courier doesn't have indexes, so it has to open all mail files.

7) "uid sort (subject) us-ascii all"

   444.13s user 56.82s system 59% cpu 14:03.14 total
   - courier doesn't have indexes, so it has to open all mail files