[Dovecot] dbmail benchmarking

[Troy Benjegerdes]

Could someone please update the dbmail wikipedia page with this info ;)

http://en.wikipedia.org/wiki/DBMail_IMAP_and_POP3_server

I suppose what would be nicer would be to extend the dovecot entry and
reference that from the dbmail entry ;)

On Sat, May 12, 2007 at 03:16:05AM +0300, Timo Sirainen wrote:
> I thought I'd try benchmarking with dbmail (v2.2.4) to see how much  
> slower a SQL backend could actually be. Skip to bottom for the  
> conclusions.
> 
> Originally I ran the tests with the databases being in XFS  
> filesystem. MySQL's performance was horrible. It went 3-7x faster  
> with ext3.
> 
> MySQL 5.0.30 backend (innodb):
> 
> ./imaptest clients=1 - append=100 seed=1 secs=30 msgs=1000000 logout=0
> Logi Sele Appe
> 100% 100% 100%
>             5%
>    1  291  303
> 
> So that's 10 messages/sec saved. Now how about with 5 concurrent  
> clients?
> 
> Logi Sele Appe
> 100% 100% 100%
>             5%
>    5 1259 1332
> 
> Pretty well. Then something more generic:
> 
> ./imaptest clients=1 seed=1 secs=30 msgs=1000000 logout=0
> Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe
> 100%  50%  50% 100% 100% 100%  50% 100% 100% 100%
>                           30%                  5%
>    1   37   36   75   73  110   34   24   73   78
> 
> Without Fetc (header/metadata fetching):
> Logi List Stat Sele Fet2 Stor Dele Expu Appe
> 100%  50%  50% 100% 100%  50% 100% 100% 100%
>                      30%                  5%
>    1   94   94  199  283  102   85  198  210
> 
> PostgreSQL 8.1.5 backend:
> 
> ./imaptest clients=1 - append=100 seed=1 secs=30 msgs=1000000 logout=0
> Logi Sele Appe
> 100% 100% 100%
>             5%
>    1  267  277
> 
> ./imaptest clients=5 - append=100 seed=1 secs=30 msgs=1000000 logout=0
> Logi Sele Appe
> 100% 100% 100%
>             5%
>    5 1094 1144
> 
> ./imaptest clients=1 seed=1 secs=30 msgs=1000000 logout=0
> Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe
> 100%  50%  50% 100% 100% 100%  50% 100% 100% 100%
>                           30%                  5%
>    9   29   40   74   72   99   22   12   64   71
> 
> ./imaptest clients=1 seed=1 secs=30 msgs=1000000 logout=0 fetch=0
> Logi List Stat Sele Fet2 Stor Dele Expu Appe
> 100%  50%  50% 100% 100%  50% 100% 100% 100%
>                      30%                  5%
>   35  105   95  200  277   54   70  165  175
> 
> The last two tests gave "Unexpected tagged reply:" errors that I  
> didn't get with MySQL. So apparently there's some PostgreSQL-specific  
> bug in dbmail.
> 
> The same values with Dovecot 1.0 + maildir:
> 
> ./imaptest clients=1 - append=100 seed=1 secs=30 msgs=1000000 logout=0
> Logi Sele Appe
> 100% 100% 100%
>             5%
>    1  346  364
> 
> ./imaptest clients=5 - append=100 seed=1 secs=30 msgs=1000000 logout=0
> Logi Sele Appe
> 100% 100% 100%
>             5%
>    5 1408 1470
> 
> ./imaptest clients=1 seed=1 secs=30 msgs=1000000 logout=0
> Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe
> 100%  50%  50% 100% 100% 100%  50% 100% 100% 100%
>                           30%                  5%
>    1  130  133  259  258  382  123  127  257  271
> 
> ./imaptest clients=1 seed=1 secs=30 msgs=1000000 logout=0 fetch=0
> Logi List Stat Sele Fet2 Stor Dele Expu Appe
> 100%  50%  50% 100% 100%  50% 100% 100% 100%
>                      30%                  5%
>    1  155  163  339  478  169  175  338  354
> 
> So, what are the conclusions?
> 
>  - In raw append speed dbmail is almost as fast as maildir.
>  - In raw read/write speed maildir is about 1,6 times faster
>  - When adding metadata fetches Dovecot is 4 times faster than  
> dbmail. This is most likely because dbmail doesn't have a cache  
> equivalent to dovecot.index.cache so it has to do the fetches the  
> slow way.
> 
> I would have liked to also run the generic tests with more than 1  
> client, but then I start hitting dbmail bugs and the test stops  
> (reported already to their bugtracker).



-- 
--------------------------------------------------------------------------
Troy Benjegerdes                'da hozer'                hozer at hozed.org  

Somone asked me why I work on this free (http://www.fsf.org/philosophy/)
software stuff and not get a real job. Charles Shultz had the best answer:

"Why do musicians compose symphonies and poets write poems? They do it
because life wouldn't have any meaning for them if they didn't. That's why
I draw cartoons. It's my life." -- Charles Shultz