[Dovecot] 1.0rc10 proxy memory usage problem
Hi, I have a rather large dovecot setup (4 proxy only servers and 21 IMAP/POP3 servers) in production for 3 days now.
The memory usage on my proxy servers only grows over time. It looks like a memory leak somewhere in dovecot. I have a RRD graph available on demand.
Any advise on the configuration ? Timo, I have strace a process before the manual restart. If it can help just ask for.
My configuration : Linux kernel 2.6. I use MySQL as authentication back-end.
dovecot --version
1.0.rc10
dovecot --build-options
Build options: ioloop=epoll notify=dnotify ipv6 openssl SQL drivers: mysql Passdb: checkpassword pam passwd passwd-file shadow sql Userdb: checkpassword passwd prefetch passwd-file sql static
----- config ----- protocols = pop3 imap disable_plaintext_auth = no ssl_disable = yes login_process_size = 128 login_process_per_connection = no login_processes_count = 16 login_max_connections = 512 login_greeting = ready auth_cache_size = 32768 auth_cache_ttl = 1800 auth_worker_max_count = 30 auth default { mechanisms = plain
Userdb settings are not used with proxy but there need to be something.
userdb static { args = static uid=5000 gid=5000 home=/dev/null } passdb sql { args = /etc/dovecot-mysql.conf } user = root count = 1 }
-- Laurent Papier
On Tue, 2006-10-24 at 09:55 +0200, Laurent Papier wrote:
Hi, I have a rather large dovecot setup (4 proxy only servers and 21 IMAP/POP3 servers) in production for 3 days now.
The memory usage on my proxy servers only grows over time. It looks like a memory leak somewhere in dovecot. I have a RRD graph available on demand.
Well, most importantly you didn't say which process's memory usage grows? imap-login, pop3-login, both of them, or dovecot-auth?
Le Thu, 02 Nov 2006 20:45:13 +0200 Timo Sirainen <tss@iki.fi> écrit:
On Tue, 2006-10-24 at 09:55 +0200, Laurent Papier wrote:
Hi, I have a rather large dovecot setup (4 proxy only servers and 21 IMAP/POP3 servers) in production for 3 days now.
The memory usage on my proxy servers only grows over time. It looks like a memory leak somewhere in dovecot. I have a RRD graph available on demand.
Well, most importantly you didn't say which process's memory usage grows? imap-login, pop3-login, both of them, or dovecot-auth?
Both of them. In fact the size seems to grow up to login_process_size. I have half most of the value in my configuration files and the memory usage is now stable around 1GB of RAM used.
My question is now what is a good setting for login_process_size with 'login_process_per_connection = no' ?
With 'login_process_per_connection = no' what is the relation between login_process_size and login_max_connections ?
My new configuration is :
protocols = pop3 imap disable_plaintext_auth = no ssl_disable = yes login_process_size = 64 login_process_per_connection = no login_processes_count = 8 login_max_connections = 256 login_greeting = ready auth_cache_size = 16384 auth_cache_ttl = 1800 auth_worker_max_count = 30 auth default { mechanisms = plain
Userdb settings are not used with proxy but there need to be something.
userdb static { args = static uid=5000 gid=5000 home=/dev/null } passdb sql { args = /etc/dovecot-mysql.conf } user = root count = 1 }
-- Laurent Papier - 03 88 75 80 50 Admin. système - SdV Plurimedia - <http://www.sdv.fr/>
On Tue, 2006-11-07 at 09:48 +0100, Laurent Papier wrote:
Le Thu, 02 Nov 2006 20:45:13 +0200 Timo Sirainen <tss@iki.fi> écrit:
On Tue, 2006-10-24 at 09:55 +0200, Laurent Papier wrote:
Hi, I have a rather large dovecot setup (4 proxy only servers and 21 IMAP/POP3 servers) in production for 3 days now.
The memory usage on my proxy servers only grows over time. It looks like a memory leak somewhere in dovecot. I have a RRD graph available on demand.
Well, most importantly you didn't say which process's memory usage grows? imap-login, pop3-login, both of them, or dovecot-auth?
Both of them. In fact the size seems to grow up to login_process_size.
Right, because after that the kernel kills the process.
My question is now what is a good setting for login_process_size with 'login_process_per_connection = no' ? .. With 'login_process_per_connection = no' what is the relation between login_process_size and login_max_connections ?
The more connections the process handles at the same time the more memory it needs.
Anyway.. I did a lot of fixes now to login processes. The largest memory leak should be fixed by these patches:
http://dovecot.org/list/dovecot-cvs/2006-November/006753.html http://dovecot.org/list/dovecot-cvs/2006-November/006754.html
But I did several other fixes also which may have leaked memory or crashed. Mostly related to AUTHENTICATE / AUTH commands, or client disconnecting before login to remote server was complete.
I guess it's time for rc13 :)
Le Tue, 07 Nov 2006 17:20:13 +0200 Timo Sirainen <tss@iki.fi> écrit:
With 'login_process_per_connection = no' what is the relation between login_process_size and login_max_connections ?
The more connections the process handles at the same time the more memory it needs.
Sure ! But can you give us some advise about the value we should use ? For example : login_process_per_connection = no login_max_connections = 512 login_process_size = ???
How dovecot handle a too small value in login_process_size ?
Anyway.. I did a lot of fixes now to login processes. The largest memory leak should be fixed by these patches:
http://dovecot.org/list/dovecot-cvs/2006-November/006753.html http://dovecot.org/list/dovecot-cvs/2006-November/006754.html
But I did several other fixes also which may have leaked memory or crashed. Mostly related to AUTHENTICATE / AUTH commands, or client disconnecting before login to remote server was complete.
I guess it's time for rc13 :)
Nice. This will give me a reason to upgrade.
-- Laurent Papier - 03 88 75 80 50 Admin. système - SdV Plurimedia - <http://www.sdv.fr/>
On Tue, 2006-11-07 at 16:44 +0100, Laurent Papier wrote:
Le Tue, 07 Nov 2006 17:20:13 +0200 Timo Sirainen <tss@iki.fi> écrit:
With 'login_process_per_connection = no' what is the relation between login_process_size and login_max_connections ?
The more connections the process handles at the same time the more memory it needs.
Sure ! But can you give us some advise about the value we should use ? For example : login_process_per_connection = no login_max_connections = 512 login_process_size = ???
Well.. If you're not using SSL, then I think one connection takes max. 16kB of memory. Probably less. So 512*16/1024 = 8MB. Then there's the memory that's needed for it to work at all. I think the default 32MB would be enough for this, but you can be sure only by looking how much memory it really uses :)
How dovecot handle a too small value in login_process_size ?
Dovecot doesn't handle it. Dovecot just tells to kernel at startup that "process's max. virtual size is this". If memory usage grows over that, the kernel kills the process. This most likely has happened to you and you should have seen "killed by signal 9" in logs (I think it was 9).
participants (2)
-
Laurent Papier
-
Timo Sirainen