October 2012 - dovecot

[Dovecot] copymail deleted
by Christian Rößner 07 Nov '12

07 Nov '12

Hi, I had enabled an option in dovecot. mail_attachment_dir = /var/mail/virtual/copymail/attachments After a while I checked /var/mail/virtual and did some cleanup. I did not remember that copymail was specified in dovecot and erased it. Oct 30 10:56:05 mx0 dovecot: imap(hidden): Error: file_istream.stat(/var/mail/virtual/copymail/attachments/6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6) failed: No such file or directory Oct 30 10:56:05 mx0 dovecot: imap(hidden): Error: istream-concat: Failed to get size of stream /var/mail/virtual/copymail/attachments/6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 Oct 30 10:56:05 mx0 dovecot: imap(hidden): Error: read() failed: Invalid argument (FETCH for mailbox INBOX UID 196) Oct 30 10:56:05 mx0 dovecot: imap(hidden): Disconnected: Internal error occurred. Refer to server log for more information. [2012-10-30 10:56:05] in=150 out=950 I have Bacula and have restored most of the stuff, but obviously not all files. That is not too important. But I do not know, how to tell dovecot that it may "forget" about files that produce a "No such file or directory" error. Can I do some "rescan/rebuild" in dovecot? Kind regards -Christian Rößner -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich

2 5

[Dovecot] how to best import Evolution/Thunderbird mail into dovecot?
by Christoph Anton Mitterer 07 Nov '12

07 Nov '12

Hi. I'm migrating all my mail archive (some 60 GB) from Evolution (which is really a broken piece of software) into dovecot. Now I face the problem how to do this best... Evolution (which is still a old 2.32.x version) itself uses mbox files, in a special hierarchical structure to allow subfolders and that like. It also stores it's own status info in X-Evolution and X-Evolution-Source mail headers. Unfortunately,... much of the mail was earlier from a Thunderbird installation, which uses it's own status headers (X-Mozilla*) that were not recognised by Evolution. I have no idea which mbox subformat was always used throughout the different programs and versions... 1) Any way to check for that? To make things worse... Thunder(burden) seems to have used a modified From_ line syntax... "^From -<address> <date>$" Ideally I'd like migrate all mail into dovecot (for storage reasons again, mbox) retaining the different status flags (read, forwarded, etc.) and getting rid of the proprietary headers (of course only when they were. First thing I tried was to simply copy mail within Evolution (i.e. dragging&dropping it from the local folders to the IMAP folders from dovecot). - that preserves the status from Evolution, but doesn't restore that from Thunderbird - it clutters up the information of all From_ lines... "<address> <date>" becomes "<address of my default evolution account> <now>" - neither does it handle the special Thunderbird From_ lines - neither does it remove the Thunderbird or the X-Evolution-Source headers - if Evolution has already had corrupted index files (and this is extremely likely... as it happens even immediately during recreating all of them)... so I may loose mail So my idea was that I need a program that: - can parse all the different mbox formats (those that use the quoted ">From" style and those that use Conent-Length) - can differentiate message headers from body (so that I can drop the proprietary headers and replace them by what dovecot uses as headers) - must of course understand multiline message headers 2) Any idea for a tool like that? The meaning of the X-Evolution and X-Mozilla* headers are easy to find on the web.... so I can convert them. So I basically "just" need a tool that parses all kinds of mbox formats... allow me to drop/add headers... and spits out the rest unmodified. 3) dovecot uses some special headers like X-UID and X-IMAPD... will it create these on it's own, the first time it processes the new mbox file? I mean these headers won't be there after creation. 4) Should I drop out (during) conversion... any other mail headers.. that dovecot uses as it's own? Thanks, Chris.

6 13

[Dovecot] maildir S= and W=
by Christoph Anton Mitterer 05 Nov '12

05 Nov '12

Hi. Even new mails delivered by my MDA don't get the ,S= and ,W= fields set... (but when I "upload" a mail via IMAP to dovecot, they are set) Is there some place in dovecot where I need to enable this? Or would it be the MDA that has to calculate and set this already when placing a file in ./new? Thanks, Chris.

2 6

[Dovecot] Pigeonhole 3.3 broken against Dovecot 2.1.10
by Daniel L. Miller 01 Nov '12

01 Nov '12

I'm compiling as I normally do. The config line for Dovecot is: configure --with-ldap --with-ssl --with-bzlib --with-zlib --with-stemmer --with-lucene --with-ldap followed by make & make install Then a 'configure' for Pigeonhole, followed by make, yields: libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -o .libs/sieve-dump sieve-dump.o -Wl,--export-dynamic ../../src/lib-sieve/.libs/libdovecot-sieve.so ../../src/lib-sieve-tool/.libs/libsieve-tool.a /usr/local/lib/dovecot/libdovecot-storage.so /usr/local/lib/dovecot/libdovecot-lda.so -L/usr/local/lib/dovecot /usr/local/lib/dovecot/libdovecot.so -lrt -Wl,-rpath -Wl,/usr/local/lib/dovecot libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -o .libs/sievec sievec.o -Wl,--export-dynamic ../../src/lib-sieve/.libs/libdovecot-sieve.so ../../src/lib-sieve-tool/.libs/libsieve-tool.a /usr/local/lib/dovecot/libdovecot-storage.so /usr/local/lib/dovecot/libdovecot-lda.so -L/usr/local/lib/dovecot /usr/local/lib/dovecot/libdovecot.so -lrt -Wl,-rpath -Wl,/usr/local/lib/dovecot /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_new_null(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_error(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_peer_certificate(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_load_verify_locations(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_use_PrivateKey(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `DH_free(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_ex_data(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_set_tmp_rsa_callback(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_use_certificate(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_alert_desc_string_long(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_ex_data(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_ex_new_index(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_set_client_CA_list(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_get_ext_d2i(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_accept(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_STORE_add_cert(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSLv23_server_method(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `OBJ_txt2nid(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_write(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_num(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_ctrl_get_write_guarantee(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_cipher_list(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_push(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_get_error(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_alert_type_string_long(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_COMP_get_name(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_by_id(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_write(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `i2d_DHparams(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_ENTRY_get_data(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CIPHER_get_name(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_use_PrivateKey(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_info_callback(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_get_subject_name(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_init(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_clear_error(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_load_error_strings(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_set_default_RSA(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_free(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CIPHER_get_bits(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_INFO_free(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_STORE_set_flags(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_new(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_dup(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_get_index_by_NID(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `d2i_DHparams(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_get_text_by_NID(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `OPENSSL_add_all_algorithms_noconf(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_set_default_DSA(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_cleanup(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_get_entry(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ASN1_STRING_type(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_library_init(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_current_cipher(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_set_cipher_list(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_connect(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_error_string_n(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_STORE_CTX_get_ex_data(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_verify(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_new_bio_pair(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSLv23_client_method(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_use_certificate(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ASN1_STRING_length(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_get_cert_store(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_set_tmp_dh_callback(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_set_default_ciphers(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `PEM_read_bio_PrivateKey(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_new_mem_buf(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_ex_data_X509_STORE_CTX_idx(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_pop_free(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_value(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `GENERAL_NAME_free(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_peek_error(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_version(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `RAND_bytes(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_new(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_ctrl_pending(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_bio(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ASN1_STRING_data(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_load_builtin_engines(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `PEM_read_bio_X509(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `EVP_cleanup(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_STORE_add_crl(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_read(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_finish(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_free(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_read(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_oneline(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_free(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_free_strings(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `EVP_PKEY_free(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_current_compression(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_verify_cert_error_string(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_ctrl(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_peek_last_error(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `RSA_generate_key(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_state_string_long(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `PEM_X509_INFO_read_bio(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `DH_generate_parameters(a)OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_free(a)OPENSSL_1.0.0' collect2: ld returned 1 exit status make[3]: *** [sieve-dump] Error 1 make[3]: *** Waiting for unfinished jobs.... -- Daniel

3 4

[Dovecot] Out of memory/Managesieve
by Jim Knuth 31 Oct '12

31 Oct '12

Hello, I have here a problem with managesieve. With the login about webmail (roundcube) comes here in the log: --snip dovecot: managesieve-login: Fatal: pool_system_realloc(4294967296): Out of memory dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 10157 returned error 83 (Out of memory (service managesieve-login { vsz_limit=1024 MB }, you may need to increase it)) --snap I've increased of 2048M and the same above. then with 4096 M happens the following --snip dovecot: managesieve-login: Panic: epoll_ctl(add, 61538840) failed: Bad file descriptor dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 9777 killed with signal 6 (core dumps disabled) --snap How can one solve then this? Any ideas greatly appreciated. Thanks. -- Mit freundlichen Grüßen, with kind regards, Jim Knuth --------- Backup interessiert niemanden - Auf Restore kommt es an!

3 7

[Dovecot] IMAP proxy between Office 365 client/Sun Messaging Server
by John Dalbec 31 Oct '12

31 Oct '12

I would like to be able to migrate messages from existing end-user accounts on Sun Messaging Server to Office 365 using an administrator login and password. Unfortunately the migration tool for Office 365 doesn't support SASL AUTHENTICATE PLAIN login. The online documentation I've found suggests that I should be able to configure dovecot as an IMAP proxy and have it log in to Sun Messaging Server with AUTHENTICATE PLAIN and encode_base64("user\0admin\0adminpw") in response to a login from the Office 365 migration tool. I'd like to configure dovecot to run only the IMAP proxy if possible. I was thinking of setting all accounts to use the same (strong) password in the proxy. Would anyone be willing to share a sample configuration? Thanks, John Dalbec ellucian Luminis system administrator Youngstown State University

2 1

[Dovecot] Unable to get Managesieve working
by Cliff Dunn 31 Oct '12

31 Oct '12

I have Roundcube webmail (v. 0.8.2) running with the managesieve plugin (v. 5.1). I am able to create sieve rules without any problems in Roundcube, but incoming mail is not being processed with the rule I specify. I suspect there is something that isn't configured correctly in dovecot, but unfortunately I am unable to find a resolution online and my very limited knowledge of dovecot isn't allowing me to solve this problem on my own. I am including my dovecot config below. Please keep my knowledge level in mind when suggesting options and let me know if there is any other information I can provide to help troubleshoot the problem. Thanks for any assistance! # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps managesieve ssl_cert_file: /etc/ssl/certs/postfix.pem ssl_key_file: /etc/ssl/private/postfix.key ssl_cipher_list: ALL:!LOW:!SSLv2 login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login mail_privileged_group: mail mail_location: maildir:~/mail:LAYOUT=fs:INBOX=~/mail/ mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve auth default: mechanisms: plain login passdb: driver: pam passdb: driver: pam userdb: driver: passwd userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix plugin: sieve: ~/.dovecot.sieve sieve_dir: ~/mail/sieve

3 5

[Dovecot] Dovecot stops to work - anvil problem
by FABIO FERRARI 31 Oct '12

31 Oct '12

Hi all, we have a problem about anvil, it seems that when we have a high load the dovecot stops to work. Sometimes it is sufficient to make a dovecot reload, but sometimes we have to restart it. These are the lines related to anvil in the dovecot.log: [root@secchia ~]# grep anvil /var/log/dovecot.log | more Oct 26 11:13:55 anvil: Error: net_accept() failed: Too many open files Oct 26 11:14:32 imap-login: Error: net_connect_unix(anvil) failed: Resource temporarily unavailable Oct 26 11:14:32 imap-login: Fatal: Couldn't connect to anvil Oct 26 11:14:33 pop3-login: Error: net_connect_unix(anvil) failed: Resource temporarily unavailable Oct 26 11:14:33 pop3-login: Fatal: Couldn't connect to anvil [...] (many lines like these) Oct 26 12:01:10 pop3-login: Fatal: Couldn't connect to anvil Oct 26 12:01:18 auth: Error: read(anvil-auth-penalty) failed: Connection reset by peer Oct 26 12:01:18 auth: Error: read(anvil-auth-penalty) failed: Connection reset by peer Oct 26 12:01:18 auth: Error: net_connect_unix(anvil-auth-penalty) failed: Connection refused Oct 26 12:01:18 auth: Error: net_connect_unix(anvil-auth-penalty) failed: Connection refused Oct 26 12:01:18 auth: Error: read(anvil-auth-penalty) failed: Connection reset by peer Oct 26 12:01:18 auth: Error: net_connect_unix(anvil-auth-penalty) failed: Connection refused And this is the output of the doveconf -n: [root@secchia ~]# doveconf -n # 2.0.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-308.11.1.el5 x86_64 Red Hat Enterprise Linux Server release 5.8 (Tikanga) xfs auth_cache_size = 1024 auth_cache_ttl = 21600 s auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes base_dir = /var/run/dovecot/ disable_plaintext_auth = no hostname = mail.unimore.it info_log_path = /var/log/dovecot.log lda_mailbox_autocreate = yes log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:/cl/mail/vhosts/sms.unimo.it/%Ln/Maildir mail_plugins = $mail_plugins quota mailbox_idle_check_interval = 60 s mbox_write_locks = fcntl namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /usr/local/etc/dovecot.masterusers driver = passwd-file master = yes } passdb { args = dovecot driver = pam } plugin { quota = maildir:User quota quota_exceeded_message = Quota exceeded (mailbox is full) quota_rule = *:storage=200MB quota_rule2 = *:messages=100000 quota_rule3 = INBOX.Trash:storage=+100M quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=85%% quota-warning 85 %u quota_warning3 = messages=95%% quota-warning 95 %u quota_warning4 = messages=80%% quota-warning 80 %u setting_name = quota } postmaster_address = postmaster(a)unimore.it quota_full_tempfail = yes service anvil { client_limit = 199999 process_limit = 199999 } service auth { client_limit = 14500 unix_listener auth-userdb { mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 } process_limit = 5000 } service imap { process_limit = 5000 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 } } service pop3 { process_limit = 1024 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { group = vmail user = vmail } user = dovecot } ssl_ca = </etc/pki/tls/certs/ca_unimore_tcs.pem ssl_cert = </etc/pki/tls/certs/cert-852-mail.unimore.it-cluster.pem ssl_key = </etc/pki/tls/certs/mailcluster.key userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_ssl = yes protocol lda { mail_plugins = $mail_plugins quota } protocol imap { mail_plugins = $mail_plugins imap_quota } protocol pop3 { mail_plugins = $mail_plugins quota pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } [root@secchia ~]# And these are the limit settings in the OS: * soft nofile 131072 * hard nofile 131072 Have someone had the same problem? thanks in advance Fabio Ferrari

2 2

[Dovecot] lmtp out of memory - raw backtrace
by Daniel Parthey 31 Oct '12

31 Oct '12

Hi, Our setup: - 4 hosts with director and mailbox instance - delivery via director lmtp into mailbox lmtp - mailbox format: mdbox - storage on NFS - OS: Linux 2.6.32-44-server x86_64 Ubuntu 10.04.4 LTS - Dovecot 2.1.10 - Pigeonhole 0.3.3 We're getting strange "out of memory" lmtp errors/backtrace with dovecot 2.1.10 accompanied by high load caused caused by a lot of lmtp deliveries to one user. First action would be to increase vsz_limit to a higher value, but I just want to make sure there is no bug - before blindly increasing this limit. I think it should not crash and corrupt mdbox, even if memory limit is reached: Oct 29 20:14:56 10.129.3.249 dovecot: lmtp(26698, user1(a)example.org) Fatal: pool_system_realloc(16777216): Out of memory Oct 29 20:14:56 10.129.3.249 dovecot: lmtp(26698, user1(a)example.org) Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x4271a) [0x7f6dcbae971a] -> /usr/lib/dovecot/libdovecot.so.0(+0x42766) [0x7f6dcbae 9766] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f6dcbabd158] -> /usr/lib/dovecot/libdovecot.so.0(+0x53690) [0x7f6dcbafa690] -> /usr/lib/dovecot/libdovecot.so.0(+0x3e6f5) [0x7f6dcbae56f5] -> /usr/lib/dove cot/libdovecot.so.0(buffer_write+0x7c) [0x7f6dcbae5e7c] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa8033) [0x7f6dcbdda033] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_record_map_move_to_private+0x3 c) [0x7f6dcbdda4ec] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_get_atomic_map+0x18) [0x7f6dcbde9c88] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb7cf9) [0x7f6dcbde9cf9] -> /usr/lib/dovecot/lib dovecot-storage.so.0(mail_index_sync_record+0x7e6) [0x7f6dcbdea626] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_map+0x23c) [0x7f6dcbdeae4c] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_m ap+0xa8) [0x7f6dcbddc2b8] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb42da) [0x7f6dcbde62da] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb458a) [0x7f6dcbde658a] -> /usr/lib/dovecot/libdovecot-storage.so.0( mail_index_sync_begin_to+0x4f) [0x7f6dcbde6ecf] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin+0x1e) [0x7f6dcbde6f4e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_map_atomic_lock+0x5e) [0x 7f6dcbd6115e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_transaction_save_commit_pre+0x46) [0x7f6dcbd64fb6] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9d4f3) [0x7f6dcbdcf4f3] -> /usr/lib/dovecot/libdov ecot-storage.so.0(mail_index_transaction_commit_full+0x9f) [0x7f6dcbddd97f] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_transaction_commit+0x8a) [0x7f6dcbdcf18a] -> /usr/lib/dovecot/modules/lib10_quota_plug in.so(+0xba7f) [0x7f6dca4eca7f] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x3d) [0x7f6d Oct 29 20:14:56 10.129.3.249 dovecot: lmtp(26698, user1(a)example.org) Fatal: master: service(lmtp): child 26698 returned error 83 (Out of memory (service lmtp { vsz_limit=256 MB }, you may need to increase it)) Oct 30 01:15:49 10.129.3.249 dovecot: lmtp(17927, user1(a)example.org) Error: mmap_anon(216690688) failed: Cannot allocate memory Oct 30 01:15:49 10.129.3.249 dovecot: lmtp(17927, user1(a)example.org) Error: mmap_anon(216690688) failed: Cannot allocate memory Oct 30 10:32:27 10.129.3.249 dovecot: mailbox: mail: imap(user1(a)example.org): <nURUbUPNBgAKgQPI>: Error: Corrupted dbox file /mail/dovecot/example.org/user1/mail/storage/m.3577 (around offset=1844402): EOF reading msg header (got 0/30 bytes) Oct 30 10:33:44 10.129.3.249 dovecot: mailbox: mail: imap(user1(a)example.org): <ZZ0pYkPN+wAKgQPp>: Error: /mail/dovecot/example.org/user1/mail/mailboxes/Lists/Cron/dbox-Mails/dovecot.index reset, view is now inconsistent Oct 30 10:33:44 10.129.3.249 dovecot: mailbox: mail: imap(user1(a)example.org): <CcYCYkPNkAAKgQPV>: Error: /mail/dovecot/example.org/user1/mail/mailboxes/Lists/postfix/dbox-Mails/dovecot.index reset, view is now inconsistent Oct 30 10:33:44 10.129.3.249 dovecot: mailbox: mail: imap(user1(a)example.org): <t+gLYkPNVQAKgQPV>: Error: /mail/dovecot/example.org/user1/mail/mailboxes/Lists/Dovecot/dbox-Mails/dovecot.index reset, view is now inconsistent Oct 30 10:33:46 10.129.3.249 dovecot: mailbox: mail: imap(user1(a)example.org): <YElVbUPNKwAKgQO/>: Error: /mail/dovecot/example.org/user1/mail/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21404, user1(a)example.org) Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21405, user1(a)example.org) Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21410, user1(a)example.org) Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21419, user1(a)example.org) Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21404, user1(a)example.org) Error: 2ZCXNsuej1CcUwAAk4785w: sieve: mailbox: deliver: session=<session}> msgid=<201210300932.q9U9URHO029943(a)common.example-hosting.net> from=MAILER-DAEMON(a)common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:32:59] Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21404, user1(a)example.org) Error: 2ZCXNsuej1CcUwAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21405, user1(a)example.org) Error: PFBjN8uej1CdUwAAk4785w: sieve: mailbox: deliver: session=<session}> msgid=<201210300932.q9U9URHP029943(a)common.example-hosting.net> from=MAILER-DAEMON(a)common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:32:59] Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21410, user1(a)example.org) Error: spYhE82ej1CiUwAAk4785w: sieve: mailbox: deliver: session=<session}> msgid=<201210300932.q9U9URHQ029943(a)common.example-hosting.net> from=MAILER-DAEMON(a)common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:33:01] Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21419, user1(a)example.org) Error: sOscMs2ej1CrUwAAk4785w: sieve: mailbox: deliver: session=<session}> msgid=<201210300932.q9U9URHS029943(a)common.example-hosting.net> from=MAILER-DAEMON(a)common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:33:01] Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21410, user1(a)example.org) Error: spYhE82ej1CiUwAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21405, user1(a)example.org) Error: PFBjN8uej1CdUwAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21419, user1(a)example.org) Error: sOscMs2ej1CrUwAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21538, user1(a)example.org) Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21538, user1(a)example.org) Error: +jXqE+uej1AiVAAAk4785w: sieve: mailbox: deliver: session=<session}> msgid=<201210300933.q9U9URHi029943(a)common.example-hosting.net> from=MAILER-DAEMON(a)common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:33:31] Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21538, user1(a)example.org) Error: +jXqE+uej1AiVAAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21543, user1(a)example.org) Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21543, user1(a)example.org) Error: CxDMM+uej1AnVAAAk4785w: sieve: mailbox: deliver: session=<session}> msgid=<201210300933.q9U9URHl029943(a)common.example-hosting.net> from=MAILER-DAEMON(a)common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:33:31] Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21543, user1(a)example.org) Error: CxDMM+uej1AnVAAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Regards Daniel -- https://plus.google.com/103021802792276734820

2 1

[Dovecot] POP3 Proxy : user format not accepted
by Christian Gregoire 30 Oct '12

30 Oct '12

Hello, I want to use Dovecot as a POP3 proxy (http://wiki.dovecot.org/HowTo/ImapProxy) All is working fine on my sample platform, except that I have plenty (several thousands) of users that login using local_part#domain, instead of local_part@domain, which is an old setting on my POP3 server. And in that case, Dovecot returns 'Authentication failed'. Here is my proxy table : mysql> select * from tbl_proxy; +--------------------+-------------+--------------------+ | user | host | destuser | +--------------------+-------------+--------------------+ | christian(a)mydom.fr | 10.10.100.1 | christian(a)mydom.fr | | christian#mydom.fr | 10.10.100.1 | christian(a)mydom.fr | +--------------------+-------------+--------------------+ If I login on the Dovecot proxy with the '@' version, everything is fine : root : ~> telnet 10.10.100.24 110 Trying 10.10.100.24... Connected to 10.10.100.24 (10.10.100.24). Escape character is '^]'. +OK Dovecot ready. user christian(a)mydom.fr +OK pass azerty42 +OK christian#mydom.fr has 3 messages (3561 octets) And MySQL logs show the query : 121030 12:55:28 3 Query SELECT NULL AS password, host, destuser, 'Y' AS nologin, 'Y' AS nodelay, 'Y' AS proxy, 'Y' AS nopassword FROM tbl_proxy WHERE user = 'christian(a)mydom.fr' If I login on the Dovecot proxy with the '#' version, it fails : root : ~> telnet 10.10.100.24 110 Trying 10.10.100.24... Connected to 10.10.100.24 (10.10.100.24). Escape character is '^]'. +OK Dovecot ready. user christian#mydom.fr +OK pass azerty42 -ERR Authentication failed. And nothing shows up the the MySQL logs. If I login directly on the POP server with the same credentials, no problem : root : ~> telnet 10.10.100.1 110 Trying 10.10.100.1... Connected to 10.10.100.1 (10.10.100.1). Escape character is '^]'. +OK Welcome to POP3 Server V 2.06. Authenticate yourself. user christian#mydom.fr +OK Password required for christian#mydom.fr pass azerty42 +OK christian#mydom.fr has 3 messages (3561 octets) Any idea ? Is the '#' not internally supported in the user login by Dovecot ? Thanks in advance. Christian

2 2