March 2007 - dovecot - dovecot.org

[Dovecot] vpopmail "last auth" question
by B. Cook 03 Apr '07

03 Apr '07

Hello, I run the domain for the school here with qmail and vpopmail on FreeBSD with dovecot from the ports. I'm running dovecot 1.0.r26 and everything is working great. I been noticing that dovecot isn't updating when the user last logged in.. and I can't seem to figure out why. I also can't seem to find anything in the archives about it.. so am I the only one? name: bcook passwd: $1$ clear passwd: comment/gecos: B. Cook uid: 0 gid: 0 flags: 0 gecos: B. Cook limits: No user limits set. dir: /usr/local/vpopmail/domains/poughkeepsieschools.org/bcook quota: 41943040S usage: 72% account created: Wed Oct 18 21:09:03 2006 last auth: Never logged in I'm wondering how I could get the 'last auth' to populate like it used to w/ courier. below is my config if it will help: protocols = imap imaps pop3 listen = * disable_plaintext_auth = no ssl_disable = no ssl_cert_file = /long/path/to/file.pem ssl_key_file = /long/path/to/file.pem verbose_ssl = yes login_user = dovecot login_process_size = 64 first_valid_uid = 89 last_valid_uid = 89 first_valid_gid = 89 protocol imap { mail_plugins = quota imap_quota imap_client_workarounds = delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep } protocol pop3 { pop3_uidl_format = %08Xu%08Xv pop3_client_workarounds = outlook-no-nuls oe-ns-eoh } auth default { mechanisms = plain passdb vpopmail { args = } userdb vpopmail { } user = root } plugin { quota = maildir } Thanks in advance. -- B. Cook Network Analyst Poughkeepsie City School District Mobile: 845.264.5827 | Desk: 845.451.4791

4 4

[Dovecot] prefetch + static + deliver
by Tom Bombadil 02 Apr '07

02 Apr '07

Hi all... The prefetch entry in the wiki says: "If you're using Dovecot's local delivery agent, you'll still need a valid userdb which it can use to locate the users. You can do this by adding a normal sql/ldap userdb after userdb prefetch." (http://wiki.dovecot.org/UserDatabase/Prefetch) My question is: Does it need to be a SQL/LDAP userdb? How about static or passwd? Thanks in advance :)

3 3

[Dovecot] Using Dovecot LDA with Sendmail
by Scott Peron 02 Apr '07

02 Apr '07

Hi all, After looking at the LDA/Sendmail page in the dovecot wiki, I wanted to contribute another method to easily configure Sendmail to use deliver. Instead of adding a new mailer definition, as already suggested, one can simply use the following line in their sendmail.mc file instead: FEATURE(`local_procmail', `/usr/libexec/dovecot/deliver', `deliver -d $u', `SPhnu9')dnl And further on, still use: MAILER(procmail)dnl The location of deliver will of course vary for each OS and distro, etc, but this redefines the "procmail" mailer in the resulting sendmail.cf to use deliver instead, with the correct flags. For most setups, I believe this is the simplest configuration path. Scott -- Scott Peron, Webmaster, Infolytica Corp. webmaster(a)infolytica.com Tel: (514) 849-8752 x269 Fax: (514) 849-4239 http://www.infolytica.com Place du Parc, 300 Leo-Pariseau, Suite 2222, Montreal, QC, Canada, H2X 4B3

4 3

[Dovecot] Unexpected behaviour when deleteing a big mailbox
by Steffen Kaiser 02 Apr '07

02 Apr '07

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, my last tests left me with a mailbox with > 80'000 messages in Maildir. Dovecot served them fine. No problem. I usually use IMAP's "Mark messages a deleted" function and expunge later. Because the "Move to Trash" is default in many MUAs, I decided to try it on this mailbox. I do think that it is hard to detect and prevent such problem, but could it be implemented that when the client connection times out, the COPY process is terminated and rolled back? I mean, as long as the messages are copied/linked to */tmp nothing is done, yet, hence, a simple cleanup of */tmp would result in a rollback. Due to the timeout, the EXPUNGE wasn't issued so far and the source mailbox is unchanged. Or, is it possible to prevent two COPYs of an overlapping message range into the same destination folder simultaneously? Or, can I prevent to COPY such a huge range of messages at once? I know 80'000 messages are much, but I have users with an INBOX of at least 40'000, eventually moving all files into an archive mailbox. I'm also surprised to find that I didn't get two full copies of the source mailbox. ============== Start situation: 4.8G .box1 (I didn't recorded the individual subdirs, unfortunately.) # find .box1 -type f | wc -l 84273 Newly created .Trash. === I started Thunderbird, let sync the mailbox, selected them all, and pressed "Delete". Some time later: Connection timed out. I, wondered, and hit "Delete" a second time. Again, connection timed out. === I was surprised, however, to see that the Dovecot log with still active telling me that mails are copied to Trash. I ended up with two processes copying all messages to .Trash/tmp on the server. The process I traced a bit later linked any file after the other. What I wondered right from the beginning was, that du -hs .??* displayed about 3.5GB in .Trash, but still 4.8GB in .box1. So, why are there 3.5GB additional files in .Trash, when I configured Dovecot to use hardlinks? The end situation is that .box1 contains all messages still, but any Unseen messages are moved to .box1/cur, and .Trash contains: 7.4G .Trash/cur 791M .Trash/new 76K .Trash/tmp # find .Trash/ -type f | wc -l 121713 The size nearly doubled. # du -hs .box1 ; du -sh .Trash/ 4.8G .box1 8.2G .Trash/ ==== Last lines in Dovecot log: dovecot: Mar 21 11:46:15 Info: IMAP(dvtest) [11535]: copy -> Trash: uid=83964, msgid=<xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx(a)xx-xxxx-xxxxx-xxxx.xx>, box=box1 dovecot: Mar 21 11:46:15 Info: IMAP(dvtest) [11535]: copy -> Trash: uid=83965, msgid=<xxxxxxxx.xxxxxxx(a)xx-xxxx-xxxxx-xxxx.xx>, box=box1 dovecot: Mar 21 11:46:29 Info: IMAP(dvtest) [11535]: Disconnected: Mailbox is in inconsistent state, please relogin. Last lines of strace: fstat64(13, {st_mode=S_IFREG|0600, st_size=1341104, ...}) = 0 stat64("/var/cache/dovecot/31045/index/.box1/dovecot.index", {st_mode=S_IFREG|0600, st_size=1341104, ...}) = 0 alarm(120) = 0 fcntl64(13, F_SETLKW64, {type=F_RDLCK, whence=SEEK_SET, start=0, len=0}, 0xbfe562d4) = 0 alarm(0) = 120 fstat64(13, {st_mode=S_IFREG|0600, st_size=1341104, ...}) = 0 mmap2(NULL, 1341104, PROT_READ|PROT_WRITE, MAP_SHARED, 13, 0) = 0xb3ffc000 munmap(0xb4285000, 1341104) = 0 write(1, "* 7998 FETCH (FLAGS ($Label2 $La"..., 2067) = 2067 write(1, "* 8091 FETCH (FLAGS (\\Answered \\"..., 2056) = -1 EPIPE (Broken pipe) - --- SIGPIPE (Broken pipe) @ 0 (0) --- munmap(0xb7cbc000, 1341104) = 0 fcntl64(13, F_SETLK64, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}, 0xbfe56394) = 0 write(2, "\1IDisconnected: Mailbox is in in"..., 66) = 66 # /usr/local/dovecot.rc27/sbin/dovecot -n # /usr/local/dovecot.rc27/etc/dovecot.conf base_dir: /var/run/dovecot/ log_path: /var/log/dovecot/dovecot.log protocols: imap imaps pop3 pop3s ssl_ca_file: /etc/ssl/certs/ca.crt ssl_cert_file(default): /etc/ssl/certs/imap.pem ssl_cert_file(imap): /etc/ssl/certs/imap.pem ssl_cert_file(pop3): /etc/ssl/certs/pop3.pem ssl_key_file(default): /etc/ssl/private/imap.key ssl_key_file(imap): /etc/ssl/private/imap.key ssl_key_file(pop3): /etc/ssl/private/pop3.key disable_plaintext_auth: no verbose_ssl: yes login_dir: /var/run/dovecot//login login_executable(default): /usr/local/dovecot.rc27/libexec/dovecot/imap-login login_executable(imap): /usr/local/dovecot.rc27/libexec/dovecot/imap-login login_executable(pop3): /usr/local/dovecot.rc27/libexec/dovecot/pop3-login login_log_format_elements: %p: user=<%u> method=%m rip=%r lip=%l %c verbose_proctitle: yes first_valid_uid: 1000 mail_location: maildir:%h/MailDir:CONTROL=/var/cache/dovecot/%i/control:INDEX=/var/cache/dovecot/%i/index mail_debug: yes dotlock_use_excl: yes maildir_copy_with_hardlinks: yes maildir_copy_preserve_filename: yes mail_drop_priv_before_exec: yes mail_executable(default): /usr/local/dovecot.rc27/libexec/dovecot/imap mail_executable(imap): /usr/local/dovecot.rc27/libexec/dovecot/imap mail_executable(pop3): /usr/local/dovecot.rc27/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota mail_log zlib mail_plugins(imap): quota imap_quota mail_log zlib mail_plugins(pop3): quota mail_log mail_plugin_dir(default): /usr/local/dovecot.rc27/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/dovecot.rc27/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/dovecot.rc27/lib/dovecot/pop3 mail_log_prefix: %Us(%u) [%p]: mail_log_max_lines_per_sec: 30 pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %u pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): oe-ns-eoh namespace: type: private separator: . inbox: yes hidden: yes auth default: mechanisms: plain login cache_size: 10 username_chars: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890 verbose: yes debug: yes passdb: driver: ldap args: /usr/local/dovecot.rc26/etc/dovecot-ldap.conf passdb: driver: shadow userdb: driver: prefetch userdb: driver: ldap args: /usr/local/dovecot.rc26/etc/dovecot-ldap.conf userdb: driver: passwd-file args: /etc/passwd socket: type: listen client: path: /var/run/dovecot/auth-client mode: 432 master: path: /var/run/dovecot/auth-master mode: 432 group: mail plugin: quota: fs Bye, - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iQEVAwUBRgFBGy9SORjhbDpvAQLBcQf+OAqiufFlyPqJTyM3Ylkmiae2afXPe2wl bNWg6AyDrdvFw+rtdc5vnYukQUSk2a8zQBggt3Rfb1zZQb0pszrzLdR+fOwnW2uE kDyHRhyy88aVpM848oQKYnlWdNHFrdwbyp2myU4JxJhu0bXsAhyMgf9txshwe5cd 23O1x+gkWUp+1wuz84uTep3Kcs1wVXWANEN5RT4YMI3MOqMrcyB+P0lj0luy0HTH HE9T6mLqzGt0Cw6yfm+1W8bU3Wnt3C5je15gT++OPd5YmDvpQs+fOjavDIITsuOf jBck4N+KjwOM+FzP8Ea1WNWCvObJKdeiISkoFWFO9bafw5Yue9hFNA== =yyP5 -----END PGP SIGNATURE-----

6 17

[Dovecot] imaptest10 and stalled messages
by Mike Brudenell 01 Apr '07

01 Apr '07

Greetings - I've now got as far as playing with the imaptest10 test utility to see if I can stress-test our development server. imaptest10 is built against dovecot-1.0rc28 It may just be that I'm excessively heavy handed, but when I let imaptest10 rip with the command... ./imaptest10 user=test%03d host=testserver.imap.york.ac.uk clients=50 mbox=./dovecot.mbox msgs=1000 secs=30 logout=10 ...after 15 seconds or so I started getting output along the lines of: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 10% 30% 5% 2 0 1 1 1 0 0 0 0 0 0 50/ 50 2 0 0 1 1 0 0 0 0 0 0 50/ 50 0 0 2 2 0 1 1 1 1 1 0 50/ 50 2 0 2 2 0 1 1 1 1 1 2 50/ 50 0 0 1 1 0 0 0 0 0 0 0 50/ 50 - 16. stalled for 16 secs in LOGIN - 17. stalled for 16 secs in LOGIN - 18. stalled for 16 secs in LOGIN 2 1 1 2 1 0 0 0 0 0 0 50/ 50 - 17. stalled for 17 secs in LOGIN - 18. stalled for 17 secs in LOGIN - 22. stalled for 16 secs in LOGIN - 23. stalled for 16 secs in LOGIN - 24. stalled for 16 secs in LOGIN 1 0 1 1 0 0 0 0 0 0 0 50/ 50 - 17. stalled for 18 secs in LOGIN - 18. stalled for 18 secs in LOGIN - 22. stalled for 17 secs in LOGIN - 23. stalled for 17 secs in LOGIN - 24. stalled for 17 secs in LOGIN - 28. stalled for 16 secs in LOGIN - 29. stalled for 16 secs in LOGIN - 30. stalled for 16 secs in LOGIN - 31. stalled for 16 secs in LOGIN - 32. stalled for 16 secs in LOGIN The "stalled" log lines continue until the test completes. I'm assuming it's just that I'm pushing our hardware beyond its reasonable limits: it is only an oldish development box... Sun v120 with 1 650MHz SPARC processor and 1Gbyte RAM Dovecot is using shadow password file authentication. The mail folders are in Maildirs, NFS-mounted from a NetApp filer, and each account's INBOX has around 1000 messages in it. Control and Index files are on local disk. By the end of the 30 second test run load average has climbed to around 11 I don't know how "active" the imaptest10 utility is when it's running, but do you think I should be concerned by the "stalled" messages? That is, do you think I have something configured amiss? Or is it likely just to be I'm expecting too much of our hardware? (The real system will have beefier Suns with Opteron-based CPUs, more memory, etc) With thanks, Mike B-) PS. I forgot to say: despite all this activity there's not a single error logged by Dovecot in the (admittedly short at this stage) test runs. -- The Computing Service, University of York, Heslington, York Yo10 5DD, UK Tel:+44-1904-433811 FAX:+44-1904-433740 * Unsolicited commercial e-mail is NOT welcome at this e-mail address. *

2 7

[Dovecot] locking question
by Stewart Dean 01 Apr '07

01 Apr '07

There are three applications that have their mitts on files on my mail server, which is running AIXV5.3 and UWIMAP and mbox format. The mail folders and INBOXES are native to that machine, but also are NFS exported to a login server and a mailing list server. All three machines are running the lockd daemon. Everybody wants to lock differently 1) procmail (delivering for sendmail), which seems to want to use dotlocking, fcntl and lockf locking; for whatever reasons, the compile time tests seem to disallow flock. 2) UWIMAP which according to wiki.dovecot.org/Migration/UW uses: mbox_read_locks = flock mbox_write_locks = dotlock flock 3) I want to run dovecot in the same environment as I switch over, for which the locking strategy is supposed to be (according to http://wiki.dovecot.org/MboxLocking) mbox_read_locks = fcntl mbox_write_locks = dotlock fcntl Since that same page in the wiki says. "*It's important that all software that's reading or writing to mboxes use the same locking settings.",* I had recompiled procmail so it only usedotlocking and fcntl (thus removing lockf in addition to the disallowed flock)...but now I bumped into the Migration/UW page and there looks to be a conflict. I want to be able to run dovecot (for testing) and UWIMAP (for all users/production) at the same time (but never at the same time with the same mailbox), but UWIMAP is doc'ed to use flock and Dovecot doesn't It's conceivable that the different locking strategies of UWIMAP and Dovecot don't matter because they would never be going after the same mail files (right?) but there is procmail, which has to interact with both. It appears to not use flock OK, even though the docs say it should since UWIMAP expects it, and I've disabled lockf as well. But will there be a conflict with procmail and dovecot's use of fcntl, which UWIMAP doesn't use? Help. * * ** -- ==== Stewart Dean, Unix System Admin, Henderson Computer Resources Center of Bard College, Annandale-on-Hudson, New York 12504 sdean(a)bard.edu voice: 845-758-7475, fax: 845-758-7035

2 1

[Dovecot] who owns dovecot files and dirs?
by Stewart Dean 01 Apr '07

01 Apr '07

...and what permissions should they have. I am thinking of /var/run/dovecot and the index directory. What ownership, group and permissions should they be? Are there any other files/dirs created for dovecot alone (not the mail folders and INBOXes); if so, how should they be owned and permed? I had thought they were to be owned by dovecot, but it turns out that they should not -- ==== Stewart Dean, Unix System Admin, Henderson Computer Resources Center of Bard College, Annandale-on-Hudson, New York 12504 sdean(a)bard.edu voice: 845-758-7475, fax: 845-758-7035

2 1

[Dovecot] convert plugin failing
by Adrian Bridgett 31 Mar '07

31 Mar '07

I'm struggling to get the convert plugin working using the Debian etch packages. When starting dovecot it was trying to create files in /.temp.hostname... convert-plugin.c looks like it has a couple of typos in it - it checks "mail" three times for NULL rather than mail, user, home. I changed this but it's getting called when these three items are set to: "maildir:/HOME_DIRECTORY_USED_BUT_NOT_GIVEN_BY_USERDB/Maildir" "dump-capability" "" I must be missing something - shouldn't the mailboxes only be converted when a user logs in? Should the conversion be skipped for the dump-capability user (whatever that is!) Thanks, Adrian -- Adrian Bridgett - adrian(a)smop.co.uk GPG key available on public key servers

2 1

[Dovecot] Version Numbering and releases
by Peter Lindgren 31 Mar '07

31 Mar '07

Hi all, Merging the two large recent threads, my recommendation is as follows: * The Release Candidates for 1.0 should be bug fixes only. * Each new Feature, or a Feature Set, goes in a separate branch. * That branch is tested by those who need/want that feature. * When testing is completed, the branch is merged into the main trunk. This way, we will have fairly well tested features/feature sets in the main trunk which people can fairly safe download and test and deploy, if their own tests succeeds. When the feature code is merged into the main trunk, the minor version is incremented by one (1.0.x --> 1.1.0). Bug fixes increment the revision (1.1.0 --> 1.1.1). Easy to follow, easy to package for package maintainers. Also, it's easy to write Readme's and other documentation on what's changed. (And for those who think that odd versions are for development (unstable), well, they will just wait until sufficient features have been added before they think it is stable... ) Timo's new features he sees for future versions, plan them for a 2.0 or a 1.1 or whatever version number, just don't call it 1.0.2rc1 :-) Another suggestion, is to create a Bugzilla (or similar) site for dovecot. That gives us all a chance to track changes and bug fixes. /Peter, building rc29 as I write this

1 0

[Dovecot] rc28 : can't set quotas with userdb + passwd file
by David Anderson 30 Mar '07

30 Mar '07

I'm using dovecot 1.0 rc28 - great software! I have a virtual hosting setup, with passdb and userdb both using a passwd file. I cannot get per-user quotas put in the relevant field of the userdb to work. They appear to be completely ignored; only the setting from /etc/dovecot.conf is applied. Can anyone confirm that this is working for them? If it is, can you post the relevant bits of your configuration, please? Many thanks. Kind regards, David

3 3

[Dovecot] Freebsd 4 error DSN stat=Service Unavailable after dovecot install
by Biffy Bob 30 Mar '07

30 Mar '07

Hello everyone, I am getting an error message on FreeBSD 4 after I install dovecot. To start off, I create two regular users, testuser and testuser1. I send each user emails then I install dovecot. After installing dovecot I set up the users in Maildir format then proceed to send them emails and here is what shows up in the maillogs: Email to testuser3 - Mar 29 22:34:52 myserver sendmail[4379]: l2TMYqwj004379: from=root, size=54, class=0, nrcpts=1, msgid=<200703292234.l2TMYqwj004379(a)myserver.vwh.net>, relay=r oot@localhost Mar 29 22:34:52 myserver sm-mta[4380]: l2TMYqMi004380: from=<root(a)myserver.vwh.net>, size=342, class=0, nrcpts=1, msgid=<200703292234.l2TMYqwj004379@myserver .vwh.net>, proto=ESMTP, daemon=MTA-v4, relay=localhost [127.0.0.1] Mar 29 22:34:52 myserver sendmail[4379]: l2TMYqwj004379: to=testuser3(a)myserver.vwh.net, ctladdr=root (0/0), delay=00:00:00, xdelay=00:00:00, mailer=relay, pr i=30054, relay=localhost.. [127.0.0.1], dsn=2.0.0, stat=Sent (l2TMYqMi004380 Message accepted for delivery) Mar 29 22:34:52 myserver sm-mta[4381]: l2TMYqMi004380: to=<testuser3(a)myserver.vwh.net>, ctladdr=<root(a)myserver.vwh.net> (0/0), delay=00:00:00, xdelay=00:00:0 0, mailer=local, pri=30560, relay=local, dsn=5.0.0, stat=Service unavailable (something didn't work) Mar 29 22:34:52 myserver sm-mta[4381]: l2TMYqMi004380: l2TMYqMi004381: DSN: Service unavailable (something didn't work) Mar 29 22:34:52 myserver sm-mta[4381]: l2TMYqMi004381: to=myserver, delay=00:00:00, xdelay=00:00:00, mailer=local, pri=31584, relay=local, dsn=5.0.0, stat=Se rvice unavailable (something didn't work) Mar 29 22:34:52 myserver sm-mta[4381]: l2TMYqMi004381: l2TMYqMj004381: return to sender: Service unavailable (something didn't work) Mar 29 22:34:52 myserver sm-mta[4381]: l2TMYqMj004381: to=myserver, delay=00:00:00, xdelay=00:00:00, mailer=local, pri=32608, relay=local, dsn=5.0.0, stat=Se rvice unavailable (something didn't work) Mar 29 22:34:52 myserver sm-mta[4381]: l2TMYqMi004381: Losing ./qfl2TMYqMi004381: savemail panic Mar 29 22:34:52 myserver sm-mta[4381]: l2TMYqMi004381: SYSERR(root): savemail: cannot save rejected email anywhere Here is what the user looks like in /etc/passwd: testuser3:*:2400:2400:My very own test user:/home/testuser3:/sbin/nologin For the other users that have shell access, they don't get this problem. It seems to be only the users with no shell. Does anybody know of a fix for this problem?

3 4

[Dovecot] make failed in doc/wiki
by Daniel 30 Mar '07

30 Mar '07

Hi! I have this problem with cvs (1.0 branch): [...] Making all in wiki make: don't know how to make \n. Stop in /home/leva/dovecot/dovecot/doc/wiki. *** Error code 1 Stop in /home/leva/dovecot/dovecot/doc (line 323 of Makefile). *** Error code 1 Stop in /home/leva/dovecot/dovecot (line 390 of Makefile). *** Error code 1 Stop in /home/leva/dovecot/dovecot (line 242 of Makefile). Using automake-1.9.6 and autoconf-2.59. What could be the problem? Daniel

2 4

[Dovecot] Security hole #3: zlib plugin allows opening any gziped mboxes
by Timo Sirainen 30 Mar '07

30 Mar '07

zlib plugin allows opening gzipped mboxes as read-only mailboxes. However when using it, the mailbox name checks are bypassed so it's possible to open for example "../otheruser/somefile.gz". Only valid gzipped mbox files can be opened, and only if their name ends with ".gz". You can fix this by upgrading to v1.0.rc29 (available soon) or with this patch: http://dovecot.org/list/dovecot-cvs/2007-March/008488.html I don't think this matters much though. zlib plugin is rarely used, and those who do use it are probably using Dovecot with systems users (per-user UIDs), so the imap process wouldn't have access to other users' mbox files anyway. I found this problem when I was cleaning up the code in CVS HEAD.

1 0

[Dovecot] dovecot-ldap for active directory 2003 r2
by Claudio Roberto Prateat 30 Mar '07

30 Mar '07

Hi, You have example of the dovecot-ldap.conf for authenticate in active directory 2003 r2 ? I have squid, apache authenticate in active directory, but dovecot return failed. Help, please... Best regards !

2 2

[Dovecot] uiddir mailbox format with benchmarks
by Timo Sirainen 30 Mar '07

30 Mar '07

I cleaned up the lib-storage code a bit so that it's easier to write support for new mailbox formats. It could still use a bit of cleaning up. Anyway, I thought I'd try how easy it would be to implement Cyrus-like mail store which would consist of only Dovecot's index files and "<uid>." message files. Flags and everything else is kept only in Dovecot's index files (so it might not be the best idea to use it in production just yet). Several hours later I ended up having it working. The code contains: ~/cvs/m/dovecot-head/src/lib-storage/index/uiddir% wc *[ch] 144 329 3552 uiddir-mail.c 240 565 6206 uiddir-save.c 532 1367 13628 uiddir-storage.c 55 126 1712 uiddir-storage.h 169 404 4057 uiddir-sync.c 21 48 573 uiddir-sync.h 103 258 2762 uiddir-transaction.c 1264 3097 32490 total uiddir-storage.c is still a bit too bloaty with lots of copy&pasted code. The same with maildir and mbox: maildir: 6022 18232 165605 total mbox: 7859 23283 213686 total Benchmarks show that with small mailboxes it's a lot faster than maildir, and with larger mailboxes it's a bit faster. I also tested it against Cyrus. I found out that fetching ENVELOPE vs. specific headers gives quite a huge performance difference. Cyrus is optimized for ENVELOPE while Dovecot is optimized for specific headers. With larger mailboxes + ENVELOPE Cyrus is faster than Dovecot, while with specific headers Dovecot is always faster. Maybe I'll some day make Dovecot optimize for ENVELOPE also if it sees that client only wants to use it. BTW. Does anyone have better naming ideas than "uiddir"? I'll commit it to CVS HEAD once I'm sure about the name. :) u: Dovecot CVS HEAD + uiddir m: Dovecot CVS HEAD + Maildir c: Cyrus IMAP 2.2.13 ./imaptest list=0 seed=1 secs=60 clients=10 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 23486 11822 23481 23476 33426 11659 8038 23472 11922 46952 m: 19367 9734 19367 19356 27453 9687 6287 19347 7983 38709 c: 5124 2553 5121 5105 7080 2559 1877 5104 3271 10228 ./imaptest list=0 dele=20 expu=20 seed=1 secs=60 msgs=10000 clients=10 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 5291 2704 5289 5278 7448 2706 1048 1061 5283 10563 m: 4162 2103 4162 4153 5977 2124 762 853 4153 8304 c: 4302 2157 4301 4289 6021 2072 806 879 4292 8584 ./imaptest list=0 dele=10 expu=10 seed=1 secs=60 msgs=100000 clients=10 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 3211 1637 3206 3195 4639 1537 287 302 3202 6402 m: 2957 1508 2955 2944 4173 1440 279 306 2948 5894 c: 3933 1929 3933 3920 5604 2020 380 379 3923 7846 ------ ./imaptest list=0 dele=10 expu=10 seed=1 secs=60 msgs=100000 clients=1 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 1529 745 1529 1528 2184 819 136 162 1528 3056 c: 1579 813 1579 1578 2257 811 171 148 1578 3156 ./imaptest list=0 dele=10 expu=10 seed=1 secs=60 msgs=100000 clients=50 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 3098 1552 3093 3085 4432 1520 293 334 3089 6173 c: 3664 1765 3659 3596 5106 1802 365 364 3614 7228 ====== FETCH BODY.PEEK[HEADER.FIELDS(from to cc subject)] instead of ENVELOPE: ./imaptest list=0 dele=20 expu=20 seed=1 secs=60 msgs=10000 clients=10 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 5777 2933 5777 5767 8215 2901 1133 1191 5767 11531 c: 3636 1794 3635 3623 5196 1843 682 722 3627 7254 ./imaptest list=0 dele=10 expu=10 seed=1 secs=60 msgs=100000 clients=10 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 3869 1959 3868 3861 5452 1943 364 394 3859 7717 c: 3191 1620 3189 3184 4501 1598 315 327 3181 6362 ./imaptest list=0 dele=10 expu=10 seed=1 secs=60 msgs=100000 clients=1 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 1912 966 1912 1910 2783 940 184 197 1911 3820 c: 1342 696 1342 1341 1888 682 109 126 1341 2682 ./imaptest list=0 dele=10 expu=10 seed=1 secs=60 msgs=100000 clients=50 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 4618 2320 4610 4604 6558 2266 452 455 4605 9205 c: 2960 1472 2956 2941 4134 1425 281 312 2910 5820

3 5

[Dovecot] sieve-body extension support for deliver LDA's sieve plugin
by Scott Peron 30 Mar '07

30 Mar '07

Hi Timo & Co, Is there any time frame for updating the deliver LDA's sieve plugin code to that from Cyrus IMAPd v2.3.x? I ask as I'm looking for the functionality of the sieve-body extension, as described here: http://www.ietf.org/internet-drafts/draft-ietf-sieve-body-06.txt This requirement has come up as we're upgrading from a Courier-IMAP/maildrop configuration that has some filtering based on message body content. As an aside, if that time frame is undefined or will be long, if I use maildrop instead as my LDA, will I loose the instant delivery notification via IMAP IDLE? Thanks very much, Scott -- Scott Peron, Webmaster, Infolytica Corp. webmaster(a)infolytica.com Tel: (514) 849-8752 x269 Fax: (514) 849-4239 http://www.infolytica.com Place du Parc, 300 Leo-Pariseau, Suite 2222, Montreal, QC, Canada, H2X 4B3

2 1

[Dovecot] Version numbering
by Timo Sirainen 30 Mar '07

30 Mar '07

After v1.0 is released, I can finally get back to sane version numbers. But any comments on which one is better: a) Postfix-style: "1.1.UNSTABLE.YYYYMMDD" -> 1.1.0 (stable) b) Odd-even numbering: 1.1.x (unstable) -> 1.2.0 (stable) With a) style the releases could be done by simply copying a nightly snapshot to releases/ directory and announcing the changes since the last release. I'm not sure if that's good or bad.

27 51

Re: [Dovecot] Version numbering
by Kenneth Porter 30 Mar '07

30 Mar '07

--On Wednesday, March 28, 2007 3:46 AM +0300 Timo Sirainen <tss(a)iki.fi> wrote: > After v1.0 is released, I can finally get back to sane version numbers. Have the RC's released new features as well as fixed bugs? I've been hesitant to upgrade to an RC because they seem to be released so frequently and I have no idea if one is more or less stable than another. This uncertainty is a big show-stopper to upgrading. It would help if, for a given RC, we knew exactly what known issues that RC had. (Is that in the wiki?) I'd suggest that a "version" be assigned only to something that should get spread around. A nightly snapshot should get the version it was based from and a timestamp suffix of when it was extracted from CVS. (Were this Subversion, I'd append the repository revision, instead.) The snapshot is not a "version" but a convenient download from CVS. It should be exactly what you get with the same timestamp used in a checkout command. Apache's versioning scheme (posted by John Peacock) looks good to me: <http://apr.apache.org/versioning.html>

1 0

[Dovecot] Forbidding folder delete
by Kenneth Porter 30 Mar '07

30 Mar '07

Is there some way I can forbid the deleting of folders? I have a Thunderbird user who twice now has accidentally deleted a folder, forcing me to go fetch it from the nightly backup. I'd like to prevent this from happening again, and the easiest way to me is to block folder deletion. I'd rather do that "manually" from the shell than leave it available from the client GUI.

2 1

[Dovecot] Copy to sent mail timeouts
by Jeff Tucker 29 Mar '07

29 Mar '07

Hi. I'm running the dovecot IMAP that comes with Ubuntu Edgy, which is 1.0rc2. I realize this is old, but I'll ask my question in case anyone knows about this problem and how to fix it, or knows that a later version definitely fixes it. What I'm seeing is that the IMAP server works perfectly, except for occasional timeouts when the user sends an email with attachments, causing that email then to be copied to their sent mail folder. The behavior seen is that Thunderbird, for instance, will say it is copying to the sent mail and the progress goes up to 100%, but then stops and Thunderbird eventually says the action timed out. I don't see anything in any dovecot logging and the file does not appear in the sent mail. I don't have any other kind of timeout issues. I can move mails around folders, read large emails, or whatever. The problem seems worse with large emails, but some times it will copy 15 MB emails and some times it won't. I can't find that any particular email always fails. Any ideas? As a last resort, I'll build the latest rc from scratch, but I like the automatic installation, notification of security problems, etc. I get with the distro packages. Thanks Jeff

2 2

[Dovecot] Help with Virtual user setup.. need pointers
by mailagent 29 Mar '07

29 Mar '07

Dear Andre, My name is Michael Joyner. I don't know if you are the right person. I had a friend in Trinidad with a brothers name, Chris. If this is you could you please reply. I live in the United States and am trying to find my friend. Please disregard is if this information does not pertain to you. Regards, Michael Joyner cid:image001.jpg@01C76B49.22E36120 * Address 21006 Devonshire St., Suite 206 Chatsworth, CA 91311 ( Phone 818-256-1939 7 Fax 818-256-1934 8 Email mj(a)refipal.com <mailto:elisa.chavez.ppys@StateFarm.com>

1 0

[Dovecot] chroot and raw logging
by Michal Soltys 29 Mar '07

29 Mar '07

I'm not sure if it's even intended to work in this particular case, but there's a problem when chroot is explicitely defined and rawlogging is used. Rawlog respects chroot point, without being actually chrooted. Consider configurtion as below, with two cases (dovecot.rawlog dir is of course created earlier, with appropriate uid/gid). 1) dovecot.userdb: test::10000:10000::/home/vmail/test.tld/test/./::userdb_mail=maildir:~/Maildir: ktrace will show: 18206 rawlog CALL sigprocmask(0x3,0) 18206 rawlog RET sigprocmask -65793/0xfffefeff 18206 rawlog CALL lstat(0x81fbe028,0xcfbcc980) 18206 rawlog NAMI "/dovecot.rawlog" 18206 rawlog RET lstat -1 errno 2 No such file or directory 18206 rawlog CALL sigprocmask(0x1,0xffffffff) 2) dovecot.userdb: test::10000:10000::/home/vmail/test.tld/./test::userdb_mail=maildir:~/Maildir: 1116 rawlog CALL lstat(0x85768028,0xcfbc1d20) 1116 rawlog NAMI "test/dovecot.rawlog" 1116 rawlog RET lstat -1 errno 2 No such file or directory 1116 rawlog CALL sigprocmask(0x1,0xffffffff) 1116 rawlog RET sigprocmask 0 I also noticed something else (not rawlog related). If I do a silly (blame my curiosity ;) thing like: valid_chroot_dirs: / and set chroot point as /./home/vmail/test.tld/test/ in passwd-file Dovecot will try to stat home/vmail/test.tld/test/ (and fail w/o beginning /), then chdir to /tmp, start pop3 or imap process, and recreate home/vmail/test.tld/test under /tmp. 17027 dovecot CALL chdir(0x87a4b29c) 17027 dovecot NAMI "home/vmail/test.tld/test/" 17027 dovecot RET chdir -1 errno 2 No such file or directory 17027 dovecot CALL setitimer(0,0xcfbe7020,0xcfbe7010) 17027 dovecot RET setitimer 0 17027 dovecot CALL seteuid(0) 17027 dovecot RET seteuid 0 17027 dovecot CALL chdir(0x3c001b72) 17027 dovecot NAMI "/tmp" 17027 dovecot RET chdir 0 ... 17027 dovecot CALL sigprocmask(0x3,0) 17027 dovecot RET sigprocmask -65793/0xfffefeff 17027 dovecot CALL execve(0x87a4bc38,0x87a4bc70,0x84fdb900) 17027 dovecot NAMI "/usr/local/libexec/dovecot/pop3" ... 17027 pop3 CALL mkdir(0x85f5a2b0,0x1ff) 17027 pop3 NAMI "home/vmail/test.tld/test//Maildir" 17027 pop3 RET mkdir -1 errno 2 No such file or directory 17027 pop3 CALL mkdir(0x85f5a2e0,0x1ff) 17027 pop3 NAMI "home/vmail/test.tld/test/" 17027 pop3 RET mkdir -1 errno 2 No such file or directory 17027 pop3 CALL mkdir(0x85f5a328,0x1ff) 17027 pop3 NAMI "home/vmail/test.tld/test" 17027 pop3 RET mkdir -1 errno 2 No such file or directory 17027 pop3 CALL mkdir(0x85f5a348,0x1ff) ... and so on, then it creates all the home/vmail/... dirs Configuration: openbsd 3.9, i386, dovecot 1.0-rc28 dovecot.userdb: test::10000:10000::/home/vmail/test.tld/test/./::userdb_mail=maildir:~/Maildir: dovecot.passdb: test:{PLAIN}test: base_dir: /var/dovecot/ protocols: imap imaps pop3 pop3s ssl_listen: * ssl_ca_file: /etc/ssl/cert_bundle.pem ssl_cert_file: /etc/ssl/ca_ppgk/certs/fetch_crt.pem ssl_key_file: /etc/ssl/ca_ppgk/private/fetch_key.pem verbose_ssl: yes login_dir: /var/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login login_greeting: Dovecot IMAP server ready. login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no valid_chroot_dirs: /var/mail:/home/regular:/home/vmail verbose_proctitle: yes first_valid_uid: 1999 first_valid_gid: 10 mail_location: mbox:~/Mail:INBOX=/var/mail/%u mail_debug: yes fsync_disable: yes lock_method: fcntl maildir_copy_with_hardlinks: yes mbox_read_locks: fcntl mbox_write_locks: fcntl mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 imap_client_workarounds(default): outlook-idle delay-newmail imap_client_workarounds(imap): outlook-idle delay-newmail imap_client_workarounds(pop3): outlook-idle pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login verbose: yes debug: yes passdb: driver: passwd-file args: /etc/dovecot.passdb passdb: driver: bsdauth userdb: driver: passwd-file args: /etc/dovecot.userdb userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master:

2 2

[Dovecot] passwd-file contents are not being used
by Randolph Kahle 29 Mar '07

29 Mar '07

I am setting up my dovecot installation and I am not getting results that I expected for the authentication phase. I expected: * PAM authentication would not be attempted * passwd-file would provide gid, uid, home, mail I notice that even thought it appears that this information is returned from auth(default) in the IMAP(randykahle) phase, it is not using that information - it used uid=502, etc. Here is my log output: ----- Mar 29 05:59:08 kahle dovecot: auth(default): pam(randykahle, 72.211.130.251): lookup service=dovecot Mar 29 05:59:08 kahle dovecot: auth(default): pam(randykahle, 72.211.130.251): pam_authenticate() failed: Authentication failure Mar 29 05:59:08 kahle dovecot: auth(default): passwd-file /etc/ dovecot/userdb: Read 1 users Mar 29 05:59:08 kahle dovecot: auth(default): passwd-file(randykahle, 72.211.130.251): lookup: user=randykahle file=/etc/dovecot/userdb Mar 29 05:59:08 kahle dovecot: auth(default): client out: OK 1 user=randykahle home=/var/mail/vhosts/com.rosalita/ randy mail=maildir:~/Maildir uid=5000 gid=5000 Mar 29 05:59:08 kahle dovecot: auth(default): master in: REQUEST 26 11935 1 Mar 29 05:59:08 kahle dovecot: auth(default): passwd(randykahle, 72.211.130.251): lookup Mar 29 05:59:08 kahle dovecot: auth(default): master out: USER 26 randykahle system_user=randykahle uid=502 gid=502 home=/ home/randykahle Mar 29 05:59:08 kahle dovecot: IMAP(randykahle): Effective uid=502, gid=502, home=/home/randykahle Mar 29 05:59:08 kahle dovecot: IMAP(randykahle): maildir: access(/ home/randykahle/Maildir, rwx): failed: No such file or directory Mar 29 05:59:08 kahle dovecot: IMAP(randykahle): maildir: couldn't find root dir Mar 29 05:59:08 kahle dovecot: IMAP(randykahle): mbox: root exists (/ home/randykahle/mail) Mar 29 05:59:08 kahle dovecot: IMAP(randykahle): mbox: INBOX exists (/ var/mail/randykahle) Mar 29 05:59:08 kahle dovecot: IMAP(randykahle): mbox: root=/home/ randykahle/mail, index=/home/randykahle/mail, inbox=/var/mail/randykahle Mar 29 05:59:08 kahle dovecot: imap-login: Login: user=<randykahle>, method=PLAIN, rip=72.211.130.251, lip=72.249.22.55 Mar 29 05:59:11 kahle dovecot: IMAP(randykahle): open(/var/ mail/.temp.kahle.org.11965.76100dca2c5b2009) failed: Permission denied Mar 29 05:59:11 kahle dovecot: IMAP(randykahle): file_lock_dotlock() failed with mbox file /var/mail/randykahle: Permission denied Mar 29 05:59:16 kahle dovecot: IMAP(randykahle): Disconnected ----- Here is my passwd-file user information line: ----- randykahle:{plain}xxxxx:5000:5000::/var/mail/vhosts/com.rosalita/ randy::home=/var/mail/vhosts/com.rosalita/randy mail=maildir:~/ Maildir uid=5000 gid=5000 ----- Any tips on how to get the information in the passwd-file used by the IMAP process would be appreciated. -- Randy

2 1

[Dovecot] dovecot mmap() complaints: No such device
by sergey ivanov 29 Mar '07

29 Mar '07

I'm trying to use dovecot-deliver with sieve plugin, delivering to Maildirs. Everything seems working O.K., but I'm getting the following lines in /var/log/maillog: --- Mar 29 00:06:00 alt64 dovecot: imap-login: Login: user=<seriv>, method=PLAIN, rip=66.80.117.2, lip=192.168.10.8, TLS Mar 29 00:06:00 alt64 dovecot: IMAP(seriv): mmap() failed with index file /home/seriv/.imap/dovecot.index: No such device ... Mar 29 00:06:19 alt64 deliver(seriv): mmap() failed with index file /home/seriv/.imap/dovecot.index: No such device Mar 29 00:06:19 alt64 deliver(seriv): msgid=<460B3B13.7010000(a)omniti.com>: saved mail to INBOX Mar 29 00:06:19 alt64 postfix/local[15166]: 6294A141C269: to=<seriv(a)alt64.menlo>, relay=local, delay=0.24, delays=0.03/0.01/0/0.2, dsn=2.0.0, status=sent (delivered to command: /usr/lib64/dovecot/deliver -d ${USER}) Mar 29 00:06:19 alt64 postfix/qmgr[20933]: 6294A141C269: removed --- I have tried to enable rawlog, but have corresponding rawlog file do not explain the problem for me: --- 3 select "INBOX" 4 UID fetch 1:* (FLAGS) 5 UID fetch 43498:43501 (UID RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type)]) 6 UID fetch 43500 (UID RFC822.SIZE BODY.PEEK[]) 7 uid store 43500 +FLAGS (NonJunk) 8 IDLE --- Dovecot compiled from cvs branch_1_0 at March 25. --- $ sudo dovecot -n Password: # /etc/dovecot/dovecot.conf ssl_cert_file: /etc/postfix/cert.pem ssl_key_file: /etc/postfix/key.pem login_dir: /var/run/dovecot/login login_executable: /usr/lib64/dovecot/imap-login mail_extra_groups: mail mail_location: maildir:/home/%u/.imap mail_executable: /usr/lib64/dovecot/rawlog /usr/lib64/dovecot/imap auth default: passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 438 --- /home mounted as zfs-fuse filesystem. I guess the problem due to fuse mounted filesystem. -- With best regards, Sergey Ivanov.

2 2

[Dovecot] can't receive mail from other hosts
by carpenter＠strongs.org 29 Mar '07

29 Mar '07

i'm running RHEL v 5 and 1.0-1.2.rc15 of dovecot and 2.3.3-2 of postfix. users can send mail to outside hosts, but can not receive them. i find no errors in the maillog -- i used a page from the redhat docs to configure postfix and dovecot so that they used the same mail directory... i think... any ideas on how to debug and/or fix this would be greatly appreciated. thanks, steve

4 3