From tss at iki.fi Sun Apr 1 00:28:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 1 Apr 2012 00:28:51 +0300 Subject: [Dovecot] Dovecot stones Message-ID: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> For the last few days I've been thinking about my company and what it really should do, and as much as the current plan seems reasonable, I think in good conscience I really can't help but to bring up an alternative plan: The support for Dovecot is quite good already, but wouldn't it be better if it was GUARANTEED that your email server was ALWAYS working with zero downtime or other problems? After some thinking, planning and consulting with my spiritual guides I realized I really should just go ahead and do it: From now on you can buy blessed Dovecot stones (http://www.dovecot.org/tmp/stone1.jpg), which are guaranteed to keep your email server running. I don't think there's really any further reason for me to provide commercial support. Probably the whole Dovecot mailing list will be pretty much useless after this as well, so I might as well stop posting there. If you have any problems, you'll just buy a Dovecot stone and the problem will go away. If for any reason that doesn't work, it means that your problem is more difficult than most and you need to buy a few more stones. Please remember that each shipment may take up to 3 weeks to reach you, so get your stones early and in sufficient quantities. A small email server will be happy enough with a single small Dovecot stone, but the more users you have the more stones you'll likely need. This is because most people are rather unhappy with their lives, and they project their negative energies towards their surroundings, including their email servers. To overcome this negative energy you need to have a counterbalancing positive energy, which the Dovecot stones will provide you. The Dovecot stones are hand painted from the finest materials available to the sister of the author of Dovecot that she could find in the local shops. Each stone contains such love and dedication towards the well being of Dovecot that you will shed tears when you lay your eyes upon their physical being. The images provided here unfortunately don't do justice to them. http://www.dovecot.org/tmp/stone2.jpg - This is how i keep my home desktop running all day every day. I have NEVER broken a single hard drive in my life. I don't even bother to keep backups, because I KNOW it is useless for me. A small Dovecot stone can be bought for 10 euros, a medium sized stone for 100 euros and a large stone for 500 euros. Each large stone can usually handle around 1000-2000 users before you need to buy more. If you have millions of users we'll need to find you a larger rock from my parents' backyard. There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago), so their prices need to be negotiated in advance. From pw at wk-serv.de Sun Apr 1 02:18:28 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Sun, 01 Apr 2012 01:18:28 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <4F7790C4.5080002@wk-serv.de> Timo Sirainen schrieb: > A small Dovecot stone can be bought for 10 euros, a medium sized stone for 100 euros and a large stone for 500 euros. Each large stone can usually handle around 1000-2000 users before you need to buy more. If you have millions of users we'll need to find you a larger rock from my parents' backyard. There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago), so their prices need to be negotiated in advance. =) From pw at wk-serv.de Sun Apr 1 02:21:45 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Sun, 01 Apr 2012 01:21:45 +0200 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F760B61.3090209@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> Message-ID: <4F779189.8000705@wk-serv.de> Patrick Westenberg schrieb: > Nick Warr schrieb: > >> I think some of the new Dovecot (director?) software is user aware, but >> I don't know if it's quite ready for production. > > Yes, with director it should be something like that: > > MTA --(lmtp)--\ /--(lmtp)--> backend1 --\ > -- director -- -- NFS > MTA --(lmtp)--/ \--(lmtp)--> backend2 --/ > > > IMAP-User --> frontend1 --\ /--(imap)--> backend1 --\ > -- director -- -- NFS > IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/ > > > So what happens if user1 at example.tld receives a mail? > > - The director decides to connect to backend1 which in turn stores > the mail on the NFS share and the index file locally? > > - Then, user1 at example.tld connects to one of the frontends. Does the > director know that, earlier, this user received a mail and proxies him > to backend1 too? Timo? This post is from March, right before you introduced the stones :D From robert at schetterer.org Sun Apr 1 09:43:13 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 01 Apr 2012 08:43:13 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <4F77F901.30605@schetterer.org> Am 31.03.2012 23:28, schrieb Timo Sirainen: > For the last few days I've been thinking about my company and what it really should do, and as much as the current plan seems reasonable, I think in good conscience I really can't help but to bring up an alternative plan: > > The support for Dovecot is quite good already, but wouldn't it be better if it was GUARANTEED that your email server was ALWAYS working with zero downtime or other problems? After some thinking, planning and consulting with my spiritual guides I realized I really should just go ahead and do it: > > From now on you can buy blessed Dovecot stones (http://www.dovecot.org/tmp/stone1.jpg), which are guaranteed to keep your email server running. I don't think there's really any further reason for me to provide commercial support. Probably the whole Dovecot mailing list will be pretty much useless after this as well, so I might as well stop posting there. If you have any problems, you'll just buy a Dovecot stone and the problem will go away. If for any reason that doesn't work, it means that your problem is more difficult than most and you need to buy a few more stones. Please remember that each shipment may take up to 3 weeks to reach you, so get your stones early and in sufficient quantities. > > A small email server will be happy enough with a single small Dovecot stone, but the more users you have the more stones you'll likely need. This is because most people are rather unhappy with their lives, and they project their negative energies towards their surroundings, including their email servers. To overcome this negative energy you need to have a counterbalancing positive energy, which the Dovecot stones will provide you. > > The Dovecot stones are hand painted from the finest materials available to the sister of the author of Dovecot that she could find in the local shops. Each stone contains such love and dedication towards the well being of Dovecot that you will shed tears when you lay your eyes upon their physical being. The images provided here unfortunately don't do justice to them. > > http://www.dovecot.org/tmp/stone2.jpg - This is how i keep my home desktop running all day every day. I have NEVER broken a single hard drive in my life. I don't even bother to keep backups, because I KNOW it is useless for me. > > A small Dovecot stone can be bought for 10 euros, a medium sized stone for 100 euros and a large stone for 500 euros. Each large stone can usually handle around 1000-2000 users before you need to buy more. If you have millions of users we'll need to find you a larger rock from my parents' backyard. There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago), so their prices need to be negotiated in advance. Hi Timo, perhaps the ghosts in the machines will now be very happy starting eat such bricks starting in the magical Ides of April so lets pray you ll never get out of Jedi power to bless the stones finally i am looking for a stone interface , meanwhile holy rays might have power enough, preparing them near the servers in the datacenter where i allready have prisoned evil trolls -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From anyaddress at gmx.net Sun Apr 1 10:28:40 2012 From: anyaddress at gmx.net (Tom Fernandes) Date: Sun, 1 Apr 2012 09:28:40 +0200 Subject: [Dovecot] namespace hierarchy Message-ID: <201204010928.41017.anyaddress@gmx.net> Hi, I'm migrating from Courier to Dovecot and would like to keep the original namespace and add two new ones on the new server. There's only one namespace in Courier which is "INBOX." . The seperator is "." In the new server I would like to have 3 namespaces (private, public, shared) and use "/" as separator. Is it possible to keep "INBOX." for private, so that the users mailboxes appear below the Inbox, but have public and shared on the same hierarchy-level like the Inbox? When I set prefix to "INBOX/" for private I get the users mailboxes below the Inbox, like I have it Courier now. But then at least the public, probably also the shared namespace doesn't work any more. I get "unknown subscription namespace" when trying to subscribe to some of the folders. My current configuration is: namespace: type: private separator: / inbox: yes list: yes subscriptions: yes namespace: type: public separator: / prefix: public/ location: maildir:/var/vmail/example.com/public list: yes namespace: type: shared separator: / prefix: shared/%%u/ location: maildir:%%h/Maildir list: yes warm regards, Tom From trashcan at odo.in-berlin.de Sun Apr 1 10:36:22 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sun, 1 Apr 2012 09:36:22 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: Nice ;-) On 31.03.2012, at 23:28, Timo Sirainen wrote: > A small Dovecot stone can be bought for 10 euros, a medium sized stone for 100 euros and a large stone for 500 euros. Each large stone can usually handle around 1000-2000 users before you need to buy more. If you have millions of users we'll need to find you a larger rock from my parents' backyard. There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago), so their prices need to be negotiated in advance. And sorry for the previous Mail. Mail.app from Apple isn't the best program ;-) Regards, Michael From marcin at mejor.pl Sun Apr 1 12:13:36 2012 From: marcin at mejor.pl (=?ISO-8859-2?Q?Marcin_Miros=B3aw?=) Date: Sun, 01 Apr 2012 11:13:36 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <4F781C40.40102@mejor.pl> W dniu 2012-03-31 23:28, Timo Sirainen pisze: > For the last few days I've been thinking about my company and what it really should do, and as much as the current plan seems reasonable, I think in good conscience I really can't help but to bring up an alternative plan: > > The support for Dovecot is quite good already, but wouldn't it be better if it was GUARANTEED that your email server was ALWAYS working with zero downtime or other problems? After some thinking, planning and consulting with my spiritual guides I realized I really should just go ahead and do it: > > From now on you can buy blessed Dovecot stones (http://www.dovecot.org/tmp/stone1.jpg), which are guaranteed to keep your email server running. I don't think there's really any further reason for me to provide commercial support. Probably the whole Dovecot mailing list will be pretty much useless after this as well, so I might as well stop posting there. If you have any problems, you'll just buy a Dovecot stone and the problem will go away. If for any reason that doesn't work, it means that your problem is more difficult than most and you need to buy a few more stones. Please remember that each shipment may take up to 3 weeks to reach you, so get your stones early and in sufficient quantities. > > A small email server will be happy enough with a single small Dovecot stone, but the more users you have the more stones you'll likely need. This is because most people are rather unhappy with their lives, and they project their negative energies towards their surroundings, including their email servers. To overcome this negative energy you need to have a counterbalancing positive energy, which the Dovecot stones will provide you. > > The Dovecot stones are hand painted from the finest materials available to the sister of the author of Dovecot that she could find in the local shops. Each stone contains such love and dedication towards the well being of Dovecot that you will shed tears when you lay your eyes upon their physical being. The images provided here unfortunately don't do justice to them. > > http://www.dovecot.org/tmp/stone2.jpg - This is how i keep my home desktop running all day every day. I have NEVER broken a single hard drive in my life. I don't even bother to keep backups, because I KNOW it is useless for me. > > A small Dovecot stone can be bought for 10 euros, a medium sized stone for 100 euros and a large stone for 500 euros. Each large stone can usually handle around 1000-2000 users before you need to buy more. If you have millions of users we'll need to find you a larger rock from my parents' backyard. There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago), so their prices need to be negotiated in advance. Hello! I'm using dovecot on vps, how can i use dovecot stone? Is it enough to use photo of stone as boot splash or should i start dovecot with LD_PRELOAD="/path_to_photo.jpg"? From patrickdk at patrickdk.com Sun Apr 1 17:22:30 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Sun, 01 Apr 2012 10:22:30 -0400 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <20120401102230.Horde.3VgjapLnE6FPeGSmb70RjgA@mail.patrickdk.com> Quoting Timo Sirainen : > For the last few days I've been thinking about my company and what > it really should do, and as much as the current plan seems > reasonable, I think in good conscience I really can't help but to > bring up an alternative plan: > > The support for Dovecot is quite good already, but wouldn't it be > better if it was GUARANTEED that your email server was ALWAYS > working with zero downtime or other problems? After some thinking, > planning and consulting with my spiritual guides I realized I really > should just go ahead and do it: > > From now on you can buy blessed Dovecot stones > (http://www.dovecot.org/tmp/stone1.jpg), which are guaranteed to > keep your email server running. I don't think there's really any > further reason for me to provide commercial support. Probably the > whole Dovecot mailing list will be pretty much useless after this as > well, so I might as well stop posting there. If you have any > problems, you'll just buy a Dovecot stone and the problem will go > away. If for any reason that doesn't work, it means that your > problem is more difficult than most and you need to buy a few more > stones. Please remember that each shipment may take up to 3 weeks > to reach you, so get your stones early and in sufficient quantities. > > A small email server will be happy enough with a single small > Dovecot stone, but the more users you have the more stones you'll > likely need. This is because most people are rather unhappy with > their lives, and they project their negative energies towards their > surroundings, including their email servers. To overcome this > negative energy you need to have a counterbalancing positive energy, > which the Dovecot stones will provide you. > > The Dovecot stones are hand painted from the finest materials > available to the sister of the author of Dovecot that she could find > in the local shops. Each stone contains such love and dedication > towards the well being of Dovecot that you will shed tears when you > lay your eyes upon their physical being. The images provided here > unfortunately don't do justice to them. > > http://www.dovecot.org/tmp/stone2.jpg - This is how i keep my home > desktop running all day every day. I have NEVER broken a single > hard drive in my life. I don't even bother to keep backups, because > I KNOW it is useless for me. > > A small Dovecot stone can be bought for 10 euros, a medium sized > stone for 100 euros and a large stone for 500 euros. Each large > stone can usually handle around 1000-2000 users before you need to > buy more. If you have millions of users we'll need to find you a > larger rock from my parents' backyard. There are only a limited > number of such rocks available (we disposed of most of them to our > neighbors' yards years ago), so their prices need to be negotiated > in advance. I'm unable to locate the order page. Or should I reply to the maillist with my creditcard info? From christian.rohmann at frittentheke.de Sun Apr 1 19:38:07 2012 From: christian.rohmann at frittentheke.de (Christian Rohmann) Date: Sun, 01 Apr 2012 18:38:07 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <4F781C40.40102@mejor.pl> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> <4F781C40.40102@mejor.pl> Message-ID: <4F78846F.5050909@frittentheke.de> Hey all, On 04/01/2012 11:13 AM, Marcin Miros?aw wrote: > I'm using dovecot on vps, how can i use dovecot stone? Is it enough to > use photo of stone as boot splash or should i start dovecot with > LD_PRELOAD="/path_to_photo.jpg"? Valid questions, what about Dovecot vStones? Apart from this practical issue, who is your marketing advisor? Timo in cloudy times like these, one needs product names that assure customers it's compatible to the Cloud: I suggest something like: Dovecot CloudProtector Stone 2000 S Dovecot CloudProtector Stone 4500 GT Dovecot CloudProtector Stone 6900 GTX turbo Also for such sheer value in less nightly calls on broken servers, you need to price the things higher. This will underline that the stones are of exceptional quality and energy. Regards Christian From robert at schetterer.org Sun Apr 1 20:51:13 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 01 Apr 2012 19:51:13 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <4F78846F.5050909@frittentheke.de> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> <4F781C40.40102@mejor.pl> <4F78846F.5050909@frittentheke.de> Message-ID: <4F789591.1000502@schetterer.org> Am 01.04.2012 18:38, schrieb Christian Rohmann: > Hey all, > > On 04/01/2012 11:13 AM, Marcin Miros?aw wrote: >> I'm using dovecot on vps, how can i use dovecot stone? Is it enough to >> use photo of stone as boot splash or should i start dovecot with >> LD_PRELOAD="/path_to_photo.jpg"? > > > Valid questions, what about Dovecot vStones? > > Apart from this practical issue, who is your marketing advisor? Timo in > cloudy times like these, one needs product names that assure customers > it's compatible to the Cloud: > > I suggest something like: > Dovecot CloudProtector Stone 2000 S > Dovecot CloudProtector Stone 4500 GT > Dovecot CloudProtector Stone 6900 GTX turbo > > > Also for such sheer value in less nightly calls on broken servers, you > need to price the things higher. This will underline that the stones are > of exceptional quality and energy. > > > > Regards > > Christian > by the way, are they kryptonite checked ? I dont wanna get in problems with Mr Kent ? http://en.wikipedia.org/wiki/Kryptonite -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From stsiol at yahoo.co.uk Mon Apr 2 00:54:40 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Sun, 1 Apr 2012 22:54:40 +0100 (BST) Subject: [Dovecot] Mail migration troubles Message-ID: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> Hello all, OK, I would really appreciate your help on this. I managed to do a pretty straight-forward and flawless migration on dovecot from one domain to another. However, I have this nagging problem with most of the thunderbird MTAs. When I try to delete some "test" messages I sent to make sure that everything works, it comes back with the following message on the bottom of the Thunderbird window : "The current operation on InBox did not succeed.? The mail server for account "Account at bloomingdomain.gr"? responded [TRYCREATE] mailbox doesn't exist : Trash" Now, I know it has to do something with automatically creating a? trash folder. But don't know anything more. Could someone shed some light on this ? Dovecot v1.2.15 AFAIK CentOS 5.5 x86 (32-bit) other stuff On Clients : Thunderbird v11.0.1 (XP Pro / 32-bit) s. ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis? From stsiol at yahoo.co.uk Mon Apr 2 00:59:25 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Sun, 1 Apr 2012 22:59:25 +0100 (BST) Subject: [Dovecot] strike my last Message-ID: <1333317565.13835.YahooMailNeo@web132204.mail.ird.yahoo.com> strike my last, looks like its a thunderbird issue Apologies... s. ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From jeetuindian at gmail.com Mon Apr 2 10:31:20 2012 From: jeetuindian at gmail.com (Jitendra Bhaskar) Date: Mon, 2 Apr 2012 13:01:20 +0530 Subject: [Dovecot] Dsync automation Message-ID: Hi, I am using dovecot-2.1.3 on centos 5.7. Its running fine. I have two systems for mail server one is for backup. Currently I am running rsync daily. But I want to use dsync. I went through all the http://wiki2.dovecot.org/Tools/Dsync and http://blog.dovecot.org/2012/02/dovecot-clustering-with-dsync-based.htmlbut I couldn't understood how to automatize dsync. Can anybody give me any idea ho to do it ? -- * Thanks & Regards * *Jitendra Kumar Bhaskar* Cell:- +91 7306311531 +91 8102997821 From stsiol at yahoo.co.uk Mon Apr 2 13:19:47 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 2 Apr 2012 11:19:47 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> Message-ID: <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> I have a suspicion that, as usual, no one is going to? answer my message. Because of the fact that this is a call for help and no-one ever bothers, I 'll just have to re-phrase my question : Here's what I found on the dovecot wiki : imapsync Here's an example of how to run imapsync for a single user: imapsync --syncinternaldates \??--host1 192.168.1.57 --authmech1 LOGIN --user1 leah at example.com --password1 secret \??--host2 127.0.0.1????--authmech2 LOGIN --user2 leah at example.com --password2 secret It is quite easy to script this for a number of users, assuming you have their passwords. Even if you do not, imapsync also supports logging in as an admin user that has the ability to copy message for sub users, and a variety of other authentication options. Is the above what I need if I migrate mailboxes FROM dovecot TO dovecot ? E.g. in my example, I only copy mailboxes from an old "user at domainA.gr" to "user at domainB.gr". Is the above all I need ? Is there a native dovecot tool for this or should I just use imapsync ? Thank you again, s. ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis ----- Original Message ----- > From: Spyros Tsiolis > To: Spyros Tsiolis > Cc: > Sent: Monday, 2 April 2012, 11:24 > Subject: Re: [Dovecot] Mail migration troubles > > Hello again, > > Well, it seems I have trouble with the migration after all. > After the migration and eventhough all folders and messages > are on their respective places, any new mail users get > inside their mailbox cannot be dragged-and-dropped to > their respective folders. > > So, if a user has a folder with my name on it and I send > him an e-mail, the user will try to drag-and-drop that > message to the folder with my name, but the message won't > move. > > Any gurus out there on this ? > Any help would be greatly appreciated. > > spyros > > > ? > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis > ? From CMarcus at Media-Brokers.com Mon Apr 2 13:30:06 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 02 Apr 2012 06:30:06 -0400 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> Message-ID: <4F797FAE.7030104@Media-Brokers.com> On 2012-04-02 6:19 AM, Spyros Tsiolis wrote: > Is the above what I need if I migrate mailboxes FROM dovecot TO dovecot ? > E.g. in my example, I only copy mailboxes from an old "user at domainA.gr" to > "user at domainB.gr". > > Is the above all I need ? > > Is there a native dovecot tool for this or should I just use imapsync ? Are these on ths same box? If so, why not just mv them?? -- Best regards, Charles From CMarcus at Media-Brokers.com Mon Apr 2 13:31:56 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 02 Apr 2012 06:31:56 -0400 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <4F79801C.8020804@Media-Brokers.com> On 2012-03-31 5:28 PM, Timo Sirainen wrote: > For the last few days I've been thinking about my company and what it > really should do, and as much as the current plan seems reasonable, I > think in good conscience I really can't help but to bring up an > alternative plan: > > The support for Dovecot is quite good already, but wouldn't it be > better if it was GUARANTEED that your email server was ALWAYS working > with zero downtime or other problems? After some thinking, planning > and consulting with my spiritual guides I realized I really should > just go ahead and do it: > > From now on you can buy blessed Dovecot stones > (http://www.dovecot.org/tmp/stone1.jpg), which are guaranteed to keep > your email server running. I don't think there's really any further > reason for me to provide commercial support. Alright Timo, where do I send the bill for my new keyboard? Mine is no ruined from the coffee I just spilled all over it... Rotflmao! Thanks, I needed that... -- Best regards, Charles From arne at drlinux.no Mon Apr 2 13:36:38 2012 From: arne at drlinux.no (Arne K. Haaje) Date: Mon, 02 Apr 2012 12:36:38 +0200 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> Message-ID: <5628413.EKgo53ll8l@trillian> Mandag 2. april 2012 11.19.47 skrev Spyros Tsiolis: [snip] > Is the above what I need if I migrate mailboxes FROM dovecot TO dovecot ? > E.g. in my example, I only copy mailboxes from an old "user at domainA.gr" to > "user at domainB.gr". > > Is the above all I need ? That worked for me when creating a backup copy of a mailbox, but with a different login. I copied mailbox A to mailbox B, then created an entry in the auth table that pointed to mailbox B. This was with maildir. I don't know if it works the same with all other types. Regards, Arne -- Arne K. Haaje http://www.drlinux.no/ LinkedIn: http://no.linkedin.com/pub/arne-haaje/27/189/bb From stsiol at yahoo.co.uk Mon Apr 2 13:51:53 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 2 Apr 2012 11:51:53 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <4F797FAE.7030104@Media-Brokers.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> Message-ID: <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> > On 2012-04-02 6:19 AM, Spyros Tsiolis wrote: >> Is the above what I need if I migrate mailboxes FROM dovecot TO dovecot ? >> E.g. in my example, I only copy mailboxes from an old? > "user at domainA.gr" to >> "user at domainB.gr". >>? >> Is the above all I need ? >>? >> Is there a native dovecot tool for this or should I just use imapsync ? >? > Are these on ths same box? If so, why not just mv them?? Hi Charles, Yes, on the same box, on the same drive, on the same partition/filesystem. I have a box with dovecot/XMail (pop3/smtp) and two domains; DomainA and DomainB The company bosses decided to move about 20 mailboxes from DomainA to DomainB. As far as I can tell everything went well except for one user (so far) that he tries to move fresh mail to (mv'd) folders and it doesn't get there. Also when he tries to delete messages, they don't get deleted. I did "cp -r" actually and kept the old mailboxes nearby just in case. Thanks, s. ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From CMarcus at Media-Brokers.com Mon Apr 2 13:55:12 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 02 Apr 2012 06:55:12 -0400 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <4F798590.6060701@Media-Brokers.com> On 2012-04-02 6:51 AM, Spyros Tsiolis wrote: > As far as I can tell everything went well except for one > user (so far) that he tries to move fresh mail to (mv'd) > folders and it doesn't get there. > Also when he tries to delete messages, they don't get > deleted. > I did "cp -r" actually and kept the old mailboxes nearby > just in case. What do the logs show when he tries this? Did you compare FS permissions on a working setup and the non-working one? -- Best regards, Charles From nmilas at noa.gr Mon Apr 2 14:44:48 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Mon, 02 Apr 2012 14:44:48 +0300 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <4F799130.2020806@noa.gr> On 2/4/2012 1:51 ??, Spyros Tsiolis wrote: > I did "cp -r" actually and kept the old mailboxes nearby > just in case. If on CentOS/RHEL (don't know for other systems), you could try using explicitly: /bin/cp -pr for better results, because cp is aliased. You might already be aware of this, but just in case... Nick From stsiol at yahoo.co.uk Mon Apr 2 15:05:23 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 2 Apr 2012 13:05:23 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <4F799130.2020806@noa.gr> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> <4F799130.2020806@noa.gr> Message-ID: <1333368323.2437.YahooMailNeo@web132204.mail.ird.yahoo.com> > On 2/4/2012 1:51 ??, Spyros Tsiolis wrote: >? >> I did "cp -r" actually and kept the old mailboxes nearby >> just in case. >? > If on CentOS/RHEL (don't know for other systems), you could try using? > explicitly: >? > ? /bin/cp -pr >? > for better results, because cp is aliased. >? > You might already be aware of this, but just in case... >? > Nick > Hi Nick, No I didn't know this. I mean I had it stored in the back of my head but? to tell you the truth I would never use it. Yes, thank you kindly for pointing that. I'll use that switch from now on. And , yes, I am using CentOS. Can't live without it, no matter what the people say :-) Thank you again, s. ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From stsiol at yahoo.co.uk Mon Apr 2 15:49:28 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 2 Apr 2012 13:49:28 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <4F798590.6060701@Media-Brokers.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> <4F798590.6060701@Media-Brokers.com> Message-ID: <1333370968.69235.YahooMailNeo@web132201.mail.ird.yahoo.com> ----- Original Message ----- > From: Charles Marcus > To: dovecot at dovecot.org > Cc: > Sent: Monday, 2 April 2012, 12:55 > Subject: Re: [Dovecot] Mail migration troubles > > On 2012-04-02 6:51 AM, Spyros Tsiolis wrote: >> As far as I can tell everything went well except for one >> user (so far) that he tries to move fresh mail to (mv'd) >> folders and it doesn't get there. >> Also when he tries to delete messages, they don't get >> deleted. >> I did "cp -r" actually and kept the old mailboxes nearby >> just in case. > > What do the logs show when he tries this? > > Did you compare FS permissions on a working setup and the non-working one? > > -- > Best regards, > > Charles > Hi Charles, Well, I checked the dovecot log files? (dovecot-info.log / dovecot.log)? It doesn't show something suspicious. Just the usual login logout and password chatter. At this point, I suspect it's thunderbird again. I asked the rest of the people how they are doing and they said they were? doing fine. So I suspect there's something wrong with this specific dovecot installation. Thank you though, s. ? ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From jdonovan at beth.k12.pa.us Mon Apr 2 16:01:04 2012 From: jdonovan at beth.k12.pa.us (jeff donovan) Date: Mon, 2 Apr 2012 09:01:04 -0400 Subject: [Dovecot] setting up ldap for sasl Message-ID: <98F8F008-100B-4CAF-BE0E-F2BBAC50B2D7@beth.k12.pa.us> Greetings im new to the list dovecot --version 2.0.13 ubuntu 11.10 i want to use dovecot to authenticate my ldap users. My local users can authenticate and send email. Using postfix sasl/tls dovecot pam. I have read many docs/tutorials, which all have a plethora of options and information. my file structure is different than most of the documentation. I have ../etc/dovecot.conf which points to. !include_try /usr/share/dovecot/protocols.d/*.protocol !include conf.d/*.conf !include_try local.conf which leads me to /etc/dovecot/conf.d, which is full of config files. All were traditionally located in dovecot.conf in previous versions. im using PAM to authenticate my local system users. I have read that could adjust PAM, to use LDAP. How do I configure dovecot to keep using pam for local users and ldap for remote ? Do i just add the passdb lines dovecot.conf: passdb ldap { args = /etc/dovecot-ldap.conf } userdb prefetch { } # The userdb below is used only by deliver. userdb ldap { args = /etc/dovecot-ldap.conf } dovecot-ldap.conf: pass_attrs = uid=user,userPassword=password, homeDirectory=userdb_home,uidNumber=userdb_uid,gidNumber=userdb_gid # For deliver: user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid any assistance would be helpful. -j -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2497 bytes Desc: not available URL: From CMarcus at Media-Brokers.com Mon Apr 2 16:20:49 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 02 Apr 2012 09:20:49 -0400 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333370968.69235.YahooMailNeo@web132201.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> <4F798590.6060701@Media-Brokers.com> <1333370968.69235.YahooMailNeo@web132201.mail.ird.yahoo.com> Message-ID: <4F79A7B1.4030908@Media-Brokers.com> On 2012-04-02 8:49 AM, Spyros Tsiolis wrote: > On2 April 2012, 12:55, Charles Marcus wrote: >> On 2012-04-02 6:51 AM, Spyros Tsiolis wrote: >>> As far as I can tell everything went well except for one >>> user (so far) that he tries to move fresh mail to (mv'd) >>> folders and it doesn't get there. >>> Also when he tries to delete messages, they don't get >>> deleted. >>> I did "cp -r" actually and kept the old mailboxes nearby >>> just in case. >> What do the logs show when he tries this? >> >> Did you compare FS permissions on a working setup and the >> non-working one? > Well, I checked the dovecot log files > (dovecot-info.log / dovecot.log) > It doesn't show something suspicious. > Just the usual login logout and > password chatter. Well, maybe one of the reasons people don't always try to help you is you often just don't respond with answers to questions... WHAT ARE THE FS PERMISSIONS ON THAT USERS MAIL FOLDERS? And don't just say 'they are ok' - SHOW them... And how about SHOWING SOME LOG EXCERPTS (and increase verbosity if needed), instead of just saying you don't see anything relevant. -- Best regards, Charles From jg at softjury.de Mon Apr 2 17:06:28 2012 From: jg at softjury.de (Jan Phillip Greimann) Date: Mon, 02 Apr 2012 16:06:28 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <4F79B264.9030007@softjury.de> STOP TALKING AND TAKE MY MONEY!!!!1111elf Am 31.03.2012 23:28, schrieb Timo Sirainen: > For the last few days I've been thinking about my company and what it really should do, and as much as the current plan seems reasonable, I think in good conscience I really can't help but to bring up an alternative plan: > > The support for Dovecot is quite good already, but wouldn't it be better if it was GUARANTEED that your email server was ALWAYS working with zero downtime or other problems? After some thinking, planning and consulting with my spiritual guides I realized I really should just go ahead and do it: > > From now on you can buy blessed Dovecot stones (http://www.dovecot.org/tmp/stone1.jpg), which are guaranteed to keep your email server running. I don't think there's really any further reason for me to provide commercial support. Probably the whole Dovecot mailing list will be pretty much useless after this as well, so I might as well stop posting there. If you have any problems, you'll just buy a Dovecot stone and the problem will go away. If for any reason that doesn't work, it means that your problem is more difficult than most and you need to buy a few more stones. Please remember that each shipment may take up to 3 weeks to reach you, so get your stones early and in sufficient quantities. > > A small email server will be happy enough with a single small Dovecot stone, but the more users you have the more stones you'll likely need. This is because most people are rather unhappy with their lives, and they project their negative energies towards their surroundings, including their email servers. To overcome this negative energy you need to have a counterbalancing positive energy, which the Dovecot stones will provide you. > > The Dovecot stones are hand painted from the finest materials available to the sister of the author of Dovecot that she could find in the local shops. Each stone contains such love and dedication towards the well being of Dovecot that you will shed tears when you lay your eyes upon their physical being. The images provided here unfortunately don't do justice to them. > > http://www.dovecot.org/tmp/stone2.jpg - This is how i keep my home desktop running all day every day. I have NEVER broken a single hard drive in my life. I don't even bother to keep backups, because I KNOW it is useless for me. > > A small Dovecot stone can be bought for 10 euros, a medium sized stone for 100 euros and a large stone for 500 euros. Each large stone can usually handle around 1000-2000 users before you need to buy more. If you have millions of users we'll need to find you a larger rock from my parents' backyard. There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago), so their prices need to be negotiated in advance. From wgrcunha at gmail.com Mon Apr 2 17:14:26 2012 From: wgrcunha at gmail.com (Francisco Wagner C. Freire) Date: Mon, 2 Apr 2012 11:14:26 -0300 Subject: [Dovecot] Backtrace on mdbox index Message-ID: Hello, I current using dovecot with mdbox and on one account i get this error: doveadm(user at domain): Panic: file mail-index-sync-ext.c: line 209 (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) doveadm(user at domain): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x3fd8a) [0x7fae684fdd8a] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7fae684fde72] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7fae684d719f] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9bdc3) [0x7fae68c1edc3] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_ext_intro+0x1e6) [0x7fae68c1fa76] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_record+0x49f) [0x7fae68c21a3f] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_map+0x234) [0x7fae68c225a4] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_map+0x86) [0x7fae68c13cd6] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x8b609) [0x7fae68c0e609] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_open+0x210) [0x7fae68c0eaa0] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x591c6) [0x7fae68bdc1c6] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_map_get_zero_ref_files+0x2d) [0x7fae68bdcd6d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_purge+0xbc) [0x7fae68bde7ac] -> doveadm() [0x40a135] -> doveadm() [0x40a813] -> doveadm(doveadm_mail_single_user+0x61) [0x40a8d1] -> doveadm() [0x40aafd] -> doveadm(doveadm_mail_try_run+0x141) [0x40af11] -> doveadm(main+0x381) [0x4109c1] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7fae67d6ec8d] -> doveadm() [0x409ff9] Aborted All doveadm commands fail on thie error, index / force-resync / search / fetch etc. Any help? From stsiol at yahoo.co.uk Mon Apr 2 17:29:39 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 2 Apr 2012 15:29:39 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <4F79A7B1.4030908@Media-Brokers.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> <4F798590.6060701@Media-Brokers.com> <1333370968.69235.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F79A7B1.4030908@Media-Brokers.com> Message-ID: <1333376979.98299.YahooMailNeo@web132205.mail.ird.yahoo.com> Hi again Charles, >>> Did you compare FS permissions on a working setup and the >>> non-working one? I don't have a non-working setup of dovecot >> Well, I checked the dovecot log files >> (dovecot-info.log / dovecot.log) >> It doesn't show something suspicious. >> Just the usual login logout and >> password chatter. >? > Well, maybe one of the reasons people don't always try to help you is you? > often just don't respond with answers to questions... Oh, I see. It's like what I ask questions about issues and? get answers on other issues... ? > WHAT ARE THE FS PERMISSIONS ON THAT USERS MAIL FOLDERS? And don't just say? > 'they are ok' - SHOW them... All users run with the virtual mail account (vmail) accounts (if I recall correctly) are stored in %maildir format. Do you want me to display user perms ? OK ! Fine. Here's the contents of the folder with the problem : [root at mailserver mailaccount]# ls -l total 8 drwx------ 6 vmail vmail 4096 Apr ?2 17:08 Maildir -rw-rw-rw- 1 vmail vmail ?141 Mar 30 17:21 user.tab Is this what you need ?? I didn't answer this not because I don't want to but because I don't know if this is what you want. There's no need to shout. ? > And how about SHOWING SOME LOG EXCERPTS (and increase verbosity if needed),? > instead of just saying you don't see anything relevant. Here goes : Apr 02 14:26:02 auth(default): Info: client out: OK1user=account at domain.gr Apr 02 14:26:02 auth(default): Info: master in: REQUEST1785294431 Apr 02 14:26:02 auth(default): Info: master out: USER1785account at domain.gruid=501gid=502home=/ho me/vmail/account at domain.gr Apr 02 14:26:02 imap-login: Info: Login: user=, method=PLAIN, rip=192.168.0.20, lip=192.168.1. 4, TLS Apr 02 14:26:02 auth(default): Info: new auth connection: pid=29450 Apr 02 14:27:02 auth(default): Info: client in: AUTH1PLAINservice=imapsecuredlip=192.168.1.4rip=192. 168.0.20lport=993rport=4494 Apr 02 14:27:02 auth(default): Info: client out: CONT1 Apr 02 14:27:02 auth(default): Info: client in: CONT1AHRpY2ldA29ZnzWzc9c5cgxTQA== Apr 02 14:27:02 auth(default): Info: passwd-file(account at domain.gr,192.168.0.20): lookup: user=account at domain.gr ?file=/etc/dovecot/passwd Apr 02 14:27:02 auth(default): Info: client out: OK1user=account at domain.gr Apr 02 14:27:02 auth(default): Info: master in: REQUEST1786294501 Apr 02 14:27:02 auth(default): Info: master out: USER1786account at domain.gruid=501gid=502home=/ho me/vmail/account at domain.gr Apr 02 14:27:02 imap-login: Info: Login: user=, method=PLAIN, rip=192.168.0.20, lip=192.168.1. 4, TLS Apr 02 14:27:02 auth(default): Info: new auth connection: pid=29453 s. ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From giom87 at libero.it Mon Apr 2 17:39:08 2012 From: giom87 at libero.it (Giovanni Mancuso) Date: Mon, 02 Apr 2012 16:39:08 +0200 Subject: [Dovecot] Connection queue full on dovecot 2.0.13 Message-ID: <4F79BA0C.1070104@libero.it> Hi, i have a problem with dovecot 2.0.13. I have one dovecot in Front-end that has only: passdb { driver = static args = proxy=y host=10.0.0.6 nopassword=y } In Back-end i have one dovecot that does auth and exports imap/pop3 ports. In dovecot's log of Front-end i see: /Apr 02 14:33:34 imap-login: Info: proxy(//user at example.com/ /): started proxying to 10.0.0.6:143: user=/>, method=PLAIN, rip=//xx.xx.xx.xx//, lip=xx.xx.xx.xx, TLS Apr 02 14:34:36 imap-login: Info: Disconnected: *Connection queue full *(auth failed, 1 attempts): user=/>, method=PLAIN, rip=//xx.xx.xx.xx//, lip=//xx.xx.xx.xx//, TLS/ I see this wiki page: http://wiki2.dovecot.org/LoginProcess but i read: It works by using a number of long running login processes, each handling a number of connections. This loses much of the security benefits of the login process design, because in case of a security hole (in Dovecot or SSL library) the attacker is now able to see other users logging in and steal their passwords, read their mails, etc. Is there another way? Thanks From zybi at talex.pl Mon Apr 2 18:45:25 2012 From: zybi at talex.pl (=?ISO-8859-2?Q?Artur_Zaprza=B3a?=) Date: Mon, 02 Apr 2012 17:45:25 +0200 Subject: [Dovecot] lmtp sometimes fails to deliver a message to all recipients Message-ID: <4F79C995.9060607@talex.pl> Dovecot version: 2.1.3 lmtp config: protocol lmtp { mail_plugins = zlib acl sieve } Running strace on lmtp reveals that when lmtp delivers a message to multiple recipients it saves the message to a file in the first mailbox and hard links that file to other mailboxes. But when in the meantime of delivery that file is deleted, lmtp fails to deliver the message to remaining mailboxes. For example a message for 46 recipients is delivered by lmtp to the first 12 only (fortunately postfix retries delivery for remaining recipients) and for the rest lmtp logs the following error messages: lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Message was expunged (guid) lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: script /vmail/domain/foo/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /vmail/domain/foo/.dovecot.sieve.log may reveal additional details) I'm seeing this error for about 10 messages per hour. Best regards, Artur Zaprza?a From adorca at uda.ad Mon Apr 2 19:11:06 2012 From: adorca at uda.ad (Aleix Dorca) Date: Mon, 2 Apr 2012 18:11:06 +0200 Subject: [Dovecot] imapsync, namespaces and Dovecot 2.x Message-ID: Hi people, I recently tried to migrate from Exchange using imapsync. It didn't work because it complained that dovecot had no namespace capablity. After looking through Dovecot's config files I found that namespaces could be defined, but if you did not a 'private' default one was created. In the 2.x wiki documentation (http://wiki2.dovecot.org/Migration) it says nothing about Namespaces. I don't know if I have to create one or not. If I add the parameters --sep2 . --prefix2 '' to imapsync then it works fine. I'm proxying using dovecot as frontend and backend, version 2.0.18 (The one that comes with Debian Wheezy). Any thoughts, please? Thanks, Aleix. From moseleymark at gmail.com Mon Apr 2 19:33:32 2012 From: moseleymark at gmail.com (Mark Moseley) Date: Mon, 2 Apr 2012 09:33:32 -0700 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: On Sat, Mar 31, 2012 at 2:28 PM, Timo Sirainen wrote: > For the last few days I've been thinking about my company and what it really should do, and as much as the current plan seems reasonable, I think in good conscience I really can't help but to bring up an alternative plan: > >... I'm slightly concerned that there's been no mention of what license these stones are going to be released under. GPL2? GPL3? Apache? I'm just hoping these aren't some sort of "open core" stones that will only work for basic features but that I'll end up needing to buy "Enterprise-grade stones" to cover large clusters. From kurt-dovecot at se.rit.edu Tue Apr 3 00:23:20 2012 From: kurt-dovecot at se.rit.edu (Kurt Mosiejczuk) Date: Mon, 02 Apr 2012 17:23:20 -0400 Subject: [Dovecot] imapdir still a configuration option? Message-ID: <4F7A18C8.3000904@se.rit.edu> I've been looking at moving an old IMAPdir-based solution to dovecot. I came across references that dovecot (to my pleasant surprise) added imapdir support in 1.1. However, in trying 2.0.13 (from an OpenBSD package) things start okay, but I see in the logs: "Initialization failed: Initializing mail storage from mail_location setting failed: Unknown mail storage driver imapdir" Poking around in the sources, I still see lots of references to imapdir, but see very little in the documentation. Is IMAPdir still supported in 2.0.x (or even 2.1.x)? I'd prefer not to have to migrate users via renaming, especially since they like the IMAPdir layout for their own searching. --Kurt From micah at riseup.net Tue Apr 3 01:26:43 2012 From: micah at riseup.net (Micah Anderson) Date: Mon, 02 Apr 2012 18:26:43 -0400 Subject: [Dovecot] Dsync automation References: Message-ID: <87k41xrcrg.fsf@algae.riseup.net> Jitendra Bhaskar writes: > I am using dovecot-2.1.3 on centos 5.7. Its running fine. I have two > systems for mail server one is for backup. Currently I am running rsync > daily. But I want to use dsync. I went through all the > http://wiki2.dovecot.org/Tools/Dsync and > http://blog.dovecot.org/2012/02/dovecot-clustering-with-dsync-based.htmlbut > I couldn't understood how to automatize dsync. Can anybody give me any > idea ho to do it ? As far as I can tell, doing a for loop around 'doveadm user \*' is the only way. That will do a dsync of all the users. On the backup server you will need to decide what to do with the data. I've been rotating it on a daily basis to try and get several daily backups, then some weekly, and monthly. I've been working on a backupninja[0] handler to do dsync backups, but its still rough. micah 0. https://labs.riseup.net/code/projects/backupninja -- From micah at riseup.net Tue Apr 3 01:34:07 2012 From: micah at riseup.net (Micah Anderson) Date: Mon, 02 Apr 2012 18:34:07 -0400 Subject: [Dovecot] 2.1.3: doveadm mailbox delete lost the -s option Message-ID: <874nt1rcf4.fsf@algae.riseup.net> Looks like the recently added -s (for unsubscribing) option to doveadm mailbox delete went missing: root at vireo# /usr/bin/doveadm mailbox delete -u micahtest -s restored delete: invalid option -- 's' doveadm mailbox delete [-u |-A] [-S ] [-s] [...] although the option still is shown as valid :) micah -- From zarf at klacto.net Tue Apr 3 02:02:17 2012 From: zarf at klacto.net (FZiegler) Date: Mon, 02 Apr 2012 19:02:17 -0400 Subject: [Dovecot] dovecot and unison Message-ID: <4F7A2FF9.7000204@klacto.net> I am successfully using dovecot purely as a personal local mail store on my desktop. (There is only one account, and it's only ever accessed by local mail clients on the machine. The point is to have a common store I can use with any client; plus, I prefer dovecot's Mailbox storage to Thunderbird's mboxes.) Now I'd like if possible, to replicate this setup on my laptop and keep both in sync with unison (http://www.cis.upenn.edu/~bcpierce/unison/), which I am already using to sync much of my home dir about once a day. I found at least one positive message regarding this topic (http://dovecot.org/list/dovecot/2010-April/048092.html), but I feel I could use some more advice. Namely, which of the ancillary files should I be syncing (or perhaps deleting before sync)? In addition to the mails themselves I'm seeing: -rw------- 1 fz 501 13 Apr 1 17:10 dovecot-keywords -rw------- 1 fz 501 2483 Apr 1 17:10 dovecot-uidlist -rw------- 1 fz 501 1040 Mar 29 14:50 dovecot.index -rw------- 1 fz 501 44032 Apr 1 17:10 dovecot.index.cache -rw------- 1 fz 501 9772 Apr 1 17:10 dovecot.index.log and at the top level (in .local/var/Maildir/): -rw------- 1 fz 501 8 Apr 1 17:10 dovecot-uidvalidity -r--r--r-- 1 fz 501 0 Apr 1 17:10 dovecot-uidvalidity.4efb323d -rw------- 1 fz 501 3288 Apr 1 17:10 dovecot.mailbox.log -rw------- 1 fz 501 4104 Mar 28 10:53 dovecot.mailbox.log.2 -rw------- 1 fz 501 5114 Apr 1 17:10 subscriptions In my naive trials I ended up at least once in a situation where Thunderbird would not see some subfolders (even though they were there). That's since fixed but I guess my question is, what's the safe recommended way to not have this happen? Thanks, FZiegler From stan at hardwarefreak.com Tue Apr 3 02:04:54 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 02 Apr 2012 18:04:54 -0500 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <4F7A3096.5050209@hardwarefreak.com> On 3/31/2012 4:28 PM, Timo Sirainen wrote: There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago) :) -- Stan From micah at riseup.net Tue Apr 3 02:15:28 2012 From: micah at riseup.net (Micah Anderson) Date: Mon, 02 Apr 2012 19:15:28 -0400 Subject: [Dovecot] dsync redesign References: <936E95DA-E032-4F8F-A323-DEBB7AAC0E1B@iki.fi> <87obrixcyp.fsf@algae.riseup.net> <4F721650.4030901@Media-Brokers.com> Message-ID: <87zkatpvxr.fsf@algae.riseup.net> Charles Marcus writes: > On 2012-03-27 11:47 AM, Micah Anderson wrote: >> One would be the ability to perform *intelligent* incremental / >> rotated backups. I can do this now by running a dsync backup >> operation and then doing manual hardlinking or moving of the backup >> directories (daily.1, daily.2, weekly.1, monthly.1, etc.), but it >> would be more intelligent if this were baked into the backup process. > > There are already numerous tools that do this flawlessly - I've been using > rsnapshot (which uses rsync) for this for years. Are you snapshotting your filesystem (using LVM, or SAN, or similar) before doing rsnapshot? Because if you aren't then rsync will not assuredly get everything in a consistent state. > I don't know if Timo should be spending his time reinventing the wheel. dsync backup is already here, and it is quite useful. > I'm much more interested in dsync working flawlessly to keep one or more > secondary servers in sync, and leave backups to backup software. I'm not against that idea, I just have not yet found a good way to use any backup software in such a way to handle large numbers of user's mail. > Although, one interesting piece that I am hopeful I'll be able to implement soon > (with Timo's professional help) is the ability to easily and automatically map > my rsnapshot snapshots directory to a read-only 'Backups' namespace that > automatically shows the snapshots by date and time as they are produced. This > way users could 'go back in time' anytime they wanted without having to call > me... :) Interesting idea, would be a great one to share with the community if you decide to do so. micah From jdonovan at beth.k12.pa.us Tue Apr 3 03:56:42 2012 From: jdonovan at beth.k12.pa.us (jeff donovan) Date: Mon, 2 Apr 2012 20:56:42 -0400 Subject: [Dovecot] setting up ldap for sasl In-Reply-To: <98F8F008-100B-4CAF-BE0E-F2BBAC50B2D7@beth.k12.pa.us> References: <98F8F008-100B-4CAF-BE0E-F2BBAC50B2D7@beth.k12.pa.us> Message-ID: <7EEE40B1-2E79-4814-B3E0-F4AE0FB45831@beth.k12.pa.us> On Apr 2, 2012, at 9:01 AM, jeff donovan wrote: > Greetings > im new to the list > dovecot --version > 2.0.13 > ubuntu 11.10 > i want to use dovecot to authenticate my ldap users. My local users can authenticate and send email. Using postfix sasl/tls dovecot pam. > > I have read many docs/tutorials, which all have a plethora of options and information. my file structure is different than most of the documentation. > I have ../etc/dovecot.conf which points to. > > !include_try /usr/share/dovecot/protocols.d/*.protocol > !include conf.d/*.conf > !include_try local.conf i included /etc/dovecot/conf.d/dovecot-ldap.conf and I get errors from dovecot config. doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/dovecot-ldap.conf line 17: Unknown setting: hosts doveconf: Error: managesieve-login: dump-capability process returned 89 i remove it, then it goes to the next option i have set; doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/dovecot-ldap.conf line 21: Unknown setting: uris doveconf: Error: managesieve-login: dump-capability process returned 89 any help would be great -j -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2497 bytes Desc: not available URL: From jdonovan at beth.k12.pa.us Tue Apr 3 05:28:13 2012 From: jdonovan at beth.k12.pa.us (jeff donovan) Date: Mon, 2 Apr 2012 22:28:13 -0400 Subject: [Dovecot] setting up ldap for sasl::solved:: In-Reply-To: <7EEE40B1-2E79-4814-B3E0-F4AE0FB45831@beth.k12.pa.us> References: <98F8F008-100B-4CAF-BE0E-F2BBAC50B2D7@beth.k12.pa.us> <7EEE40B1-2E79-4814-B3E0-F4AE0FB45831@beth.k12.pa.us> Message-ID: <18E8FC85-525D-4B46-8246-7B9A6FDC113F@beth.k12.pa.us> I had to compile with ldap-- duh. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2497 bytes Desc: not available URL: From tss at iki.fi Tue Apr 3 09:32:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 09:32:37 +0300 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> Message-ID: <1333434757.21461.32.camel@innu> On Sun, 2012-04-01 at 22:54 +0100, Spyros Tsiolis wrote: > However, I have this nagging problem with most of the thunderbird > MTAs. When I try to delete some "test" messages I sent to make sure > that everything works, it comes back with the following message on > the bottom of the Thunderbird window : > > "The current operation on InBox did not succeed. > The mail server for account "Account at bloomingdomain.gr" > responded [TRYCREATE] mailbox doesn't exist : Trash" That error probably happens when deleting a message. > Dovecot v1.2.15 AFAIK dovecot -n output would have helped. Mainly: do you have a namespace prefix or not? That's about the only thing I can think of related to that error message (e.g. dovecot has non-empty namespace prefix, while TB has empty namespace prefix). From tss at iki.fi Tue Apr 3 09:33:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 09:33:29 +0300 Subject: [Dovecot] Backtrace on mdbox index In-Reply-To: References: Message-ID: <1333434809.21461.33.camel@innu> On Mon, 2012-04-02 at 11:14 -0300, Francisco Wagner C. Freire wrote: > I current using dovecot with mdbox and on one account i get this error: > > doveadm(user at domain): Panic: file mail-index-sync-ext.c: line 209 > (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) Dovecot version? I think I've fixed this in recent v2.1.x. From tss at iki.fi Tue Apr 3 09:44:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 09:44:10 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F760B61.3090209@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> Message-ID: <1333435450.21461.35.camel@innu> On Fri, 2012-03-30 at 21:37 +0200, Patrick Westenberg wrote: > Nick Warr schrieb: > > > I think some of the new Dovecot (director?) software is user aware, but > > I don't know if it's quite ready for production. > > Yes, with director it should be something like that: > > MTA --(lmtp)--\ /--(lmtp)--> backend1 --\ > -- director -- -- NFS > MTA --(lmtp)--/ \--(lmtp)--> backend2 --/ > > > IMAP-User --> frontend1 --\ /--(imap)--> backend1 --\ > -- director -- -- NFS > IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/ The director above must be the same one for both IMAP and MTA. > So what happens if user1 at example.tld receives a mail? > > - The director decides to connect to backend1 which in turn stores > the mail on the NFS share and the index file locally? > > - Then, user1 at example.tld connects to one of the frontends. Does the > director know that, earlier, this user received a mail and proxies him > to backend1 too? Yes. Director process actually doesn't know anything about mail protocols, it simply does username -> backend mapping. The regular Dovecot IMAP/POP3/LMTP/doveadm proxying code handles the rest. From lukas.mueller at newmedia.ch Tue Apr 3 10:24:58 2012 From: lukas.mueller at newmedia.ch (=?iso-8859-1?Q?M=FCller_Lukas?=) Date: Tue, 3 Apr 2012 07:24:58 +0000 Subject: [Dovecot] Dovecot 1.2.9. next_uid was lowered Message-ID: Hi There A while ago I posted here about a crash of dovecot, that had to do with nfs and was quite certainly due to bad luck. But still one problem persists: We have a Mailbox, that is accessed from multiple users using IMAP. Occasionally all the Emails in the mailbox seem to be missing, when accessing the Mailbox through IMAP, although they're still there on the server. This seems to happen, if the users are accessing the Mailbox on Server1 and at the same time an email gets delivered to the mailbox on server2. Since it happens quite seldom, there must be other conditions or circumstances that have to be met for the problem to occur. Unfortunately I don't know which. Shortly after the mail gets delivered on server2, I get the following errors on server1: mail01 dovecot: IMAP(akzidenz at triner.ch): fdatasync(/data/vmail/example.com/user/dovecot-uidlist) failed: Input/output error mail01 dovecot: IMAP(akzidenz at triner.ch): /data/vmail/example.com/user/dovecot-uidlist: next_uid was lowered (2143 -> 2142, hdr=2142) The new mail is than missing in the uidlist and as long as no new mail arrives all folders of the mailbox appear to be empty. If a new mail arrives, it will get the uid that should have been given to the pervious mail (the one that caused the error). The previous mail still won't be listed in the uidlist, but everything works as expected. Best regards Lukas M?ller Systems Engineer _______________________________________________ NEWMEDIA S?dostschweiz Newmedia AG http://www.newmedia.ch _______________________________________________ TYPO3 & Drupal - Wir wissen wie. Ihre professionelle Web Agentur in Chur, Ilanz, Glarus und Z?rich. From lee at standen.id.au Tue Apr 3 11:02:05 2012 From: lee at standen.id.au (Lee Standen) Date: Tue, 3 Apr 2012 16:02:05 +0800 Subject: [Dovecot] Dovecot 1.2.9. next_uid was lowered In-Reply-To: References: Message-ID: I'll just say what I'm sure someone else will shortly :) The problem is NFS: http://wiki2.dovecot.org/NFS The solution is 2.x Director: http://wiki2.dovecot.org/Director Although it looks like you're currently on 1.x so it'd be a fairly significant upgrade for you. On Tue, Apr 3, 2012 at 3:24 PM, M?ller Lukas wrote: > Shortly after the mail gets delivered on server2, I get the following > errors on server1: > mail01 dovecot: IMAP(akzidenz at triner.ch): fdatasync(/data/vmail/ > example.com/user/dovecot-uidlist) failed: Input/output error > mail01 dovecot: IMAP(akzidenz at triner.ch): /data/vmail/ > example.com/user/dovecot-uidlist: next_uid was lowered (2143 -> 2142, > hdr=2142) > > From CMarcus at Media-Brokers.com Tue Apr 3 14:33:39 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 03 Apr 2012 07:33:39 -0400 Subject: [Dovecot] dsync redesign In-Reply-To: <87zkatpvxr.fsf@algae.riseup.net> References: <936E95DA-E032-4F8F-A323-DEBB7AAC0E1B@iki.fi> <87obrixcyp.fsf@algae.riseup.net> <4F721650.4030901@Media-Brokers.com> <87zkatpvxr.fsf@algae.riseup.net> Message-ID: <4F7AE013.6040402@Media-Brokers.com> On 2012-04-02 7:15 PM, Micah Anderson wrote: > Charles Marcus writes: >> On 2012-03-27 11:47 AM, Micah Anderson wrote: >>> One would be the ability to perform *intelligent* incremental / >>> rotated backups. I can do this now by running a dsync backup >>> operation and then doing manual hardlinking or moving of the >>> backup directories (daily.1, daily.2, weekly.1, monthly.1, etc.), >>> but it would be more intelligent if this were baked into the >>> backup process. >> There are already numerous tools that do this flawlessly - I've >> beenusing rsnapshot (which uses rsync) for this for years. > Are you snapshotting your filesystem (using LVM, or SAN, or similar) > before doing rsnapshot? Because if you aren't then rsync will not > assuredly get everything in a consistent state. No, and you are correct... but I run it in the middle of the night, and the system is only barely utilized at the time, so the very minor inconsistencies are not a problem overall. I will, however, be changing this to using FS snapshots once I get my mailserver virtualized (already being planned for when our new office location comes online), so that will allow me to perform snapshots multiple times during the day (I'm thinking 4 times per day will be enough). >> I don't know if Timo should be spending his time reinventing the >> wheel. > dsync backup is already here, and it is quite useful. I'm not saying it isn't, I'm just saying that there are already *plenty* of different backup tools, and I don't see the sense in Timo spending lots of time on creating a new one just for dovecot. His time would be better spent just making it easier for any other backup tool to work better. >> Although, one interesting piece that I am hopeful I'll be able to >> implement soon (with Timo's professional help) is the ability to >> easily and automatically map my rsnapshot snapshots directory to a >> read-only 'Backups' namespace that automatically shows the >> snapshots by date and time as they are produced. This way users >> could 'go back in time' anytime they wanted without having to call >> me... :) > Interesting idea, would be a great one to share with the community > if you decide to do so. Absolutely - that is already on my list for when I pay Timo's company to do this - document it on the wiki. Hopefully if any code changes are needed to make it work right, they will be minor. -- Best regards, Charles From xnasx at yandex.ru Tue Apr 3 15:33:11 2012 From: xnasx at yandex.ru (xnasx) Date: Tue, 03 Apr 2012 16:33:11 +0400 Subject: [Dovecot] Courier to Dovecot migration script issue Message-ID: <125901333456391@web125.yandex.ru> Hi! Forgive my bad English. I trying to migrate to Dovecot 2.0.18 from Courier 4.10. I using a script 'courier-dovecot-migrate.pl' from oficial wiki - it works without errors and creating dovecot-uidlist, etc. . I running Dovecot on a different port: 11149. When I connecting to port 11149 from Thunderbird, all my message headers are reloading. Please help me to solve this issue. Best regards, Victor From tss at iki.fi Tue Apr 3 16:00:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 16:00:47 +0300 Subject: [Dovecot] Per-user flags/indexes for shared mailboxes Message-ID: <1333458047.21461.48.camel@innu> The attached patch adds support for doing e.g.: namespace { type = public prefix = Public/ location = mdbox:/var/lib/public/mdbox:INDEXPVT=~/mdbox/public list = yes } So the private flags are stored in indexes under ~/mdbox/public/, while the shared flags are stored under /var/lib/public/mdbox/. The patch currently hardcodes the shared flags as only MAIL_SEEN. I'm not really sure what would be the best place to configure which flags are private. Perhaps a per-namespace default, but how to configure it per-mailbox? Also per-user keywords aren't currently supported. And it's a little bit annoying that each private flag update must increase the shared modseq for the message, but that can't really be helped. (Alternative would be for each user to have their own private modseqs, which could get difficult.) I'll probably commit this to v2.1 after some more testing. -------------- next part -------------- A non-text attachment was scrubbed... Name: private-index.diff Type: text/x-patch Size: 28899 bytes Desc: not available URL: From feltrin at gmail.com Tue Apr 3 17:17:59 2012 From: feltrin at gmail.com (Jean Michel) Date: Tue, 3 Apr 2012 11:17:59 -0300 Subject: [Dovecot] Backtrace on mdbox index In-Reply-To: <1333434809.21461.33.camel@innu> References: <1333434809.21461.33.camel@innu> Message-ID: I tried with dovecot 2.1.3 and got the same error, any ideia ? 2012/4/3 Timo Sirainen > On Mon, 2012-04-02 at 11:14 -0300, Francisco Wagner C. Freire wrote: > > > I current using dovecot with mdbox and on one account i get this error: > > > > doveadm(user at domain): Panic: file mail-index-sync-ext.c: line 209 > > (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) > > Dovecot version? I think I've fixed this in recent v2.1.x. > > > -- ------------------------------ Jean Michel Feltrin From tss at iki.fi Tue Apr 3 17:37:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 17:37:26 +0300 Subject: [Dovecot] Backtrace on mdbox index In-Reply-To: References: <1333434809.21461.33.camel@innu> Message-ID: <1333463846.21461.50.camel@innu> Could you privately send me your all of your dovecot.index, dovecot.index.log, dovecot.map.index and dovecot.map.index.log files under mdbox? None of those contain any sensitive data. On Tue, 2012-04-03 at 11:17 -0300, Jean Michel wrote: > I tried with dovecot 2.1.3 and got the same error, any ideia ? > > 2012/4/3 Timo Sirainen > > > On Mon, 2012-04-02 at 11:14 -0300, Francisco Wagner C. Freire wrote: > > > > > I current using dovecot with mdbox and on one account i get this error: > > > > > > doveadm(user at domain): Panic: file mail-index-sync-ext.c: line 209 > > > (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) > > > > Dovecot version? I think I've fixed this in recent v2.1.x. > > > > > > > > From tss at iki.fi Tue Apr 3 17:40:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 17:40:34 +0300 Subject: [Dovecot] imapdir still a configuration option? In-Reply-To: <4F7A18C8.3000904@se.rit.edu> References: <4F7A18C8.3000904@se.rit.edu> Message-ID: <1333464034.21461.51.camel@innu> On Mon, 2012-04-02 at 17:23 -0400, Kurt Mosiejczuk wrote: > I've been looking at moving an old IMAPdir-based solution to dovecot. I > came across references that dovecot (to my pleasant surprise) added > imapdir support in 1.1. However, in trying 2.0.13 (from an OpenBSD > package) things start okay, but I see in the logs: > > "Initialization failed: Initializing mail storage from mail_location > setting failed: Unknown mail storage driver imapdir" > > Poking around in the sources, I still see lots of references to imapdir, > but see very little in the documentation. > > Is IMAPdir still supported in 2.0.x (or even 2.1.x)? I'd prefer not to > have to migrate users via renaming, especially since they like the > IMAPdir layout for their own searching. mail_location = maildir:~/Maildir:LAYOUT=imapdir From tss at iki.fi Tue Apr 3 17:43:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 17:43:50 +0300 Subject: [Dovecot] Connection queue full on dovecot 2.0.13 In-Reply-To: <4F79BA0C.1070104@libero.it> References: <4F79BA0C.1070104@libero.it> Message-ID: <1333464230.21461.53.camel@innu> On Mon, 2012-04-02 at 16:39 +0200, Giovanni Mancuso wrote: > Apr 02 14:34:36 imap-login: Info: Disconnected: *Connection queue full > *(auth failed, 1 attempts): user= I see this wiki page: http://wiki2.dovecot.org/LoginProcess but i read: > > It works by using a number of long running login processes, each > handling a number of connections. This loses much of the security > benefits of the login process design, because in case of a security hole > (in Dovecot or SSL library) the attacker is now able to see other users > logging in and steal their passwords, read their mails, etc. > > Is there another way? Just increase the number of processes: service imap-login { process_limit = 10000 } From kurt-dovecot at se.rit.edu Tue Apr 3 17:50:59 2012 From: kurt-dovecot at se.rit.edu (Kurt Mosiejczuk) Date: Tue, 03 Apr 2012 10:50:59 -0400 Subject: [Dovecot] imapdir still a configuration option? In-Reply-To: <1333464034.21461.51.camel@innu> References: <4F7A18C8.3000904@se.rit.edu> <1333464034.21461.51.camel@innu> Message-ID: <4F7B0E53.50601@se.rit.edu> Timo Sirainen wrote: > mail_location = maildir:~/Maildir:LAYOUT=imapdir Excellent. That works *perfectly*. --Kurt From stsiol at yahoo.co.uk Tue Apr 3 17:55:54 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 3 Apr 2012 15:55:54 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333434757.21461.32.camel@innu> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333434757.21461.32.camel@innu> Message-ID: <1333464954.38480.YahooMailNeo@web132206.mail.ird.yahoo.com> Hi Timo,? >________________________________ > From: Timo Sirainen >To: Spyros Tsiolis ? >Cc: Dovecot ? >Sent: Tuesday, 3 April 2012, 8:32 >Subject: Re: [Dovecot] Mail migration troubles >? >On Sun, 2012-04-01 at 22:54 +0100, Spyros Tsiolis wrote: >> However, I have this nagging problem with most of the thunderbird >> MTAs. When I try to delete some "test" messages I sent to make sure >> that everything works, it comes back with the following message on >> the bottom of the Thunderbird window : >>? >> "The current operation on InBox did not succeed.? >> The mail server for account "Account at bloomingdomain.gr"? >> responded [TRYCREATE] mailbox doesn't exist : Trash" > >That error probably happens when deleting a message. Yes, it does Timo. Thank you. >> Dovecot v1.2.15 AFAIK Actually it's v1.2.16 >dovecot -n output would have helped. Mainly: do you have a namespace >prefix or not? That's about the only thing I can think of related to >that error message (e.g. dovecot has non-empty namespace prefix, while >TB has empty namespace prefix). Why do I keep forgetting this ? Damn the gods :-) !@#$ :-) Here's the output of "dovecot -n" : -------------------------------------------------------------------- [root at mailgate ~]# dovecot -n # 1.2.16: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 base_dir: /var/run/dovecot/ log_path: /var/log/dovecot/dovecot.log info_log_path: /var/log/dovecot/dovecot-info.log ssl_parameters_regenerate: 48 verbose_ssl: yes login_dir: /var/run/dovecot//login login_executable: /usr/local/dovecot/libexec/dovecot/imap-login login_greeting: * Dovecot ready * login_max_processes_count: 64 mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir mail_plugins: zlib auth default: ? verbose: yes ? debug: yes ? debug_passwords: yes ? passdb: ? ? driver: passwd-file ? ? args: /etc/dovecot/passwd ? passdb: ? ? driver: pam ? userdb: ? ? driver: static ? ? args: uid=vmail gid=vmail home=/home/vmail/%u ? userdb: ? ? driver: passwd [root at mailgate ~]#? -------------------------------------------------------------------- I suspect it's some freak thunderbird bug. It's the only explanation. I've migrated about 20 mailboxes and it's the only one acting up. Thanks again Timo, s. ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From stsiol at yahoo.co.uk Tue Apr 3 18:06:42 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 3 Apr 2012 16:06:42 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333464954.38480.YahooMailNeo@web132206.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333434757.21461.32.camel@innu> <1333464954.38480.YahooMailNeo@web132206.mail.ird.yahoo.com> Message-ID: <1333465602.15453.YahooMailNeo@web132203.mail.ird.yahoo.com> Also, What do you mean by namespace ? I am sorry I didn't get that one : >> dovecot -n output would have helped. Mainly: do you have a namespace >> prefix or not? That's about the only thing I can think of related to >> that error message (e.g. dovecot has non-empty namespace prefix, while >> TB has empty namespace prefix). I missed the "namespace" thing. s. ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From nmilas at noa.gr Tue Apr 3 18:33:59 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Tue, 03 Apr 2012 18:33:59 +0300 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333465602.15453.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333434757.21461.32.camel@innu> <1333464954.38480.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333465602.15453.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <4F7B1867.3030608@noa.gr> On 3/4/2012 6:06 ??, Spyros Tsiolis wrote: > What do you mean by namespace ? See: http://wiki2.dovecot.org/Namespaces Best regards, Nick From trashcan at odo.in-berlin.de Tue Apr 3 20:29:41 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Tue, 3 Apr 2012 19:29:41 +0200 Subject: [Dovecot] dovecot and unison In-Reply-To: <4F7A2FF9.7000204@klacto.net> References: <4F7A2FF9.7000204@klacto.net> Message-ID: <8EAB42AF-BC4A-4235-9CF9-A444BE553B10@odo.in-berlin.de> Hi -- [Sorry for my private response, instead of public one] On 03.04.2012, at 01:02, FZiegler wrote: > Now I'd like if possible, to replicate this setup on my laptop and keep > both in sync with unison (http://www.cis.upenn.edu/~bcpierce/unison/), > which I am already using to sync much of my home dir about once a day. > > I found at least one positive message regarding this topic > (http://dovecot.org/list/dovecot/2010-April/048092.html), but I feel I > could use some more advice. I used to sync my two server's maildirs for some years, but I moved to dsync mirroring some months ago, and now I'm into using replicator/dsync (see archive of March 2012). Unsion mirroring is working in general, but sometimes you might realize some duplicates being produced. Not bad, but it happens. I did realize the same with using dsync mirror instead. It's much better with the new replicator/dsync mirroring scheme, but it's not perfect, yet. (Again, see ML archives of the last weeks). > Namely, which of the ancillary files should I be syncing (or perhaps > deleting before sync)? In addition to the mails themselves I'm seeing: > > -rw------- 1 fz 501 13 Apr 1 17:10 dovecot-keywords > -rw------- 1 fz 501 2483 Apr 1 17:10 dovecot-uidlist > -rw------- 1 fz 501 1040 Mar 29 14:50 dovecot.index > -rw------- 1 fz 501 44032 Apr 1 17:10 dovecot.index.cache > -rw------- 1 fz 501 9772 Apr 1 17:10 dovecot.index.log > > and at the top level (in .local/var/Maildir/): > > -rw------- 1 fz 501 8 Apr 1 17:10 dovecot-uidvalidity > -r--r--r-- 1 fz 501 0 Apr 1 17:10 dovecot-uidvalidity.4efb323d > -rw------- 1 fz 501 3288 Apr 1 17:10 dovecot.mailbox.log > -rw------- 1 fz 501 4104 Mar 28 10:53 dovecot.mailbox.log.2 I used ... ignore = Name dovecot* ignore = Regex .*/tmp$ ignore = Regex .*/\.Drafts$ ? to ignore all of them. When runnig mirroring for the very first time, I started with one empty maildir at one server, IIRC. You better check that with a test maildir beforehand. > -rw------- 1 fz 501 5114 Apr 1 17:10 subscriptions That became mirrored. Regards, Michael From dave-lists-dovecot at weller-fahy.com Tue Apr 3 21:44:32 2012 From: dave-lists-dovecot at weller-fahy.com (David J. Weller-Fahy) Date: Tue, 3 Apr 2012 14:44:32 -0400 Subject: [Dovecot] Dovecot stones In-Reply-To: References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <20120403184431.GA792@weller-fahy.com> * Mark Moseley [2012-04-02 12:41 -0400]: > On Sat, Mar 31, 2012 at 2:28 PM, Timo Sirainen wrote: > > For the last few days I've been thinking about my company and what > > > >... > > I'm slightly concerned that there's been no mention of what license > these stones are going to be released under. GPL2? GPL3? Apache? I'm > just hoping these aren't some sort of "open core" stones that will > only work for basic features but that I'll end up needing to buy > "Enterprise-grade stones" to cover large clusters. From the description and images, I'd say they are probably binary blobs to be included in the firmware via hardware dongle. ;) -- dave [ please don't CC me ] -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From dm-list-email-dovecot at scs.stanford.edu Tue Apr 3 22:22:02 2012 From: dm-list-email-dovecot at scs.stanford.edu (dm-list-email-dovecot at scs.stanford.edu) Date: Tue, 03 Apr 2012 12:22:02 -0700 Subject: [Dovecot] dovecot and unison In-Reply-To: <4F7A2FF9.7000204@klacto.net> References: <4F7A2FF9.7000204@klacto.net> Message-ID: <87398kei3p.wl@ta.scs.stanford.edu> At Mon, 02 Apr 2012 19:02:17 -0400, FZiegler wrote: > > I am successfully using dovecot purely as a personal local mail store on > my desktop. (There is only one account, and it's only ever accessed by > local mail clients on the machine. The point is to have a common store I > can use with any client; plus, I prefer dovecot's Mailbox storage to > Thunderbird's mboxes.) > > Now I'd like if possible, to replicate this setup on my laptop and keep > both in sync with unison (http://www.cis.upenn.edu/~bcpierce/unison/), > which I am already using to sync much of my home dir about once a day. > > I found at least one positive message regarding this topic > (http://dovecot.org/list/dovecot/2010-April/048092.html), but I feel I > could use some more advice. I have a similar setup, but I use offlineimap instead of unison: http://offlineimap.org/ It seems to work pretty well. That's not to say that unison wouldn't work as well also. However, offlineimap has the advantage that it doesn't restrict you to a star topology. You can, for instance, since to your laptop at work and from your laptop at home. Note that offlineimap is slow if you don't use imap at both ends. Therefore, I use it on the local end. A simplified excerpt of my .offlineimaprc looks like this: ======== [general] accounts = DefaultAccount [Account DefaultAccount] localrepository = MyLocal remoterepository = MyRemote [Repository MyRemote] type = IMAP preauthtunnel = ssh -qax -oBatchMode=yes -oServerAliveInterval=60 MY-MAIL-SERVER 'exec env CONFIG_FILE=/PATH/TO/PRIVATE/dovecot.conf /usr/lib/dovecot/imap' [Repository MyLocal] type = IMAP preauthtunnel = CONFIG_FILE=$HOME/etc/dovecot.conf /usr/lib/dovecot/imap ======== Unfortunately, in dovecot 2.1, the full text search no longer seems to work in pre-auth mode, but I don't think that has anything to do with offlineimap. I think maybe dovecot is deprecating pre-auth mode or requires a more complicated setup. From tss at iki.fi Wed Apr 4 03:45:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 03:45:36 +0300 Subject: [Dovecot] Backtrace on mdbox index In-Reply-To: References: Message-ID: <7F642313-D6F2-4032-A913-63E9BDAB6848@iki.fi> On 2.4.2012, at 17.14, Francisco Wagner C. Freire wrote: > I current using dovecot with mdbox and on one account i get this error: > > doveadm(user at domain): Panic: file mail-index-sync-ext.c: line 209 > (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) Fixed: http://hg.dovecot.org/dovecot-2.1/rev/47526bf86c57 From tss at iki.fi Wed Apr 4 03:48:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 03:48:12 +0300 Subject: [Dovecot] 2.1.3: doveadm mailbox delete lost the -s option In-Reply-To: <874nt1rcf4.fsf@algae.riseup.net> References: <874nt1rcf4.fsf@algae.riseup.net> Message-ID: On 3.4.2012, at 1.34, Micah Anderson wrote: > Looks like the recently added -s (for unsubscribing) option to doveadm > mailbox delete went missing: > > root at vireo# /usr/bin/doveadm mailbox delete -u micahtest -s restored > delete: invalid option -- 's' > doveadm mailbox delete [-u |-A] [-S ] [-s] [...] > > although the option still is shown as valid :) Fixed: http://hg.dovecot.org/dovecot-2.1/rev/411344f9daf3 From tss at iki.fi Wed Apr 4 03:56:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 03:56:05 +0300 Subject: [Dovecot] dovecot 2.1 breaks FTS + pre-auth? In-Reply-To: <878vigok53.wl@ta.scs.stanford.edu> References: <878vigok53.wl@ta.scs.stanford.edu> Message-ID: <335CB623-DFF7-4676-AA0E-53D7AD42E5C0@iki.fi> On 31.3.2012, at 18.38, dm-list-email-dovecot at scs.stanford.edu wrote: > Hi. I use dovecot in the simplest possible way, as an IMAP server in > pre-auth mode over ssh or just locally over a unix-domain socket > (e.g., with offlineimap, which runs much faster using dovecot for the > local message store). Ideally I would like to avoid running any extra > daemons or setting up anything as root. Until recently, this has > worked fine by just setting the CONFIG_FILE environment variable to > something in my home directory. You can run Dovecot without root: http://wiki2.dovecot.org/HowTo/Rootless > Full text search used to work just fine with this configuration, and > still does on a machine I have running dovecot 2.0.13. However, on > the machine with 2.1, I get errors about /var/run/dovecot/index not > existing. > > $ printf "a select INBOX\nb search text xyzzy\nc logout\n" \ > | /usr/lib/dovecot/imap > * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE] Logged in as dm > imap(dm): Error: net_connect_unix(/var/run/dovecot/indexer) failed: No such file or directory I looked at the code and looks like there is no easy way to make it work the old way. The new way is the only way to make fts-lucene work reliably, and it's also better for fts-solr. And fts-squat is really on its way out to die. From tss at iki.fi Wed Apr 4 03:57:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 03:57:33 +0300 Subject: [Dovecot] Is it possible to migrating mail to dovecot using imapc? In-Reply-To: <695D3A65-CFBF-4DC3-9DAC-E0C299ED0E6D@tao.org.uk> References: <72230FBC-F639-415E-AD1D-7D49CE4C6287@tao.org.uk> <07E5D123-E233-4EB5-B5F8-99B90AF6A809@tao.org.uk> <76BAD212-0807-4EA6-9211-4C3D4F6518CC@iki.fi> <2F722E50-B9AC-4660-AAC2-39FBEEE14364@iki.fi> <25DE9E4A-478B-4D70-8B65-0A727B0DEFAF@tao.org.uk> <9F05AEAA-6D32-417E-85F2-240FF39D0426@iki.fi> <695D3A65-CFBF-4DC3-9DAC-E0C299ED0E6D@tao.org.uk> Message-ID: <1C12148A-959F-453C-AA98-15ED001F53B6@iki.fi> On 30.3.2012, at 23.42, Dr Josef Karthauser wrote: > % dsync -Dv -u joe at local.com -o imapc_host=mail.remoteimap.com -o imapc_port=143 -o imapc_username=joe@ remoteimap.com -o imapc_password='somepass' mirror imapc: > & /tmp/output > > produced the following output. > dsync(joe at email.com): Error: Mailbox INBOX changed its GUID (f38c263a4919764f1b560100c516a7a4 -> c92f64f79f0d1ed01e6d5b314f04886c) dsync + imapc requires indexes. So you need to use e.g. imapc:/tmp/dsync-imapc From tss at iki.fi Wed Apr 4 04:05:52 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 04:05:52 +0300 Subject: [Dovecot] Many messages clustered around the same date.saved value In-Reply-To: References: Message-ID: <31C97BDA-5B4E-4289-9087-9AC72F11F0F5@iki.fi> On 29.3.2012, at 5.41, Joseph Tam wrote: >> Ah, with mbox there isn't any usable fallback for date.saved. If it's >> not in dovecot.index.cache, the current time is used. > > I'm a little confused as to why it needed a fallback. In other words, > why wasn't date.saved put into the index as soon as the IMAP operation > copied it into "Trash"? > > If this data isn't set at that time, when does it get instantiated? > When I actually ask for it? Well..: - date.saved is stored only in dovecot.index.cache file - if it doesn't exist and is requested, the current time is returned and it's added to the cache - when date.saved has already fetched once (so it already exists in dovecot.index.cache file), and mail is saved via LDA/IMAP then it gets added there immediately when saving - dovecot.index.cache has caching decisions, and some old/unused fields may get dropped from it once in a while - maybe due to some bugs or whatever, the fields or the entire cache may get dropped for some other reason So it probably should have worked, but for some reason didn't. It would be possible to store date.saved in dovecot.index file, like mdbox does, so cache decisions wouldn't matter. But probably too much trouble to be worth it, very few mbox installations care about it. From tss at iki.fi Wed Apr 4 04:08:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 04:08:46 +0300 Subject: [Dovecot] Dovecot upgrade from 1.2.x to 2.0.x: roundcube/squirrelmail sent folder doesn't work any more In-Reply-To: References: <4F71F8D8.6040700@wiesinger.com> <95C664AD-0532-4F3B-A2EB-2FD25B79187B@iki.fi> Message-ID: On 29.3.2012, at 8.25, Gerhard Wiesinger wrote: >>> 2.0.x: with Prefix ~/Mail >>> A0003 LIST "" ~/Mail/sent >>> A0003 OK List completed.: >> > > mail_full_filesystem_access = yes > mail_location = mbox:~:INBOX=/var/mail/%u Yes, the problem happens only with mail_full_filesystem_access=yes. It already works in v2.1, so I don't think I'll bother looking into why it's not working in v2.0. From tss at iki.fi Wed Apr 4 04:11:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 04:11:04 +0300 Subject: [Dovecot] File/folder permission issues in 2.1.3 In-Reply-To: <8B296F70-22B8-487B-AD7A-47BEB8C84F62@roessner-network-solutions.com> References: <8B296F70-22B8-487B-AD7A-47BEB8C84F62@roessner-network-solutions.com> Message-ID: On 29.3.2012, at 13.24, Christian R??ner wrote: > I figured out that Dovecot does not honer secondary groups with auth/auth-worker (??), if doing LDAP/TLS stuff. I had to use file system acls to add the user "vmail" to /etc/ssl/private and to the corresponding key file: service auth { extra_groups = any groups you want auth process to have } Don't give secondary groups to any Dovecot related users. From tss at iki.fi Wed Apr 4 04:35:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 04:35:13 +0300 Subject: [Dovecot] Dovecot allows creation of folders outside of a user's directory In-Reply-To: <4F759B08.1060603@in.tum.de> References: <4F759B08.1060603@in.tum.de> Message-ID: On 30.3.2012, at 14.37, Christoph Bu?enius wrote: > in our dovecot 2.0 setup with shared folders, users can make dovecot create directories outside their mail directory. Which is a bit scary imho. > > The following command: > > . create inbox.shared.abc123 > > or even > > . create "inbox.shared.strange &ANY- characters" > > -- even though it will fail with a "permission denied" error -- will create a directory like "/mail/users/strange &ANY- characters". That directory will only contain a subdirectory "Maildir" and therein dovecot-acl-list. Fixed: http://hg.dovecot.org/dovecot-2.0/rev/b15889b82258 From tss at iki.fi Wed Apr 4 04:47:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 04:47:33 +0300 Subject: [Dovecot] namespace hierarchy In-Reply-To: <201204010928.41017.anyaddress@gmx.net> References: <201204010928.41017.anyaddress@gmx.net> Message-ID: On 1.4.2012, at 10.28, Tom Fernandes wrote: > I'm migrating from Courier to Dovecot and would like to keep the original > namespace and add two new ones on the new server. > > There's only one namespace in Courier which is "INBOX." . The seperator is "." > > In the new server I would like to have 3 namespaces (private, public, shared) > and use "/" as separator. Is it possible to keep "INBOX." for private, so that > the users mailboxes appear below the Inbox, but have public and shared on the > same hierarchy-level like the Inbox? > > When I set prefix to "INBOX/" for private I get the users mailboxes below the > Inbox, like I have it Courier now. But then at least the public, probably also > the shared namespace doesn't work any more. I get "unknown subscription > namespace" when trying to subscribe to some of the folders. If you have any subscriptions=no namespaces, you need a parent namespace that has subscriptions=yes. You could have e.g.: namespace { prefix = INBOX/ list = no inbox = yes } namespace { prefix = list = no hidden = yes alias_for = INBOX/ } So now the prefix="" is used for the subscriptions but nothing else really. Anyway, INBOX/ isn't the same as INBOX. so if any client configs are using INBOX. as namespace prefix then they don't work with INBOX/. From tss at iki.fi Wed Apr 4 04:49:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 04:49:48 +0300 Subject: [Dovecot] imapsync, namespaces and Dovecot 2.x In-Reply-To: References: Message-ID: <65D3A596-EC48-430C-8F53-4A11F30437C4@iki.fi> On 2.4.2012, at 19.11, Aleix Dorca wrote: > I recently tried to migrate from Exchange using imapsync. It didn't work because it complained that dovecot had no namespace capablity. After looking through Dovecot's config files I found that namespaces could be defined, but if you did not a 'private' default one was created. The problem is that Dovecot doesn't advertise NAMESPACE capability before client has logged in. The better solution would be to change imapsync to figure this out automatically. The other solution would be to set in Dovecot: imap_capability = +NAMESPACE From tss at iki.fi Wed Apr 4 05:35:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 05:35:33 +0300 Subject: [Dovecot] lmtp sometimes fails to deliver a message to all recipients In-Reply-To: <4F79C995.9060607@talex.pl> References: <4F79C995.9060607@talex.pl> Message-ID: <1B1369FF-6748-46DE-AB94-AF2F32061757@iki.fi> On 2.4.2012, at 18.45, Artur Zaprza?a wrote: > Running strace on lmtp reveals that when lmtp delivers a message to multiple recipients it saves the message to a file in the first mailbox and hard links that file to other mailboxes. But when in the meantime of delivery that file is deleted, lmtp fails to deliver the message to remaining mailboxes. For example a message for 46 recipients is delivered by lmtp to the first 12 only (fortunately postfix retries delivery for remaining recipients) and for the rest lmtp logs the following error messages: > > lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Message was expunged (guid) > lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: script /vmail/domain/foo/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /vmail/domain/foo/.dovecot.sieve.log may reveal additional details) Fixed in hg. From tss at iki.fi Wed Apr 4 05:49:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 05:49:40 +0300 Subject: [Dovecot] dovecot-auth restaring and caching In-Reply-To: <4F6C696C.5030900@um.es> References: <4F6AF72E.9030206@um.es> <7B9D052D-5864-42A9-AE9A-6FCE858F48C0@iki.fi> <4F6C696C.5030900@um.es> Message-ID: On 23.3.2012, at 14.15, Angel L. Mateo wrote: > El 22/03/12 19:57, Timo Sirainen escribi?: >> On 22.3.2012, at 11.55, Angel L. Mateo wrote: >> >>> The problem I'm having is that if I have no activity in the server, dovecot stops its auth process and when another message is received, it restarted it, but with an empty cache. >> >> service auth { >> idle_kill = 0 >> } >> > > In a test server I have, this have solved the problem. In my productions servers it is still being restarted. Could it be another parameter involve in this? Oops, I remembered wrong. idle_kill=0 uses default_idle_kill limit. You could set for example: idle_kill = 4294967295s From tss at iki.fi Wed Apr 4 05:51:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 05:51:19 +0300 Subject: [Dovecot] dovecot 2.0.19 Panic: file mail-index-sync-ext.c: line 209 (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) In-Reply-To: <4F6A1BA7.7030208@uvm.edu> References: <4F69DD6E.1090502@uvm.edu> <94A1158A-2F42-406A-9212-C58A4D4FF879@iki.fi> <4F6A1BA7.7030208@uvm.edu> Message-ID: <18B56699-7AF4-4ED8-A21F-4EBA9E7C6752@iki.fi> On 21.3.2012, at 20.19, Jim Lawson wrote: > On 3/21/12 10:02 AM, Timo Sirainen wrote: >> On 21.3.2012, at 15.53, Jim Lawson wrote: >> >>> Had a user who couldn't access his INBOX: >>> >>>> Mar 21 09:21:17 penguina dovecot: imap([USER]): Panic: file >>>> mail-index-sync-ext.c: line 209 (sync_ext_reorder): assertion fai >>>> led: (offset < (uint16_t)-1) >> I kind of remember that this was fixed by http://hg.dovecot.org/dovecot-2.1/rev/b4d8e950eb9d but I'm not entirely sure. I guess I should have included in the commit the error message it fixed. > > > This applies cleanly against 2.0.19; should I try it on that version, or > not recommended? http://hg.dovecot.org/dovecot-2.1/rev/47526bf86c57 should fix this. From tss at iki.fi Wed Apr 4 05:57:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 05:57:03 +0300 Subject: [Dovecot] Courier to Dovecot migration script issue In-Reply-To: <125901333456391@web125.yandex.ru> References: <125901333456391@web125.yandex.ru> Message-ID: <9BBB03B5-4742-403B-8720-FBEFB554266D@iki.fi> On 3.4.2012, at 15.33, xnasx wrote: > I trying to migrate to Dovecot 2.0.18 from Courier 4.10. > I using a script 'courier-dovecot-migrate.pl' from oficial wiki - it works without errors and creating dovecot-uidlist, etc. . > > I running Dovecot on a different port: 11149. > When I connecting to port 11149 from Thunderbird, all my message headers are reloading. > > Please help me to solve this issue. telnet localhost 143 a login user pass b select inbox c status inbox (uidvalidity) d fetch 1:* uid telnet localhost 11149 a login user pass b select inbox c status inbox (uidvalidity) d fetch 1:* uid Check if the "c" and "d" outputs are the same for both servers. If they are, then the migration was successful and the problem is something else, such as maybe the client always redownloading messages because you changed the port (or something else). From tss at iki.fi Wed Apr 4 06:16:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 06:16:54 +0300 Subject: [Dovecot] ldap userdb warning in v2.1.1 In-Reply-To: References: <4F69989A.3000106@noa.gr> Message-ID: On 21.3.2012, at 14.06, Timo Sirainen wrote: > On 21.3.2012, at 11.00, Nikolaos Milas wrote: > >> Mar 21 10:07:23 imapserver dovecot: master: Dovecot v2.1.1 starting up (core dumps disabled) >> Mar 21 10:08:17 imapserverdovecot: auth: Warning: ldap: Ignoring changed user_attrs in /etc/dovecot/dovecot-passdb-ldap.conf, because userdb ldap not used. (If this is intentional, set userdb_warning_disable=yes) >> >> I didn't see such warnings in 2.0.13. >> >> I guess I should/could remove the "user_attrs" line from dovecot-passdb-ldap.conf because it's not needed? > > Hmm. Yes, if dovecot-usrdb-ldap.conf is a separate file from dovecot-passdb-ldap.conf you can just remove it. But this reminds me that in several places I've suggested to make one of them a symlink to the other, and you can't really do it then. Perhaps I'll need to remove this warning, or maybe make it recognize the symlink case. Removed, at least for now: http://hg.dovecot.org/dovecot-2.1/rev/324df4134049 From tss at iki.fi Wed Apr 4 06:25:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 04 Apr 2012 06:25:51 +0300 Subject: [Dovecot] Dovecot 2.1 with custom OpenSSL fails to build In-Reply-To: References: <1331816286.10319.23.camel@innu.invalid> Message-ID: <4F7BBF3F.9060103@iki.fi> On 03/15/2012 11:43 PM, Andreas M. Kirchwitz wrote: > Compilation works. Great! The binaries find all their libraries. > > But two libraries are not quite okay. They don't find their SSL libs: > > libdovecot-lda.so > libdovecot-storage.so > > Since libdovecot-lda.so doesn't contain the words libssl or libcrypto, > I guess that ldd just complains because it uses libdovecot-storage.so. > Thus, libdovecot-storage.so is the (only) one left with an incomplete > library search path. > > Luckily, all binaries use some additional libraries which come with > a proper library path. So the whole things works, but it's more like > some kind of magic. It would be great if libdovecot-storage.so could > be fixed as well to make things finally perfect. Maybe this fixes it? http://hg.dovecot.org/dovecot-2.1/rev/8b91367bc3e1 From tss at iki.fi Wed Apr 4 06:31:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 06:31:07 +0300 Subject: [Dovecot] Dovecot and scalable database storage In-Reply-To: References: Message-ID: <7CD2C788-D258-458B-A915-1360B861DEE6@iki.fi> On 23.3.2012, at 4.36, wrote: > I saw some interesting mails from TImo back in 2009 talking about the idea > of using something like Cassandra db or similar as a storage platform for > both email and index/logs. I was wondering if this has been discussed > since then, and if there are any plans to support something like this in > the future? I have been playing with Cassandra and found that their > RackAwareStrategy gives you the ability to replicate writes to as many > nodes as you would like, but more importantly what nodes and one of those > nodes could be defined by what rack it lives in or what data center it > lives in. This means multiple sites high available storage clusters, > seemingly a system that dovecot could benefit from in terms of performance > and redundancy and simplicity. Any takers? There are still plans, but not in near future. I'm still planning on implementing it by first making Dovecot's code simpler and more asynchronous, and then the key-value database support could be implemented in just a few hours. From tss at iki.fi Wed Apr 4 07:33:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 07:33:02 +0300 Subject: [Dovecot] doveadm user -f index In-Reply-To: <87pqc3l1xk.fsf@algae.riseup.net> References: <87pqc3l1xk.fsf@algae.riseup.net> Message-ID: <4C96B184-EF85-4301-AEFD-F9201A423D63@iki.fi> http://hg.dovecot.org/dovecot-2.1/rev/601014feade4 doveadm user -m -f home user at domain I'm still not sure if I implemented this the user friendliest way.. It's kind of annoying to have these existing "doveadm auth" and "doveadm user" commands. Perhaps "doveadm user -m" should have been the default, with some added fields, and then the current "doveadm user" would have been as an added option. Hm. Maybe I should change it to v2.2.. On 23.3.2012, at 18.25, Micah Anderson wrote: > > I've configured my mail_location to have a different location for > performance reasons so they aren't in the same location as the > mail_location. > > The 'doveadm user -f home' is useful to find where a user's home > directory is for various scripting purposes, but I can't seem to find a > way to determine the location of the user's indexes. > > I can do something with the output of dovecot -a to find the > mail_location and then look for a configured INDEX, but then I don't > have a good way of translating the %d/%1n/%n type string formatters into > their values for a user. > > thanks for any suggestions! > micah > > -- > > From xnasx at yandex.ru Wed Apr 4 10:36:07 2012 From: xnasx at yandex.ru (xnasx) Date: Wed, 04 Apr 2012 11:36:07 +0400 Subject: [Dovecot] Courier to Dovecot migration script issue In-Reply-To: <9BBB03B5-4742-403B-8720-FBEFB554266D@iki.fi> References: <125901333456391@web125.yandex.ru> <9BBB03B5-4742-403B-8720-FBEFB554266D@iki.fi> Message-ID: <331561333524967@web30.yandex.ru> Thank you! I really don't know what I made, but now all works fine with Thunderbird and Outlook 2007. But something strange: 'uidvalidity' are the same for both servers, but UIDs of mesages are not: Courier: d fetch 1:* uid * 1 FETCH (UID 64295) * 2 FETCH (UID 64296) * 3 FETCH (UID 64297) Dovecot: * 1 FETCH (UID 64296) * 2 FETCH (UID 64297) * 3 FETCH (UID 64298) There can be it because of that that the server is used? 04.04.2012, 06:57, "Timo Sirainen" : > On 3.4.2012, at 15.33, xnasx wrote: > >> ?I trying to migrate to Dovecot 2.0.18 from Courier 4.10. >> ?I using a script 'courier-dovecot-migrate.pl' from oficial wiki - it works without errors and creating dovecot-uidlist, etc. . >> >> ?I running Dovecot on a different port: 11149. >> ?When I connecting to port 11149 from Thunderbird, all my message headers are reloading. >> >> ?Please help me to solve this issue. > > telnet localhost 143 > a login user pass > b select inbox > c status inbox (uidvalidity) > d fetch 1:* uid > > telnet localhost 11149 > a login user pass > b select inbox > c status inbox (uidvalidity) > d fetch 1:* uid > > Check if the "c" and "d" outputs are the same for both servers. If they are, then the migration was successful and the problem is something else, such as maybe the client always redownloading messages because you changed the port (or something else). From adorca at uda.ad Wed Apr 4 13:28:36 2012 From: adorca at uda.ad (Aleix Dorca) Date: Wed, 4 Apr 2012 12:28:36 +0200 Subject: [Dovecot] imapsync, namespaces and Dovecot 2.x In-Reply-To: <65D3A596-EC48-430C-8F53-4A11F30437C4@iki.fi> References: <65D3A596-EC48-430C-8F53-4A11F30437C4@iki.fi> Message-ID: El 04/04/2012, a les 3:49, Timo Sirainen va escriure: > On 2.4.2012, at 19.11, Aleix Dorca wrote: > >> I recently tried to migrate from Exchange using imapsync. It didn't work because it complained that dovecot had no namespace capablity. After looking through Dovecot's config files I found that namespaces could be defined, but if you did not a 'private' default one was created. > > The problem is that Dovecot doesn't advertise NAMESPACE capability before client has logged in. The better solution would be to change imapsync to figure this out automatically. The other solution would be to set in Dovecot: > > imap_capability = +NAMESPACE > Dead on! Thanks a lot. Aleix. From feltrin at gmail.com Wed Apr 4 14:08:21 2012 From: feltrin at gmail.com (Jean Michel) Date: Wed, 4 Apr 2012 08:08:21 -0300 Subject: [Dovecot] Backtrace on mdbox index In-Reply-To: <7F642313-D6F2-4032-A913-63E9BDAB6848@iki.fi> References: <7F642313-D6F2-4032-A913-63E9BDAB6848@iki.fi> Message-ID: Almost 100%, while running force-resync got this message this time: doveadm(acc at domain): Error: Log synchronization error at seq=49,offset=3876 for /storage/3/bd/05/server00001/users/domain/cache/storage/dovecot.map.index: Broken extension introduction: Record alignment is too large doveadm(acc at domain): Warning: fscking index file /storage/3/bd/05/server00001/users/domain/cache/storage/dovecot.map.index doveadm(acc at domain): Warning: mdbox /storage/3/bd/05/server00001/users/domain/mdbox/storage: rebuilding indexes .... doveadm(acc at domain): Panic: file mail-storage.c: line 787 (mailbox_check_mismatching_separators): assertion failed: (strncmp(vname, ns->prefix, ns->prefix_len-1) == 0) doveadm(acc at domain): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x443da) [0x7fa9c7bbd3da] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7fa9c7bbd4c2] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7fa9c7b9451f] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_open_stream+0) [0x7fa9c7e813c0] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_rebuild_in_context+0xb2b) [0x7fa9c7e43aeb] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync_begin+0x7ec) [0x7fa9c7e41abc] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync+0x46) [0x7fa9c7e41b26] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_sync_init+0x87) [0x7fa9c7e41c07] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x31) [0x7fa9c7e7ff91] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x27) [0x7fa9c7e80aa7] -> doveadm() [0x4103d7] -> doveadm() [0x40f898] -> doveadm() [0x40fbf4] -> doveadm(doveadm_mail_try_run+0x141) [0x410071] -> doveadm(main+0x3a1) [0x416c31] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7fa9c7835c8d] -> doveadm() [0x40f1c9] I'm sending again a link with the indexes and more details on your email. 2012/4/3 Timo Sirainen > On 2.4.2012, at 17.14, Francisco Wagner C. Freire wrote: > > > I current using dovecot with mdbox and on one account i get this error: > > > > doveadm(user at domain): Panic: file mail-index-sync-ext.c: line 209 > > (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/47526bf86c57 > > > -- ------------------------------ Jean Michel Feltrin From mlists at edicom.eu Wed Apr 4 14:18:00 2012 From: mlists at edicom.eu (Miguel Tormo) Date: Wed, 4 Apr 2012 13:18:00 +0200 Subject: [Dovecot] Proxy and SSO (single sign-on) Message-ID: <201204041318.00760.mlists@edicom.eu> Hello, I have a running setup with a dovecot imap4/pop3 proxy to a few dovecot backend servers which actually store the mailboxes. This is running smoothly and allows me to transparently distribute mailboxes. I'm using some "extrafield" configured in the LDAP passdb. However, now I would like to use GSSAPI (preferred) and NTLM for single sign-on. Both are pretty straightforward to configure in a single instance environment, but I don't know if they would work with proxy. For example, with GSSAPI there are two cases: 1) Just use gssapi mechanism, without PAM. Then, it a user presents a ticket the passdb ldap is not used, so the extrafields are never read. 2) Use gssapi and PAM (thus allowing using a kerberos password). But the extrafields feature isn't available with PAM passdb driver, so again the proxy won't work. The case for NTLM would fall into the first case, I think. Am I right regarding this scenario? Is there a way I could make SSO and proxying work? I'm currently using dovecot 2.0.16 (had to patch it to increment LOGIN_MAX_INBUF_SIZE to 4096 for GSSAPI to work, as I read somewhere in this list), but I could upgrade to a newer version if that allows all this to work. Thanks! From tss at iki.fi Wed Apr 4 14:18:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 14:18:53 +0300 Subject: [Dovecot] Backtrace on mdbox index In-Reply-To: References: <7F642313-D6F2-4032-A913-63E9BDAB6848@iki.fi> Message-ID: <277DFE38-DBF3-4806-B73A-CA797763321F@iki.fi> On 4.4.2012, at 14.08, Jean Michel wrote: > doveadm(acc at domain): Panic: file mail-storage.c: line 787 > (mailbox_check_mismatching_separators): assertion failed: (strncmp(vname, > ns->prefix, ns->prefix_len-1) == 0) That's a bit strange. Can you get a core of it? http://dovecot.org/bugreport.html If you can access the crash with gdb, try commands like: fr 6 (or maybe 5, or maybe 4, or 7, or whatever makes the following commands work) p *box o *box.list.ns From tss at iki.fi Wed Apr 4 14:21:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 14:21:33 +0300 Subject: [Dovecot] Proxy and SSO (single sign-on) In-Reply-To: <201204041318.00760.mlists@edicom.eu> References: <201204041318.00760.mlists@edicom.eu> Message-ID: <48AD358F-ECC4-48F5-8CAB-9B91C8BDEEE3@iki.fi> On 4.4.2012, at 14.18, Miguel Tormo wrote: > I have a running setup with a dovecot imap4/pop3 proxy to a few dovecot backend servers which actually store the mailboxes. This is running smoothly and allows me to transparently distribute mailboxes. > I'm using some "extrafield" configured in the LDAP passdb. > > However, now I would like to use GSSAPI (preferred) and NTLM for single sign-on. Both are pretty straightforward to configure in a single instance environment, but I don't know if they would work with proxy. For example, with GSSAPI there are two cases: > 1) Just use gssapi mechanism, without PAM. Then, it a user presents a ticket the passdb ldap is not used, so the extrafields are never read. The patch in http://dovecot.org/list/dovecot/2012-March/064331.html makes this work I think. I still haven't managed to look into it much though. From pw at wk-serv.de Wed Apr 4 14:38:55 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Wed, 04 Apr 2012 13:38:55 +0200 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <1333435450.21461.35.camel@innu> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> Message-ID: <4F7C32CF.7010908@wk-serv.de> Timo Sirainen schrieb: > On Fri, 2012-03-30 at 21:37 +0200, Patrick Westenberg wrote: >> MTA --(lmtp)--\ /--(lmtp)--> backend1 --\ >> -- director -- -- NFS >> MTA --(lmtp)--/ \--(lmtp)--> backend2 --/ >> >> >> IMAP-User --> frontend1 --\ /--(imap)--> backend1 --\ >> -- director -- -- NFS >> IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/ > > The director above must be the same one for both IMAP and MTA. I know and it is meant to be the same in this drawing. However, in this scenario the director is a single point of failure. What would be the best way to get loadbalancing and high-availability? Patrick From tss at iki.fi Wed Apr 4 14:46:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 14:46:11 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F7C32CF.7010908@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> Message-ID: <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> On 4.4.2012, at 14.38, Patrick Westenberg wrote: > Timo Sirainen schrieb: >> On Fri, 2012-03-30 at 21:37 +0200, Patrick Westenberg wrote: >>> MTA --(lmtp)--\ /--(lmtp)--> backend1 --\ >>> -- director -- -- NFS >>> MTA --(lmtp)--/ \--(lmtp)--> backend2 --/ >>> >>> >>> IMAP-User --> frontend1 --\ /--(imap)--> backend1 --\ >>> -- director -- -- NFS >>> IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/ >> >> The director above must be the same one for both IMAP and MTA. > > I know and it is meant to be the same in this drawing. > > However, in this scenario the director is a single point of failure. > What would be the best way to get loadbalancing and high-availability? Another director. They're meant to connect to each others and do LB/HA. From mlists at edicom.eu Wed Apr 4 14:47:47 2012 From: mlists at edicom.eu (Miguel Tormo) Date: Wed, 4 Apr 2012 13:47:47 +0200 Subject: [Dovecot] Proxy and SSO (single sign-on) In-Reply-To: <48AD358F-ECC4-48F5-8CAB-9B91C8BDEEE3@iki.fi> References: <201204041318.00760.mlists@edicom.eu> <48AD358F-ECC4-48F5-8CAB-9B91C8BDEEE3@iki.fi> Message-ID: <201204041347.47817.mlists@edicom.eu> El Mi?rcoles, 4 de Abril de 2012 13:21:33 Timo Sirainen escribi?: > On 4.4.2012, at 14.18, Miguel Tormo wrote: > > > I have a running setup with a dovecot imap4/pop3 proxy to a few dovecot backend servers which actually store the mailboxes. This is running smoothly and allows me to transparently distribute mailboxes. > > I'm using some "extrafield" configured in the LDAP passdb. > > > > However, now I would like to use GSSAPI (preferred) and NTLM for single sign-on. Both are pretty straightforward to configure in a single instance environment, but I don't know if they would work with proxy. For example, with GSSAPI there are two cases: > > 1) Just use gssapi mechanism, without PAM. Then, it a user presents a ticket the passdb ldap is not used, so the extrafields are never read. > > The patch in http://dovecot.org/list/dovecot/2012-March/064331.html makes this work I think. I still haven't managed to look into it much though. > > It definitely is worth a look. I wonder if it would make ldap extrafields lookups work with gssapi auth, I will try it and post the results. Thank you! From pw at wk-serv.de Wed Apr 4 15:02:02 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Wed, 04 Apr 2012 14:02:02 +0200 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> Message-ID: <4F7C383A.2060401@wk-serv.de> Timo Sirainen schrieb: > Another director. They're meant to connect to each others and do LB/HA. But what about my MTAs? How can I tell my two postfix servers that there are two directors and it should/can use the other one if the first is down? Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one director. From tss at iki.fi Wed Apr 4 15:06:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 15:06:29 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F7C383A.2060401@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> Message-ID: <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> On 4.4.2012, at 15.02, Patrick Westenberg wrote: > Timo Sirainen schrieb: > >> Another director. They're meant to connect to each others and do LB/HA. > > But what about my MTAs? How can I tell my two postfix servers that there are two directors and it should/can use the other one if the > first is down? > > Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. > Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one director. I don't know if Postfix supports that. Typically people use a load balancer (cluster). From jerry at seibercom.net Wed Apr 4 15:29:21 2012 From: jerry at seibercom.net (Jerry) Date: Wed, 4 Apr 2012 08:29:21 -0400 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> Message-ID: <20120404082921.7d404024@scorpio> On Wed, 4 Apr 2012 15:06:29 +0300 Timo Sirainen articulated: > On 4.4.2012, at 15.02, Patrick Westenberg wrote: > > > Timo Sirainen schrieb: > > > >> Another director. They're meant to connect to each others and do > >> LB/HA. > > > > But what about my MTAs? How can I tell my two postfix servers that > > there are two directors and it should/can use the other one if the > > first is down? > > > > Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. > > Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one > > director. > > I don't know if Postfix supports that. Typically people use a load > balancer (cluster). Perhaps posting on the Postfix forum and asking Wietse Venema would be the logical course of action. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From tss at iki.fi Wed Apr 4 15:39:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 15:39:37 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <20120404082921.7d404024@scorpio> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> <20120404082921.7d404024@scorpio> Message-ID: <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> On 4.4.2012, at 15.29, Jerry wrote: >>> Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. >>> Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one >>> director. >> >> I don't know if Postfix supports that. Typically people use a load >> balancer (cluster). > > Perhaps posting on the Postfix forum and asking Wietse Venema would be > the logical course of action. I have a guess though, or alternatively a suggestion if it already doesn't work like that: lmtp:tcp:lmtp.example.com:24 where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places. From patrickdk at patrickdk.com Wed Apr 4 15:54:28 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Wed, 04 Apr 2012 08:54:28 -0400 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> <20120404082921.7d404024@scorpio> <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> Message-ID: <20120404085428.Horde.9RyhTJLnE6FPfESE0TaTcQA@kishi.patrickdk.com> Quoting Timo Sirainen : > On 4.4.2012, at 15.29, Jerry wrote: > >>>> Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. >>>> Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one >>>> director. >>> >>> I don't know if Postfix supports that. Typically people use a load >>> balancer (cluster). >> >> Perhaps posting on the Postfix forum and asking Wietse Venema would be >> the logical course of action. > > I have a guess though, or alternatively a suggestion if it already > doesn't work like that: > > lmtp:tcp:lmtp.example.com:24 > > where lmtp.example.com expands to your two IP addresses. DNS server > handles load balancing by returning IPs in round robin and Postfix > handles fallbacking to the second IP if the first one doesn't work. > Dovecot has similar behavior in several places. Personally I use MX dns entries, lowest mx entry is localhost, then others. From pw at wk-serv.de Wed Apr 4 16:04:11 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Wed, 04 Apr 2012 15:04:11 +0200 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> <20120404082921.7d404024@scorpio> <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> Message-ID: <4F7C46CB.3050709@wk-serv.de> Timo Sirainen schrieb: > where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places. As far as I understand, lmtp.example.com with out records would result in a MX-lookup. If postfix behaves for lmtp like it does for smtp, it will try all listed MX (my directors) in turn. From tss at iki.fi Wed Apr 4 16:09:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 16:09:46 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F7C46CB.3050709@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> <20120404082921.7d404024@scorpio> <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> <4F7C46CB.3050709@wk-serv.de> Message-ID: On 4.4.2012, at 16.04, Patrick Westenberg wrote: > Timo Sirainen schrieb: > >> where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places. > > As far as I understand, lmtp.example.com with out records would result in a MX-lookup. If postfix behaves for lmtp like it does for smtp, it will try all listed MX (my directors) in turn. MX lookups are intended for SMTP, I very much doubt LMTP uses them? From p at state-of-mind.de Wed Apr 4 16:10:38 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Wed, 4 Apr 2012 15:10:38 +0200 Subject: [Dovecot] Modifying LDAP search results for user_attrs Message-ID: <20120404131037.GB23371@state-of-mind.de> I need to authenticate users via LDAP either by mailaddress or login name. In both cases the mailbox location is /src/mail/%d/%n. I can easily deduct that path from the mailaddress, but not so from the login name. Can I get the mail address as part of fetching user_attrs and modify it on the fly? Something along the lines of this: user_attrs = mail:/srv/mail/%d/%n=home,uidNumber=uid,gidNumber=gid Or would I be able to modify this with a post-login script? p at rick -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From tss at iki.fi Wed Apr 4 16:16:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 16:16:22 +0300 Subject: [Dovecot] Modifying LDAP search results for user_attrs In-Reply-To: <20120404131037.GB23371@state-of-mind.de> References: <20120404131037.GB23371@state-of-mind.de> Message-ID: On 4.4.2012, at 16.10, Patrick Ben Koetter wrote: > I need to authenticate users via LDAP either by mailaddress or login name. > In both cases the mailbox location is /src/mail/%d/%n. I can easily deduct > that path from the mailaddress, but not so from the login name. If you have something like mailAddress=user in your pass_attrs and user_attrs, so that the username gets translated to the mailaddress then you can use %n/%d in dovecot.conf. > Can I get the mail address as part of fetching user_attrs and modify it on the > fly? Something along the lines of this: > > user_attrs = mail:/srv/mail/%d/%n=home,uidNumber=uid,gidNumber=gid > > Or would I be able to modify this with a post-login script? The correct syntax is: user_attrs = =home=mail:/srv/mail/%d/%n, .. but this assumes that the login is user at domain. Without changing the username to mail address you can't use %d/%n directly, and post-login script would be the only way to go. From patrickdk at patrickdk.com Wed Apr 4 16:57:01 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Wed, 04 Apr 2012 09:57:01 -0400 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> <20120404082921.7d404024@scorpio> <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> <4F7C46CB.3050709@wk-serv.de> Message-ID: <20120404095701.Horde.oqR4cpLnE6FPfFMtl5U1tdA@kishi.patrickdk.com> Quoting Timo Sirainen : > On 4.4.2012, at 16.04, Patrick Westenberg wrote: > >> Timo Sirainen schrieb: >> >>> where lmtp.example.com expands to your two IP addresses. DNS >>> server handles load balancing by returning IPs in round robin and >>> Postfix handles fallbacking to the second IP if the first one >>> doesn't work. Dovecot has similar behavior in several places. >> >> As far as I understand, lmtp.example.com with out records would >> result in a MX-lookup. If postfix behaves for lmtp like it does for >> smtp, it will try all listed MX (my directors) in turn. > > MX lookups are intended for SMTP, I very much doubt LMTP uses them? Yes, but postfix lmtp is the same binary as smtp, at least currently. From tss at iki.fi Wed Apr 4 17:10:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 17:10:31 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <20120404095701.Horde.oqR4cpLnE6FPfFMtl5U1tdA@kishi.patrickdk.com> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> <20120404082921.7d404024@scorpio> <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> <4F7C46CB.3050709@wk-serv.de> <20120404095701.Horde.oqR4cpLnE6FPfFMtl5U1tdA@kishi.patrickdk.com> Message-ID: On 4.4.2012, at 16.57, Patrick Domack wrote: > Quoting Timo Sirainen : > >> On 4.4.2012, at 16.04, Patrick Westenberg wrote: >> >>> Timo Sirainen schrieb: >>> >>>> where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places. >>> >>> As far as I understand, lmtp.example.com with out records would result in a MX-lookup. If postfix behaves for lmtp like it does for smtp, it will try all listed MX (my directors) in turn. >> >> MX lookups are intended for SMTP, I very much doubt LMTP uses them? > > Yes, but postfix lmtp is the same binary as smtp, at least currently. That's about how to handle incoming lmtp vs. smtp connections. The outgoing smtp vs. lmtp connections are always handled by the same binary anyway. From zybi at talex.pl Wed Apr 4 19:09:50 2012 From: zybi at talex.pl (=?ISO-8859-2?Q?Artur_Zaprza=B3a?=) Date: Wed, 04 Apr 2012 18:09:50 +0200 Subject: [Dovecot] lmtp sometimes fails to deliver a message to all recipients In-Reply-To: <1B1369FF-6748-46DE-AB94-AF2F32061757@iki.fi> References: <4F79C995.9060607@talex.pl> <1B1369FF-6748-46DE-AB94-AF2F32061757@iki.fi> Message-ID: <4F7C724E.8040905@talex.pl> Timo Sirainen wrote: > On 2.4.2012, at 18.45, Artur Zaprza?a wrote: > >> Running strace on lmtp reveals that when lmtp delivers a message to multiple recipients it saves the message to a file in the first mailbox and hard links that file to other mailboxes. But when in the meantime of delivery that file is deleted, lmtp fails to deliver the message to remaining mailboxes. For example a message for 46 recipients is delivered by lmtp to the first 12 only (fortunately postfix retries delivery for remaining recipients) and for the rest lmtp logs the following error messages: >> >> lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Message was expunged (guid) >> lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: script /vmail/domain/foo/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /vmail/domain/foo/.dovecot.sieve.log may reveal additional details) > Fixed in hg. > Tested with Maildir. Works great. Thanks. The above problem was appearing when some recipients (including first one) had a sieve filter with discard action for current message. In this case, depending on the pattern of recipients having a sieve discard action, lmtp can create more than one instance of the message for a few dozen recipients. It would be nice if lmtp could create a single hardlinked instance of the message even in this case. Best regards, Artur Zaprza?a From property.of.mike.jones at gmail.com Wed Apr 4 20:52:27 2012 From: property.of.mike.jones at gmail.com (Mike Jones!) Date: Wed, 4 Apr 2012 13:52:27 -0400 Subject: [Dovecot] Listener not binding to port Message-ID: Hello. Dovecot is not binding or listening on port 993 when I start it. $ sudo netstat -tlnp | grep -c 993 0 Dovecot starts fine and runs without complaints. $ sudo service dovecot restart Restarting IMAP/POP3 mail server: dovecot. $ sudo tail /var/log/mail.log Apr 4 13:42:39 mwjones dovecot: master: Warning: Killed with signal 15 (by pid=16238 uid=0 code=kill) Apr 4 13:42:39 mwjones dovecot: master: Dovecot v2.0.18 starting up (core dumps disabled) $ sudo ps -ef | grep -i [d]ovecot root 16243 1 0 13:42 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf dovecot 16245 16243 0 13:42 ? 00:00:00 dovecot/anvil root 16246 16243 0 13:42 ? 00:00:00 dovecot/log root 16248 16243 0 13:42 ? 00:00:00 dovecot/config Still no listener :( $ sudo netstat -tlnp | grep -c 993 0 Other infos for your scrutiny. $ sudo dovecot --version 2.0.18 $ doveconf -n # 2.0.18: /etc/dovecot/dovecot.conf # OS: Linux 3.2.5-grsec x86_64 Debian wheezy/sid ext4 auth_debug = yes auth_mechanisms = plain login auth_verbose = yes first_valid_uid = 1000 log_timestamp = "%Y-%m-%d %H:%M:%S " mail_debug = yes mail_location = maildir:/home/vmail/%d/%n/Maildir passdb { args = /home/vmail/%d/etc/passwd driver = passwd-file } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } service imap-login { service_count = 1 } service ssl-params { type = startup } ssl_cert = References: <20120404131037.GB23371@state-of-mind.de> Message-ID: <6709704ac05c4ad9e121b73a76e5ae23@in.tum.de> On 2012-04-04 15:10, Patrick Ben Koetter wrote: > Or would I be able to modify this with a post-login script? Just one note: as far as I know, if your mail directory depends on a post-login script, it might be tricky to get LMTP or doveadm commands to work properly. Cheers, Christoph From eliezer at ngtech.co.il Thu Apr 5 06:44:18 2012 From: eliezer at ngtech.co.il (Eliezer Croitoru) Date: Thu, 05 Apr 2012 06:44:18 +0300 Subject: [Dovecot] Listener not binding to port In-Reply-To: References: Message-ID: <4F7D1512.5080102@ngtech.co.il> you should configure imap\pop3 services as far as i remeber... Regards, Eliezer On 04/04/2012 20:52, Mike Jones! wrote: > Hello. > > Dovecot is not binding or listening on port 993 when I start it. > > $ sudo netstat -tlnp | grep -c 993 > 0 > > Dovecot starts fine and runs without complaints. > > $ sudo service dovecot restart > Restarting IMAP/POP3 mail server: dovecot. > > $ sudo tail /var/log/mail.log > Apr 4 13:42:39 mwjones dovecot: master: Warning: Killed with signal > 15 (by pid=16238 uid=0 code=kill) > Apr 4 13:42:39 mwjones dovecot: master: Dovecot v2.0.18 starting up > (core dumps disabled) > > $ sudo ps -ef | grep -i [d]ovecot > root 16243 1 0 13:42 ? 00:00:00 /usr/sbin/dovecot -c > /etc/dovecot/dovecot.conf > dovecot 16245 16243 0 13:42 ? 00:00:00 dovecot/anvil > root 16246 16243 0 13:42 ? 00:00:00 dovecot/log > root 16248 16243 0 13:42 ? 00:00:00 dovecot/config > > Still no listener :( > > $ sudo netstat -tlnp | grep -c 993 > 0 > > Other infos for your scrutiny. > > $ sudo dovecot --version > 2.0.18 > > $ doveconf -n > # 2.0.18: /etc/dovecot/dovecot.conf > # OS: Linux 3.2.5-grsec x86_64 Debian wheezy/sid ext4 > auth_debug = yes > auth_mechanisms = plain login > auth_verbose = yes > first_valid_uid = 1000 > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_debug = yes > mail_location = maildir:/home/vmail/%d/%n/Maildir > passdb { > args = /home/vmail/%d/etc/passwd > driver = passwd-file > } > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > user = root > } > service imap-login { > service_count = 1 > } > service ssl-params { > type = startup > } > ssl_cert = ssl_key = userdb { > args = /home/vmail/%d/etc/passwd > driver = passwd-file > } > verbose_ssl = yes > > > What am I missing and how do I fix this? > > Thanks, > mwjones -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer ngtech.co.il From tss at iki.fi Thu Apr 5 09:08:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 5 Apr 2012 09:08:42 +0300 Subject: [Dovecot] Listener not binding to port In-Reply-To: References: Message-ID: <0EBD739C-2E2E-4CFB-B149-852AFD7279F4@iki.fi> On 4.4.2012, at 20.52, Mike Jones! wrote: > Dovecot is not binding or listening on port 993 when I start it. Try adding protocols=imap setting. From lists at wiesinger.com Thu Apr 5 09:16:33 2012 From: lists at wiesinger.com (Gerhard Wiesinger) Date: Thu, 5 Apr 2012 08:16:33 +0200 (CEST) Subject: [Dovecot] Dovecot upgrade from 1.2.x to 2.0.x: roundcube/squirrelmail sent folder doesn't work any more In-Reply-To: References: <4F71F8D8.6040700@wiesinger.com> <95C664AD-0532-4F3B-A2EB-2FD25B79187B@iki.fi> Message-ID: On Wed, 4 Apr 2012, Timo Sirainen wrote: > On 29.3.2012, at 8.25, Gerhard Wiesinger wrote: > >>>> 2.0.x: with Prefix ~/Mail >>>> A0003 LIST "" ~/Mail/sent >>>> A0003 OK List completed.: >>> >> >> mail_full_filesystem_access = yes >> mail_location = mbox:~:INBOX=/var/mail/%u > > Yes, the problem happens only with mail_full_filesystem_access=yes. It already works in v2.1, so I don't think I'll bother looking into why it's not working in v2.0. Hello Timo, As discussed per private mail you finally fixed it: http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e Thnx for your support. Ciao, Gerhard -- http://www.wiesinger.com/ From dvorak at cosign.jp Thu Apr 5 10:48:45 2012 From: dvorak at cosign.jp (Akihiko Sato) Date: Thu, 5 Apr 2012 16:48:45 +0900 Subject: [Dovecot] problem to configuration dovecot and postfix Message-ID: Hi, I'm having a problem with dovecot and postfix. I'm using a dovecot version 2.0.13. Postfix (Version: 2.8.5) config is changed according to the http://wiki2.dovecot.org/LDA/Postfix: I have thought to build a virtual user and virtual mailbox, and use the functionality of the Sieve. However, when the test set, the mail not received the mailbox. Record that there was a "temporary failure" in the log of Postfix. It delivered and to change the "virtual" to "dovecot", I think that's a matter of dovecot. To ensure that mail is delivered as intended, will How do I? Can anyone give me a hint what to do? # doveconf -n: # 2.0.13: /etc/dovecot/dovecot.conf # OS: Linux 3.0.0-12-server x86_64 Ubuntu 11.10 ext4 auth_debug = yes auth_debug_passwords = yes auth_verbose = yes disable_plaintext_auth = no info_log_path = /var/log/dovecot-info.log lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:/var/mail/vhosts/%d/%n managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables bo dy enotify environment mailbox date namespace { inbox = yes list = yes location = prefix = INBOX/ separator = / subscriptions = yes type = private } namespace { hidden = no list = yes location = maildir:/var/mail/shared/:INDEX=/var/mail/vhosts/%d/%n/shared/:CONTROL=/var/mail/vhosts/%d/%n/shared/ prefix = &UXFnCQ-/ separator = / subscriptions = yes type = public } passdb { args = /etc/auth/passwd.main driver = passwd-file } passdb { args = username_format=%n /etc/auth/%d/passwd driver = passwd-file } passdb { driver = pam } plugin { sieve = /var/mail/vhosts/%d/%n/_dovecot.sieve sieve_dir = /var/mail/vhosts/%d/%n/sieve/ } postmaster_address = postmaster at dom3.gs3 protocols = imap sieve service auth { user = root } ssl_cert = Apr 5 16:38:36 qbu postfix/cleanup[25500]: BDC3221212: message-id=<20120405073836.BDC3221212 at qbu> Apr 5 16:38:36 qbu postfix/qmgr[25471]: BDC3221212: from=, size=230, nrcpt=1 (queue active) Apr 5 16:38:36 qbu postfix/pipe[25502]: BDC3221212: to=, relay=dovecot, delay=0.04, delays=0.02/0/0/0.02, dsn=4.3.0, status=deferred (temporary failure) Regards, Akihiko From cor at xs4all.nl Thu Apr 5 11:15:09 2012 From: cor at xs4all.nl (Cor Bosman) Date: Thu, 5 Apr 2012 10:15:09 +0200 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F7C383A.2060401@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> Message-ID: <20120405081509.GA1598@xs4all.nl> On Wed, Apr 04, 2012 at 02:02:02PM +0200, Patrick Westenberg wrote: > Timo Sirainen schrieb: > >> Another director. They're meant to connect to each others and do LB/HA. > > But what about my MTAs? How can I tell my two postfix servers that there > are two directors and it should/can use the other one if the > first is down? We use hardware loadbalancers in front of the directors. If a director goes down, the hardware loadbalancers sees that. We also use the poolmon script to tell the directors if a certain imap server is missing. Cor From tlx at leuxner.net Thu Apr 5 11:18:14 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 5 Apr 2012 10:18:14 +0200 Subject: [Dovecot] problem to configuration dovecot and postfix In-Reply-To: References: Message-ID: <20120405081814.GA7578@nihlus.leuxner.net> On Thu, Apr 05, 2012 at 04:48:45PM +0900, Akihiko Sato wrote: > # postconf -n: > mydestination = qbu.example.com, qbu, localhost.localdomain, localhost http://www.postfix.org/VIRTUAL_README.html [...] NEVER list a virtual MAILBOX domain name as a mydestination domain! > /var/log/mail.log: > Apr 5 16:38:36 qbu postfix/pickup[25470]: BDC3221212: uid=0 from= > Apr 5 16:38:36 qbu postfix/cleanup[25500]: BDC3221212: > message-id=<20120405073836.BDC3221212 at qbu> > Apr 5 16:38:36 qbu postfix/qmgr[25471]: BDC3221212: > from=, size=230, nrcpt=1 (queue active) > Apr 5 16:38:36 qbu postfix/pipe[25502]: BDC3221212: > to=, relay=dovecot, delay=0.04, delays=0.02/0/0/0.02, > dsn=4.3.0, status=deferred (temporary failure) Also with 2.x you may want to use LMTP rather than the LDA Piping. http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From p at state-of-mind.de Thu Apr 5 11:44:08 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Thu, 5 Apr 2012 10:44:08 +0200 Subject: [Dovecot] problem to configuration dovecot and postfix In-Reply-To: <20120405081814.GA7578@nihlus.leuxner.net> References: <20120405081814.GA7578@nihlus.leuxner.net> Message-ID: <20120405084408.GB10931@state-of-mind.de> * Thomas Leuxner : > On Thu, Apr 05, 2012 at 04:48:45PM +0900, Akihiko Sato wrote: > > > # postconf -n: > > mydestination = qbu.example.com, qbu, localhost.localdomain, localhost > > http://www.postfix.org/VIRTUAL_README.html > > [...] > NEVER list a virtual MAILBOX domain name as a mydestination domain! ... unless you know what you do and wish to benefit from the side-effects. p at rick -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 316 bytes Desc: Digital signature URL: From dvorak at cosign.jp Thu Apr 5 11:54:13 2012 From: dvorak at cosign.jp (Akihiko Sato) Date: Thu, 5 Apr 2012 17:54:13 +0900 Subject: [Dovecot] problem to configuration dovecot and postfix In-Reply-To: <20120405081814.GA7578@nihlus.leuxner.net> References: <20120405081814.GA7578@nihlus.leuxner.net> Message-ID: 2012/4/5 Thomas Leuxner : > On Thu, Apr 05, 2012 at 04:48:45PM +0900, Akihiko Sato wrote: > >> # postconf -n: >> mydestination = qbu.example.com, qbu, localhost.localdomain, localhost > > http://www.postfix.org/VIRTUAL_README.html > > [...] > NEVER list a virtual MAILBOX domain name as a mydestination domain! Oops, this is my careless mistake...mydestination is empty. But status does not change. Regards, Akihiko > > Also with 2.x you may want to use LMTP rather than the LDA Piping. > > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP > > Regards > Thomas > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > > iEYEARECAAYFAk99VUYACgkQ+meF/S97aXi/4wCgkLGKNByWuxYb0E7YIdZceHHU > UwQAnArESXtRggXYRbnk64WaVfItfAY9 > =N8Aq > -----END PGP SIGNATURE----- > From tlx at leuxner.net Thu Apr 5 12:31:20 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 5 Apr 2012 11:31:20 +0200 Subject: [Dovecot] problem to configuration dovecot and postfix In-Reply-To: References: <20120405081814.GA7578@nihlus.leuxner.net> Message-ID: <20120405093120.GB7578@nihlus.leuxner.net> On Thu, Apr 05, 2012 at 05:54:13PM +0900, Akihiko Sato wrote: > Oops, this is my careless mistake...mydestination is empty. > But status does not change. > > Regards, > Akihiko Hmm. Haven't used LDA since LMTP was introduced. Are there corresponding log entries on the Dovecot side? [...] protocol lda { postmaster_address = postmaster at domain.tld log_path = /var/log/dovecot-deliver.log info_log_path = /var/log/dovecot-deliver.log } Still would go the LMTP route rather than using Pipes: > > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP > > Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From dvorak at cosign.jp Thu Apr 5 13:10:07 2012 From: dvorak at cosign.jp (Akihiko Sato) Date: Thu, 5 Apr 2012 19:10:07 +0900 Subject: [Dovecot] problem to configuration dovecot and postfix In-Reply-To: <20120405093120.GB7578@nihlus.leuxner.net> References: <20120405081814.GA7578@nihlus.leuxner.net> <20120405093120.GB7578@nihlus.leuxner.net> Message-ID: 2012/4/5 Thomas Leuxner : > On Thu, Apr 05, 2012 at 05:54:13PM +0900, Akihiko Sato wrote: >> Oops, this is my careless mistake...mydestination is empty. >> But status does not change. >> >> Regards, >> Akihiko > > Hmm. Haven't used LDA since LMTP was introduced. Are there corresponding > log entries on the Dovecot side? > > [...] > protocol lda { > ? ? ? ?postmaster_address = postmaster at domain.tld > ? ? ? ?log_path = /var/log/dovecot-deliver.log > ? ? ? ?info_log_path = /var/log/dovecot-deliver.log > } I tried to restart the dovecot put these settings, the log was not generated. Existing log (dovecot.log, dovecot-info.log) that an error is also recorded in the was not. ;-( In this case, is it better to use LMTP? > > Still would go the LMTP route rather than using Pipes: > >> > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP >> > > > Thomas > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > > iEYEARECAAYFAk99ZmgACgkQ+meF/S97aXh1/ACgo/VaG3SP587MvPDwkXGRn/lG > o08An1nKrVecLm6yS9HuG60PULJgw0I3 > =hUSv > -----END PGP SIGNATURE----- > Regards, Akihiko From busseniu at in.tum.de Thu Apr 5 13:36:03 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Thu, 05 Apr 2012 12:36:03 +0200 Subject: [Dovecot] Dovecot allows creation of folders outside of a user's directory In-Reply-To: References: <4F759B08.1060603@in.tum.de> Message-ID: <4F7D7593.2090005@in.tum.de> On 04.04.2012 03:35, Timo Sirainen wrote: > On 30.3.2012, at 14.37, Christoph Bu?enius wrote: >> -- even though it will fail with a "permission denied" error -- will create a directory like "/mail/users/strange&ANY- characters". That directory will only contain a subdirectory "Maildir" and therein dovecot-acl-list. > > Fixed: http://hg.dovecot.org/dovecot-2.0/rev/b15889b82258 > Thanks, this fixed it. By the way, your tireless work on Dovecot is amazing :) -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From CMarcus at Media-Brokers.com Thu Apr 5 15:59:22 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 05 Apr 2012 08:59:22 -0400 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header Message-ID: <4F7D972A.8050406@Media-Brokers.com> On 2012-04-05 4:18 AM, Thomas Leuxner wrote: > Also with 2.x you may want to use LMTP rather than the LDA Piping. > > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP I am preparing to convert my main client's postfix_courier-imap setup to dovecot 2.1, which currently just uses the postfix virtual delivery agent... Does anyone know if the use of LMTP (or even the dovecot LDA) still loses the x-original-to header that the postfix vda adds and that I rely heavily on (since I use a lot of aliases), and if it does, is there any solution to get the original recipient added back in before final delivery? Everything I'm reading says that LMTP is better, but I really do need this header (or one like it) to be there so I know who the original recipient was (for filtering and other purposes). Thanks, -- Best regards, Charles From l.chelchowski at eurocar.pl Thu Apr 5 16:29:13 2012 From: l.chelchowski at eurocar.pl (l.chelchowski at eurocar.pl) Date: Thu, 05 Apr 2012 15:29:13 +0200 Subject: [Dovecot] Problem with polish characters and sieve fileinto Message-ID: <4F7D9E29.3050802@eurocar.pl> Hi, I have problems with sieve fileinto and polish characters in folder names: # rule:[??????????????????] elsif anyof (header :contains "Subject" "??????????????????") { fileinto "INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-"; } dovecot log: Apr 5 15:17:42 mailer dovecot: lmtp(54203, test at test1.com): rmHVJHabfU+70wAA9hAjDg: sieve: msgid=<4F7D9B68.4030206 at test2.com>: stored mail into mailbox 'INBOX' Apr 5 15:17:42 mailer dovecot: lmtp(54203, test at test1.com): Error: rmHVJHabfU+70wAA9hAjDg: sieve: execution of script /home/vmail/test1.com/test//.dovecot.sieve failed, but implicit keep was successful (user logfile /home/vmail/test1.com/test//.dovecot.sieve.log may reveal additional details) Apr 5 15:17:42 mailer postfix/lmtp[54202]: 941B1E2681F: to=, relay=m.test1.com[private/dovecot-lmtp], delay=0.04, delays=0/0.01/0/0.03, dsn=2.0.0, status=sent (250 2.0.0 rmHVJHabfU+70wAA9hAjDg Saved) dovecot.sieve.log: sieve: info: started log at Apr 05 15:17:42. error: msgid=<4F7D9B68.4030206 at test2.com>: failed to store into mailbox 'INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-' (INBOX/&-AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-): Mailbox doesn't exist: INBOX.&-AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-. Thanks for help. -- Pozdrawiam ?ukasz From zybi at talex.pl Thu Apr 5 16:45:24 2012 From: zybi at talex.pl (=?UTF-8?B?QXJ0dXIgWmFwcnphxYJh?=) Date: Thu, 05 Apr 2012 15:45:24 +0200 Subject: [Dovecot] Problem with polish characters and sieve fileinto In-Reply-To: <4F7D9E29.3050802@eurocar.pl> References: <4F7D9E29.3050802@eurocar.pl> Message-ID: <4F7DA1F4.40207@talex.pl> l.chelchowski at eurocar.pl wrote: > Hi, > > I have problems with sieve fileinto and polish characters in folder names: > # rule:[??????????????????] > elsif anyof (header :contains "Subject" "??????????????????") > { > fileinto "INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-"; > } > Sieve script expects UTF-8 folder names. This should work: fileinto "INBOX/??????????????????"; You can also use UTF-8 mailbox names on disk if you add UTF-8 option to mail_location: mail_location = maildir:~/Maildir:UTF-8 but you have to convert existing names to UTF-8 somehow. > > dovecot log: > Apr 5 15:17:42 mailer dovecot: lmtp(54203, test at test1.com): > rmHVJHabfU+70wAA9hAjDg: sieve: msgid=<4F7D9B68.4030206 at test2.com>: stored mail > into mailbox 'INBOX' > Apr 5 15:17:42 mailer dovecot: lmtp(54203, test at test1.com): Error: > rmHVJHabfU+70wAA9hAjDg: sieve: execution of script > /home/vmail/test1.com/test//.dovecot.sieve failed, but implicit keep was > successful (user logfile /home/vmail/test1.com/test//.dovecot.sieve.log may > reveal additional details) > Apr 5 15:17:42 mailer postfix/lmtp[54202]: 941B1E2681F: to=, > relay=m.test1.com[private/dovecot-lmtp], delay=0.04, delays=0/0.01/0/0.03, > dsn=2.0.0, status=sent (250 2.0.0 rmHVJHabfU+70wAA9hAjDg Saved) > > > dovecot.sieve.log: > sieve: info: started log at Apr 05 15:17:42. > error: msgid=<4F7D9B68.4030206 at test2.com>: failed to store into mailbox > 'INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-' > (INBOX/&-AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-): Mailbox doesn't > exist: INBOX.&-AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-. > > Thanks for help. -- Talex Sp??ka Akcyjna z siedzib? w Poznaniu adres: ul. Karpia 27d, 61-619 Pozna? NIP 782-00-21-045 zarejestrowana w S?dzie Rejonowym Pozna? ? Nowe Miasto i Wilda w Poznaniu VIII Wydzia? Gospodarczy - KRS pod nr 000048779 kapita? zak?adowy: 3.000.092,00 PLN (w ca?o?ci wp?acony) Uwaga: Niniejsza wiadomo??, w szczeg?lno?ci jej tre?? oraz za??czniki, mo?e by? poufna. W przypadku, gdy nie jest Pan/Pani zamierzonym jej adresatem, informujemy, ?e wszelkie rozpowszechnianie, dystrybucja lub powielanie powy?szej wiadomo?ci jest zabronione. Jednocze?nie prosimy o powiadomienie nadawcy oraz niezw?oczne usuni?cie powy?szej wiadomo?ci wraz z za??cznikami. Dzi?kujemy, Talex S.A. w Poznaniu. Confidentiality Notice: This email, particularly its content and any attached files, may be confidential. If you are not an intended recipient, any disclosure, distribution and reproduction of this message is prohibited. In this case please notify the sender immediately and then delete this message and any attachments. Thank you, Talex S.A., Poznan. From stephan at rename-it.nl Thu Apr 5 16:47:16 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 05 Apr 2012 15:47:16 +0200 Subject: [Dovecot] Problem with polish characters and sieve fileinto In-Reply-To: <4F7D9E29.3050802@eurocar.pl> References: <4F7D9E29.3050802@eurocar.pl> Message-ID: <4F7DA264.8020004@rename-it.nl> Op 4/5/2012 3:29 PM, l.chelchowski at eurocar.pl schreef: > Hi, > > I have problems with sieve fileinto and polish characters in folder > names: > # rule:[??????????????????] > elsif anyof (header :contains "Subject" "??????????????????") > { > fileinto "INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-"; > } Looks like your Sieve GUI uses the wrong Unicode format. The above is mUTF7 as used by IMAP. However, Sieve expects UTF-8. So it should use this instead: fileinto "INBOX/??????????????????"; Regards, Stephan. From xnasx at yandex.ru Thu Apr 5 16:51:32 2012 From: xnasx at yandex.ru (Victor) Date: Thu, 05 Apr 2012 17:51:32 +0400 Subject: [Dovecot] Username bug in '%%' variables selects Message-ID: <10301333633892@web94.yandex.ru> Hi! Forgive my bad English. I trying to set up shared folders, but I can't. I shared INBOX of 'victor at domain.com' for 'victor at testmail.domain.com' There is my debugging results: doveadm acl debug -u victor at testmail.domain.com INBOX.shared.victor at domain.com. doveadm(victor at testmail.domain.com): Error: Couldn't create namespace 'INBOX.shared.' for user victor at domain: userdb didn't return a home directory, but mail_location used it (%h): maildir:%h That's I seen in logs: auth-worker: Debug: sql(victor at domain): SELECT CONCAT('/var/spool/mail/', SUBSTRING_INDEX(email, '@', -1), '/', SUBSTRING_INDEX(email, '@', 1)) as home from `users` WHERE email='victor at domain' What happened with my username? 'victor at domain' is not correct value. My config: namespace { inbox = yes location = prefix = INBOX. separator = . type = private } namespace { hidden = no list = yes location = maildir:%%h prefix = INBOX.shared.%%u. separator = . subscriptions = no type = shared } password_query = \ SELECT email as username, password \ FROM users WHERE email = '%u' user_query = SELECT CONCAT('/var/spool/mail/', SUBSTRING_INDEX(email, '@', -1), '/', SUBSTRING_INDEX(email, '@', 1)) as home from `users` WHERE email='%u' Please, help me! -- Best regards, Victor From property.of.mike.jones at gmail.com Thu Apr 5 18:08:59 2012 From: property.of.mike.jones at gmail.com (Mike Jones!) Date: Thu, 5 Apr 2012 11:08:59 -0400 Subject: [Dovecot] Listener not binding to port In-Reply-To: <0EBD739C-2E2E-4CFB-B149-852AFD7279F4@iki.fi> References: <0EBD739C-2E2E-4CFB-B149-852AFD7279F4@iki.fi> Message-ID: Thanks so much all. I was missing a few items: protocols = imap # thanks to Timo Sirainen # thanks to Noel Butler service imap-login { inet_listener imap { port = 0 } inet_listener imaps { address = * port = 993 } service_count = 1 } Here is my final doveconf -n with listener successfully running on 993: # 2.0.18: /etc/dovecot/dovecot.conf # OS: Linux 3.2.5-grsec x86_64 Debian wheezy/sid ext4 auth_debug = yes auth_mechanisms = plain login auth_verbose = yes first_valid_gid = 1030 first_valid_uid = 1030 info_log_path = /var/log/dovecot-info.log last_valid_gid = 1030 last_valid_uid = 1030 log_path = /var/log/dovecot.log log_timestamp = "%Y-%m-%d %H:%M:%S " mail_debug = yes mail_location = maildir:/home/vmail/%d/%n/Maildir passdb { args = /home/vmail/%d/etc/passwd driver = passwd-file } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { address = * port = 993 } service_count = 1 } service ssl-params { type = startup } ssl_cert = Hi all, I'm planning on implementing this in my new upcoming dovecot instance, and would like to hear thoughts on how best to accomplish this. We will be paying Timo's support company to do the work, but obviously, the less work in the form of coding he has to do to get this working (I'm hoping it won't be a lot), the more money it will save us... ;) First - I currently use rsnapshot to backup emails, so that is the use-case I'm most interested in getting working. It is rsync based, and like other rsync based backup programs it uses hardlinks to save storage space - so you can have a *lot* of backups (going back months, or even years), where each snapshot only adds a little more to the total disk space being used. The snapshots are stored with the following filesystem layout: /path/to/snapshotsdir/hourly.0 ... /path/to/snapshotsdir/hourly.4 /path/to/snapshotsdir/daily.0 ... /path/to/snapshotsdir/daily.7 /path/to/snapshotsdir/weekly.0 ... /path/to/snapshotsdir/weekly.4 /path/to/snapshotsdir/monthly.0 ... /path/to/snapshotsdir/monthly.12 /path/to/snapshotsdir/yearly.0 ... /path/to/snapshotsdir/yearly.5 The 'names' (hourly, daily, weekly, monthly, yearly) are arbitrary (this is a bit confusing to people new to rsnapshot), and would *not* be used for displaying the mail folders to the users - it is the Date/Time stamps of each of the snapshot dirs above that would be used to display the folder names under the 'Time Machine' namespace. This is, I imagine, the part that will need some actual coding by Timo to get working - maybe just some new config variables added to the namespace code for mapping the date/time stamps of the directories to user friendly folder names in the namespace. That said, I'd like to design this and have it coded such that it will work with almost any type of backup storage that stores snapshots as date/time stamped directories like this (there must be others, right?). Also, it goes without saying that this code will be (if Timo is ok with it) part of the core dovecot code going forward, so anyone else will be able to benefit from it. What I'm envisioning is something like this... 1. Define a namespace - for this example we'll call it 'Time Machine' 2. Under this namespace, each user will see their, and *only* their snapshots So, each user would see something like this: My Mail Account Inbox Drafts Templates Sent Time Machine (sorted above user created folders if possible) -4/3/12, 8:00am (first subfolder) Inbox Drafts etc... (all other folders and sub-folders shown here) +4/3/12, 12:00pm (first subfolder) etc... Other User Folders ... Or even better, I'm thinking some magical code that can group them by Date, like: -4/3/12 (first subfolder) -8:00am (next sub-folder) Inbox Drafts Etc... (all folders and sub-folders shown here) +12:00pm +4:00pm +8:00pm +4/4/12 etc... Comments? Suggestions? Flames? -- Best regards, Charles From tom at whyscream.net Thu Apr 5 19:37:30 2012 From: tom at whyscream.net (Tom Hendrikx) Date: Thu, 05 Apr 2012 18:37:30 +0200 Subject: [Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups In-Reply-To: <4F7DBA3B.5090803@Media-Brokers.com> References: <4F7DBA3B.5090803@Media-Brokers.com> Message-ID: <4F7DCA4A.70009@whyscream.net> On 05-04-12 17:28, Charles Marcus wrote: > Hi all, > > I'm planning on implementing this in my new upcoming dovecot instance, > and would like to hear thoughts on how best to accomplish this. We will > be paying Timo's support company to do the work, but obviously, the less > work in the form of coding he has to do to get this working (I'm hoping > it won't be a lot), the more money it will save us... ;) > > First - I currently use rsnapshot to backup emails, so that is the > use-case I'm most interested in getting working. It is rsync based, and > like other rsync based backup programs it uses hardlinks to save storage > space - so you can have a *lot* of backups (going back months, or even > years), where each snapshot only adds a little more to the total disk > space being used. > > What I'm envisioning is something like this... > > 1. Define a namespace - for this example we'll call it 'Time Machine' > > 2. Under this namespace, each user will see their, and *only* their > snapshots > > So, each user would see something like this: > > My Mail Account > Inbox > Drafts > Templates > Sent > Time Machine (sorted above user created folders if possible) > -4/3/12, 8:00am (first subfolder) > Inbox > Drafts > etc... (all other folders and sub-folders shown here) > +4/3/12, 12:00pm (first subfolder) > etc... > Other User Folders > ... > > Or even better, I'm thinking some magical code that can group them by > Date, like: > > -4/3/12 (first subfolder) > -8:00am (next sub-folder) > Inbox > Drafts > Etc... (all folders and sub-folders shown here) > +12:00pm > +4:00pm > +8:00pm > +4/4/12 > etc... > > Comments? Suggestions? Flames? > The first interesting point I'd see with this, is that you supply the mail client with a near endless supply of folders, which would take a lot of caching space on the clients end, either (depending on the client and its configuration) from the moment that you enable this fort hem, or after someone starts searching in their 'time machine' for some old mail. I see my mail client on a new install working quite hard to download mail headers for 2 years of postfix/dovecot/etc mailing lists, so what happens if you provide a 'time machine' namespace going 1 month back, 4 with snapshots a day (i.e. 31x4 =~ 120 times more headers to download/index). -- Tom From CMarcus at Media-Brokers.com Thu Apr 5 20:02:51 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 05 Apr 2012 13:02:51 -0400 Subject: [Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups In-Reply-To: <4F7DCA4A.70009@whyscream.net> References: <4F7DBA3B.5090803@Media-Brokers.com> <4F7DCA4A.70009@whyscream.net> Message-ID: <4F7DD03B.30909@Media-Brokers.com> On 2012-04-05 12:37 PM, Tom Hendrikx wrote: > The first interesting point I'd see with this, is that you supply the > mail client with a near endless supply of folders, which would take a > lot of caching space on the clients end, either (depending on the client > and its configuration) from the moment that you enable this fort hem, or > after someone starts searching in their 'time machine' for some old mail. > > I see my mail client on a new install working quite hard to download > mail headers for 2 years of postfix/dovecot/etc mailing lists, so what > happens if you provide a 'time machine' namespace going 1 month back, 4 > with snapshots a day (i.e. 31x4 =~ 120 times more headers to > download/index). Interesting and valid point... hmmmm..... First, these folders would be read-only - a user could copy something from there back to one of his other folders, but couldn't write anything in them - so nothing would be changing under this namespace, except new snapshots magically appearing, which means that once they are indexed, the indexes would never need to be rebuilt (unless they got corrupted somehow). But, yeah, I can imagine some problems especially if someone has a ton of email. And while these would probably only be accessed rarely, in those cases where someone would want to access them, they would very likely want to be able to search, so disabling indexes wouldn't be a good idea... Since we use Thunderbird, I can of course disable offline mode for everyone, so the only time headers would be downloaded would be when the user selects (or performs a search on) one (or more) of the folders. Maybe Timo can think of something creative to minimize this problem... -- Best regards, Charles From zarf at klacto.net Thu Apr 5 20:37:37 2012 From: zarf at klacto.net (FZiegler) Date: Thu, 05 Apr 2012 13:37:37 -0400 Subject: [Dovecot] dovecot and unison In-Reply-To: <8EAB42AF-BC4A-4235-9CF9-A444BE553B10@odo.in-berlin.de> References: <4F7A2FF9.7000204@klacto.net> <8EAB42AF-BC4A-4235-9CF9-A444BE553B10@odo.in-berlin.de> Message-ID: <4F7DD861.7060303@klacto.net> Michael Grimm wrote: >> Namely, which of the ancillary files should I be syncing (or perhaps >> deleting before sync)? > I used ... > > ignore = Name dovecot* > ignore = Regex .*/tmp$ > ignore = Regex .*/\.Drafts$ > > ? to ignore all of them. Thanks Michael for this (and thanks also mazieres at stanford for your reply). I am now merrily ignoring these files too, and synced 400 MB of mail without seeing the disappearing subfolder issue again. Francois Z. From hsn at filez.com Thu Apr 5 21:46:50 2012 From: hsn at filez.com (Radim Kolar) Date: Thu, 05 Apr 2012 20:46:50 +0200 Subject: [Dovecot] create mailboxes in /var/mail via doveadm Message-ID: <4F7DE89A.4020409@filez.com> what usedid is used by doveadm for creating mailboxes? can it be setup to run as dovecot:mail? cant create mailbox from command line ponto# ls -ld /var/mail drwxrwxr-x 8 root mail 24 Apr 5 20:35 /var/mail ponto# doveadm mailbox create -u jboss INBOX doveadm(jboss): Error: Can't create mailbox INBOX: Permission denied mail_location = maildir:%h/Maildir:INBOX=/var/mail/%u From slusarz at curecanti.org Thu Apr 5 21:59:20 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Thu, 05 Apr 2012 12:59:20 -0600 Subject: [Dovecot] 2.1.3: Overly lax FETCH parsing Message-ID: <20120405125920.Horde.wazxEIF5lbhPfeuIm_gQ4uA@bigworm.curecanti.org> These commands both work in Dovecot (as of 2.1.3): 1 UID FETCH 1:* () (CHANGEDSINCE 1) * 1 FETCH (UID 30097 MODSEQ (19554)) [...] 1 OK Fetch completed. 2 FETCH 1:* () (CHANGEDSINCE 1) * 2 FETCH (MODSEQ (19554)) [...] 2 OK Fetch completed. However, RFC 3501 indicates that an empty FETCH attribute list is incorrect: fetch = "FETCH" SP sequence-set SP ("ALL" / "FULL" / "FAST" / fetch-att / "(" fetch-att *(SP fetch-att) ")") fetch-att = "ENVELOPE" / "FLAGS" / "INTERNALDATE" / "RFC822" [".HEADER" / ".SIZE" / ".TEXT"] / "BODY" ["STRUCTURE"] / "UID" / "BODY" section ["<" number "." nz-number ">"] / "BODY.PEEK" section ["<" number "." nz-number ">"] In other words, fetch-att is not allowed to be empty. (RFC 4466 does not alter this requirement when using FETCH modifiers). While useful that Dovecot is more liberal about what it receives, 3501 seems pretty clear that incorrect FETCH parameters must return a BAD. I can verify that the above commands fail on Cyrus. I don't disagree that the way Dovecot handles this is correct - or at least is the way that I wish the RFCs were written. Since this seems a bit redundant, but is necessary under the ABNF: 1 UID FETCH 1:* UID (CHANGEDSINCE 1) -or- 1 UID FETCH 1:* (UID) (CHANGEDSINCE 1) michael From centos.admin at gmail.com Thu Apr 5 23:02:17 2012 From: centos.admin at gmail.com (Emmanuel Noobadmin) Date: Fri, 6 Apr 2012 04:02:17 +0800 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? Message-ID: I'm trying to improve the setup of our Dovecot/Exim mail servers to handle the increasingly huge accounts (everybody thinks it's like infinitely growing storage like gmail and stores everything forever in their email accounts) by changing from Maildir to mdbox, and to take advantage of offloading older emails to alternative networked storage nodes. The question now is whether having a single large server or will a number of 1U servers with the same total capacity be better? Will be using RAID 1 pairs, likely XFS based on reading Hoeppner's recommendation on this and the mdadm list. Currently, I'm leaning towards multiple small servers because I think it should be better in terms of performance. At the very least even if one node gets jammed up, the rest should still be able serve up the emails for other accounts that is unless Dovecot will get locked up by that jammed transaction. Also, I could possibly arrange them in a sort of network raid 1 to gain redundancy over single machine failure. Would I be correct in these or do actual experiences say otherwise? From bruce.hammond at contractor.verio.net Fri Apr 6 01:05:53 2012 From: bruce.hammond at contractor.verio.net (Bruce Hammond) Date: Thu, 5 Apr 2012 18:05:53 -0400 Subject: [Dovecot] dovecot: auth: Error: ldap(...) failed: No such object Message-ID: <4803B289543E1244868D9FF82F670AA002E999AD@IAD-WPRD-XCHB01.corp.verio.net> I just installed the 2.1.3 version of dovecot. When I telnet to my dovecot linux server on the imap port (143) and try to login I get the error "No [UNAVAILABLE] Temporary authentication failure" on the telnet display and dovecot: auth: Error: ldap(bhammond_test at 131.103.212.38,128.121.121.193): ldap_search(base=uid=bhammond_test at 131.103.212.38,ou=users,dc=aspect filter=(&(objectClass=posixAccount)(productFeature=mail))) failed: No such object in the /var/log/maillog. I am not familiar with ldap and I haven't been able to find anything using google. Anyone have any ideas with the problem? Thanks. Here is my configuration: # 2.1.3: /usr/local/packages/dovecot2/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-274.3.1.el5.secure i686 Red Hat Enterprise Linux Server release 5.7 (Tikanga) auth_mechanisms = plain login auth_verbose = yes default_login_user = dovecot disable_plaintext_auth = no listen = * mail_location = maildir:~/mail passdb { args = /etc/devel/passdb.conf driver = ldap } pop3_lock_session = yes protocols = imap pop3 service auth { user = dovecot } service imap-login { process_limit = 128 process_min_avail = 30 service_count = 0 user = dovecot vsz_limit = 64 M } service pop3-login { process_limit = 128 process_min_avail = 30 service_count = 0 user = dovecot vsz_limit = 64 M } shutdown_clients = no ssl_cert = References: Message-ID: A timely topic as I was just mulling over ways to provide this to my users. Charles Marcus wrote: > The snapshots are stored with the following filesystem layout: > > /path/to/snapshotsdir/hourly.0 > ... This is familiar to NetApp users. > The 'names' (hourly, daily, weekly, monthly, yearly) are arbitrary (this > is a bit confusing to people new to rsnapshot), and would *not* be used > for displaying the mail folders to the users - it is the Date/Time > stamps of each of the snapshot dirs above that would be used to display > the folder names under the 'Time Machine' namespace. This is, I imagine, > the part that will need some actual coding by Timo to get working - > maybe just some new config variables added to the namespace code for > mapping the date/time stamps of the directories to user friendly folder > names in the namespace. > > That said, I'd like to design this and have it coded such that it will > work with almost any type of backup storage that stores snapshots as > date/time stamped directories like this (there must be others, right?). One idea is to take this complexity entirely out of dovecot and create a synthetic filesystem using hard or soft links (as rsnapshot has done) and create your own, with whatever weird and wonderful naming scheme you want. /path/to/TimeMachine// -> /path/to/snapshotsdir// namespace { prefix = TimeMachine location = maildir:/path/to/TimeMachine:INDEX=MEMORY ... } This might not be very scalable depending on how big your userbase is. I would probably define memory indices for this namespace, and take the performance hit on the assumption that access will be a once-in-a- while thing. On-disk indices will probably get out of date with each snapshot rollover, if if you have a lot of snapshots/mailboxes, it could consume a non-trivial amount of space without a lot of benefits. Or you could run a cron script to rename or remove old indices, but that seems more trouble than it's worth. One other consideration (at least for me) is if the INBOX and personal mail folders are stored in two separate FS's. It would be nice to fuse the two sets of backups under the same namespace, but I don't know how the namespace prefix matching works and whether you can define hierarchical namespaces like namespace { prefix = backup/inbox location = mbox:/path/to/inbox-snapdir/%u ... } namespace { prefix = backup/mail location = mbox:/path/to/mail-snapdir/%u ... } The above can also be accomplished with a synthetic filesystem. Joseph Tam From fxmulder at gmail.com Fri Apr 6 04:16:10 2012 From: fxmulder at gmail.com (James Devine) Date: Thu, 5 Apr 2012 19:16:10 -0600 Subject: [Dovecot] distributed mdbox In-Reply-To: <1ff190f691892c01a980927422f38753@mail.airstreamcomm.net> References: <1ff190f691892c01a980927422f38753@mail.airstreamcomm.net> Message-ID: On Fri, Mar 23, 2012 at 7:39 AM, wrote: > On Wed, 21 Mar 2012 09:56:12 -0600, James Devine > wrote: > > Anyone know how to setup dovecot with mdbox so that it can be used > through > > shared storage from multiple hosts? I've setup a gluster volume and am > > sharing it between 2 test clients. I'm using postfix/dovecot LDA for > > delivery and I'm using postal to send mail between 40 users. In doing > > this, I'm seeing these errors in the logs > > > > Mar 21 09:36:29 test-gluster-client2 dovecot: lda(testuser34): Error: > Fixed > > index file /mnt/testuser34/mdbox/storage/dovecot.map.index: > messages_count > > 272 -> 271 > > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: > Log > > synchronization error at seq=4,offset=3768 for > > /mnt/testuser28/mdbox/storage/dovecot.map.index: Append with UID 516, > but > > next_uid = 517 > > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: > Log > > synchronization error at seq=4,offset=4220 for > > /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update > > for invalid uid=517 > > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: > Log > > synchronization error at seq=4,offset=5088 for > > /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update > > for invalid uid=517 > > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Warning: > > fscking index file /mnt/testuser28/mdbox/storage/dovecot.map.index > > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser34): Warning: > > fscking index file /mnt/testuser34/mdbox/storage/dovecot.map.index > > > > > > This is my dovecot config currently: > > > > jdevine at test-gluster-client2:~> dovecot -n > > # 2.0.13: /etc/dovecot/dovecot.conf > > # OS: Linux 3.0.0-13-server x86_64 Ubuntu 11.10 > > lock_method = dotlock > > mail_fsync = always > > mail_location = mdbox:~/mdbox > > mail_nfs_index = yes > > mail_nfs_storage = yes > > mmap_disable = yes > > passdb { > > driver = pam > > } > > protocols = " imap" > > ssl_cert = > ssl_key = > userdb { > > driver = passwd > > } > > I was able to get dovecot working across a gluster cluster a few weeks ago > and it worked just fine. I would recommend using the native gluster mount > option (need to install gluster software on clients), and using distributed > replicated as your replication mechanism. If you're running two gluster > servers you should have a replica count of two with distributed replicated. > You should test first to make sure you can create a file in both mounts > and see it from every mount point in the cluster, as well as interact with > it. It's also very important to make sure your servers are running with > synchronized clocks from an NTP server. Very bad things happen to a > (dovecot or gluster) cluster out of sync with NTP. > > What storage method are you using? I'm able to produce errors within seconds of starting postal with more than one thread From thomasvoneyben at gmail.com Fri Apr 6 12:52:34 2012 From: thomasvoneyben at gmail.com (Thomas von Eyben) Date: Fri, 6 Apr 2012 11:52:34 +0200 Subject: [Dovecot] Outlook (2010) -> Dovecot (IMAP) >10x slower with high network load and many folders Message-ID: Hi, I am seeing a >10x as slow performance when trying to complete a "send/receive" from an Outlook 2010 client to Dovecot via IMAP, but only when the LAN is fully loaded with other traffic, EG file copying. It seems the problem is when outlook is trying to identify folders that have changed since last "send/receive" thus traversing the hierachy. Observations: - Apple's Mail.app does not have problems when exposed to the same environment. - Eliminating Outlook is not an option. - The tests have been performed during the easter vacation, thus almost no other clients are using the network/mailserver. Under normal load there are 2-300 users using the network. Description of the environment: Server sw: Mac OS X Server 10.6.8 running dovecot: 1.1.20apple0.5 (OS = fully updated 10.6.x) Server hw: Xserve Quadcore intel Xeon 2.26 GHz 12 GB RAM - Never having a cpu load > 20%, mail is stored on a Promise vtrack RAID connected via Fibre Channel Client: Windows 7, Outlook 2010 The client have maybe 50 folders (and each folder about a handfull of subfolders) the size of the mailbox is around 3-5 GB. Test results: CLIENT-1 is having the problems when CLIENT-2 is using all the (100Mbps) bandwidth eg. copying files to MAIL-SRV. If I move CLIENT-1 to CLIENT-3 then almost all the delay is gone. NB.: I have not (yet) tested if the problem also exists when CLIENT-2 generates traffic to MAIL-SRV as opposed to OTHER-SRV (but I am expecting the same problems). When dumping the traffic on CLIENT-1 (with Wireshark) one thing is catching my eyes for the 'bad' case: There is a very long delay after each "REQUEST IDLE" until the next "REQUEST DONE" ca. 0,3 seconds. Comparing to the 'good' setup the pause at the same place in the communication is less than 1 ms ! So why this delay? Where/how shall I continue my debugging? - Run dtruss/dtrace scripts on the server? - Get dovecot to output more debug info (I guess it's doing it's best so no problems will be seen there?)? Are there any dovecot configurations that can be altered to workaround [what I think is a limitation in Outlook's IMAP implementation]? The current dovecot configuration is Apple's defaults (+POP3 disabled)) I am seing one warning from dovecotd -n though: - But I see the same warning on a clean installed not-yet-configured OS X Server so I guess it's not 'that' bad: Warning: fd limit 256 is lower than what Dovecot can use under full load (more than 456). Either grow the limit or change login_max_processes_count and max_mail_processes settings Physical setup: +---------------+ ?+------------------------------+ ? ?+-----------------+ | CLIENT-1 ? ? ?| ?| CLIENT-2 ? ? ? ? ? ? ? ? ? ? | ? ?| CLIENT-3 ? ? ? ?| | Outlook "bad" | ?| Traffic generator, eg. Samba | ? ?| Outlook ?"good" | +---------------+ ?+------------------------------+ ? ?+-----------------+ ? ? ? ? | ? ? ? ? ? ? ? ? ? ? ? | ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| +-------------------------------------------------+ ? ? ? ? ? ? | | ? ? ? ? ? 100Mbps switch (clients) ? ? ? ? ? ? ?| ? ? ? ? ? ? | +-------------------------------------------------+ ? ? ? ? ? ? | ? ? ? ? ? ? ? ? ? ? ?| ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? | +------------------------------------------------------------------------+ | ? ? ? ? ? ? ? ? ? ? 1Gbps switch (for servers) ? ? ? ? ? ? ? ? ? ? ? ? | +------------------------------------------------------------------------+ ? ? ? ? ? ? ? | ? ? ? ? ? ? ? ? ? ? ? ? | +-----------------------------+ ? ?+-----------+ | MAIL-SRV: Dovecot, AFP, SMB | ? ?| OTHER-SRV | +-----------------------------+ ? ?+-----------+ Wireshark dump - IMAP communication: |Time ? ? | 10.211.55.3 ? ? ? ? ? ? ? ? ? ? ? ? ? | | ? ? ? ? | ? ? ? ? ? ? ? ? ? | 10.0.0.10 ? ? ? ? | |9.851 ? ?| ? ? ? ? Response: gatq OK L ? ? ? ? ? |IMAP: Response: gatq OK Logged in. | ? ? ? ? |(49433) ?<------------------ ?(143) ? ?| |9.852 ? ?| ? ? ? ? Request: o47u SELEC ? ? ? ? ? |IMAP: Request: o47u SELECT "1_GROUPS" | ? ? ? ? |(49433) ?------------------> ?(143) ? ?| |9.853 ? ?| ? ? ? ? Response: * FLAGS ( ? ? ? ? ? |IMAP: Response: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) | ? ? ? ? |(49433) ?<------------------ ?(143) ? ?| |9.854 ? ?| ? ? ? ? Request: 3y4b NOOP ? ? ? ? ? ?|IMAP: Request: 3y4b NOOP | ? ? ? ? |(49433) ?------------------> ?(143) ? ?| |9.854 ? ?| ? ? ? ? Response: 3y4b OK N ? ? ? ? ? |IMAP: Response: 3y4b OK NOOP completed. | ? ? ? ? |(49433) ?<------------------ ?(143) ? ?| |9.855 ? ?| ? ? ? ? Request: 4vlj IDLE ? ? ? ? ? ?|IMAP: Request: 4vlj IDLE | ? ? ? ? |(49433) ?------------------> ?(143) ? ?| |9.856 ? ?| ? ? ? ? Response: + idling ? ? ? ? ? ?|IMAP: Response: + idling | ? ? ? ? |(49433) ?<------------------ ?(143) ? ?| |10.108 ? | ? ? ? ? Request: DONE ? ? ? ? ? ? ? ? |IMAP: Request: DONE | ? ? ? ? |(49433) ?------------------> ?(143) ? ?| |10.108 ? | ? ? ? ? Response: 4vlj OK I ? ? ? ? ? |IMAP: Response: 4vlj OK Idle completed. | ? ? ? ? |(49433) ?<------------------ ?(143) ? ?| |10.108 ? | ? ? ? ? Request: wh89 SELEC ? ? ? ? ? |IMAP: Request: wh89 SELECT "1_GROUPS.Adm" BR Thomas von Eyben From adrian.minta at gmail.com Fri Apr 6 13:23:32 2012 From: adrian.minta at gmail.com (Adrian Minta) Date: Fri, 06 Apr 2012 13:23:32 +0300 Subject: [Dovecot] Outlook (2010) -> Dovecot (IMAP) >10x slower with high network load and many folders In-Reply-To: References: Message-ID: <4F7EC424.3010504@gmail.com> It seems that you have packets lost in the network. MAC and Windows have different network stacks so this may count for the different behavior. -- Best regards, Adrian Minta From xnasx at yandex.ru Fri Apr 6 14:21:53 2012 From: xnasx at yandex.ru (Victor) Date: Fri, 06 Apr 2012 15:21:53 +0400 Subject: [Dovecot] Username bug in '%%' variables selects In-Reply-To: <10301333633892@web94.yandex.ru> References: <10301333633892@web94.yandex.ru> Message-ID: <915031333711313@web74.yandex.ru> I solved this issue by changing a namespace to native type. namespace { separator = / prefix = inbox = yes } namespace { separator = . prefix = INBOX. inbox = no hidden = yes list = no alias_for = } namespace { type = shared separator = / prefix = shared/%%u/ list = yes subscriptions = no hidden = no location = maildir:%%h:INDEX=%h/shared/%%u } It did not turn out to make the transparent migration. :( Outlooks 2007 reloaded all scheme and frozen. It's seems a bug in Dovecot. From busseniu at in.tum.de Fri Apr 6 14:51:14 2012 From: busseniu at in.tum.de (=?UTF-8?B?Q2hyaXN0b3BoIEJ1w59lbml1cw==?=) Date: Fri, 06 Apr 2012 13:51:14 +0200 Subject: [Dovecot] Courier to Dovecot migration script issue In-Reply-To: <331561333524967@web30.yandex.ru> References: <125901333456391@web125.yandex.ru> <9BBB03B5-4742-403B-8720-FBEFB554266D@iki.fi> <331561333524967@web30.yandex.ru> Message-ID: <4F7ED8B2.6030809@in.tum.de> Hi, on 04/04/2012 09:36 AM, xnasx wrote: > 'uidvalidity' are the same for both servers, but UIDs of mesages are not: > > Courier: > d fetch 1:* uid > * 1 FETCH (UID 64295) > * 2 FETCH (UID 64296) > * 3 FETCH (UID 64297) > > Dovecot: > > * 1 FETCH (UID 64296) > * 2 FETCH (UID 64297) > * 3 FETCH (UID 64298) Does your Courier directory include a courierpop3sizelist file? The migration script places higher importance on the POP3 order than on IMAP UIDs. Also, in order to see if the old 64295 corresponds to the new 64298, you can use "fetch 1:* (uid rfc822.size body.peek[header.fields (message-id)])". Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From pw at wk-serv.de Fri Apr 6 16:58:18 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Fri, 06 Apr 2012 15:58:18 +0200 Subject: [Dovecot] Director (was: Hints for a NFS-Setup) In-Reply-To: <4F75A929.9090902@mobilia.it> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> Message-ID: <4F7EF67A.8060001@wk-serv.de> Hi again, I tried to setup a test invironemnt like this: MTA --(lmtp)--\ /--(lmtp)--> backend1 --\ -- director -- -- NFS MTA --(lmtp)--/ \--(lmtp)--> backend2 --/ IMAP-User --> frontend1 --\ /--(imap)--> backend1 --\ -- director -- -- NFS IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/ but now I'm very confused. Is it actually possible to setup a host (or two) as a director only or will I have to enable the director service on each frontend and MTA? Is anyone using a similar setup and could post his configuration for the different hosts? Happy Easter! Patrick From fxmulder at gmail.com Fri Apr 6 18:39:43 2012 From: fxmulder at gmail.com (James Devine) Date: Fri, 6 Apr 2012 09:39:43 -0600 Subject: [Dovecot] distributed mdbox In-Reply-To: References: <1ff190f691892c01a980927422f38753@mail.airstreamcomm.net> Message-ID: As it turns out I can duplicate this problem with a single dovecot server and a single gluster server using mdbox, so maybe not caching? This being the case I don't think director would help On Thu, Apr 5, 2012 at 7:16 PM, James Devine wrote: > > > On Fri, Mar 23, 2012 at 7:39 AM, wrote: > >> On Wed, 21 Mar 2012 09:56:12 -0600, James Devine >> wrote: >> > Anyone know how to setup dovecot with mdbox so that it can be used >> through >> > shared storage from multiple hosts? I've setup a gluster volume and am >> > sharing it between 2 test clients. I'm using postfix/dovecot LDA for >> > delivery and I'm using postal to send mail between 40 users. In doing >> > this, I'm seeing these errors in the logs >> > >> > Mar 21 09:36:29 test-gluster-client2 dovecot: lda(testuser34): Error: >> Fixed >> > index file /mnt/testuser34/mdbox/storage/dovecot.map.index: >> messages_count >> > 272 -> 271 >> > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: >> Log >> > synchronization error at seq=4,offset=3768 for >> > /mnt/testuser28/mdbox/storage/dovecot.map.index: Append with UID 516, >> but >> > next_uid = 517 >> > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: >> Log >> > synchronization error at seq=4,offset=4220 for >> > /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update >> > for invalid uid=517 >> > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: >> Log >> > synchronization error at seq=4,offset=5088 for >> > /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update >> > for invalid uid=517 >> > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Warning: >> > fscking index file /mnt/testuser28/mdbox/storage/dovecot.map.index >> > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser34): Warning: >> > fscking index file /mnt/testuser34/mdbox/storage/dovecot.map.index >> > >> > >> > This is my dovecot config currently: >> > >> > jdevine at test-gluster-client2:~> dovecot -n >> > # 2.0.13: /etc/dovecot/dovecot.conf >> > # OS: Linux 3.0.0-13-server x86_64 Ubuntu 11.10 >> > lock_method = dotlock >> > mail_fsync = always >> > mail_location = mdbox:~/mdbox >> > mail_nfs_index = yes >> > mail_nfs_storage = yes >> > mmap_disable = yes >> > passdb { >> > driver = pam >> > } >> > protocols = " imap" >> > ssl_cert = > > ssl_key = > > userdb { >> > driver = passwd >> > } >> >> I was able to get dovecot working across a gluster cluster a few weeks ago >> and it worked just fine. I would recommend using the native gluster mount >> option (need to install gluster software on clients), and using >> distributed >> replicated as your replication mechanism. If you're running two gluster >> servers you should have a replica count of two with distributed >> replicated. >> You should test first to make sure you can create a file in both mounts >> and see it from every mount point in the cluster, as well as interact with >> it. It's also very important to make sure your servers are running with >> synchronized clocks from an NTP server. Very bad things happen to a >> (dovecot or gluster) cluster out of sync with NTP. >> >> What storage method are you using? I'm able to produce errors within > seconds of starting postal with more than one thread From stephan at rename-it.nl Fri Apr 6 20:08:17 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 06 Apr 2012 19:08:17 +0200 Subject: [Dovecot] Problem with polish characters and sieve fileinto In-Reply-To: <4F7DA264.8020004@rename-it.nl> References: <4F7D9E29.3050802@eurocar.pl> <4F7DA264.8020004@rename-it.nl> Message-ID: <4F7F2301.4020702@rename-it.nl> On 4/5/2012 3:47 PM, Stephan Bosch wrote: > Op 4/5/2012 3:29 PM, l.chelchowski at eurocar.pl schreef: >> Hi, >> >> I have problems with sieve fileinto and polish characters in folder >> names: >> # rule:[??????????????????] >> elsif anyof (header :contains "Subject" "??????????????????") >> { >> fileinto "INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-"; >> } > > Looks like your Sieve GUI uses the wrong Unicode format. The above is > mUTF7 as used by IMAP. However, Sieve expects UTF-8. So it should use > this instead: > > fileinto "INBOX/??????????????????"; Added item to the troubleshooting wiki page: http://master.wiki2.dovecot.org/Pigeonhole/Sieve/Troubleshooting#Mailbox_Names_with_non-ASCII_Characters_Cause_Problems By the looks of it, you are also using the RoundCube SieveRules plugin. The solution for RoundCube mentioned in the wiki should solve it for you. Regards, Stephan. From dmiller at amfes.com Fri Apr 6 21:53:31 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Fri, 06 Apr 2012 11:53:31 -0700 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: <4F7D972A.8050406@Media-Brokers.com> References: <4F7D972A.8050406@Media-Brokers.com> Message-ID: On 4/5/2012 5:59 AM, Charles Marcus wrote: > On 2012-04-05 4:18 AM, Thomas Leuxner wrote: > > Also with 2.x you may want to use LMTP rather than the LDA Piping. > > > > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP > > I am preparing to convert my main client's postfix_courier-imap setup > to dovecot 2.1, which currently just uses the postfix virtual delivery > agent... > > Does anyone know if the use of LMTP (or even the dovecot LDA) still > loses the x-original-to header that the postfix vda adds and that I > rely heavily on (since I use a lot of aliases), and if it does, is > there any solution to get the original recipient added back in before > final delivery? > > Everything I'm reading says that LMTP is better, but I really do need > this header (or one like it) to be there so I know who the original > recipient was (for filtering and other purposes). > I'm currently using Postfix 2.7, Dovecot 2.1, and the Dovecot LDA. I have a pure virtual user environment stored in LDAP. My messages include X-Original-To and Delivered-To headers. I had difficulty getting the LMTP transport to work previously - I may revisit that. -- Daniel From CMarcus at Media-Brokers.com Fri Apr 6 23:00:19 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 06 Apr 2012 16:00:19 -0400 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: References: <4F7D972A.8050406@Media-Brokers.com> Message-ID: <4F7F4B53.2090700@Media-Brokers.com> On 2012-04-06 2:53 PM, Daniel L. Miller wrote: > I'm currently using Postfix 2.7, Dovecot 2.1, and the Dovecot LDA. I > have a pure virtual user environment stored in LDAP. My messages include > X-Original-To and Delivered-To headers. Well that is great news... at least I'll be able to use the LDA, if not LMTP... Thanks! :) > I had difficulty getting the LMTP transport to work previously - I may > revisit that. If you do, by all means reply back on whether or not the headers are still there... Thanks again, -- Best regards, Charles From kgc at corp.sonic.net Sat Apr 7 02:40:01 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Fri, 6 Apr 2012 16:40:01 -0700 Subject: [Dovecot] POP3 dele to Trash? Message-ID: <20120406234001.GG89906@corp.sonic.net> To simplify recovery of accidentally deleted messages we'd like to send all messages that are deleted by pop clients to their Trash folder. (Which is auto expired already.) This allows a POP client who deletes an important email to restore it themselves by logging into our webmail client and move the lost message from Trash to the Inbox where it would be fetched by their client again. Has anyone already done this? Should this be possible via a plugin? I see the deleted-to-trash imap plugin. We are using Maildir if it makes a difference. -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From kgc at corp.sonic.net Sat Apr 7 03:10:26 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Fri, 06 Apr 2012 17:10:26 -0700 Subject: [Dovecot] POP3 dele to Trash? In-Reply-To: <20120406234001.GG89906@corp.sonic.net> References: <20120406234001.GG89906@corp.sonic.net> Message-ID: <4F7F85F2.3000909@corp.sonic.net> On 04/06/12 16:40, Kelsey Cummings wrote: > Has anyone already done this? Should this be possible via a plugin? > I see the deleted-to-trash imap plugin. We are using Maildir if it > makes a difference. Of course, this is exactly what the Lazy Expunge plugin does, isn't it? -K From wgillespie+dovecot at es2eng.com Sat Apr 7 04:16:14 2012 From: wgillespie+dovecot at es2eng.com (Willie Gillespie) Date: Fri, 06 Apr 2012 19:16:14 -0600 Subject: [Dovecot] Outlook (2010) -> Dovecot (IMAP) >10x slower with high network load and many folders In-Reply-To: References: Message-ID: <4F7F955E.2050508@es2eng.com> On 4/6/2012 3:52 AM, Thomas von Eyben wrote: > Test results: > CLIENT-1 is having the problems when CLIENT-2 is using all the > (100Mbps) bandwidth eg. copying files to MAIL-SRV. > If I move CLIENT-1 to CLIENT-3 then almost all the delay is gone. > NB.: I have not (yet) tested if the problem also exists when CLIENT-2 > generates traffic to MAIL-SRV as opposed to OTHER-SRV (but I am > expecting the same problems). So the link between your 100 Mbps switch and the 1 Gbps switch is saturated by CLIENT-2, so CLIENT-1 is just getting the leftovers? Since CLIENT-3 doesn't go through that 100 Mbps switch, it obviously doesn't see that issue. From plmb21 at gmail.com Sat Apr 7 06:48:43 2012 From: plmb21 at gmail.com (PL MB) Date: Fri, 6 Apr 2012 20:48:43 -0700 Subject: [Dovecot] Setting ACL for master user after login Message-ID: Hello, I'd like to log in to normal user accounts as a master user but retain the normal users' ACLs. The Master Users page on the Dovecot 1.x wiki (1) says that I can set the master user's ACLs in a postlogin script. The documentation for master users on the 2.x wiki (2) no longer has any statements about master user ACLs. Has something important in this regard changed? Can I no longer override the ACLs in a postlogin script? Thanks in advance. (1) http://wiki.dovecot.org/Authentication/MasterUsers (2) http://wiki2.dovecot.org/Authentication/MasterUsers From andy at xecu.net Sat Apr 7 10:13:42 2012 From: andy at xecu.net (Andy Dills) Date: Sat, 7 Apr 2012 03:13:42 -0400 (EDT) Subject: [Dovecot] Director pop-login and imap-login processes exiting on signal 11 Message-ID: <20120407030811.I46246@shell.xecu.net> We recently upgraded our cluster to 2.1.3, to enable director proxying. Everything appears to be working fine for the most part; the only odd thing is that I'm seeing a lot of entries in the logs like this: Apr 7 02:18:05 mail-out06 dovecot: pop3-login: Fatal: master: service(pop3-login): child 75029 killed with signal 11 (core not dumped - set service pop3-login { drop_priv_before_exec=yes }) This is on the proxy side, not that backend side. When I try to get a dump out of it, and add drop_prive_before_exec and chroot= to the pop3-login statement on the proxy, I keep running into permissions errors with the various service sockets. Any suggestions? Thanks, Andy --- Andy Dills Xecunet, Inc. www.xecu.net 301-682-9972 --- From stan at hardwarefreak.com Sat Apr 7 13:19:46 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 07 Apr 2012 05:19:46 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: References: Message-ID: <4F8014C2.3010606@hardwarefreak.com> On 4/5/2012 3:02 PM, Emmanuel Noobadmin wrote: Hi Emmanuel, > I'm trying to improve the setup of our Dovecot/Exim mail servers to > handle the increasingly huge accounts (everybody thinks it's like > infinitely growing storage like gmail and stores everything forever in > their email accounts) by changing from Maildir to mdbox, and to take > advantage of offloading older emails to alternative networked storage > nodes. I'll assume "networked storage nodes" means NFS, not FC/iSCSI SAN, in which case you'd have said "SAN". > The question now is whether having a single large server or will a > number of 1U servers with the same total capacity be better? Less complexity and cost is always better. CPU throughput isn't a factor in mail workloads--it's all about IO latency. A 1U NFS server with 12 drive JBOD is faster, cheaper, easier to setup and manage, sucks less juice and dissipates less heat than 4 1U servers each w/ 4 drives. I don't recall seeing your user load or IOPS requirements so I'm making some educated guesses WRT your required performance and total storage. I came up with the following system that should be close to suitable, for ~$10k USD. The 4 node system runs ~$12k USD. At $2k this isn't substantially higher. But when we double the storage of each architecture we're at ~$19k, vs ~$26k for an 8 node cluster, a difference of ~$7k. That's $1k shy of another 12 disk JBOD. Since CPU is nearly irrelevant for a mail workload, you can see it's much cheaper to scale capacity and IOPS with a single node w/fat storage than with skinny nodes w/thin storage. Ok, so here's the baseline config I threw together: http://h10010.www1.hp.com/wwpc/us/en/sm/WF06b/15351-15351-3328412-241644-3328421-4091396-4158470-4158440.html?dnr=1 8 cores is plenty, 2 boot drives mirrored on B110i, 16GB (4x4GB) http://www.lsi.com/products/storagecomponents/Pages/LSISAS9205-8e.aspx http://h10010.www1.hp.com/wwpc/us/en/sm/WF06b/12169-304616-3930445-3930445-3930445-3954787-4021626-4021628.html?dnr=1 w/ 12 2TB 7.2K SATA drives, configured as md concat+RAID1 pairs with 12 allocation groups, 12TB usable. Format the md device with the defaults: $ mkfs.xfs /dev/md0 Mount with inode64. No XFS stripe alignment to monkey with. No md chunk size or anything else to worry about. XFS' allocation group design is pure elegance here. If 12 TB isn't sufficient, or if you need more space later, you can daisy chain up to 3 additional D2600 JBODs for ~$8500 USD each, just add cables. This quadruples IOPS, throughput, and capacity--96TB total, 48TB net. Simply create 6 more mdraid1 devices and grow the linear array with them. Then do an xfs_growfs to bring the extra 12TB of free space into the filesystem. If you're budget conscious and/or simply prefer quality inexpensive whitebox/DIY type gear, as I do, you can get 24 x 2TB drives in one JBOD chassis for $7400 USD. That twice the drives, capacity, IOPS, for ~$2500 less than the HP JBOD. And unlike the HP 'enterprise SATA' drives, the 2TB WD Black series have a 5 year warranty, and work great with mdraid. Chassis and drives at Newegg: http://www.newegg.com/Product/Product.aspx?Item=N82E16816133047 http://www.newegg.com/Product/Product.aspx?Item=N82E16822136792 You can daisy chain 3 of these off one HBA SFF8088 port, 6 total on our LSI 9205-8e above, for a total of 144 2TB drives, 72 effective spindles in our concat+RAID1 setup, 144TB net space. > Will be > using RAID 1 pairs, likely XFS based on reading Hoeppner's > recommendation on this and the mdadm list. To be clear, the XFS configuration I recommend/promote for mailbox storage is very specific and layered. The layers must all be used together to get the performance. These layers consist of using multiple hardware or software RAID1 pairs and concatenating them with an md linear array. You then format that md device with the XFS defaults, or a specific agcount if you know how to precisely tune AG layout based on disk size and your anticipated concurrency level of writers. Putting XFS on a singe RAID1 pair, as you seem to be describing above for the multiple "thin" node case, and hitting one node with parallel writes to multiple user mail dirs, you'll get less performance than EXT3/4 on that mirror pair--possibly less than half, depending on the size of the disks and thus the number of AGs created. The 'secret' to XFS performance with this workload is concatenation of spindles. Without it you can't spread the AGs--thus directories, thus parallel file writes--horizontally across the spindles--and this is the key. By spreading AGs 'horizontally' across the disks in a concat, instead of 'vertically' down a striped array, you accomplish two important things: 1. You dramatically reduce disk head seeking by using the concat array. With XFS on a RAID10 array of 24 2TB disks you end up with 24 AGs evenly spaced vertically down each disk in the array, following the stripe pattern. Each user mailbox is stored in a different directory. Each directory was created in a different AG. So if you have 96 users writing their dovecot index concurrently, you have at worst case a minimum 192 head movements occurring back and forth across the entire platter of each disk, and likely not well optimized by TCQ/NCQ. Why 192 instead of 96? The modification time in the directory metadata must be updated for each index file, among other things. 2. Because we decrease seeks dramatically we also decrease response latency significantly. With the RAID1+concat+XFS we have 12 disks each with only 2 AGs spaced evenly down each platter. We have the same 4 user mail dirs in each AG, but in this case only 8 user mail dirs are contained on each disk instead of portions all 96. With the same 96 concurrent writes to indexes, in this case end up with only 16 seeks per drive--again, one to update each index file and one to update the metadata. Assuming these drives have a max seek rate of 150 which is the average for 7.2k drives, it will take 192/150 = 1.28 seconds for these operations on the RAID10 array. With the concat array it will only take 16/150 = 0.11 seconds. Extrapolating from that demonstrates that the concat array can handle 1.28/0.11 = 11.6*96 = 1,111 concurrent user index updates in the same time as the RAID10 array, just over 10 times more users. Granted, these are rough theoretical numbers--an index plus metadata update isn't always going to cause a seek on every chunk in a stripe, etc. But this does paint a very accurate picture of the differences in mailbox workload disk seek patterns between XFS on concat and RAID10 with the same hardware. In production one should be able to handle at minimum 2x more users, probably many more, with the RAID1+concat+XFS vs RAID10+XFS setup on the same hardware. > Currently, I'm leaning towards multiple small servers because I think > it should be better in terms of performance. This usually isn't the case with mail. It's impossible to split up the user files across the storage nodes in a way that balances block usage on each node and user access to those blocks. Hotspots are inevitable in both categories. You may achieve the same total performance of a single server, maybe slightly surpass it depending on user load, but you end up spending extra money on building resources that are idle most of the time, in the case of CPU and NICs, or under/over utilized, in the case of disk capacity in each node. Switch ports aren't horribly expensive today, but you're still wasting some with the farm setup. > At the very least even if > one node gets jammed up, the rest should still be able serve up the > emails for other accounts that is unless Dovecot will get locked up by > that jammed transaction. Some host failure redundancy is about all you'd gain from the farm setup. Dovecot shouldn't barf due to one NFS node being down, only hiccup. I.e. only imap process accessing files on the downed node would have trouble. > Also, I could possibly arrange them in a sort > of network raid 1 to gain redundancy over single machine failure. Now you're sounding like Charles Marcus, but worse. ;) Stay where you are, and brush your hair away from your forehead. I'm coming over with my branding iron that says "K.I.S.S" > Would I be correct in these or do actual experiences say otherwise? Oracles on Mount Interweb profess that 2^5 nodes wide scale out is the holy grail. IBM's mainframe evangelists tell us to put 5 million mail users on a SystemZ with hundreds of Linux VMs. I think bliss for most of us is found somewhere in the middle. -- Stan From centos.admin at gmail.com Sat Apr 7 17:43:09 2012 From: centos.admin at gmail.com (Emmanuel Noobadmin) Date: Sat, 7 Apr 2012 22:43:09 +0800 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F8014C2.3010606@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> Message-ID: On 4/7/12, Stan Hoeppner wrote: Firstly, thanks for the comprehensive reply. :) > I'll assume "networked storage nodes" means NFS, not FC/iSCSI SAN, in > which case you'd have said "SAN". I haven't decided on that but it would either be NFS or iSCSI over Gigabit. I don't exactly get a big budget for this. iSCSI because I planned to do md/mpath over two separate switches so that if one switch explodes, the email service would still work. > Less complexity and cost is always better. CPU throughput isn't a > factor in mail workloads--it's all about IO latency. A 1U NFS server > with 12 drive JBOD is faster, cheaper, easier to setup and manage, sucks > less juice and dissipates less heat than 4 1U servers each w/ 4 drives. My worry is that if that one server dies, everything is dead. With at least a pair of servers, I could keep it running, or if necessary, restore the accounts on the dead servers from backup, make some config changes and have everything back running while waiting for replacement hardware. > I don't recall seeing your user load or IOPS requirements so I'm making > some educated guesses WRT your required performance and total storage. I'm embarrassed to admit I don't have hard numbers on the user load except the rapidly dwindling disk space count and the fact when the web-based mail application try to list and check disk quota, it can bring the servers to a crawl. My lame excuse is that I'm just the web dev who got caught holding the server admin potato. > is nearly irrelevant for a mail workload, you can see it's much cheaper > to scale capacity and IOPS with a single node w/fat storage than with > skinny nodes w/thin storage. Ok, so here's the baseline config I threw > together: One of my concern is that heavy IO on the same server slow the overall performance even though the theoretical IOPS of the total drives are the same on 1 and on X servers. Right now, the servers are usually screeching to a halt, to the point of even locking out SSH access due to IOWait sending the load in top to triple digits. > Some host failure redundancy is about all you'd gain from the farm > setup. Dovecot shouldn't barf due to one NFS node being down, only > hiccup. I.e. only imap process accessing files on the downed node would > have trouble. But if I only have one big storage node and that went down, Dovecot would barf wouldn't it? Or would the mdbox format mean Dovecot would still use the local storage, just that users can't access the offloaded messages? >> Also, I could possibly arrange them in a sort >> of network raid 1 to gain redundancy over single machine failure. > > Now you're sounding like Charles Marcus, but worse. ;) Stay where you > are, and brush your hair away from your forehead. I'm coming over with > my branding iron that says "K.I.S.S" Lol, I have no idea who Charles is, but I always feel safer if there was some kind of backup. Especially since I don't have the time to dedicate myself to server administration, by the time I notice something is bad, it might be too late for anything but the backup. Of course management and clients don't agree with me since backup/redundancy costs money. :) From dmiller at amfes.com Sat Apr 7 21:06:48 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Sat, 07 Apr 2012 11:06:48 -0700 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: <4F7F4B53.2090700@Media-Brokers.com> References: <4F7D972A.8050406@Media-Brokers.com> <4F7F4B53.2090700@Media-Brokers.com> Message-ID: On 4/6/2012 1:00 PM, Charles Marcus wrote: > On 2012-04-06 2:53 PM, Daniel L. Miller wrote: >> I'm currently using Postfix 2.7, Dovecot 2.1, and the Dovecot LDA. I >> have a pure virtual user environment stored in LDAP. My messages include >> X-Original-To and Delivered-To headers. > > Well that is great news... at least I'll be able to use the LDA, if > not LMTP... > > Thanks! :) > >> I had difficulty getting the LMTP transport to work previously - I may >> revisit that. > > If you do, by all means reply back on whether or not the headers are > still there... > > Thanks again, > From the documentation... http://www.postfix.org/virtual.8.html The*virtual*(8) delivery agent prepends a "*From* /sender/ /time/*_*/stamp/" envelope header to each message, prepends a *Delivered-To:* message header with the envelope recipient address, prepends an*X-Original-To:* header with the recip- ient address as given to Postfix, prepends a*Return-Path:* message header with the envelope sender address, prepends a> character to lines beginning with "*From* ", and appends an empty line. Using the Postfix pipe agent, which is what is used with the Dovecot LDA, http://www.postfix.org/pipe.8.html *flags=BDFORXhqu.*> (optional) Optional message processing flags. By default, a message is copied unchanged. *B* Append a blank line at the end of each mes- sage. This is required by some mail user agents that recognize "*From* " lines only when preceded by a blank line. *D* Prepend a "*Delivered-To:* /recipient/" message header with the envelope recipient address. Note: for this to work, the/transport/*_desti-* *nation_recipient_limit* must be 1 (see SIN- GLE-RECIPIENT DELIVERY above for details). The*D* flag also enforces loop detection (Postfix 2.5 and later): if a message already contains a*Delivered-To:* header with the same recipient address, then the message is returned as undeliverable. The address comparison is case insensitive. This feature is available as of Postfix 2.0. *F* Prepend a "*From* /sender time/*_*/stamp/" envelope header to the message content. This is expected by, for example,*UUCP* software. *O* Prepend an "*X-Original-To:* /recipient/" mes- sage header with the recipient address as given to Postfix. Note: for this to work, the*/transport/_destination_recipient_limit * must be 1 (see SINGLE-RECIPIENT DELIVERY above for details). Unfortunately, the docs for the ltmp agent http://www.postfix.org/lmtp.8.html don't say anything about adding these headers. I tried asking on the Postfix list - didn't get much of an answer. -- Daniel From jerry at seibercom.net Sat Apr 7 21:30:38 2012 From: jerry at seibercom.net (Jerry) Date: Sat, 7 Apr 2012 14:30:38 -0400 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: References: <4F7D972A.8050406@Media-Brokers.com> <4F7F4B53.2090700@Media-Brokers.com> Message-ID: <20120407143038.5252acf9@scorpio> On Sat, 07 Apr 2012 11:06:48 -0700 Daniel L. Miller articulated: > Unfortunately, the docs for the ltmp agent > http://www.postfix.org/lmtp.8.html don't say anything about adding > these headers. I tried asking on the Postfix list - didn't get much > of an answer. I may be wrong; however, from what I have been able to understand in regards to the Postfix documentation, if it does not explicitly claim to have a feature, then that feature is not available. In other words, if it doesn't state it can do it, it can't. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From thomasvoneyben at gmail.com Sat Apr 7 23:13:15 2012 From: thomasvoneyben at gmail.com (Thomas von Eyben) Date: Sat, 7 Apr 2012 22:13:15 +0200 Subject: [Dovecot] Outlook (2010) -> Dovecot (IMAP) >10x slower with high network load and many folders In-Reply-To: <4F7F955E.2050508@es2eng.com> References: <4F7F955E.2050508@es2eng.com> Message-ID: On Sat, Apr 7, 2012 at 3:16 AM, Willie Gillespie wrote: > On 4/6/2012 3:52 AM, Thomas von Eyben wrote: >> >> Test results: >> CLIENT-1 is having the problems when CLIENT-2 is using all the >> (100Mbps) bandwidth eg. copying files to MAIL-SRV. >> If I move CLIENT-1 to CLIENT-3 then almost all the delay is gone. >> NB.: I have not (yet) tested if the problem also exists when CLIENT-2 >> generates traffic to MAIL-SRV as opposed to OTHER-SRV (but I am >> expecting the same problems). > > > So the link between your 100 Mbps switch and the 1 Gbps switch is saturated > by CLIENT-2, so CLIENT-1 is just getting the leftovers? > > Since CLIENT-3 doesn't go through that 100 Mbps switch, it obviously doesn't > see that issue. Yes - that's my current "workaround" (perhaps also solution), I'm wondering if the performance is really expected to be _so_ bad when other users are utilizing the LAN. (You seem to indicate that what I am observing is expected and is "just" caused by [un-intended] semi-bad behavior from other users?) BR TvE From dovecot at r.paypc.com Sat Apr 7 23:45:08 2012 From: dovecot at r.paypc.com (Robin) Date: Sat, 07 Apr 2012 13:45:08 -0700 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F8014C2.3010606@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> Message-ID: <4F80A754.4000703@r.paypc.com> > Putting XFS on a singe RAID1 pair, as you seem to be describing above > for the multiple "thin" node case, and hitting one node with parallel > writes to multiple user mail dirs, you'll get less performance than > EXT3/4 on that mirror pair--possibly less than half, depending on the > size of the disks and thus the number of AGs created. The 'secret' to > XFS performance with this workload is concatenation of spindles. > Without it you can't spread the AGs--thus directories, thus parallel > file writes--horizontally across the spindles--and this is the key. By > spreading AGs 'horizontally' across the disks in a concat, instead of > 'vertically' down a striped array, you accomplish two important things: > > 1. You dramatically reduce disk head seeking by using the concat array. > With XFS on a RAID10 array of 24 2TB disks you end up with 24 AGs > evenly spaced vertically down each disk in the array, following the > stripe pattern. Each user mailbox is stored in a different directory. > Each directory was created in a different AG. So if you have 96 users > writing their dovecot index concurrently, you have at worst case a > minimum 192 head movements occurring back and forth across the entire > platter of each disk, and likely not well optimized by TCQ/NCQ. Why 192 > instead of 96? The modification time in the directory metadata must be > updated for each index file, among other things. Does the XFS allocator automatically distribute AGs in this way even when disk usage is extremely light, i.e, a freshly formatted system with user directories initially created, and then the actual mailbox contents copied into them? If this is indeed the case, then what you describe is a wondrous revelation, since you're scaling out the number of simultaneous metadata reads+writes/second as you add RAID1 pairs, if my understanding of this is correct. I'm assuming of course, but should look at the code, that metadata locks imposed by the filesystem "distribute" as the number of pairs increase - if it's all just one Big Lock, then that wouldn't be the case. Forgive my laziness, as I could just experiment and take a look at the on-disk structures myself, but I don't have four empty drives handy to experiment. The bandwidth improvements due to striping (RAID0/5/6 style) are no help for metadata-intensive IO loads, and probably of little value for even mdbox loads too, I suspect, unless the mdbox max size is set to something pretty large, no? Have you tried other filesystems and seen if they distribute metadata in a similarly efficient and scalable manner across concatenated drive sets? Is there ANY point to using striping at all, a la "RAID10" in this? I'd have thought just making as many RAID1 pairs out of your drives as possible would be the ideal strategy - is this not the case? =R= From stan at hardwarefreak.com Sun Apr 8 03:46:20 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 07 Apr 2012 19:46:20 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F80A754.4000703@r.paypc.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F80A754.4000703@r.paypc.com> Message-ID: <4F80DFDC.2030703@hardwarefreak.com> On 4/7/2012 3:45 PM, Robin wrote: > >> Putting XFS on a singe RAID1 pair, as you seem to be describing above >> for the multiple "thin" node case, and hitting one node with parallel >> writes to multiple user mail dirs, you'll get less performance than >> EXT3/4 on that mirror pair--possibly less than half, depending on the >> size of the disks and thus the number of AGs created. The 'secret' to >> XFS performance with this workload is concatenation of spindles. >> Without it you can't spread the AGs--thus directories, thus parallel >> file writes--horizontally across the spindles--and this is the key. By >> spreading AGs 'horizontally' across the disks in a concat, instead of >> 'vertically' down a striped array, you accomplish two important things: >> >> 1. You dramatically reduce disk head seeking by using the concat array. >> With XFS on a RAID10 array of 24 2TB disks you end up with 24 AGs >> evenly spaced vertically down each disk in the array, following the >> stripe pattern. Each user mailbox is stored in a different directory. >> Each directory was created in a different AG. So if you have 96 users >> writing their dovecot index concurrently, you have at worst case a >> minimum 192 head movements occurring back and forth across the entire >> platter of each disk, and likely not well optimized by TCQ/NCQ. Why 192 >> instead of 96? The modification time in the directory metadata must be >> updated for each index file, among other things. > > Does the XFS allocator automatically distribute AGs in this way even > when disk usage is extremely light, i.e, a freshly formatted system with > user directories initially created, and then the actual mailbox contents > copied into them? It doesn't distribute AGs. There are a static number created during mkfs.xfs. The inode64 allocator round robins new directory creation across the AGs, and does the same with files created in those directories. Having the directory metadata and file extents in the same AG decreases head movement and thus seek latency for mixed metadata/extent high IOPS workloads. > If this is indeed the case, then what you describe is a wondrous > revelation, since you're scaling out the number of simultaneous metadata > reads+writes/second as you add RAID1 pairs, if my understanding of this > is correct. Correct. And adding more space and IOPS is uncomplicated. No chunk calculations, no restriping of the array. You simply grow the md linear array adding the new disk device. Then grow XFS to add the new free space to the filesystem. AFAIK this can be done infinitely, theoretically. I'm guessing md has a device count limit somewhere. If not your bash line buffer might. ;) > I'm assuming of course, but should look at the code, that > metadata locks imposed by the filesystem "distribute" as the number of > pairs increase - if it's all just one Big Lock, then that wouldn't be > the case. XFS locking is done as minimally as possibly and is insanely fast. I've not come across any reported performance issues relating to it. And yes, any single metadata lock will occur in a single AG on one mirror pair using the concat setup. > Forgive my laziness, as I could just experiment and take a look at the > on-disk structures myself, but I don't have four empty drives handy to > experiment. Don't sweat it. All of this stuff is covered in the XFS Filesystem Structure Guide, exciting reading if you enjoy a root canal while watching snales race: http://xfs.org/docs/xfsdocs-xml-dev/XFS_Filesystem_Structure//tmp/en-US/html/index.html > The bandwidth improvements due to striping (RAID0/5/6 style) are no help > for metadata-intensive IO loads, and probably of little value for even > mdbox loads too, I suspect, unless the mdbox max size is set to > something pretty large, no? The problem with striped parity RAID is not allocation, which takes place in free space and is pretty fast. The problem is the extra read seeks and bandwidth of the RMW cycle when you modify an existing stripe. Updating a single flag in a Dovecot index causes md or the hardware RAID controller to read the entire stripe into buffer space or RAID cache, modify the flag byte, recalculate parity, then write the whole stripe and parity block back out across all the disks. With a linear concat of RAID1 pairs we're simply rewriting a single 4KB filesystem block, maybe only a single 512B sector. I'm at the edge of my knowledge here. I don't know exactly how Timo does the index updates. Regardless of the method, the index update is light years faster with the concat setup as there is no RMW and full stripe writeback as with the RAID5/6 case. > Have you tried other filesystems and seen if they distribute metadata in > a similarly efficient and scalable manner across concatenated drive sets? EXT, any version, does not. ReiserFS does not. Both require disk striping to achieve any parallelism. With concat they both simply start writing at the beginning sectors of the first RAID1 pair and 4 years later maybe reach the last pair as they fill up the volume. ;) JFS has a more advanced allocation strategy that EXT or ReiserFS, not as advanced as XFS. I've never read of a concat example with JFS and I've never tested it. It's all but a dead filesystem at this point anyway, less than 2 dozen commits in 8 years last I checked, and these were simple bug fixes and changes to keep it building on new kernels. If it's not suffering bit rot now I'm sure it will be in the near future. > Is there ANY point to using striping at all, a la "RAID10" in this? I'd > have thought just making as many RAID1 pairs out of your drives as > possible would be the ideal strategy - is this not the case? If you're using XFS, and your workload is overwhelmingly mail, RAID1+concat is the only way to fly, and it flies. If the workload is not mail, say large file streaming writes, then you're limited to 100-200MB/s, a single drive of throughput, as each file is written to a single directory on a single AG on a single disk. For streaming write performance you'll need striping. If you have many concurrent large streaming writes, you'll want to concat multiple striped arrays. -- Stan From stan at hardwarefreak.com Sun Apr 8 21:21:47 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sun, 08 Apr 2012 13:21:47 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: References: <4F8014C2.3010606@hardwarefreak.com> Message-ID: <4F81D73B.9030901@hardwarefreak.com> On 4/7/2012 9:43 AM, Emmanuel Noobadmin wrote: > On 4/7/12, Stan Hoeppner wrote: > > Firstly, thanks for the comprehensive reply. :) > >> I'll assume "networked storage nodes" means NFS, not FC/iSCSI SAN, in >> which case you'd have said "SAN". > > I haven't decided on that but it would either be NFS or iSCSI over > Gigabit. I don't exactly get a big budget for this. iSCSI because I > planned to do md/mpath over two separate switches so that if one > switch explodes, the email service would still work. So it seems you have two courses of action: 1. Identify individual current choke points and add individual systems and storage to eliminate those choke points. 2. Analyze your entire workflow and all systems, identifying all choke points, then design a completely new well integrated storage architecture that solves all current problems and addresses future needs Adding an NFS server and moving infrequently (old) accessed files to alternate storage will alleviate your space problems. But it will probably not fix some of the other problems you mention, such as servers bogging down and becoming unresponsive, as that's not a space issue. The cause of that would likely be an IOPS issue, meaning you don't have enough storage spindles to service requests in a timely manner. >> Less complexity and cost is always better. CPU throughput isn't a >> factor in mail workloads--it's all about IO latency. A 1U NFS server >> with 12 drive JBOD is faster, cheaper, easier to setup and manage, sucks >> less juice and dissipates less heat than 4 1U servers each w/ 4 drives. > > My worry is that if that one server dies, everything is dead. With at > least a pair of servers, I could keep it running, or if necessary, > restore the accounts on the dead servers from backup, make some config > changes and have everything back running while waiting for replacement > hardware. You are a perfect candidate for VMware ESX. The HA feature will do exactly what you want. If one physical node in the cluster dies, HA automatically restarts the dead VMs on other nodes, transparently. Clients will will have to reestablish connections, but everything else will pretty much be intact. Worse case scenario will possibly be a few corrupted mailboxes that were being written when the hardware crashed. A SAN is required for such a setup. I had extensive experience with ESX and HA about 5 years ago and it works as advertised. After 5 years it can only have improved. It's not "cheap" but usually pays for itself due to being able to consolidate the workload of dozens of physical servers into just 2 or 3 boxes. >> I don't recall seeing your user load or IOPS requirements so I'm making >> some educated guesses WRT your required performance and total storage. > > I'm embarrassed to admit I don't have hard numbers on the user load > except the rapidly dwindling disk space count and the fact when the > web-based mail application try to list and check disk quota, it can > bring the servers to a crawl. Maybe just starting with a description of your current hardware setup and number of total users/mailboxes would be a good starting point. How many servers do you have, what storage is connected to each, percent of MUA POP/IMAP connections from user PCs versus those from webmail applications, etc, etc. Probably the single most important piece of information would be the hardware specs of your current Dovecot server, CPUs/RAM/disk array, etc, and what version of Dovecot you're running. The focus of your email is building a storage server strictly to offload old mail and free up space on the Dovecot server. From the sound of things, this may not be sufficient to solve all your problems. > My lame excuse is that I'm just the web > dev who got caught holding the server admin potato. Baptism by fire. Ouch. What doesn't kill you makes you stronger. ;) >> is nearly irrelevant for a mail workload, you can see it's much cheaper >> to scale capacity and IOPS with a single node w/fat storage than with >> skinny nodes w/thin storage. Ok, so here's the baseline config I threw >> together: > > One of my concern is that heavy IO on the same server slow the overall > performance even though the theoretical IOPS of the total drives are > the same on 1 and on X servers. Right now, the servers are usually > screeching to a halt, to the point of even locking out SSH access due > to IOWait sending the load in top to triple digits. If multiple servers are screeching to a halt due to iowait, either all of your servers individual disks are overloaded, or you already have shared storage. We really need more info on your current architecture. Right now we don't knw if we're talking about 4 servers or 40., 100 users or 10,000. >> Some host failure redundancy is about all you'd gain from the farm >> setup. Dovecot shouldn't barf due to one NFS node being down, only >> hiccup. I.e. only imap process accessing files on the downed node would >> have trouble. > > But if I only have one big storage node and that went down, Dovecot > would barf wouldn't it? > Or would the mdbox format mean Dovecot would still use the local > storage, just that users can't access the offloaded messages? If the big storage node is strictly alt storage, and it dies, Dovecot will still access its main mdbox storage just fine. It simply wouldn't be able to access the alt storage and would log errors for those requests. If you design a whole new architecture from scratch, going with ESX and an iSCSI SAN this whole line of thinking is moot as there is no SPOF. This can be done with as little as two physical servers and one iSCSI SAN array which has dual redundant controllers in the base config. Depending on your actual IOPS needs, you could possibly consolidate everything you have now into two physical servers and one iSCSI SAN array, for between $30-40K USD in hardware and $8-10K in ESX licenses. That just a guess on ESX as I don't know the current pricing. Even if it's that "high" it's far more than worth the price due to the capability. Such a setup allows you to run all of your Exim, webmail, Dovecot, etc servers on two machines, and you usually get much better performance than with individual boxes, especially if you manually place the VMs on the nodes for lowest network latency. For instance, if you place your webmail server VM on the same host as the Dovecot VM, TCP packet latency drops from the high micro/low milliscond range into the mid nanosecond range--a 1000x decrease in latency. Why? The packet transfer is now a memory-to-memory copy through the hypervisor. The packets never reach a physical network interface. You can do some of these things with free Linux hypervisors, but AFAIK the poor management interfaces for them make the price of ESX seem like a bargain. >>> Also, I could possibly arrange them in a sort >>> of network raid 1 to gain redundancy over single machine failure. >> >> Now you're sounding like Charles Marcus, but worse. ;) Stay where you >> are, and brush your hair away from your forehead. I'm coming over with >> my branding iron that says "K.I.S.S" > Lol, I have no idea who Charles is, but I always feel safer if there > was some kind of backup. Especially since I don't have the time to > dedicate myself to server administration, by the time I notice > something is bad, it might be too late for anything but the backup. Search the list archives for Charles' thread about bringing up a 2nd office site. His desire was/is to duplicate machines at the 2nd site for redundancy, when the proper thing to do is duplicate them at the primary site, and simply duplicate the network links between sites. My point to you and Charles is that you never add complexity for the sake of adding complexity. > Of course management and clients don't agree with me since > backup/redundancy costs money. :) So does gasoline, but even as the price has more than doubled in 3 years in the States, people haven't stopped buying it. Why? They have to have it. The case is the same for certain levels of redundancy. You simply have to have it. You job is properly explaining that need. Ask the CEO/CFO how much money the company will lose in productivity if nobody has email for 1 workday, as that is how long it will take to rebuild it from scratch and restore all the data when it fails. Then ask what the cost is if all the email is completely lost because they were to cheap to pay for a backup solution? To executives, money in the bank is like the family jewels in their trousers. Kicking the family jewels and generating that level of pain seriously gets their attention. Likewise, if a failed server plus rebuild/restore costs $50K in lost productivity, spending $20K on a solution to prevent that from happening is a good investment. Explain it in terms execs understand. Have industry data to back your position. There plenty of it available. -- Stan From dmiller at amfes.com Mon Apr 9 04:20:19 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Sun, 08 Apr 2012 18:20:19 -0700 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: <20120407143038.5252acf9@scorpio> References: <4F7D972A.8050406@Media-Brokers.com> <4F7F4B53.2090700@Media-Brokers.com> <20120407143038.5252acf9@scorpio> Message-ID: On Sat, 7 Apr 2012 14:30:38 -0400, Jerry wrote: > On Sat, 07 Apr 2012 11:06:48 -0700 > Daniel L. Miller articulated: > >> Unfortunately, the docs for the ltmp agent http://www.postfix.org/lmtp.8.html [1] don't say anything about adding these headers. I tried asking on the Postfix list - didn't get much of an answer. > > I may be wrong; however, from what I have been able to understand in > regards to the Postfix documentation, if it does not explicitly claim to > have a feature, then that feature is not available. In other words, if > it doesn't state it can do it, it can't. As I just stated on that list - even though a given feature may be documented, the possible uses of that feature may not be immediately apparent. And while the Postfix lda & virtual transports have the "flag" parameters, and the lmtp transport does not - the lmtp transport DOES have a whole slew of other parameters not available in the lda. So I was simply inquiring if there was a way to achieve my goal - given that my understanding of smtp handling in general, and Postfix in particular, are extremely limited. For some reason, I seem to irritate people with my polite questions - while others who are (in my opinion) downright rude and aggressive get assistance and acceptance. Maybe I need to start being more of a jerk on purpose... -- Daniel Links: ------ [1] http://www.postfix.org/lmtp.8.html From tss at iki.fi Mon Apr 9 09:47:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 09:47:11 +0300 Subject: [Dovecot] lmtp sometimes fails to deliver a message to all recipients In-Reply-To: <4F7C724E.8040905@talex.pl> References: <4F79C995.9060607@talex.pl> <1B1369FF-6748-46DE-AB94-AF2F32061757@iki.fi> <4F7C724E.8040905@talex.pl> Message-ID: <981BEDB0-63E3-43C1-906A-390E4BD053CA@iki.fi> On 4.4.2012, at 19.09, Artur Zaprza?a wrote: >>> lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Message was expunged (guid) >>> lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: script /vmail/domain/foo/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /vmail/domain/foo/.dovecot.sieve.log may reveal additional details) >> Fixed in hg. >> > Tested with Maildir. Works great. Thanks. > > The above problem was appearing when some recipients (including first one) had a sieve filter with discard action for current message. In this case, depending on the pattern of recipients having a sieve discard action, lmtp can create more than one instance of the message for a few dozen recipients. It would be nice if lmtp could create a single hardlinked instance of the message even in this case. The problem here isn't the discard action, but that Sieve is used at all. The hard linking happens currently only for users who don't have Sieve scripts. I've a plan to fix this, but it's not a simple fix and it's pretty low priority currently. From tss at iki.fi Mon Apr 9 09:54:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 09:54:11 +0300 Subject: [Dovecot] 2.1.3: Overly lax FETCH parsing In-Reply-To: <20120405125920.Horde.wazxEIF5lbhPfeuIm_gQ4uA@bigworm.curecanti.org> References: <20120405125920.Horde.wazxEIF5lbhPfeuIm_gQ4uA@bigworm.curecanti.org> Message-ID: On 5.4.2012, at 21.59, Michael M Slusarz wrote: > While useful that Dovecot is more liberal about what it receives, 3501 seems pretty clear that incorrect FETCH parameters must return a BAD. I can verify that the above commands fail on Cyrus. It's a SHOULD, not a MUST: Servers SHOULD enforce the syntax outlined in this specification strictly. Any client command with a protocol syntax error, including (but not limited to) missing or extraneous spaces or arguments, SHOULD be rejected, and the client given a BAD server completion response. But since it's not much trouble to fix it: http://hg.dovecot.org/dovecot-2.1/rev/19e09ab09383 From tss at iki.fi Mon Apr 9 09:59:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 09:59:55 +0300 Subject: [Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups In-Reply-To: References: Message-ID: <558E2E13-E08D-46F3-B68A-0AA63DF43D38@iki.fi> On 6.4.2012, at 1.46, Joseph Tam wrote: > One other consideration (at least for me) is if the INBOX and > personal mail folders are stored in two separate FS's. It would be nice > to fuse the two sets of backups under the same namespace, but I don't > know how the namespace prefix matching works and whether you can define > hierarchical namespaces like > > namespace { > prefix = backup/inbox > location = mbox:/path/to/inbox-snapdir/%u > ... > } > > namespace { > prefix = backup/mail > location = mbox:/path/to/mail-snapdir/%u > ... > } You can define hierarchical namespaces, although they've probably not been used outside my few tests. Well, except shared// autocreated namespaces are already children to shared/ namespace, so I guess they should work. From tss at iki.fi Mon Apr 9 10:03:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:03:39 +0300 Subject: [Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups In-Reply-To: <4F7DD03B.30909@Media-Brokers.com> References: <4F7DBA3B.5090803@Media-Brokers.com> <4F7DCA4A.70009@whyscream.net> <4F7DD03B.30909@Media-Brokers.com> Message-ID: On 5.4.2012, at 20.02, Charles Marcus wrote: > On 2012-04-05 12:37 PM, Tom Hendrikx wrote: >> The first interesting point I'd see with this, is that you supply the >> mail client with a near endless supply of folders, which would take a >> lot of caching space on the clients end, either (depending on the client >> and its configuration) from the moment that you enable this fort hem, or >> after someone starts searching in their 'time machine' for some old mail. > > Since we use Thunderbird, I can of course disable offline mode for everyone, so the only time headers would be downloaded would be when the user selects (or performs a search on) one (or more) of the folders. Do they need to be accessible via Thunderbird, or maybe only via a webmail? Or perhaps a secondary (normally disabled?) TB account where you've specified a "backup/" namespace prefix (which is normally hidden)? From tss at iki.fi Mon Apr 9 10:07:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:07:40 +0300 Subject: [Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups In-Reply-To: <4F7DBA3B.5090803@Media-Brokers.com> References: <4F7DBA3B.5090803@Media-Brokers.com> Message-ID: On 5.4.2012, at 18.28, Charles Marcus wrote: > The snapshots are stored with the following filesystem layout: > > /path/to/snapshotsdir/hourly.0 > ... > /path/to/snapshotsdir/hourly.4 > /path/to/snapshotsdir/daily.0 .. > The 'names' (hourly, daily, weekly, monthly, yearly) are arbitrary (this is a bit confusing to people new to rsnapshot), and would *not* be used for displaying the mail folders to the users - it is the Date/Time stamps of each of the snapshot dirs above that would be used to display the folder names under the 'Time Machine' namespace. This is, I imagine, the part that will need some actual coding by Timo to get working - maybe just some new config variables added to the namespace code for mapping the date/time stamps of the directories to user friendly folder names in the namespace. I guess there could be kind of a "filter fs layout" that modifies the filesystem layout a bit and lets the underlying layout handle the rest: namespace { location = maildir:/path/to/snapshotsdir:LAYOUT=timestamp } Although it's annoying that it's not possible to have per-layout settings currently.. But I guess if this was implemented as plugin it would be enough to have: plugin { timestamp_layout = maildir++ } From tss at iki.fi Mon Apr 9 10:13:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:13:39 +0300 Subject: [Dovecot] Director (was: Hints for a NFS-Setup) In-Reply-To: <4F7EF67A.8060001@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F7EF67A.8060001@wk-serv.de> Message-ID: <613701AE-34D1-4366-887E-72F576A01A1D@iki.fi> On 6.4.2012, at 16.58, Patrick Westenberg wrote: > Hi again, > > I tried to setup a test invironemnt like this: > > MTA --(lmtp)--\ /--(lmtp)--> backend1 --\ > -- director -- -- NFS > MTA --(lmtp)--/ \--(lmtp)--> backend2 --/ > > > IMAP-User --> frontend1 --\ /--(imap)--> backend1 --\ > -- director -- -- NFS > IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/ > > but now I'm very confused. Is it actually possible to setup a host (or two) as a director only or will I have to enable the director service on each frontend and MTA? The cleanest way to run director is to have 2 or more servers running only director itself. If you want to have less servers, it's also possible to place a Dovecot director configuration to any other servers as well, but that's conceptually more complex. For MTA you'd simply tell its LMTP client to connect to director servers, which could be one of: a) Load balancer's IP address b) Host name that expands to all directors' IP addresses. If the first one is down, the LMTP client (hopefully! verify!) connects to the second one. From tss at iki.fi Mon Apr 9 10:15:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:15:37 +0300 Subject: [Dovecot] distributed mdbox In-Reply-To: References: <1ff190f691892c01a980927422f38753@mail.airstreamcomm.net> Message-ID: <7864D800-17D2-48E3-832D-D44FA912FC5C@iki.fi> Yeah, not caching then. I know Glusterfs people implemented some fixes/workarounds to make Dovecot work better. I don't know if all of those fixes are in the public glusterfs. On 6.4.2012, at 18.39, James Devine wrote: > As it turns out I can duplicate this problem with a single dovecot server > and a single gluster server using mdbox, so maybe not caching? This being > the case I don't think director would help > > On Thu, Apr 5, 2012 at 7:16 PM, James Devine wrote: > >> >> >> On Fri, Mar 23, 2012 at 7:39 AM, wrote: >> >>> On Wed, 21 Mar 2012 09:56:12 -0600, James Devine >>> wrote: >>>> Anyone know how to setup dovecot with mdbox so that it can be used >>> through >>>> shared storage from multiple hosts? I've setup a gluster volume and am >>>> sharing it between 2 test clients. I'm using postfix/dovecot LDA for >>>> delivery and I'm using postal to send mail between 40 users. In doing >>>> this, I'm seeing these errors in the logs >>>> >>>> Mar 21 09:36:29 test-gluster-client2 dovecot: lda(testuser34): Error: >>> Fixed >>>> index file /mnt/testuser34/mdbox/storage/dovecot.map.index: >>> messages_count >>>> 272 -> 271 >>>> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: >>> Log >>>> synchronization error at seq=4,offset=3768 for >>>> /mnt/testuser28/mdbox/storage/dovecot.map.index: Append with UID 516, >>> but >>>> next_uid = 517 >>>> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: >>> Log >>>> synchronization error at seq=4,offset=4220 for >>>> /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update >>>> for invalid uid=517 >>>> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: >>> Log >>>> synchronization error at seq=4,offset=5088 for >>>> /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update >>>> for invalid uid=517 >>>> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Warning: >>>> fscking index file /mnt/testuser28/mdbox/storage/dovecot.map.index >>>> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser34): Warning: >>>> fscking index file /mnt/testuser34/mdbox/storage/dovecot.map.index >>>> >>>> >>>> This is my dovecot config currently: >>>> >>>> jdevine at test-gluster-client2:~> dovecot -n >>>> # 2.0.13: /etc/dovecot/dovecot.conf >>>> # OS: Linux 3.0.0-13-server x86_64 Ubuntu 11.10 >>>> lock_method = dotlock >>>> mail_fsync = always >>>> mail_location = mdbox:~/mdbox >>>> mail_nfs_index = yes >>>> mail_nfs_storage = yes >>>> mmap_disable = yes >>>> passdb { >>>> driver = pam >>>> } >>>> protocols = " imap" >>>> ssl_cert = >>> ssl_key = >>> userdb { >>>> driver = passwd >>>> } >>> >>> I was able to get dovecot working across a gluster cluster a few weeks ago >>> and it worked just fine. I would recommend using the native gluster mount >>> option (need to install gluster software on clients), and using >>> distributed >>> replicated as your replication mechanism. If you're running two gluster >>> servers you should have a replica count of two with distributed >>> replicated. >>> You should test first to make sure you can create a file in both mounts >>> and see it from every mount point in the cluster, as well as interact with >>> it. It's also very important to make sure your servers are running with >>> synchronized clocks from an NTP server. Very bad things happen to a >>> (dovecot or gluster) cluster out of sync with NTP. >>> >>> What storage method are you using? I'm able to produce errors within >> seconds of starting postal with more than one thread From tss at iki.fi Mon Apr 9 10:17:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:17:42 +0300 Subject: [Dovecot] POP3 dele to Trash? In-Reply-To: <4F7F85F2.3000909@corp.sonic.net> References: <20120406234001.GG89906@corp.sonic.net> <4F7F85F2.3000909@corp.sonic.net> Message-ID: <540057BA-FA72-4A34-B9C1-EBEA8458A210@iki.fi> On 7.4.2012, at 3.10, Kelsey Cummings wrote: > On 04/06/12 16:40, Kelsey Cummings wrote: >> Has anyone already done this? Should this be possible via a plugin? >> I see the deleted-to-trash imap plugin. We are using Maildir if it >> makes a difference. > > Of course, this is exactly what the Lazy Expunge plugin does, isn't it? Not exactly, the messages would go to lazyexpunge-namespace-prefix/INBOX. But maybe close enough? Otherwise would require writing a new plugin. From tss at iki.fi Mon Apr 9 10:27:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:27:04 +0300 Subject: [Dovecot] Setting ACL for master user after login In-Reply-To: References: Message-ID: <1F913824-E90D-4807-ABC4-78E119303CF0@iki.fi> On 7.4.2012, at 6.48, PL MB wrote: > I'd like to log in to normal user accounts as a master user but retain > the normal users' ACLs. > > The Master Users page on the Dovecot 1.x wiki (1) says that I can set > the master user's ACLs in a postlogin script. The documentation for > master users on the 2.x wiki (2) no longer has any statements about > master user ACLs. > > Has something important in this regard changed? Can I no longer > override the ACLs in a postlogin script? No, it's just that the ACL text was added there after wiki2 was forked. I updated now http://master.wiki2.dovecot.org/Authentication/MasterUsers#ACLs I'm pretty sure the userdb way works in v2.1, possibly also in v2.0 and probably not in v1.x. From tss at iki.fi Mon Apr 9 10:29:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:29:20 +0300 Subject: [Dovecot] Director pop-login and imap-login processes exiting on signal 11 In-Reply-To: <20120407030811.I46246@shell.xecu.net> References: <20120407030811.I46246@shell.xecu.net> Message-ID: <70AB2E58-42C9-49B6-BE13-C553893E19EF@iki.fi> On 7.4.2012, at 10.13, Andy Dills wrote: > Apr 7 02:18:05 mail-out06 dovecot: pop3-login: Fatal: master: > service(pop3-login): child 75029 killed with signal 11 (core not dumped - > set service pop3-login { drop_priv_before_exec=yes }) v2.1.3 proxying was buggy with SSL connections. Probably crashes because of that. I was supposed to release v2.1.4 already but.. From tss at iki.fi Mon Apr 9 10:33:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:33:18 +0300 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: <4F7D972A.8050406@Media-Brokers.com> References: <4F7D972A.8050406@Media-Brokers.com> Message-ID: On 5.4.2012, at 15.59, Charles Marcus wrote: > Does anyone know if the use of LMTP (or even the dovecot LDA) still loses the x-original-to header that the postfix vda adds and that I rely heavily on (since I use a lot of aliases), and if it does, is there any solution to get the original recipient added back in before final delivery? LMTP adds a new Delivered-To: header when there is a single RCPT TO. You can force a single RCPT TO from Postfix side by setting lmtp_destination_recipient_limit=1. LMTP doesn't add/remove/change X-Original-To: header. From stu at spacehopper.org Mon Apr 9 10:49:35 2012 From: stu at spacehopper.org (Stuart Henderson) Date: Mon, 9 Apr 2012 07:49:35 +0000 (UTC) Subject: [Dovecot] Outlook (2010) -> Dovecot (IMAP) >10x slower with high network load and many folders References: Message-ID: On 2012-04-06, Thomas von Eyben wrote: > I am seeing a >10x as slow performance when trying to complete a > "send/receive" from an Outlook 2010 client to Dovecot via IMAP, but > only when the LAN is fully loaded with other traffic, EG file copying. > It seems the problem is when outlook is trying to identify folders > that have changed since last "send/receive" thus traversing the > hierachy. Not sure why it would only affect Outlook clients, but if your switches are managed, you might like to check if flow control is enabled and, if so, try disabling it. From tss at iki.fi Mon Apr 9 11:19:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 11:19:03 +0300 Subject: [Dovecot] Director simplification? Message-ID: <71D6ACD2-8FF6-4E74-8C27-BF54DDACA3E5@iki.fi> An idea I just had: Director basically works by assigning the backend IP address by: ip = vhosts[ md5(username) mod vhosts_count ].ip The rest of director is about what happens when vhosts[] or vhosts_count changes. What about instead doing this on IP address level? ip = ip_pool[ md5(username) mod ip_pool_size ] When a backend dies, you'll reassign the backend's IPs to other backends. Each backend should have many IPs. The main restriction here is that the IP pool cannot change without stopping the entire Dovecot. But if you initially allocate enough IPs, that shouldn't be a problem. And the advantage of this over the current director? To guarantee that one director can't break others, because they don't need to communicate with each others. The disadvantage of course is that it's a little less flexible and requires more planning ahead. The IP address reassignment would also need some distro-specific scripts. This could be implemented as an alternative director-lite or something. The doveadm director status-related commands could still work with it. From tss at iki.fi Mon Apr 9 13:19:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 13:19:50 +0300 Subject: [Dovecot] v2.1.4 released Message-ID: http://dovecot.org/releases/2.1/dovecot-2.1.4.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.4.tar.gz.sig + Added mail_temp_scan_interval setting and changed its default value from 8 hours to 1 week. + Added pop3-migration plugin for easily doing a transparent IMAP+POP3 migration to Dovecot: http://wiki2.dovecot.org/Migration/Dsync + doveadm user: Added -m parameter to show some of the mail settings. - Proxying SSL connections crashed in v2.1.[23] - fts-solr: Indexing mail bodies was broken. - director: Several changes to significantly improve error handling - doveadm import didn't import messages' flags - mail_full_filesystem_access=yes was broken - Make sure IMAP clients can't create directories when accessing nonexistent users' mailboxes via shared namespace. - Dovecot auth clients authenticating via TCP socket could have failed with bogus "PID already in use" errors. From tss at iki.fi Mon Apr 9 13:19:52 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 13:19:52 +0300 Subject: [Dovecot] v2.0.20 released Message-ID: <3972C017-7380-4936-8231-AF660DEC9106@iki.fi> http://dovecot.org/releases/2.0/dovecot-2.0.20.tar.gz http://dovecot.org/releases/2.0/dovecot-2.0.20.tar.gz.sig + doveadm user: Added -m parameter to show some of the mail settings. - doveadm import didn't import messages' flags - Make sure IMAP clients can't create directories when accessing nonexistent users' mailboxes via shared namespace. - Dovecot auth clients authenticating via TCP socket could have failed with bogus "PID already in use" errors. From andy at xecu.net Mon Apr 9 13:58:33 2012 From: andy at xecu.net (Andy Dills) Date: Mon, 9 Apr 2012 06:58:33 -0400 (EDT) Subject: [Dovecot] Director pop-login and imap-login processes exiting on signal 11 In-Reply-To: <70AB2E58-42C9-49B6-BE13-C553893E19EF@iki.fi> References: <20120407030811.I46246@shell.xecu.net> <70AB2E58-42C9-49B6-BE13-C553893E19EF@iki.fi> Message-ID: <20120409065728.K14159@shell.xecu.net> On Mon, 9 Apr 2012, Timo Sirainen wrote: > On 7.4.2012, at 10.13, Andy Dills wrote: > > > Apr 7 02:18:05 mail-out06 dovecot: pop3-login: Fatal: master: > > service(pop3-login): child 75029 killed with signal 11 (core not dumped - > > set service pop3-login { drop_priv_before_exec=yes }) > > v2.1.3 proxying was buggy with SSL connections. Probably crashes because > of that. I was supposed to release v2.1.4 already but.. Thanks Timo. I can confirm this is fixed in 2.1.4. Andy --- Andy Dills Xecunet, Inc. www.xecu.net 301-682-9972 --- From CMarcus at Media-Brokers.com Mon Apr 9 15:50:42 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 09 Apr 2012 08:50:42 -0400 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: References: <4F7D972A.8050406@Media-Brokers.com> Message-ID: <4F82DB22.2050201@Media-Brokers.com> On 2012-04-09 3:33 AM, Timo Sirainen wrote: > On 5.4.2012, at 15.59, Charles Marcus wrote: > >> Does anyone know if the use of LMTP (or even the dovecot LDA) still >> loses the x-original-to header that the postfix vda adds and that I >> rely heavily on (since I use a lot of aliases), and if it does, is >> there any solution to get the original recipient added back in >> before final delivery? > LMTP adds a new Delivered-To: header when there is > a single RCPT TO. You can force a single RCPT TO from Postfix side by > setting lmtp_destination_recipient_limit=1. LMTP doesn't > add/remove/change X-Original-To: header. Ok, thanks Timo... but... Are you saying that this 'Delivered-To:' header can somehow be leveraged to provide the same info as the x-original-to header? If not, since it was the postfix virtual delivery agent that added the x-original-to, and since using lmtp means I would not be using the postfix vda, is the appropriate place to add this header in dovecot's lmtp implementation (and if so, how hard would it be)? Or would this need to be done somehow on the postfix side (if so, I'll go ask on the postfix list)? Sorry for my ignorance - but as I said, I rely on this header (I use a ton of aliases, and without it I can't see the original (alias) recipient), so I need to determine if I'm going to be able to use lmtp or not (obviously, I would much prefer to do so)... Thanks again Timo... -- Best regards, Charles From tss at iki.fi Mon Apr 9 15:53:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 15:53:56 +0300 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: <4F82DB22.2050201@Media-Brokers.com> References: <4F7D972A.8050406@Media-Brokers.com> <4F82DB22.2050201@Media-Brokers.com> Message-ID: On 9.4.2012, at 15.50, Charles Marcus wrote: >> LMTP adds a new Delivered-To: header when there is >> a single RCPT TO. You can force a single RCPT TO from Postfix side by >> setting lmtp_destination_recipient_limit=1. LMTP doesn't >> add/remove/change X-Original-To: header. > > Ok, thanks Timo... but... > > Are you saying that this 'Delivered-To:' header can somehow be leveraged to provide the same info as the x-original-to header? I guess X-Original-To is the same address as what Postfix sees as the original RCPT TO address before alias expansion and such? In that case, see my today's mail in Postfix list.. From pw at wk-serv.de Mon Apr 9 16:25:41 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Mon, 09 Apr 2012 15:25:41 +0200 Subject: [Dovecot] Director In-Reply-To: <613701AE-34D1-4366-887E-72F576A01A1D@iki.fi> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F7EF67A.8060001@wk-serv.de> <613701AE-34D1-4366-887E-72F576A01A1D@iki.fi> Message-ID: <4F82E355.8000901@wk-serv.de> Timo Sirainen schrieb: > b) Host name that expands to all directors' IP addresses. If the first one is down, the LMTP client (hopefully! verify!) connects to the second one. Yes, Postfix' LMTP works fine with a hostname expanding to two IPs. From pw at wk-serv.de Mon Apr 9 16:35:01 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Mon, 09 Apr 2012 15:35:01 +0200 Subject: [Dovecot] Problem with quota warning Message-ID: <4F82E585.7030201@wk-serv.de> Hi again, my test environment is working now and finally I tried to implement my quota warning but it is not working. LDA says the mailbox is full even it is not (only 81% and that is why the quota warning is executed). Further I used :noenforcing in my script. Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: Namespace INBOX.: Using permissions from /var/mail/k-team.info/test/mdbox: mode=0700 gid=-1 Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: Quota root: name=User quota backend=dict args=:file://mdbox/dovecot-quota:noenforcing Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: Quota warning: bytes=0 (90%) messages=0 reverse=no command=quota-warning 90 raw mail user Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: Quota warning: bytes=0 (75%) messages=0 reverse=no command=quota-warning 75 raw mail user Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: dict quota: user=raw mail user, uri=file://mdbox/dovecot-quota:noenforcing, noenforcing=0 Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: none: root=, index=, control=, inbox=, alt= Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: Destination address: test at k-team.info (source: user at hostname) Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): msgid=unspecified: save failed to INBOX: Quota exceeded (mailbox for user is full) Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): msgid=unspecified: rejected: Quota exceeded (mailbox for user is full) Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): msgid=: Return-Path missing, rejection reason: Quota exceeded (mailbox for user is full) # 2.1.3: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.4 auth_debug = yes auth_debug_passwords = yes auth_verbose = yes listen = 172.17.99.96 mail_debug = yes mail_gid = vmail mail_home = /var/mail/%d/%n mail_location = mdbox:~/mdbox:INDEX=/var/indexes/%d/%n mail_plugins = quota mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { autocreate = INBOX.Sent autocreate2 = INBOX.Trash autocreate3 = INBOX.Drafts autocreate4 = INBOX.Spamverdacht autosubscribe = INBOX.Sent autosubscribe2 = INBOX.Trash autosubscribe3 = INBOX.Drafts autosubscribe4 = INBOX.Spamverdacht quota = dict:User quota::file:%h/mdbox/dovecot-quota quota_rule2 = INBOX.Trash:storage=+100M quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=75%% quota-warning 75 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_global_path = /usr/local/etc/dovecot/sieve/default.sieve } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { user = vmail } } service imap-login { service_count = 1 } service lmtp { inet_listener lmtp { address = 172.17.99.96 port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 } service quota-warning { executable = script /usr/local/etc/dovecot/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = required ssl_cert = References: <4F7D972A.8050406@Media-Brokers.com> <4F82DB22.2050201@Media-Brokers.com> Message-ID: <4F82E7CB.6010206@Media-Brokers.com> On 2012-04-09 8:53 AM, Timo Sirainen wrote: > I guess X-Original-To is the same address as what Postfix sees as the > original RCPT TO address before alias expansion and such? In that > case, see my today's mail in Postfix list. Yep... and hoping that you and Wietse can work out some way to support it... Thanks for participating in the discussion over there... :) -- Best regards, Charles From lists at luigirosa.com Mon Apr 9 16:44:46 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Mon, 09 Apr 2012 15:44:46 +0200 Subject: [Dovecot] mount Message-ID: <4F82E7CE.3070502@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I have a Dovecot installation on CentOS 5 where I sometimes mount external fs in /mnt Every Dovecot data is in local / file system, nothing is mounted elswhere After upgrading to 1.2.4 I rebooted the system for other reasons and at startup I got this on Dovecot log: master: Warning: /mnt is no longer mounted. If this is intentional, remove it with doveadm mount No /mnt entry in /etc/fstab and nothing pmounted under /mnt I THINK that the last time I used /mnt to mount something was few weeks ago to update VMware tools. Is there a way toi tell Dovecot to ignore /mnt ? Ciao, luigi - -- / +--[Luigi Rosa]-- \ $100 invested at 7% interest for 100 years will become $100,000, at which time it will be worth absolutely nothing. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk+C58gACgkQ3kWu7Tfl6ZRc0wCgl0Z4OtblYbfYwzvOp1/vUifV PqYAoIvfltvmq3cijvDbOEKV2Tai2rpu =hyrI -----END PGP SIGNATURE----- From =?UTF-8?Q?=D0=91=D1=80=D0=B0=D0=BD=D0=BA=D0=BE_=D0=9C=D0=B0=D1=98?= Mon Apr 9 16:51:49 2012 From: =?UTF-8?Q?=D0=91=D1=80=D0=B0=D0=BD=D0=BA=D0=BE_=D0=9C=D0=B0=D1=98?= (=?UTF-8?Q?=D0=91=D1=80=D0=B0=D0=BD=D0=BA=D0=BE_=D0=9C=D0=B0=D1=98?=) Date: Mon, 09 Apr 2012 15:51:49 +0200 Subject: [Dovecot] Username from rfc822Name subject alternative name Message-ID: Hello, I'm looking into adding support for extracting the username from client certificate's rfc822Name (from the subjectAltName extension). The question I have is what would be the best approach to do this? Current implementation has a kind of clean code since it just goes through the subject name, extracting the values with X509_NAME_get_text_by_NID (while NID is obtained with OBJ_txt2nid). If I were to add this, it's bound to make the code a little bit more complicated since SAN's can't be retrieved in the same way. So far in terms of options I have, I can see the following: 1. Create a distinct configuration option for the ssl_cert_username_field (i.e. specify something like "sanrfc822Name" to have Dovecot extract the username from the designated alternative name). 2. Make the current code fail-over to rfc822Name SAN if emailAddress is provided for ssl_cert_username (less invasion in code, but less flexibility as well). Any input/recommendation/directioning is welcome. I've wanted to actually first write a patch, and then submit it, but I think it might be better to check what would be preferable by Dovecot maintainers/devs. Best regards -- Branko Majic Jabber: branko at majic.rs Please use only Free formats when sending attachments to me. ?????? ????? ?????: branko at majic.rs ????? ??? ?? ??????? ?????? ????????? ? ????????? ?????????. From tss at iki.fi Mon Apr 9 16:57:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 16:57:48 +0300 Subject: [Dovecot] mount In-Reply-To: <4F82E7CE.3070502@luigirosa.com> References: <4F82E7CE.3070502@luigirosa.com> Message-ID: On 9.4.2012, at 16.44, Luigi Rosa wrote: > I have a Dovecot installation on CentOS 5 where I sometimes mount external fs > in /mnt > > Every Dovecot data is in local / file system, nothing is mounted elswhere .. > Is there a way toi tell Dovecot to ignore /mnt ? doveadm mount add /mnt ignore From lists at luigirosa.com Mon Apr 9 16:59:05 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Mon, 09 Apr 2012 15:59:05 +0200 Subject: [Dovecot] mount In-Reply-To: References: <4F82E7CE.3070502@luigirosa.com> Message-ID: <4F82EB29.8030709@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Timo Sirainen said the following on 09/04/12 15:57: >> Is there a way toi tell Dovecot to ignore /mnt ? > doveadm mount add /mnt ignore Thanks, next time I will RTFM first. Ciao, luigi - -- / +--[Luigi Rosa]-- \ [UNIX] n. A random quote, item of trivia, joke or maxim printed to the user's tty at login time or (less commonly) at logout time. Items from this jargon file have often been used as fortune cookies. --Jargon File -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk+C6ykACgkQ3kWu7Tfl6ZQI6QCgt4E3Imx1OeaB6SqjIjWDhjS0 xqUAoKizTRivIkvKkQE5SS7zwCtPlL9B =RCsz -----END PGP SIGNATURE----- From achekalin at lazurit.com Mon Apr 9 17:58:23 2012 From: achekalin at lazurit.com (Alexander Chekalin) Date: Mon, 09 Apr 2012 17:58:23 +0300 Subject: [Dovecot] Multiply mailboxes vs one huge Message-ID: <4F82F90F.8020902@lazurit.com> Hello, as I need to store a lot of messages on my IMAP server (order of 900K-1000K; this is an archive for some time, maybe a year or so), I see some "slowness" in dealing with such a huge amount. I mainly need to do searches like "get all messages from user1 at domain1.com to user2 at domain2.tld recieved between date1 and date2". So I really interested will it be wise to a) split all messages into several smaller mailboxes (per-month, or per-day, or create 2-level-structure like "month/day/") b) use dbox (vs currently used mbox) storage scheme (I'm afraid of mdbox as I still not sure I'll be able to parse it by scripts later "just in case") Dovecot is the latest one (2.1.3). No compression Dovecot-side, but it mails are in zfs volume with compression on. I ask this mainly due to my not fully understand how Dovecot indexes are working. I also test another approach: to use my own index somewhere outside Dovecot which will store reference between emails and UIDs, and dates and UIDs, so I'll simple query my index for things I need. But then, that's exactly what IMAP index can do, so I simple slow my search down, isn't it? The only reason I think about my own index is I won't use 'all header' as search scope, I need to deal only with From:, To:, Cc:, Bcc: (if any), Recieved (if nowehere else I see the from/to info), and date field(s) - I doubt IMAP will care for that for me. Yours, Alexander From tss at iki.fi Mon Apr 9 18:03:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 18:03:38 +0300 Subject: [Dovecot] Multiply mailboxes vs one huge In-Reply-To: <4F82F90F.8020902@lazurit.com> References: <4F82F90F.8020902@lazurit.com> Message-ID: On 9.4.2012, at 17.58, Alexander Chekalin wrote: > as I need to store a lot of messages on my IMAP server (order of 900K-1000K; this is an archive for some time, maybe a year or so), I see some "slowness" in dealing with such a huge amount. I mainly need to do searches like "get all messages from user1 at domain1.com to user2 at domain2.tld recieved between date1 and date2". So by "received between date" you mean the IMAP INTERNALDATE as opposed to Date: header? These kind of searches are looked up from the index/cache files, and the performance should be exactly the same with all of the mailbox formats. It would be useful to figure out what exactly is causing the slowness. Is the SEARCH command slow? Something else? Is the slowness about user CPU, system CPU or disk IO? From andre.rodier at gmail.com Mon Apr 9 19:26:50 2012 From: andre.rodier at gmail.com (Andre Rodier) Date: Mon, 9 Apr 2012 17:26:50 +0100 Subject: [Dovecot] per user sieve after filters Message-ID: <20120409162650.GA29690@london.sagso.home> Hello, Thanks for dovecot, as it's still the best mail server. I'd like to use per users sieve_after scripts. Can I put in my dovecot config file, something like that: sieve_after = %h/Mails/Sieve/After/ It would be very useful for me, as I'd like to add vacation script to be executed from this place. Kind regards, Andr?. From stephan at rename-it.nl Mon Apr 9 21:01:42 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 09 Apr 2012 20:01:42 +0200 Subject: [Dovecot] per user sieve after filters In-Reply-To: <20120409162650.GA29690@london.sagso.home> References: <20120409162650.GA29690@london.sagso.home> Message-ID: <4F832406.8060108@rename-it.nl> On 4/9/2012 6:26 PM, Andre Rodier wrote: > Hello, > > Thanks for dovecot, as it's still the best mail server. > > I'd like to use per users sieve_after scripts. > > Can I put in my dovecot config file, something like that: > > sieve_after = %h/Mails/Sieve/After/ > > It would be very useful for me, as I'd like to add vacation script to be > executed from this place. I must say I've never tested something like that, but it should work. Regards, Stephan. From centos.admin at gmail.com Mon Apr 9 22:15:02 2012 From: centos.admin at gmail.com (Emmanuel Noobadmin) Date: Tue, 10 Apr 2012 03:15:02 +0800 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F81D73B.9030901@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> Message-ID: On 4/9/12, Stan Hoeppner wrote: > So it seems you have two courses of action: > 1. Identify individual current choke points and add individual systems > and storage to eliminate those choke points. > > 2. Analyze your entire workflow and all systems, identifying all choke > points, then design a completely new well integrated storage > architecture that solves all current problems and addresses future needs I started to do this and realize I have a serious mess on hand that makes delving in other people's uncommented source code seem like a joy :D Management added to this by deciding if we're going to offload the email storage to a network storage, we might as well consolidate everything into that shared storage system so we don't have TBs of un-utilized space. So I might not even be able to use your tested XFS + concat solution since it may not be optimal for VM images and databases. As the requirements' changed, I'll stop asking here as it's no longer really relevant just for Dovecot purposes. > You are a perfect candidate for VMware ESX. The HA feature will do > exactly what you want. If one physical node in the cluster dies, HA > automatically restarts the dead VMs on other nodes, transparently. > Clients will will have to reestablish connections, but everything else > will pretty much be intact. Worse case scenario will possibly be a few > corrupted mailboxes that were being written when the hardware crashed. > A SAN is required for such a setup. Thanks for the suggestion, I will need to find some time to look into this although we've mostly been using KVM for virtualization so far. Although the "SAN" part will probably prevent this from being accepted due to cost. >> My lame excuse is that I'm just the web >> dev who got caught holding the server admin potato. > > Baptism by fire. Ouch. What doesn't kill you makes you stronger. ;) True, but I'd hate to be the customer who get to pick up the pieces when things explode due to unintended negligence by a dev trying to level up by multi-classing as an admin. > physical network interface. You can do some of these things with free > Linux hypervisors, but AFAIK the poor management interfaces for them > make the price of ESX seem like a bargain. Unfortunately, the usual kind of customers we have here, spending that kind of budget isn't justifiable. The only reason we're providing email services is because customers wanted freebies and they felt there was no reason why we can't give them emails on our servers, they are all "servers" after all. So I have to make do with OTS commodity parts and free software for the most parts. From achekalin at lazurit.com Mon Apr 9 22:39:40 2012 From: achekalin at lazurit.com (Alexander Chekalin) Date: Mon, 09 Apr 2012 22:39:40 +0300 Subject: [Dovecot] Multiply mailboxes vs one huge In-Reply-To: References: <4F82F90F.8020902@lazurit.com> Message-ID: <4F833AFC.60301@lazurit.com> Hello, Timo, I feel a bit unsure about "which 'date' I mean", since I always consider the only date from Date: header. But which value is used as INTERNALDATE then? As soon as I use (for now) maildir storage type, all the metadata are stored in messages. So I expect Dovecot somehow parse and use Date: field itself, or I'm wrong with it? And also what's about messages without Date header at all? But the Date isn't the worst thing. Look, to have my archive work I setup server-side filter which redirect all messages it processed also to my archive mailbox. This way, each message (after such a redirect) targeted to 'archive at mydomain', instead of its original destination email. The only place I can find out the original recipient is to parse 'Recieved' field(-s). As I think I understand that none of these headers (Date or Received) are to be used for SEARCH anyway, and this was the idea behind creating my own index. But wait, is there any way I can make Dovecot also index additional fields (yes, I talk about 'Received') - then it'll be the best solution! Thank you, Timo, for your work, yours, Alexander 09.04.2012 18:03, Timo Sirainen ???????: > On 9.4.2012, at 17.58, Alexander Chekalin wrote: > >> as I need to store a lot of messages on my IMAP server (order of 900K-1000K; this is an archive for some time, maybe a year or so), I see some "slowness" in dealing with such a huge amount. I mainly need to do searches like "get all messages from user1 at domain1.com to user2 at domain2.tld recieved between date1 and date2". > So by "received between date" you mean the IMAP INTERNALDATE as opposed to Date: header? These kind of searches are looked up from the index/cache files, and the performance should be exactly the same with all of the mailbox formats. It would be useful to figure out what exactly is causing the slowness. Is the SEARCH command slow? Something else? Is the slowness about user CPU, system CPU or disk IO? > From marc at perkel.com Tue Apr 10 04:28:32 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 09 Apr 2012 18:28:32 -0700 Subject: [Dovecot] v2.1.4 released - broken In-Reply-To: References: Message-ID: <4F838CC0.2000900@perkel.com> I'm seeing this immediately after upgrading from 2.1.3 Apr 09 18:22:43 imap(chris at powerpage.org): Error: user chris at powerpage.org: Initialization failed: Initializing mail storage from mail_location setting failed: Home directory not set for user. Can't expand ~/ for mail root dir in: /vhome/powerpage.org/home/chris:INDEX=/email/imap-cache/powerpage.org-chris mail_location = maildir:/vhome/%d/home/%n:INDEX=/email/imap-cache/%d-%n From a.kostyrev at serverc.ru Tue Apr 10 05:37:27 2012 From: a.kostyrev at serverc.ru (=?koi8-r?B?68/T1NnSxdcg4czFy9PBzsTSIOHMxcvTxcXXyd4=?=) Date: Tue, 10 Apr 2012 13:37:27 +1100 Subject: [Dovecot] Authentication mechanism and Password scheme Message-ID: <213B51F00051AE48A9F0E112880177178F794E@Delta.sc.local> Good day! I'm just trying to figure out that my understanding of subject is correct. So, if I want to store passwords in my database encrypted with SSHA512 scheme, my only choice for Authentication mechanism is plaintext? Thanks in advance! -- ? ?????????, ???????? ????????? ????????? ????????????? ??? "??????-?????" ???.: (423) 262-02-62 (???. 2037) ????: (423) 262-02-10 From stan at hardwarefreak.com Tue Apr 10 08:00:19 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 10 Apr 2012 00:00:19 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> Message-ID: <4F83BE63.9050603@hardwarefreak.com> On 4/9/2012 2:15 PM, Emmanuel Noobadmin wrote: > Unfortunately, the usual kind of customers we have here, spending that > kind of budget isn't justifiable. The only reason we're providing > email services is because customers wanted freebies and they felt > there was no reason why we can't give them emails on our servers, they > are all "servers" after all. > > So I have to make do with OTS commodity parts and free software for > the most parts. OTS meaning you build your own systems from components? Too few in the business realm do so today. :( It sounds like budget overrides redundancy then. You can do an NFS cluster with SAN and GFS2, or two servers with their own storage and DRBD mirroring. Here's how to do the latter: http://www.howtoforge.com/high_availability_nfs_drbd_heartbeat The total cost is about the same for each solution as an iSCSI SAN array of drive count X is about the same cost as two JBOD disk arrays of count X*2. Redundancy in this case is expensive no matter the method. Given how infrequent host failures are, and the fact your storage is redundant, it may make more sense to simply keep spare components on hand and swap what fails--PSU, mobo, etc. Interestingly, I designed a COTS server back in January to handle at least 5k concurrent IMAP users, using best of breed components. If you or someone there has the necessary hardware skills, you could assemble this system and simply use it for NFS instead of Dovecot. The parts list: secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=17069985 In case the link doesn't work, the core components are: SuperMicro H8SGL G34 mobo w/dual Intel GbE, 2GHz 8-core Opteron 32GB Kingston REG ECC DDR3, LSI 9280-4i4e, Intel 24 port SAS expander 20 x 1TB WD RE4 Enterprise 7.2K SATA2 drives NORCO RPC-4220 4U 20 Hot-Swap Bays, SuperMicro 865W PSU All other required parts are in the Wish List. I've not written assembly instructions. I figure anyone who would build this knows what s/he is doing. Price today: $5,376.62 Configuring all 20 drives as a RAID10 LUN in the MegaRAID HBA would give you a 10TB net Linux device and 10 stripe spindles of IOPS and bandwidth. Using RAID6 would yield 18TB net and 18 spindles of read throughput, however parallel write throughput will be at least 3-6x slower than RAID10, which is why nobody uses RAID6 for transactional workloads. If you need more transactional throughput you could use 20 WD6000HLHX 600GB 10K RPM WD Raptor drives. You'll get 40% more throughput and 6TB net space with RAID10. They'll cost you $1200 more, or $6,576.62 total. Well worth the $1200 for 40% more throughput, if 6TB is enough. Both of the drives I've mentioned here are enterprise class drives, feature TLER, and are on the LSI MegaRAID SAS hardware compatibility list. The price of the 600GB Raptor has come down considerably since I designed this system, or I'd have used them instead. Anyway, lots of option out there. But $6,500 is pretty damn cheap for a quality box with 32GB RAM, enterprise RAID card, and 20x10K RPM 600GB drives. The MegaRAID 9280-4i4e has an external SFF8088 port For an additional $6,410 you could add an external Norco SAS expander JBOD chassis and 24 more 600GB 10K RPM Raptors, for 13.2TB of total net RAID10 space, and 22 10k spindles of IOPS performance from 44 total drives. That's $13K for a 5K random IOPS, 13TB, 44 drive NFS RAID COTS server solution, $1000/TB, $2.60/IOPS. Significantly cheaper than an HP, Dell, IBM solution of similar specs, each of which will set you back at least 20 large. Note the chassis I've spec'd have single PSUs, not the dual or triple redundant supplies you'll see on branded hardware. With a relatively stable climate controlled environment and a good UPS with filtering, quality single supplies are fine. In fact, in the 4U form factor single supplies are usually more reliable due to superior IC packaging and airflow through the heatsinks, not to mention much quieter. -- Stan From lists at wiesinger.com Tue Apr 10 08:46:31 2012 From: lists at wiesinger.com (Gerhard Wiesinger) Date: Tue, 10 Apr 2012 07:46:31 +0200 Subject: [Dovecot] v2.1.4 released - broken In-Reply-To: <4F838CC0.2000900@perkel.com> References: <4F838CC0.2000900@perkel.com> Message-ID: <4F83C937.1080707@wiesinger.com> On 10.04.2012 03:28, Marc Perkel wrote: > > I'm seeing this immediately after upgrading from 2.1.3 > > Apr 09 18:22:43 imap(chris at powerpage.org): Error: user > chris at powerpage.org: Initialization failed: Initializing mail storage > from mail_location setting failed: Home directory not set for user. > Can't expand ~/ for mail root dir in: > /vhome/powerpage.org/home/chris:INDEX=/email/imap-cache/powerpage.org-chris > > mail_location = maildir:/vhome/%d/home/%n:INDEX=/email/imap-cache/%d-%n > I'm guessing this occurs from the following bugfix: http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e Reverting this patch helps? What is your config (doveconf -n)? Ciao, Gerhard From centos.admin at gmail.com Tue Apr 10 09:09:18 2012 From: centos.admin at gmail.com (Emmanuel Noobadmin) Date: Tue, 10 Apr 2012 14:09:18 +0800 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F83BE63.9050603@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> Message-ID: On 4/10/12, Stan Hoeppner wrote: >> So I have to make do with OTS commodity parts and free software for >> the most parts. > > OTS meaning you build your own systems from components? Too few in the > business realm do so today. :( For the inhouse stuff and budget customers yes, in fact both the email servers are on seconded hardware that started life as something else. I spec HP servers for our app servers to customers who are willing to pay for their own colocated or onsite servers but still there are customers who balk at the cost and so go OTS or virtualized. > SuperMicro H8SGL G34 mobo w/dual Intel GbE, 2GHz 8-core Opteron > 32GB Kingston REG ECC DDR3, LSI 9280-4i4e, Intel 24 port SAS expander > 20 x 1TB WD RE4 Enterprise 7.2K SATA2 drives > NORCO RPC-4220 4U 20 Hot-Swap Bays, SuperMicro 865W PSU > All other required parts are in the Wish List. I've not written > assembly instructions. I figure anyone who would build this knows what > s/he is doing. > > Price today: $5,376.62 This price looks like something I might be able to push through although I'll probably have to go SATA instead of SAS due to cost of keeping spares. > Configuring all 20 drives as a RAID10 LUN in the MegaRAID HBA would give > you a 10TB net Linux device and 10 stripe spindles of IOPS and > bandwidth. Using RAID6 would yield 18TB net and 18 spindles of read > throughput, however parallel write throughput will be at least 3-6x > slower than RAID10, which is why nobody uses RAID6 for transactional > workloads. Not likely to go with RAID 5 or 6 due to concerns about the uncorrectable read errors risks on rebuild with large arrays. Is the MegaRAID being used as the actual RAID controller or just as a HBA? I have been avoiding hardware RAID because of a really bad experience with RAID 5 on an obsolete controller that eventually died without replacement and couldn't be recovered. Since then, it's always been RAID 1 and, after I discovered mdraid, using them as purely HBA with mdraid for the flexibility of being able to just pull the drives into a new system if necessary without having to worry about the controller. > Both of the drives I've mentioned here are enterprise class drives, > feature TLER, and are on the LSI MegaRAID SAS hardware compatibility > list. The price of the 600GB Raptor has come down considerably since I > designed this system, or I'd have used them instead. > > Anyway, lots of option out there. But $6,500 is pretty damn cheap for a > quality box with 32GB RAM, enterprise RAID card, and 20x10K RPM 600GB > drives. > > The MegaRAID 9280-4i4e has an external SFF8088 port For an additional > $6,410 you could add an external Norco SAS expander JBOD chassis and 24 > more 600GB 10K RPM Raptors, for 13.2TB of total net RAID10 space, and 22 > 10k spindles of IOPS performance from 44 total drives. That's $13K for > a 5K random IOPS, 13TB, 44 drive NFS RAID COTS server solution, > $1000/TB, $2.60/IOPS. Significantly cheaper than an HP, Dell, IBM > solution of similar specs, each of which will set you back at least 20 > large. Would this setup work well too for serving up VM images? I've been trying to find a solution for the virtualized app servers images as well but the distributed FSes currently are all bad with random reads/writes it seems. XFS seem to be good with large files like db and vm images with random internal write/read so given my time constraints, it would be nice to have a single configuration that works generally well for all the needs I have to oversee. > Note the chassis I've spec'd have single PSUs, not the dual or triple > redundant supplies you'll see on branded hardware. With a relatively > stable climate controlled environment and a good UPS with filtering, > quality single supplies are fine. In fact, in the 4U form factor single > supplies are usually more reliable due to superior IC packaging and > airflow through the heatsinks, not to mention much quieter. Same reason I do my best to avoid 1U servers, the space/heat issues worries me. Yes, I'm guilty of worrying too much but that had saved me on several occasions. From tss at iki.fi Tue Apr 10 10:10:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 10:10:28 +0300 Subject: [Dovecot] v2.1.4 released - broken In-Reply-To: <4F838CC0.2000900@perkel.com> References: <4F838CC0.2000900@perkel.com> Message-ID: On 10.4.2012, at 4.28, Marc Perkel wrote: > I'm seeing this immediately after upgrading from 2.1.3 > > Apr 09 18:22:43 imap(chris at powerpage.org): Error: user chris at powerpage.org: Initialization failed: Initializing mail storage from mail_location setting failed: Home directory not set for user. Can't expand ~/ for mail root dir in: /vhome/powerpage.org/home/chris:INDEX=/email/imap-cache/powerpage.org-chris > > mail_location = maildir:/vhome/%d/home/%n:INDEX=/email/imap-cache/%d-%n Now would be a good time to add home directories for users. :) Or apply: http://hg.dovecot.org/dovecot-2.1/rev/20e1aa322b1e From tss at iki.fi Tue Apr 10 10:11:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 10:11:05 +0300 Subject: [Dovecot] Authentication mechanism and Password scheme In-Reply-To: <213B51F00051AE48A9F0E112880177178F794E@Delta.sc.local> References: <213B51F00051AE48A9F0E112880177178F794E@Delta.sc.local> Message-ID: On 10.4.2012, at 5.37, ???????? ????????? ?????????? wrote: > Good day! > I'm just trying to figure out that my understanding of subject is correct. > > So, if I want to store passwords in my database encrypted with SSHA512 scheme, > my only choice for Authentication mechanism is plaintext? Yeah, that's correct. From tss at iki.fi Tue Apr 10 10:16:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 10:16:47 +0300 Subject: [Dovecot] Multiply mailboxes vs one huge In-Reply-To: <4F833AFC.60301@lazurit.com> References: <4F82F90F.8020902@lazurit.com> <4F833AFC.60301@lazurit.com> Message-ID: On 9.4.2012, at 22.39, Alexander Chekalin wrote: > Hello, Timo, > > I feel a bit unsure about "which 'date' I mean", since I always consider the only date from Date: header. But which value is used as INTERNALDATE then? As soon as I use (for now) maildir storage type, all the metadata are stored in messages. So I expect Dovecot somehow parse and use Date: field itself, or I'm wrong with it? The INTERNALDATE means the same as "received date", while the Date: header is the "sent date". With mbox format the received date is stored in the separating From-lines. IMAP supports searching and sorting messages by either INTERNALDATE or Date: header > And also what's about messages without Date header at all? The searching just doesn't match those messages then. Sorting falls back to using received date. > But the Date isn't the worst thing. Look, to have my archive work I setup server-side filter which redirect all messages it processed also to my archive mailbox. This way, each message (after such a redirect) targeted to 'archive at mydomain', instead of its original destination email. The only place I can find out the original recipient is to parse 'Recieved' field(-s). > > As I think I understand that none of these headers (Date or Received) are to be used for SEARCH anyway, and this was the idea behind creating my own index. But wait, is there any way I can make Dovecot also index additional fields (yes, I talk about 'Received') - then it'll be the best solution! If you do a SEARCH HEADER Received, then Dovecot adds the Received headers to dovecot.index.cache file and the subsequent searches should be quite fast, although the Received headers increase the cache file's size quite a lot. Also alternatively you can enable full text search indexes (Lucene or Solr) and the search is then done from them. From adrian.minta at gmail.com Tue Apr 10 13:22:18 2012 From: adrian.minta at gmail.com (Adrian Minta) Date: Tue, 10 Apr 2012 13:22:18 +0300 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F83BE63.9050603@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> Message-ID: <4F8409DA.1090202@gmail.com> On 04/10/12 08:00, Stan Hoeppner wrote: > Interestingly, I designed a COTS server back in January to handle at > least 5k concurrent IMAP users, using best of breed components. If you > or someone there has the necessary hardware skills, you could assemble > this system and simply use it for NFS instead of Dovecot. The parts > list: > secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=17069985 Don't forget the Battery Backup Unit for RAID card !!! From pw at wk-serv.de Tue Apr 10 13:44:08 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 10 Apr 2012 12:44:08 +0200 Subject: [Dovecot] Problem with quota warning In-Reply-To: <4F82E585.7030201@wk-serv.de> References: <4F82E585.7030201@wk-serv.de> Message-ID: <4F840EF8.7080109@wk-serv.de> Hi, I switched from dovecot-lda to lmtp for my quota warnings but there is something wrong with the parameters: /usr/local/libexec/dovecot/lmtp -d $USER -o "plugin/quota=dict:User quota::file:%h/mdbox/dovecot-quota:noenforcing" /usr/local/libexec/dovecot/lmtp: invalid option -- 'd' /usr/local/libexec/dovecot/lmtp -r $USER -o "plugin/quota=dict:User quota::file:%h/mdbox/dovecot-quota:noenforcing" /usr/local/libexec/dovecot/lmtp: invalid option -- 'r' Both parameters are listed on http://wiki2.dovecot.org/LMTP. So is this my fault or wrong information on the page? Patrick From tss at iki.fi Tue Apr 10 15:31:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 15:31:13 +0300 Subject: [Dovecot] Problem with quota warning In-Reply-To: <4F840EF8.7080109@wk-serv.de> References: <4F82E585.7030201@wk-serv.de> <4F840EF8.7080109@wk-serv.de> Message-ID: <3F52251E-6411-448E-B397-207EFD176478@iki.fi> On 10.4.2012, at 13.44, Patrick Westenberg wrote: > I switched from dovecot-lda to lmtp for my quota warnings but there is something wrong with the parameters: > > /usr/local/libexec/dovecot/lmtp -d $USER -o "plugin/quota=dict:User quota::file:%h/mdbox/dovecot-quota:noenforcing" This is not how to use LMTP. You need to tell your MTA to connect to it via UNIX socket or via TCP. You don't execute the LMTP. > Both parameters are listed on http://wiki2.dovecot.org/LMTP. > So is this my fault or wrong information on the page? It says "Compared to dovecot-lda parameters", so it's describing where dovecot-lda parameters are taken with LMTP. I guess it could be said in some clearer way, but I don't really know how. From pw at wk-serv.de Tue Apr 10 15:54:45 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 10 Apr 2012 14:54:45 +0200 Subject: [Dovecot] Problem with quota warning In-Reply-To: <3F52251E-6411-448E-B397-207EFD176478@iki.fi> References: <4F82E585.7030201@wk-serv.de> <4F840EF8.7080109@wk-serv.de> <3F52251E-6411-448E-B397-207EFD176478@iki.fi> Message-ID: <4F842D95.3070005@wk-serv.de> Timo Sirainen schrieb: > On 10.4.2012, at 13.44, Patrick Westenberg wrote: > >> I switched from dovecot-lda to lmtp for my quota warnings but there is something wrong with the parameters: >> >> /usr/local/libexec/dovecot/lmtp -d $USER -o "plugin/quota=dict:User quota::file:%h/mdbox/dovecot-quota:noenforcing" > > This is not how to use LMTP. You need to tell your MTA to connect to it via UNIX socket or via TCP. You don't execute the LMTP. There is no MTA installed on this system. This is the backend server. dovecot-lda doesn't work too, look at my first post with this subject. From tss at iki.fi Tue Apr 10 15:58:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 15:58:16 +0300 Subject: [Dovecot] Problem with quota warning In-Reply-To: <4F82E585.7030201@wk-serv.de> References: <4F82E585.7030201@wk-serv.de> Message-ID: <28D02445-BBDC-41F5-A7AA-2AD9C5E5093F@iki.fi> On 9.4.2012, at 16.35, Patrick Westenberg wrote: > cat << EOF | /usr/local/libexec/dovecot/dovecot-lda -d $USER -o "plugin/quota=dict:User quota::file:%h/mdbox/dovecot-quota:noenforcing" The :noenforcing isn't in the right location. dict:User quota::noenforcing:file:%h/... From pw at wk-serv.de Tue Apr 10 16:26:11 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 10 Apr 2012 15:26:11 +0200 Subject: [Dovecot] Problem with quota warning In-Reply-To: <28D02445-BBDC-41F5-A7AA-2AD9C5E5093F@iki.fi> References: <4F82E585.7030201@wk-serv.de> <28D02445-BBDC-41F5-A7AA-2AD9C5E5093F@iki.fi> Message-ID: <4F8434F3.3080708@wk-serv.de> Timo Sirainen schrieb: > The :noenforcing isn't in the right location. > > dict:User quota::noenforcing:file:%h/... Aww.. thank you! From luc at bigdata.nb.ca Tue Apr 10 16:45:13 2012 From: luc at bigdata.nb.ca (Luc Belliveau) Date: Tue, 10 Apr 2012 10:45:13 -0300 Subject: [Dovecot] old messages being delivered to blackberry Message-ID: <4F843969.6040806@bigdata.nb.ca> Several users are reporting old messages (sometime years old) are being delivered as new on their blackberry devices... I've confirmed that this happens to all blackberry users... I believe the problem is related to this error: Maildir: Expunged message reappeared, giving a new UID In fact, this is the only mail related error message on the server... the mail is stored on a glusterfs share, and I'm using director to prevent simultaneous access... although I wasn't able to get LMTP to work with Director... to simplify things, I've temporarily only enabled 1 dovecot server anyway, to get everything else working first... I'll fiddle with lmtp later... so right now only 1 server is accessing the mailboxes. Delivery is also done from that one server, from postfix. Possibly caused by GlusterFS, the indexes were still getting corrupted, even with only 1 server... so I've moved the indexes to a local directory on each server. I'm no longer getting index related errors... but the expunged error is still occuring... and the error seems to happen a few moments before the BB gets a bunch of messages... so it's very likely related. I would like to continue using GlusterFS if possible; I haven't found any indication that it is outright unsupported, as long as I'm using Director... I am using dovecot 2.0.16. Any help would be appreciated! Here is my configuration: # 2.0.16: /etc/dovecot/dovecot.conf # OS: Linux 3.1.9-1.4-desktop i686 openSUSE 12.1 (i586) auth_mechanisms = plain login disable_plaintext_auth = no log_timestamp = "%Y-%m-%d %H:%M:%S " mail_fsync = always mail_location = maildir:~/Maildir:INDEX=/var/indexes/%u mail_nfs_storage = yes mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve = /var/vmail/%d/%n/.sieve } postmaster_address = postmaster at bigdata.nb.ca service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } user = root } service lmtp { inet_listener lmtp { port = 24 } unix_listener /var/spool/postfix/private/dovecot-lmtp { mode = 0660 user = postfix } } ssl_ca = References: <4F843969.6040806@bigdata.nb.ca> Message-ID: <5CDED2FA-3654-4378-A2D3-8A0BFAF7B5CC@iki.fi> On 10.4.2012, at 16.45, Luc Belliveau wrote: > Several users are reporting old messages (sometime years old) are being delivered as new on their blackberry devices... I've confirmed that this happens to all blackberry users... I believe the problem is related to this error: > > Maildir: Expunged message reappeared, giving a new UID This means: 1. readdir() returns maildir file X 2. Later readdir() doesn't return file X, so Dovecot thinks the mail is expunged and also tells the client the mail is expunged 3. Later readdir() once again returns file X, so Dovecot can't really do anything except treat it as a new mail. > Possibly caused by GlusterFS, the indexes were still getting corrupted, even with only 1 server... so I've moved the indexes to a local directory on each server. I'm no longer getting index related errors... but the expunged error is still occuring... and the error seems to happen a few moments before the BB gets a bunch of messages... so it's very likely related. > > I would like to continue using GlusterFS if possible; I haven't found any indication that it is outright unsupported, as long as I'm using Director... If glusterfs can't do readdir() reliably, there's not a whole lot you can do.. Except maybe switch to another mailbox format. But if you're getting index corruption also with 1 Dovecot server, it doesn't look very promising. From apm at one.com Tue Apr 10 18:28:29 2012 From: apm at one.com (Peter Mogensen) Date: Tue, 10 Apr 2012 17:28:29 +0200 Subject: [Dovecot] 2.0/2.1 - different behavior for LIST-EXTENDED Message-ID: <4F84519D.9070104@one.com> Hi Timo, We are sitting here wondering if this difference in behaviour between dovecot 2.0.17 and 2.1.3 is intended. When you create a folder, subscribe to it and rename it (without changing the subscription) these are the behaviours: For 2.0.17: . list (SUBSCRIBED) "" "*" RETURN (STATUS (MESSAGES)) * LIST (\Subscribed \NonExistent) "." "INBOX.test" For 2.1.3: . list (SUBSCRIBED) "" "*" RETURN (STATUS (MESSAGES)) * LIST (\Subscribed) "." "INBOX.test" * NO Mailbox doesn't exist: test If you don't use rfc5819 the folder will just get silently ignored by dovecot 2.1.x, but if you actually try to get the number of messages you'll get the error. It seems to me from reading rfc5258 that the 2.0.x behaviour is the correct ?? /Peter From s.locatelli at hexanet.fr Tue Apr 10 18:29:45 2012 From: s.locatelli at hexanet.fr (=?ISO-8859-1?Q?St=E9phane_Locatelli?=) Date: Tue, 10 Apr 2012 17:29:45 +0200 Subject: [Dovecot] Per IP virtual domain Message-ID: Hello, My dovecot server (2.0.18 on FreeBSD 9) has multiple interface. I would like to configure the authentication (pop and imap) to append different domain according to the local IP address. I've tested this: local 10.0.0.1 { auth_default_realm = domain1.com } local 10.0.0.2 { auth_default_realm = domain2.com } But it doesn't work. I get this error message: doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf line 58: Auth settings not supported inside local/remote blocks: auth_default_realm -- Stephane Locatelli From tss at iki.fi Tue Apr 10 19:04:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 19:04:59 +0300 Subject: [Dovecot] 2.0/2.1 - different behavior for LIST-EXTENDED In-Reply-To: <4F84519D.9070104@one.com> References: <4F84519D.9070104@one.com> Message-ID: <73DC1DBA-930F-41F3-9220-68C2D53339C9@iki.fi> On 10.4.2012, at 18.28, Peter Mogensen wrote: > For 2.0.17: > . list (SUBSCRIBED) "" "*" RETURN (STATUS (MESSAGES)) > * LIST (\Subscribed \NonExistent) "." "INBOX.test" > > For 2.1.3: > . list (SUBSCRIBED) "" "*" RETURN (STATUS (MESSAGES)) > * LIST (\Subscribed) "." "INBOX.test" > * NO Mailbox doesn't exist: test This was fixed in v2.1.4. From tss at iki.fi Tue Apr 10 19:06:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 19:06:38 +0300 Subject: [Dovecot] Per IP virtual domain In-Reply-To: References: Message-ID: <39EBDB8E-C0E2-43D5-9548-6CDF274D51E3@iki.fi> On 10.4.2012, at 18.29, St?phane Locatelli wrote: > My dovecot server (2.0.18 on FreeBSD 9) has multiple interface. I would > like to configure the authentication (pop and imap) to append different > domain according to the local IP address. > > I've tested this: > > local 10.0.0.1 { > auth_default_realm = domain1.com > } > > local 10.0.0.2 { > auth_default_realm = domain2.com > } > > But it doesn't work. I get this error message: > > doveconf: Fatal: Error in configuration file > /usr/local/etc/dovecot/dovecot.conf line 58: Auth settings not supported > inside local/remote blocks: auth_default_realm As it says, currently they aren't supported. It would need some extra code. Depending on your passdb you may be able to do this in your passdb configuration. From tss at iki.fi Tue Apr 10 19:10:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 19:10:04 +0300 Subject: [Dovecot] Username from rfc822Name subject alternative name In-Reply-To: References: Message-ID: <578B8B92-3EC4-4137-B4D0-3B68A767CAB5@iki.fi> On 9.4.2012, at 16.51, ?????? ????? wrote: > I'm looking into adding support for extracting the username from client certificate's rfc822Name (from the subjectAltName extension). > > The question I have is what would be the best approach to do this? Current implementation has a kind of clean code since it just goes through the subject name, extracting the values with X509_NAME_get_text_by_NID (while NID is obtained with OBJ_txt2nid). If I were to add this, it's bound to make the code a little bit more complicated since SAN's can't be retrieved in the same way. > > So far in terms of options I have, I can see the following: > > 1. Create a distinct configuration option for the ssl_cert_username_field (i.e. specify something like "sanrfc822Name" to have Dovecot extract the username from the designated alternative name). I'm not sure if this is a good idea, but what about: ssl_cert_username_field = subjectAltName:rfc822Name > 2. Make the current code fail-over to rfc822Name SAN if emailAddress is provided for ssl_cert_username (less invasion in code, but less flexibility as well). Automatic failover seems dangerous. From s.locatelli at hexanet.fr Tue Apr 10 19:12:41 2012 From: s.locatelli at hexanet.fr (=?ISO-8859-1?Q?St=E9phane_Locatelli?=) Date: Tue, 10 Apr 2012 18:12:41 +0200 Subject: [Dovecot] Per IP virtual domain In-Reply-To: <39EBDB8E-C0E2-43D5-9548-6CDF274D51E3@iki.fi> References: <39EBDB8E-C0E2-43D5-9548-6CDF274D51E3@iki.fi> Message-ID: 2012/4/10 Timo Sirainen > > As it says, currently they aren't supported. It would need some extra > code. Depending on your passdb you may be able to do this in your passdb > configuration. Currently I'm using dovecot 1.x on my production server with mysql passdb. I have ugly SQL queries to do this and my idea was to upgrade to dovecot 2.x to avoid this. password_query = SELECT mbox AS user , passwd as password FROM mbox WHERE mbox=IF('%s' = 'smtp', '%u', IF('%u' like '%%@%%', '%u', concat('%u', case '%l' WHEN '10.0.0.1' THEN '@domain1.com' WHEN '10.0.0.2' THEN '@domain2.com' WHEN '10.0.0.3' THEN '@domain3.com' END ) ) ) AND passwd IS NOT NULL AND (status='active' OR status ='inactive'); -- Stephane Locatelli From lists at wildgooses.com Tue Apr 10 19:35:09 2012 From: lists at wildgooses.com (Ed W) Date: Tue, 10 Apr 2012 17:35:09 +0100 Subject: [Dovecot] Authentication mechanism and Password scheme In-Reply-To: References: <213B51F00051AE48A9F0E112880177178F794E@Delta.sc.local> Message-ID: <4F84613D.8040508@wildgooses.com> On 10/04/2012 08:11, Timo Sirainen wrote: > On 10.4.2012, at 5.37, ???????? ????????? ?????????? wrote: > >> Good day! >> I'm just trying to figure out that my understanding of subject is correct. >> >> So, if I want to store passwords in my database encrypted with SSHA512 scheme, >> my only choice for Authentication mechanism is plaintext? > Yeah, that's correct. > Does dovecot 2.0 also support SCRAM-SHA? I only mention because it's come up on my radar recently and as I understand it, it solves the issue of either having - plain text db of passwords, encrypted login - encrypted db of passwords, plaintext login With SCRAM you have both sides "encrypted". (Clearly it's also desirable that the hash algorithm is well chosen to be resistant to bruteforce, so some might argue that bcrypt/scrypt is even more desirable since there is not yet a GPU implementation - However, at least SHA is a decent stab at things) Can you confirm my understanding is correct? Next question is whether any current mail client supports SCRAM..? Regards Ed W From gwurster at rim.com Tue Apr 10 20:17:23 2012 From: gwurster at rim.com (Glenn Wurster) Date: Tue, 10 Apr 2012 17:17:23 +0000 Subject: [Dovecot] Bug in HIGHESTMODSEQ when MODSEQ tracking is not enabled for version 2.0.18. Message-ID: <48A61B0976102F488C64EF8A8F1E929D074BBA@XMB118CNC.rim.net> Greetings, I'm new to the list. I have encountered what I believe to be a bug in the return of HIGHESTMODSEQ when talking to a IMAP client. HIGHESTMODSEQ does not appear to be incrementing when a new message arrives. The exact steps I took are: 1) Create a new mail account 2) Send a mail to the account, check that you can receive the email. Send a second mail to the account. 3) perform a SELECT on the mailbox from a new session. PMLK4 SELECT INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 2 EXISTS * 1 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1334074920] UIDs valid * OK [UIDNEXT 3] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest PMLK4 OK [READ-WRITE] Select completed. 4) Retrieve the mail. Then perform another SELECT on the mailbox from yet another session. LLEP4 SELECT INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 2 EXISTS * 0 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1334074920] UIDs valid * OK [UIDNEXT 3] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest LLEP4 OK [READ-WRITE] Select completed. 5) Send another mail to the account, and check SELECT again. FKDA4 SELECT INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 3 EXISTS * 1 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1334074920] UIDs valid * OK [UIDNEXT 4] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest FKDA4 OK [READ-WRITE] Select completed. It appears as if HIGHESTMODSEQ is not being updated. I can get HIGHESTMODSEQ to start updating correctly if I send a "UID FETCH 1 MODSEQ" or similar command, which appears to enable MODSEQ tracking at the server (according to the comment around line 173 in file src/lib-index/mail-index-modseq.c), but until that command is sent, MODSEQ tracking is not enabled and hence HIGHESTMODSEQ is always going to return 1. According to RFC4551, the combination of HIGHESTMODSEQ and UIDVALIDITY should be sufficient to determine if the metadata associated with the mailbox has changed, but in this case looking at only those two parameters does not yield sufficient information about changes in the mailbox. The mail client I'm using relies on the combination of HIGHESTMODSEQ and UIDVALIDITY to determine if there are changes in the mailbox, and hence does not see new mail come in. It seems that Dovecot should not be returning HIGHESTMODSEQ in response to a command if MODSEQ tracking is not enabled, but I could be wrong. I've attached my configuration (it's Dovecot 2.0.18 running on Debian Stable). Let me know if you need more information. Glenn. --------------------------------------------------------------------- This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful. -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot.conf Type: application/octet-stream Size: 452 bytes Desc: dovecot.conf URL: From tss at iki.fi Tue Apr 10 21:21:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 21:21:25 +0300 Subject: [Dovecot] Authentication mechanism and Password scheme In-Reply-To: <4F84613D.8040508@wildgooses.com> References: <213B51F00051AE48A9F0E112880177178F794E@Delta.sc.local> <4F84613D.8040508@wildgooses.com> Message-ID: <8AB6491F-9277-45BF-8FF0-2C90D7941E8A@iki.fi> On 10.4.2012, at 19.35, Ed W wrote: > Does dovecot 2.0 also support SCRAM-SHA? v2.1 does. > I only mention because it's come up on my radar recently and as I understand it, it solves the issue of either having > - plain text db of passwords, encrypted login > - encrypted db of passwords, plaintext login > > With SCRAM you have both sides "encrypted". The same way as with DIGEST-MD5 and several others. Each mechanism requires that the server-side password is saved using a hash specific to that auth mechanism, none of them support generic MD5/SHA/etc hashes or other mechanisms' hashes. Looks like Dovecot's current SCRAM-SHA1 supports only plaintext passwords, but it would be possible to add SCRAM-SHA1 password scheme similar to others. From tss at iki.fi Tue Apr 10 21:42:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 21:42:28 +0300 Subject: [Dovecot] Bug in HIGHESTMODSEQ when MODSEQ tracking is not enabled for version 2.0.18. In-Reply-To: <48A61B0976102F488C64EF8A8F1E929D074BBA@XMB118CNC.rim.net> References: <48A61B0976102F488C64EF8A8F1E929D074BBA@XMB118CNC.rim.net> Message-ID: <11BD08AA-2E29-48DF-973D-4D5B4CCE2FF7@iki.fi> On 10.4.2012, at 20.17, Glenn Wurster wrote: > It appears as if HIGHESTMODSEQ is not being updated. I can get HIGHESTMODSEQ to start updating correctly if I send a "UID FETCH 1 MODSEQ" or similar command, which appears to enable MODSEQ tracking at the server (according to the comment around line 173 in file src/lib-index/mail-index-modseq.c), but until that command is sent, MODSEQ tracking is not enabled and hence HIGHESTMODSEQ is always going to return 1. Yes, modseqs aren't tracked in a mailbox until client expresses an interest for them. It would be a waste of disk space to save them since 99% of users don't need them. > According to RFC4551, the combination of HIGHESTMODSEQ and UIDVALIDITY should be sufficient to determine if the metadata associated with the mailbox has changed, but in this case looking at only those two parameters does not yield sufficient information about changes in the mailbox. The mail client I'm using relies on the combination of HIGHESTMODSEQ and UIDVALIDITY to determine if there are changes in the mailbox, and hence does not see new mail come in. Yeah, it does seem that the RFC says that.. > It seems that Dovecot should not be returning HIGHESTMODSEQ in response to a command if MODSEQ tracking is not enabled, but I could be wrong. I've attached my configuration (it's Dovecot 2.0.18 running on Debian Stable). RFC 4551 says that HIGHESTMODSEQ or NOMODSEQ MUST be returned. Hmm. Perhaps: 1) If the session is known to have modseqs enabled, immediately enable modseqs for newly created mailboxes 2) If a mailbox doesn't have modseqs enabled, return NOMODSEQ. This isn't ideal, but seems like the only possibility. From jeetuindian at gmail.com Tue Apr 10 22:07:48 2012 From: jeetuindian at gmail.com (Jitendra Bhaskar) Date: Wed, 11 Apr 2012 00:37:48 +0530 Subject: [Dovecot] Dsync test on 2.1.3 Message-ID: Hi, I am using dovecot version 2.1.3 on centos. I tested dsync from my live server to one of backup server. But I got that its working for most of the user and there is problem for few user. Some users got duplicated mail to their mailbox and I saw there is some extra folders i.e Sent Items_* , Draft_* and Trash_* in few users mail box. and I synced only from main server to backup server. I didn't enable mail_plugin in backup server. My configuration for testing of Dsync on master server is as: mail_plugins = $mail_plugins notify replication service aggregator { fifo_listener replication-notify-fifo { user = vmail mode = 0666 } unix_listener replication-notify { user = vmail mode = 0666 } } service replicator { # start replication at startup process_min_avail = 1 } plugin { #host1 replicates to host2 mail_replica = remote:root at backup_server } #dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -l%{lock_timeout} -n%{namespace} dsync_remote_cmd = ssh -l root %{host} doveadm dsync-server -u%{user} -l%{10} -n%{namespace} service doveadm { user = root } And in backup server : service doveadm { user = root } 1. Problem is I am not getting why some user getting lots of duplicate mails. 2. Why duplicate folders like Sent Items_* and Trash_* created for few users 3. I used only one way syncing then why both replicated. Note: i am using root only for testing purpose * Thanks & Regards * *Jitendra Kumar Bhaskar* Cell:- +91 7306311531 +91 8102997821 From fxmulder at gmail.com Tue Apr 10 23:19:26 2012 From: fxmulder at gmail.com (James Devine) Date: Tue, 10 Apr 2012 14:19:26 -0600 Subject: [Dovecot] lazy_expunge questions Message-ID: I'm messing around with lazy expunge and I just had a couple of questions. If I delete messages or folders with messages, I see them being moved to the expunge namespace, but if I delete an empty folder, all traces of it seem to disappear. Is there a way to move empty folders to the expunge namespace also? My second question is in regards to restoring folders. If I try to execute the imap command 'RENAME EXPUNGED/TEST INBOX/TEST' I get 'NO [CANNOT] Can't rename mailboxes to/from expunge namespace'. Just wondering if this is possible or do I need to recreate the folder and then move messages into it. From gwurster at rim.com Tue Apr 10 23:50:07 2012 From: gwurster at rim.com (Glenn Wurster) Date: Tue, 10 Apr 2012 20:50:07 +0000 Subject: [Dovecot] Bug in HIGHESTMODSEQ when MODSEQ tracking is not enabled for version 2.0.18. In-Reply-To: <11BD08AA-2E29-48DF-973D-4D5B4CCE2FF7@iki.fi> References: <48A61B0976102F488C64EF8A8F1E929D074BBA@XMB118CNC.rim.net> <11BD08AA-2E29-48DF-973D-4D5B4CCE2FF7@iki.fi> Message-ID: <48A61B0976102F488C64EF8A8F1E929D074E9F@XMB118CNC.rim.net> > Yes, modseqs aren't tracked in a mailbox until client expresses an > interest for them. It would be a waste of disk space to save them > since 99% of users don't need them. Makes sense, our mail client gets caught in the middle though, because it uses HIGHESTMODSEQ to track mailbox updates without using MODSEQ options on SELECT/FETCH to track message updates. > RFC 4551 says that HIGHESTMODSEQ or NOMODSEQ MUST be returned. Hmm. > Perhaps: > > 1) If the session is known to have modseqs enabled, immediately enable > modseqs for newly created mailboxes MODSEQ seems to be enabled on the mailbox, not on the session. For disconnected IMAP, there will frequently be new sessions established, and so we may not have a good indicator of whether the client is using MODSEQ (based on my reading). The current approach of looking for a MODSEQ related command seems to work in most cases, and so I'd be tempted to keep the current functionality (minus the incorrect reporting of HIGHESTMODSEQ if we can find a solution). > 2) If a mailbox doesn't have modseqs enabled, return NOMODSEQ. This > isn't ideal, but seems like the only possibility. The RFC also states that if we return NOMODSEQ we'd have to return a tagged BAD response to "UID FETCH 1 MODSEQ", which appears to one of the commands that enables MODSEQ for Dovecot ("SELECT INBOX (CONDSTORE)" also enables it...). What about returning a BAD response and at the same time start tracking MODSEQ so that future SELECT commands would return HIGHESTMODSEQ? Do we know what email clients are using CONDSTORE options and how they'd react to a mailbox suddenly having MODSEQ capabilities after we just told them it didn't? Glenn. --------------------------------------------------------------------- This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful. From branko at majic.rs Wed Apr 11 00:07:54 2012 From: branko at majic.rs (=?UTF-8?B?0JHRgNCw0L3QutC+INCc0LDRmNC40Zs=?=) Date: Tue, 10 Apr 2012 23:07:54 +0200 Subject: [Dovecot] Username from rfc822Name subject alternative name In-Reply-To: <578B8B92-3EC4-4137-B4D0-3B68A767CAB5@iki.fi> References: <578B8B92-3EC4-4137-B4D0-3B68A767CAB5@iki.fi> Message-ID: <20120410230754.1ab2831b@majic.rs> Another solution could be having another property that specifies whether the information should be pulled in from subject or subjectAltName. But I think what you've proposed would be ok as well, and it'd allow for future expansion too (if anyone decides to use some other SAN). I can get working on adding the support for this (might take a couple of days until I sort it out - getting quite busy here :). Would a patch both against the 1.x and 2.x be welcome? ???? Tue, 10 Apr 2012 19:10:04 +0300 Timo Sirainen ??????: > On 9.4.2012, at 16.51, ?????? ????? wrote: > > > I'm looking into adding support for extracting the username from > > client certificate's rfc822Name (from the subjectAltName extension). > > > > The question I have is what would be the best approach to do this? > > Current implementation has a kind of clean code since it just goes > > through the subject name, extracting the values with > > X509_NAME_get_text_by_NID (while NID is obtained with OBJ_txt2nid). > > If I were to add this, it's bound to make the code a little bit > > more complicated since SAN's can't be retrieved in the same way. > > > > So far in terms of options I have, I can see the following: > > > > 1. Create a distinct configuration option for the > > ssl_cert_username_field (i.e. specify something like > > "sanrfc822Name" to have Dovecot extract the username from the > > designated alternative name). > > I'm not sure if this is a good idea, but what about: > > ssl_cert_username_field = subjectAltName:rfc822Name > > > 2. Make the current code fail-over to rfc822Name SAN if > > emailAddress is provided for ssl_cert_username (less invasion in > > code, but less flexibility as well). > > Automatic failover seems dangerous. -- Branko Majic Jabber: branko at majic.rs Please use only Free formats when sending attachments to me. ?????? ????? ?????: branko at majic.rs ????? ??? ?? ??????? ?????? ????????? ? ????????? ?????????. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From robert at schetterer.org Wed Apr 11 00:08:56 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 10 Apr 2012 23:08:56 +0200 Subject: [Dovecot] sieve replace dot string for foldername Message-ID: <4F84A168.5090700@schetterer.org> Hi, i am playing about mail archive this is what allready works for me require ["variables","date","fileinto","mailbox","envelope","subaddress","regex"]; # Extract date info if currentdate :matches "year" "*" { set "year" "${1}"; } if currentdate :matches "month" "*" { set "month" "${1}"; } if currentdate :matches "day" "*" { set "day" "${1}"; } if envelope :detail :matches "to" "*" { set "to" "${1}"; } if true { fileinto :create "INBOX.${year}.${month}.${day}.${to}"; stop; } but ${to} may have a dot included, foldernames with dot are not allowed in my dove setup yet ( and i dont want to change it ) i am looking for a way ( guess regex ) for replace a dot in string replaced by a space char on the fly -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Wed Apr 11 00:44:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 00:44:04 +0300 Subject: [Dovecot] Username from rfc822Name subject alternative name In-Reply-To: <20120410230754.1ab2831b@majic.rs> References: <578B8B92-3EC4-4137-B4D0-3B68A767CAB5@iki.fi> <20120410230754.1ab2831b@majic.rs> Message-ID: <83E9DEB2-6986-4097-95EF-1532D1FB873F@iki.fi> I can add the patch to v2.1, but probably not to earlier versions. On 11.4.2012, at 0.07, ?????? ????? wrote: > Another solution could be having another property that specifies > whether the information should be pulled in from subject or > subjectAltName. > > But I think what you've proposed would be ok as well, and it'd allow > for future expansion too (if anyone decides to use some other SAN). I > can get working on adding the support for this (might take a couple > of days until I sort it out - getting quite busy here :). > > Would a patch both against the 1.x and 2.x be welcome? > > ???? Tue, 10 Apr 2012 19:10:04 +0300 > Timo Sirainen ??????: > >> On 9.4.2012, at 16.51, ?????? ????? wrote: >> >>> I'm looking into adding support for extracting the username from >>> client certificate's rfc822Name (from the subjectAltName extension). >>> >>> The question I have is what would be the best approach to do this? >>> Current implementation has a kind of clean code since it just goes >>> through the subject name, extracting the values with >>> X509_NAME_get_text_by_NID (while NID is obtained with OBJ_txt2nid). >>> If I were to add this, it's bound to make the code a little bit >>> more complicated since SAN's can't be retrieved in the same way. >>> >>> So far in terms of options I have, I can see the following: >>> >>> 1. Create a distinct configuration option for the >>> ssl_cert_username_field (i.e. specify something like >>> "sanrfc822Name" to have Dovecot extract the username from the >>> designated alternative name). >> >> I'm not sure if this is a good idea, but what about: >> >> ssl_cert_username_field = subjectAltName:rfc822Name >> >>> 2. Make the current code fail-over to rfc822Name SAN if >>> emailAddress is provided for ssl_cert_username (less invasion in >>> code, but less flexibility as well). >> >> Automatic failover seems dangerous. > > -- > Branko Majic > Jabber: branko at majic.rs > Please use only Free formats when sending attachments to me. > > ?????? ????? > ?????: branko at majic.rs > ????? ??? ?? ??????? ?????? ????????? ? ????????? ?????????. From stephan at rename-it.nl Wed Apr 11 01:06:48 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 11 Apr 2012 00:06:48 +0200 Subject: [Dovecot] sieve replace dot string for foldername In-Reply-To: <4F84A168.5090700@schetterer.org> References: <4F84A168.5090700@schetterer.org> Message-ID: <4F84AEF8.8000504@rename-it.nl> On 4/10/2012 11:08 PM, Robert Schetterer wrote: > Hi, i am playing about mail archive > > this is what allready works for me > > require > ["variables","date","fileinto","mailbox","envelope","subaddress","regex"]; > > # Extract date info > if currentdate :matches "year" "*" { set "year" "${1}"; } > if currentdate :matches "month" "*" { set "month" "${1}"; } > if currentdate :matches "day" "*" { set "day" "${1}"; } > if envelope :detail :matches "to" "*" { set "to" "${1}"; } > > if true { > fileinto :create "INBOX.${year}.${month}.${day}.${to}"; > stop; > } > > but ${to} may have a dot included, foldernames > with dot are not allowed in my dove setup yet ( and i dont want to > change it ) > > i am looking for a way ( guess regex ) > for replace a dot in string replaced by a space char > on the fly Hmm, no, I see no possibilities for that really. Sieve doesn't have such advanced string manipulation capabilities unfortunately. You could use the vnd.dovecot.execute extension (extprograms plugin) for that, but I wouldn't call that a nice solution by any means. Regards, Stephan. From robert at schetterer.org Wed Apr 11 01:44:33 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 11 Apr 2012 00:44:33 +0200 Subject: [Dovecot] sieve replace dot string for foldername In-Reply-To: <4F84AEF8.8000504@rename-it.nl> References: <4F84A168.5090700@schetterer.org> <4F84AEF8.8000504@rename-it.nl> Message-ID: <4F84B7D1.1050507@schetterer.org> Am 11.04.2012 00:06, schrieb Stephan Bosch: > On 4/10/2012 11:08 PM, Robert Schetterer wrote: >> Hi, i am playing about mail archive >> >> this is what allready works for me >> >> require >> ["variables","date","fileinto","mailbox","envelope","subaddress","regex"]; >> >> >> # Extract date info >> if currentdate :matches "year" "*" { set "year" "${1}"; } >> if currentdate :matches "month" "*" { set "month" "${1}"; } >> if currentdate :matches "day" "*" { set "day" "${1}"; } >> if envelope :detail :matches "to" "*" { set "to" "${1}"; } >> >> if true { >> fileinto :create "INBOX.${year}.${month}.${day}.${to}"; >> stop; >> } >> >> but ${to} may have a dot included, foldernames >> with dot are not allowed in my dove setup yet ( and i dont want to >> change it ) >> >> i am looking for a way ( guess regex ) >> for replace a dot in string replaced by a space char >> on the fly > > Hmm, no, I see no possibilities for that really. Sieve doesn't have such > advanced string manipulation capabilities unfortunately. > > You could use the vnd.dovecot.execute extension (extprograms plugin) for > that, but I wouldn't call that a nice solution by any means. > > Regards, > > Stephan. > > > Thx Stephan, so i dont have to search anymore perhaps i ll use listescape plugin for dot foldernames no idea if it breaks existing setup stuff yet -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From lee at standen.id.au Wed Apr 11 06:54:35 2012 From: lee at standen.id.au (Lee Standen) Date: Wed, 11 Apr 2012 11:54:35 +0800 Subject: [Dovecot] F5 SLB iRule - POP3 TLS Offload & Add Realm Message-ID: Hey Guys, I figured someone might have use of this at some point. I've created an iRule for our F5 Load Balancer which performs the following tasks: - Appends STLS to the POP3 Capability list (it does this blindly, so expects the backend POP3 server to not return this) - Watches for the STLS command and initiates SSL negotiation with the client (leaves the connection to the backend POP3 server unencrypted) - Watches for the USER command (login attempt) and appends a realm if the user hasn't supplied one - Stops watching once a successful login occurs It's designed to be attached to port 110 and port 995 at the same time, and will disable SSL if the user is connecting to port 110 until the STLS command is issued. This is a first pass, so there's probably room for improvement, but it does seem to work. This will require LTM v10 (at least), as the SSL::collect command didn't exist prior to that version. Here's the rule: when CLIENT_ACCEPTED { if { [TCP::local_port clientside] == "110" } { # Disable SSL if we're on port 110 (as we're doing TLS Offload) set secure 0 SSL::disable } else { set secure 1 } set realm "mydomain.com" set debug 0 set loggedin 0 } when SERVER_CONNECTED { TCP::collect } when CLIENTSSL_HANDSHAKE { SSL::collect set secure 1 } when CLIENTSSL_DATA { if { $debug } { log local0. "CLIENT DATA: [SSL::payload]" } set lcpayload [string tolower [SSL::payload]] if { $lcpayload starts_with "user" } { scan [SSL::payload] {%s%s} command user set pos [string first $user [SSL::payload] 0] SSL::payload replace $pos [string length $user] "$user@$realm" if { $debug } { log local0. [SSL::payload] } } SSL::release if { $loggedin == 0 } { SSL::collect } } when CLIENT_DATA { if { $debug } { log local0. "CLIENT DATA: [TCP::payload]" } set lcpayload [string tolower [TCP::payload]] if { $lcpayload starts_with "stls" } { TCP::respond "+OK Begin TLS negotiation\r\n" TCP::payload replace 0 [TCP::payload length] "" TCP::release SSL::enable return } elseif { $lcpayload starts_with "user" } { scan [TCP::payload] {%s%s} command user set pos [string first $user [TCP::payload] 0] TCP::payload replace $pos [string length $user] "$user@$realm" if { $debug } { log local0. [TCP::payload] } } TCP::release if { $loggedin == 0 } { TCP::collect } } when SERVER_DATA { if { $debug } { log local0. "SERVER DATA: [TCP::payload]" } set lcpayload [string tolower [TCP::payload]] if { $lcpayload starts_with "+ok logged in" } { TCP::release set loggedin 1 event disable if { $debug } { log local0. "POP3 Login Successful" } return } elseif { $lcpayload contains "capa" } { TCP::payload replace [expr [TCP::payload length] - 3] 0 "STLS\r\n" } if { $secure == 1 && $loggedin == 0 } { clientside { SSL::collect } } elseif { $secure == 0 && $loggedin == 0 } { clientside { TCP::collect } } TCP::release if { $loggedin == 0 } { TCP::collect } } From dmiller at amfes.com Wed Apr 11 08:06:34 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Tue, 10 Apr 2012 22:06:34 -0700 Subject: [Dovecot] Folder Aliases - Revisited Yet Again Four Years Later In-Reply-To: <1281704852.3046.120.camel@kurkku.sapo.corppt.com> References: <4C648BF7.4020209@amfes.com> <1281704852.3046.120.camel@kurkku.sapo.corppt.com> Message-ID: On 8/13/2010 6:07 AM, Timo Sirainen wrote: > On Thu, 2010-08-12 at 17:04 -0700, Daniel L. Miller wrote: >> If we can't get clients to agree on a "standard" set of folders >> (probably reasonably safe assumption) > [...] > > > I'm kind of annoyed with listescape though. Hopefully for v2.1 I can > figure out some API changes to make it much easier to modify mailbox > names, and I guess also make such aliasing plugin much easier to > implement. > Did you happen to look into this in your spare time? -- Daniel From tss at iki.fi Wed Apr 11 08:12:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 08:12:36 +0300 Subject: [Dovecot] Folder Aliases - Revisited Yet Again Four Years Later In-Reply-To: References: <4C648BF7.4020209@amfes.com> <1281704852.3046.120.camel@kurkku.sapo.corppt.com> Message-ID: On 11.4.2012, at 8.06, Daniel L. Miller wrote: > On 8/13/2010 6:07 AM, Timo Sirainen wrote: >> On Thu, 2010-08-12 at 17:04 -0700, Daniel L. Miller wrote: >>> If we can't get clients to agree on a "standard" set of folders >>> (probably reasonably safe assumption) >> [...] >> >> >> I'm kind of annoyed with listescape though. Hopefully for v2.1 I can >> figure out some API changes to make it much easier to modify mailbox >> names, and I guess also make such aliasing plugin much easier to >> implement. > > Did you happen to look into this in your spare time? Yes, this is much easier with v2.1 I think. The new listescape plugin is just a tiny wrapper that will probably just be replaced by a setting some day in future. I haven't actually tried to write such alias plugin though. From tss at iki.fi Wed Apr 11 08:55:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 08:55:04 +0300 Subject: [Dovecot] Folder Aliases - Revisited Yet Again Four Years Later In-Reply-To: References: <4C648BF7.4020209@amfes.com> <1281704852.3046.120.camel@kurkku.sapo.corppt.com> Message-ID: <91485582-FA24-41C8-B478-62B4EF174C15@iki.fi> On 11.4.2012, at 8.12, Timo Sirainen wrote: > Yes, this is much easier with v2.1 I think. The new listescape plugin is just a tiny wrapper that will probably just be replaced by a setting some day in future. I haven't actually tried to write such alias plugin though. Here: http://dovecot.org/patches/2.1/mailbox-alias-plugin.c But as it is, it doesn't really work in a useful way, since e.g.: x create real x OK Create completed. x list "" real x OK List completed. x list "" alias * LIST (\HasNoChildren) "/" "alias" x OK List completed. x create real x NO [ALREADYEXISTS] Mailbox already exists x status real (messages) * STATUS "real" (MESSAGES 0) x OK Status completed. Perhaps a more useful feature would be if LIST showed both alias and real? That would need a bit more code. From tss at iki.fi Wed Apr 11 09:01:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 09:01:16 +0300 Subject: [Dovecot] lazy_expunge questions In-Reply-To: References: Message-ID: <4791F1D6-6BF2-4FB8-82CE-7C99DB50B4EA@iki.fi> On 10.4.2012, at 23.19, James Devine wrote: > I'm messing around with lazy expunge and I just had a couple of questions. > > If I delete messages or folders with messages, I see them being moved to > the expunge namespace, but if I delete an empty folder, all traces of it > seem to disappear. Is there a way to move empty folders to the expunge > namespace also? If you give 3 namespaces to lazyexpunge, it uses another way of moving stuff, and that includes also moving empty folders. But with 1 namespace it doesn't currently move it. Perhaps it should. > My second question is in regards to restoring folders. If I try to execute > the imap command 'RENAME EXPUNGED/TEST INBOX/TEST' I get 'NO [CANNOT] Can't > rename mailboxes to/from expunge namespace'. Just wondering if this is > possible or do I need to recreate the folder and then move messages into it. You need to recreate + move. Renaming across namespaces is a little tricky. From tss at iki.fi Wed Apr 11 09:26:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 09:26:20 +0300 Subject: [Dovecot] Bug tracker Message-ID: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> I started thinking that perhaps I should move my TODO list to a bug tracker. But because of reasons I've explained a few times before, I don't want a full blown public bug tracking system. The requirements for it are: * I am the only person who can add new bugs. Everyone else reports bugs/requests to this mailing list as before. (Well, I guess Stephan could use this as well if he wants to.) * Everyone can comment existing bugs. * Dovecot mailing list integration: Commenting a bug sends a mail to the mailing list. Replies to those comments go back to bug tracker (probably based on some [#1234] tag in subject). I would have the option of adding a comment that doesn't go to the mailing list (= adding some internal comment that nobody else cares about). Notifications about new bugs won't go to the mailing list (most likely it was created due to a recent mailing list post). So the main difference to how things work now is that people would be able to easily browse existing bugs and add comments to them. I would add bugs there only when I'm not planning on fixing them within a few days. I wouldn't add each and every feature request there, only the things that I'm actually interested in developing. So the idea would be to actually get the bug tracker emptied at some point, not to be a graveyard of unimportant feature requests that about 1-2 people in the world would want. So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. From martin.rabl at rablnet.de Wed Apr 11 09:43:46 2012 From: martin.rabl at rablnet.de (Martin Rabl) Date: Wed, 11 Apr 2012 08:43:46 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: <8095D302-D3CE-4A4E-AB12-64DB2EF7BD96@rablnet.de> What about Mantis? --- Martin Rabl Am 11.04.2012 um 08:26 schrieb Timo Sirainen : > I started thinking that perhaps I should move my TODO list to a bug tracker. But because of reasons I've explained a few times before, I don't want a full blown public bug tracking system. The requirements for it are: > > * I am the only person who can add new bugs. Everyone else reports bugs/requests to this mailing list as before. (Well, I guess Stephan could use this as well if he wants to.) > > * Everyone can comment existing bugs. > > * Dovecot mailing list integration: Commenting a bug sends a mail to the mailing list. Replies to those comments go back to bug tracker (probably based on some [#1234] tag in subject). I would have the option of adding a comment that doesn't go to the mailing list (= adding some internal comment that nobody else cares about). Notifications about new bugs won't go to the mailing list (most likely it was created due to a recent mailing list post). > > So the main difference to how things work now is that people would be able to easily browse existing bugs and add comments to them. I would add bugs there only when I'm not planning on fixing them within a few days. I wouldn't add each and every feature request there, only the things that I'm actually interested in developing. So the idea would be to actually get the bug tracker emptied at some point, not to be a graveyard of unimportant feature requests that about 1-2 people in the world would want. > > So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. > From janfrode at tanso.net Wed Apr 11 09:44:19 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 11 Apr 2012 08:44:19 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: <20120411064419.GA25675@dibs.tanso.net> On Wed, Apr 11, 2012 at 09:26:20AM +0300, Timo Sirainen wrote: > > So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. > I didn't see open source as a requirement, so then I would give a plug for Jira, which is the nicest/prettiest thing :-) And they provide free hosted solution: http://www.atlassian.com/software/jira/pricing Apache/ASF is a heavy jira user, in case you're not familiar with it: http://wiki.apache.org/general/ApacheJira https://issues.apache.org/jira/ -jf From tss at iki.fi Wed Apr 11 09:46:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 09:46:47 +0300 Subject: [Dovecot] Bug tracker In-Reply-To: <8095D302-D3CE-4A4E-AB12-64DB2EF7BD96@rablnet.de> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> <8095D302-D3CE-4A4E-AB12-64DB2EF7BD96@rablnet.de> Message-ID: <65B699AA-10F7-4CF8-81F7-AA59895D54E9@iki.fi> Probably the ugliest/user-unfriendliest bug tracker UI that I've ever seen :) On 11.4.2012, at 9.43, Martin Rabl wrote: > What about Mantis? > > > --- > Martin Rabl > > Am 11.04.2012 um 08:26 schrieb Timo Sirainen : > >> I started thinking that perhaps I should move my TODO list to a bug tracker. But because of reasons I've explained a few times before, I don't want a full blown public bug tracking system. The requirements for it are: >> >> * I am the only person who can add new bugs. Everyone else reports bugs/requests to this mailing list as before. (Well, I guess Stephan could use this as well if he wants to.) >> >> * Everyone can comment existing bugs. >> >> * Dovecot mailing list integration: Commenting a bug sends a mail to the mailing list. Replies to those comments go back to bug tracker (probably based on some [#1234] tag in subject). I would have the option of adding a comment that doesn't go to the mailing list (= adding some internal comment that nobody else cares about). Notifications about new bugs won't go to the mailing list (most likely it was created due to a recent mailing list post). >> >> So the main difference to how things work now is that people would be able to easily browse existing bugs and add comments to them. I would add bugs there only when I'm not planning on fixing them within a few days. I wouldn't add each and every feature request there, only the things that I'm actually interested in developing. So the idea would be to actually get the bug tracker emptied at some point, not to be a graveyard of unimportant feature requests that about 1-2 people in the world would want. >> >> So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. >> > From tss at iki.fi Wed Apr 11 09:49:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 09:49:18 +0300 Subject: [Dovecot] Bug tracker In-Reply-To: <20120411064419.GA25675@dibs.tanso.net> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> <20120411064419.GA25675@dibs.tanso.net> Message-ID: On 11.4.2012, at 9.44, Jan-Frode Myklebust wrote: > On Wed, Apr 11, 2012 at 09:26:20AM +0300, Timo Sirainen wrote: >> >> So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. >> > > I didn't see open source as a requirement, so then I would give a plug > for Jira, which is the nicest/prettiest thing :-) I don't think it supports one of my requirements: > I would have the option of adding a comment that doesn't go to the mailing list Unless that's been added in a newer version. From ngu.antoine at gmail.com Wed Apr 11 09:49:45 2012 From: ngu.antoine at gmail.com (Antoine Nguyen) Date: Wed, 11 Apr 2012 08:49:45 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: Le 11 avril 2012 08:26, Timo Sirainen a ?crit : > I started thinking that perhaps I should move my TODO list to a bug > tracker. But because of reasons I've explained a few times before, I don't > want a full blown public bug tracking system. The requirements for it are: > > * I am the only person who can add new bugs. Everyone else reports > bugs/requests to this mailing list as before. (Well, I guess Stephan could > use this as well if he wants to.) > > * Everyone can comment existing bugs. > > * Dovecot mailing list integration: Commenting a bug sends a mail to the > mailing list. Replies to those comments go back to bug tracker (probably > based on some [#1234] tag in subject). I would have the option of adding a > comment that doesn't go to the mailing list (= adding some internal comment > that nobody else cares about). Notifications about new bugs won't go to the > mailing list (most likely it was created due to a recent mailing list post). > > So the main difference to how things work now is that people would be able > to easily browse existing bugs and add comments to them. I would add bugs > there only when I'm not planning on fixing them within a few days. I > wouldn't add each and every feature request there, only the things that I'm > actually interested in developing. So the idea would be to actually get the > bug tracker emptied at some point, not to be a graveyard of unimportant > feature requests that about 1-2 people in the world would want. > > So, any suggestions for what software could do these things? I think > Request Tracker has those features, but it's not really the > nicest/prettiest thing. > > Maybe Redmine ? (http://www.redmine.org/ It's more than just a bug tracker but I think it answers your needs. Antoine Modoboa developer (http://modoboa.org/) From martin.rabl at rablnet.de Wed Apr 11 09:53:24 2012 From: martin.rabl at rablnet.de (Martin Rabl) Date: Wed, 11 Apr 2012 08:53:24 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: <65B699AA-10F7-4CF8-81F7-AA59895D54E9@iki.fi> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> <8095D302-D3CE-4A4E-AB12-64DB2EF7BD96@rablnet.de> <65B699AA-10F7-4CF8-81F7-AA59895D54E9@iki.fi> Message-ID: ;-) Ok, next one: TRAC ? --- Martin Rabl Am 11.04.2012 um 08:46 schrieb Timo Sirainen : > Probably the ugliest/user-unfriendliest bug tracker UI that I've ever seen :) > > On 11.4.2012, at 9.43, Martin Rabl wrote: > >> What about Mantis? >> >> >> --- >> Martin Rabl >> >> Am 11.04.2012 um 08:26 schrieb Timo Sirainen : >> >>> I started thinking that perhaps I should move my TODO list to a bug tracker. But because of reasons I've explained a few times before, I don't want a full blown public bug tracking system. The requirements for it are: >>> >>> * I am the only person who can add new bugs. Everyone else reports bugs/requests to this mailing list as before. (Well, I guess Stephan could use this as well if he wants to.) >>> >>> * Everyone can comment existing bugs. >>> >>> * Dovecot mailing list integration: Commenting a bug sends a mail to the mailing list. Replies to those comments go back to bug tracker (probably based on some [#1234] tag in subject). I would have the option of adding a comment that doesn't go to the mailing list (= adding some internal comment that nobody else cares about). Notifications about new bugs won't go to the mailing list (most likely it was created due to a recent mailing list post). >>> >>> So the main difference to how things work now is that people would be able to easily browse existing bugs and add comments to them. I would add bugs there only when I'm not planning on fixing them within a few days. I wouldn't add each and every feature request there, only the things that I'm actually interested in developing. So the idea would be to actually get the bug tracker emptied at some point, not to be a graveyard of unimportant feature requests that about 1-2 people in the world would want. >>> >>> So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. >>> >> > From tlx at leuxner.net Wed Apr 11 09:54:28 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Wed, 11 Apr 2012 08:54:28 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: <20120411065428.GA3099@nihlus.leuxner.net> On Wed, Apr 11, 2012 at 09:26:20AM +0300, Timo Sirainen wrote: > So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. > May be worth to take a look at http://www.thebuggenie.com/. Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From p at state-of-mind.de Wed Apr 11 10:10:07 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Wed, 11 Apr 2012 09:10:07 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: <20120411071006.GC28281@state-of-mind.de> * Antoine Nguyen : > > I started thinking that perhaps I should move my TODO list to a bug > > tracker. But because of reasons I've explained a few times before, I don't > > want a full blown public bug tracking system. The requirements for it are: > > > > * I am the only person who can add new bugs. Everyone else reports > > bugs/requests to this mailing list as before. (Well, I guess Stephan could > > use this as well if he wants to.) > > > > * Everyone can comment existing bugs. > > > > * Dovecot mailing list integration: Commenting a bug sends a mail to the > > mailing list. Replies to those comments go back to bug tracker (probably > > based on some [#1234] tag in subject). I would have the option of adding a > > comment that doesn't go to the mailing list (= adding some internal comment > > that nobody else cares about). Notifications about new bugs won't go to the > > mailing list (most likely it was created due to a recent mailing list post). > > > > So the main difference to how things work now is that people would be able > > to easily browse existing bugs and add comments to them. I would add bugs > > there only when I'm not planning on fixing them within a few days. I > > wouldn't add each and every feature request there, only the things that I'm > > actually interested in developing. So the idea would be to actually get the > > bug tracker emptied at some point, not to be a graveyard of unimportant > > feature requests that about 1-2 people in the world would want. > > > > So, any suggestions for what software could do these things? I think > > Request Tracker has those features, but it's not really the > > nicest/prettiest thing. > > > > > Maybe Redmine ? (http://www.redmine.org/ > > It's more than just a bug tracker but I think it answers your needs. +1 We moved from trac to redmine about a year ago and still are very happy about it. Our non-technical customers can deal with it well. Redmine allows to hide projects and it knows the concept of sub-projects. It's versatile in terms of repositories (mercurial supported) and ticket handling is fine too. Sometimes I miss a more comfortable wiki editor, but that's a minor tradeoff. p at rick -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From warren at decoy.co.za Wed Apr 11 10:18:28 2012 From: warren at decoy.co.za (Warren Baker) Date: Wed, 11 Apr 2012 09:18:28 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: <20120411071006.GC28281@state-of-mind.de> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> <20120411071006.GC28281@state-of-mind.de> Message-ID: On Wed, Apr 11, 2012 at 9:10 AM, Patrick Ben Koetter

wrote: > * Antoine Nguyen : >> >> It's more than just a bug tracker but I think it answers your needs. > > +1 > > We moved from trac to redmine about a year ago and still are very happy about > it. Our non-technical customers can deal with it well. Redmine allows to hide > projects and it knows the concept of sub-projects. It's versatile in terms of > repositories (mercurial supported) and ticket handling is fine too. > > Sometimes I miss a more comfortable wiki editor, but that's a minor tradeoff. +2 - we have been using it for another open source project for a number of years and it works well. We have it hooked up into github and it works without a glitch. thx -- .warren From stan at hardwarefreak.com Wed Apr 11 10:18:49 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 11 Apr 2012 02:18:49 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> Message-ID: <4F853059.1000706@hardwarefreak.com> On 4/10/2012 1:09 AM, Emmanuel Noobadmin wrote: > On 4/10/12, Stan Hoeppner wrote: >> SuperMicro H8SGL G34 mobo w/dual Intel GbE, 2GHz 8-core Opteron >> 32GB Kingston REG ECC DDR3, LSI 9280-4i4e, Intel 24 port SAS expander >> 20 x 1TB WD RE4 Enterprise 7.2K SATA2 drives >> NORCO RPC-4220 4U 20 Hot-Swap Bays, SuperMicro 865W PSU >> All other required parts are in the Wish List. I've not written >> assembly instructions. I figure anyone who would build this knows what >> s/he is doing. >> >> Price today: $5,376.62 > > This price looks like something I might be able to push through It's pretty phenomenally low considering what all you get, especially 20 enterprise class drives. > although I'll probably have to go SATA instead of SAS due to cost of > keeping spares. The 10K drives I mentioned are SATA not SAS. WD's 7.2k RE and 10k Raptor series drives are both SATA but have RAID specific firmware, better reliability, longer warranties, etc. The RAID specific firmware is why both are tested and certified by LSI with their RAID cards. >> Configuring all 20 drives as a RAID10 LUN in the MegaRAID HBA would give >> you a 10TB net Linux device and 10 stripe spindles of IOPS and >> bandwidth. Using RAID6 would yield 18TB net and 18 spindles of read >> throughput, however parallel write throughput will be at least 3-6x >> slower than RAID10, which is why nobody uses RAID6 for transactional >> workloads. > > Not likely to go with RAID 5 or 6 due to concerns about the > uncorrectable read errors risks on rebuild with large arrays. Is the Not to mention rebuild times for large width RAID5/6. > MegaRAID being used as the actual RAID controller or just as a HBA? It's a top shelf RAID controller, 512MB cache, up to 240 drives, SSD support, the works. It's an LSI "Feature Line" card: http://www.lsi.com/products/storagecomponents/Pages/6GBSATA_SASRAIDCards.aspx The specs: http://www.lsi.com/products/storagecomponents/Pages/MegaRAIDSAS9280-4i4e.aspx You'll need the cache battery module for safe write caching, which I forgot in the wish list (now added), $160: http://www.newegg.com/Product/Product.aspx?Item=N82E16816118163&Tpk=LSIiBBU08 With your workload and RAID10 you should run with all 512MB configured as write cache. Linux caches all reads so using any controller cache for reads is a waste. Using all 512MB for write cache will increase random write IOPS. Note the 9280 allows up to 64 LUNs, so you can do tiered storage within this 20 bay chassis. For spares management you'd probably not want to bother with two different sized drives. I didn't mention the 300GB 10K Raptors previously due to their limited capacity. Note they're only $15 more apiece than the 1TB RE4 drives in the original parts list. For a total of $300 more you get the same 40% increase in IOPs of the 600GB model, but you'll only have 3TB net space after RAID10. If 3TB is sufficient space for your needs, that extra 40% IOPS makes this config a no brainer. The decreased latency of the 10K drives will give a nice boost to VM read performance, especially when using NFS. Write performance probably won't be much different due to the generous 512MB write cache on the controller. I also forgot to mention that with BBWC enabled you can turn off XFS barriers, which will dramatically speed up Exim queues and Dovecot writes, all writes actually. Again, you probably don't want the spares management overhead of two different disk types on the shelf, but you could stick these 10K 300s in the first 16 slots, and put the 2TB RE4 drive in the last 4 slots, RAID10 on the 10K drives, RAID5 on the 2TB drives. This yields an 8 spindle high IOPS RAID10 of 2.4TB and a lower performance RAID5 of 6TB for near line storage such as your Dovecot alt storage, VM templates, etc, 8.4TB net, 1.6TB less than the original 10TB setup. Total additional cost is $920 for this setup. You'd have two XFS filesystems (with quite different mkfs parameters). > I have been avoiding hardware RAID because of a really bad experience > with RAID 5 on an obsolete controller that eventually died without > replacement and couldn't be recovered. Since then, it's always been > RAID 1 and, after I discovered mdraid, using them as purely HBA with > mdraid for the flexibility of being able to just pull the drives into > a new system if necessary without having to worry about the > controller. Assuming you have the right connector configuration for your drive/enclosure on the replacement card, you can usually swap out one LSI RAID card with any other LSI RAID card in the same, or newer, generation. It'll read the configuration metadata from the disks and be up an running in minutes. This feature has been around all the way back to the AMI/Mylex cards of the late 1990s. LSI acquired both companies, who were #1 and #2 in RAID, which is why LSI is so successful today. Back in those days LSI simply supplied the ASICs to AMI and Mylex. I have an AMI MegaRAID 428, top of the line in 1998, lying around somewhere. Still working when I retired it many years ago. FYI, LSI is the OEM provider of RAID and SAS/SATA HBA ASIC silicon for the tier 1 HBA and mobo down markets. Dell, HP, IBM, Intel, Oracle (Sun), Siemens/Fujitsu, all use LSI silicon and firmware. Some simply rebadge OEM LSI cards with their own model and part numbers. IBM and Dell specifically have been doing this rebadging for well over a decade, long before LSI acquired Mylex and AMI. The Dell PERC/2 is a rebadged AMI MegaRAID 428. Software and hardware RAID each have their pros and cons. I prefer hardware RAID for write cache performance and many administrative reasons, including SAF-TE enclosure management (fault LEDs, alarms, etc) so you know at a glance which drive has failed and needs replacing, email and SNMP notification of events, automatic rebuild, configurable rebuild priority, etc, etc, and good performance with striping and mirroring. Parity RAID performance often lags behind md with heavy workloads but not with light/medium. FWIW I rarely use parity RAID, due to the myriad performance downsides. For ultra high random IOPS workloads, or when I need a single filesystem space larger than the drive limit or practical limit for one RAID HBA, I'll stitch hardware RAID1 or small stripe width RAID 10 arrays (4-8 drives, 2-4 spindles) together with md RAID 0 or 1. >> Both of the drives I've mentioned here are enterprise class drives, >> feature TLER, and are on the LSI MegaRAID SAS hardware compatibility >> list. The price of the 600GB Raptor has come down considerably since I >> designed this system, or I'd have used them instead. >> >> Anyway, lots of option out there. But $6,500 is pretty damn cheap for a >> quality box with 32GB RAM, enterprise RAID card, and 20x10K RPM 600GB >> drives. >> >> The MegaRAID 9280-4i4e has an external SFF8088 port For an additional >> $6,410 you could add an external Norco SAS expander JBOD chassis and 24 >> more 600GB 10K RPM Raptors, for 13.2TB of total net RAID10 space, and 22 >> 10k spindles of IOPS performance from 44 total drives. That's $13K for >> a 5K random IOPS, 13TB, 44 drive NFS RAID COTS server solution, >> $1000/TB, $2.60/IOPS. Significantly cheaper than an HP, Dell, IBM >> solution of similar specs, each of which will set you back at least 20 >> large. > > Would this setup work well too for serving up VM images? I've been > trying to find a solution for the virtualized app servers images as > well but the distributed FSes currently are all bad with random > reads/writes it seems. XFS seem to be good with large files like db > and vm images with random internal write/read so given my time > constraints, it would be nice to have a single configuration that > works generally well for all the needs I have to oversee. Absolutely. If you setup these 20 drives as a single RAID10, soft/hard or hybrid, with the LSI cache set to 100% write-back, with a single XFS filesystem with 10 allocation groups and proper stripe alignment, you'll get maximum performance for pretty much any conceivable workload. Your only limitations will be possible NFS or TCP tuning issues, and maybe having only two GbE ports. For small random IOPS such as Exim queues, Dovecot store, VM image IO, etc, the two GbE ports are plenty. But if you add any large NFS file copies into the mix, such as copying new VM templates or ISO images over, etc, or do backups over NFS instead of directly on the host machine at the XFS level, then two bonded GbE ports might prove a bottleneck. The mobo has 2 PCIe x8 slots and one x4 slot. One of the x8 slots is an x16 physical connector. You'll put the LSI card in the x16 slot. If you mount the Intel SAS expander to the chassis as I do instead of in a slot, you have one free x8 and one free x4 slot. Given the $250 price, I'd simply ad an Intel quad port GbE NIC to the order. Link aggregate all 4 ports on day one and use one IP address for the NFS traffic. Use the two on board ports for management etc. This should give you a theoretical 400MB/s of peak NFS throughput, which should be plenty no matter what workload you throw at it. >> Note the chassis I've spec'd have single PSUs, not the dual or triple >> redundant supplies you'll see on branded hardware. With a relatively >> stable climate controlled environment and a good UPS with filtering, >> quality single supplies are fine. In fact, in the 4U form factor single >> supplies are usually more reliable due to superior IC packaging and >> airflow through the heatsinks, not to mention much quieter. > > Same reason I do my best to avoid 1U servers, the space/heat issues > worries me. Yes, I'm guilty of worrying too much but that had saved me > on several occasions. Just about every 1U server I've seen that's been racked for 3 or more years has warped under its own weight. I even saw an HPQ 2U that was warped this way, badly warped. In this instance the slide rail bolts had never been tightened down to the rack--could spin them by hand. Since the chassis side panels weren't secured, and there was lateral play, the weight of the 6 drives caused the side walls of the case to fold into a mild trapezoid, which allowed the bottom and top panels to bow. Let this be a lesson boys and girls: always tighten your rack bolts. :) -- Stan From tss at iki.fi Wed Apr 11 10:23:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 10:23:56 +0300 Subject: [Dovecot] Bug tracker In-Reply-To: References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: On 11.4.2012, at 9.49, Antoine Nguyen wrote: > Maybe Redmine ? (http://www.redmine.org/ I don't think it either supports: > I would have the option of adding a comment that doesn't go to the mailing list But I'm beginning to think that no BTS supports that in the way I want. And it's not a huge problem for me really, just means that this mailing list may get a bit more pointless notifications than necessary. Although I'm also not seeing much documentation about email notifications in general. Can it be configured to send comment updates, but no other updates? I guess this could also be kludged by dropping unwanted emails in MTA. From janfrode at tanso.net Wed Apr 11 10:25:22 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 11 Apr 2012 09:25:22 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> <20120411064419.GA25675@dibs.tanso.net> Message-ID: <20120411072522.GA26184@dibs.tanso.net> On Wed, Apr 11, 2012 at 09:49:18AM +0300, Timo Sirainen wrote: > > > > I didn't see open source as a requirement, so then I would give a plug > > for Jira, which is the nicest/prettiest thing :-) > > I don't think it supports one of my requirements: > > > I would have the option of adding a comment that doesn't go to the mailing list > > Unless that's been added in a newer version. > There is an option for restricting who can view your comment, plus "Email notifications will only be sent to people who have permission to view the relevant issue" http://confluence.atlassian.com/display/JIRA/Creating+a+Notification+Scheme so I would expect it to be possible to define that the mailinglist is not member of a group-b, while everyone else is, and restrict the comment to that group. But best would probably be to discuss it with atlassion support... -jf From lee at standen.id.au Wed Apr 11 10:59:28 2012 From: lee at standen.id.au (Lee Standen) Date: Wed, 11 Apr 2012 15:59:28 +0800 Subject: [Dovecot] Bug tracker In-Reply-To: <20120411072522.GA26184@dibs.tanso.net> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> <20120411064419.GA25675@dibs.tanso.net> <20120411072522.GA26184@dibs.tanso.net> Message-ID: We use JIRA internally and it's probably the best bug tracker I've ever seen. The permissions system is very flexible, and to the best of my knowledge it should support everything you've described in your original post. On Wed, Apr 11, 2012 at 3:25 PM, Jan-Frode Myklebust wrote: > On Wed, Apr 11, 2012 at 09:49:18AM +0300, Timo Sirainen wrote: > > > > > > I didn't see open source as a requirement, so then I would give a plug > > > for Jira, which is the nicest/prettiest thing :-) > > > > I don't think it supports one of my requirements: > > > > > I would have the option of adding a comment that doesn't go to the > mailing list > > > > Unless that's been added in a newer version. > > > > There is an option for restricting who can view your comment, plus > "Email notifications will only be sent to people who have permission to > view the relevant issue" > > > http://confluence.atlassian.com/display/JIRA/Creating+a+Notification+Scheme > > so I would expect it to be possible to define that the mailinglist is > not member of a group-b, while everyone else is, and restrict the comment > to that group. > > But best would probably be to discuss it with atlassion support... > > > -jf > From ngu.antoine at gmail.com Wed Apr 11 11:05:05 2012 From: ngu.antoine at gmail.com (Antoine Nguyen) Date: Wed, 11 Apr 2012 10:05:05 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: Le 11 avril 2012 09:23, Timo Sirainen a ?crit : > On 11.4.2012, at 9.49, Antoine Nguyen wrote: > > > Maybe Redmine ? (http://www.redmine.org/ > > I don't think it either supports: > > > I would have the option of adding a comment that doesn't go to the > mailing list > > But I'm beginning to think that no BTS supports that in the way I want. > And it's not a huge problem for me really, just means that this mailing > list may get a bit more pointless notifications than necessary. > > Although I'm also not seeing much documentation about email notifications > in general. Can it be configured to send comment updates, but no other > updates? I guess this could also be kludged by dropping unwanted emails in > MTA. > > AFAIK, yes. You can choose on which kind of event Redmine sends a notification. The documentation is not the best I've seen but the administration panel is pretty simple to understand. Antoine From warren at decoy.co.za Wed Apr 11 11:24:55 2012 From: warren at decoy.co.za (Warren Baker) Date: Wed, 11 Apr 2012 10:24:55 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: On Wed, Apr 11, 2012 at 10:05 AM, Antoine Nguyen wrote: > Le 11 avril 2012 09:23, Timo Sirainen a ?crit : >> > I would have the option of adding a comment that doesn't go to the >> mailing list >> >> But I'm beginning to think that no BTS supports that in the way I want. >> And it's not a huge problem for me really, just means that this mailing >> list may get a bit more pointless notifications than necessary. >> >> Although I'm also not seeing much documentation about email notifications >> in general. Can it be configured to send comment updates, but no other >> updates? I guess this could also be kludged by dropping unwanted emails in >> MTA. >> >> > AFAIK, yes. You can choose on which kind of event Redmine sends a > notification. The documentation is not the best I've seen but the > administration panel is pretty simple to understand. > There is a default notification option which is set to 'Only for things I watch or I'm involved in' - which in your case Timo you would receive everything since you would be the project owner. There are also global settings that you can configure which provide the ability to send notifications for certain events only: - Issue added - Issue updated - Note added - Status updated - Priority updated - News added - Comment added to a news - Document added - File added - Message added - Wiki page added - Wiki page updated I think there is also a plugin to disable email notifications entirely for a user. -- .warren From mlists at edicom.eu Wed Apr 11 12:35:25 2012 From: mlists at edicom.eu (Miguel Tormo) Date: Wed, 11 Apr 2012 11:35:25 +0200 Subject: [Dovecot] Proxy and SSO (single sign-on) In-Reply-To: <201204041347.47817.mlists@edicom.eu> References: <201204041318.00760.mlists@edicom.eu> <48AD358F-ECC4-48F5-8CAB-9B91C8BDEEE3@iki.fi> <201204041347.47817.mlists@edicom.eu> Message-ID: <201204111135.25742.mlists@edicom.eu> El Mi?rcoles, 4 de Abril de 2012 13:47:47 Miguel Tormo escribi?: > El Mi?rcoles, 4 de Abril de 2012 13:21:33 Timo Sirainen escribi?: > > On 4.4.2012, at 14.18, Miguel Tormo wrote: > > > > > I have a running setup with a dovecot imap4/pop3 proxy to a few dovecot backend servers which actually store the mailboxes. This is running smoothly and allows me to transparently distribute mailboxes. > > > I'm using some "extrafield" configured in the LDAP passdb. > > > > > > However, now I would like to use GSSAPI (preferred) and NTLM for single sign-on. Both are pretty straightforward to configure in a single instance environment, but I don't know if they would work with proxy. For example, with GSSAPI there are two cases: > > > 1) Just use gssapi mechanism, without PAM. Then, it a user presents a ticket the passdb ldap is not used, so the extrafields are never read. > > > > The patch in http://dovecot.org/list/dovecot/2012-March/064331.html makes this work I think. I still haven't managed to look into it much though. > > > > > > It definitely is worth a look. I wonder if it would make ldap extrafields lookups work with gssapi auth, I will try it and post the results. Well just in case someone is interested: I applied Sam Morris' patch and got this working. Very nice indeed. To keep in mind: - It's necessary to set a master password for the backend server so the proxy instance can impersonate any user when connecting to the backend instance. I needed to do this to make GSSAPI with proxy work (thus the proxy instance is who actually does the authentication), however NTLM worked without a masster password, it seems this authenticatiom mechanism can be forwarded as is. - In the LDAP configuration, "auth bind" must be set to "no". Obviously, when using sso you don't provide a password, so there is no way it can bind to the LDAP server with your credentials. There is still a problem that might be more related to the MUA, but still if anyone has a suggestion I would appreciate it. The problem is the following: if you are a roaming user (ie: with a laptop) when you are outside the LAN you cannot get a kerberos ticket thus the GSSAPI auth fails. Then you have to change your account configuration and select a password-based authentication mechanism. This is not very convenient. However, this is different if compared with kerberos authentication with a web browser: if you have a valid ticket, access is granted; if not, the browser prompts for user/password and then you are granted access if the supplied credentials are valid. Do you have any idea if something like this is possible to accomplish with IMAP? Thank you and regards, From CMarcus at Media-Brokers.com Wed Apr 11 13:20:47 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 11 Apr 2012 06:20:47 -0400 Subject: [Dovecot] Bug tracker In-Reply-To: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: <4F855AFF.7050704@Media-Brokers.com> On 2012-04-11 2:26 AM, Timo Sirainen wrote: > Notifications about new bugs won't go to the mailing list (most > likely it was created due to a recent mailing list post). I actually would like to see these, and I imagine I'm not alone... If you'd prefer they don't go to the users list, maybe create a new 'notifications' (or maybe even a -dev) list that anyone could subscribe to to see these? > So, any suggestions for what software could do these things? I think > Request Tracker has those features, but it's not really the > nicest/prettiest thing. +1 to Redmine. I only used it for a short time, but I liked it, and the fact that you can integrate it with git or mercurial is a big plus. Yes, it is a full blown bug tracker, but if, in the future, you add more developers (maybe you get some angel funding), you already have a full blown bug tracker in place, including the extras it comes with. -- Best regards, Charles From interfasys at gmail.com Wed Apr 11 12:05:59 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Wed, 11 Apr 2012 10:05:59 +0100 Subject: [Dovecot] Bug tracker Message-ID: <4F854977.9000804@gmail.com> I second that. New version 3.2 supports sub-projects, editions and components and is user-friendly. Cheers, Olivier > On Wed, Apr 11, 2012 at 09:26:20AM +0300, Timo Sirainen wrote: > > So, any suggestions for what software could do these things? I think > Request > Tracker has those features, but it's not really the nicest/prettiest > thing. > > May be worth to take a look at http://www.thebuggenie.com/. > > Thomas > From helga.mayer at uni-hohenheim.de Wed Apr 11 17:13:21 2012 From: helga.mayer at uni-hohenheim.de (Helga Mayer) Date: Wed, 11 Apr 2012 16:13:21 +0200 Subject: [Dovecot] Problems with Apple Mail and attachments Message-ID: <20120411161321.165760epurnu7i4h@webmail.uni-hohenheim.de> Hello, has anyone experiences with the Apple mail client, to be precise : Mail 5.2? When I copy a message with an attachment from the inbox to a different folder using 'move' or 'copy' , Apple Mail does not properly display the attachment in the new folder. When I copy the same message using 'archive' the attachment shows up properly. There is no difference on the server side. We are using maildirs. The raw messages in the various folders are identical. Deleting plist on the Mac didn't help. After removing and recreation of the account (client side) the attachments are displayed correctly again. Is this an apple problem or has it got anything to do with dovecot.index.cache ? Kind Regards Helga Mayer From yzhou at medplus.com Wed Apr 11 17:49:05 2012 From: yzhou at medplus.com (Zhou, Yan) Date: Wed, 11 Apr 2012 14:49:05 +0000 Subject: [Dovecot] ldap idle connection timeout in DoveCot 1.0.13? Message-ID: Hi there, We are using DoveCot 1.0.13, it connects to LDAP server for authentication. It seems that DoveCot keeps the idle LDAP connection open. Our firewall is terminating these connections after some time of idle activity (2 hours), then, we run into authentication problem. If we restart either LDAP or DoveCot, then it is fine. Can we set some kind of LDAP idle connection timeout in DoveCot? /etc/dovecot-ldap.conf. I do not see any configuration available for 1.0.13. Another alternative is to set idle connection timeout in LDAP, but we prefer doing that in DoveCot. Thanks, Yan Confidentiality Notice: The information contained in this electronic transmission is confidential and may be legally privileged. It is intended only for the addressee(s) named above. If you are not an intended recipient, be aware that any disclosure, copying, distribution or use of the information contained in this transmission is prohibited and may be unlawful. If you have received this transmission in error, please notify us by telephone (513) 229-5500 or by email (postmaster at MedPlus.com). After replying, please erase it from your computer system. From fxmulder at gmail.com Wed Apr 11 18:34:54 2012 From: fxmulder at gmail.com (James Devine) Date: Wed, 11 Apr 2012 09:34:54 -0600 Subject: [Dovecot] lazy_expunge questions In-Reply-To: <4791F1D6-6BF2-4FB8-82CE-7C99DB50B4EA@iki.fi> References: <4791F1D6-6BF2-4FB8-82CE-7C99DB50B4EA@iki.fi> Message-ID: On Wed, Apr 11, 2012 at 12:01 AM, Timo Sirainen wrote: > On 10.4.2012, at 23.19, James Devine wrote: > > > I'm messing around with lazy expunge and I just had a couple of > questions. > > > > If I delete messages or folders with messages, I see them being moved to > > the expunge namespace, but if I delete an empty folder, all traces of it > > seem to disappear. Is there a way to move empty folders to the expunge > > namespace also? > > If you give 3 namespaces to lazyexpunge, it uses another way of moving > stuff, and that includes also moving empty folders. But with 1 namespace it > doesn't currently move it. Perhaps it should. > > > My second question is in regards to restoring folders. If I try to > execute > > the imap command 'RENAME EXPUNGED/TEST INBOX/TEST' I get 'NO [CANNOT] > Can't > > rename mailboxes to/from expunge namespace'. Just wondering if this is > > possible or do I need to recreate the folder and then move messages into > it. > > You need to recreate + move. Renaming across namespaces is a little tricky. > > It looks like 3 namespace support in the lazy expunge plugin was removed in 2.1+ ? From tss at iki.fi Wed Apr 11 19:10:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 19:10:36 +0300 Subject: [Dovecot] lazy_expunge questions In-Reply-To: References: <4791F1D6-6BF2-4FB8-82CE-7C99DB50B4EA@iki.fi> Message-ID: On 11.4.2012, at 18.34, James Devine wrote: > It looks like 3 namespace support in the lazy expunge plugin was removed in > 2.1+ ? Yeah, so it seems. Updated the wiki2. From tss at iki.fi Wed Apr 11 19:36:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 19:36:30 +0300 Subject: [Dovecot] ldap idle connection timeout in DoveCot 1.0.13? In-Reply-To: References: Message-ID: On 11.4.2012, at 17.49, Zhou, Yan wrote: > We are using DoveCot 1.0.13, it connects to LDAP server for authentication. It seems that DoveCot keeps the idle LDAP connection open. Yes. > Our firewall is terminating these connections after some time of idle activity (2 hours), then, we run into authentication problem. If we restart either LDAP or DoveCot, then it is fine. > > Can we set some kind of LDAP idle connection timeout in DoveCot? /etc/dovecot-ldap.conf. I do not see any configuration available for 1.0.13. No. But if you upgrade to a newer Dovecot (v2.x probably) this is solved by automatic transparent reconnection. From tss at iki.fi Wed Apr 11 19:37:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 19:37:59 +0300 Subject: [Dovecot] Problems with Apple Mail and attachments In-Reply-To: <20120411161321.165760epurnu7i4h@webmail.uni-hohenheim.de> References: <20120411161321.165760epurnu7i4h@webmail.uni-hohenheim.de> Message-ID: On 11.4.2012, at 17.13, Helga Mayer wrote: > has anyone experiences with the Apple mail client, to be precise : Mail 5.2? > When I copy a message with an attachment from the inbox to a different folder using 'move' or 'copy' , Apple Mail does not properly display the attachment in the new folder. .. > Is this an apple problem or has it got anything to do with dovecot.index.cache ? Highly unlikely to be a Dovecot problem. Of course, you can verify if it's a caching problem by disabling indexes and seeing if you can reproduce the problem: mail_location = maildir:~/Maildir:INDEX=MEMORY From thierry at odry.net Wed Apr 11 19:44:59 2012 From: thierry at odry.net (Thierry de Montaudry) Date: Wed, 11 Apr 2012 18:44:59 +0200 Subject: [Dovecot] Problems with Apple Mail and attachments In-Reply-To: <20120411161321.165760epurnu7i4h@webmail.uni-hohenheim.de> References: <20120411161321.165760epurnu7i4h@webmail.uni-hohenheim.de> Message-ID: On 11 Apr 2012, at 16:13, Helga Mayer wrote: > Hello, > > has anyone experiences with the Apple mail client, to be precise : Mail 5.2? > When I copy a message with an attachment from the inbox to a different folder using 'move' or 'copy' , Apple Mail does not properly display the attachment in the new folder. > > When I copy the same message using 'archive' the attachment shows up properly. > There is no difference on the server side. We are using maildirs. The raw > messages in the various folders are identical. > Deleting plist on the Mac didn't help. > After removing and recreation of the account (client side) the attachments are displayed correctly again. > > Is this an apple problem or has it got anything to do with dovecot.index.cache ? > > Kind Regards > Helga Mayer > > Hi Helga, I'm using Apple Mail 5.2 with a dovecot server (2.0.13, with maildir), and recently had no trouble like that, even though working with a mailbox that has about 10000 emails, a lot of folders, and moving mails around all the time. I've seen a similar problem a while ago (1 year maybe more), but used the mailbox Rebuild option on the client, which fixed it without having to delete and recreate the account. Your problem might just be a local index corruption, which can happen when loosing your Internet connection. Regards, Thierry From lists at wildgooses.com Wed Apr 11 19:50:09 2012 From: lists at wildgooses.com (Ed W) Date: Wed, 11 Apr 2012 17:50:09 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F853059.1000706@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> Message-ID: <4F85B641.5060103@wildgooses.com> Re XFS. Have you been watching BTRFS recently? I will concede that despite the authors considering it production ready I won't be using it for my servers just yet. However, it's benchmarking on single disk benchmarks fairly similarly to XFS and in certain cases (multi-threaded performance) can be somewhat better. I haven't yet seen any benchmarks on larger disk arrays yet, eg 6+ disks, so no idea how it scales up. Basically what I have seen seems "competitive" I don't have such hardware spare to benchmark, but I would be interested to hear from someone who benchmarks your RAID1+linear+XFS suggestion, especially if they have compared a cutting edge btrfs kernel on the same array? One of the snags of md RAID1 vs RAID6 is the lack of checksumming in the event of bad blocks. (I'm not sure what actually happens when md scrubbing finds a bad sector with raid1..?). For low performance requirements I have become paranoid and been using RAID6 vs RAID10, filesystems with sector checksums seem attractive... Regards Ed W From adrian.minta at gmail.com Wed Apr 11 23:48:00 2012 From: adrian.minta at gmail.com (Adrian Minta) Date: Wed, 11 Apr 2012 23:48:00 +0300 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F85B641.5060103@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> Message-ID: <4F85EE00.9030208@gmail.com> On 04/11/12 19:50, Ed W wrote: ... > One of the snags of md RAID1 vs RAID6 is the lack of checksumming in > the event of bad blocks. (I'm not sure what actually happens when md > scrubbing finds a bad sector with raid1..?). For low performance > requirements I have become paranoid and been using RAID6 vs RAID10, > filesystems with sector checksums seem attractive... > RAID6 is very slow for write operations. That's why is the worst choice for maildir. From CMarcus at Media-Brokers.com Wed Apr 11 21:50:11 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 11 Apr 2012 14:50:11 -0400 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F85EE00.9030208@gmail.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F85EE00.9030208@gmail.com> Message-ID: <4F85D263.3070201@Media-Brokers.com> On 2012-04-11 4:48 PM, Adrian Minta wrote: > On 04/11/12 19:50, Ed W wrote: >> One of the snags of md RAID1 vs RAID6 is the lack of checksumming in >> the event of bad blocks. (I'm not sure what actually happens when md >> scrubbing finds a bad sector with raid1..?). For low performance >> requirements I have become paranoid and been using RAID6 vs RAID10, >> filesystems with sector checksums seem attractive... > RAID6 is very slow for write operations. That's why is the worst choice > for maildir. He did say '"For *low* *performance* requirements..." ... ;) -- Best regards, Charles From stan at hardwarefreak.com Thu Apr 12 01:46:44 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 11 Apr 2012 17:46:44 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F8409DA.1090202@gmail.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F8409DA.1090202@gmail.com> Message-ID: <4F8609D4.2040205@hardwarefreak.com> On 4/10/2012 5:22 AM, Adrian Minta wrote: > On 04/10/12 08:00, Stan Hoeppner wrote: >> Interestingly, I designed a COTS server back in January to handle at >> least 5k concurrent IMAP users, using best of breed components. If you >> or someone there has the necessary hardware skills, you could assemble >> this system and simply use it for NFS instead of Dovecot. The parts >> list: >> secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=17069985 > > Don't forget the Battery Backup Unit for RAID card !!! Heh, thanks for the reminder Adrian. :) I got to your email a little late--already corrected the omission. Yes, battery or flash backup for the RAID cache is always a necessity when doing write-back caching. -- Stan From alietsantiesteban at gmail.com Thu Apr 12 02:03:04 2012 From: alietsantiesteban at gmail.com (Aliet Santiesteban Sifontes) Date: Wed, 11 Apr 2012 18:03:04 -0500 Subject: [Dovecot] ldap idle connection timeout in DoveCot 1.0.13? In-Reply-To: References: Message-ID: I had this problem running Dovecot 2.x where LDAP servers are located on another firewall zone, we use Juniper SSG550. The problem was that the firewall was dropping the ldap idle connections so client authentication was failing in dovecot for a while and after a time it reconnects, Dovecot/Openldap-Server never knows that the firewall has dropped the connection because this is the default, the firewall doesn't send TCP -Reset to the client and the server, in Juniper/Netscreen you can do a workaround to speed up the process by configuring the zone to send reset back to the client and the server. Check you have on the firewall: set flow tcp-mss unset flow no-tcp-seq-check set flow tcp-syn-check unset flow tcp-syn-bit-check set flow reverse-route clear-text prefer set flow reverse-route tunnel always Edit your zone and enable "If TCP non SYN, send RESET back" checkbox: This fixed the delay for us, it would be a nice feature at dovecot side... best regards El 11 de abril de 2012 11:36, Timo Sirainen escribi?: > On 11.4.2012, at 17.49, Zhou, Yan wrote: > > > We are using DoveCot 1.0.13, it connects to LDAP server for > authentication. It seems that DoveCot keeps the idle LDAP connection open. > > Yes. > > > Our firewall is terminating these connections after some time of idle > activity (2 hours), then, we run into authentication problem. If we restart > either LDAP or DoveCot, then it is fine. > > > > Can we set some kind of LDAP idle connection timeout in DoveCot? > /etc/dovecot-ldap.conf. I do not see any configuration available for > 1.0.13. > > No. But if you upgrade to a newer Dovecot (v2.x probably) this is solved > by automatic transparent reconnection. > > From stan at hardwarefreak.com Thu Apr 12 04:18:08 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 11 Apr 2012 20:18:08 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F85B641.5060103@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> Message-ID: <4F862D50.8000206@hardwarefreak.com> On 4/11/2012 11:50 AM, Ed W wrote: > Re XFS. Have you been watching BTRFS recently? > > I will concede that despite the authors considering it production ready > I won't be using it for my servers just yet. However, it's benchmarking > on single disk benchmarks fairly similarly to XFS and in certain cases > (multi-threaded performance) can be somewhat better. I haven't yet seen > any benchmarks on larger disk arrays yet, eg 6+ disks, so no idea how it > scales up. Basically what I have seen seems "competitive" Links? > I don't have such hardware spare to benchmark, but I would be interested > to hear from someone who benchmarks your RAID1+linear+XFS suggestion, > especially if they have compared a cutting edge btrfs kernel on the same > array? http://btrfs.boxacle.net/repository/raid/history/History_Mail_server_simulation._num_threads=128.html This is with an 8 wide LVM stripe over 8 17 drive hardware RAID0 arrays. If the disks had been setup as a concat of 68 RAID1 pairs, XFS would have turned in numbers significantly higher, anywhere from a 100% increase to 500%. It's hard to say because the Boxacle folks didn't show the XFG AG config they used. The concat+RAID1 setup can decrease disk seeks by many orders of magnitude vs striping. Everyone knows as seeks go down IOPS go up. Even with this very suboptimal disk setup, XFS still trounces everything but JFS which is a close 2nd. BTRFS is way down in the pack. It would be nice to see these folks update these results with a 3.2.6 kernel, as both BTRFS and XFS have improved significantly since 2.6.35. EXT4 and JFS have seen little performance work since. In fact JFS has seen no commits but bug fixes and changes to allow compiling with recent kernels. > One of the snags of md RAID1 vs RAID6 is the lack of checksumming in the > event of bad blocks. (I'm not sure what actually happens when md > scrubbing finds a bad sector with raid1..?). For low performance > requirements I have become paranoid and been using RAID6 vs RAID10, > filesystems with sector checksums seem attractive... Except we're using hardware RAID1 here and mdraid linear. Thus the controller takes care of sector integrity. RAID6 yields nothing over RAID10, except lower performance, and more usable space if more than 4 drives are used. -- Stan From centos.admin at gmail.com Thu Apr 12 05:23:19 2012 From: centos.admin at gmail.com (Emmanuel Noobadmin) Date: Thu, 12 Apr 2012 10:23:19 +0800 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F862D50.8000206@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> Message-ID: On 4/12/12, Stan Hoeppner wrote: > On 4/11/2012 11:50 AM, Ed W wrote: >> One of the snags of md RAID1 vs RAID6 is the lack of checksumming in the >> event of bad blocks. (I'm not sure what actually happens when md >> scrubbing finds a bad sector with raid1..?). For low performance >> requirements I have become paranoid and been using RAID6 vs RAID10, >> filesystems with sector checksums seem attractive... > > Except we're using hardware RAID1 here and mdraid linear. Thus the > controller takes care of sector integrity. RAID6 yields nothing over > RAID10, except lower performance, and more usable space if more than 4 > drives are used. How would the control ensure sector integrity unless it is writing additional checksum information to disk? I thought only a few filesystems like ZFS does the sector checksum to detect if any data corruption occurred. I suppose the controller could throw an error if the two drives returned data that didn't agree with each other but it wouldn't know which is the accurate copy but that wouldn't protect the integrity of the data, at least not directly without additional human intervention I would think. From tlx at leuxner.net Thu Apr 12 10:11:32 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 12 Apr 2012 09:11:32 +0200 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes Message-ID: <20120412071132.GA2583@nihlus.leuxner.net> Some change between bf5ae73e9475 and 584bd77c38fd seems to have broken something in the SSL Handshake. A previously valid server certificate is deemed invalid by various mail clients. http://hg.dovecot.org/dovecot-2.1/rev/bf5ae73e9475 works fine while http://hg.dovecot.org/dovecot-2.1/rev/584bd77c38fd does not. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From tss at iki.fi Thu Apr 12 10:43:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Apr 2012 10:43:22 +0300 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: <20120412071132.GA2583@nihlus.leuxner.net> References: <20120412071132.GA2583@nihlus.leuxner.net> Message-ID: On 12.4.2012, at 10.11, Thomas Leuxner wrote: > Some change between bf5ae73e9475 and 584bd77c38fd seems to have broken > something in the SSL Handshake. A previously valid server certificate is > deemed invalid by various mail clients. > > http://hg.dovecot.org/dovecot-2.1/rev/bf5ae73e9475 works fine while > http://hg.dovecot.org/dovecot-2.1/rev/584bd77c38fd does not. What kind of a certificate do you have? You have an intermediary cert that exists only in ssl_ca file? I couldn't reproduce this with a test. But anyway, reverted for now: http://hg.dovecot.org/dovecot-2.1/rev/f80f18d0ffa3 Now how do I fix the memory leak then?... From tss at iki.fi Thu Apr 12 10:49:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Apr 2012 10:49:12 +0300 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: References: <20120412071132.GA2583@nihlus.leuxner.net> Message-ID: On 12.4.2012, at 10.43, Timo Sirainen wrote: > On 12.4.2012, at 10.11, Thomas Leuxner wrote: > >> Some change between bf5ae73e9475 and 584bd77c38fd seems to have broken >> something in the SSL Handshake. A previously valid server certificate is >> deemed invalid by various mail clients. >> >> http://hg.dovecot.org/dovecot-2.1/rev/bf5ae73e9475 works fine while >> http://hg.dovecot.org/dovecot-2.1/rev/584bd77c38fd does not. > > What kind of a certificate do you have? You have an intermediary cert that exists only in ssl_ca file? I couldn't reproduce this with a test. But anyway, reverted for now: http://hg.dovecot.org/dovecot-2.1/rev/f80f18d0ffa3 > > Now how do I fix the memory leak then?... http://hg.dovecot.org/dovecot-2.1/rev/85ad4baedd43 ? From tlx at leuxner.net Thu Apr 12 11:16:45 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 12 Apr 2012 10:16:45 +0200 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: References: <20120412071132.GA2583@nihlus.leuxner.net> Message-ID: <20120412081645.GB2583@nihlus.leuxner.net> On Thu, Apr 12, 2012 at 10:43:22AM +0300, Timo Sirainen wrote: > What kind of a certificate do you have? You have an intermediary cert that exists only in ssl_ca file? I couldn't reproduce this with a test. But anyway, reverted for now: http://hg.dovecot.org/dovecot-2.1/rev/f80f18d0ffa3 > Thawte. They only do intermediates for some time now. $ openssl x509 -in /etc/ssl/certs/spectre_leuxner_net_2011.crt -noout -subject -issuer -dates subject= /O=spectre.leuxner.net/OU=Go to https://www.thawte.com/repository/index.html/OU=Thawte SSL123 certificate/OU=Domain Validated/CN=spectre.leuxner.net issuer= /C=US/O=Thawte, Inc./OU=Domain Validated SSL/CN=Thawte DV SSL CA notBefore=May 16 00:00:00 2011 GMT notAfter=Jun 14 23:59:59 2012 GMT [...] ssl_ca = From tss at iki.fi Thu Apr 12 11:17:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Apr 2012 11:17:50 +0300 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: <20120412081645.GB2583@nihlus.leuxner.net> References: <20120412071132.GA2583@nihlus.leuxner.net> <20120412081645.GB2583@nihlus.leuxner.net> Message-ID: <11B240E5-087D-43A2-A687-633AB3C4A367@iki.fi> On 12.4.2012, at 11.16, Thomas Leuxner wrote: > On Thu, Apr 12, 2012 at 10:43:22AM +0300, Timo Sirainen wrote: >> What kind of a certificate do you have? You have an intermediary cert that exists only in ssl_ca file? I couldn't reproduce this with a test. But anyway, reverted for now: http://hg.dovecot.org/dovecot-2.1/rev/f80f18d0ffa3 >> > > Thawte. They only do intermediates for some time now. But do you keep your intermediate cert in ssl_ca file or ssl_cert file? From tlx at leuxner.net Thu Apr 12 11:33:58 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 12 Apr 2012 10:33:58 +0200 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: <11B240E5-087D-43A2-A687-633AB3C4A367@iki.fi> References: <20120412071132.GA2583@nihlus.leuxner.net> <20120412081645.GB2583@nihlus.leuxner.net> <11B240E5-087D-43A2-A687-633AB3C4A367@iki.fi> Message-ID: <20120412083358.GA14349@nihlus.leuxner.net> On Thu, Apr 12, 2012 at 11:17:50AM +0300, Timo Sirainen wrote: > But do you keep your intermediate cert in ssl_ca file or ssl_cert file? Separate. Root and intermediate are in ssl_ca: $ cat /etc/ssl/certs/SSL123_CA_Bundle.pem -----BEGIN CERTIFICATE----- MIIEjzCCA3egAwIBAgIQdhASihe2grs6H50amjXAkjANBgkqhkiG9w0BAQUFADCB qTELMAkGA1UEBhMCVVMxFTATBgNVBAoTDHRoYXd0ZSwgSW5jLjEoMCYGA1UECxMf Q2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjE4MDYGA1UECxMvKGMpIDIw MDYgdGhhd3RlLCBJbmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxHzAdBgNV BAMTFnRoYXd0ZSBQcmltYXJ5IFJvb3QgQ0EwHhcNMTAwMjE4MDAwMDAwWhcNMjAw MjE3MjM1OTU5WjBeMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMVGhhd3RlLCBJbmMu MR0wGwYDVQQLExREb21haW4gVmFsaWRhdGVkIFNTTDEZMBcGA1UEAxMQVGhhd3Rl IERWIFNTTCBDQTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMuYyTY/ 0pzYFgfUSWP5g7DoAi3MXFp0l6YT7xMT3gV8p+bKACPaOfnvE89Sxa+a48q+84LZ iz2q4cyuiFBmoy3sYRR1SasOJPGsRFsLKKIzIHYeBmBqZwVxi7pmYhZ6s20Nx9CU QMaMPR6SDGI0DUSJ1feJ/intGI/2mysI92qr2EiXWvSf7Qx1UiL31V6EAJ/ASg0x d0xk0BLmDzrwocDVXB3nXy3C99Y2GNmVbkROyVgUTbaOu83eYh76W7W9GCuYrKyT P1Ba9RQLos+2855PWs1awzYj2hqvsE3WSiIDj0MCGb3qrN3EejUyFPFyLghVQAz0 B0FBrzg3hClCslUCAwEAAaOB/DCB+TAyBggrBgEFBQcBAQQmMCQwIgYIKwYBBQUH MAGGFmh0dHA6Ly9vY3NwLnRoYXd0ZS5jb20wEgYDVR0TAQH/BAgwBgEB/wIBADA0 BgNVHR8ELTArMCmgJ6AlhiNodHRwOi8vY3JsLnRoYXd0ZS5jb20vVGhhd3RlUENB LmNybDAOBgNVHQ8BAf8EBAMCAQYwKQYDVR0RBCIwIKQeMBwxGjAYBgNVBAMTEVZl cmlTaWduTVBLSS0yLTExMB0GA1UdDgQWBBSrRORd7IPH2cCFn/fhxpeQsIw/mDAf BgNVHSMEGDAWgBR7W0XPr87Lev0xkhpqtvNG61dIUDANBgkqhkiG9w0BAQUFAAOC AQEABLr7rLv8S1QRoy2Iszy9AG2KGraNxMGD+MdTKsEybjqBoVR92ho/OkVPNudC sApChZegrPvlh6eDT+ixt5tYZW4mgAuSTUdVuWEWUWXpK/Fo2Vi4A4HRt2Yc07zF pntfPsU4RnbndbSgDEvOosKpwcw2c3v7uSQkoF6n9vq7DChDnh3wTvA/2CSwIdxt Le6/Wjv6iJx0bK8h3ZLswxXvlHUmRtamP79mSKod790n5rdRiTh9E4QMQPzQtfHg 2/lPL0ActI5HImG4TJbe8F8Rfk8R2exQRyIOxR3iZEnnaGNFOorZcfRe8W63FE0+ bxQe3FL+vN8MvSk/dvsRX2hoFQ== -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- MIIERTCCA66gAwIBAgIQM2VQCHmtc+IwueAdDX+skTANBgkqhkiG9w0BAQUFADCB zjELMAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJ Q2FwZSBUb3duMR0wGwYDVQQKExRUaGF3dGUgQ29uc3VsdGluZyBjYzEoMCYGA1UE CxMfQ2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjEhMB8GA1UEAxMYVGhh d3RlIFByZW1pdW0gU2VydmVyIENBMSgwJgYJKoZIhvcNAQkBFhlwcmVtaXVtLXNl cnZlckB0aGF3dGUuY29tMB4XDTA2MTExNzAwMDAwMFoXDTIwMTIzMDIzNTk1OVow gakxCzAJBgNVBAYTAlVTMRUwEwYDVQQKEwx0aGF3dGUsIEluYy4xKDAmBgNVBAsT H0NlcnRpZmljYXRpb24gU2VydmljZXMgRGl2aXNpb24xODA2BgNVBAsTLyhjKSAy MDA2IHRoYXd0ZSwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MR8wHQYD VQQDExZ0aGF3dGUgUHJpbWFyeSBSb290IENBMIIBIjANBgkqhkiG9w0BAQEFAAOC AQ8AMIIBCgKCAQEArKDw+4BZ1JzHpM+doVlzCRBFDA0sbmjxbFtIaElZN/wLMxnC d3/MEC2VNBzm600JpxzSuMmXNgK3idQkXwbAzESUlI0CYm/rWt0RjSiaXISQEHoN vXRmL2o4oOLVVETrHQefB7pv7un9Tgsp9T6EoAHxnKv4HH6JpOih2HFlDaNRe+68 0iJgDblbnd+6/FFbC6+Ysuku6QToYofeK8jXTsFMZB7dz4dYukpPymgHHRydSsbV L5HMfHFyHMXAZ+sy/cmSXJTahcCbv1N9Kwn0jJ2RH5dqUsveCTakd9h7h1BE1T5u KWn7OUkmHgmlgHtALevoJ4XJ/mH9fuZ8lx3VnQIDAQABo4HCMIG/MA8GA1UdEwEB /wQFMAMBAf8wOwYDVR0gBDQwMjAwBgRVHSAAMCgwJgYIKwYBBQUHAgEWGmh0dHBz Oi8vd3d3LnRoYXd0ZS5jb20vY3BzMA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQU e1tFz6/Oy3r9MZIaarbzRutXSFAwQAYDVR0fBDkwNzA1oDOgMYYvaHR0cDovL2Ny bC50aGF3dGUuY29tL1RoYXd0ZVByZW1pdW1TZXJ2ZXJDQS5jcmwwDQYJKoZIhvcN AQEFBQADgYEAhKhMyT4qvJrizI8LsiV3xGGJiWNa1KMVQNT7Xj+0Q+pjFytrmXSe Cajd1FYVLnp5MV9jllMbNNkV6k9tcMq+9oKp7dqFd8x2HGqBCiHYQZl/Xi6Cweiq 95OBBaqStB+3msAHF/XLxrRMDtdW3HEgdDjWdMbWj2uvi42gbCkLYeA= -----END CERTIFICATE----- $ dovecot --version 2.1.4 (584bd77c38fd) Seems to have fixed it. Thanks. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From tss at iki.fi Thu Apr 12 11:35:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Apr 2012 11:35:48 +0300 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: <20120412083358.GA14349@nihlus.leuxner.net> References: <20120412071132.GA2583@nihlus.leuxner.net> <20120412081645.GB2583@nihlus.leuxner.net> <11B240E5-087D-43A2-A687-633AB3C4A367@iki.fi> <20120412083358.GA14349@nihlus.leuxner.net> Message-ID: On 12.4.2012, at 11.33, Thomas Leuxner wrote: > On Thu, Apr 12, 2012 at 11:17:50AM +0300, Timo Sirainen wrote: >> But do you keep your intermediate cert in ssl_ca file or ssl_cert file? > > Separate. Root and intermediate are in ssl_ca: The documentation tells to put the intermediary to ssl_cert though. I didn't even know it worked in ssl_ca. But I guess I won't intentionally break it.. From tlx at leuxner.net Thu Apr 12 11:41:52 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 12 Apr 2012 10:41:52 +0200 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: References: <20120412071132.GA2583@nihlus.leuxner.net> <20120412081645.GB2583@nihlus.leuxner.net> <11B240E5-087D-43A2-A687-633AB3C4A367@iki.fi> <20120412083358.GA14349@nihlus.leuxner.net> Message-ID: <20120412084152.GB14349@nihlus.leuxner.net> On Thu, Apr 12, 2012 at 11:35:48AM +0300, Timo Sirainen wrote: > On 12.4.2012, at 11.33, Thomas Leuxner wrote: > > > On Thu, Apr 12, 2012 at 11:17:50AM +0300, Timo Sirainen wrote: > >> But do you keep your intermediate cert in ssl_ca file or ssl_cert file? > > > > Separate. Root and intermediate are in ssl_ca: > > The documentation tells to put the intermediary to ssl_cert though. I didn't even know it worked in ssl_ca. But I guess I won't intentionally break it.. Hmmm. I did emulate Thawte instructions though: https://search.thawte.com/support/ssl-digital-certificates/index?page=content&id=SO15464&actp=LIST&viewlocale=en_US https://search.thawte.com/library/VERISIGN/ALL_OTHER/thawte%20ca/SSL123_CA_Bundle.pem [...] SSLCertificateFile /usr/local/ssl/crt/domainname.crt SSLCertificateKeyFile /usr/local/ssl/private/server.key SSLCACertificateFile /usr/local/ssl/crt/cabundle.crt -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From stan at hardwarefreak.com Thu Apr 12 13:20:31 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 12 Apr 2012 05:20:31 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> Message-ID: <4F86AC6F.4070304@hardwarefreak.com> On 4/11/2012 9:23 PM, Emmanuel Noobadmin wrote: > On 4/12/12, Stan Hoeppner wrote: >> On 4/11/2012 11:50 AM, Ed W wrote: >>> One of the snags of md RAID1 vs RAID6 is the lack of checksumming in the >>> event of bad blocks. (I'm not sure what actually happens when md >>> scrubbing finds a bad sector with raid1..?). For low performance >>> requirements I have become paranoid and been using RAID6 vs RAID10, >>> filesystems with sector checksums seem attractive... >> >> Except we're using hardware RAID1 here and mdraid linear. Thus the >> controller takes care of sector integrity. RAID6 yields nothing over >> RAID10, except lower performance, and more usable space if more than 4 >> drives are used. > > How would the control ensure sector integrity unless it is writing > additional checksum information to disk? I thought only a few > filesystems like ZFS does the sector checksum to detect if any data > corruption occurred. I suppose the controller could throw an error if > the two drives returned data that didn't agree with each other but it > wouldn't know which is the accurate copy but that wouldn't protect the > integrity of the data, at least not directly without additional human > intervention I would think. When a drive starts throwing uncorrectable read errors, the controller faults the drive and tells you to replace it. Good hardware RAID controllers are notorious for their penchant to kick drives that would continue to work just fine in mdraid or as a single drive for many more years. The mindset here is that anyone would rather spent $150-$2500 dollars on a replacement drive than take a chance with his/her valuable data. Yes I typed $2500. EMC charges over $2000 for a single Seagate disk drive with an EMC label and serial# on it. The serial number is what prevents one from taking the same off the shelf Seagate drive at $300 and mounting it in a $250,000 EMC array chassis. The controller firmware reads the S/N from each connected drive and will not allow foreign drives to be used. HP, IBM, Oracle/Sun, etc do this as well. Which is why they make lots of profit, and is why I prefer open storage systems. -- Stan From lists at wildgooses.com Thu Apr 12 13:58:52 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 12 Apr 2012 11:58:52 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F86AC6F.4070304@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> Message-ID: <4F86B56C.6010201@wildgooses.com> On 12/04/2012 11:20, Stan Hoeppner wrote: > On 4/11/2012 9:23 PM, Emmanuel Noobadmin wrote: >> On 4/12/12, Stan Hoeppner wrote: >>> On 4/11/2012 11:50 AM, Ed W wrote: >>>> One of the snags of md RAID1 vs RAID6 is the lack of checksumming in the >>>> event of bad blocks. (I'm not sure what actually happens when md >>>> scrubbing finds a bad sector with raid1..?). For low performance >>>> requirements I have become paranoid and been using RAID6 vs RAID10, >>>> filesystems with sector checksums seem attractive... >>> Except we're using hardware RAID1 here and mdraid linear. Thus the >>> controller takes care of sector integrity. RAID6 yields nothing over >>> RAID10, except lower performance, and more usable space if more than 4 >>> drives are used. >> How would the control ensure sector integrity unless it is writing >> additional checksum information to disk? I thought only a few >> filesystems like ZFS does the sector checksum to detect if any data >> corruption occurred. I suppose the controller could throw an error if >> the two drives returned data that didn't agree with each other but it >> wouldn't know which is the accurate copy but that wouldn't protect the >> integrity of the data, at least not directly without additional human >> intervention I would think. > When a drive starts throwing uncorrectable read errors, the controller > faults the drive and tells you to replace it. Good hardware RAID > controllers are notorious for their penchant to kick drives that would > continue to work just fine in mdraid or as a single drive for many more > years. The mindset here is that anyone would rather spent $150-$2500 > dollars on a replacement drive than take a chance with his/her valuable > data. > I'm asking a subtlely different question. The claim by ZFS/BTRFS authors and others is that data silently "bit rots" on it's own. The claim is therefore that you can have a raid1 pair where neither drive reports a hardware failure, but each gives you different data? I can't personally claim to have observed this, so it remains someone else's theory... (for background my experience is simply: RAID10 for high performance arrays and RAID6 for all my personal data - I intend to investigate your linear raid idea in the future though) I do agree that if one drive reports a read error, then it's quite easy to guess which pair of the array is wrong... Just as an aside, I don't have a lot of failure experience. However, the few I have had (perhaps 6-8 events now) is that there is a massive correlation in failure time with RAID1, eg one pair I had lasted perhaps 2 years and then both failed within 6 hours of each other. I also had a bad experience with RAID 5 that wasn't being scrubbed regularly and when one drive started reporting errors (ie lack of monitoring meant it had been bad for a while), the rest of the array turned out to be a patchwork of read errors - linux raid then turns out to be quite fragile in the presence of a small number of read failures and it's extremely difficult to salvage the 99% of the array which is ok due to the disks getting kicked out... (of course regular scrubs would have prevented getting so deep into that situation - it was a small cheap nas box without such features) Ed W From tss at iki.fi Thu Apr 12 14:09:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Apr 2012 14:09:31 +0300 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F86B56C.6010201@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> Message-ID: <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> On 12.4.2012, at 13.58, Ed W wrote: > The claim by ZFS/BTRFS authors and others is that data silently "bit rots" on it's own. The claim is therefore that you can have a raid1 pair where neither drive reports a hardware failure, but each gives you different data? That's one reason why I planned on adding a checksum to each message in dbox. But I forgot to actually do that. I guess I could add it for new messages in some upcoming version. Then Dovecot could optionally verify the checksum before returning the message to client, and if it detects corruption perhaps automatically read it from some alternative location (e.g. if dsync replication is enabled ask from another replica). And Dovecot index files really should have had some small (8/16/32bit) checksums of stuff as well.. From lists at wildgooses.com Thu Apr 12 14:45:51 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 12 Apr 2012 12:45:51 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F862D50.8000206@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> Message-ID: <4F86C06F.3060109@wildgooses.com> On 12/04/2012 02:18, Stan Hoeppner wrote: > On 4/11/2012 11:50 AM, Ed W wrote: >> Re XFS. Have you been watching BTRFS recently? >> >> I will concede that despite the authors considering it production ready >> I won't be using it for my servers just yet. However, it's benchmarking >> on single disk benchmarks fairly similarly to XFS and in certain cases >> (multi-threaded performance) can be somewhat better. I haven't yet seen >> any benchmarks on larger disk arrays yet, eg 6+ disks, so no idea how it >> scales up. Basically what I have seen seems "competitive" > Links? http://btrfs.ipv5.de/index.php?title=Main_Page#Benchmarking See the regular Phoronix benchmarks in particular. However, I believe these are all single disk? >> I don't have such hardware spare to benchmark, but I would be interested >> to hear from someone who benchmarks your RAID1+linear+XFS suggestion, >> especially if they have compared a cutting edge btrfs kernel on the same >> array? > http://btrfs.boxacle.net/repository/raid/history/History_Mail_server_simulation._num_threads=128.html > > This is with an 8 wide LVM stripe over 8 17 drive hardware RAID0 arrays. > If the disks had been setup as a concat of 68 RAID1 pairs, XFS would > have turned in numbers significantly higher, anywhere from a 100% > increase to 500%. My instinct is that this is an irrelevant benchmark for BTRFS because its performance characteristics for these workloads have changed so significantly? I would be far more interested in a 3.2 and then a 3.6/3.7 benchmark in a years time In particular recent benchmarks on Phoronix show btrfs exceeding XFS performance on heavily threaded benchmarks - however, I doubt this is representative of performance on a multi-disk benchmark? > It would be nice to see these folks update these > results with a 3.2.6 kernel, as both BTRFS and XFS have improved > significantly since 2.6.35. EXT4 and JFS have seen little performance > work since. My understanding is that there was a significant multi-thread performance boost for EXT4 in the last year kind of timeframe? I don't have a link to hand, but someone did some work to reduce lock contention (??) which I seem to recall made a very large difference on multi-user or multi-cpu workloads? I seem to recall that the summary was that it allowed Ext4 to scale up to a good fraction of XFS performance on "medium sized" systems? (I believe that XFS still continues to scale far better than anything else on large systems) Point is that I think it's a bit unfair to say that little has changed on Ext4? It still seems to be developing faster than "maintenance only" However, well OT... The original question was: anyone tried very recent BTRFS on a multi-disk system. Seems like the answer is no. My proposal is that it may be worth watching in the future Cheers Ed W P.S. I have always been intrigued by the idea that a COW based filesystem could potentially implement much faster "RAID" parity, because it can avoid reading the whole stripe. The idea is that you treat unallocated space as "zero", which means you can compute the incremental parity with only a read/write of the checksum value (and with a COW filesystem you only ever update by rewriting to new "zero'd" space). I had in mind something like a fixed parity disk (RAID4?) and allowing the parity disk to be "write behind" cached in ram (ie exposed to risk of: power fails AND data disk fails at the same time). My code may not be following along for a while though... From zybi at talex.pl Thu Apr 12 14:47:49 2012 From: zybi at talex.pl (=?ISO-8859-2?Q?Artur_Zaprza=B3a?=) Date: Thu, 12 Apr 2012 13:47:49 +0200 Subject: [Dovecot] "doveadm mailbox status" destroys compressed messages without W= attribute Message-ID: <4F86C0E5.6090709@talex.pl> I still have some old messages from previous mail server. This messages have no W= attribute either in file name or dovecot-uidlist and are compressed with gzip. Running "doveadm mailbox status -A vsize \*" will result in the following messages: doveadm(foo at domain): Error: Cached message size larger than expected (2580 > 1451) doveadm(foo at domain): Error: Maildir filename has wrong S value, renamed the file from /vmail/domain/foo/Maildir/.Sent/cur/1206550323.M125837P13306V0000000000000812I0000809E.oldname,S=2580:2,SZ to /vmail/domain/foo/Maildir/.Sent/cur/1206550323.M125837P13306V0000000000000812I0000809E.oldname,S=1451:2,SZ doveadm(foo at domain): Error: Corrupted index cache file /vmail/domain/foo/Maildir/.Sent/dovecot.index.cache: Broken physical size for mail UID 2 doveadm(foo at domain): Error: Cached message size larger than expected (2580 > 1451) doveadm(foo at domain): Error: Corrupted index cache file /vmail/domain/foo/Maildir/.Sent/dovecot.index.cache: Broken physical size for mail UID 2 doveadm(foo at domain): Error: read(/vmail/domain/foo/Maildir/.Sent/cur/1206550323.M125837P13306V0000000000000812I0000809E.oldname,S=2580:2,SZ) failed: Input/output error (uid=2) (Size of uncompressed message is 2580 and compressed size is 1451) I have enabled zlib plugin for imap, pop3, lda and lmtp. But how to enable it for doveadm? -- Best regards, Artur Zaprza?a From tss at iki.fi Thu Apr 12 14:49:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Apr 2012 14:49:44 +0300 Subject: [Dovecot] "doveadm mailbox status" destroys compressed messages without W= attribute In-Reply-To: <4F86C0E5.6090709@talex.pl> References: <4F86C0E5.6090709@talex.pl> Message-ID: <5CA9296A-87CB-4F47-AB1B-67533F866248@iki.fi> On 12.4.2012, at 14.47, Artur Zaprza?a wrote: > I have enabled zlib plugin for imap, pop3, lda and lmtp. But how to enable it for doveadm? Just set it globally: mail_plugins = zlib From lists at wildgooses.com Thu Apr 12 15:10:20 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 12 Apr 2012 13:10:20 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> Message-ID: <4F86C62C.8030305@wildgooses.com> On 12/04/2012 12:09, Timo Sirainen wrote: > On 12.4.2012, at 13.58, Ed W wrote: > >> The claim by ZFS/BTRFS authors and others is that data silently "bit rots" on it's own. The claim is therefore that you can have a raid1 pair where neither drive reports a hardware failure, but each gives you different data? > That's one reason why I planned on adding a checksum to each message in dbox. But I forgot to actually do that. I guess I could add it for new messages in some upcoming version. Then Dovecot could optionally verify the checksum before returning the message to client, and if it detects corruption perhaps automatically read it from some alternative location (e.g. if dsync replication is enabled ask from another replica). And Dovecot index files really should have had some small (8/16/32bit) checksums of stuff as well.. > I have to say - I haven't actually seen this happen... Do any of your big mailstore contacts observe this, eg rackspace, etc? I think it's worth thinking about the failure cases before implementing something to be honest? Just sticking in a checksum possibly doesn't help anyone unless it's on the right stuff and in the right place? Off the top of my head: - Someone butchers the file on disk (disk error or someone edits it with vi) - Restore of some files goes subtly wrong, eg tool tries to be clever and fails, snapshot taken mid-write, etc? - Filesystem crash (sudden power loss), how to deal with partial writes? Things I might like to do *if* there were some suitable "checksums" available: - Use the checksum as some kind of guid either for the whole message, the message minus the headers, or individual mime sections - Use the checksums to assist with replication speed/efficiency (dsync or custom imap commands) - File RFCs for new imap features along the "lemonde" lines which allow clients to have faster recovery from corrupted offline states... - Single instance storage (presumably already done, and of course this has some subtleties in the face of deliberate attack) - Possibly duplicate email suppression (but really this is an LDA problem...) - Storage backends where emails are redundantly stored and might not ALL be on a single server (find me the closest copy of email X) - derivations of this might be interesting for compliance archiving of messages? - Fancy key-value storage backends might use checksums as part of the key value (either for the whole or parts of the message) The mail server has always looked like a kind of key-value store to my eye. However, traditional key-value isn't usually optimised for "streaming reads", hence dovecot seems like a "key value store, optimised for sequential high speed streaming access to the key values"... Whilst it seems increasingly unlikely that a traditional key-value store will work well to replace say mdbox, I wonder if it's not worth looking at the replication strategies of key-value stores to see if those ideas couldn't lead to new features for mdbox? Cheers Ed W From dlie76 at yahoo.com.au Thu Apr 12 16:27:38 2012 From: dlie76 at yahoo.com.au (Daminto Lie) Date: Thu, 12 Apr 2012 06:27:38 -0700 (PDT) Subject: [Dovecot] vacation plugins for squirrelmail Message-ID: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> Hi, I am afraid I have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. Any help would be very much appreciated. Thank you From zybi at talex.pl Thu Apr 12 16:47:41 2012 From: zybi at talex.pl (=?UTF-8?B?QXJ0dXIgWmFwcnphxYJh?=) Date: Thu, 12 Apr 2012 15:47:41 +0200 Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> Message-ID: <4F86DCFD.5020007@talex.pl> Daminto Lie wrote: > Hi, > > I am afraid I have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. > > Any help would be very much appreciated. > > Thank you > I'm using Avelsieve 1.9.9 with a set of my own bugfixes: http://email.uoa.gr/avelsieve/ From dirk.jahnke-zumbusch at desy.de Thu Apr 12 17:08:31 2012 From: dirk.jahnke-zumbusch at desy.de (Dirk Jahnke-Zumbusch) Date: Thu, 12 Apr 2012 16:08:31 +0200 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F86C62C.8030305@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> <4F86C62C.8030305@wildgooses.com> Message-ID: <4F86E1DF.30800@desy.de> Hi there, > I have to say - I haven't actually seen this happen... Do any of your > big mailstore contacts observe this, eg rackspace, etc? Just to throw in to the discussion that with (silent) data corruption not only "the disk" is involved but many other parts of your systems. So perhaps you would like to have a look at https://indico.desy.de/getFile.py/access?contribId=65&sessionId=42&resId=0&materialId=slides&confId=257 http://indico.cern.ch/getFile.py/access?contribId=3&sessionId=0&resId=1&materialId=paper&confId=13797 The documents are from 2007 but the principals are still the same. Kind regards Dirk From helga.mayer at uni-hohenheim.de Thu Apr 12 18:31:35 2012 From: helga.mayer at uni-hohenheim.de (Helga Mayer) Date: Thu, 12 Apr 2012 17:31:35 +0200 Subject: [Dovecot] Problems with Apple Mail and attachments In-Reply-To: References: <20120411161321.165760epurnu7i4h@webmail.uni-hohenheim.de> Message-ID: <20120412173135.10654u9eo7r2pumv@webmail.uni-hohenheim.de> Quoting Thierry de Montaudry : > I've seen a similar problem a while ago (1 year maybe more), but > used the mailbox Rebuild option on the client, which fixed it > without having to delete and recreate the account. Your problem > might just be a local index corruption, which can happen when > loosing your Internet connection. Thank you, we will will try it. Regards Helga Helga Mayer Universit?t Hohenheim Kommunikations-, Informations- und Medienzentrum (630) IT-Dienste | Mail Schloss-Westhof-S?d | 70599 Stuttgart Tel.: +49 711 459-22838 | Fax: +49 711 459-23449 https://kim.uni-hohenheim.de From am at am.cx Thu Apr 12 18:35:15 2012 From: am at am.cx (Andrea Mistrali) Date: Thu, 12 Apr 2012 17:35:15 +0200 Subject: [Dovecot] Problems with master user Message-ID: Hi to all! I?m trying to setup master users, but I have some problems. Namely I can authenticate, but after it I cannot access INBOX or other mailboxes of the user. My configuration is: passdb { driver = ldap args = /etc/dovecot/ldap-passdb.conf } passdb { driver = sql args = /etc/dovecot/sql.conf } passdb { driver = passwd-file args = /etc/dovecot/passwd.masterusers master = yes pass = yes } userdb { driver = sql args = /etc/dovecot/sql.conf } (I look up for auth in LDAP server first, if it fails I look up in DB, else I check for master user) and relevant files are /etc/dovecot/sql.conf ?????????? password_query = SELECT fullusername as user, \ password, \ uid AS userdb_uid, \ gid AS userdb_gid, \ home AS userdb_home, \ mail AS userdb_mail, \ groups as userdb_acl_groups, \ quota_rule as userdb_quota_rule \ FROM pd_users_full WHERE \ username = '%n' AND \ domain = '%d' AND \ external_auth IS FALSE AND \ master_user IS FALSE AND \ %Ls_ok IS TRUE user_query = SELECT fullusername as user, \ uid, \ gid, \ home, \ mail, \ groups as acl_groups, \ quota_rule \ FROM pd_users_full WHERE \ username = '%n' AND \ domain = '%d' AND \ master_user IS FALSE iterate_query = SELECT fullusername as username, fullusername as user \ FROM pd_users_full where master_user IS FALSE ORDER BY domain,username /etc/dovecot/ldap-passdb.conf ?????????????? uris = ldap://dioniso.cube.lan base = cn=users,dc=cube,dc=lan auth_bind = yes auth_bind_userdn = uid=%n,cn=users,dc=cube,dc=lan pass_attrs = uid=username, \ userPassword=password, \ # uidNumber=userdb_uid, \ # =userdb_home=/var/mail/cubeholding.com/%Lu, \ # =userdb_domain=cubeholding.com, \ # =userdb_mail=maildir:~/maildir/:INBOX=~/maildir/INBOX:LAYOUT=fs:INDEX=~/indexes/ pass_filter = (&(objectClass=posixAccount)(uid=%n)(mail=*@%d)) # Attributes and filter to get a list of all users # iterate_attrs = uid=username iterate_attrs = uid=user iterate_filter = (&(objectClass=posixAccount)(mail=*@%d)) If I test with doveadm auth and doveadm user I receive this: # doveadm auth -x service=imap andre at am.cx\*master at am.cx XXXXXX passdb: andre at am.cx*master at am.cx auth succeeded extra fields: user=andre at am.cx # doveadm user andre at am.cx userdb: andre at am.cx uid : 10010 gid : 8 home : /var/mail/am.cx/andre mail : maildir:~/maildir:INBOX=~/maildir/INBOX:LAYOUT=fs:INDEX=~/indexes/ acl_groups: quota_rule: *:storage=10G and in log files I see: 20120412 17:31:26 auth: Info: passdb(master at am.cx,master): Master user logging in as andre at am.cx 20120412 17:31:26 auth: Info: ldap(andre at am.cx): invalid credentials (given password: XXXXXX) but if I try the real thing: # telnet localhost 143 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE ACL QUOTA NAMESPACE STARTTLS AUTH=PLAIN AUTH=LOGIN AUTH=GSSAPI] Dovecot ready. 0 login andre at am.cx*master at am.cx XXXXXX 0 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE ACL QUOTA NAMESPACE COMPRESS=DEFLATE QUOTA ACL RIGHTS=texk] Logged in 0 select INBOX 0 NO [SERVERBUG] Internal error occurred. Refer to server log for more information. [2012-04-12 17:33:15] and in log file I have: 20120412 17:34:25 auth: Info: passdb(master at am.cx,127.0.0.1,master): Master user logging in as andre at am.cx 20120412 17:34:25 auth: Info: ldap(andre at am.cx,127.0.0.1): invalid credentials (given password: silmaril) 20120412 17:34:25 imap-login: Info: Login: pid=1673, , 127.0.0.1/127.0.0.1, PLAIN, secured 20120412 17:34:47 imap(andre at am.cx): Error: Opening INBOX failed: Mailbox doesn't exist: INBOX Can someone tell me what is wrong in my setup? TIA A. From am at am.cx Thu Apr 12 19:11:36 2012 From: am at am.cx (Andrea Mistrali) Date: Thu, 12 Apr 2012 18:11:36 +0200 Subject: [Dovecot] Problems with master user In-Reply-To: <533C3A4E-2E3A-4B95-84DB-98A4903BBD2E@am.cx> References: <533C3A4E-2E3A-4B95-84DB-98A4903BBD2E@am.cx> Message-ID: Il giorno 12/apr/2012, alle ore 17.35, Andrea Mistrali ha scritto: > Hi to all! > I?m trying to setup master users, but I have some problems. Namely I can authenticate, but after it I cannot access INBOX or other mailboxes of the user. > snip > Can someone tell me what is wrong in my setup? Solved! It is a problem of ACL as stated at http://master.wiki2.dovecot.org/Authentication/MasterUsers#ACLs Sorry A. From root.kev at gmail.com Thu Apr 12 23:48:12 2012 From: root.kev at gmail.com (Root Kev) Date: Thu, 12 Apr 2012 16:48:12 -0400 Subject: [Dovecot] POP3 Dovecot Auth CPU usage 75%+ In-Reply-To: References: Message-ID: Hello all, I hope someone can help me, I have been testing out Dovecot to switch from popa3d which I use at the moment. When I get several users connecting and disconnection multiple times, the Dovecot process with command Auth uses 50-90% of the CPU for the period which they are connecting. I am wondering if there is something that I may have misconfigured, or if there is something that I can change so that this spike doesn't occur. If anyone could shed some light on the issue, I would appreciate it, Kevin /var/mail# dovecot -n # 2.1.4: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-33-generic-pae i686 Ubuntu 10.04.4 LTS ext4 auth_cache_size = 10 M auth_verbose = yes disable_plaintext_auth = no instance_name = Mail Popper 1 listen = 172.20.20.222 login_greeting = Mail Popper 1 Ready mail_location = mbox:/var/empty:INBOX=/var/mail/%u:INDEX=MEMORY mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = shadow } protocols = pop3 service pop3-login { service_count = 0 } ssl = no userdb { driver = passwd } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } From root.kev at gmail.com Thu Apr 12 23:46:46 2012 From: root.kev at gmail.com (Root Kev) Date: Thu, 12 Apr 2012 16:46:46 -0400 Subject: [Dovecot] POP3 Dovecot Auth CPU usage 75%+ Message-ID: Hello all, I hope someone can help me, I have been testing out Dovecot to switch from popa3d which I use at the moment. When I get several users connecting and disconnection multiple times, the Dovecot process with command Auth uses 50-90% of the CPU for the period which they are connecting. I am wondering if there is something that I may have misconfigured, or if there is something that I can change so that this spike doesn't occur. If anyone could shed some light on the issue, I would appreciate it, Kevin /var/mail# dovecot -n # 2.1.4: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-33-generic-pae i686 Ubuntu 10.04.4 LTS ext4 auth_cache_size = 10 M auth_verbose = yes disable_plaintext_auth = no instance_name = Mail Popper 1 listen = 172.20.20.222 login_greeting = Mail Popper 1 Ready mail_location = mbox:/var/empty:INBOX=/var/mail/%u:INDEX=MEMORY mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = shadow } protocols = pop3 service pop3-login { service_count = 0 } ssl = no userdb { driver = passwd } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } From michael at orlitzky.com Fri Apr 13 02:33:22 2012 From: michael at orlitzky.com (Michael Orlitzky) Date: Thu, 12 Apr 2012 19:33:22 -0400 Subject: [Dovecot] [OT] Outlook identities Message-ID: <4F876642.1040701@orlitzky.com> Nothing to do with Dovecot, but I figured this is the best place to ask. Do any of the newer versions of Outlook have proper identities support like Thunderbird, mutt, Roundcube, i.e. every other mail client on Earth? We have customers who set up ten different mailboxes for one person because otherwise Outlook won't Do the Right Thing. Is there some way to make it behave like the others? * When sending new mail, you can choose which address to use. * When replying to mail, it sends from the address that the message was sent to by default. * All mail winds up in one inbox. Outlook (2003, 2007) does do this if you set up different mail accounts, but we shouldn't have to do that. From stan at hardwarefreak.com Fri Apr 13 08:29:52 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 13 Apr 2012 00:29:52 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F86B56C.6010201@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> Message-ID: <4F87B9D0.5010300@hardwarefreak.com> On 4/12/2012 5:58 AM, Ed W wrote: > The claim by ZFS/BTRFS authors and others is that data silently "bit > rots" on it's own. The claim is therefore that you can have a raid1 pair > where neither drive reports a hardware failure, but each gives you > different data? You need to read those articles again very carefully. If you don't understand what they mean by "1 in 10^15 bits non-recoverable read error rate" and combined probability, let me know. And this has zero bearing on RAID1. And RAID1 reads don't work the way you describe above. I explained this in some detail recently. > I do agree that if one drive reports a read error, then it's quite easy > to guess which pair of the array is wrong... Been working that way for more than 2 decades Ed. :) Note that "RAID1" has that "1" for a reason. It was the first RAID level. It was in production for many many years before parity RAID hit the market. It is the most well understood of all RAID levels, and the simplest. -- Stan From centos.admin at gmail.com Fri Apr 13 09:12:48 2012 From: centos.admin at gmail.com (Emmanuel Noobadmin) Date: Fri, 13 Apr 2012 14:12:48 +0800 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F86AC6F.4070304@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> Message-ID: On 4/12/12, Stan Hoeppner wrote: > On 4/11/2012 9:23 PM, Emmanuel Noobadmin wrote: > I suppose the controller could throw an error if >> the two drives returned data that didn't agree with each other but it >> wouldn't know which is the accurate copy but that wouldn't protect the >> integrity of the data, at least not directly without additional human >> intervention I would think. > > When a drive starts throwing uncorrectable read errors, the controller > faults the drive and tells you to replace it. Good hardware RAID > controllers are notorious for their penchant to kick drives that would > continue to work just fine in mdraid or as a single drive for many more > years. What I meant wasn't the drive throwing uncorrectable read errors but the drives are returning different data that each think is correct or both may have sent the correct data but one of the set got corrupted on the fly. After reading the articles posted, maybe the correct term would be the controller receiving silently corrupted data, say due to bad cable on one. If the controller simply returns the fastest result, it could be the bad sector and that doesn't protect the integrity of the data right? if the controller gets 1st half from one drive and 2nd half from the other drive to speed up performance, we could still get the corrupted half and the controller itself still can't tell if the sector it got was corrupted isn't it? If the controller compares the two sectors from the drives, it may be able to tell us something is wrong but there isn't anyway for it to know which one of the sector was a good read and which isn't, or is there? From blevi.linux at gmail.com Fri Apr 13 09:22:34 2012 From: blevi.linux at gmail.com (Birta Levente) Date: Fri, 13 Apr 2012 09:22:34 +0300 Subject: [Dovecot] [OT] Outlook identities In-Reply-To: <4F876642.1040701@orlitzky.com> References: <4F876642.1040701@orlitzky.com> Message-ID: <4F87C62A.8060105@gmail.com> On 13/04/2012 02:33, Michael Orlitzky wrote: > Nothing to do with Dovecot, but I figured this is the best place to ask. > > Do any of the newer versions of Outlook have proper identities support > like Thunderbird, mutt, Roundcube, i.e. every other mail client on Earth? > > We have customers who set up ten different mailboxes for one person > because otherwise Outlook won't Do the Right Thing. Is there some way to > make it behave like the others? > > * When sending new mail, you can choose which address to use. > * When replying to mail, it sends from the address that the message > was sent to by default. > * All mail winds up in one inbox. > > Outlook (2003, 2007) does do this if you set up different mail accounts, > but we shouldn't have to do that. Control Panel / E-mail (Mail in win7) / Profiles ... I hope this help you .... Levi From robert at schetterer.org Fri Apr 13 09:35:53 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 13 Apr 2012 08:35:53 +0200 Subject: [Dovecot] [OT] Outlook identities In-Reply-To: <4F876642.1040701@orlitzky.com> References: <4F876642.1040701@orlitzky.com> Message-ID: <4F87C949.8010405@schetterer.org> Am 13.04.2012 01:33, schrieb Michael Orlitzky: > Nothing to do with Dovecot, but I figured this is the best place to ask. > > Do any of the newer versions of Outlook have proper identities support > like Thunderbird, mutt, Roundcube, i.e. every other mail client on Earth? > > We have customers who set up ten different mailboxes for one person > because otherwise Outlook won't Do the Right Thing. Is there some way to > make it behave like the others? > > * When sending new mail, you can choose which address to use. > * When replying to mail, it sends from the address that the message > was sent to by default. > * All mail winds up in one inbox. > > Outlook (2003, 2007) does do this if you set up different mail accounts, > but we shouldn't have to do that. you can do it with exchange, no wonder outlook isnt a internet mail client in first line its the client of exchange, so people should use real internet mail clients ( TB etc ), if m$ would more be compatible, the need of exchange may more less , but this should not be , as cashmakers should live forever *g http://www.msexchange.org/tutorials/Sending-As.html there are a few tricks to goal with profiles etc, but i saw nothing that was equal to thunderbird identities but there a many lists an payed support for outlook, you may asked there i also looked in my newest outlook version and didnt found something related i found this add on http://www.emailaddressmanager.com/outlook/email_accounts.html -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From am at am.cx Fri Apr 13 10:02:42 2012 From: am at am.cx (Andrea Mistrali) Date: Fri, 13 Apr 2012 09:02:42 +0200 Subject: [Dovecot] [OT] Outlook identities In-Reply-To: <4F876642.1040701@orlitzky.com> References: <4F876642.1040701@orlitzky.com> Message-ID: Il giorno 13/apr/2012, alle ore 01.33, Michael Orlitzky ha scritto: > Nothing to do with Dovecot, but I figured this is the best place to ask. > > Do any of the newer versions of Outlook have proper identities support > like Thunderbird, mutt, Roundcube, i.e. every other mail client on Earth? > > We have customers who set up ten different mailboxes for one person > because otherwise Outlook won't Do the Right Thing. Is there some way to > make it behave like the others? > > * When sending new mail, you can choose which address to use. > * When replying to mail, it sends from the address that the message > was sent to by default. > * All mail winds up in one inbox. > > Outlook (2003, 2007) does do this if you set up different mail accounts, > but we shouldn't have to do that. If it can comfort you even Apple Mail.app ha no support for full identities like Thunderbird and every other good mail client (I think of Eudora, back to 1999, that had full identities support) have, except for some little trick that does not do ?The Right Thing". We ended writing a little PyObjc application that adds support for full identities to Mail.app, but it is just a hack and dunno if it will work forever :/ A. From tss at iki.fi Fri Apr 13 14:51:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 13 Apr 2012 14:51:06 +0300 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F86C62C.8030305@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> <4F86C62C.8030305@wildgooses.com> Message-ID: <918556F5-6EE9-4651-A481-46720480FE46@iki.fi> On 12.4.2012, at 15.10, Ed W wrote: > On 12/04/2012 12:09, Timo Sirainen wrote: >> On 12.4.2012, at 13.58, Ed W wrote: >> >>> The claim by ZFS/BTRFS authors and others is that data silently "bit rots" on it's own. The claim is therefore that you can have a raid1 pair where neither drive reports a hardware failure, but each gives you different data? >> That's one reason why I planned on adding a checksum to each message in dbox. But I forgot to actually do that. I guess I could add it for new messages in some upcoming version. Then Dovecot could optionally verify the checksum before returning the message to client, and if it detects corruption perhaps automatically read it from some alternative location (e.g. if dsync replication is enabled ask from another replica). And Dovecot index files really should have had some small (8/16/32bit) checksums of stuff as well.. >> > > I have to say - I haven't actually seen this happen... Do any of your big mailstore contacts observe this, eg rackspace, etc? I haven't heard. But then again people don't necessarily notice if it has. > Things I might like to do *if* there were some suitable "checksums" available: > - Use the checksum as some kind of guid either for the whole message, the message minus the headers, or individual mime sections Messages already have a GUID. And the rest of that is kind of done with the single instance storage stuff.. I was thinking of using SHA1 of the entire message with headers as the checksum, and save it into dbox metadata field. I also thought about checksumming the metadata fields as well, but that would need another checksum as the first one can have other uses as well besides verifying the message integrity. > - Use the checksums to assist with replication speed/efficiency (dsync or custom imap commands) It would be of some use with dbox index rebuilding. I don't think it would help with dsync. > - File RFCs for new imap features along the "lemonde" lines which allow clients to have faster recovery from corrupted offline states... Too much trouble, no one would implement it :) > - Storage backends where emails are redundantly stored and might not ALL be on a single server (find me the closest copy of email X) - derivations of this might be interesting for compliance archiving of messages? > - Fancy key-value storage backends might use checksums as part of the key value (either for the whole or parts of the message) GUID would work for these as well, without the possibility of a hash collision. From tss at iki.fi Fri Apr 13 14:55:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 13 Apr 2012 14:55:07 +0300 Subject: [Dovecot] POP3 Dovecot Auth CPU usage 75%+ In-Reply-To: References: Message-ID: <0B6D0FE2-2831-460E-8B2B-C255C49DEBAE@iki.fi> On 12.4.2012, at 23.46, Root Kev wrote: > I hope someone can help me, I have been testing out Dovecot to switch from > popa3d which I use at the moment. When I get several users connecting and > disconnection multiple times, the Dovecot process with command Auth uses > 50-90% of the CPU for the period which they are connecting. I am wondering > if there is something that I may have misconfigured, or if there is > something that I can change so that this spike doesn't occur. So is it the "auth" process or "auth worker" process? What if you add: > passdb { > driver = shadow > } > userdb { > driver = passwd args = blocking=yes > } does that move the CPU usage from "auth" to "auth worker" process? Is it using /etc/shadow and /etc/passwd files? Are they large? Do you have enabled other weird stuff in /etc/nsswitch.conf (and were there some other files related to them as well?) From lists at wildgooses.com Fri Apr 13 15:17:19 2012 From: lists at wildgooses.com (Ed W) Date: Fri, 13 Apr 2012 13:17:19 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <918556F5-6EE9-4651-A481-46720480FE46@iki.fi> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> <4F86C62C.8030305@wildgooses.com> <918556F5-6EE9-4651-A481-46720480FE46@iki.fi> Message-ID: <4F88194F.4030807@wildgooses.com> On 13/04/2012 12:51, Timo Sirainen wrote: >> - Use the checksums to assist with replication speed/efficiency (dsync or custom imap commands) > It would be of some use with dbox index rebuilding. I don't think it would help with dsync. .. >> - File RFCs for new imap features along the "lemonde" lines which allow clients to have faster recovery from corrupted offline states... > Too much trouble, no one would implement it :) I presume you have seen that cyrus is working on various distributed options? Standardising this through imap might work if they also buy into it? >> - Storage backends where emails are redundantly stored and might not ALL be on a single server (find me the closest copy of email X) - derivations of this might be interesting for compliance archiving of messages? >> - Fancy key-value storage backends might use checksums as part of the key value (either for the whole or parts of the message) > GUID would work for these as well, without the possibility of a hash collision. I was thinking that the win for key-value store as a backend is if you can reduce the storage requirements or do better placement of the data (mail text replicated widely, attachments stored on higher latency storage?). Hence whilst I don't see this being a win with current options, if it were done then it would almost certainly be "per mime part", eg storing all large attachments in one place and the rest of the message somewhere else, perhaps with different redundancy levels per type OK, this is all completely pie in the sky. Please don't build it! All I meant was that these are the kind of things that someone might one day desire to do and hence they would have competing requirements for what to checksum... Cheers Ed W From tss at iki.fi Fri Apr 13 15:21:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 13 Apr 2012 15:21:49 +0300 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F88194F.4030807@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> <4F86C62C.8030305@wildgooses.com> <918556F5-6EE9-4651-A481-46720480FE46@iki.fi> <4F88194F.4030807@wildgooses.com> Message-ID: <6A4E7A1D-3B55-48DB-A4B0-EF5C01FA829A@iki.fi> On 13.4.2012, at 15.17, Ed W wrote: > On 13/04/2012 12:51, Timo Sirainen wrote: >>> - Use the checksums to assist with replication speed/efficiency (dsync or custom imap commands) >> It would be of some use with dbox index rebuilding. I don't think it would help with dsync. > .. >>> - File RFCs for new imap features along the "lemonde" lines which allow clients to have faster recovery from corrupted offline states... >> Too much trouble, no one would implement it :) > > I presume you have seen that cyrus is working on various distributed options? Standardising this through imap might work if they also buy into it? Probably more trouble than worth. I doubt anyone would want to run a cross-Dovecot/Cyrus cluster. >>> - Storage backends where emails are redundantly stored and might not ALL be on a single server (find me the closest copy of email X) - derivations of this might be interesting for compliance archiving of messages? >>> - Fancy key-value storage backends might use checksums as part of the key value (either for the whole or parts of the message) >> GUID would work for these as well, without the possibility of a hash collision. > > I was thinking that the win for key-value store as a backend is if you can reduce the storage requirements or do better placement of the data (mail text replicated widely, attachments stored on higher latency storage?). Hence whilst I don't see this being a win with current options, if it were done then it would almost certainly be "per mime part", eg storing all large attachments in one place and the rest of the message somewhere else, perhaps with different redundancy levels per type > > OK, this is all completely pie in the sky. Please don't build it! All I meant was that these are the kind of things that someone might one day desire to do and hence they would have competing requirements for what to checksum... That can almost be done already .. the attachments are saved and accessed via a lib-fs API. It wouldn't be difficult to write a backend for some key-value databases. So with about one day's coding you could already have Dovecot save all message attachments to a key-value db, and you can configure redundancy in the db's configs. From stan at hardwarefreak.com Fri Apr 13 15:33:19 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 13 Apr 2012 07:33:19 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> Message-ID: <4F881D0F.2090802@hardwarefreak.com> On 4/13/2012 1:12 AM, Emmanuel Noobadmin wrote: > On 4/12/12, Stan Hoeppner wrote: >> On 4/11/2012 9:23 PM, Emmanuel Noobadmin wrote: >> I suppose the controller could throw an error if >>> the two drives returned data that didn't agree with each other but it >>> wouldn't know which is the accurate copy but that wouldn't protect the >>> integrity of the data, at least not directly without additional human >>> intervention I would think. >> >> When a drive starts throwing uncorrectable read errors, the controller >> faults the drive and tells you to replace it. Good hardware RAID >> controllers are notorious for their penchant to kick drives that would >> continue to work just fine in mdraid or as a single drive for many more >> years. > > What I meant wasn't the drive throwing uncorrectable read errors but > the drives are returning different data that each think is correct or > both may have sent the correct data but one of the set got corrupted > on the fly. After reading the articles posted, maybe the correct term > would be the controller receiving silently corrupted data, say due to > bad cable on one. This simply can't happen. What articles are you referring to? If the author is stating what you say above, he simply doesn't know what he's talking about. > If the controller simply returns the fastest result, it could be the > bad sector and that doesn't protect the integrity of the data right? I already answered this in a previous post. > if the controller gets 1st half from one drive and 2nd half from the > other drive to speed up performance, we could still get the corrupted > half and the controller itself still can't tell if the sector it got > was corrupted isn't it? No, this is not correct. > If the controller compares the two sectors from the drives, it may be > able to tell us something is wrong but there isn't anyway for it to > know which one of the sector was a good read and which isn't, or is > there? Yes it can, and it does. Emmanuel, Ed, we're at a point where I simply don't have the time nor inclination to continue answering these basic questions about the base level functions of storage hardware. You both have serious misconceptions about how many things work. To answer the questions you're asking will require me to teach you the basics of hardware signaling protocols, SCSI, SATA, Fiber Channel, and Ethernet transmission error detection protocols, disk drive firmware error recovery routines, etc, etc, etc. I don't mind, and actually enjoy, passing knowledge. But the amount that seems to be required here to bring you up to speed is about 2^15 times above and beyond the scope of mailing list conversation. In closing, I'll simply say this: If hardware, whether a mobo-down SATA chip, or a $100K SGI SAN RAID controller, allowed silent data corruption or transmission to occur, there would be no storage industry, and we'll all still be using pen and paper. The questions you're asking were solved by hardware and software engineers decades ago. You're fretting and asking about things that were solved decades ago. -- Stan From jtl+dovecot at uvm.edu Fri Apr 13 16:12:02 2012 From: jtl+dovecot at uvm.edu (Jim Lawson) Date: Fri, 13 Apr 2012 09:12:02 -0400 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F881D0F.2090802@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> Message-ID: <4F882622.8040701@uvm.edu> On 04/13/2012 08:33 AM, Stan Hoeppner wrote: >> What I meant wasn't the drive throwing uncorrectable read errors but >> the drives are returning different data that each think is correct or >> both may have sent the correct data but one of the set got corrupted >> on the fly. After reading the articles posted, maybe the correct term >> would be the controller receiving silently corrupted data, say due to >> bad cable on one. > This simply can't happen. What articles are you referring to? If the > author is stating what you say above, he simply doesn't know what he's > talking about. ?! Stan, are you really saying that silent data corruption "simply can't happen"? People who have been studying this have been talking about it for years now. It can happen in the same way that Emmanuel describes. USENIX FAST08: http://static.usenix.org/event/fast08/tech/bairavasundaram.html CERN: http://storagemojo.com/2007/09/19/cerns-data-corruption-research/ http://fuji.web.cern.ch/fuji/talk/2007/kelemen-2007-C5-Silent_Corruptions.pdf LANL: http://institute.lanl.gov/resilience/conferences/2009/HPCResilience09_Michalak.pdf There are others if you search for it. This problem has been well-known in large (petabyte+) data storage systems for some time. Jim From lists at wildgooses.com Fri Apr 13 17:04:17 2012 From: lists at wildgooses.com (Ed W) Date: Fri, 13 Apr 2012 15:04:17 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <6A4E7A1D-3B55-48DB-A4B0-EF5C01FA829A@iki.fi> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> <4F86C62C.8030305@wildgooses.com> <918556F5-6EE9-4651-A481-46720480FE46@iki.fi> <4F88194F.4030807@wildgooses.com> <6A4E7A1D-3B55-48DB-A4B0-EF5C01FA829A@iki.fi> Message-ID: <4F883261.2040309@wildgooses.com> On 13/04/2012 13:21, Timo Sirainen wrote: > On 13.4.2012, at 15.17, Ed W wrote: > >> On 13/04/2012 12:51, Timo Sirainen wrote: >>>> - Use the checksums to assist with replication speed/efficiency (dsync or custom imap commands) >>> It would be of some use with dbox index rebuilding. I don't think it would help with dsync. >> .. >>>> - File RFCs for new imap features along the "lemonde" lines which allow clients to have faster recovery from corrupted offline states... >>> Too much trouble, no one would implement it :) >> I presume you have seen that cyrus is working on various distributed options? Standardising this through imap might work if they also buy into it? > Probably more trouble than worth. I doubt anyone would want to run a cross-Dovecot/Cyrus cluster. No definitely not. Sorry I just meant that you are both working on similar things. Standardising the basics that each use might be useful in the future > That can almost be done already .. the attachments are saved and accessed via a lib-fs API. It wouldn't be difficult to write a backend for some key-value databases. So with about one day's coding you could already have Dovecot save all message attachments to a key-value db, and you can configure redundancy in the db's configs. Hmm, super. Ed W From stan at hardwarefreak.com Fri Apr 13 17:20:29 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 13 Apr 2012 09:20:29 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F882622.8040701@uvm.edu> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F882622.8040701@uvm.edu> Message-ID: <4F88362D.9080400@hardwarefreak.com> On 4/13/2012 8:12 AM, Jim Lawson wrote: > On 04/13/2012 08:33 AM, Stan Hoeppner wrote: >>> What I meant wasn't the drive throwing uncorrectable read errors but >>> the drives are returning different data that each think is correct or >>> both may have sent the correct data but one of the set got corrupted >>> on the fly. After reading the articles posted, maybe the correct term >>> would be the controller receiving silently corrupted data, say due to >>> bad cable on one. >> This simply can't happen. What articles are you referring to? If the >> author is stating what you say above, he simply doesn't know what he's >> talking about. > > > ?! Stan, are you really saying that silent data corruption "simply > can't happen"? Yes, I did. Did you read the context in which I made that statement? > People who have been studying this have been talking > about it for years now. Yes, they have. Did you miss the paragraph where I stated exactly that? Did you also miss the part about the probably of such being dictated by total storage system size and access rate? > It can happen in the same way that Emmanuel > describes. No, it can't. Not in the way Emmanuel described. I already stated the reason, and all of this research backs my statement. You won't see this with a 2 drive mirror, or a 20 drive RAID10. Not until each drive has a capacity in the 15TB+ range, if not more, and again, depending on the total system size. This doesn't address the "RAID5", better known as "parity RAID" write hole, which is a separate issue. Which is also one of the reasons I don't use it. In lieu of an actual controller firmware bug, or mdraid or lvm bug, you'll never see this on small scale systems. > USENIX FAST08: > > http://static.usenix.org/event/fast08/tech/bairavasundaram.html > > CERN: > > http://storagemojo.com/2007/09/19/cerns-data-corruption-research/ > > http://fuji.web.cern.ch/fuji/talk/2007/kelemen-2007-C5-Silent_Corruptions.pdf > > LANL: > > http://institute.lanl.gov/resilience/conferences/2009/HPCResilience09_Michalak.pdf > > There are others if you search for it. This problem has been well-known > in large (petabyte+) data storage systems for some time. And again, this is the crux of it. One doesn't see this problem until one hits extreme scale, which I spent at least a paragraph or two explaining, referencing the same research. Please re-read my post at least twice, critically. Then tell me if I've stated anything substantively different than what any of these researches have. The statements "shouldn't" "wouldn't" and "can't" are based on probabilities. "Can't" or "won't" does not need equal probability 0. The probability of this type of silent data corruption occurring on a 2 disk or 20 disk array of today's drives is not zero over 10 years, but it is so low the effective statement is "can't" or "won't" see this corruption. As I said, when we reach 15-30TB+ disk drives, this may change for small count arrays. -- Stan From root.kev at gmail.com Fri Apr 13 17:32:34 2012 From: root.kev at gmail.com (Root Kev) Date: Fri, 13 Apr 2012 10:32:34 -0400 Subject: [Dovecot] POP3 Dovecot Auth CPU usage 75%+ In-Reply-To: <0B6D0FE2-2831-460E-8B2B-C255C49DEBAE@iki.fi> References: <0B6D0FE2-2831-460E-8B2B-C255C49DEBAE@iki.fi> Message-ID: I tried making the changes that you suggested but it didn't seem to make a noticeable difference. It should be using the shadow file directly. The shadow file has the default Ubuntu system accounts and 16 user accounts, so overall fairly small. The nsswitch.conf file is set as default: # /etc/nsswitch.conf # # Example configuration of GNU Name Service Switch functionality. # If you have the `glibc-doc-reference' and `info' packages installed, try: # `info libc "Name Service Switch"' for information about this file. passwd: compat group: compat shadow: compat hosts: files dns networks: files protocols: db files services: db files ethers: db files rpc: db files netgroup: nis An example of users connecting and the Auth process using alot of CPU (from top): Cpu(s): 87.4%us, 8.0%sy, 0.0%ni, 2.3%id, 0.0%wa, 0.7%hi, 1.7%si, 0.0%st Mem: 1026096k total, 533924k used, 492172k free, 60340k buffers Swap: 1757176k total, 0k used, 1757176k free, 414212k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 643 dovecot 20 0 3096 1616 1208 S 50.7 0.2 0:01.76 auth 644 root 20 0 3096 1524 1140 S 1.3 0.1 0:00.08 auth 642 dovenull 20 0 4276 1612 1256 S 1.0 0.2 0:00.03 pop3-login 623 root 20 0 2704 1020 772 S 0.7 0.1 0:00.02 dovecot 627 root 20 0 4344 2808 1056 S 0.7 0.3 0:00.03 config 631 syslog 20 0 33916 1924 1036 S 0.3 0.2 0:01.61 rsyslogd 696 serverma 20 0 5464 2564 2040 R 0.3 0.2 0:00.01 pop3 1 root 20 0 2652 1604 1216 S 0.0 0.2 0:01.59 init 2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd Thanks for any other ideas.... Kevin On Fri, Apr 13, 2012 at 7:55 AM, Timo Sirainen wrote: > On 12.4.2012, at 23.46, Root Kev wrote: > > So is it the "auth" process or "auth worker" process? What if you add: > > > passdb { > > driver = shadow > > } > > userdb { > > driver = passwd > args = blocking=yes > > } > > does that move the CPU usage from "auth" to "auth worker" process? Is it > using /etc/shadow and /etc/passwd files? Are they large? Do you have > enabled other weird stuff in /etc/nsswitch.conf (and were there some other > files related to them as well?) > > From tss at iki.fi Fri Apr 13 17:37:43 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 13 Apr 2012 17:37:43 +0300 Subject: [Dovecot] POP3 Dovecot Auth CPU usage 75%+ In-Reply-To: References: <0B6D0FE2-2831-460E-8B2B-C255C49DEBAE@iki.fi> Message-ID: <337BF2E0-9F19-4AB4-A81C-2DCE76484662@iki.fi> On 13.4.2012, at 17.32, Root Kev wrote: > 643 dovecot 20 0 3096 1616 1208 S 50.7 0.2 0:01.76 > auth Well, I can't think of any good reason for this. Try strace -p 643 (or whatever pid of the process eating CPU) for a few seconds and send its output? Also maybe get a couple of gdb backtraces to see where it's doing stuff: gdb -p 643 bt quit and repeat it a few times From f.bonnet at esiee.fr Fri Apr 13 18:01:59 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Fri, 13 Apr 2012 17:01:59 +0200 Subject: [Dovecot] French characters from 2.0.x to 2.1.3 Message-ID: <4F883FE7.9080401@esiee.fr> Hello Maybe it's a coincidence but it "seems" since I upgrade from 2.0.18 to 2.1.3 that some French characters ( a with accents by example ) are not well displayed anymore using thunderbird and are replaced by a "?" character Thanks for any info From lists at wildgooses.com Fri Apr 13 18:09:31 2012 From: lists at wildgooses.com (Ed W) Date: Fri, 13 Apr 2012 16:09:31 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F87B9D0.5010300@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <4F87B9D0.5010300@hardwarefreak.com> Message-ID: <4F8841AB.6010108@wildgooses.com> On 13/04/2012 06:29, Stan Hoeppner wrote: > On 4/12/2012 5:58 AM, Ed W wrote: > >> The claim by ZFS/BTRFS authors and others is that data silently "bit >> rots" on it's own. The claim is therefore that you can have a raid1 pair >> where neither drive reports a hardware failure, but each gives you >> different data? > You need to read those articles again very carefully. If you don't > understand what they mean by "1 in 10^15 bits non-recoverable read error > rate" and combined probability, let me know. OK, I'll bite. I only have an honours degree in mathematics from a well known university, so grateful if you could dumb it down appropriately? Lets start with what "those articles" are you referring to? I don't see any articles if I go literally up the chain from this email, but you might be talking about any one of the lots of other emails in this thread or even some other email thread? Wikipedia has it's faults, but it dumbs the "silent corruption" claim down to: http://en.wikipedia.org/wiki/ZFS "an undetected error for every 67TB" And a CERN study apparently claims "far higher than one in every 10^16 bits" Now, I'm NOT professing any experience of axe to grind here. I'm simply asking by what feature do you believe either software or hardware RAID1 is capable of detecting which pair is correct when both pairs of a raid one disk return different results and there is no hardware failure to clue us that one pair suffered a read error? Please don't respond with a maths pissing competition, it's an innocent question about what levels of data checking are done on each piece of the hardware chain? My (probably flawed) understanding is that popular RAID 1 implementations don't add any additional sector checksums over and above what the drives/filesystem/etc add already offer - is this the case? > And this has zero bearing on RAID1. And RAID1 reads don't work the way > you describe above. I explained this in some detail recently. Where? > Been working that way for more than 2 decades Ed. :) Note that "RAID1" > has that "1" for a reason. It was the first RAID level. What should I make of RAID0 then? Incidentally do you disagree with the history of RAID evolution on Wikipedia? http://en.wikipedia.org/wiki/RAID Regards Ed W From lists at wildgooses.com Fri Apr 13 18:31:35 2012 From: lists at wildgooses.com (Ed W) Date: Fri, 13 Apr 2012 16:31:35 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F881D0F.2090802@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> Message-ID: <4F8846D7.7020900@wildgooses.com> On 13/04/2012 13:33, Stan Hoeppner wrote: >> What I meant wasn't the drive throwing uncorrectable read errors but >> the drives are returning different data that each think is correct or >> both may have sent the correct data but one of the set got corrupted >> on the fly. After reading the articles posted, maybe the correct term >> would be the controller receiving silently corrupted data, say due to >> bad cable on one. > This simply can't happen. What articles are you referring to? If the > author is stating what you say above, he simply doesn't know what he's > talking about. It quite clearly can??! Just grab your drive, lever the connector off a little bit until it's a bit flaky and off you go? *THIS* type of problem I have heard of and you can find easy examples with a quick google search of any hobbyist storage board. Very common other examples are such problems due to failing PSUs and other interference driven examples causing explicit disk errors (and once the error rate goes up, some will make it past the checksum) Note this is NOT what I was originally asking about. My interest is more about when the hardware is working reliably and as you agree, the error levels are vastly lower. However, it would be incredibly foolish to claim that it's not trivial to construct a scenario where bad hardware causes plenty of silent corruption? >> If the controller simply returns the fastest result, it could be the >> bad sector and that doesn't protect the integrity of the data right? > I already answered this in a previous post. Not obviously?! I will also add my understanding that linux software RAID1,5&6 *DO NOT* read all disks and hence will not be aware when disks have different data. In fact with software raid you need to run a regular "scrub" job to check this consistency. I also believe that most commodity hardware raid implementations work exactly the same way and a background scrub is needed to detect inconsistent arrays. However, feel free to correct that understanding? >> if the controller gets 1st half from one drive and 2nd half from the >> other drive to speed up performance, we could still get the corrupted >> half and the controller itself still can't tell if the sector it got >> was corrupted isn't it? > No, this is not correct. I definitely think you are wrong and Emmanuel is right? If the controller gets a good read from the disk then it will trust that read and will NOT check the result with the other disk (or parity in the case of RAID5/6). If that read was incorrect for some reason then the data will be passed as good. >> If the controller compares the two sectors from the drives, it may be >> able to tell us something is wrong but there isn't anyway for it to >> know which one of the sector was a good read and which isn't, or is >> there? > Yes it can, and it does. No it definitely does not!! At least not with linux software raid and I don't believe on commodity hardware controllers either! (You would be able to tell because the disk IO would be doubled) Linux software raid 1 isn't that smart, but reads only one disk and trusts the answer if the read did not trigger an error. It does not check the other disk except during an explicit disk scrub. > Emmanuel, Ed, we're at a point where I simply don't have the time nor > inclination to continue answering these basic questions about the base > level functions of storage hardware. You mean those "answers" like: "I answered that in another thread" or "you need to read 'those' articles again" Referring to some unknown and hard to find previous emails is not the same as answering? Also you are wondering off at extreme tangents. The question is simple: - Disk 1 Read good, checksum = A - Disk 2 Read good, checksum = B Disks are a raid 1 pair. How do we know which disk is correct. Please specify raid 1 implementation and mechanism used with any answer > To answer the questions > you're asking will require me to teach you the basics of hardware > signaling protocols, SCSI, SATA, Fiber Channel, and Ethernet > transmission error detection protocols, disk drive firmware error > recovery routines, etc, etc, etc. I really think not... A simple statement of: - Each sector on disk has a certain sized checksum - Controller checks checksum on read - Sent back over SATA connection, with a certain sized checksum - After that you are on your own vs corruption ...Should cover it I think? > In closing, I'll simply say this: If hardware, whether a mobo-down SATA > chip, or a $100K SGI SAN RAID controller, allowed silent data corruption > or transmission to occur, there would be no storage industry, and we'll > all still be using pen and paper. The questions you're asking were > solved by hardware and software engineers decades ago. You're fretting > and asking about things that were solved decades ago. So why are so many people getting excited about it now? Note, there have been plenty of shoddy disk controller implementations before today - ie there exists hardware on sale with *known* defects. Despite that the industry continues without collapse. Now you claim that if corruption is silent and people only tend to notice it much later and under certain edge conditions that this can't be possible because it should cause the industry to collapse..??? ...Not buying your logic... Ed W From michael at orlitzky.com Fri Apr 13 19:08:31 2012 From: michael at orlitzky.com (Michael Orlitzky) Date: Fri, 13 Apr 2012 12:08:31 -0400 Subject: [Dovecot] [OT] Outlook identities In-Reply-To: <4F87C62A.8060105@gmail.com> References: <4F876642.1040701@orlitzky.com> <4F87C62A.8060105@gmail.com> Message-ID: <4F884F7F.5070002@orlitzky.com> On 04/13/12 02:22, Birta Levente wrote: >> >> Outlook (2003, 2007) does do this if you set up different mail accounts, >> but we shouldn't have to do that. > > Control Panel / E-mail (Mail in win7) / Profiles ... > > I hope this help you .... This is more work than setting up multiple accounts =) From michael at orlitzky.com Fri Apr 13 19:13:30 2012 From: michael at orlitzky.com (Michael Orlitzky) Date: Fri, 13 Apr 2012 12:13:30 -0400 Subject: [Dovecot] [OT] Outlook identities In-Reply-To: <4F87C949.8010405@schetterer.org> References: <4F876642.1040701@orlitzky.com> <4F87C949.8010405@schetterer.org> Message-ID: <4F8850AA.80903@orlitzky.com> On 04/13/12 02:35, Robert Schetterer wrote: > > you can do it with exchange, > no wonder outlook isnt a internet mail client in first line > its the client of exchange, so people should use real internet mail > clients ( TB etc ), if m$ would more be compatible, the need of exchange > may more less , but this should not be , as cashmakers should live > forever *g > > http://www.msexchange.org/tutorials/Sending-As.html > > there are a few tricks to goal with profiles etc, but i saw nothing > that was equal to thunderbird identities > Exchange... the cure is worse than the disease! This isn't looking good -- I guess I'll continue to do what I have been: telling people to switch off of Outlook if they want their mail client to not suck. From robert at schetterer.org Fri Apr 13 20:51:59 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 13 Apr 2012 19:51:59 +0200 Subject: [Dovecot] sieve pipe gpg archive mails Message-ID: <4F8867BF.7030207@schetterer.org> hi @ll would it be possible to use http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Pipe to gpg sign archived mail, incoming via lmtp the intention is to store all archived mail gpg signed to notice some changes the mails at possible restore case has anyone done it bevor , or something equal ? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Fri Apr 13 21:33:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 13 Apr 2012 21:33:05 +0300 Subject: [Dovecot] sieve pipe gpg archive mails In-Reply-To: <4F8867BF.7030207@schetterer.org> References: <4F8867BF.7030207@schetterer.org> Message-ID: On 13.4.2012, at 20.51, Robert Schetterer wrote: > would it be possible to use > > http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Pipe > > to gpg sign archived mail, incoming via lmtp > > the intention is to store all archived mail gpg signed > to notice some changes the mails at possible restore case > > has anyone done it bevor , or something equal ? What about mails saved via IMAP APPEND? From mcbdovecot at robuust.nl Fri Apr 13 22:10:04 2012 From: mcbdovecot at robuust.nl (Maarten Bezemer) Date: Fri, 13 Apr 2012 21:10:04 +0200 (CEST) Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F8846D7.7020900@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F8846D7.7020900@wildgooses.com> Message-ID: On Fri, 13 Apr 2012, Ed W wrote: > On 13/04/2012 13:33, Stan Hoeppner wrote: >>> What I meant wasn't the drive throwing uncorrectable read errors but >>> the drives are returning different data that each think is correct or >>> both may have sent the correct data but one of the set got corrupted >>> on the fly. After reading the articles posted, maybe the correct term >>> would be the controller receiving silently corrupted data, say due to >>> bad cable on one. >> This simply can't happen. What articles are you referring to? If the >> author is stating what you say above, he simply doesn't know what he's >> talking about. > It quite clearly can??! I totally agree with Ed here. Drives sure can and sometimes really do return different data, without reporting errors. Also, data can get corrupted on any of the busses or chips it passes through. The math about 10^15 or 10^16 and all that stuff is not only about array sizes. It's also about data transfer. I've seen silent corruption on a few systems myself. (Luckily, only 3 times in a couple years.) Those systems were only in the 2TB-5TB size category, which is substantially lower than the 67TB claimed elsewhere. Yet, statistically, it's well within normal probability levels. Linux mdraid only reads one mirror as long as the drives don't return an error. Easy to check, the read speeds are way beyond a single drive's read speed. When the kernel would have to read all (possibly more than two) mirrors, and compare them, and make a decision based on this comparison, things would be horribly slow. Hardware raid typically uses this exact same approach. This goes for Areca, 3ware, LSI, which cover most of the regular (i.e. non-SAN) professional hardware raid setups. If you don't believe it, just don't take my word for it but test it for yourself. Cleanly power down a raid1 array, take the individual drives, put them into a simple desktop machine, and write different data to both, using some raw disk writing tool like dd. Then put the drives back into the raid1 array, power it up, and re-read the information. You'll see data from both drives will be intermixed as parts of the reads come from one disk, and parts come from the other. Only when you order the raid array to do a verification pass, it'll start screaming and yelling. At least, I hope it will... But as explained elsewhere, silent corruption can occur at numerous places. If you don't have an explicit checksumming/checking mechanism, there are indeed cases that will haunt you if you don't do regular scrubbing or at least do regular verification runs. Heck, that's why Linux mdadm comes with cron jobs to do just that, and hardware raid controllers have similar scheduling capabilities. Of course, scrubbing/verification is not going to magically protect you from all problems. But you would at least get notifications if it detects problems. >>> If the controller compares the two sectors from the drives, it may be >>> able to tell us something is wrong but there isn't anyway for it to >>> know which one of the sector was a good read and which isn't, or is >>> there? >> Yes it can, and it does. > > No it definitely does not!! At least not with linux software raid and I don't > believe on commodity hardware controllers either! (You would be able to tell > because the disk IO would be doubled) Obviously there is no way to tell which versions of a story are correct if you are not biased to believe one of the storytellers and distrust the other. You would have to add a checksum layer for that. (And hope the checksum isn't the part that got corrupted!) >> To answer the questions >> you're asking will require me to teach you the basics of hardware >> signaling protocols, SCSI, SATA, Fiber Channel, and Ethernet >> transmission error detection protocols, disk drive firmware error >> recovery routines, etc, etc, etc. I'm quite familiar with the basics of these protocols. I'm also quite familiar with the flaws in several implementations of "seemingly straightforward protocols". More often than not, there's a pressing need to get new devices onto the market before the competition has something similar and you loose your advantage. More often than not, this results in suboptimal implementations of all those fine protocols and algorithms. And let's face it: flaws in error recovery routines often don't surface until someone actually needs those routines. As long as drives (or any other device) are functioning as expected, everything is all right. But as soon as something starts to get flaky, error recovery has to kick in but may just as well fail to do the right thing. Just consider the real-world analogy of politicians. They do or say something stupid every once in a while, and error recovery (a.k.a. damage control) has to kick in. But even though those well trained professionals, having decades of experience in the political arena, sometimes simply fail to do the right thing. They may have overlooked some pesky details, or they may take actions that don't have the expected outcome because... indeed, things work differently in damage control mode, and the only law you can trust is physics: you always go down when you can't stay on your feet. With hard drives, raid controllers, mainboards, data buses, it's exactly the same. If _something_ isn't working as it should, how should we know which part of it we _can_ trust? >> In closing, I'll simply say this: If hardware, whether a mobo-down SATA >> chip, or a $100K SGI SAN RAID controller, allowed silent data corruption >> or transmission to occur, there would be no storage industry, and we'll >> all still be using pen and paper. The questions you're asking were >> solved by hardware and software engineers decades ago. You're fretting >> and asking about things that were solved decades ago. Isn't it just "worked around" by adding more layers of checksuming and adding more redundancy into the mix? Don't believe this "storage industry" because they tell you it's OK. It simply is not OK. You might want to talk to people in the data and computing cluster business about their opinion on "storage industry professionals"... Timo's suggestion to add checksums to mailboxes/metadata could help to (at least) report these types of failures. Re-reading from different storage when available could also recover the data that got corrupted, but I'm not sure what would be the best way to handle these situations. If you know there is a corruption problem on one of your storage locations, you might want to switch that to read-only asap. Automagically trying to recover might not be the best thing to do. Given all kinds of different use cases, I think that should at least be configurable :-P -- Maarten From lists at necoro.eu Fri Apr 13 23:41:02 2012 From: lists at necoro.eu (=?UTF-8?B?UmVuw6kgTmV1bWFubg==?=) Date: Fri, 13 Apr 2012 22:41:02 +0200 Subject: [Dovecot] imapc / namespace question Message-ID: <4F888F5E.5030709@necoro.eu> Hi, I've got two questions regarding imapc and namespaces: 1) Is there a way of stating, that I only want part of the hierarchy on the end of an imapc connection to be available here? My "problem": On the other end the layout is: INBOX.Shared.{some,nice,folders} If I mirror this into my dovecot hierarchy with prefix "Gemeinsam" I get the following folder structure: Gemeinsam.INBOX.Shared.{some,nice,folders} But I want to ignore the two top-most parts of the hierarchy, i.e. I want it to look: Gemeinsam.{some,nice,folders} Is this possible? I tried with 'location=imapc:INBOX.Shared.' but this did not change anything. 2) "Private" namespaces only make sense, if location contains something user-specific like "%u" or "~". Is this correct? If yes: Is there some other way to make something like an imapc connection only appear for one user, if there is nothing in the location or imapc_user to denote the user it belongs to and hence I cannot use "private"? The only way I see currently is to use "shared" and set up ACLs to forbid all users except one to access the namespace (which gets slightly ugly as I have to use global ACLs, as -- I think -- there is no way to store ACLs for an imapc storage). Thanks, Ren? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From joe at tao.org.uk Sat Apr 14 00:59:36 2012 From: joe at tao.org.uk (Dr Josef Karthauser) Date: Fri, 13 Apr 2012 22:59:36 +0100 Subject: [Dovecot] How do I test if the anti spam plugin is working? In-Reply-To: <3779AD95-CA9E-484B-8B63-039F50B0426E@tao.org.uk> References: <3779AD95-CA9E-484B-8B63-039F50B0426E@tao.org.uk> Message-ID: <4692B327-5985-4227-82A4-C30B4A291C4E@tao.org.uk> On 30 Mar 2012, at 12:29, Dr Josef Karthauser wrote: > I've configured the dspam anti spam plugin, but it doesn't appear to be doing anything when I move mail between mailboxes. Can anyone help me determine what's going on? > > The plugin appears to be loading; at least if I don't define all the required configuration options I get a complaint in the log file. But beyond that I don't see any activity. > > [cut] > I don't even appear to be seeing any log entries from the plugin. > > I've moving an email from my main mailbox into a mailbox called 'SPAM', which is how I thought that it was supposed to be triggered. It turns out that I had the signature header configured incorrectly. :). Joe From stan at hardwarefreak.com Sat Apr 14 06:31:04 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 13 Apr 2012 22:31:04 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F8846D7.7020900@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F8846D7.7020900@wildgooses.com> Message-ID: <4F88EF78.7040202@hardwarefreak.com> On 4/13/2012 10:31 AM, Ed W wrote: > On 13/04/2012 13:33, Stan Hoeppner wrote: >> In closing, I'll simply say this: If hardware, whether a mobo-down SATA >> chip, or a $100K SGI SAN RAID controller, allowed silent data corruption >> or transmission to occur, there would be no storage industry, and we'll >> all still be using pen and paper. The questions you're asking were >> solved by hardware and software engineers decades ago. You're fretting >> and asking about things that were solved decades ago. > > So why are so many people getting excited about it now? "So many"? I know of one person "getting excited" about it. Data densities and overall storage sizes and complexity at the top end of the spectrum are increasing at a faster rate than the consistency/validation mechanisms. That's the entire point of the various academic studies on the issue. Note that the one study required a sample set of 1.5 million disk drives. If the phenomenon were a regular occurrence as you would have everyone here believe, they could have used a much smaller sample set. Ed, this is an academic exercise. Academia leads industry. Almost always has. Academia blows the whistle and waves hands, prompting industry to take action. There is nothing normal users need to do to address this problem. The hardware and software communities will make the necessary adjustments to address this issue before it filters down to the general user community in a half decade or more--when normal users have a 10-20 drive array of 500TB to 1PB or more. Having the prestigious degree that you do, you should already understand the relationship between academic research and industry, and the considerable lead times involved. -- Stan From stan at hardwarefreak.com Sat Apr 14 06:48:07 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 13 Apr 2012 22:48:07 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F8846D7.7020900@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F8846D7.7020900@wildgooses.com> Message-ID: <4F88F377.1070801@hardwarefreak.com> On 4/13/2012 10:31 AM, Ed W wrote: > You mean those "answers" like: > "you need to read 'those' articles again" > > Referring to some unknown and hard to find previous emails is not the > same as answering? No, referring to this: On 4/12/2012 5:58 AM, Ed W wrote: > The claim by ZFS/BTRFS authors and others is that data silently "bit > rots" on it's own. Is it not a correct assumption that you read this in articles? If you read this in books, scrolls, or chiseled tablets, my apologies for assuming it was articles. -- Stan From ngu.antoine at gmail.com Sat Apr 14 12:50:32 2012 From: ngu.antoine at gmail.com (Antoine Nguyen) Date: Sat, 14 Apr 2012 11:50:32 +0200 Subject: [Dovecot] Unseen messages question Message-ID: Hi list, this question is related to the IMAP protocol itself, not really to Dovecot. I'm trying to understand what is the more efficient way to maintain the number of unseen messages of the currently selected mailbox. RFC3501 says a client must not issue a STATUS command to the selected mailbox and that information sent by a SELECT is enough. My current idea follows these steps : * Issue a STATUS before the mailbox is selected => I know how many unseen messages it contains * SELECT the mailbox => I got the eventual first unseen message in this mailbox but I don't understand how this info can be useful * Maintain the unseen counter (on client side) according to what the user do * Send a NOOP command every X minutes and look at the RECENT response to see if there are new messages I think it works pretty well when the mailbox is opened only once. Let's imagine this mailbox is opened twice, by different clients. If one client marks a message as \Seen, how can the second client know about this change? Thanks for your help, Antoine Nguyen http://modoboa.org/ From lists at wildgooses.com Sat Apr 14 13:00:40 2012 From: lists at wildgooses.com (Ed W) Date: Sat, 14 Apr 2012 11:00:40 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F88F377.1070801@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F8846D7.7020900@wildgooses.com> <4F88F377.1070801@hardwarefreak.com> Message-ID: <4F894AC8.9060406@wildgooses.com> On 14/04/2012 04:48, Stan Hoeppner wrote: > On 4/13/2012 10:31 AM, Ed W wrote: > >> You mean those "answers" like: >> "you need to read 'those' articles again" >> >> Referring to some unknown and hard to find previous emails is not the >> same as answering? > No, referring to this: > > On 4/12/2012 5:58 AM, Ed W wrote: > >> The claim by ZFS/BTRFS authors and others is that data silently "bit >> rots" on it's own. > Is it not a correct assumption that you read this in articles? If you > read this in books, scrolls, or chiseled tablets, my apologies for > assuming it was articles. > WHAT?!! The original context was that you wanted me to learn some very specific thing that you accused me of misunderstanding, and then it turns out that the thing I'm supposed to learn comes from re-reading every email, every blog post, every video, every slashdot post, every wiki, every ... that mentions ZFS's reason for including end to end checksumming?!! Please stop wasting our time and get specific You have taken my email which contained a specific question, been asked of you multiple times now and yet you insist on only answering irrelevant details with a pointed and personal dig on each answer. The rudeness is unnecessary, and your evasiveness of answers does not fill me with confidence that you actually know the answer... For the benefit of anyone reading this via email archives or whatever, I think the conclusion we have reached is that: modern systems are now a) a complex sum of pieces, any of which can cause an error to be injected, b) the level of error correction which was originally specified as being sufficient is now starting to be reached in real systems, possibly even consumer systems. There is no "solution", however, the first step is to enhance "detection". Various solutions have been proposed, all increase cost, computation or have some disadvantage - however, one of the more promising detection mechanisms is an end to end checksum, which will then have the effect of augmenting ALL the steps in the chain, not just one specific step. As of today, only a few filesystems offer this, roll on more adopting it Regards Ed W From janfrode at tanso.net Sat Apr 14 13:04:22 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sat, 14 Apr 2012 12:04:22 +0200 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F881D0F.2090802@hardwarefreak.com> References: <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> Message-ID: <20120414100422.GA5353@dibs.tanso.net> On Fri, Apr 13, 2012 at 07:33:19AM -0500, Stan Hoeppner wrote: > > > > What I meant wasn't the drive throwing uncorrectable read errors but > > the drives are returning different data that each think is correct or > > both may have sent the correct data but one of the set got corrupted > > on the fly. After reading the articles posted, maybe the correct term > > would be the controller receiving silently corrupted data, say due to > > bad cable on one. > > This simply can't happen. What articles are you referring to? If the > author is stating what you say above, he simply doesn't know what he's > talking about. It has happened to me, with RAID5 not RAID1. It was a firmware bug in the raid controller that caused the RAID array to go silently corrupted. The HW reported everything green -- but the filesystem was reporting lots of strange errors.. This LUN was part of a larger filesystem striped over multiple LUNs, so parts of the fs was OK, while other parts was corrupt. It was this bug: http://delivery04.dhe.ibm.com/sar/CMA/SDA/02igj/7/ibm_fw1_ds4kfc_07605200_anyos_anycpu.chg - Fix 432525 - CR139339 Data corruption found on drive after reconstruct from GHSP (Global Hot Spare) > In closing, I'll simply say this: If hardware, whether a mobo-down SATA > chip, or a $100K SGI SAN RAID controller, allowed silent data corruption > or transmission to occur, there would be no storage industry, and we'll > all still be using pen and paper. The questions you're asking were > solved by hardware and software engineers decades ago. You're fretting > and asking about things that were solved decades ago. Look at the plans are for your favorite fs: http://www.youtube.com/watch?v=FegjLbCnoBw They're planning on doing metadata checksumming to be sure they don't receive corrupted metadata from the backend storage, and say that data validation is a storage subsystem *or* application problem. Hardly a solved problem.. -jf From lists at wildgooses.com Sat Apr 14 13:22:37 2012 From: lists at wildgooses.com (Ed W) Date: Sat, 14 Apr 2012 11:22:37 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F88EF78.7040202@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F8846D7.7020900@wildgooses.com> <4F88EF78.7040202@hardwarefreak.com> Message-ID: <4F894FED.70901@wildgooses.com> On 14/04/2012 04:31, Stan Hoeppner wrote: > On 4/13/2012 10:31 AM, Ed W wrote: >> On 13/04/2012 13:33, Stan Hoeppner wrote: >>> In closing, I'll simply say this: If hardware, whether a mobo-down SATA >>> chip, or a $100K SGI SAN RAID controller, allowed silent data corruption >>> or transmission to occur, there would be no storage industry, and we'll >>> all still be using pen and paper. The questions you're asking were >>> solved by hardware and software engineers decades ago. You're fretting >>> and asking about things that were solved decades ago. >> So why are so many people getting excited about it now? > "So many"? I know of one person "getting excited" about it. You love being vague don't you? Go on, I'll bite again, do you mean yourself? :-) > Data densities and overall storage sizes and complexity at the top end > of the spectrum are increasing at a faster rate than the > consistency/validation mechanisms. That's the entire point of the > various academic studies on the issue. Again, you love being vague. By your dismissive "academic studies" phrase, do you mean studies done on a major industrial player, ie NetApp in this case? Or do you mean that it's rubbish because they asked someone with some background in statistics to do the work, rather than asking someone sitting nearby in the office to do it? I don't think the researcher broke into NetApp to do this research, so we have to conclude that the industrial partner was onboard. NetApp seem to do a bunch of engineering of their own (got enough patents..) that I think we can safely assume they very much do their own research on this and it's not just "academic"... I doubt they publish all their own internal research, be thankful you got to see some of the results this way... > Note that the one study required > a sample set of 1.5 million disk drives. If the phenomenon were a > regular occurrence as you would have everyone here believe, they could > have used a much smaller sample set. Sigh... You could criticise the study if it had a small number of drives as being under-representive and now you criticise a large study for having too many observations... You cannot have "too many" observations when measuring a small and unpredictable phenomena... Where does it say that they could NOT have reproduced this study with just 10 drives? If you have 1.5 million available, why not use all the results?? > Ed, this is an academic exercise. Academia leads industry. Almost > always has. Academia blows the whistle and waves hands, prompting > industry to take action. Sigh... We are back to the start of the email thread again... Gosh you seem to love arguing and muddying the water for zero reason but to have the last word? It's *trivial* to do a google search and hit *lots* of reports of corruptions in various parts of the system, from corrupting drivers, to hardware which writes incorrectly, to operating system flaws. I just found a bunch more in the Redhat database today while looking for something else. You yourself are very vocal on avoiding certain brands of HD controller which have been rumoured to cause corrupted data... (and thankyou for revealing that kind of thing - it's very helpful) Don't veer off at a tangent now: The *original* email this has spawned is about a VERY specific point. RAID1 appears to offer less protection against a class of error conditions than does RAID6. Nothing more, nothing less. Don't veer off and talk about the minutiae of testing studies at universities, this is a straightforward claim that you have been jumping around and avoiding answering with claims of needing to educate me on SCSI protocols and other fatuous responses. Nor deviate and discuss that RAID6 is inappropriate for many situations - we all get that... > There is nothing normal users need to do to address this problem. ...except sit tight and hope they don't loose anything important! :-) > Having the prestigious degree that you do, you should already understand > the relationship between academic research and industry, and the > considerable lead times involved. I'm guessing you haven't attended higher education then? You are confusing graduate and post-graduate systems... Byee Ed W From jerry at seibercom.net Sat Apr 14 13:51:13 2012 From: jerry at seibercom.net (Jerry) Date: Sat, 14 Apr 2012 06:51:13 -0400 Subject: [Dovecot] [OT] Outlook identities In-Reply-To: <4F8850AA.80903@orlitzky.com> References: <4F876642.1040701@orlitzky.com> <4F87C949.8010405@schetterer.org> <4F8850AA.80903@orlitzky.com> Message-ID: <20120414065113.79eb4065@scorpio> On Fri, 13 Apr 2012 12:13:30 -0400 Michael Orlitzky articulated: > Exchange... the cure is worse than the disease! This isn't looking > good -- I guess I'll continue to do what I have been: telling people > to switch off of Outlook if they want their mail client to not suck. First of all, there are no existing RFC's that require any MUA to meet the requirements that you desire. So please, stop your wining and crying. It is embarrassing. Second, there are avenues available that can make Outlook behave in a fashion that should be acceptable to you. If you choose not to pursue them, then that is you business. I have had to endure hours of tedious nonsense to get a simple sound card to work under a *.nix environment when I could have simply plugged it into a machine running Microsoft Windows and had it working immediately. Your "the cure is worse than the disease" is just self-serving bull-shit. Outlook + MS Exchange offers features that no other MUA presently comes close to being able to duplicate in an office environment. If these don't fit your needs, then please find an MUA that does. No one is holding a gun to your head. However, your desire to force others to abandon something that works fine for them to simple suit your narrow view of what an MUA should or should not do stinks of fascism. I use Outlook at work and claws-mail at home. Each one fits perfectly into the environment I have chosen to use it in. By the way, after examining your original post, I cannot find a single thing that the proper use of filtering rules and plugins cannot easily accomplish. Instead of your customers using a different MUA, they should consider changing to a new service provider. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From amk at spamfence.net Sat Apr 14 14:04:17 2012 From: amk at spamfence.net (Andreas M. Kirchwitz) Date: Sat, 14 Apr 2012 11:04:17 +0000 (UTC) Subject: [Dovecot] Dovecot 2.1 with custom OpenSSL fails to build References: <1331816286.10319.23.camel@innu.invalid> <4F7BBF3F.9060103@iki.fi> Message-ID: Timo Sirainen wrote: >> But two libraries are not quite okay. They don't find their SSL libs: >> >> libdovecot-lda.so >> libdovecot-storage.so > > Maybe this fixes it? > > http://hg.dovecot.org/dovecot-2.1/rev/8b91367bc3e1 Works perfectly! Great, now all components find their libraries by themselves. Thanks a lot for fixing this issue which seemed quite complicated. Very good, thank you ... Andreas From kjonca at o2.pl Sat Apr 14 14:27:59 2012 From: kjonca at o2.pl (Kamil =?iso-8859-2?Q?Jo=F1ca?=) Date: Sat, 14 Apr 2012 13:27:59 +0200 Subject: [Dovecot] Compressed mbox - patch Message-ID: <871unq7dts.fsf@alfa.kjonca> Some time ago I complained about very slow access to compressed mboxes. Unfortunately it looks like that it is very little interest in it, so I have to investigate some things by myself. Firstly: some rationale. Why do I prefer use mbox/maildir over mdbox. Short answer "bus factor" for support mdbox (not only dovecot) Longer answer: if something goes wrong withm maildir/mbox i can use other tools (mutt, or formail or even text editor) and with mdbox ... I am not ISP, I use dovecot as a "gateway" to my (rather huge) mail archive. Most of these mails are rather valuable for me, so I prefer use something "well-known-and-tested". (I can't do like most ISP's do: write in "Terms of Service" that mail can be lost or damaged and we give no warranty :) ) So then: Below my patch. It contains 2 changes: 1. when buffer is compressed, we try to save last marked offset. 2. Increase temporary buffer for decompression. without these changes 1.5 GB of bzip compressed mbox with ~20K messages can't be open in 1.5 day After applying 1. change it can be open in ~1.5 h With both changes it was a few minutes. Maybe it is a good idea to add config parameter to specify size of decompress buffer? Patch is against v2.0.18 -------------- next part -------------- A non-text attachment was scrubbed... Name: compress.patch Type: text/x-diff Size: 3898 bytes Desc: not available URL: -------------- next part -------------- -- Gdyby kto? mia? zb?dny Toshiba G450 - to ch?tnie przejm? ;) < asuffield> a workstation is anything you can stick on somebodies desk and con them into using -- in #debian-devel From kjonca at o2.pl Sat Apr 14 15:13:58 2012 From: kjonca at o2.pl (Kamil =?iso-8859-2?Q?Jo=F1ca?=) Date: Sat, 14 Apr 2012 14:13:58 +0200 Subject: [Dovecot] Sieve pipe extension - can it retur something? Message-ID: <87ty0m5x4p.fsf@alfa.kjonca> I have a question about sieve pipe: can it return something to further processing? For example in procmail I can do: --8<---------------cut here---------------start------------->8--- :0 VIRUS=`$CLAMSCAN --mbox --disable-summary --stdout -` --8<---------------cut here---------------end--------------->8--- and then test VIRUS variable. Maybe I missing something, when read http://hg.rename-it.nl/pigeonhole-0.2-sieve-pipe/raw-file/tip/doc/rfc/spec-bosch-sieve-pipe.txt KJ -- http://sporothrix.wordpress.com/2011/01/16/usa-sie-krztusza-kto-nastepny/ Gloffing is a state of mine. From branko at majic.rs Sat Apr 14 16:30:06 2012 From: branko at majic.rs (=?UTF-8?B?0JHRgNCw0L3QutC+INCc0LDRmNC40Zs=?=) Date: Sat, 14 Apr 2012 15:30:06 +0200 Subject: [Dovecot] Dovecot 2.1.4 and client certificates Message-ID: <20120414153006.19ce7e3c@majic.rs> Version: 2.1.4 OS: Gentoo stable/amd64 OpenSSL version: 1.0.0h I'm having a slight problem with the client certificates in Dovecot 2.1.4. I've set-up the client certificate verification/authentication, and it seems that Dovecot is choking on the trustchain with CRL's that I'm providing to it (attached to this mail). When I enable the client authentication using certificates, and pick the certificate from my client (I've also tried it out with gnutls-cli as well), I get the following errors in Dovecot's log: imap-login: Info: Invalid certificate: Different CRL scope: /CN=Example Root CA/O=Example Inc./C=RS As per the wiki2 configuration page, I've set up the truststore in the following order (everything PEM-encoded): Example Person CA Certificate Example Person CA CRL Example Root CA Certificate Example Root CA CRL Person CA is the one issuing the end-entity certificates, of course. I'm also attaching the certificate I've used for testing. On additional note, the imap-login process also got stuck writing out the error message to the log file, refusing to die when receiving the SIGTERM (had to send SIGKILL). A similar set-up used to work under Dovecot in Debian Squeeze (version 1.2.15). The same file copied over to Dovecot 2.1.4's configuration won't work. I've compiled Dovecot by hand, and I'm not running it in any kind of chroot (this is a developer set-up so I could add support for rfc822Name username extraction I mentioned a week or so ago without messing around as root). Best regards -- Branko Majic Jabber: branko at majic.rs Please use only Free formats when sending attachments to me. ?????? ????? ?????: branko at majic.rs ????? ??? ?? ??????? ?????? ????????? ? ????????? ?????????. -------------- next part -------------- A non-text attachment was scrubbed... Name: trustchain.pem Type: application/x-x509-ca-cert Size: 6640 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: branko_majic.crt Type: application/x-x509-ca-cert Size: 1700 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From cor at xs4all.nl Sat Apr 14 19:24:24 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sat, 14 Apr 2012 18:24:24 +0200 Subject: [Dovecot] LMTP auth problem Message-ID: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> hey all, im getting the following error: Apr 14 14:29:44 lmtpdirector1 dovecot: auth: Error: passdb(scorpio,127.0.0.1): Auth client doesn't have permissions to do a PASS lookup: /var/run/dovecot/auth-userdb mode=0666, but not owned by UID 112(dovecot) Apr 14 14:29:44 lmtpdirector1 dovecot: lmtp(18298): Error: user scorpio: Auth PASS lookup failed My config. Director servers running both imap and lmtp with a matching set of real servers accepting imap/lmtp. Imap is working fine, and has been working fine for a while. Im trying to add lmtp to the director, but i cant seem to get that working. We're passing passdb on to the real servers. How does this work with lmtp? protocols = imap lmtp protocol lmtp { auth_socket_path = director-userdb } lmtp_proxy = yes # passdb check on real servers passdb { driver = static args = proxy=y nopassword=y } Cor From cor at xs4all.nl Sat Apr 14 19:52:40 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sat, 14 Apr 2012 18:52:40 +0200 Subject: [Dovecot] LMTP auth problem In-Reply-To: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> References: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> Message-ID: <20120414165240.GA31983@xs4all.nl> Of course the moment I post I seem to have figured it out.. service auth { unix_listener auth-userdb { mode = 0777 } } Is this safe if your servers are secure? Cor From tlx at leuxner.net Sat Apr 14 20:10:44 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sat, 14 Apr 2012 19:10:44 +0200 Subject: [Dovecot] LMTP auth problem In-Reply-To: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> References: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> Message-ID: <71BEFF67-4A0A-442A-9ED0-C4B1B23A4E7F@leuxner.net> Am 14.04.2012 um 18:24 schrieb Cor Bosman: > Apr 14 14:29:44 lmtpdirector1 dovecot: auth: Error: passdb(scorpio,127.0.0.1): Auth client doesn't have permissions to do a PASS lookup: /var/run/dovecot/auth-userdb mode=0666, but not owned by UID 112(dovecot) > Apr 14 14:29:44 lmtpdirector1 dovecot: lmtp(18298): Error: user scorpio: Auth PASS lookup failed I'd just try 'user = dovecot' rather than making it wide open because that's what the log basically says. $ doveconf -d | grep 'unix_listener auth-userdb' -A 4 unix_listener auth-userdb { group = mode = 0666 user = } Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From cor at xs4all.nl Sat Apr 14 20:21:35 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sat, 14 Apr 2012 19:21:35 +0200 Subject: [Dovecot] LMTP auth problem In-Reply-To: <71BEFF67-4A0A-442A-9ED0-C4B1B23A4E7F@leuxner.net> References: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> <71BEFF67-4A0A-442A-9ED0-C4B1B23A4E7F@leuxner.net> Message-ID: <20120414172135.GA32608@xs4all.nl> > > Apr 14 14:29:44 lmtpdirector1 dovecot: auth: Error: passdb(scorpio,127.0.0.1): Auth client doesn't have permissions to do a PASS lookup: /var/run/dovecot/auth-userdb mode=0666, but not owned by UID 112(dovecot) > > Apr 14 14:29:44 lmtpdirector1 dovecot: lmtp(18298): Error: user scorpio: Auth PASS lookup failed > > I'd just try 'user = dovecot' rather than making it wide open because that's what the log basically says. > > $ doveconf -d | grep 'unix_listener auth-userdb' -A 4 > unix_listener auth-userdb { > group = > mode = 0666 > user = > } > My config was the same as yours. That didnt work for me. But if I add user = dovecot mode = 0666 That does work. Of course, the difference between 777 and 666 is minimal. I think 666 is handled as a special case in the code? Cor From stephan at rename-it.nl Sun Apr 15 01:36:08 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 15 Apr 2012 00:36:08 +0200 Subject: [Dovecot] Sieve pipe extension - can it retur something? In-Reply-To: <87ty0m5x4p.fsf@alfa.kjonca> References: <87ty0m5x4p.fsf@alfa.kjonca> Message-ID: <4F89FBD8.9040501@rename-it.nl> Op 4/14/2012 2:13 PM, Kamil Jo?ca schreef: > > I have a question about sieve pipe: can it return something to further > processing? > For example in procmail I can do: > --8<---------------cut here---------------start------------->8--- > :0 > VIRUS=`$CLAMSCAN --mbox --disable-summary --stdout -` > --8<---------------cut here---------------end--------------->8--- > and then test VIRUS variable. > > Maybe I missing something, when read > http://hg.rename-it.nl/pigeonhole-0.2-sieve-pipe/raw-file/tip/doc/rfc/spec-bosch-sieve-pipe.txt For Pigeonhole 0.3/Dovecot 2.1 there is a new plugin called ExtPrograms. Apart from the 'pipe' extension it adds the 'execute' extension that should match just what you want: http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/raw-file/d4683490a878/doc/rfc/spec-bosch-sieve-extprograms.txt Regards, Stephan. From stan at hardwarefreak.com Sun Apr 15 01:39:55 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 14 Apr 2012 17:39:55 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <20120414100422.GA5353@dibs.tanso.net> References: <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <20120414100422.GA5353@dibs.tanso.net> Message-ID: <4F89FCBB.4070305@hardwarefreak.com> On 4/14/2012 5:04 AM, Jan-Frode Myklebust wrote: > On Fri, Apr 13, 2012 at 07:33:19AM -0500, Stan Hoeppner wrote: >>> >>> What I meant wasn't the drive throwing uncorrectable read errors but >>> the drives are returning different data that each think is correct or >>> both may have sent the correct data but one of the set got corrupted >>> on the fly. After reading the articles posted, maybe the correct term >>> would be the controller receiving silently corrupted data, say due to >>> bad cable on one. >> >> This simply can't happen. What articles are you referring to? If the >> author is stating what you say above, he simply doesn't know what he's >> talking about. > > It has happened to me, with RAID5 not RAID1. It was a firmware bug > in the raid controller that caused the RAID array to go silently > corrupted. The HW reported everything green -- but the filesystem was > reporting lots of strange errors.. This LUN was part of a larger > filesystem striped over multiple LUNs, so parts of the fs was OK, while > other parts was corrupt. > > It was this bug: > > http://delivery04.dhe.ibm.com/sar/CMA/SDA/02igj/7/ibm_fw1_ds4kfc_07605200_anyos_anycpu.chg > - Fix 432525 - CR139339 Data corruption found on drive after > reconstruct from GHSP (Global Hot Spare) Note my comments were specific to the RAID1 case, or a concatenated set of RAID1 devices. And note the discussion was framed around silent corruption in the absence of bugs and hardware failure, or should I say, where no bugs or hardware failures can be identified. > > >> In closing, I'll simply say this: If hardware, whether a mobo-down SATA >> chip, or a $100K SGI SAN RAID controller, allowed silent data corruption >> or transmission to occur, there would be no storage industry, and we'll >> all still be using pen and paper. The questions you're asking were >> solved by hardware and software engineers decades ago. You're fretting >> and asking about things that were solved decades ago. > > Look at the plans are for your favorite fs: > > http://www.youtube.com/watch?v=FegjLbCnoBw > > They're planning on doing metadata checksumming to be sure they don't > receive corrupted metadata from the backend storage, and say that data > validation is a storage subsystem *or* application problem. You can't made sure you don't receive corrupted data. You take steps to mitigate the negative effects of it if and when it happens. The XFS devs are planning this for the future. If the problem was here now, this work would have already been done. > Hardly a solved problem.. It has been up to this point. The issue going forward is that current devices don't employ sufficient consistency checking to meet future needs. And the disk drive makers apparently don't want to consume the additional bits required to properly do this in the drives. If they'd dedicate far more bits to ECC we may not have this issue. But since it appears this isn't going to change, kernel, filesystem and application developers are taking steps to mitigate it. Again, this "silent corruption" issue as described in the various academic papers is a future problem for most, not a current problem. It's only a current problem for those are the bleeding edge of large scale storage. Note that firmware bugs in individual products aren't part of this issue. Those will be with us forever in various products because humans make mistakes. No amount of filesystem or application code can mitigate those. The solution to that is standard best practices: snapshots, backups, or even mirroring all your storage across different vendor hardware. -- Stan From stan at hardwarefreak.com Sun Apr 15 03:05:19 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 14 Apr 2012 19:05:19 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F894AC8.9060406@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F8846D7.7020900@wildgooses.com> <4F88F377.1070801@hardwarefreak.com> <4F894AC8.9060406@wildgooses.com> Message-ID: <4F8A10BF.3020706@hardwarefreak.com> On 4/14/2012 5:00 AM, Ed W wrote: > On 14/04/2012 04:48, Stan Hoeppner wrote: >> On 4/13/2012 10:31 AM, Ed W wrote: >> >>> You mean those "answers" like: >>> "you need to read 'those' articles again" >>> >>> Referring to some unknown and hard to find previous emails is not the >>> same as answering? >> No, referring to this: >> >> On 4/12/2012 5:58 AM, Ed W wrote: >> >>> The claim by ZFS/BTRFS authors and others is that data silently "bit >>> rots" on it's own. >> Is it not a correct assumption that you read this in articles? If you >> read this in books, scrolls, or chiseled tablets, my apologies for >> assuming it was articles. >> > > WHAT?!! The original context was that you wanted me to learn some very > specific thing that you accused me of misunderstanding, and then it > turns out that the thing I'm supposed to learn comes from re-reading > every email, every blog post, every video, every slashdot post, every > wiki, every ... that mentions ZFS's reason for including end to end > checksumming?!! No, the original context was your town crier statement that the sky is falling due to silent data corruption. I pointed out that this is not the case, currently, that most wouldn't see this until quite a few years down the road. I provided facts to back my statement, which you didn't seem to grasp or comprehend. I pointed this out and your top popped with a cloud of steam. > Please stop wasting our time and get specific Whose time am I wasting Ed? You're the primary person one on this list who wastes everyone's time with these drawn out threads, usually unrelated to Dovecot. I have been plenty specific. The problem is you lack the knowledge and understanding of hardware communication. You're upset because I'm not pointing out the knowledge you seem to lack? Is that not a waste of everyone's time? Is that not be even "more insulting"? Causing even more excited/heated emails from you? > You have taken my email which contained a specific question, been asked > of you multiple times now and yet you insist on only answering > irrelevant details with a pointed and personal dig on each answer. The > rudeness is unnecessary, and your evasiveness of answers does not fill > me with confidence that you actually know the answer... Ed, I have not been rude. I've been attempting to prevent you dragging us into the mud, which you've done, as you often do. How specific would you like me to get? This is what you seem to be missing: Drives perform per sector CRC before transmitting data to the HBA. ATA, SATA, SCSI, SAS, fiber channel devices and HBAs all perform CRC on wire data. The PCI/PCI-X/PCIe buses/channels and Southbridge all perform CRC on wire data. HyperTransport, and Intel's proprietary links also perform CRC on wire transmissions. Server memory is protected by ECC, some by ChipKill which can tolerate double bit errors. With today's systems and storage densities, with error correcting code on all data paths within the system, and on the drives themselves, "silent data corruption" is not an issue--in absence of defective hardware or a bug, which are not relevant to the discussion. > For the benefit of anyone reading this via email archives or whatever, I > think the conclusion we have reached is that: modern systems are now a) > a complex sum of pieces, any of which can cause an error to be injected, Errors occur all the time. And they're corrected nearly all of the time, on modern complex systems. Silent errors do not occur frequently, usually not at all, on most modern systems. > b) the level of error correction which was originally specified as being > sufficient is now starting to be reached in real systems, FSVO 'real systems'. The few occurrences of "silent data corruption" I'm aware of have been documented in academic papers published by researches working at taxpayer funded institutions. In the case of CERN, the problem was a firmware bug in the Western Digital drives that caused an issue with the 3Ware controllers. This kind of thing happens when using COTS DIY hardware in the absence of proper load validation testing. So this case doesn't really fit the Henny-penny silent data corruption scenario as a firmware bug caused it. One that should have been caught and corrected during testing. In the other cases I'm aware of, all were HPC systems which generated SDC under extended high loads, and these SDCs nearly all occurred somewhere other than the storage systems--CPUs, RAM, interconnect, etc. HPC apps tend to run the CPUs, interconnects, storage, etc, at full bandwidth for hours at a time, across tens of thousands of nodes, so the probability of SDC is much higher simply due to scale. > possibly even > consumer systems. Possibly? If you're going to post pure conjecture why not say "possibly even iPhones or Androids"? There's no data to back either claim. Stick to the facts. > There is no "solution", however, the first step is to > enhance "detection". Various solutions have been proposed, all increase > cost, computation or have some disadvantage - however, one of the more > promising detection mechanisms is an end to end checksum, which will > then have the effect of augmenting ALL the steps in the chain, not just > one specific step. As of today, only a few filesystems offer this, roll > on more adopting it So after all the steam blowing, we're back to where we started. I disagree with your assertion that this is an issue that we--meaning "average" users not possessing 1PB storage systems or massive clusters--need to be worried about TODAY. I gave sound reasons as to why this is the case. You've given us 'a couple of academic papers say the sky is falling so I'm repeating the sky is falling'. Without apparently truly understanding the issue. The data available and the experience of the vast majority of IT folks backs my position--which is why that's my position. There is little to no data supporting your position. I say this isn't going to be an issue for average users, if at all, for a few years to come. You say it's here now. That's a fairly minor point of disagreement to cause such a heated (on your part) lengthy exchange. BTW, if you see anything I've stated as rude you've apparently not been on the Interwebs long. ;) -- Stan From dovecot-user at spambox.dk Sun Apr 15 12:42:16 2012 From: dovecot-user at spambox.dk (Henrik Larsson) Date: Sun, 15 Apr 2012 11:42:16 +0200 Subject: [Dovecot] 2.1.2 Corrupted squat uidlist In-Reply-To: <4F7164DC.7010706@unict.it> References: <4F7164DC.7010706@unict.it> Message-ID: <4F8A97F8.70808@spambox.dk> On 27-03-2012 08:57, Luca Palazzo wrote: > Hi Timo and All, > after upgrading to 2.1.2 i'm getting a lot of these messages: > Error: Corrupted squat uidlist file XXXXXX wrong indexid After an upgrade to 2.1.3 i see this as well. Any thoughts? Apr 15 03:43:43 imap(xxxx): Error: Corrupted squat uidlist file /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid Apr 15 06:30:27 imap(xxxx): Error: Corrupted squat uidlist file /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid Apr 15 07:43:55 pop3(xxxx): Error: Corrupted squat uidlist file /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid Apr 15 09:00:01 imap(xxxx): Error: Corrupted squat uidlist file /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid # doveconf -n # 2.1.3: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 8.2-STABLE amd64 auth_mechanisms = plain login digest-md5 cram-md5 first_valid_uid = 125 listen = * log_path = /var/log/dovecot mail_plugins = fts fts_squat zlib mail_privileged_group = postfix mail_temp_dir = /var/db/dovecot managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = prefix = separator = . type = private } namespace { hidden = yes inbox = no list = no location = prefix = INBOX. separator = . type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { fts = squat fts_squat = partial=4 full=10 sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve service auth-worker { user = $default_internal_user } service auth { unix_listener /home/mail/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } service lmtp { executable = lmtp -L unix_listener /home/mail/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } ssl_cert = Hi Timo the virtual setup got broke after update to 2.1 ( its still working under 2.0 ) on other loadbalanced server namespace default { separator = / prefix = "" list = yes subscriptions = yes hidden = no } namespace virtual { prefix = "virtual/" separator = / location = virtual:/etc/dovecot/virtual:LAYOUT=maildir++:INDEX=~/virtual hidden = yes list = no subscriptions= no } namespace real { prefix = "RealMails/" separator = / list = no hidden = yes } after upgrade i have Error: Couldn't open INBOX: Virtual mailbox open failed because of mailbox /RealMails/: Invalid mailbox name --------------------- also i wanted to include namespace inbox { #mailbox name { # auto=create will automatically create this mailbox. # auto=subscribe will both create and subscribe to the mailbox. #auto = no # Space separated list of IMAP SPECIAL-USE attributes as specified by # RFC 6154: \All \Archive \Drafts \Flagged \Junk \Sent \Trash #special_use = #} # These mailboxes are widely used and could perhaps be created automatically: mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Trash { special_use = \Trash } # For \Sent mailboxes there are two widely used names. We'll mark both of # them as \Sent. User typically deletes one of them if duplicates are created. mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } # If you have a virtual "All messages" mailbox: mailbox virtual/All { special_use = \All } # If you have a virtual "Flagged" mailbox: mailbox virtual/Flagged { special_use = \Flagged } } but i found no working namespace combination with virtual -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Sun Apr 15 16:14:25 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 15 Apr 2012 15:14:25 +0200 Subject: [Dovecot] http://xi.rename-it.nl upgrade 2.0 - 2.1 dovecot-common (<< 2:2.1.alpha1-0~auto+47) Message-ID: <4F8AC9B1.2030307@schetterer.org> Hi, i got depency problem like dovecot-common (<< 2:2.1.alpha1-0~auto+47) dpkg -i dovecot-common_2.1.4-0~auto+3_all.de after all dove seems to run without install dovecot-common basicly -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From mhlavink at redhat.com Mon Apr 16 12:55:28 2012 From: mhlavink at redhat.com (Michal Hlavinka) Date: Mon, 16 Apr 2012 11:55:28 +0200 Subject: [Dovecot] dovecot and systemd In-Reply-To: <1331820329.10319.32.camel@innu> References: <4F61EFE8.1000901@redhat.com> <1331820329.10319.32.camel@innu> Message-ID: <4F8BEC90.8060504@redhat.com> On 03/15/2012 03:05 PM, Timo Sirainen wrote: > On Thu, 2012-03-15 at 14:34 +0100, Michal Hlavinka wrote: >> What exactly should happen when >> dovecot.conf does not match dovecot.socket configuration? > > Dovecot's systemd code was written by one of you Redhat guys. I had some > similar thoughts when I applied the patch, but didn't really know what > to do about it, so I didn't do anything. So: I don't know. Maybe some > other project has solved this somehow already? Seems other projects did not solve this yet. Most projects provide just one service. In that case, they serve any connection they get, because they know what to do. I've discussed this with systemd upstream and we've decided that the best solution is to log error message and close that socket. > Dovecot anyway needs its own internal UNIX listeners. Should all > internal inet listeners be disabled? Could Dovecot somehow talk to > systemd and ask what listeners it's using for Dovecot and log warnings > if they don't match? I don't understand this question completely. What it does already is that during start up, when dovecot creates sockets, it checks what sockets already exist and creates only the missing ones. Systemd provides following functions: sd_is_fifo (3) - Check the type of a file descriptor sd_is_mq (3) - Check the type of a file descriptor sd_is_socket (3) - Check the type of a file descriptor sd_is_socket_inet (3) - Check the type of a file descriptor sd_is_socket_unix (3) - Check the type of a file descriptor sd_listen_fds (3) - Check for file descriptors passed by the init system. http://0pointer.de/public/systemd-man/sd_listen_fds.html I wrote simple patch that close the extra sockets. It's tested and works fine. You'll maybe want to move that function to different place and/or change wording of error messages. Michal -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-2.0.19-systemdfix.patch Type: text/x-patch Size: 2669 bytes Desc: not available URL: From root.kev at gmail.com Mon Apr 16 17:55:54 2012 From: root.kev at gmail.com (Root Kev) Date: Mon, 16 Apr 2012 10:55:54 -0400 Subject: [Dovecot] POP3 Dovecot Auth CPU usage 75%+ In-Reply-To: <337BF2E0-9F19-4AB4-A81C-2DCE76484662@iki.fi> References: <0B6D0FE2-2831-460E-8B2B-C255C49DEBAE@iki.fi> <337BF2E0-9F19-4AB4-A81C-2DCE76484662@iki.fi> Message-ID: I think my last email may have been bounced due to attachment size, I have put a snippet of the captures below. The CPU is still going to high percent of usage when my test mailboxes are used. An ideas on how to bring down the Auth CPU usage would be greatly appreciated! Thanks, Kevin Stace on the Auth process: epoll_wait(13, {{EPOLLIN, {u32=150109008, u64=150109008}}}, 29, 149958) = 1 gettimeofday({1334328634, 21072}, NULL) = 0 read(29, "VERSION\t1\t1\nREQUEST\t1011351553\t3"..., 1024) = 72 time(NULL) = 1334328634 writev(29, [{"USER\t1011351553\tservermailbox1\ts"..., 108}, {"\n", 1}], 2) = 109 gettimeofday({1334328634, 27993}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=149927248, u64=149927248}}}, 29, 149992) = 1 gettimeofday({1334328634, 32215}, NULL) = 0 accept(11, {sa_family=AF_FILE, NULL}, [2]) = 30 fcntl64(30, F_GETFL) = 0x2 (flags O_RDWR) fcntl64(30, F_SETFL, O_RDWR|O_NONBLOCK) = 0 gettimeofday({1334328634, 32342}, NULL) = 0 fstat64(30, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0 _llseek(30, 0, 0xbffd24c0, SEEK_CUR) = -1 ESPIPE (Illegal seek) getsockname(30, {sa_family=AF_FILE, path="/usr/local/var/run/dovecot"}, [41]) = 0 epoll_ctl(13, EPOLL_CTL_ADD, 30, {EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP, {u32=150123496, u64=150123496}}) = 0 write(30, "VERSION\t1\t1\nSPID\t2093\n", 22) = 22 gettimeofday({1334328634, 32625}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=150123496, u64=150123496}}}, 29, 1000) = 1 gettimeofday({1334328634, 32721}, NULL) = 0 read(30, "VERSION\t1\t1\n", 1024) = 12 gettimeofday({1334328634, 32792}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=150123496, u64=150123496}}}, 29, 1000) = 1 gettimeofday({1334328634, 32883}, NULL) = 0 read(30, "REQUEST\t3624009729\t3062\t16\tbe004"..., 1012) = 60 time(NULL) = 1334328634 writev(30, [{"USER\t3624009729\tservermailbox\tsy"..., 105}, {"\n", 1}], 2) = 106 gettimeofday({1334328634, 33062}, NULL) = 0 epoll_wait(13, {{EPOLLIN|EPOLLHUP, {u32=150094520, u64=150094520}}}, 29, 999) = 1 gettimeofday({1334328634, 33766}, NULL) = 0 read(28, "", 6243) = 0 epoll_ctl(13, EPOLL_CTL_DEL, 28, {0, {u32=150094520, u64=150094520}}) = 0 close(28) = 0 epoll_wait(13, {{EPOLLIN|EPOLLHUP, {u32=150109008, u64=150109008}}}, 29, -1) = 1 gettimeofday({1334328634, 40036}, NULL) = 0 read(29, "", 952) = 0 epoll_ctl(13, EPOLL_CTL_DEL, 29, {0, {u32=150109008, u64=150109008}}) = 0 close(29) = 0 gettimeofday({1334328634, 40163}, NULL) = 0 gettimeofday({1334328634, 40197}, NULL) = 0 epoll_wait(13, {{EPOLLIN|EPOLLHUP, {u32=150123496, u64=150123496}}}, 29, 1000) = 1 gettimeofday({1334328634, 44007}, NULL) = 0 read(30, "", 952) = 0 epoll_ctl(13, EPOLL_CTL_DEL, 30, {0, {u32=150123496, u64=150123496}}) = 0 close(30) = 0 gettimeofday({1334328634, 44148}, NULL) = 0 gettimeofday({1334328634, 44184}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=150065544, u64=150065544}}}, 29, 1000) = 1 gettimeofday({1334328634, 52466}, NULL) = 0 read(26, "AUTH\t1\tPLAIN\tservice=pop3\tlip=17"..., 8170) = 122 gettimeofday({1334328634, 52582}, NULL) = 0 writev(12, [{"PENALTY-GET\t172.20.20.110", 25}, {"\n", 1}], 2) = 26 gettimeofday({1334328634, 52698}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=149924840, u64=149924840}}}, 29, 992) = 1 gettimeofday({1334328634, 52760}, NULL) = 0 read(12, "0 0\n", 424) = 4 time(NULL) = 1334328634 gettimeofday({1334328634, 93200}, NULL) = 0 writev(26, [{"OK\t1\tuser=servermailbox1", 24}, {"\n", 1}], 2) = 25 read(12, 0x8f36c14, 420) = -1 EAGAIN (Resource temporarily unavailable) gettimeofday({1334328634, 93651}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=150065544, u64=150065544}}}, 29, 951) = 1 gettimeofday({1334328634, 93715}, NULL) = 0 read(26, "AUTH\t2\tPLAIN\tservice=pop3\tlip=17"..., 8048) = 118 gettimeofday({1334328634, 93808}, NULL) = 0 writev(12, [{"PENALTY-GET\t172.20.20.110", 25}, {"\n", 1}], 2) = 26 gettimeofday({1334328634, 93919}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=149924840, u64=149924840}}}, 29, 951) = 1 gettimeofday({1334328634, 93980}, NULL) = 0 read(12, "0 0\n", 420) = 4 time(NULL) = 1334328634 gettimeofday({1334328634, 133578}, NULL) = 0 writev(26, [{"OK\t2\tuser=servermailbox", 23}, {"\n", 1}], 2) = 24 read(12, 0x8f36c18, 416) = -1 EAGAIN (Resource temporarily unavailable) gettimeofday({1334328634, 133998}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=149927248, u64=149927248}}}, 29, 911) = 1 gettimeofday({1334328634, 134064}, NULL) = 0 accept(11, {sa_family=AF_FILE, NULL}, [2]) = 28 fcntl64(28, F_GETFL) = 0x2 (flags O_RDWR) fcntl64(28, F_SETFL, O_RDWR|O_NONBLOCK) = 0 gettimeofday({1334328634, 134200}, NULL) = 0 fstat64(28, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0 _llseek(28, 0, 0xbffd24c0, SEEK_CUR) = -1 ESPIPE (Illegal seek) getsockname(28, {sa_family=AF_FILE, path="/usr/local/var/run/dovecot"}, [41]) = 0 epoll_ctl(13, EPOLL_CTL_ADD, 28, {EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP, {u32=150094520, u64=150094520}}) = 0 Debug on the auth process: Apr 13 11:01:58 devsmtp dovecot: auth: Debug: cache(qamailbox,172.20.20.222): miss Apr 13 11:01:58 devsmtp dovecot: auth-worker(3432): Debug: Loading modules from directory: /usr/local/lib/dovecot/auth Apr 13 11:01:58 devsmtp dovecot: auth-worker(3432): Debug: shadow(qamailbox,172.20.20.222): lookup Apr 13 11:01:58 devsmtp dovecot: auth: Debug: client out: OK#0111#011user=qamailbox Apr 13 11:01:58 devsmtp dovecot: auth: Debug: master in: REQUEST#0111220673537#0113397#0111#0115609887f745a84903ce3699d23e7b886 Apr 13 11:01:58 devsmtp dovecot: auth: Debug: userdb-cache(qamailbox,172.20.20.222): miss Apr 13 11:01:58 devsmtp dovecot: auth-worker(3432): Debug: passwd(qamailbox,172.20.20.222): lookup Apr 13 11:01:58 devsmtp dovecot: auth: Debug: master out: USER#0111220673537#011qamailbox#011system_groups_user=qamailbox#011uid=1002#011gid=1002#011home=/home/qamailbox Apr 13 11:01:58 devsmtp dovecot: pop3-login: Login: user=, method=PLAIN, rip=172.20.20.222, lip=172.20.20.222, mpid=3433, secured Apr 13 11:02:05 devsmtp dovecot: pop3(qamailbox): Disconnected: Logged out top=0/0, retr=1/15637, del=0/50, size=779917 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: auth client connected (pid=3397) Apr 13 11:05:31 devsmtp dovecot: auth: Debug: client in: AUTH#0112#011PLAIN#011service=pop3#011lip=172.20.20.222#011rip=172.20.20.110#011lport=110#011rport=53254#011resp=AHNlcnZlcm1haWxib3gxADEyMzQ1Ng== Apr 13 11:05:31 devsmtp dovecot: auth: Debug: cache(servermailbox1,172.20.20.110): miss Apr 13 11:05:31 devsmtp dovecot: auth-worker(3459): Debug: Loading modules from directory: /usr/local/lib/dovecot/auth Apr 13 11:05:31 devsmtp dovecot: auth-worker(3459): Debug: shadow(servermailbox1,172.20.20.110): lookup Apr 13 11:05:31 devsmtp dovecot: auth: Debug: client out: OK#0112#011user=servermailbox1 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: master in: REQUEST#011358219777#0113397#0112#01132ecf6b93729a90ece98bbb643446ea6 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: userdb-cache(servermailbox1,172.20.20.110): miss Apr 13 11:05:31 devsmtp dovecot: auth-worker(3459): Debug: passwd(servermailbox1,172.20.20.110): lookup Apr 13 11:05:31 devsmtp dovecot: auth: Debug: master out: USER#011358219777#011servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:31 devsmtp dovecot: pop3-login: Login: user=, method=PLAIN, rip=172.20.20.110, lip=172.20.20.222, mpid=3460 Apr 13 11:05:31 devsmtp dovecot: pop3(servermailbox1): Disconnected: Logged out top=5/6281, retr=5/77906, del=0/50, size=778637 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: client in: AUTH#0113#011PLAIN#011service=pop3#011lip=172.20.20.222#011rip=172.20.20.110#011lport=110#011rport=53255#011resp=AHNlcnZlcm1haWxib3gxADEyMzQ1Ng== Apr 13 11:05:31 devsmtp dovecot: auth: Debug: cache(servermailbox1,172.20.20.110): hit: {CRYPT}$6$bpFXtlOP$zczdubFhGyRjJA0PD9lr/QbWAdweg3jThwkxkfCEGgxPxFpualm1ea.8rECmM1UZ0YuTNKWpiGPHwLhys1luy0#011user=servermailbox1#011user=servermailbox1 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: client out: OK#0113#011user=servermailbox1 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: master in: REQUEST#0112057699329#0113397#0113#01132ecf6b93729a90ece98bbb643446ea6 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: userdb-cache(servermailbox1,172.20.20.110): hit: servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: master out: USER#0112057699329#011servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:31 devsmtp dovecot: pop3-login: Login: user=, method=PLAIN, rip=172.20.20.110, lip=172.20.20.222, mpid=3461 Apr 13 11:05:31 devsmtp dovecot: pop3(servermailbox1): Disconnected: Logged out top=0/0, retr=0/0, del=0/50, size=778637 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=pop3#011lip=172.20.20.222#011rip=172.20.20.110#011lport=110#011rport=53261#011resp=AHNlcnZlcm1haWxib3gxADEyMzQ1Ng== Apr 13 11:05:33 devsmtp dovecot: auth: Debug: cache(servermailbox1,172.20.20.110): hit: {CRYPT}$6$bpFXtlOP$zczdubFhGyRjJA0PD9lr/QbWAdweg3jThwkxkfCEGgxPxFpualm1ea.8rECmM1UZ0YuTNKWpiGPHwLhys1luy0#011user=servermailbox1#011user=servermailbox1 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: client out: OK#0111#011user=servermailbox1 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: master in: REQUEST#0113081109505#0113386#0111#01146c7d95e8b31022008fef693a1ef018c Apr 13 11:05:33 devsmtp dovecot: auth: Debug: userdb-cache(servermailbox1,172.20.20.110): hit: servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: master out: USER#0113081109505#011servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:33 devsmtp dovecot: pop3-login: Login: user=, method=PLAIN, rip=172.20.20.110, lip=172.20.20.222, mpid=3462 Apr 13 11:05:33 devsmtp dovecot: pop3(servermailbox1): Disconnected: Logged out top=0/0, retr=0/0, del=1/50, size=778637 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: client in: AUTH#0114#011PLAIN#011service=pop3#011lip=172.20.20.222#011rip=172.20.20.110#011lport=110#011rport=53262#011resp=AHNlcnZlcm1haWxib3gxADEyMzQ1Ng== Apr 13 11:05:33 devsmtp dovecot: auth: Debug: cache(servermailbox1,172.20.20.110): hit: {CRYPT}$6$bpFXtlOP$zczdubFhGyRjJA0PD9lr/QbWAdweg3jThwkxkfCEGgxPxFpualm1ea.8rECmM1UZ0YuTNKWpiGPHwLhys1luy0#011user=servermailbox1#011user=servermailbox1 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: client out: OK#0114#011user=servermailbox1 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: master in: REQUEST#0112632187905#0113397#0114#01132ecf6b93729a90ece98bbb643446ea6 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: userdb-cache(servermailbox1,172.20.20.110): hit: servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: master out: USER#0112632187905#011servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:33 devsmtp dovecot: pop3-login: Login: user=, method=PLAIN, rip=172.20.20.110, lip=172.20.20.222, mpid=3463 Apr 13 11:05:33 devsmtp dovecot: pop3(servermailbox1): Disconnected: Logged out top=0/0, retr=0/0, del=0/49, size=763044 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: client in: AUTH#0115#011PLAIN#011service=pop3#011lip=172.20.20.222#011rip=172.20.20.110#011lport=110#011rport=53263#011resp=AHNlcnZlcm1haWxib3gxADEyMzQ1Ng== Apr 13 11:05:33 devsmtp dovecot: auth: Debug: cache(servermailbox1,172.20.20.110): hit: {CRYPT}$6$bpFXtlOP$zczdubFhGyRjJA0PD9lr/QbWAdweg3jThwkxkfCEGgxPxFpualm1ea.8rECmM1UZ0YuTNKWpiGPHwLhys1luy0#011user=servermailbox1#011user=servermailbox1 From robert at schetterer.org Mon Apr 16 18:48:13 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 16 Apr 2012 17:48:13 +0200 Subject: [Dovecot] problems upgrade 2.0 - 2.1 virtual inbox broken / special use namespace In-Reply-To: <4F8AC473.7030307@schetterer.org> References: <4F8AC473.7030307@schetterer.org> Message-ID: <4F8C3F3D.3030103@schetterer.org> Am 15.04.2012 14:52, schrieb Robert Schetterer: > Hi Timo > > the virtual setup got broke after update to 2.1 > ( its still working under 2.0 ) on other loadbalanced server > > namespace default { > separator = / > prefix = "" > list = yes > subscriptions = yes > hidden = no > } > > namespace virtual { > prefix = "virtual/" > separator = / > location = virtual:/etc/dovecot/virtual:LAYOUT=maildir++:INDEX=~/virtual > hidden = yes > list = no > subscriptions= no > } > > namespace real { > prefix = "RealMails/" > separator = / > list = no > hidden = yes > } > > > after upgrade i have > > Error: Couldn't open INBOX: Virtual mailbox open failed because of > mailbox /RealMails/: Invalid mailbox name > > --------------------- ok i identified the problem but didnt find a working other solution +RealMails/ << not longer work in 2.1 +RealMails/* -RealMails/Trash -RealMails/Trash/* -RealMails/Sent -RealMails/Sent/* -RealMails/Templates -RealMails/Templates/* -RealMails/Drafts -RealMails/Drafts/* -RealMails/Archives -RealMails/Archives/* all > > also > > i wanted > to include > > namespace inbox { > > #mailbox name { > # auto=create will automatically create this mailbox. > # auto=subscribe will both create and subscribe to the mailbox. > #auto = no > > # Space separated list of IMAP SPECIAL-USE attributes as specified by > # RFC 6154: \All \Archive \Drafts \Flagged \Junk \Sent \Trash > #special_use = > #} > > # These mailboxes are widely used and could perhaps be created > automatically: > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Trash { > special_use = \Trash > } > > # For \Sent mailboxes there are two widely used names. We'll mark both of > # them as \Sent. User typically deletes one of them if duplicates are > created. > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > > # If you have a virtual "All messages" mailbox: > mailbox virtual/All { > special_use = \All > } > > # If you have a virtual "Flagged" mailbox: > mailbox virtual/Flagged { > special_use = \Flagged > } > } > > but i found no working namespace combination with virtual -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Mon Apr 16 22:01:26 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 16 Apr 2012 21:01:26 +0200 Subject: [Dovecot] problems upgrade 2.0 - 2.1 virtual inbox broken / special use namespace /solved In-Reply-To: <4F8C3F3D.3030103@schetterer.org> References: <4F8AC473.7030307@schetterer.org> <4F8C3F3D.3030103@schetterer.org> Message-ID: <4F8C6C86.7000707@schetterer.org> Am 16.04.2012 17:48, schrieb Robert Schetterer: > Am 15.04.2012 14:52, schrieb Robert Schetterer: >> Hi Timo >> >> the virtual setup got broke after update to 2.1 >> ( its still working under 2.0 ) on other loadbalanced server >> >> namespace default { >> separator = / >> prefix = "" >> list = yes >> subscriptions = yes >> hidden = no >> } >> >> namespace virtual { >> prefix = "virtual/" >> separator = / >> location = virtual:/etc/dovecot/virtual:LAYOUT=maildir++:INDEX=~/virtual >> hidden = yes >> list = no >> subscriptions= no >> } >> >> namespace real { >> prefix = "RealMails/" >> separator = / >> list = no >> hidden = yes >> } >> >> >> after upgrade i have >> >> Error: Couldn't open INBOX: Virtual mailbox open failed because of >> mailbox /RealMails/: Invalid mailbox name >> >> --------------------- > > ok i identified the problem > but didnt find a working other solution > > > > +RealMails/ << not longer work in 2.1 > +RealMails/* > -RealMails/Trash > -RealMails/Trash/* > -RealMails/Sent > -RealMails/Sent/* > -RealMails/Templates > -RealMails/Templates/* > -RealMails/Drafts > -RealMails/Drafts/* > -RealMails/Archives > -RealMails/Archives/* > all > > > >> >> also >> >> i wanted >> to include >> >> namespace inbox { >> >> #mailbox name { >> # auto=create will automatically create this mailbox. >> # auto=subscribe will both create and subscribe to the mailbox. >> #auto = no >> >> # Space separated list of IMAP SPECIAL-USE attributes as specified by >> # RFC 6154: \All \Archive \Drafts \Flagged \Junk \Sent \Trash >> #special_use = >> #} >> >> # These mailboxes are widely used and could perhaps be created >> automatically: >> mailbox Drafts { >> special_use = \Drafts >> } >> mailbox Junk { >> special_use = \Junk >> } >> mailbox Trash { >> special_use = \Trash >> } >> >> # For \Sent mailboxes there are two widely used names. We'll mark both of >> # them as \Sent. User typically deletes one of them if duplicates are >> created. >> mailbox Sent { >> special_use = \Sent >> } >> mailbox "Sent Messages" { >> special_use = \Sent >> } >> >> # If you have a virtual "All messages" mailbox: >> mailbox virtual/All { >> special_use = \All >> } >> >> # If you have a virtual "Flagged" mailbox: >> mailbox virtual/Flagged { >> special_use = \Flagged >> } >> } >> >> but i found no working namespace combination with virtual > > for small test this setting should work with 2.1 and special use namespace notice default namespace virtual example in the wiki must change to namespace inbox so the i.e sql lookup must change CASE '%s' WHEN 'pop3' THEN NULL ELSE 'yes' END AS 'namespace/inbox /inbox', \ CASE '%s' WHEN 'pop3' THEN 'yes' ELSE NULL END AS 'namespace/virtual/inbox', \ 10-mail.conf .. namespace inbox { prefix = separator = / list = yes } 15-mailboxes.conf as you like --------------------------------- the virtual fix at my setup is RealMails +RealMails/* -RealMails/Trash -RealMails/Trash/* -RealMails/Sent -RealMails/Sent/* -RealMails/Templates -RealMails/Templates/* -RealMails/Drafts -RealMails/Drafts/* -RealMails/Archives -RealMails/Archives/* all -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From dlie76 at yahoo.com.au Tue Apr 17 05:31:56 2012 From: dlie76 at yahoo.com.au (Daminto Lie) Date: Mon, 16 Apr 2012 19:31:56 -0700 (PDT) Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> <4F86DCA7.1060806@talex.pl> <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> Message-ID: <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> Thank you folks for your replies. I realised that the dovecot 1.2.9 that I installed via ubuntu synaptic package manager has already come with sieve plugins. After making some modifications on the dovecot.conf file, I went to download and install avelsieve 1.0.1 (stable version) onto the squirrelmail. Then, I run in squirrelmail conf.pl and added the plugin. I logged in and could see "Filter" option right on the top header row. I went to click it and nothing came up. I checked in the mail.log but did not see any errors there. Here is my snapshot of dovecot configuration # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS ext4 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap listen: *:143 ssl: no disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login verbose_proctitle: yes first_valid_uid: 1001 last_valid_uid: 600 mail_privileged_group: mail mail_location: maildir:/var/vmail/%u/Maildir mbox_write_locks: fcntl dotlock imap_client_workarounds: outlook-idle delay-newmail netscape-eoh tb-extra-mailbox-sep lda: ? postmaster_address: postmaster at ourcompany.com ? mail_plugins: sieve ? quota_full_tempfail: yes ? deliver_log_format: msgid=%m: %$ ? sendmail_path: /usr/lib/sendmail ? rejection_reason: Your message to <%t> was automatically rejected:%n%r ? auth_socket_path: /var/run/dovecot-auth-master auth default: ? mechanisms: plain login ? username_format: %Lu ? verbose: yes ? debug: yes ? debug_passwords: yes ? passdb: ??? driver: pam ? passdb: ??? driver: ldap ??? args: /etc/dovecot/dovecot-ldap.conf ? userdb: ??? driver: passwd ? userdb: ??? driver: static ??? args: uid=1001 gid=1001 home=/var/vmail/%u allow_all_users=yes ? socket: ??? type: listen ??? client: ????? path: /var/spool/postfix/private/auth ????? mode: 432 ????? user: postfix ????? group: mail ??? master: ????? path: /var/run/dovecot-auth-master ????? mode: 432 ????? user: vmail ????? group: vmail plugin: ? sieve: /var/vmail/%u/.sieve Also, I have restarted dovecot after making changes to the dovecot.conf. Any help would be very much appreciated. Thank you ? ________________________________ From: Artur Zaprza?a To: Dovecot Mailing List Cc: Daminto Lie Sent: Thursday, 12 April 2012 11:46 PM Subject: Re: [Dovecot] vacation plugins for squirrelmail Daminto Lie wrote: > Hi, > > I am afraid I have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. > > Any help would be very much appreciated. > > Thank you > I'm using Avelsieve 1.9.9 with a set of my own bugfixes: http://email.uoa.gr/avelsieve/ -- Talex Sp??ka Akcyjna z siedzib? w Poznaniu adres: ul. Karpia 27d, 61-619 Pozna? NIP 782-00-21-045 zarejestrowana w S?dzie Rejonowym Pozna? ? Nowe Miasto i Wilda w Poznaniu VIII Wydzia? Gospodarczy - KRS pod nr 000048779 kapita? zak?adowy: 3.000.092,00 PLN (w ca?o?ci wp?acony) Uwaga: Niniejsza wiadomo??, w szczeg?lno?ci jej tre?? oraz za??czniki, mo?e by? poufna. W przypadku, gdy nie jest Pan/Pani zamierzonym jej adresatem, informujemy, ?e wszelkie rozpowszechnianie, dystrybucja lub powielanie powy?szej wiadomo?ci jest zabronione. Jednocze?nie prosimy o powiadomienie nadawcy oraz niezw?oczne usuni?cie powy?szej wiadomo?ci wraz z za??cznikami. Dzi?kujemy, Talex S.A. w Poznaniu. Confidentiality Notice: This email, particularly its content and any attached files, may be confidential. If you are not an intended recipient, any disclosure, distribution and reproduction of this message is prohibited. In this case please notify the sender immediately and then delete this message and any attachments. Thank you, Talex S.A., Poznan. From dovecot-list at mohtex.net Tue Apr 17 06:25:44 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Tue, 17 Apr 2012 10:25:44 +0700 Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> <4F86DCA7.1060806@talex.pl> <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> Message-ID: <4F8CE2B8.3080801@mohtex.net> Show us your avelsieve-config in your-squirrelmail-dir/plugins/avelsieve/config/config.php Daminto Lie wrote the following on 17.04.2012 09:31: > Thank you folks for your replies. > > > I realised that the dovecot 1.2.9 that I installed via ubuntu synaptic package manager has already come with sieve plugins. After making some modifications on the dovecot.conf file, I went to download and install avelsieve 1.0.1 (stable version) onto the squirrelmail. Then, I run in squirrelmail conf.pl and added the plugin. I logged in and could see "Filter" option right on the top header row. I went to click it and nothing came up. I checked in the mail.log but did not see any errors there. > > Here is my snapshot of dovecot configuration > # 1.2.9: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS ext4 > log_timestamp: %Y-%m-%d %H:%M:%S > protocols: imap > listen: *:143 > ssl: > no > disable_plaintext_auth: no > login_dir: /var/run/dovecot/login > login_executable: /usr/lib/dovecot/imap-login > verbose_proctitle: yes > first_valid_uid: 1001 > last_valid_uid: 600 > mail_privileged_group: mail > mail_location: maildir:/var/vmail/%u/Maildir > mbox_write_locks: fcntl dotlock > imap_client_workarounds: outlook-idle delay-newmail netscape-eoh tb-extra-mailbox-sep > lda: > postmaster_address: postmaster at ourcompany.com > mail_plugins: sieve > quota_full_tempfail: yes > deliver_log_format: msgid=%m: %$ > sendmail_path: /usr/lib/sendmail > rejection_reason: Your message to<%t> was automatically rejected:%n%r > auth_socket_path: /var/run/dovecot-auth-master > auth default: > mechanisms: plain login > username_format: %Lu > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: > pam > passdb: > driver: ldap > args: /etc/dovecot/dovecot-ldap.conf > userdb: > driver: passwd > userdb: > driver: static > args: uid=1001 gid=1001 home=/var/vmail/%u allow_all_users=yes > socket: > type: listen > client: > path: /var/spool/postfix/private/auth > mode: 432 > user: postfix > group: mail > master: > path: /var/run/dovecot-auth-master > mode: 432 > user: vmail > group: vmail > plugin: > sieve: /var/vmail/%u/.sieve > > Also, I have restarted dovecot after making changes to the dovecot.conf. > > Any help would be very much appreciated. > > Thank you > > > > > ________________________________ > From: Artur Zaprza?a > To: Dovecot Mailing List > Cc: Daminto Lie > Sent: Thursday, 12 April 2012 11:46 PM > Subject: Re: [Dovecot] vacation plugins for squirrelmail > > Daminto Lie wrote: >> Hi, >> >> I am afraid I > have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. >> Any help would be very much appreciated. >> >> Thank you >> > I'm using Avelsieve 1.9.9 with a set of my own bugfixes: > http://email.uoa.gr/avelsieve/ > > From dlie76 at yahoo.com.au Tue Apr 17 09:14:57 2012 From: dlie76 at yahoo.com.au (Daminto Lie) Date: Mon, 16 Apr 2012 23:14:57 -0700 (PDT) Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <4F8CE2B8.3080801@mohtex.net> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> <4F86DCA7.1060806@talex.pl> <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> <4F8CE2B8.3080801@mohtex.net> Message-ID: <1334643297.87733.YahooMailNeo@web113417.mail.gq1.yahoo.com> Thank you for your reply. Here it is as follows if(!defined('AVELSIEVE_DEBUG')) { ??? define('AVELSIEVE_DEBUG', 0); } global $avelsieve_backend; $avelsieve_backend = 'File'; /*global $sieveport; $sieveport = 2000;*/ global $sieve_preferred_sasl_mech; $sieve_preferred_sasl_mech = 'PLAIN'; global $avelsieve_disabletls; $avelsieve_disabletls = false; global $avelsieve_file_backend_options, $data_dir, $username; $avelsieve_file_backend_options = array( ??? 'avelsieve_default_file' => "$data_dir/$username.sievesource" ); /*global $avelsieve_oldcyrus; $avelsieve_oldcyrus = true;*/ global $avelsieve_enable_envelope_auth; $avelsieve_enable_envelope_auth = true; global $avelsieve_custom_sieve_implementation; $avelsieve_custom_sieve_implementation = ''; global $avelsieve_striproot; $avelsieve_striproot = ''; global $avelsieve_hardcoded_capabilities; $avelsieve_hardcoded_capabilities = array( ??? 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' ); /*global $avelsieve_imapproxymode, $avelsieve_imapproxyserv; $avelsieve_imapproxymode = false; $avelsieve_imapproxyserv = array( ??? 'localhost' => 'imap.example.org' );*/ global $avelsieve_ldapuserdatamode; $avelsieve_ldapuserdatamode = false; /*global $avelsieve_cyrusadmins_map; $avelsieve_cyrusadmins_map = array( ??? 'cyrusimap' => 'cyrussieve' );*/ $conservative = false; $useimages = true; global $translate_return_msgs; $translate_return_msgs = false; $imagetheme = 'famfamfam'; //$imagetheme = 'bluecurve_24x24'; //$imagetheme = 'bluecurve_16x16'; global $startitems; $startitems = 3; global $maxitems; $maxitems = 10; global $headers; $headers = array( ?'From', 'To', 'Cc', 'Bcc', 'Subject', 'Reply-To', 'Sender', 'List-Id', ?'MailingList', 'Mailing-List', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', ?'Resent-From',? 'Resent-To', 'X-Mailer', 'X-Mailing-List', ?'X-Spam-Flag', 'X-Spam-Status', ?'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', ?'Return-Path', 'Received', 'Auto-Submitted', ?'X-Spam-Flag', 'X-Spam-Status','X-Spam-Tests' ?); global $notifymethods; $notifymethods = array( 'mailto', 'sms' ); //$notifymethods = false; // $disable_avelsieve_capabilities = array("notify"); global $disable_avelsieve_capabilities; $disable_avelsieve_capabilities = array(); global $avelsieveheaderlink; $avelsieveheaderlink = true; global $avelsieve_default_mode; $avelsieve_default_mode = 'terse'; global $avelsieve_enable_rules; $avelsieve_enable_rules = array(); global $avelsieve_rules_settings; $avelsieve_rules_settings = array(); foreach($avelsieve_enable_rules as $r) { ??? if(file_exists(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php')) { ??????? require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php'); ??? } else { ??????? require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.default.php'); ??? } } $avelsieve_spam_highlight_enable = false; Hope this help you to help me. Thank you ________________________________ From: Tamsy To: "dovecot at dovecot.org" Sent: Tuesday, 17 April 2012 1:25 PM Subject: Re: [Dovecot] vacation plugins for squirrelmail Show us your avelsieve-config in your-squirrelmail-dir/plugins/avelsieve/config/config.php Daminto Lie wrote the following on 17.04.2012 09:31: > Thank you folks for your replies. > > > I realised that the dovecot 1.2.9 that I installed via ubuntu synaptic package manager has already come with sieve plugins. After making some modifications on the dovecot.conf file, I went to download and install avelsieve 1.0.1 (stable version) onto the squirrelmail. Then, I run in squirrelmail conf.pl and added the plugin. I logged in and could see "Filter" option right on the top header row. I went to click it and nothing came up. I checked in the mail.log but did not see any errors there. > > Here is my snapshot of dovecot configuration > # 1.2.9: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS ext4 > log_timestamp: %Y-%m-%d %H:%M:%S > protocols: imap > listen: *:143 > ssl: >? no > disable_plaintext_auth: no > login_dir: /var/run/dovecot/login > login_executable: /usr/lib/dovecot/imap-login > verbose_proctitle: yes > first_valid_uid: 1001 > last_valid_uid: 600 > mail_privileged_group: mail > mail_location: maildir:/var/vmail/%u/Maildir > mbox_write_locks: fcntl dotlock > imap_client_workarounds: outlook-idle delay-newmail netscape-eoh tb-extra-mailbox-sep > lda: >? ? postmaster_address: postmaster at ourcompany.com >? ? mail_plugins: sieve >? ? quota_full_tempfail: yes >? ? deliver_log_format: msgid=%m: %$ >? ? sendmail_path: /usr/lib/sendmail >? ? rejection_reason: Your message to<%t>? was automatically rejected:%n%r >? ? auth_socket_path: /var/run/dovecot-auth-master > auth default: >? ? mechanisms: plain login >? ? username_format: %Lu >? ? verbose: yes >? ? debug: yes >? ? debug_passwords: yes >? ? passdb: >? ? ? driver: >? pam >? ? passdb: >? ? ? driver: ldap >? ? ? args: /etc/dovecot/dovecot-ldap.conf >? ? userdb: >? ? ? driver: passwd >? ? userdb: >? ? ? driver: static >? ? ? args: uid=1001 gid=1001 home=/var/vmail/%u allow_all_users=yes >? ? socket: >? ? ? type: listen >? ? ? client: >? ? ? ? path: /var/spool/postfix/private/auth >? ? ? ? mode: 432 >? ? ? ? user: postfix >? ? ? ? group: mail >? ? ? master: >? ? ? ? path: /var/run/dovecot-auth-master >? ? ? ? mode: 432 >? ? ? ? user: vmail >? ? ? ? group: vmail > plugin: >? ? sieve: /var/vmail/%u/.sieve > > Also, I have restarted dovecot after making changes to the dovecot.conf. > > Any help would be very much appreciated. > > Thank you > >? > > > ________________________________ >? From: Artur Zaprza?a > To: Dovecot Mailing List > Cc: Daminto Lie > Sent: Thursday, 12 April 2012 11:46 PM > Subject: Re: [Dovecot] vacation plugins for squirrelmail > > Daminto Lie wrote: >> Hi, >> >> I am afraid I >? have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. >> Any help would be very much appreciated. >> >> Thank you >> > I'm using Avelsieve 1.9.9 with a set of my own bugfixes: > http://email.uoa.gr/avelsieve/ > > From dovecot-list at mohtex.net Tue Apr 17 09:51:11 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Tue, 17 Apr 2012 13:51:11 +0700 Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <1334643297.87733.YahooMailNeo@web113417.mail.gq1.yahoo.com> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> <4F86DCA7.1060806@talex.pl> <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> <4F8CE2B8.3080801@mohtex.net> <1334643297.87733.YahooMailNeo@web113417.mail.gq1.yahoo.com> Message-ID: <4F8D12DF.9020508@mohtex.net> Hi Daminto, That looks pretty much unconfigured. No wonder nothing is coming up. I have attached the avelsieve config file we are using on one of our domains. Don't copy it blindly. At least you should configure $sieveport to the port you are using on your server for Sieve. And put your domain name where it reads YOURDOMAINNAMEHERE.com. For the arrays at $headers you should tune the filters you are using with your headers. Hope this helps (working fine with our domain and Squirrel-Mail). Rgds Tamsy Daminto Lie wrote the following on 17.04.2012 13:14: > Thank you for your reply. > > Here it is as follows > > > if(!defined('AVELSIEVE_DEBUG')) { > define('AVELSIEVE_DEBUG', 0); > } > > global $avelsieve_backend; > $avelsieve_backend = 'File'; > > /*global $sieveport; > $sieveport = 2000;*/ > > global $sieve_preferred_sasl_mech; > $sieve_preferred_sasl_mech = 'PLAIN'; > > global $avelsieve_disabletls; > $avelsieve_disabletls = false; > > global $avelsieve_file_backend_options, $data_dir, $username; > $avelsieve_file_backend_options = array( > 'avelsieve_default_file' => "$data_dir/$username.sievesource" > ); > > /*global $avelsieve_oldcyrus; > $avelsieve_oldcyrus = true;*/ > > global $avelsieve_enable_envelope_auth; > $avelsieve_enable_envelope_auth = true; > > global $avelsieve_custom_sieve_implementation; > $avelsieve_custom_sieve_implementation = ''; > > global $avelsieve_striproot; > $avelsieve_striproot = ''; > > global $avelsieve_hardcoded_capabilities; > $avelsieve_hardcoded_capabilities = array( > 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' > ); > > /*global $avelsieve_imapproxymode, $avelsieve_imapproxyserv; > $avelsieve_imapproxymode = false; > $avelsieve_imapproxyserv = array( > 'localhost' => 'imap.example.org' > );*/ > > global $avelsieve_ldapuserdatamode; > $avelsieve_ldapuserdatamode = false; > > /*global $avelsieve_cyrusadmins_map; > $avelsieve_cyrusadmins_map = array( > 'cyrusimap' => 'cyrussieve' > );*/ > > $conservative = false; > > $useimages = true; > > global $translate_return_msgs; > $translate_return_msgs = false; > > $imagetheme = 'famfamfam'; > //$imagetheme = 'bluecurve_24x24'; > //$imagetheme = 'bluecurve_16x16'; > > global $startitems; > $startitems = 3; > > global $maxitems; > $maxitems = 10; > > global $headers; > $headers = array( > 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Reply-To', 'Sender', 'List-Id', > 'MailingList', 'Mailing-List', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', > 'Resent-From', 'Resent-To', 'X-Mailer', 'X-Mailing-List', > 'X-Spam-Flag', 'X-Spam-Status', > 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', > 'Return-Path', 'Received', 'Auto-Submitted', > 'X-Spam-Flag', 'X-Spam-Status','X-Spam-Tests' > ); > > global $notifymethods; > $notifymethods = array( > 'mailto', 'sms' > ); > > //$notifymethods = false; > > // $disable_avelsieve_capabilities = array("notify"); > global $disable_avelsieve_capabilities; > $disable_avelsieve_capabilities = array(); > > global $avelsieveheaderlink; > $avelsieveheaderlink = true; > > global $avelsieve_default_mode; > $avelsieve_default_mode = 'terse'; > > global $avelsieve_enable_rules; > $avelsieve_enable_rules = array(); > > global $avelsieve_rules_settings; > $avelsieve_rules_settings = array(); > foreach($avelsieve_enable_rules as $r) { > if(file_exists(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php')) { > require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php'); > } else { > require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.default.php'); > } > } > > $avelsieve_spam_highlight_enable = false; > > > Hope this help you to help me. > > Thank you > > > > > ________________________________ > From: Tamsy > To: "dovecot at dovecot.org" > Sent: Tuesday, 17 April 2012 1:25 PM > Subject: Re: [Dovecot] vacation plugins for squirrelmail > > Show us your avelsieve-config in > your-squirrelmail-dir/plugins/avelsieve/config/config.php > > > Daminto Lie wrote the following on 17.04.2012 09:31: >> Thank you folks for your replies. >> >> >> I realised that the dovecot 1.2.9 that I installed via ubuntu synaptic package manager has already come with sieve plugins. After making some modifications on the dovecot.conf file, I went to download and install avelsieve 1.0.1 (stable version) onto the squirrelmail. Then, I run in squirrelmail conf.pl and added the plugin. I logged in and could see "Filter" option right on the top header row. I went to click it and nothing came up. I checked in the mail.log but did not see any errors there. >> >> Here is my snapshot of dovecot configuration >> # 1.2.9: /etc/dovecot/dovecot.conf >> # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS ext4 >> log_timestamp: %Y-%m-%d %H:%M:%S >> protocols: imap >> listen: *:143 >> ssl: >> no >> disable_plaintext_auth: no >> login_dir: /var/run/dovecot/login >> login_executable: /usr/lib/dovecot/imap-login >> verbose_proctitle: yes >> first_valid_uid: 1001 >> last_valid_uid: 600 >> mail_privileged_group: mail >> mail_location: maildir:/var/vmail/%u/Maildir >> mbox_write_locks: fcntl dotlock >> imap_client_workarounds: outlook-idle delay-newmail netscape-eoh tb-extra-mailbox-sep >> lda: >> postmaster_address: postmaster at ourcompany.com >> mail_plugins: sieve >> quota_full_tempfail: yes >> deliver_log_format: msgid=%m: %$ >> sendmail_path: /usr/lib/sendmail >> rejection_reason: Your message to<%t> was automatically rejected:%n%r >> auth_socket_path: /var/run/dovecot-auth-master >> auth default: >> mechanisms: plain login >> username_format: %Lu >> verbose: yes >> debug: yes >> debug_passwords: yes >> passdb: >> driver: >> pam >> passdb: >> driver: ldap >> args: /etc/dovecot/dovecot-ldap.conf >> userdb: >> driver: passwd >> userdb: >> driver: static >> args: uid=1001 gid=1001 home=/var/vmail/%u allow_all_users=yes >> socket: >> type: listen >> client: >> path: /var/spool/postfix/private/auth >> mode: 432 >> user: postfix >> group: mail >> master: >> path: /var/run/dovecot-auth-master >> mode: 432 >> user: vmail >> group: vmail >> plugin: >> sieve: /var/vmail/%u/.sieve >> >> Also, I have restarted dovecot after making changes to the dovecot.conf. >> >> Any help would be very much appreciated. >> >> Thank you >> >> >> >> >> ________________________________ >> From: Artur Zaprza?a >> To: Dovecot Mailing List >> Cc: Daminto Lie >> Sent: Thursday, 12 April 2012 11:46 PM >> Subject: Re: [Dovecot] vacation plugins for squirrelmail >> >> Daminto Lie wrote: >>> Hi, >>> >>> I am afraid I >> have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. >>> Any help would be very much appreciated. >>> >>> Thank you >>> >> I'm using Avelsieve 1.9.9 with a set of my own bugfixes: >> http://email.uoa.gr/avelsieve/ >> >> From dovecot-list at mohtex.net Tue Apr 17 10:00:23 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Tue, 17 Apr 2012 14:00:23 +0700 Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <1334643297.87733.YahooMailNeo@web113417.mail.gq1.yahoo.com> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> <4F86DCA7.1060806@talex.pl> <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> <4F8CE2B8.3080801@mohtex.net> <1334643297.87733.YahooMailNeo@web113417.mail.gq1.yahoo.com> Message-ID: <4F8D1507.7060609@mohtex.net> Seems not to be attached to my former mail. Here it comes again, inline: avelsieve config.php ############################### * @copyright 2002-2004 Alexandros Vellis * @package plugins * @subpackage avelsieve */ /** * Debug Mode. Enable this (change to 1) if you need to send a bug report, * or to 2 or 3 if you are a developer! * * Valid values: * 0 = No debugging output * 1 = Normal debugging output * 2 = Firebug-enhanced debugging output * 3 = Enhanced debugging output and no Sieve capabilities checks - enables * all UI features! */ if(!defined('AVELSIEVE_DEBUG')) { define('AVELSIEVE_DEBUG', 1); } /* ======================================================================== */ /* =================== IMAP Server / SIEVE Setup ========================= */ /* ======================================================================== */ /** @var string Backend to use. Available backends are: * 'ManageSieve': Uses the ManageSieve protocol. e.g. Cyrus * 'File': Writes files straight to disk. e.g. Exim4, Dovecot LDA. */ global $avelsieve_backend; $avelsieve_backend = 'ManageSieve'; /* ======================================================================== */ /* =================== ManageSieve Backend Options ======================== */ /* ======================================================================== */ /* Port where ManageSieve servers (e.g. Cyrus IMAP's timsieved) listens to. * Default is 2000. Note that another possible port that your server might * listen to, is 4190, which is now the official (IANA-assigned) port for * Sieve. */ global $sieveport; $sieveport = 4190; /** * @var string Space separated list of preferred SASL mechanisms for the * authentication to ManageSieve server. e.g. "PLAIN DIGEST-MD5";*/ global $sieve_preferred_sasl_mech; $sieve_preferred_sasl_mech = 'PLAIN'; /** * @var boolean Disable STARTTLS for ManageSieve. You can set this to true, * if you do not wish to use encryption via TLS mechanisms (i.e. the server * is not configured properly, or this is a local connection and TLS is not * needed. * Note that STARTTLS is supported only in PHP5+. In PHP4 this option will * have no effect and STARTTLS will be disabled anyway. */ global $avelsieve_disabletls; $avelsieve_disabletls = false; /* ======================================================================== */ /* ======================= File Backend Options =========================== */ /* ======================================================================== */ global $avelsieve_file_backend_options, $data_dir, $username; $avelsieve_file_backend_options = array( 'avelsieve_default_file' => "$data_dir/$username.sievesource" ); /* ======================================================================== */ /* ====== Implementation- and Server-Specific Options ==================== */ /* ======================================================================== */ /* In Cyrus 2.3+, the notification action is a bit more complex than the * others. The oldcyrus variable is for supporting the partially implemented * notify extension implementation of Cyrus < 2.3. If you have Cyrus < 2.3, * just set this to true. * * This only changes the informational / help text displayed in avelsieve. * * Cyrus < 2.3 : $from$, $env-from$, $subject$ * Cyrus 2.3+ : $from$, $env-from$, $subject$, $text$, $text[n]$ */ global $avelsieve_oldcyrus; $avelsieve_oldcyrus = true; /* If you have Cyrus with an lmtpd that can understand the "auth" argument to * the :envelope test as the SMTP/LMTP auth, or any other Sieve implementation, * then you can enable this to provide this functionality to the user. * * This was not clarified in the base spec of RFC 3028. It will be done * correctly in a new version of Cyrus, based on a new draft / spec. */ global $avelsieve_enable_envelope_auth; $avelsieve_enable_envelope_auth = true; /* Some Implementations of Sieve need certain things in order to operate * correctly. If you use any of the following server implementations, you MUST * set this variable to the corresponding value for the filtering to work * correctly. * Valid values are: * - Any RFC3028-mostly-compatible implementation: '' (empty). * - Exim MTA: 'exim' * - MFL (as supported by mvmf): 'mfl' */ global $avelsieve_custom_sieve_implementation; $avelsieve_custom_sieve_implementation = ''; /* For delivery agents that don't know how to handle some mailbox prefixes, you * can enable this option. Example for Dovecot LDA: 'INBOX.'; */ global $avelsieve_striproot; $avelsieve_striproot = 'INBOX.'; /* If the backend does not support capabilities reporting, such as the File * Backend, then you should define which capabilities are used by the server * implementation. * * The following are the capabilities supported by Exim4 as of Exim version * 4.60, according to README.SIEVE. You can change them if a new version of * Exim provides more functionality: * 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' * * The following are the capabilities that are suported by MFM, according to: * http://www.mvmf.org/mfl/language.shtml#sieve * 'envelope', 'fileinto', 'reject', 'relational', 'subaddress', 'regex', * 'editheader', 'copy', 'vacation', 'comparator-i;ascii-casemap', * 'comparator-i;octet' * * The following capabilities have been reported to work with Dovecot LDA: * 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric', * 'imapflags', 'subaddress','relational','regex' * */ /* global $avelsieve_hardcoded_capabilities; * $avelsieve_hardcoded_capabilities = array( * 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' * ); */ global $avelsieve_hardcoded_capabilities; $avelsieve_hardcoded_capabilities = array( 'fileinto', 'reject', 'envelope', 'encoded-character', 'vacation', 'subaddress', 'comparator-i;ascii-numeric', 'relational', 'regex', 'imap4flags', 'copy', 'include', 'variables', 'body', 'enotify', 'environment', 'mailbox', 'date' ); /** @var boolean Enable ImapProxy mode. * If you use imapproxy, because imapproxy cannot understand and proxy the * SIEVE protocol, you must connect to the SIEVE daemon (usually on the IMAP * server) itself. So you need to set $imapproxymode to true, and define a * mapping, from the imapproxy host (usually localhost) to your real IMAP * server (usually the same that is defined on Imapproxy's configuration). * * This will not work if you use a perdition-style proxy, where different users * go to different IMAP servers; it applies mostly to people running imapproxy * for speed and want a quick hack. */ global $avelsieve_imapproxymode, $avelsieve_imapproxyserv; $avelsieve_imapproxymode = false; $avelsieve_imapproxyserv = array( 'localhost' => 'YOURDOMAINNAMEHERE.com' ); /** @var boolean Ldapuserdata mode: Gets user's email addresses (including * mailAlternate & mailAuthorized) from LDAP Prefs Backend plugin's cache */ global $avelsieve_ldapuserdatamode; $avelsieve_ldapuserdatamode = false; /** @var array Map of cyrus administrator users, for proxy authentication */ global $avelsieve_cyrusadmins_map; $avelsieve_cyrusadmins_map = array( 'cyrusimap' => 'cyrussieve' ); /* ======================================================================== */ /* =============== Avelsieve Interface / Behavior Setup ================== */ /* ======================================================================== */ /* Be conservative to our updates on the SIEVE server? If true, a button * entitled "Save Changes" will appear, which will give the user the * functionality to register her changes. 'false' is recommended. */ $conservative = false; /* Use images for the move up / down, delete rule buttons and STOP? */ $useimages = true; /* Translate the messages returned by the "Reject" and "Vacation" actions? The * default behaviour since 0.9 is not to translate them. Change to true if in * an intranet environment or in a same-language environment. */ global $translate_return_msgs; $translate_return_msgs = false; /* Theme to use for the images. A directory with the same name must exist under * plugins/avelsieve/$imagetheme, that contains the files: up.png, down.png, * del.png, dup.png, edit.png, top.png, bottom.png. */ $imagetheme = 'famfamfam'; //$imagetheme = 'bluecurve_24x24'; //$imagetheme = 'bluecurve_16x16'; /* Number of items to display _initially_, when displaying the header match * rule */ global $startitems; $startitems = 3; /* Maximum number of items to allow in one header match rule. */ global $maxitems; $maxitems = 10; /* Headers to display in listbox widget, when adding a new header rule. */ /* global $headers; * $headers = array( * 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Reply-To', 'Sender', 'List-Id', * 'MailingList', 'Mailing-List', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', * 'Resent-From', 'Resent-To', 'X-Mailer', 'X-Mailing-List', * 'X-Spam-Flag', 'X-Spam-Status', * 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', * 'Return-Path', 'Received', 'Auto-Submitted', * 'X-Spam-Flag', 'X-Spam-Status','X-Spam-Tests' * ); */ global $headers; $headers = array( 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Sender', 'Reply-To', 'List-Id', 'MailingList', 'Mailing-List', 'X-DKIM-Authentication-Results', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', 'Resent-From', 'Resent-To', 'X-Mailer', 'X-Mailing-List', 'X-SpamTest-Status-Extended', 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', 'Return-Path', 'Received', 'Auto-Submitted', ); /* Available :method's for the :notify extension (if applicable) */ global $notifymethods; $notifymethods = array( 'mailto', 'sms' ); /* use the value "false" if you want to provide a simple input box so that * users can edit the method themselves : */ //$notifymethods = false; /* Capabilities to disable. If you would like to force avelsieve not to display * certain features, even though there _is_ a capability for them in Sieve * server, you should specify these here. For instance, if you would * like to disable the notify extension, even though ManageSieve advertises it, * you should add 'notify' in this array: $force_disable_avelsieve_capability = * array("notify");. This will still leave the defined feature on, and if the * user can upload her own scripts then she can use that feature; this option * just disables the GUI of it. Leave as-is (empty array) if you do not need * that. * * Look in $implemented_capabilities array in include/constants.inc.php for * valid values */ // $disable_avelsieve_capabilities = array("notify"); global $disable_avelsieve_capabilities; $disable_avelsieve_capabilities = array(); /* Display Filters link in the top Squirrelmail header? */ global $avelsieveheaderlink; $avelsieveheaderlink = true; /* Default rules table display mode, one of 'verbose' or 'terse' */ global $avelsieve_default_mode; $avelsieve_default_mode = 'terse'; /* ======================================================================== */ /* ========================= Custom rules Configuration =================== */ /* ======================================================================== */ /** * @var array * List of additional rules to enable. This must be the numeric id of the * rule. * * 10: Spam Rule (as existed in Avelsieve) * 11: Junk-Mail Rule (new-style SPAM Rule, only one exists per Sieve script) * 12: Global Whitelist (only one exists per Sieve script) * 13: Custom Sieve Code (allows users to enter whatever Sieve code they like - * Alpha/Experimental) * * Example: * array(11, 12); */ global $avelsieve_enable_rules; $avelsieve_enable_rules = array(); global $avelsieve_rules_settings; $avelsieve_rules_settings = array(); foreach($avelsieve_enable_rules as $r) { if(file_exists(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php')) { require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php'); } else { require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.default.php'); } } /* Please keep the following setting false; it is alpha + needs Squirrelmail * to be patched in three or four places. */ $avelsieve_spam_highlight_enable = false; ############################### Hi Daminto, That looks pretty much unconfigured. No wonder nothing is coming up. I have attached the avelsieve config file we are using on one of our domains. Don't copy it blindly. At least you should configure $sieveport to the port you are using on your server for Sieve. And put your domain name where it reads YOURDOMAINNAMEHERE.com. For the arrays at $headers you should tune the filters you are using with your headers. Hope this helps (working fine with our domain and Squirrel-Mail). Rgds Tamsy Daminto Lie wrote the following on 17.04.2012 13:14: > Thank you for your reply. > > Here it is as follows > > > if(!defined('AVELSIEVE_DEBUG')) { > define('AVELSIEVE_DEBUG', 0); > } > > global $avelsieve_backend; > $avelsieve_backend = 'File'; > > /*global $sieveport; > $sieveport = 2000;*/ > > global $sieve_preferred_sasl_mech; > $sieve_preferred_sasl_mech = 'PLAIN'; > > global $avelsieve_disabletls; > $avelsieve_disabletls = false; > > global $avelsieve_file_backend_options, $data_dir, $username; > $avelsieve_file_backend_options = array( > 'avelsieve_default_file' => "$data_dir/$username.sievesource" > ); > > /*global $avelsieve_oldcyrus; > $avelsieve_oldcyrus = true;*/ > > global $avelsieve_enable_envelope_auth; > $avelsieve_enable_envelope_auth = true; > > global $avelsieve_custom_sieve_implementation; > $avelsieve_custom_sieve_implementation = ''; > > global $avelsieve_striproot; > $avelsieve_striproot = ''; > > global $avelsieve_hardcoded_capabilities; > $avelsieve_hardcoded_capabilities = array( > 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' > ); > > /*global $avelsieve_imapproxymode, $avelsieve_imapproxyserv; > $avelsieve_imapproxymode = false; > $avelsieve_imapproxyserv = array( > 'localhost' => 'imap.example.org' > );*/ > > global $avelsieve_ldapuserdatamode; > $avelsieve_ldapuserdatamode = false; > > /*global $avelsieve_cyrusadmins_map; > $avelsieve_cyrusadmins_map = array( > 'cyrusimap' => 'cyrussieve' > );*/ > > $conservative = false; > > $useimages = true; > > global $translate_return_msgs; > $translate_return_msgs = false; > > $imagetheme = 'famfamfam'; > //$imagetheme = 'bluecurve_24x24'; > //$imagetheme = 'bluecurve_16x16'; > > global $startitems; > $startitems = 3; > > global $maxitems; > $maxitems = 10; > > global $headers; > $headers = array( > 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Reply-To', 'Sender', 'List-Id', > 'MailingList', 'Mailing-List', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', > 'Resent-From', 'Resent-To', 'X-Mailer', 'X-Mailing-List', > 'X-Spam-Flag', 'X-Spam-Status', > 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', > 'Return-Path', 'Received', 'Auto-Submitted', > 'X-Spam-Flag', 'X-Spam-Status','X-Spam-Tests' > ); > > global $notifymethods; > $notifymethods = array( > 'mailto', 'sms' > ); > > //$notifymethods = false; > > // $disable_avelsieve_capabilities = array("notify"); > global $disable_avelsieve_capabilities; > $disable_avelsieve_capabilities = array(); > > global $avelsieveheaderlink; > $avelsieveheaderlink = true; > > global $avelsieve_default_mode; > $avelsieve_default_mode = 'terse'; > > global $avelsieve_enable_rules; > $avelsieve_enable_rules = array(); > > global $avelsieve_rules_settings; > $avelsieve_rules_settings = array(); > foreach($avelsieve_enable_rules as $r) { > if(file_exists(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php')) { > require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php'); > } else { > require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.default.php'); > } > } > > $avelsieve_spam_highlight_enable = false; > > > Hope this help you to help me. > > Thank you > > > > > ________________________________ > From: Tamsy > To: "dovecot at dovecot.org" > Sent: Tuesday, 17 April 2012 1:25 PM > Subject: Re: [Dovecot] vacation plugins for squirrelmail > > Show us your avelsieve-config in > your-squirrelmail-dir/plugins/avelsieve/config/config.php > > > Daminto Lie wrote the following on 17.04.2012 09:31: >> Thank you folks for your replies. >> >> >> I realised that the dovecot 1.2.9 that I installed via ubuntu synaptic package manager has already come with sieve plugins. After making some modifications on the dovecot.conf file, I went to download and install avelsieve 1.0.1 (stable version) onto the squirrelmail. Then, I run in squirrelmail conf.pl and added the plugin. I logged in and could see "Filter" option right on the top header row. I went to click it and nothing came up. I checked in the mail.log but did not see any errors there. >> >> Here is my snapshot of dovecot configuration >> # 1.2.9: /etc/dovecot/dovecot.conf >> # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS ext4 >> log_timestamp: %Y-%m-%d %H:%M:%S >> protocols: imap >> listen: *:143 >> ssl: >> no >> disable_plaintext_auth: no >> login_dir: /var/run/dovecot/login >> login_executable: /usr/lib/dovecot/imap-login >> verbose_proctitle: yes >> first_valid_uid: 1001 >> last_valid_uid: 600 >> mail_privileged_group: mail >> mail_location: maildir:/var/vmail/%u/Maildir >> mbox_write_locks: fcntl dotlock >> imap_client_workarounds: outlook-idle delay-newmail netscape-eoh tb-extra-mailbox-sep >> lda: >> postmaster_address: postmaster at ourcompany.com >> mail_plugins: sieve >> quota_full_tempfail: yes >> deliver_log_format: msgid=%m: %$ >> sendmail_path: /usr/lib/sendmail >> rejection_reason: Your message to<%t> was automatically rejected:%n%r >> auth_socket_path: /var/run/dovecot-auth-master >> auth default: >> mechanisms: plain login >> username_format: %Lu >> verbose: yes >> debug: yes >> debug_passwords: yes >> passdb: >> driver: >> pam >> passdb: >> driver: ldap >> args: /etc/dovecot/dovecot-ldap.conf >> userdb: >> driver: passwd >> userdb: >> driver: static >> args: uid=1001 gid=1001 home=/var/vmail/%u allow_all_users=yes >> socket: >> type: listen >> client: >> path: /var/spool/postfix/private/auth >> mode: 432 >> user: postfix >> group: mail >> master: >> path: /var/run/dovecot-auth-master >> mode: 432 >> user: vmail >> group: vmail >> plugin: >> sieve: /var/vmail/%u/.sieve >> >> Also, I have restarted dovecot after making changes to the dovecot.conf. >> >> Any help would be very much appreciated. >> >> Thank you >> >> >> >> >> ________________________________ >> From: Artur Zaprza?a >> To: Dovecot Mailing List >> Cc: Daminto Lie >> Sent: Thursday, 12 April 2012 11:46 PM >> Subject: Re: [Dovecot] vacation plugins for squirrelmail >> >> Daminto Lie wrote: >>> Hi, >>> >>> I am afraid I >> have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. >>> Any help would be very much appreciated. >>> >>> Thank you >>> >> I'm using Avelsieve 1.9.9 with a set of my own bugfixes: >> http://email.uoa.gr/avelsieve/ >> >> From f.bonnet at esiee.fr Tue Apr 17 10:54:35 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Tue, 17 Apr 2012 09:54:35 +0200 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) Message-ID: <4F8D21BB.2030407@esiee.fr> Hello I need some feedbacks advices of experienced admins I will have to setup in few monthes an email system for approx 50K "intensives" users. The only mandatory thing will be I must use HP proliant servers The operating system will be FreeBSD or Linux Thank you for any advices From dchenusa at yahoo.com Tue Apr 17 12:16:22 2012 From: dchenusa at yahoo.com (D Chen) Date: Tue, 17 Apr 2012 02:16:22 -0700 (PDT) Subject: [Dovecot] Dovecot and Outlook Express "plaintext authentication not allowed..." Message-ID: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> Server: Ubuntu 11.10 server with postfix (MTA), dovecot (MDA) Mail client: Outlook Express in Windows XP with IMAP on port 143, SMTP on port 25 When tried to set up a new Outlook Express mail acct, the following error encountered: ? ? ?Your IMAP server wishes to alert you to the following: ? ? ?plaintext authentication not allowed without SSL/TLS, but your client did it anyway. ?If ? ? ?anyone was listening, the password was exposed. When asked "would you like to download folders from the mail server you added?" then got the following error: The server has rejected your login. ?Please verify that your user name and password are correct. I re-entered several time with the correct username and password, but still responded the same error. Any insights would be greatly appreciated. Here is the message from the /var/log/mail.log: Apr 17 01:54:17 server dovecot: imap-login: Disconnected (tried to use disabled plaintext auth): rip= 192.168.20.51, lip=192.168.20.100Apr 17 01:54:28 server dovecot: imap-login: Disconnected (tried to use disabled plaintext auth): rip= 192.168.20.51, lip=192.168.20.100 What might be wrong in my dovecot.conf file. ?Below is my dovecot.conf file: # 2.0.13: /etc/dovecot/dovecot.conf # OS: Linux 3.0.0-16-server x86_64 Ubuntu 11.10? log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:~/Maildir mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date passdb { ? driver = pam } passdb { ? driver = pam } plugin { ? sieve = ~/.dovecot.sieve ? sieve_dir = ~/sieve } protocols = imap pop3 sieve service auth { ? unix_listener /var/spool/postfix/private/auth-client { ? ? group = postfix ? ? mode = 0660 ? ? user = postfix ? } ? unix_listener /var/spool/postfix/private/dovecot-auth { ? ? group = postfix ? ? mode = 0660 ? ? user = postfix ? } ? user = root } ssl_cert = was automatically rejected:%n%r } From CMarcus at Media-Brokers.com Tue Apr 17 13:26:37 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 17 Apr 2012 06:26:37 -0400 Subject: [Dovecot] Dovecot and Outlook Express "plaintext authentication not allowed..." In-Reply-To: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> References: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> Message-ID: <4F8D455D.9060805@Media-Brokers.com> On 2012-04-17 5:16 AM, D Chen wrote: > When tried to set up a new Outlook Express mail acct, the following error encountered: > > Your IMAP server wishes to alert you to the following: > plaintext authentication not allowed without SSL/TLS, but your client did it anyway. Ummm... hello? McFly? ;) You need to configure SSL on Outlook Express (why on gods green earth you are using that is beyond me), as the error plainly says. Until you do this it doesn't matter what you enter for a username/password. Also, you should be using SSL on port 993, because I'm fairly certain that Outlook Express doesn't support STARTTLS on port 143. And off topic but in the same vein, for postfix/SMTP relay you should be using the submission port (STARTTLS on port 587) for most clients, but I think Outlook Express again doesn't work with it, so you will need to use the deprecated SSL on port 465 (these two need to be enabled in postfix/master.cf - just uncomment the examples). -- Best regards, Charles From blevi.linux at gmail.com Tue Apr 17 13:47:26 2012 From: blevi.linux at gmail.com (Birta Levente) Date: Tue, 17 Apr 2012 13:47:26 +0300 Subject: [Dovecot] Dovecot and Outlook Express "plaintext authentication not allowed..." In-Reply-To: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> References: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> Message-ID: <4F8D4A3E.8000401@gmail.com> On 17/04/2012 12:16, D Chen wrote: > Server: Ubuntu 11.10 server with postfix (MTA), dovecot (MDA) > Mail client: Outlook Express in Windows XP with IMAP on port 143, SMTP on port 25 > > When tried to set up a new Outlook Express mail acct, the following error encountered: > > Your IMAP server wishes to alert you to the following: > plaintext authentication not allowed without SSL/TLS, but your client did it anyway. If > anyone was listening, the password was exposed. > > When asked "would you like to download folders from the mail server you added?" then got the > following error: > > The server has rejected your login. Please verify that your user name and password are correct. > > I re-entered several time with the correct username and password, but still responded the same error. > > Any insights would be greatly appreciated. > > > Here is the message from the /var/log/mail.log: > > > Apr 17 01:54:17 server dovecot: imap-login: Disconnected (tried to use disabled plaintext auth): rip= > 192.168.20.51, lip=192.168.20.100Apr 17 01:54:28 server dovecot: imap-login: Disconnected (tried to use disabled plaintext auth): rip= > 192.168.20.51, lip=192.168.20.100 > > What might be wrong in my dovecot.conf file. Below is my dovecot.conf file: > > # 2.0.13: /etc/dovecot/dovecot.conf > # OS: Linux 3.0.0-16-server x86_64 Ubuntu 11.10 > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_location = maildir:~/Maildir > mail_privileged_group = mail > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date > passdb { > driver = pam > } > passdb { > driver = pam > } > plugin { > sieve = ~/.dovecot.sieve > sieve_dir = ~/sieve > } > protocols = imap pop3 sieve > service auth { > unix_listener /var/spool/postfix/private/auth-client { > group = postfix > mode = 0660 > user = postfix > } > unix_listener /var/spool/postfix/private/dovecot-auth { > group = postfix > mode = 0660 > user = postfix > } > user = root > } > ssl_cert = ssl_cipher_list = ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM > ssl_key = userdb { > driver = passwd > } > userdb { > driver = passwd > } > protocol imap { > imap_client_workarounds = delay-newmail > mail_max_userip_connections = 10 > } > protocol pop3 { > mail_max_userip_connections = 10 > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > pop3_uidl_format = %08Xu%08Xv > } > protocol lda { > deliver_log_format = msgid=%m: %$ > mail_plugins = sieve > postmaster_address = postmaster > quota_full_tempfail = yes > rejection_reason = Your message to<%t> was automatically rejected:%n%r > } > Maybe auth_mechanisms = plain login cram-md5 From robert at schetterer.org Tue Apr 17 14:47:57 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 17 Apr 2012 13:47:57 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken Message-ID: <4F8D586D.3010706@schetterer.org> Hi Tim, after upgrade to 2.1 acl with shared namespace seems to be broken it looks like depend to No lookup right to mailbox: shared/ it runs perfect under 2.0.20 -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From mlopez at gtdinternet.com Tue Apr 17 14:54:15 2012 From: mlopez at gtdinternet.com (=?ISO-8859-1?Q?Mauricio_L=F3pez_Riffo?=) Date: Tue, 17 Apr 2012 08:54:15 -0300 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8D21BB.2030407@esiee.fr> References: <4F8D21BB.2030407@esiee.fr> Message-ID: <4F8D59E7.6010403@gtdinternet.com> Frank, Here we have approx. 200K users with 4000 concurrent connections (90% POP3 users) All servers in virtual environment Vmware, supermicro servers and Netapp Metrocluster storage solutions (nfs storage with 10G ethernet network) POP3 sessions take betwen 40 and 300 milisecons at connect, auth and list. All accounts lives in LDAP, CentOS 5 and exim like a mta relay. Regards El 17-04-2012 4:54, Frank Bonnet escribi?: > Hello > > I need some feedbacks advices of experienced admins > I will have to setup in few monthes an email system > for approx 50K "intensives" users. > > The only mandatory thing will be I must use HP proliant servers > > The operating system will be FreeBSD or Linux > > Thank you for any advices > > -- Mauricio L?pez Riffo Red Hat Certified Engineer 804006455319519 Administrador de Servicios Internet ?rea Ingenier?a Gtd Internet S.A. http://www.grupogtd.com/ Moneda 920, Oficina 602 - Fono : +562 4139742 From janfrode at tanso.net Tue Apr 17 15:42:41 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 17 Apr 2012 14:42:41 +0200 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8D59E7.6010403@gtdinternet.com> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> Message-ID: <20120417124241.GA23120@dibs.tanso.net> On Tue, Apr 17, 2012 at 08:54:15AM -0300, Mauricio L?pez Riffo wrote: > > Here we have approx. 200K users with 4000 concurrent connections > (90% POP3 users) How do you measure "concurrent" POP3 users? > All servers in virtual environment Vmware, > supermicro servers and Netapp Metrocluster storage solutions (nfs > storage with 10G ethernet network) POP3 sessions take betwen 40 and > 300 milisecons at connect, auth and list. All accounts lives in > LDAP, CentOS 5 and exim like a mta relay. Very interesting config. We're close to 1M accounts, GPFS cluster fs, LDAP, RHEL5/6 and postfix + dovecot director for pop/imap/lmtp, and moving from maildir to mdbox. What mailbox-format are you using? Do you have a director, or accounts sticky to a server some other way? How's the NFS performance? I've always bean weary that NFS works terribly with many small files (i.e. maildir).. What does the metrocluster give you? Is it for disaster recovery on second location, or do you have two active locations working against the same filesystem? -jf From f.bonnet at esiee.fr Tue Apr 17 16:01:31 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Tue, 17 Apr 2012 15:01:31 +0200 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8D59E7.6010403@gtdinternet.com> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> Message-ID: <4F8D69AB.5090706@esiee.fr> hello Thanks for your answer , MY problem will be IMAPS connections I dunno how much I will have but it would be possible that we'll have 4000/6000 imaps concurent connections during working hours . POP3 users will be very few Le 17/04/2012 13:54, Mauricio L?pez Riffo a ?crit : > Frank, > > Here we have approx. 200K users with 4000 concurrent connections > (90% POP3 users) All servers in virtual environment Vmware, > supermicro servers and Netapp Metrocluster storage solutions (nfs > storage with 10G ethernet network) POP3 sessions take betwen 40 and > 300 milisecons at connect, auth and list. All accounts lives in LDAP, > CentOS 5 and exim like a mta relay. > > Regards > > > > El 17-04-2012 4:54, Frank Bonnet escribi?: >> Hello >> >> I need some feedbacks advices of experienced admins >> I will have to setup in few monthes an email system >> for approx 50K "intensives" users. >> >> The only mandatory thing will be I must use HP proliant servers >> >> The operating system will be FreeBSD or Linux >> >> Thank you for any advices >> >> > From mlopez at gtdinternet.com Tue Apr 17 16:10:02 2012 From: mlopez at gtdinternet.com (=?ISO-8859-1?Q?Mauricio_L=F3pez_Riffo?=) Date: Tue, 17 Apr 2012 10:10:02 -0300 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <20120417124241.GA23120@dibs.tanso.net> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <20120417124241.GA23120@dibs.tanso.net> Message-ID: <4F8D6BAA.7060501@gtdinternet.com> Jan, >> How do you measure "concurrent" POP3 users? We use cacti for metrics like concurrent connections or pop3 delay and Zabbix for alarms. >> What mailbox-format are you using? Do you have a director, or accounts >> sticky to a server some other way? Maildir like a mailformat and currently without a director, but we testing a new environment with director to reduce number of servers (7 virtual servers with 4 Vcpu, 6gb ram) At the meantime a lvs piranha do the work of steaky connections but its not enought, for this reason will use a director. >> How's the NFS performance? I've always bean weary that NFS works terribly with many small files (i.e. maildir).. At peak hours our storage traffic about 10 thousend total ops (bewten read and write, 80/20) and cpu mark 70% of use. >> What does the metrocluster give you? Is it for disaster recovery on >> second location, or do you have two active locations working against the >>same filesystem? Our storage have two missions, first hight availability because we have two datacenter (environment N +1) and posibility of use two storages in cluster for the same filesystems en this two datacenters (Vmware and Mail storage resides in NFS filesystem) >> Very interesting config. We're close to 1M accounts, GPFS cluster fs, LDAP, >> RHEL5/6 and postfix + dovecot director for pop/imap/lmtp, and moving from >> maildir to mdbox. 1M = 1 milion ? How many servers you have? hardware? Any help o contribution, you welcome :) Regards El 17-04-2012 9:42, Jan-Frode Myklebust escribi?: > On Tue, Apr 17, 2012 at 08:54:15AM -0300, Mauricio L?pez Riffo wrote: >> Here we have approx. 200K users with 4000 concurrent connections >> (90% POP3 users) > How do you measure "concurrent" POP3 users? > >> All servers in virtual environment Vmware, >> supermicro servers and Netapp Metrocluster storage solutions (nfs >> storage with 10G ethernet network) POP3 sessions take betwen 40 and >> 300 milisecons at connect, auth and list. All accounts lives in >> LDAP, CentOS 5 and exim like a mta relay. > Very interesting config. We're close to 1M accounts, GPFS cluster fs, LDAP, > RHEL5/6 and postfix + dovecot director for pop/imap/lmtp, and moving from > maildir to mdbox. > > What mailbox-format are you using? Do you have a director, or accounts > sticky to a server some other way? > > How's the NFS performance? I've always bean weary that NFS works > terribly with many small files (i.e. maildir).. > > What does the metrocluster give you? Is it for disaster recovery on > second location, or do you have two active locations working against the > same filesystem? > > > -jf -- Mauricio L?pez Riffo Red Hat Certified Engineer 804006455319519 Administrador de Servicios Internet ?rea Ingenier?a Gtd Internet S.A. http://www.grupogtd.com/ Moneda 920, Oficina 602 - Fono : +562 4139742 From goetz.reinicke at filmakademie.de Tue Apr 17 17:21:39 2012 From: goetz.reinicke at filmakademie.de (=?ISO-8859-15?Q?G=F6tz_Reinicke?=) Date: Tue, 17 Apr 2012 16:21:39 +0200 Subject: [Dovecot] 487 imap-login processes - is this ok Message-ID: <4F8D7C73.9090700@filmakademie.de> Hi, recently I checked our mailserver for some more tuning and optimizing and noticed, that currently I do have 487 imap-login processes and I'm asking myself if that's o.k. We do have about 1.000 user in total and not everybody is currently logged in. May be this is o.k. but if not, what might I check? We run the dovecot package provided by redhat EL 5.8 dovecot-1.0.7-7.el5_7.1 Thanks for any comment or suggestion. Regards . G?tz -- G?tz Reinicke IT-Koordinator Tel. +49 7141 969 420 Fax +49 7141 969 55 420 E-Mail goetz.reinicke at filmakademie.de Filmakademie Baden-W?rttemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: J?rgen Walter MdL Staatssekret?r im Ministerium f?r Wissenschaft, Forschung und Kunst Baden-W?rttemberg Gesch?ftsf?hrer: Prof. Thomas Schadt -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5161 bytes Desc: S/MIME Kryptografische Unterschrift URL: From robert at schetterer.org Tue Apr 17 17:40:19 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 17 Apr 2012 16:40:19 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken In-Reply-To: <4F8D586D.3010706@schetterer.org> References: <4F8D586D.3010706@schetterer.org> Message-ID: <4F8D80D3.2050505@schetterer.org> Am 17.04.2012 13:47, schrieb Robert Schetterer: > Hi Tim, i should get some "o" > after upgrade to 2.1 > acl with shared namespace seems to be broken > it looks like depend to > > No lookup right to mailbox: shared/ > > it runs perfect under 2.0.20 looks its depended to bug handling doveadm acl debug shows on both versions Fatal: ACL shared dict iteration failed but dove 2.1 does handle this in another way means, results in no rights , and no list for that -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From jd.beaubien at gmail.com Tue Apr 17 19:19:49 2012 From: jd.beaubien at gmail.com (Jean-Daniel Beaubien) Date: Tue, 17 Apr 2012 12:19:49 -0400 Subject: [Dovecot] Howto reindex with solr? Message-ID: Hi, I am currently testing a 2.1 dovecot setup with fts-solr. Search speed is simply amazing. I decided to try to delete and rebuild the solr indexes, but i can't seem to rebuild them thru dovecot. At first, when telnetting to the imap server, the search initially built the index, and then subsequent searches are blazingly fast. 1- I deleted all the solr indexes with the following query: *:* 2- Flag the mailbox to be re-indexed: doveadm force-resync INBOX At this point, it doesn't work. When I telnet into the imap server, the searches always come back empty 3- I also tried a few other commands: doveadm fts rescan, doveadm index INBOX. Still no search results. Nothing seems to be appearing in the logs when I issue all the doveadm commands. What did I miss? Thanks, -JD From cor at xs4all.nl Tue Apr 17 20:57:01 2012 From: cor at xs4all.nl (Cor Bosman) Date: Tue, 17 Apr 2012 19:57:01 +0200 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8D59E7.6010403@gtdinternet.com> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> Message-ID: <20120417175701.GA28791@xs4all.nl> > Here we have approx. 200K users with 4000 concurrent connections > (90% POP3 users) All servers in virtual environment Vmware, supermicro > servers and Netapp Metrocluster storage solutions (nfs storage with 10G > ethernet network) POP3 sessions take betwen 40 and 300 milisecons at > connect, auth and list. All accounts lives in LDAP, CentOS 5 and exim > like a mta relay. Similar setup here. Maybe 15-20K concurrent connections, imap only (pop is not handled by dovecot yet) about 800K mailboxes. We have all bare metal servers, linux. Currently 35 servers handling the load easily. We could run on 1/3rd of the load probably. In front of the 35 servers are 3 directors, handling imap only, although im in the process of adding LMTP/sieve to the mix. Backend storage is NetApp Metrocluster over 2 datacenters. Cor From joe.beaubien at gmail.com Tue Apr 17 21:36:33 2012 From: joe.beaubien at gmail.com (Joe Beaubien) Date: Tue, 17 Apr 2012 14:36:33 -0400 Subject: [Dovecot] Re-indexing of fts-solr not working Message-ID: Hi, I am currently testing a 2.1 dovecot setup with fts-solr. Search speed is simply amazing. I decided to try to delete and rebuild the solr indexes, but i can't seem to rebuild them thru dovecot. At first, when telnetting to the imap server, the search initially built the index, and then subsequent searches are blazingly fast. 1- I deleted all the solr indexes with the following query: *:* 2- Flag the mailbox to be re-indexed: doveadm force-resync INBOX At this point, it doesn't work. When I telnet into the imap server, the searches always come back empty 3- I also tried a few other commands: doveadm fts rescan, doveadm index INBOX. Still no search results. Nothing seems to be appearing in the logs when I issue all the doveadm commands. What did I miss? Thanks, Joe From janfrode at tanso.net Tue Apr 17 23:08:20 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 17 Apr 2012 22:08:20 +0200 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8D6BAA.7060501@gtdinternet.com> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <20120417124241.GA23120@dibs.tanso.net> <4F8D6BAA.7060501@gtdinternet.com> Message-ID: <20120417200820.GA30743@dibs.tanso.net> On Tue, Apr 17, 2012 at 10:10:02AM -0300, Mauricio L?pez Riffo wrote: > > 1M = 1 milion ? 976508 to be exact :-) but it's very much a useless number. Lots and lots of these are inactive. A better number is probably that we're seeing about 80 logins/second for the last hour.. (just checked now, not sure if this is the most busy hour or not). > How many servers you have? hardware? 7 backend dovecot servers (two IBM x336, three x346 and two x3550, with a 8 GB for the x336/x346 and 16GB memory memory for the x3550's). 2 frontend dovecot directors (IBM x3550). None of these are really very busy, so we could probably reduce the number of backends a bit if we wanted. Our struggle is the number of iops we're able to get from the backend storage (IBM DS4800), mostly a problem when we have storms of incoming marketing messages in addition to the pop/imap traffic. -jf From dchenusa at yahoo.com Tue Apr 17 23:41:20 2012 From: dchenusa at yahoo.com (D Chen) Date: Tue, 17 Apr 2012 13:41:20 -0700 (PDT) Subject: [Dovecot] Dovecot and Outlook Express "plaintext authentication not allowed..." In-Reply-To: <4F8D455D.9060805@Media-Brokers.com> References: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> <4F8D455D.9060805@Media-Brokers.com> Message-ID: <1334695280.2248.YahooMailNeo@web161606.mail.bf1.yahoo.com> Hi Charles, Thanks for the response. Per your suggestion, in Outlook Express I changed to use SSL on port 993 for IMAP, and port 465 for SMTP,?the creation of the new account succeeded without error! ?However, when tried to send to my yahoo mail or gmail acct, ?got the following error: (apparently failed to send!) "The message could not be sent because one of the recipients was rejected by the server. ?The rejected email address was 'dchenusa at yahoo.com.' ?Subject: 'test out to yahoo mail' Account:'dchen192.168.20.100' Server: 192.168.20.100 Protocol: SMTP, server response: 554 5.7.1 : client host rejected access denied, port: 465, Secured [SSL]: Yes, server Error: 554, Error number: 0x800CCC79" PS. in the same client PC, using Thunderbird client, IMAP and SMTP configured with either with SSL or without SSL, both outgoing mails were sent successfully! Any thought? Thx again, Dennis ________________________________ From: Charles Marcus To: dovecot at dovecot.org Sent: Tuesday, April 17, 2012 3:26 AM Subject: Re: [Dovecot] Dovecot and Outlook Express "plaintext authentication not allowed..." On 2012-04-17 5:16 AM, D Chen wrote: > When tried to set up a new Outlook Express mail acct, the following error encountered: > >? ? ? Your IMAP server wishes to alert you to the following: >? ? ? plaintext authentication not allowed without SSL/TLS, but your client did it anyway. Ummm... hello? McFly? ;) You need to configure SSL on Outlook Express (why on gods green earth you are using that is beyond me), as the error plainly says. Until you do this it doesn't matter what you enter for a username/password. Also, you should be using SSL on port 993, because I'm fairly certain that Outlook Express doesn't support STARTTLS on port 143. And off topic but in the same vein, for postfix/SMTP relay you should be using the submission port (STARTTLS on port 587) for most clients, but I think Outlook Express again doesn't work with it, so you will need to use the deprecated SSL on port 465 (these two need to be enabled in postfix/master.cf - just uncomment the examples). -- Best regards, Charles From jerry at seibercom.net Wed Apr 18 00:27:18 2012 From: jerry at seibercom.net (Jerry) Date: Tue, 17 Apr 2012 17:27:18 -0400 Subject: [Dovecot] Dovecot and Outlook Express "plaintext authentication not allowed..." In-Reply-To: <1334695280.2248.YahooMailNeo@web161606.mail.bf1.yahoo.com> References: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> <4F8D455D.9060805@Media-Brokers.com> <1334695280.2248.YahooMailNeo@web161606.mail.bf1.yahoo.com> Message-ID: <20120417172718.3af8127d@scorpio> On Tue, 17 Apr 2012 13:41:20 -0700 (PDT) D Chen articulated: > Per your suggestion, in Outlook Express I changed to use SSL on port > 993 for IMAP, and port 465 for SMTP,?the creation of the new account > succeeded without error! ?However, when tried to send to my yahoo > mail or gmail acct, ?got the following error: (apparently failed to > send!) > > "The message could not be sent because one of the recipients was > rejected by the server. ?The rejected email address was > 'dchenusa at yahoo.com.' ?Subject: 'test out to yahoo mail' > Account:'dchen192.168.20.100' Server: 192.168.20.100 > Protocol: SMTP, server response: 554 5.7.1 > : client host rejected access denied, port: > 465, Secured [SSL]: Yes, server Error: 554, Error number: 0x800CCC79" > > PS. in the same client PC, using Thunderbird client, IMAP and SMTP > configured with either with SSL or without SSL, both outgoing mails > were sent successfully! > > Any thought? Well, my first thought is that you should lost the tendency to "Top Post". Second, why are you screwing around with the SMTP settings? Dovecot could not care less about them. Third, Outlook Express has been abandoned years ago. Why are you still using it? If you really need it, try Windows Live Essentials . I know users who have it working with Dovecot just fine. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From kgc at corp.sonic.net Wed Apr 18 01:29:17 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Tue, 17 Apr 2012 15:29:17 -0700 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> Message-ID: <4F8DEEBD.9080206@corp.sonic.net> On 03/30/12 07:35, Timo Sirainen wrote: > IP stickiness isn't enough if user uses more than one IMAP client, which is pretty common nowadays. And doesn't help at all with LDA. Timo, having the LDA flow through the director with maildir and NFS is only an issue if you want to use the dovecot delivery agent since it manipulates the indexes on delivery, right? This shouldn't be a problem if you are running a completely distinct delivery cluster running a LDA like procmail. -K From noel.butler at ausics.net Wed Apr 18 05:24:29 2012 From: noel.butler at ausics.net (Noel Butler) Date: Wed, 18 Apr 2012 12:24:29 +1000 Subject: [Dovecot] 487 imap-login processes - is this ok In-Reply-To: <4F8D7C73.9090700@filmakademie.de> References: <4F8D7C73.9090700@filmakademie.de> Message-ID: <1334715869.4078.3.camel@tardis> Older versions of dovecot (all of v1) can under some circumstances screwup and leave hung login processes, this more often than not occurs if you have network issues, requiring a restart of dovecot to clear them. Timo has apparently, IIRC, worked around this for v2.x and it should not happen. On Tue, 2012-04-17 at 16:21 +0200, G?tz Reinicke wrote: > Hi, > > recently I checked our mailserver for some more tuning and optimizing > and noticed, that currently I do have 487 imap-login processes and I'm > asking myself if that's o.k. > > We do have about 1.000 user in total and not everybody is currently > logged in. > > May be this is o.k. but if not, what might I check? > > We run the dovecot package provided by redhat EL 5.8 dovecot-1.0.7-7.el5_7.1 > > Thanks for any comment or suggestion. > > Regards . G?tz -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From dlie76 at yahoo.com.au Wed Apr 18 04:12:49 2012 From: dlie76 at yahoo.com.au (Daminto Lie) Date: Tue, 17 Apr 2012 18:12:49 -0700 (PDT) Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <4F8D1507.7060609@mohtex.net> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> <4F86DCA7.1060806@talex.pl> <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> <4F8CE2B8.3080801@mohtex.net> <1334643297.87733.YahooMailNeo@web113417.mail.gq1.yahoo.com> <4F8D1507.7060609@mohtex.net> Message-ID: <1334711569.21467.YahooMailNeo@web113409.mail.gq1.yahoo.com> Thank you Tamsy for sharing with me your config file. Initially, I thought I tried not to use managesieve and that's why I skipped that part there. But after seeing your config, I'd try to use managesieve. I would need to modify my dovecot config file first to incorporate the managesieve. And I also learnt that I need to put in my domain name. In this case, I would try this again next week as this server has not been made as a production server just yet. I thought if I could get all the things I need to work before the day of putting it online to the Internet. That would save me a lot of work. Thank you once again ________________________________ From: Tamsy To: Dovecot Mailing List Sent: Tuesday, 17 April 2012 5:00 PM Subject: Re: [Dovecot] vacation plugins for squirrelmail Seems not to be attached to my former mail. Here it comes again, inline: avelsieve config.php ############################### * @copyright 2002-2004 Alexandros Vellis * @package plugins * @subpackage avelsieve */ /** * Debug Mode. Enable this (change to 1) if you need to send a bug report, * or to 2 or 3 if you are a developer! * * Valid values: * 0 = No debugging output * 1 = Normal debugging output * 2 = Firebug-enhanced debugging output * 3 = Enhanced debugging output and no Sieve capabilities checks - enables *? ? all UI features! */ if(!defined('AVELSIEVE_DEBUG')) { ? ? define('AVELSIEVE_DEBUG', 1); } /* ======================================================================== */ /* =================== IMAP Server / SIEVE Setup? ========================= */ /* ======================================================================== */ /** @var string Backend to use. Available backends are: * 'ManageSieve': Uses the ManageSieve protocol. e.g. Cyrus * 'File': Writes files straight to disk. e.g. Exim4, Dovecot LDA. */ global $avelsieve_backend; $avelsieve_backend = 'ManageSieve'; /* ======================================================================== */ /* =================== ManageSieve Backend Options ======================== */ /* ======================================================================== */ /* Port where ManageSieve servers (e.g. Cyrus IMAP's timsieved) listens to. * Default is 2000. Note that another possible port that your server might * listen to, is 4190, which is now the official (IANA-assigned) port for * Sieve. */ global $sieveport; $sieveport = 4190; /** * @var string Space separated list of preferred SASL mechanisms for the * authentication to ManageSieve server. e.g. "PLAIN DIGEST-MD5";*/ global $sieve_preferred_sasl_mech; $sieve_preferred_sasl_mech = 'PLAIN'; /** * @var boolean Disable STARTTLS for ManageSieve. You can set this to true, * if you do not wish to use encryption via TLS mechanisms (i.e. the server * is not configured properly, or this is a local connection and TLS is not * needed. * Note that STARTTLS is supported only in PHP5+. In PHP4 this option will * have no effect and STARTTLS will be disabled anyway. */ global $avelsieve_disabletls; $avelsieve_disabletls = false; /* ======================================================================== */ /* ======================= File Backend Options =========================== */ /* ======================================================================== */ global $avelsieve_file_backend_options, $data_dir, $username; $avelsieve_file_backend_options = array( ? ? 'avelsieve_default_file' => "$data_dir/$username.sievesource" ); /* ======================================================================== */ /* ====== Implementation- and Server-Specific? Options ==================== */ /* ======================================================================== */ /* In Cyrus 2.3+, the notification action is a bit more complex than the * others. The oldcyrus variable is for supporting the partially implemented * notify extension implementation of Cyrus < 2.3. If you have Cyrus < 2.3, * just set this to true. * * This only changes the informational / help text displayed in avelsieve. * * Cyrus < 2.3 : $from$, $env-from$, $subject$ * Cyrus 2.3+? : $from$, $env-from$, $subject$, $text$, $text[n]$ */ global $avelsieve_oldcyrus; $avelsieve_oldcyrus = true; /* If you have Cyrus with an lmtpd that can understand the "auth" argument to * the :envelope test as the SMTP/LMTP auth, or any other Sieve implementation, * then you can enable this to provide this functionality to the user. * * This was not clarified in the base spec of RFC 3028. It will be done * correctly in a new version of Cyrus, based on a new draft / spec. */ global $avelsieve_enable_envelope_auth; $avelsieve_enable_envelope_auth = true; /* Some Implementations of Sieve need certain things in order to operate * correctly. If you use any of the following server implementations, you MUST * set this variable to the corresponding value for the filtering to work * correctly. * Valid values are: * - Any RFC3028-mostly-compatible implementation: '' (empty). * - Exim MTA: 'exim' * - MFL (as supported by mvmf): 'mfl' */ global $avelsieve_custom_sieve_implementation; $avelsieve_custom_sieve_implementation = ''; /* For delivery agents that don't know how to handle some mailbox prefixes, you * can enable this option. Example for Dovecot LDA: 'INBOX.'; */ global $avelsieve_striproot; $avelsieve_striproot = 'INBOX.'; /* If the backend does not support capabilities reporting, such as the File * Backend, then you should define which capabilities are used by the server * implementation. * * The following are the capabilities supported by Exim4 as of Exim version * 4.60, according to README.SIEVE. You can change them if a new version of * Exim provides more functionality: *? 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' * * The following are the capabilities that are suported by MFM, according to: * http://www.mvmf.org/mfl/language.shtml#sieve *? 'envelope', 'fileinto', 'reject', 'relational', 'subaddress', 'regex', *? 'editheader', 'copy', 'vacation', 'comparator-i;ascii-casemap', *? 'comparator-i;octet' * * The following capabilities have been reported to work with Dovecot LDA: *? 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric', *? 'imapflags', 'subaddress','relational','regex' * */ /* global $avelsieve_hardcoded_capabilities; * $avelsieve_hardcoded_capabilities = array( *? ? 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' * ); */ global $avelsieve_hardcoded_capabilities; $avelsieve_hardcoded_capabilities = array( ? ? 'fileinto', 'reject', 'envelope', 'encoded-character', 'vacation', ? ? 'subaddress', 'comparator-i;ascii-numeric', 'relational', 'regex', ? ? 'imap4flags', 'copy', 'include', 'variables', 'body', 'enotify', ? ? 'environment', 'mailbox', 'date' ); /** @var boolean Enable ImapProxy mode. * If you use imapproxy, because imapproxy cannot understand and proxy the * SIEVE protocol, you must connect to the SIEVE daemon (usually on the IMAP * server) itself. So you need to set $imapproxymode to true, and define a * mapping, from the imapproxy host (usually localhost) to your real IMAP * server (usually the same that is defined on Imapproxy's configuration). * * This will not work if you use a perdition-style proxy, where different users * go to different IMAP servers; it applies mostly to people running imapproxy * for speed and want a quick hack. */ global $avelsieve_imapproxymode, $avelsieve_imapproxyserv; $avelsieve_imapproxymode = false; $avelsieve_imapproxyserv = array( ? ? 'localhost' => 'YOURDOMAINNAMEHERE.com' ); /** @var boolean Ldapuserdata mode: Gets user's email addresses (including * mailAlternate & mailAuthorized) from LDAP Prefs Backend plugin's cache */ global $avelsieve_ldapuserdatamode; $avelsieve_ldapuserdatamode = false; /** @var array Map of cyrus administrator users, for proxy authentication */ global $avelsieve_cyrusadmins_map; $avelsieve_cyrusadmins_map = array( ? ? 'cyrusimap' => 'cyrussieve' ); /* ======================================================================== */ /* =============== Avelsieve Interface / Behavior Setup? ================== */ /* ======================================================================== */ /* Be conservative to our updates on the SIEVE server? If true, a button * entitled "Save Changes" will appear, which will give the user the * functionality to register her changes. 'false' is recommended. */ $conservative = false; /* Use images for the move up / down, delete rule buttons and STOP? */ $useimages = true; /* Translate the messages returned by the "Reject" and "Vacation" actions? The * default behaviour since 0.9 is not to translate them. Change to true if in * an intranet environment or in a same-language environment. */ global $translate_return_msgs; $translate_return_msgs = false; /* Theme to use for the images. A directory with the same name must exist under * plugins/avelsieve/$imagetheme, that contains the files: up.png, down.png, * del.png, dup.png, edit.png, top.png, bottom.png. */ $imagetheme = 'famfamfam'; //$imagetheme = 'bluecurve_24x24'; //$imagetheme = 'bluecurve_16x16'; /* Number of items to display _initially_, when displaying the header match * rule */ global $startitems; $startitems = 3; /* Maximum number of items to allow in one header match rule. */ global $maxitems; $maxitems = 10; /* Headers to display in listbox widget, when adding a new header rule. */ /* global $headers; * $headers = array( *? 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Reply-To', 'Sender', 'List-Id', *? 'MailingList', 'Mailing-List', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', *? 'Resent-From',? 'Resent-To', 'X-Mailer', 'X-Mailing-List', *? 'X-Spam-Flag', 'X-Spam-Status', *? 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', *? 'Return-Path', 'Received', 'Auto-Submitted', *? 'X-Spam-Flag', 'X-Spam-Status','X-Spam-Tests' *? ); */ global $headers; $headers = array( 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Sender', 'Reply-To', 'List-Id', 'MailingList', 'Mailing-List', 'X-DKIM-Authentication-Results', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', 'Resent-From',? 'Resent-To', 'X-Mailer', 'X-Mailing-List', 'X-SpamTest-Status-Extended', 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', 'Return-Path', 'Received', 'Auto-Submitted', ); /* Available :method's for the :notify extension (if applicable) */ global $notifymethods; $notifymethods = array( 'mailto', 'sms' ); /* use the value "false" if you want to provide a simple input box so that * users can edit the method themselves : */ //$notifymethods = false; /* Capabilities to disable. If you would like to force avelsieve not to display * certain features, even though there _is_ a capability for them in Sieve * server, you should specify these here. For instance, if you would * like to disable the notify extension, even though ManageSieve advertises it, * you should add 'notify' in this array: $force_disable_avelsieve_capability = * array("notify");. This will still leave the defined feature on, and if the * user can upload her own scripts then she can use that feature; this option * just disables the GUI of it. Leave as-is (empty array) if you do not need * that. * * Look in $implemented_capabilities array in include/constants.inc.php for * valid values */ // $disable_avelsieve_capabilities = array("notify"); global $disable_avelsieve_capabilities; $disable_avelsieve_capabilities = array(); /* Display Filters link in the top Squirrelmail header? */ global $avelsieveheaderlink; $avelsieveheaderlink = true; /* Default rules table display mode, one of 'verbose' or 'terse' */ global $avelsieve_default_mode; $avelsieve_default_mode = 'terse'; /* ======================================================================== */ /* ========================= Custom rules Configuration =================== */ /* ======================================================================== */ /** * @var array * List of additional rules to enable. This must be the numeric id of the * rule. * * 10: Spam Rule (as existed in Avelsieve) * 11: Junk-Mail Rule (new-style SPAM Rule, only one exists per Sieve script) * 12: Global Whitelist (only one exists per Sieve script) * 13: Custom Sieve Code (allows users to enter whatever Sieve code they like - *? ? Alpha/Experimental) * * Example: * array(11, 12); */ global $avelsieve_enable_rules; $avelsieve_enable_rules = array(); global $avelsieve_rules_settings; $avelsieve_rules_settings = array(); foreach($avelsieve_enable_rules as $r) { ? ? if(file_exists(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php')) { ? ? ? ? require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php'); ? ? } else { ? ? ? ? require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.default.php'); ? ? } } /* Please keep the following setting false; it is alpha + needs Squirrelmail * to be patched in three or four places. */ $avelsieve_spam_highlight_enable = false; ############################### Hi Daminto, That looks pretty much unconfigured. No wonder nothing is coming up. I have attached the avelsieve config file we are using on one of our domains. Don't copy it blindly. At least you should configure $sieveport to the port you are using on your server for Sieve. And put your domain name where it reads YOURDOMAINNAMEHERE.com. For the arrays at $headers you should tune the filters you are using with your headers. Hope this helps (working fine with our domain and Squirrel-Mail). Rgds Tamsy Daminto Lie wrote the following on 17.04.2012 13:14: > Thank you for your reply. > > Here it is as follows > > > if(!defined('AVELSIEVE_DEBUG')) { >? ? ? define('AVELSIEVE_DEBUG', 0); > } > > global $avelsieve_backend; > $avelsieve_backend = 'File'; > > /*global $sieveport; > $sieveport = 2000;*/ > > global $sieve_preferred_sasl_mech; > $sieve_preferred_sasl_mech = 'PLAIN'; > > global $avelsieve_disabletls; > $avelsieve_disabletls = false; > > global $avelsieve_file_backend_options, $data_dir, $username; > $avelsieve_file_backend_options = array( >? ? ? 'avelsieve_default_file' =>? "$data_dir/$username.sievesource" > ); > > /*global $avelsieve_oldcyrus; > $avelsieve_oldcyrus = true;*/ > > global $avelsieve_enable_envelope_auth; > $avelsieve_enable_envelope_auth = true; > > global $avelsieve_custom_sieve_implementation; > $avelsieve_custom_sieve_implementation = ''; > > global $avelsieve_striproot; > $avelsieve_striproot = ''; > > global $avelsieve_hardcoded_capabilities; > $avelsieve_hardcoded_capabilities = array( >? ? ? 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' > ); > > /*global $avelsieve_imapproxymode, $avelsieve_imapproxyserv; > $avelsieve_imapproxymode = false; > $avelsieve_imapproxyserv = array( >? ? ? 'localhost' =>? 'imap.example.org' > );*/ > > global $avelsieve_ldapuserdatamode; > $avelsieve_ldapuserdatamode = false; > > /*global $avelsieve_cyrusadmins_map; > $avelsieve_cyrusadmins_map = array( >? ? ? 'cyrusimap' =>? 'cyrussieve' > );*/ > > $conservative = false; > > $useimages = true; > > global $translate_return_msgs; > $translate_return_msgs = false; > > $imagetheme = 'famfamfam'; > //$imagetheme = 'bluecurve_24x24'; > //$imagetheme = 'bluecurve_16x16'; > > global $startitems; > $startitems = 3; > > global $maxitems; > $maxitems = 10; > > global $headers; > $headers = array( >? 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Reply-To', 'Sender', 'List-Id', >? 'MailingList', 'Mailing-List', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', >? 'Resent-From',? 'Resent-To', 'X-Mailer', 'X-Mailing-List', >? 'X-Spam-Flag', 'X-Spam-Status', >? 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', >? 'Return-Path', 'Received', 'Auto-Submitted', >? 'X-Spam-Flag', 'X-Spam-Status','X-Spam-Tests' >? ); > > global $notifymethods; > $notifymethods = array( > 'mailto', 'sms' > ); > > //$notifymethods = false; > > // $disable_avelsieve_capabilities = array("notify"); > global $disable_avelsieve_capabilities; > $disable_avelsieve_capabilities = array(); > > global $avelsieveheaderlink; > $avelsieveheaderlink = true; > > global $avelsieve_default_mode; > $avelsieve_default_mode = 'terse'; > > global $avelsieve_enable_rules; > $avelsieve_enable_rules = array(); > > global $avelsieve_rules_settings; > $avelsieve_rules_settings = array(); > foreach($avelsieve_enable_rules as $r) { >? ? ? if(file_exists(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php')) { >? ? ? ? ? require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php'); >? ? ? } else { >? ? ? ? ? require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.default.php'); >? ? ? } > } > > $avelsieve_spam_highlight_enable = false; > > > Hope this help you to help me. > > Thank you > > > > > ________________________________ >? From: Tamsy > To: "dovecot at dovecot.org" > Sent: Tuesday, 17 April 2012 1:25 PM > Subject: Re: [Dovecot] vacation plugins for squirrelmail > > Show us your avelsieve-config in > your-squirrelmail-dir/plugins/avelsieve/config/config.php > > > Daminto Lie wrote the following on 17.04.2012 09:31: >> Thank you folks for your replies. >> >> >> I realised that the dovecot 1.2.9 that I installed via ubuntu synaptic package manager has already come with sieve plugins. After making some modifications on the dovecot.conf file, I went to download and install avelsieve 1.0.1 (stable version) onto the squirrelmail. Then, I run in squirrelmail conf.pl and added the plugin. I logged in and could see "Filter" option right on the top header row. I went to click it and nothing came up. I checked in the mail.log but did not see any errors there. >> >> Here is my snapshot of dovecot configuration >> # 1.2.9: /etc/dovecot/dovecot.conf >> # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS ext4 >> log_timestamp: %Y-%m-%d %H:%M:%S >> protocols: imap >> listen: *:143 >> ssl: >>? ? no >> disable_plaintext_auth: no >> login_dir: /var/run/dovecot/login >> login_executable: /usr/lib/dovecot/imap-login >> verbose_proctitle: yes >> first_valid_uid: 1001 >> last_valid_uid: 600 >> mail_privileged_group: mail >> mail_location: maildir:/var/vmail/%u/Maildir >> mbox_write_locks: fcntl dotlock >> imap_client_workarounds: outlook-idle delay-newmail netscape-eoh tb-extra-mailbox-sep >> lda: >>? ? ? postmaster_address: postmaster at ourcompany.com >>? ? ? mail_plugins: sieve >>? ? ? quota_full_tempfail: yes >>? ? ? deliver_log_format: msgid=%m: %$ >>? ? ? sendmail_path: /usr/lib/sendmail >>? ? ? rejection_reason: Your message to<%t>? was automatically rejected:%n%r >>? ? ? auth_socket_path: /var/run/dovecot-auth-master >> auth default: >>? ? ? mechanisms: plain login >>? ? ? username_format: %Lu >>? ? ? verbose: yes >>? ? ? debug: yes >>? ? ? debug_passwords: yes >>? ? ? passdb: >>? ? ? ? driver: >>? ? pam >>? ? ? passdb: >>? ? ? ? driver: ldap >>? ? ? ? args: /etc/dovecot/dovecot-ldap.conf >>? ? ? userdb: >>? ? ? ? driver: passwd >>? ? ? userdb: >>? ? ? ? driver: static >>? ? ? ? args: uid=1001 gid=1001 home=/var/vmail/%u allow_all_users=yes >>? ? ? socket: >>? ? ? ? type: listen >>? ? ? ? client: >>? ? ? ? ? path: /var/spool/postfix/private/auth >>? ? ? ? ? mode: 432 >>? ? ? ? ? user: postfix >>? ? ? ? ? group: mail >>? ? ? ? master: >>? ? ? ? ? path: /var/run/dovecot-auth-master >>? ? ? ? ? mode: 432 >>? ? ? ? ? user: vmail >>? ? ? ? ? group: vmail >> plugin: >>? ? ? sieve: /var/vmail/%u/.sieve >> >> Also, I have restarted dovecot after making changes to the dovecot.conf. >> >> Any help would be very much appreciated. >> >> Thank you >> >>? >> >> ________________________________ >>? ? From: Artur Zaprza?a >> To: Dovecot Mailing List >> Cc: Daminto Lie >> Sent: Thursday, 12 April 2012 11:46 PM >> Subject: Re: [Dovecot] vacation plugins for squirrelmail >> >> Daminto Lie wrote: >>> Hi, >>> >>> I am afraid I >>? ? have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. >>> Any help would be very much appreciated. >>> >>> Thank you >>> >> I'm using Avelsieve 1.9.9 with a set of my own bugfixes: >> http://email.uoa.gr/avelsieve/ >> >> From cor at xs4all.nl Wed Apr 18 11:22:32 2012 From: cor at xs4all.nl (Cor Bosman) Date: Wed, 18 Apr 2012 10:22:32 +0200 Subject: [Dovecot] trash plugin Message-ID: <6332F986-C1C4-421E-9EFC-362D429047F4@xs4all.nl> The trash plugin docs say: "Normally if a message can't be saved/copied because it would bring user over quota, the save/copy fails with "Quota exceeded" error. The trash plugin can be used to avoid such situations by making Dovecot automatically expunge oldest messages from configured mailboxes until the message can be saved" This leads me to believe that if a message cant be stored, the plugin would avoid returning a quota error, but instead delete/expunge messages until the original message can fit, and then complete the command. But I see something different, at least using the plugin in imap (havent tried it on lmtp). When I drag a message to a folder thats almost full, and the message doesnt fit, I get an actual quota error: 23 uid copy 454343 "Spam" 23 NO [OVERQUOTA] Quota exceeded (mailbox for user is full) After my client sees this error, I see the trash plugin creating space. But then it's too late right? The client already has a visible error. Am I just misunderstanding how the trash plugin is supposed to work? Dovecot 2.1.4, config is on a set of test servers: namespace { type = private separator = / inbox = yes } namespace { type = private separator = / inbox = no prefix = Spam/ location = maildir:/var/spool/mail/%1u/%1.1u/%u/.Spam:INDEX=/var/spool/mail/dovecot-control/indexes/%1u/%2u/%u/.Spam:CONTROL=/var/spool/mail/dovecot-control/%1u/%2u/%u/.Spam subscriptions = no list = yes hidden = yes } plugin { quota = maildir:Spam quota:ns=Spam/ quota_rule = *:storage=5000K } My goal is to create a FIFO spam folder that never returns a quota error, but instead just deletes the oldest emails as new spam comes in. Regards, Cor From cor at xs4all.nl Wed Apr 18 11:53:34 2012 From: cor at xs4all.nl (Cor Bosman) Date: Wed, 18 Apr 2012 10:53:34 +0200 Subject: [Dovecot] returning a namespace location from userdb? Message-ID: <567E4A7D-930E-4C57-B79D-8C94554D7C72@xs4all.nl> hey all, is it possible to return the location of namespace from the userdb lookup? The code is a bit unclear about it. There seems to be a part of the docs saying: If you want to override settings inside sections, you can separate the section name and key with '/'. For example: namespace default { inbox = yes } The inbox setting can be overridden by returning namespace/default/inbox=no extra field I realize you can set 'location=' in the namespace, but we have mailbox locations that cant be expressed by the normal variable substitution vars in dovecot (dont ask). So we have a userdb that returns the correct mail location. This works fine. But im currently trying to implement a second namespace, and Im having trouble returning a mailbox location for this specific namespace from our userdb. Is this possible? If so, what would the userdb have to return exactly? namespace spam { type = private separator = / inbox = no prefix = Spam/ subscriptions = no list = yes hidden = yes } Regards, Cor From pw at wk-serv.de Wed Apr 18 12:49:14 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Wed, 18 Apr 2012 11:49:14 +0200 Subject: [Dovecot] Missing 20-managesieve.conf in 2.1.4? Message-ID: <4F8E8E1A.3080408@wk-serv.de> Hello, I installed Dovecot 2.1.4 and Pigeonhole 0.3.0 and wondered that there is no 20-managesieve.conf file anymore. It is no problem to use one of my old files. I just want to know whether this is a bug or not. Regards Patrick From CMarcus at Media-Brokers.com Wed Apr 18 13:22:27 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 18 Apr 2012 06:22:27 -0400 Subject: [Dovecot] OT Re: Dovecot and Outlook Express "plaintext authentication not allowed..." In-Reply-To: <1334695280.2248.YahooMailNeo@web161606.mail.bf1.yahoo.com> References: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> <4F8D455D.9060805@Media-Brokers.com> <1334695280.2248.YahooMailNeo@web161606.mail.bf1.yahoo.com> Message-ID: <4F8E95E3.6020500@Media-Brokers.com> On 2012-04-17 4:41 PM, D Chen wrote: > Per your suggestion, in Outlook Express I changed to use SSL on port > 993 for IMAP, and port 465 for SMTP, the creation of the new account > succeeded without error! However, when tried to send to my yahoo mail or > gmail acct, got the following error: (apparently failed to send!) Changed subject to OT because this is not relevant to dovecot... > "The message could not be sent because one of the recipients was rejected by the server. The > rejected email address was 'dchenusa at yahoo.com.' Subject: 'test out to yahoo mail' > Account:'dchen192.168.20.100' > Server: 192.168.20.100 > Protocol: SMTP, server response: 554 5.7.1 That is a 'Relay access denied' error (assuming yahoo uses correct error codes, which is probably a bad assumption)... Either Yahoo doesn't support port 465 (try the submission port for your yahoo smtp server - 587)... or you have a free account which doesn't allow relaying using their servers (so you'd have to use something else for sending - what does the Thunderbird client use for sending for that account?)... But as suggested... don't use Outlook Express, use a modern mail client (Thunderbird is my preference, but someone else suggested Windows Live Mail which does suck much less than Outlook Express)... -- Best regards, Charles From stephan at rename-it.nl Wed Apr 18 13:34:15 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 18 Apr 2012 12:34:15 +0200 Subject: [Dovecot] Missing 20-managesieve.conf in 2.1.4? In-Reply-To: <4F8E8E1A.3080408@wk-serv.de> References: <4F8E8E1A.3080408@wk-serv.de> Message-ID: <4F8E98A7.2040305@rename-it.nl> On 4/18/2012 11:49 AM, Patrick Westenberg wrote: > Hello, > > I installed Dovecot 2.1.4 and Pigeonhole 0.3.0 and wondered that there > is no 20-managesieve.conf file anymore. > > It is no problem to use one of my old files. I just want to know whether > this is a bug or not. I wouldn't know why it is missing in your installation. It is still contained in the tarball. How did you install it exactly? Regards, Stephan. From pw at wk-serv.de Wed Apr 18 14:06:34 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Wed, 18 Apr 2012 13:06:34 +0200 Subject: [Dovecot] Missing 20-managesieve.conf in 2.1.4? In-Reply-To: <4F8E98A7.2040305@rename-it.nl> References: <4F8E8E1A.3080408@wk-serv.de> <4F8E98A7.2040305@rename-it.nl> Message-ID: <4F8EA03A.7000601@wk-serv.de> Stephan Bosch schrieb: > I wouldn't know why it is missing in your installation. It is still > contained in the tarball. How did you install it exactly? I installed it the same way as usual: ./configure make make install No errors reported. I checked the configure options and also tried --with-managesieve but the file is still missing. However, this is my first installation on Scientific Linux. Patrick From john.robinson at anonymous.org.uk Wed Apr 18 14:36:51 2012 From: john.robinson at anonymous.org.uk (John Robinson) Date: Wed, 18 Apr 2012 12:36:51 +0100 Subject: [Dovecot] mbox folders on-disc layout Message-ID: <4F8EA753.4070605@anonymous.org.uk> I'd like to change the on-disk layout for my mboxes from being ~/mail/foo/bar to ~/mail/foo.bar so that I can have folders containing both messages and subfolders, without having subfolders beginning with . and then having to do all the other fiddling with locations of index files etc. that goes along with LAYOUT=maildir++, and also leaving the filesystem layout looking sensible so my users see their mailboxes instead of the index and control files. Is this possible or do I need to patch something? (CentOS 6, dovecot 2.0.9) Cheers, John. From stephan at rename-it.nl Wed Apr 18 15:42:14 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 18 Apr 2012 14:42:14 +0200 Subject: [Dovecot] http://xi.rename-it.nl upgrade 2.0 - 2.1 dovecot-common (<< 2:2.1.alpha1-0~auto+47) In-Reply-To: <4F8AC9B1.2030307@schetterer.org> References: <4F8AC9B1.2030307@schetterer.org> Message-ID: <4F8EB6A6.6010206@rename-it.nl> Op 4/15/2012 3:14 PM, Robert Schetterer schreef: > Hi, > i got > depency problem like dovecot-common (<< 2:2.1.alpha1-0~auto+47) > > dpkg -i dovecot-common_2.1.4-0~auto+3_all.de > > after all dove seems to run without install dovecot-common basicly You should be able to drop dovecot-common entirely. It is nothing but a dummy package now. Regards, Stephan. From tss at iki.fi Wed Apr 18 16:45:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 16:45:45 +0300 Subject: [Dovecot] Statistics scripts Message-ID: <42DEDB45-6B9A-47BB-BFB1-B82ECC086839@iki.fi> I wrote two scripts, which you can use with the stats plugin enabled to get a better "doveadm stats top" than it currently is: http://dovecot.org/tools/stats-top.pl http://dovecot.org/tools/stats.pl From tss at iki.fi Wed Apr 18 19:29:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 19:29:58 +0300 Subject: [Dovecot] trash plugin In-Reply-To: <6332F986-C1C4-421E-9EFC-362D429047F4@xs4all.nl> References: <6332F986-C1C4-421E-9EFC-362D429047F4@xs4all.nl> Message-ID: <14B830E2-9EFD-45CA-B0F7-BEDDF0FCDFF1@iki.fi> On 18.4.2012, at 11.22, Cor Bosman wrote: > The trash plugin docs say: > > "Normally if a message can't be saved/copied because it would bring user over quota, the save/copy fails with "Quota exceeded" error. The trash plugin can be used to avoid such situations by making Dovecot automatically expunge oldest messages from configured mailboxes until the message can be saved" > > This leads me to believe that if a message cant be stored, the plugin would avoid returning a quota error, but instead delete/expunge messages until the original message can fit, and then complete the command. Yes. > But I see something different, at least using the plugin in imap (havent tried it on lmtp). When I drag a message to a folder thats almost full, and the message doesnt fit, I get an actual quota error: > > 23 uid copy 454343 "Spam" > 23 NO [OVERQUOTA] Quota exceeded (mailbox for user is full) > > After my client sees this error, I see the trash plugin creating space. But then it's too late right? The client already has a visible error. Am I just misunderstanding how the trash plugin is supposed to work? Fixed: http://hg.dovecot.org/dovecot-2.1/rev/51b40cffb628 From tss at iki.fi Wed Apr 18 19:42:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 19:42:21 +0300 Subject: [Dovecot] returning a namespace location from userdb? In-Reply-To: <567E4A7D-930E-4C57-B79D-8C94554D7C72@xs4all.nl> References: <567E4A7D-930E-4C57-B79D-8C94554D7C72@xs4all.nl> Message-ID: On 18.4.2012, at 11.53, Cor Bosman wrote: > hey all, is it possible to return the location of namespace from the userdb lookup? Yes. > But im currently trying to implement a second namespace, and Im having trouble returning a mailbox location for this specific namespace from our userdb. Is this possible? If so, what would the userdb have to return exactly? > > namespace spam { namespace/spam/location=maildir:/something/different From tss at iki.fi Wed Apr 18 19:45:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 19:45:13 +0300 Subject: [Dovecot] mbox folders on-disc layout In-Reply-To: <4F8EA753.4070605@anonymous.org.uk> References: <4F8EA753.4070605@anonymous.org.uk> Message-ID: <35C2B470-438E-4CEC-A42A-0BC2C39A0159@iki.fi> On 18.4.2012, at 14.36, John Robinson wrote: > I'd like to change the on-disk layout for my mboxes from being > ~/mail/foo/bar > to > ~/mail/foo.bar > so that I can have folders containing both messages and subfolders, without having subfolders beginning with . and then having to do all the other fiddling with locations of index files etc. that goes along with LAYOUT=maildir++, and also leaving the filesystem layout looking sensible so my users see their mailboxes instead of the index and control files. > > Is this possible or do I need to patch something? Not possible without patching. There's also the other possibility of doing something like: # Trick mbox configuration which allows a mail folder which contains both # messages and sub-folders mail_location = mbox:~/mail/mailboxes:DIRNAME=mBoX-MeSsAgEs:INDEX=~/mail/index:CONTROL=~/mail/control from http://wiki2.dovecot.org/MboxChildFolders From tss at iki.fi Wed Apr 18 19:46:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 19:46:42 +0300 Subject: [Dovecot] 487 imap-login processes - is this ok In-Reply-To: <4F8D7C73.9090700@filmakademie.de> References: <4F8D7C73.9090700@filmakademie.de> Message-ID: <74640698-B3E5-46BB-88FC-98B088CF0330@iki.fi> On 17.4.2012, at 17.21, G?tz Reinicke wrote: > recently I checked our mailserver for some more tuning and optimizing > and noticed, that currently I do have 487 imap-login processes and I'm > asking myself if that's o.k. > > We do have about 1.000 user in total and not everybody is currently > logged in. > > May be this is o.k. but if not, what might I check? Normal in "high-security mode": http://wiki.dovecot.org/LoginProcess Most of those 487 processes are probably handling SSL traffic. From tss at iki.fi Wed Apr 18 19:47:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 19:47:11 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F8DEEBD.9080206@corp.sonic.net> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F8DEEBD.9080206@corp.sonic.net> Message-ID: <8B60CEC2-4945-4513-9873-CCEEA0344C7A@iki.fi> On 18.4.2012, at 1.29, Kelsey Cummings wrote: > On 03/30/12 07:35, Timo Sirainen wrote: >> IP stickiness isn't enough if user uses more than one IMAP client, which is pretty common nowadays. And doesn't help at all with LDA. > > Timo, having the LDA flow through the director with maildir and NFS is only an issue if you want to use the dovecot delivery agent since it manipulates the indexes on delivery, right? This shouldn't be a problem if you are running a completely distinct delivery cluster running a LDA like procmail. Right. If indexes or dovecot-uidlist isn't updated then it doesn't matter. From tss at iki.fi Wed Apr 18 19:57:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 19:57:26 +0300 Subject: [Dovecot] Re-indexing of fts-solr not working In-Reply-To: References: Message-ID: <1766B7F6-4A32-41DA-9655-13E0AE43A8C7@iki.fi> On 17.4.2012, at 21.36, Joe Beaubien wrote: > I am currently testing a 2.1 dovecot setup with fts-solr. Search speed is > simply amazing. I decided to try to delete and rebuild the solr indexes, > but i can't seem to rebuild them thru dovecot. With Maildir you could just delete dovecot.index* files. But with dbox that's not a good idea. Anyway, implemented now the minimum needed to get doveadm fts rescan to work: http://hg.dovecot.org/dovecot-2.1/rev/d145ea6f7061 From tss at iki.fi Wed Apr 18 20:01:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 20:01:14 +0300 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken In-Reply-To: <4F8D80D3.2050505@schetterer.org> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> Message-ID: On 17.4.2012, at 17.40, Robert Schetterer wrote: > doveadm acl debug > > shows on both versions > > Fatal: ACL shared dict iteration failed Doesn't it show any other error message before this? Either to terminal or to error log? What's your doveconf -n? From tss at iki.fi Wed Apr 18 20:06:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 20:06:45 +0300 Subject: [Dovecot] LMTP auth problem In-Reply-To: <20120414172135.GA32608@xs4all.nl> References: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> <71BEFF67-4A0A-442A-9ED0-C4B1B23A4E7F@leuxner.net> <20120414172135.GA32608@xs4all.nl> Message-ID: On 14.4.2012, at 20.21, Cor Bosman wrote: > My config was the same as yours. That didnt work for me. But if I add > > user = dovecot > mode = 0666 Hmm. Perhaps I should make this the default. > That does work. Of course, the difference between 777 and 666 is > minimal. I think 666 is handled as a special case in the code? Yes, it's handled specially. 0666 is now a safe default for multi-user systems, since it returns data for a user only if the calling process's UID matches the returned "uid". 0777 makes it truly world-accessible. From news0104.6.weezy at spamgourmet.com Wed Apr 18 20:00:48 2012 From: news0104.6.weezy at spamgourmet.com (Andreas Helmcke) Date: Wed, 18 Apr 2012 19:00:48 +0200 Subject: [Dovecot] namespace hierarchy In-Reply-To: References: <201204010928.41017.anyaddress@gmx.net> Message-ID: <4F8EF340.4010407@spamgourmet.com> Am 04.04.2012 03:47, schrieb Timo Sirainen: > On 1.4.2012, at 10.28, Tom Fernandes wrote: > >> In the new server I would like to have 3 namespaces (private, public, shared) >> and use "/" as separator. Is it possible to keep "INBOX." for private, so that >> the users mailboxes appear below the Inbox, but have public and shared on the >> same hierarchy-level like the Inbox? >> > > If you have any subscriptions=no namespaces, you need a parent namespace that has subscriptions=yes. You could have e.g.: > > namespace { > prefix = INBOX/ > list = no > inbox = yes > } > namespace { > prefix = > list = no > hidden = yes > alias_for = INBOX/ > } > > So now the prefix="" is used for the subscriptions but nothing else really. I tried this, but sadly with dovecot V2.1.1 it is not working. Steps to reproduce: + create empty mail directory + create one folder (Allgemeines/Tabel) in public namespace with doveadm + start mail-client (in my case thunderbird) + try to subscribe to the new public folder Error message in log: Apr 18 18:19:34 elablnmail02 dovecot: imap(ahelmcke): Warning: Subscriptions file /home/mail/user/ahelmcke/subscriptions: Removing invalid entry: Allgemeines/Tabel doveconf -n: # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) ext4 auth_master_user_separator = * first_valid_uid = 400 mail_gid = vmail mail_location = maildir:/home/mail/user/%u mail_uid = vmail mbox_write_locks = fcntl namespace Allgemeines { location = maildir:/home/mail/Allgemeines:INDEX=/home/mail/user/%u/SEEN.Allgemeines prefix = Allgemeines/ separator = / subscriptions = no type = public } namespace dummy { alias_for = INBOX/ hidden = yes list = no location = prefix = separator = / type = private } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile sieve = /var/lib/dovecot/sieve/%u.sieve sieve_global_dir = /var/lib/dovecot/sieve/global/ sieve_global_path = /var/lib/dovecot/sieve/default.sieve } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } service imap-login { inet_listener imap { address = * } inet_listener imaps { address = * } } ssl = required ssl_cert = References: <201204010928.41017.anyaddress@gmx.net> <4F8EF340.4010407@spamgourmet.com> Message-ID: <305E07F6-1D75-4B6D-862A-D7697442A770@iki.fi> On 18.4.2012, at 20.00, Andreas Helmcke wrote: >> If you have any subscriptions=no namespaces, you need a parent namespace that has subscriptions=yes. You could have e.g.: >> >> namespace { >> prefix = INBOX/ >> list = no >> inbox = yes >> } >> namespace { >> prefix = >> list = no >> hidden = yes >> alias_for = INBOX/ >> } >> >> So now the prefix="" is used for the subscriptions but nothing else really. > > I tried this, but sadly with dovecot V2.1.1 it is not working. Oh, I hadn't thought of one problem: It's now using the same subscriptions file for both prefix="" and prefix=INBOX/ namespaces. This confuses it. So you need to place the prefix="" namespace somewhere else, for example: > namespace dummy { > alias_for = INBOX/ location = maildir:/home/mail/user/%u/public > hidden = yes > list = no > location = > prefix = > separator = / > type = private > } From robert at schetterer.org Wed Apr 18 20:44:23 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 18 Apr 2012 19:44:23 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken In-Reply-To: References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> Message-ID: <4F8EFD77.90103@schetterer.org> Am 18.04.2012 19:01, schrieb Timo Sirainen: > On 17.4.2012, at 17.40, Robert Schetterer wrote: > >> doveadm acl debug >> >> shows on both versions >> >> Fatal: ACL shared dict iteration failed > > Doesn't it show any other error message before this? Either to terminal or to error log? What's your doveconf -n? Hi Timo, i didnt found more usefull errors before all are like Debug: acl: No lookup right to mailbox: but that istn true, i checked it in the filesystem and tried new acls etc ( and they working under 2.0.20 ) to me it looks like some acl check/list problem perhaps depend on stuff like in i.e http://hg.dovecot.org/dovecot-2.1/rev/65a75939ac2c as i said the old 2.0.20 works still fine on the other server the only change i did was with virtual and namespace inbox to get special_use working > this is from the dove version # 2.1.4 (85ad4baedd43): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-37-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 50 M auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes auth_verbose_passwords = plain auth_worker_max_count = 300 dict { acl = mysql:/etc/dovecot/dovecot-dict-acl-sql.conf.ext expire = mysql:/etc/dovecot/dovecot-dict-expire-sql.conf.ext quotadict = mysql:/etc/dovecot/dovecot-dict-quota-sql.conf.ext } disable_plaintext_auth = no first_valid_gid = 1001 first_valid_uid = 1001 hostname = mail01.example.com last_valid_gid = 1001 last_valid_uid = 1001 listen = * login_greeting = imap, pop ready mail_access_groups = vmail mail_debug = yes mail_fsync = always mail_gid = 1001 mail_location = maildir:~/ mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = " stats zlib expire virtual fts fts_squat mail_log autocreate notify acl listescape" mail_privileged_group = vmail mail_uid = 1001 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags notify mmap_disable = yes namespace { list = yes location = maildir:/usr/local/virtual/%%d/%%u:INDEX=~/shared/%%d/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { hidden = no list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } mailbox name { special_use = \Drafts \Junk \Sent \Trash } prefix = separator = / subscriptions = yes } namespace real { hidden = yes list = no location = prefix = RealMails/ separator = / } namespace virtual { hidden = yes list = no location = virtual:/etc/dovecot/virtual2:LAYOUT=maildir++:INDEX=~/virtual prefix = virtual/ separator = / } passdb { args = /etc/dovecot/dovecot-sql-master.conf.ext driver = sql master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl autocreate = Trash autocreate2 = Junk autocreate3 = Sent autocreate4 = Drafts autocreate5 = Templates autocreate6 = Hostmaster-Backup autocreate7 = archiv-backup-mailspooler autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Drafts autosubscribe5 = Templates expire = Trash expire2 = Trash/* expire3 = Junk expire4 = Junk/* expire5 = Hostmaster-Backup expire6 = Hostmaster-Backup/* expire7 = archiv-backup-mailspooler/* expire_dict = proxy::expire fts = squat home = /usr/local/virtual/%d/%u mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size mail_log_group_events = yes quota = dict:::proxy::quotadict quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /usr/local/virtual/%d/%u/dovecot.sieve sieve_before = /etc/dovecot/sieve/global.sieve sieve_dir = /usr/local/virtual/%d/%u/sieve sieve_execute_bin_dir = /usr/lib/dovecot/sieve-execute sieve_execute_socket_dir = sieve-execute sieve_extensions = +notify +imapflags sieve_filter_bin_dir = /usr/lib/dovecot/sieve-filter sieve_filter_socket_dir = sieve-filter sieve_global_dir = /etc/dovecot/sieve/ sieve_global_extensions = +vnd.dovecot.execute +vnd.dovecot.pipe +vnd.dovecot.filter sieve_global_path = /etc/dovecot/sieve/global.sieve sieve_pipe_bin_dir = /usr/lib/dovecot/sieve-pipe sieve_pipe_socket_dir = sieve-pipe sieve_plugins = sieve_extprograms stats_refresh = 30 secs stats_track_cmds = yes zlib_save = gz zlib_save_level = 6 } postmaster_address = hostmaster at mail01.example.com protocols = imap pop3 lmtp sieve sendmail_path = /usr/lib/sendmail service anvil { client_limit = 1000 } service auth-worker { user = root } service auth { client_limit = 1000 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service dict { extra_groups = vmail unix_listener dict { group = vmail mode = 0660 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 8 service_count = 1 } service imap-postlogin { executable = script-login /usr/local/bin/postlogin_imap.sh } service imap { executable = imap imap-postlogin process_limit = 1024 vsz_limit = 256 M } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } vsz_limit = 256 M } service managesieve-login { inet_listener { address = 127.0.0.1 212.52.224.210 212.52.224.205 port = 2000 } vsz_limit = 256 M } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service pop3-postlogin { executable = script-login /usr/local/bin/postlogin_pop3.sh } service pop3 { executable = pop3 pop3-postlogin process_limit = 1024 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { mode = 0666 user = vmail } user = vmail } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl_ca = References: <1766B7F6-4A32-41DA-9655-13E0AE43A8C7@iki.fi> Message-ID: Awesome, thanks. Will this appear in the 2.1.5 release? On Wed, Apr 18, 2012 at 12:57 PM, Timo Sirainen wrote: > On 17.4.2012, at 21.36, Joe Beaubien wrote: > > > I am currently testing a 2.1 dovecot setup with fts-solr. Search speed is > > simply amazing. I decided to try to delete and rebuild the solr indexes, > > but i can't seem to rebuild them thru dovecot. > > With Maildir you could just delete dovecot.index* files. But with dbox > that's not a good idea. Anyway, implemented now the minimum needed to get > doveadm fts rescan to work: > > http://hg.dovecot.org/dovecot-2.1/rev/d145ea6f7061 > > From tss at iki.fi Wed Apr 18 22:43:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 22:43:46 +0300 Subject: [Dovecot] Re-indexing of fts-solr not working In-Reply-To: References: <1766B7F6-4A32-41DA-9655-13E0AE43A8C7@iki.fi> Message-ID: Sure. Everything in hg goes to the next release. On 18.4.2012, at 22.41, Joe Beaubien wrote: > Awesome, thanks. > > Will this appear in the 2.1.5 release? > > > On Wed, Apr 18, 2012 at 12:57 PM, Timo Sirainen wrote: > >> On 17.4.2012, at 21.36, Joe Beaubien wrote: >> >>> I am currently testing a 2.1 dovecot setup with fts-solr. Search speed is >>> simply amazing. I decided to try to delete and rebuild the solr indexes, >>> but i can't seem to rebuild them thru dovecot. >> >> With Maildir you could just delete dovecot.index* files. But with dbox >> that's not a good idea. Anyway, implemented now the minimum needed to get >> doveadm fts rescan to work: >> >> http://hg.dovecot.org/dovecot-2.1/rev/d145ea6f7061 >> >> From jtam.home at gmail.com Thu Apr 19 02:11:17 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 18 Apr 2012 16:11:17 -0700 (PDT) Subject: [Dovecot] alias_for Message-ID: I noticed the addition of a new namespace configuration "alias_for". What pracical difference is adding this over and above making namespace aliases hidden and disabling list, as in older versions of dovecot configurations? Joseph Tam From janfrode at tanso.net Thu Apr 19 11:22:45 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 19 Apr 2012 10:22:45 +0200 Subject: [Dovecot] migrate 15K users to new domain name Message-ID: <20120419082245.GA4050@dibs.tanso.net> I need to migrate 15K users to a new domain name, and plan to use "dsync mirror" in the transition phase. Could someone confirm that this should work: Before giving users access to new-domain do a first sync to get all the stale data over: for user in $old-domain; do dsync mirror $user at old-domain $user at new-domain done Configure sieve vacation filter to forward all messages from $user at old-domain to $user at new-domain, and notify sender of changed address. Give users access to both new-domain and old-domain, and do a final sync. for user in $old-domain; do dsync mirror $user at old-domain $user at new-domain dsync mirror $user at old-domain $user at new-domain # twice in case the first was slow drop all messages for $user at old-domain Leave notice message for $user at old-domain saying he should use new-domain done Wait a few weeks/months, and then drop all users at old-domain. Does this look sensible? -jf From dchenusa at yahoo.com Thu Apr 19 11:44:35 2012 From: dchenusa at yahoo.com (D Chen) Date: Thu, 19 Apr 2012 01:44:35 -0700 (PDT) Subject: [Dovecot] IMAP SSL incoming test need help ! Message-ID: <1334825075.56841.YahooMailNeo@web161604.mail.bf1.yahoo.com> Ubuntu 11.10 server with postfix/dovecot/squirrelmail configured. Outgoing SMTP server seemed worked fine, however, the incoming IMAP with SSL on port 993 doesn't seem to receive any mail at all !? ?Here is the mail.log during the test session from an external yahoo mail (dchenusa at yahoo.com) to the internal server testmail.biokeyinc.com (dchen at testmail.biokeyinc.com) The log did NOT show any error ! ?The sender showed sent mail! but no mail reach to the destination! Any help would be appreciated. Apr 19 01:20:02 testmail postfix/smtpd[4103]: connect from nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:03 testmail postfix/smtpd[4103]: 091ED200973: client=nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:03 testmail postfix/cleanup[4108]: 091ED200973: message-id=<32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com> Apr 19 01:20:03 testmail postfix/qmgr[2007]: 091ED200973: from=, size=2840, nrcpt=1 (queue active) Apr 19 01:20:03 testmail postfix/smtpd[4103]: disconnect from nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:12 testmail postfix/smtpd[4113]: connect from localhost[127.0.0.1] Apr 19 01:20:12 testmail postfix/smtpd[4113]: 33AE9200AE9: client=localhost[127.0.0.1] Apr 19 01:20:12 testmail postfix/cleanup[4108]: 33AE9200AE9: message-id=<32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com> Apr 19 01:20:12 testmail postfix/qmgr[2007]: 33AE9200AE9: from=, size=3536, nrcpt=1 (queue active) Apr 19 01:20:12 testmail postfix/smtpd[4113]: disconnect from localhost[127.0.0.1] Apr 19 01:20:12 testmail amavis[2042]: (02042-06) Passed CLEAN, [98.138.91.180] [98.138.87.4] -> , Message-ID: <32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com>, mail_id: rqyVZiQDwSsG, Hits: -0.109, size: 2837, queued_as: 33AE9200AE9, dkim_id=dchenusa at yahoo.com, at yahoo.com, 8647 ms Apr 19 01:20:12 testmail postfix/smtp[4109]: 091ED200973: to=, relay=127.0.0.1[127.0.0.1]:10024, delay=9.5, delays=0.87/0.01/0/8.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 33AE9200AE9) Apr 19 01:20:12 testmail postfix/qmgr[2007]: 091ED200973: removed Apr 19 01:20:12 testmail postfix/local[4114]: 33AE9200AE9: to=, relay=local, delay=0.18, delays=0.1/0/0/0.07, dsn=2.0.0, status=sent (delivered to command: procmail -a "$EXTENSION") Apr 19 01:20:12 testmail postfix/qmgr[2007]: 33AE9200AE9: removed Apr 19 01:21:30 testmail dovecot: imap-login: Login: user=, method=PLAIN, rip=63.195.90.22, lip=192.168.20.100, mpid=4122, TLS Apr 19 01:23:13 testmail dovecot: imap(dchen): Disconnected: Disconnected in IDLE bytes=99/708 Apr 19 01:23:23 testmail postfix/anvil[4105]: statistics: max connection rate 1/60s for (smtp:98.138.91.180) at Apr 19 01:20:02 Apr 19 01:23:23 testmail postfix/anvil[4105]: statistics: max connection count 1 for (smtp:98.138.91.180) at Apr 19 01:20:02 Apr 19 01:23:23 testmail postfix/anvil[4105]: statistics: max cache size 1 at Apr 19 01:20:02 From pw at wk-serv.de Thu Apr 19 12:34:50 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 19 Apr 2012 11:34:50 +0200 Subject: [Dovecot] =?utf-8?q?Missing_20-managesieve=2Econf_in_2=2E1=2E4=3F?= In-Reply-To: <4F8E98A7.2040305@rename-it.nl> References: <4F8E8E1A.3080408@wk-serv.de> <4F8E98A7.2040305@rename-it.nl> Message-ID: Hi Stephan, after re-installing the whole system the files have been installed properly. Patrick From stan at hardwarefreak.com Thu Apr 19 12:40:43 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 19 Apr 2012 04:40:43 -0500 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8D69AB.5090706@esiee.fr> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <4F8D69AB.5090706@esiee.fr> Message-ID: <4F8FDD9B.6060007@hardwarefreak.com> On 4/17/2012 8:01 AM, Frank Bonnet wrote: > have 4000/6000 imaps concurent connections during working hours . > > POP3 users will be very few How much disk space do you plan to offer per user mail directory? Will you be using quotas? >>> I need some feedbacks advices of experienced admins >>> I will have to setup in few monthes an email system >>> for approx 50K "intensives" users. >>> >>> The only mandatory thing will be I must use HP proliant servers >>> >>> The operating system will be FreeBSD or Linux Quite a coincidence Frank. It's a shame it has to be an HP solution. I just finished designing a high quality high performance 4U 72 drive server yesterday that will easily handle 15K concurrent IMAP users, for only ~$24K USD, $0.48/user @50K users. So it may not be of interest to you, but maybe to others. It is capable of ~7K random 4KB r/w IOPS sustained, has 10TB net space for an average ~200MB/user mail directory assuming 50K users. The parts for this machine run ~$24K USD at Newegg. I just made the wishlist public so it should be available tomorrow or Friday. I'll provide the link when it's available. All components used are top quality, best available in the channel. The reliability of the properly assembled server will rival that of any HP/Dell/IBM machine. For those not familiar with SuperMicro, they manufacture many of Intel's retail boards and have for a decade+. The majority of the COTS systems used in large academic HPC clusters are built with SuperMicro chassis and motherboards, as well as some 1000+ node US DOE clusters. Here are the basics: 72x 2.5" bay 4U chassis, 3x SAS backplanes each w/redundant expanders: http://www.newegg.com/Product/Product.aspx?Item=N82E16811152212 78x Seagate 10K SAS 300GB drives--includes 6 spares Triple LSI 9261-8i dual port 512MB BBWC RAID controllers each with 2 redundant load balanced connections to a backplane 24 drives per controller for lowest latency, maximum throughput, 1.5GB total write cache, a rebuild affects only one controller, etc SuperMicro mainboard, 2x 6-core 3.3GHz AMD Interlagos Opteron CPUs 64GB Reg ECC DDR3-1066, 8x8GB DIMMs, 34GB/s aggregate bandwidth Dual Intel Quad port GbE NICs, 10 total Intel GbE ports Use the 2 mobo ports for redundant management links Aggregate 4 ports, 2 on each quad NIC, for mail traffic Aggregate the remaining 4 for remote backup, future connection to an iSCSI SAN array, etc Or however works best--having 8 GbEs gives flexibility and these two cards are only $500 of the total 2x Intel 20GB SSD internal fixed drives, hardware mirrored by the onboard LSI SAS chip, for boot/OS The key to performance, and yielding a single file tree, is once again using XFS to take advantage of this large spindle count across 3 RAID controllers. Unlike previous configurations where I recommended using a straight md concatenation of hardware RAID1 pairs, in this case we're going to use a concatenation of 6 hardware RAID10 arrays. There are a couple of reasons for doing so in this case: 1. Using 36 device names in a single md command line is less than intuitive and possibly error prone. Using 6 is more manageable. 2. We have 3 BBWC RAID controllers w/24 drives each. This is a high performance server and will see a high IO load in production. In many cases one would use an external filesystem journal, which we could easily do and get great performance with our mirrored SSDs. However, the SSDs are not backed by BBWC, so a UPS failure or system crash could hose the log journal. So we'll go with the default internal journal which will be backed by the BBWC. Going internal with the log in this mail scenario can cause a serious amount of extra IOPS on the filesystem data section, this being Allocation Group 0. If we did the "normal" RAID1 concat, all the log IO would hit the first RAID1 pair. On this system, the load may hit that spindle pretty hard, making access to mailboxes in AG0 slower than others. With 6 RAID10 arrays in a concat, the internal log writes will be striped across 6 spindles in the first array. With 512MB BBWC backing that array and optimizing writeout, and with delaylog, this will yield optimal log write performance without slowing down mailbox file access in AG0. To create such a setup we'd do something like this, assuming the mobo LSI controller yields sd[ab], and the 6 array devices on the PCIe LSI cards yield sd[cdefgh] 1. Create two RAID10 arrays, each of 12 drives, in the WebBIOS GUI of each LSI card, using a strip size of 32KB which should yield good random r/w performance for any mailbox format. Use the following policies for each array: RW, Normal, Wback, Direct, Disable, No, and use the full size. Create the concatenated md device: $ mdadm -C /dev/md0 -l linear -n 6 /dev/sd[cdefgh] Then we format it with XFS, optimizing the AG layout for our mailbox workload, and allocation write stripe alignment to each hardware array: $ mkfs.xfs -d agcount=24 su=32k sw=6 /dev/md0 This yields 4 AGs per RAID10 array which will minimize the traditional inode64 head seeking overhead on striped arrays, while still yielding fantastic allocation parallelism with 24 AGs. Optimal fstab for MTA queue/mailbox workload, assuming kernel 2.6.39+: /dev/md0 /mail xfs defaults,inode64,nobarrier 0 0 We disable write barriers as we have BBWC. And that 1.5GB of BBWC will yield extremely low Dovecot write latency and throughput. Given the throughput available, if you're running Postfix on this box, you will want to create a directory on this filesystem for the Postfix spool. Postfix puts the spool files in many dozens, hundreds of subdirectories, so you'll get 100% parallelism across all AGs, thus all disks. It's very likely none of you will decide to build this system. My hope is that some of the design concepts and components used, along with the low cost but high performance of this machine, may be educational or simply give people new ideas, steer them in directions they may not have previously considered. -- Stan From odhiambo at gmail.com Thu Apr 19 14:01:44 2012 From: odhiambo at gmail.com (Odhiambo Washington) Date: Thu, 19 Apr 2012 14:01:44 +0300 Subject: [Dovecot] migrate 15K users to new domain name In-Reply-To: <20120419082245.GA4050@dibs.tanso.net> References: <20120419082245.GA4050@dibs.tanso.net> Message-ID: On Thu, Apr 19, 2012 at 11:22, Jan-Frode Myklebust wrote: > I need to migrate 15K users to a new domain name, and plan to use "dsync > mirror" in the transition phase. Could someone confirm that this should > work: > > Before giving users access to new-domain do a first sync to get all the > stale data over: > > for user in $old-domain; do > dsync mirror $user at old-domain $user at new-domain > done > > Configure sieve vacation filter to forward all messages from > $user at old-domain to $user at new-domain, and notify sender of changed > address. > > Give users access to both new-domain and old-domain, and do a final > sync. > > for user in $old-domain; do > dsync mirror $user at old-domain $user at new-domain > dsync mirror $user at old-domain $user at new-domain # twice in > case the first was slow > drop all messages for $user at old-domain > Leave notice message for $user at old-domain saying he should > use new-domain > done > > Wait a few weeks/months, and then drop all users at old-domain. > > > Does this look sensible? > > > -jf > What do you mean by a "new domain" in this context? Is the server changing? Is the storage changing? In my thinking, a domain change is as simple as using a rewrite rule in your MTA. I am sure I didn't understand your requirement and as such my response is equally vague. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I can't hear you -- I'm using the scrambler. Please consider the environment before printing this email. -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 652 bytes Desc: not available URL: From robert at schetterer.org Thu Apr 19 15:09:10 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 14:09:10 +0200 Subject: [Dovecot] doveadm delete folders ? Message-ID: <4F900066.9030808@schetterer.org> Hi , is there a feature like doveadm expunge savedbefore ... for imap folder delete ? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From luc at bigdata.nb.ca Thu Apr 19 15:14:16 2012 From: luc at bigdata.nb.ca (Luc Belliveau) Date: Thu, 19 Apr 2012 09:14:16 -0300 Subject: [Dovecot] old messages being delivered to blackberry In-Reply-To: <5CDED2FA-3654-4378-A2D3-8A0BFAF7B5CC@iki.fi> References: <4F843969.6040806@bigdata.nb.ca> <5CDED2FA-3654-4378-A2D3-8A0BFAF7B5CC@iki.fi> Message-ID: <4F900198.1080703@bigdata.nb.ca> I've switched to NFS instead of GlusterFS, and the problems have disappeared. It's too bad... GlusterFS had some nice features ... On 04/10/2012 12:11 PM, Timo Sirainen wrote: > On 10.4.2012, at 16.45, Luc Belliveau wrote: > >> Several users are reporting old messages (sometime years old) are being delivered as new on their blackberry devices... I've confirmed that this happens to all blackberry users... I believe the problem is related to this error: >> >> Maildir: Expunged message reappeared, giving a new UID > This means: > > 1. readdir() returns maildir file X > 2. Later readdir() doesn't return file X, so Dovecot thinks the mail is expunged and also tells the client the mail is expunged > 3. Later readdir() once again returns file X, so Dovecot can't really do anything except treat it as a new mail. > >> Possibly caused by GlusterFS, the indexes were still getting corrupted, even with only 1 server... so I've moved the indexes to a local directory on each server. I'm no longer getting index related errors... but the expunged error is still occuring... and the error seems to happen a few moments before the BB gets a bunch of messages... so it's very likely related. >> >> I would like to continue using GlusterFS if possible; I haven't found any indication that it is outright unsupported, as long as I'm using Director... > If glusterfs can't do readdir() reliably, there's not a whole lot you can do.. Except maybe switch to another mailbox format. But if you're getting index corruption also with 1 Dovecot server, it doesn't look very promising. > From gedalya at gedalya.net Thu Apr 19 16:19:54 2012 From: gedalya at gedalya.net (Gedalya) Date: Thu, 19 Apr 2012 15:19:54 +0200 Subject: [Dovecot] doveadm delete folders ? In-Reply-To: <4F900066.9030808@schetterer.org> References: <4F900066.9030808@schetterer.org> Message-ID: <4F9010FA.8000704@gedalya.net> On 4/19/2012 2:09 PM, Robert Schetterer wrote: > Hi , > is there a feature like > doveadm expunge savedbefore ... > for imap folder delete ? I was wondering the same thing - when a user moves a whole folder into Trash, doveadm expunge mailbox "Trash.*" savedbefore would eventually remove all the messages, but the folder itself remains a subfolder of Trash. It would make sense to be able to remove subfolders which have been there for more than x days and are now empty. From janfrode at tanso.net Thu Apr 19 16:28:03 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 19 Apr 2012 15:28:03 +0200 Subject: [Dovecot] migrate 15K users to new domain name In-Reply-To: References: <20120419082245.GA4050@dibs.tanso.net> Message-ID: <20120419132803.GA9396@dibs.tanso.net> On Thu, Apr 19, 2012 at 02:01:44PM +0300, Odhiambo Washington wrote: > > What do you mean by a "new domain" in this context? The user's email addresses are changing from username at old.domain to username at new-domain. > Is the server changing? No. > Is the storage changing? The user's home directory is based on the user's email address, so this is changing. > In my thinking, a domain change is as simple as using a rewrite rule in > your MTA. Also the user's login-names needs to change from old to new domain, and all their data needs to move from old to new domain. -jf From cor at xs4all.nl Thu Apr 19 16:37:25 2012 From: cor at xs4all.nl (Cor Bosman) Date: Thu, 19 Apr 2012 15:37:25 +0200 Subject: [Dovecot] multiple Return-Path headers added using lmtp director Message-ID: <72F28E8A-EE19-48B9-AD36-0147DACD73B2@xs4all.nl> Emails arrive with 2 Return-Paths when using lmtp director. Is this something configurable in the director or is this a bug? Return-Path: Delivered-To: Received: from lmtpdirector1.xs4all.net ([194.109.26.176]) by lmtp2.xs4all.net (Dovecot) with LMTP id gOe2E3YSkE/aEAAAYH/+MA for ; Thu, 19 Apr 2012 15:26:14 +0200 Return-Path: Received: from mxdrop-builder.xs4all.nl ([194.109.23.200]) by lmtpdirector1.xs4all.net (Dovecot) with LMTP id CnOvKTkzjU9sJQAAbddgHA ; Thu, 19 Apr 2012 15:26:14 +0200 Regards From robert at schetterer.org Thu Apr 19 16:57:21 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 15:57:21 +0200 Subject: [Dovecot] doveadm delete folders ? In-Reply-To: <4F9010FA.8000704@gedalya.net> References: <4F900066.9030808@schetterer.org> <4F9010FA.8000704@gedalya.net> Message-ID: <4F9019C1.4070106@schetterer.org> Am 19.04.2012 15:19, schrieb Gedalya: > On 4/19/2012 2:09 PM, Robert Schetterer wrote: >> Hi , >> is there a feature like >> doveadm expunge savedbefore ... >> for imap folder delete ? > I was wondering the same thing - when a user moves a whole folder into > Trash, doveadm expunge mailbox "Trash.*" savedbefore would eventually > remove all the messages, but the folder itself remains a subfolder of > Trash. It would make sense to be able to remove subfolders which have > been there for more than x days and are now empty. > jep i.e for all empty and not longer needed archive date folders -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From odhiambo at gmail.com Thu Apr 19 17:03:01 2012 From: odhiambo at gmail.com (Odhiambo Washington) Date: Thu, 19 Apr 2012 17:03:01 +0300 Subject: [Dovecot] migrate 15K users to new domain name In-Reply-To: <20120419132803.GA9396@dibs.tanso.net> References: <20120419082245.GA4050@dibs.tanso.net> <20120419132803.GA9396@dibs.tanso.net> Message-ID: On Thu, Apr 19, 2012 at 16:28, Jan-Frode Myklebust wrote: > On Thu, Apr 19, 2012 at 02:01:44PM +0300, Odhiambo Washington wrote: > > > > What do you mean by a "new domain" in this context? > > The user's email addresses are changing from username at old.domain to > username at new-domain. > That can be handled by address rewrites within the MTA. > > > Is the server changing? > > No. > > > Is the storage changing? > > The user's home directory is based on the user's email address, so this is > changing. > In my setup, I have virtual users. So the home directory is in the /var/spool/virtual/$domain/$user/mdbox How is yours setup? If the domain name changed, from domain1 to domain2, I believe it would be easy to change as follows: cd /var/spool/virtual/ mv $domain1 $domain2 Well, it the size of $domain1 is in TBs, I'd probably do it another way as doing mv is a bit dangerous with very large datasets: cd /var/spool/virtual mkdir domain2 cd domain1 for d in `ls -1`; do mv $d domain2/$d; done [ or something closer] > > In my thinking, a domain change is as simple as using a rewrite rule in > > your MTA. > > Also the user's login-names needs to change from old to new domain, and > all their data needs to move from old to new domain. > > And the login names are stored in a flatfile or db?? Either way, you can do a rename. Dump the database and just use vi to rename old-domain to new-domain, then drop the db and import the dump. mysqldump dbname dbname.sql vi dbname.sql :g/old-domain/s//new-domain/g mysqladmin drop dbname mysqladmin create dbname mysql dbname < dbname.sql You can also edit the flatfile to s/old-domain/new-domain/g cat flatfile | xargs sed -i.BAK 's/old-domain/new-domain/g' Maybe I still don't understand you:-) -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I can't hear you -- I'm using the scrambler. Please consider the environment before printing this email. -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 652 bytes Desc: not available URL: From janfrode at tanso.net Thu Apr 19 17:19:30 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 19 Apr 2012 16:19:30 +0200 Subject: [Dovecot] migrate 15K users to new domain name In-Reply-To: References: <20120419082245.GA4050@dibs.tanso.net> <20120419132803.GA9396@dibs.tanso.net> Message-ID: <20120419141930.GB10069@dibs.tanso.net> On Thu, Apr 19, 2012 at 05:03:01PM +0300, Odhiambo Washington wrote: > > > > In my setup, I have virtual users. So the home directory is in the > /var/spool/virtual/$domain/$user/mdbox > > How is yours setup? mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln > If the domain name changed, from domain1 to domain2, I > believe it would be easy to change as follows: > > cd /var/spool/virtual/ > mv $domain1 $domain2 If I could figure out what the %256LRHu hash is, mv would probably be a very good solution.. > > And the login names are stored in a flatfile or db?? LDAP > Either way, you can do a rename. No, we need to keep the old username/password working, so that all users will get notified of the changed -- even if they take off on a 6 month vacation the day before the change. > > Maybe I still don't understand you:-) You seem to be understanding perfectly well. I've been looking myself blind on dsync mirror, when a simple "mv" probably will work just as well :-) -jf From robert at schetterer.org Thu Apr 19 17:45:34 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 16:45:34 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works In-Reply-To: <4F8EFD77.90103@schetterer.org> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> Message-ID: <4F90250E.3000200@schetterer.org> Am 18.04.2012 19:44, schrieb Robert Schetterer: > Am 18.04.2012 19:01, schrieb Timo Sirainen: >> On 17.4.2012, at 17.40, Robert Schetterer wrote: >> >>> doveadm acl debug >>> >>> shows on both versions >>> >>> Fatal: ACL shared dict iteration failed >> >> Doesn't it show any other error message before this? Either to terminal or to error log? What's your doveconf -n? > > > Hi Timo, > i didnt found more usefull errors before > all are like Debug: acl: No lookup right to mailbox: > but that istn true, i checked it in the filesystem > and tried new acls etc > ( and they working under 2.0.20 ) > > to me it looks like some acl check/list problem > perhaps depend on stuff like in i.e > http://hg.dovecot.org/dovecot-2.1/rev/65a75939ac2c > > as i said the old 2.0.20 works still fine > on the other server > the only change i did was with virtual and namespace inbox > to get special_use working > >> > this is from the dove version > > # 2.1.4 (85ad4baedd43): /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-37-server x86_64 Ubuntu 10.04.4 LTS > auth_cache_negative_ttl = 0 > auth_cache_size = 50 M > auth_debug = yes > auth_debug_passwords = yes > auth_master_user_separator = * > auth_mechanisms = plain login > auth_socket_path = /var/run/dovecot/auth-userdb > auth_verbose = yes > auth_verbose_passwords = plain > auth_worker_max_count = 300 > dict { > acl = mysql:/etc/dovecot/dovecot-dict-acl-sql.conf.ext > expire = mysql:/etc/dovecot/dovecot-dict-expire-sql.conf.ext > quotadict = mysql:/etc/dovecot/dovecot-dict-quota-sql.conf.ext > } > disable_plaintext_auth = no > first_valid_gid = 1001 > first_valid_uid = 1001 > hostname = mail01.example.com > last_valid_gid = 1001 > last_valid_uid = 1001 > listen = * > login_greeting = imap, pop ready > mail_access_groups = vmail > mail_debug = yes > mail_fsync = always > mail_gid = 1001 > mail_location = maildir:~/ > mail_nfs_index = yes > mail_nfs_storage = yes > mail_plugins = " stats zlib expire virtual fts fts_squat mail_log > autocreate notify acl listescape" > mail_privileged_group = vmail > mail_uid = 1001 > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date ihave imapflags notify > mmap_disable = yes > namespace { > list = yes > location = maildir:/usr/local/virtual/%%d/%%u:INDEX=~/shared/%%d/%%u > prefix = shared/%%u/ > separator = / > subscriptions = no > type = shared > } > namespace inbox { > hidden = no > list = yes > location = > mailbox Drafts { > auto = subscribe > special_use = \Drafts > } > mailbox Junk { > auto = subscribe > special_use = \Junk > } > mailbox Sent { > auto = subscribe > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > auto = subscribe > special_use = \Trash > } > mailbox name { > special_use = \Drafts \Junk \Sent \Trash > } > prefix = > separator = / > subscriptions = yes > } > namespace real { > hidden = yes > list = no > location = > prefix = RealMails/ > separator = / > } > namespace virtual { > hidden = yes > list = no > location = virtual:/etc/dovecot/virtual2:LAYOUT=maildir++:INDEX=~/virtual > prefix = virtual/ > separator = / > } > passdb { > args = /etc/dovecot/dovecot-sql-master.conf.ext > driver = sql > master = yes > pass = yes > } > passdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > plugin { > acl = vfile:/etc/dovecot/global-acls:cache_secs=300 > acl_shared_dict = proxy::acl > autocreate = Trash > autocreate2 = Junk > autocreate3 = Sent > autocreate4 = Drafts > autocreate5 = Templates > autocreate6 = Hostmaster-Backup > autocreate7 = archiv-backup-mailspooler > autosubscribe = Trash > autosubscribe2 = Junk > autosubscribe3 = Sent > autosubscribe4 = Drafts > autosubscribe5 = Templates > expire = Trash > expire2 = Trash/* > expire3 = Junk > expire4 = Junk/* > expire5 = Hostmaster-Backup > expire6 = Hostmaster-Backup/* > expire7 = archiv-backup-mailspooler/* > expire_dict = proxy::expire > fts = squat > home = /usr/local/virtual/%d/%u > mail_log_events = delete undelete expunge copy mailbox_delete > mailbox_rename > mail_log_fields = uid box msgid size > mail_log_group_events = yes > quota = dict:::proxy::quotadict > quota_warning = storage=95%% quota-warning 95 %u > quota_warning2 = storage=80%% quota-warning 80 %u > sieve = /usr/local/virtual/%d/%u/dovecot.sieve > sieve_before = /etc/dovecot/sieve/global.sieve > sieve_dir = /usr/local/virtual/%d/%u/sieve > sieve_execute_bin_dir = /usr/lib/dovecot/sieve-execute > sieve_execute_socket_dir = sieve-execute > sieve_extensions = +notify +imapflags > sieve_filter_bin_dir = /usr/lib/dovecot/sieve-filter > sieve_filter_socket_dir = sieve-filter > sieve_global_dir = /etc/dovecot/sieve/ > sieve_global_extensions = +vnd.dovecot.execute +vnd.dovecot.pipe > +vnd.dovecot.filter > sieve_global_path = /etc/dovecot/sieve/global.sieve > sieve_pipe_bin_dir = /usr/lib/dovecot/sieve-pipe > sieve_pipe_socket_dir = sieve-pipe > sieve_plugins = sieve_extprograms > stats_refresh = 30 secs > stats_track_cmds = yes > zlib_save = gz > zlib_save_level = 6 > } > postmaster_address = hostmaster at mail01.example.com > protocols = imap pop3 lmtp sieve > sendmail_path = /usr/lib/sendmail > service anvil { > client_limit = 1000 > } > service auth-worker { > user = root > } > service auth { > client_limit = 1000 > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0666 > user = postfix > } > unix_listener auth-userdb { > group = vmail > mode = 0600 > user = vmail > } > } > service dict { > extra_groups = vmail > unix_listener dict { > group = vmail > mode = 0660 > } > } > service imap-login { > inet_listener imap { > port = 143 > } > inet_listener imaps { > port = 993 > ssl = yes > } > process_min_avail = 8 > service_count = 1 > } > service imap-postlogin { > executable = script-login /usr/local/bin/postlogin_imap.sh > } > service imap { > executable = imap imap-postlogin > process_limit = 1024 > vsz_limit = 256 M > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0660 > user = postfix > } > vsz_limit = 256 M > } > service managesieve-login { > inet_listener { > address = 127.0.0.1 212.52.224.210 212.52.224.205 > port = 2000 > } > vsz_limit = 256 M > } > service pop3-login { > inet_listener pop3 { > port = 110 > } > inet_listener pop3s { > port = 995 > ssl = yes > } > } > service pop3-postlogin { > executable = script-login /usr/local/bin/postlogin_pop3.sh > } > service pop3 { > executable = pop3 pop3-postlogin > process_limit = 1024 > } > service quota-warning { > executable = script /usr/local/bin/quota-warning.sh > unix_listener quota-warning { > mode = 0666 > user = vmail > } > user = vmail > } > service stats { > fifo_listener stats-mail { > mode = 0600 > user = vmail > } > } > ssl_ca = ssl_cert = ssl_key = userdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > userdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > verbose_proctitle = yes > protocol lda { > mail_plugins = quota sieve virtual acl mail_log notify expire listescape > } > protocol imap { > imap_client_workarounds = delay-newmail > imap_id_log = * > imap_id_send = * > imap_idle_notify_interval = 2 mins > imap_logout_format = bytes=%i/%o > imap_max_line_length = 128 k > mail_max_userip_connections = 100 > mail_plugins = imap_acl acl fts fts_squat mail_log autocreate notify > expire virtual quota imap_quota zlib imap_zlib listescape stats imap_stats > } > protocol lmtp { > mail_plugins = quota sieve virtual acl fts fts_squat mail_log notify > expire listescape > } > protocol sieve { > mail_max_userip_connections = 100 > } > protocol pop3 { > mail_max_userip_connections = 100 > mail_plugins = expire virtual quota mail_log fts fts_squat notify > listescape > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > pop3_enable_last = yes > pop3_fast_size_lookups = yes > pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s > pop3_no_flag_updates = no > pop3_reuse_xuidl = no > pop3_uidl_format = %g > } > > > Hi Timo, i did some more tests in shared namespace prefix shared/%%u/ is not working shared/%%n/ is working i cant use that cause of mutiple domains naming convention ,users are user at domain.de etc -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Thu Apr 19 17:59:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 17:59:39 +0300 Subject: [Dovecot] migrate 15K users to new domain name In-Reply-To: <20120419141930.GB10069@dibs.tanso.net> References: <20120419082245.GA4050@dibs.tanso.net> <20120419132803.GA9396@dibs.tanso.net> <20120419141930.GB10069@dibs.tanso.net> Message-ID: On 19.4.2012, at 17.19, Jan-Frode Myklebust wrote: > mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln > > If I could figure out what the %256LRHu hash is, mv would probably be a > very good solution.. With v2.1.4 you could do something like: doveadm -c dummy.conf user -m user at domain where dummy.conf contains the minimum configuration needed: mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln ssl = no It still does a userdb lookup though, so either the user at domain needs to exist in userdb or you need to create a separate dummy Dovecot installation using userdb static that the dummy.conf uses. From tss at iki.fi Thu Apr 19 18:16:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 18:16:37 +0300 Subject: [Dovecot] doveadm delete folders ? In-Reply-To: <4F900066.9030808@schetterer.org> References: <4F900066.9030808@schetterer.org> Message-ID: <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> On 19.4.2012, at 15.09, Robert Schetterer wrote: > Hi , > is there a feature like > doveadm expunge savedbefore ... > for imap folder delete ? Here: http://hg.dovecot.org/dovecot-2.1/rev/2d8bafd11569 http://hg.dovecot.org/dovecot-2.1/rev/782570f644f7 From tss at iki.fi Thu Apr 19 18:23:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 18:23:22 +0300 Subject: [Dovecot] multiple Return-Path headers added using lmtp director In-Reply-To: <72F28E8A-EE19-48B9-AD36-0147DACD73B2@xs4all.nl> References: <72F28E8A-EE19-48B9-AD36-0147DACD73B2@xs4all.nl> Message-ID: On 19.4.2012, at 16.37, Cor Bosman wrote: > Emails arrive with 2 Return-Paths when using lmtp director. Is this something configurable in the director or is this a bug? Fixed: http://hg.dovecot.org/dovecot-2.1/rev/85b3975c9008 From zhbmaillistonly at gmail.com Thu Apr 19 18:30:17 2012 From: zhbmaillistonly at gmail.com (Zhang Huangbin) Date: Thu, 19 Apr 2012 23:30:17 +0800 Subject: [Dovecot] Custom LDAP filter for doveadm-mailbox? Message-ID: <1E42E3B7EAAF4EC4A4D8D70AC3AFB045@gmail.com> Dear Timo, With Dovecot-2.0.16, 'doveadm mailbox list -A' will perform LDAP search with filter "(objectClass=posixAccount)", and get value of attribute 'uid' as mail user. Is it possible to custom the ldap filter and returned attribute name? For example: - LDAP filter: (&(objectClass=mailUser)(accountStatus=active)) - Returned attribute name: mail Did i miss something? Thanks. ---- Zhang Huangbin iRedMail: Open Source Mail Server Solution for Red Hat Enterprise Linux, CentOS, Scientific Linux, Debian, Ubuntu, Mint, Gentoo, openSUSE, FreeBSD: http://www.iredmail.org/ From tss at iki.fi Thu Apr 19 18:41:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 18:41:28 +0300 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works In-Reply-To: <4F90250E.3000200@schetterer.org> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> <4F90250E.3000200@schetterer.org> Message-ID: <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> On 19.4.2012, at 17.45, Robert Schetterer wrote: > Hi Timo, > i did some more tests > in shared namespace > prefix > > shared/%%u/ is not working > > > shared/%%n/ is working > > i cant use that cause of mutiple domains > naming convention ,users are user at domain.de etc I can't really think of why that would make a difference. I did a few tests and couldn't reproduce the problem. Try if changing the location in the shared namespace makes a difference: namespace { list = yes location = maildir:%%h:INDEX=~/shared/%%d/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } BTW. You could remove autocreate plugin by moving the rest of the autocreates to mailbox { auto=create } settings. From tss at iki.fi Thu Apr 19 18:44:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 18:44:07 +0300 Subject: [Dovecot] alias_for In-Reply-To: References: Message-ID: <49931079-28AE-4A18-AE32-1F38735D7255@iki.fi> On 19.4.2012, at 2.11, Joseph Tam wrote: > I noticed the addition of a new namespace configuration "alias_for". > What pracical difference is adding this over and above making namespace > aliases hidden and disabling list, as in older versions of dovecot > configurations? It's used by doveadm and dsync to skip duplicate mailboxes. I've also been thinking that it could be made automatic by comparing which namespaces have identical "location" settings. From tss at iki.fi Thu Apr 19 18:44:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 18:44:56 +0300 Subject: [Dovecot] Custom LDAP filter for doveadm-mailbox? In-Reply-To: <1E42E3B7EAAF4EC4A4D8D70AC3AFB045@gmail.com> References: <1E42E3B7EAAF4EC4A4D8D70AC3AFB045@gmail.com> Message-ID: <41EDF5DE-49DF-4900-86CA-52F8A6B05717@iki.fi> On 19.4.2012, at 18.30, Zhang Huangbin wrote: > With Dovecot-2.0.16, 'doveadm mailbox list -A' will perform LDAP search with filter "(objectClass=posixAccount)", and get value of attribute 'uid' as mail user. Is it possible to custom the ldap filter and returned attribute name? For example: > > - LDAP filter: (&(objectClass=mailUser)(accountStatus=active)) > - Returned attribute name: mail > > Did i miss something? See iterate_* settings in example dovecot-ldap.conf.ext. From news1204.10.weezy at spamgourmet.com Wed Apr 18 20:09:58 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Wed, 18 Apr 2012 19:09:58 +0200 Subject: [Dovecot] namespace hierarchy In-Reply-To: References: <201204010928.41017.anyaddress@gmx.net> Message-ID: Am 04.04.2012 03:47, schrieb Timo Sirainen: > On 1.4.2012, at 10.28, Tom Fernandes wrote: > >> In the new server I would like to have 3 namespaces (private, public, shared) >> and use "/" as separator. Is it possible to keep "INBOX." for private, so that >> the users mailboxes appear below the Inbox, but have public and shared on the >> same hierarchy-level like the Inbox? >> > > If you have any subscriptions=no namespaces, you need a parent namespace that has subscriptions=yes. You could have e.g.: > > namespace { > prefix = INBOX/ > list = no > inbox = yes > } > namespace { > prefix = > list = no > hidden = yes > alias_for = INBOX/ > } > > So now the prefix="" is used for the subscriptions but nothing else really. I tried this, but sadly with dovecot V2.1.1 it is not working. Steps to reproduce: + create empty mail directory + create one folder (Allgemeines/Tabel) in public namespace with doveadm + start mail-client (in my case thunderbird) + try to subscribe to the new public folder Error message in log: Apr 18 18:19:34 elablnmail02 dovecot: imap(ahelmcke): Warning: Subscriptions file /home/mail/user/ahelmcke/subscriptions: Removing invalid entry: Allgemeines/Tabel doveconf -n: # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) ext4 auth_master_user_separator = * first_valid_uid = 400 mail_gid = vmail mail_location = maildir:/home/mail/user/%u mail_uid = vmail mbox_write_locks = fcntl namespace Allgemeines { location = maildir:/home/mail/Allgemeines:INDEX=/home/mail/user/%u/SEEN.Allgemeines prefix = Allgemeines/ separator = / subscriptions = no type = public } namespace dummy { alias_for = INBOX/ hidden = yes list = no location = prefix = separator = / type = private } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile sieve = /var/lib/dovecot/sieve/%u.sieve sieve_global_dir = /var/lib/dovecot/sieve/global/ sieve_global_path = /var/lib/dovecot/sieve/default.sieve } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } service imap-login { inet_listener imap { address = * } inet_listener imaps { address = * } } ssl = required ssl_cert = Hello, after finally having managed to setup up dovecot with virtual users and public folders I tried to get it connected to postfix. The problem is, that I can't get postix to deliver mail via dovecot-lda. After some testing it seems to come down to dovecot-lda behaving different when called as root then when called as user vmail. Basic setting: - dovecot 2.1.1 - all mailboxes owned by user vmail (using mail_uid and mail_gid) Test 1: as user root: /usr/libexec/dovecot/dovecot-lda -d ahelmcke < some_mail working as expected; some_mail gets stored in users INBOX Test 2: same command executed as user vmail. dovecot-lda terminates immediately. syslog-output with mail_debug = yes: Test 1: Apr 19 16:38:04 elablnmail02 dovecot: lda: Debug: auth input: ahelmcke Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: Effective uid=494, gid=491, home=/home/mail/user/ahelmcke Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=/home/mail/user/ahelmcke, alt= Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke/public, index=, control=, inbox=, alt= Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace Allgemeines: type=public, prefix=Allgemeines/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no location=maildir:/home/mail/Allgemeines:INDEX=/home/mail/user/ahelmcke/SEEN.Allgemeines Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/Allgemeines, index=/home/mail/user/ahelmcke/SEEN.Allgemeines, control=, inbox=, alt= Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: none: root=, index=, control=, inbox=, alt= Apr 19 16:51:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Destination address: ahelmcke at elablnmail02.ela-bln.local (source: user at hostname) Apr 19 16:51:37 elablnmail02 dovecot: lda(ahelmcke): msgid=unspecified: saved mail to INBOX Test 2: Apr 19 16:37:37 elablnmail02 dovecot: lda: Debug: auth input: ahelmcke Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Effective uid=494, gid=491, home=/home/mail/user/ahelmcke Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=/home/mail/user/ahelmcke, alt= Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke/public, index=, control=, inbox=, alt= Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, subscriptions=yes location= Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mdbox: access(/home/mail/user/ahelmcke/mdbox, rwx): failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mdbox: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: sdbox: access(/home/mail/user/ahelmcke/sdbox, rwx): failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: sdbox: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir: access(/home/mail/user/ahelmcke/Maildir, rwx): failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has .imap/: stat(/home/mail/user/ahelmcke/mail/.imap) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has inbox: stat(/home/mail/user/ahelmcke/mail/inbox) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has mbox: stat(/home/mail/user/ahelmcke/mail/mbox) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has .imap/: stat(/home/mail/user/ahelmcke/Mail/.imap) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has inbox: stat(/home/mail/user/ahelmcke/Mail/inbox) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has mbox: stat(/home/mail/user/ahelmcke/Mail/mbox) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: sdbox: access(/home/mail/user/ahelmcke/sdbox, rwx): failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: sdbox: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Error: user ahelmcke: Initialization failed: Namespace '': Mail storage autodetection failed with home=/home/mail/user/ahelmcke Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Fatal: Invalid user settings. Refer to server log for more information. doveconf -n # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) auth_master_user_separator = * auth_verbose = yes first_valid_uid = 400 mail_debug = yes mail_gid = vmail mail_home = /home/mail/user/%u mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl namespace Allgemeines { location = maildir:/home/mail/Allgemeines:INDEX=/home/mail/user/%u/SEEN.Allgemeines prefix = Allgemeines/ separator = / subscriptions = no type = public } namespace dummy { alias_for = INBOX/ hidden = yes list = no location = maildir:/home/mail/user/%u/public/ prefix = separator = / type = private } namespace inbox { inbox = yes location = maildir:/home/mail/user/%u prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile sieve = /home/mail/user/%u/sieve.active sieve_dir = /home/mail/user/%u/sieve.scripts sieve_global_dir = /var/lib/dovecot/sieve/global/ sieve_global_path = /var/lib/dovecot/sieve/default.sieve } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = * } inet_listener imaps { address = * } } service managesieve-login { inet_listener sieve { address = * port = 4190 } } ssl = required ssl_cert = References: <1E42E3B7EAAF4EC4A4D8D70AC3AFB045@gmail.com> <41EDF5DE-49DF-4900-86CA-52F8A6B05717@iki.fi> Message-ID: <16B8C19B02364F77908C1521F76E4AA8@gmail.com> On Thursday, April 19, 2012 at 11:44 PM, Timo Sirainen wrote: > See iterate_* settings in example dovecot-ldap.conf.ext. It works as expected. Thanks. :) ---- Zhang Huangbin iRedMail: Open Source Mail Server Solution for Red Hat Enterprise Linux, CentOS, Scientific Linux, Debian, Ubuntu, Mint, Gentoo, openSUSE, FreeBSD: http://www.iredmail.org/ From tss at iki.fi Thu Apr 19 18:53:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 18:53:32 +0300 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: References: Message-ID: On 19.4.2012, at 17.56, Andreas Helmcke wrote: > namespace inbox { > inbox = yes > location = maildir:/home/mail/user/%u > prefix = INBOX/ > separator = / > type = private > } Move the location from the above namespace to global: mail_location = maildir:/home/mail/user/%u Although it probably still shouldn't have failed.. From news1204.10.weezy at spamgourmet.com Thu Apr 19 19:09:52 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Thu, 19 Apr 2012 18:09:52 +0200 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: References: Message-ID: <2vp569xe2o.ln2@mail.ela-soft.com> Am 19.04.2012 17:53, schrieb Timo Sirainen: > On 19.4.2012, at 17.56, Andreas Helmcke wrote: > >> namespace inbox { >> location = maildir:/home/mail/user/%u >> .... > > Move the location from the above namespace to global: > > mail_location = maildir:/home/mail/user/%u Actually it /had/ been there and I moved it to inbox while trying to find the problem. So, change reverted (excerpt from doveconf -n): mail_location = maildir:/home/mail/user/%u namespace inbox { inbox = yes location = prefix = INBOX/ separator = / type = private } Now dovecot-lda is complaining about duplicate namespaces (still only when run as vmail; run it as root and it is working): Apr 19 17:58:07 elablnmail02 dovecot: lda: Debug: auth input: ahelmcke Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Effective uid=494, gid=491, home=/home/mail/user/ahelmcke Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=/home/mail/user/ahelmcke, alt= Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke/public, index=, control=, inbox=, alt= Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=, alt= Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Error: user ahelmcke: Initialization failed: namespace configuration error: Duplicate namespace prefix: "" Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Fatal: Invalid user settings. Refer to server log for more information. Greetings Andreas From robert at schetterer.org Thu Apr 19 19:30:57 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 18:30:57 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works In-Reply-To: <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> <4F90250E.3000200@schetterer.org> <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> Message-ID: <4F903DC1.5070501@schetterer.org> Am 19.04.2012 17:41, schrieb Timo Sirainen: > On 19.4.2012, at 17.45, Robert Schetterer wrote: > >> Hi Timo, >> i did some more tests >> in shared namespace >> prefix >> >> shared/%%u/ is not working >> >> >> shared/%%n/ is working >> >> i cant use that cause of mutiple domains >> naming convention ,users are user at domain.de etc > > I can't really think of why that would make a difference. I did a few tests and couldn't reproduce the problem. Try if changing the location in the shared namespace makes a difference: > > namespace { > list = yes > location = maildir:%%h:INDEX=~/shared/%%d/%%u hi Timo, > prefix = shared/%%u/ is not working !!! under 2.1 / it works under 2.0.20 prefix = shared/%%n/ is working but then i see only users of my domain acl shared with their user part of i.e user at domain.de --shared | user ( of my own domain ) i cant use this because i have to share cross domain and users may have same names in different domains what i need is --shared | user at domainwhatever.de > separator = / > subscriptions = no > type = shared > } > > BTW. You could remove autocreate plugin by moving the rest of the autocreates to mailbox { auto=create } settings. i know this nevertheless i try location change -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From almarzuki2001 at hotmail.com Thu Apr 19 21:10:33 2012 From: almarzuki2001 at hotmail.com (Hadi Salem) Date: Thu, 19 Apr 2012 21:10:33 +0300 Subject: [Dovecot] dovecot LDA with sendmail Message-ID: Hi, Im configuring sendmail with dovecot for virtual users using password file and file for user name. on centos. dovecot-1.0.7-7.el5 sendmail-8.13.8-2.el5 dovecot ?n # 1.0.7: /etc/dovecot.conf base_dir: /var/run/dovecot/ log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot.log ssl_disable: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login default_mail_env: maildir:/home/vmail/%d/%n mail_location: maildir:/home/vmail/%d/%n mail_debug: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib64/dovecot/imap mail_plugin_dir(imap): /usr/lib64/dovecot/imap mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3 auth default: verbose: yes debug: yes debug_passwords: yes passdb: driver: passwd-file args: /etc/dovecot/passwd userdb: driver: passwd-file args: /etc/dovecot/users socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: root group: root I have also added dovecot LDA for sendmail /usr/share/sendmail-cf/mailer/dovecot.m4 ######################*****############## ### DOVECOT Mailer specification ### ##################*****################## Mdovecot, P=/usr/libexec/dovecot/deliver, F=DFMPhnu9,S=EnvFromSMTP/HdrFromSMTP, R=EnvToSMTP/HdrFromSMTP,T=DNS/RFC822/X-Unix,A=/usr/libexec/dovecot/deliver -d $u And also [root at host1 mail]# cat /etc/mail/mailertable example.com dovecot:dovecot however when im sending email to the virtual user im getting this error, Apr 19 20:52:13 host1 sendmail[22311]: q3JHqDAR022309: to=, ctladdr= (0/0), delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120292, relay=dovecot, dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/libexec/dovecot/deliver) exited with EX_TEMPFAIL Dovecot LDA can?t deliver the mail to the user!!! From robert at schetterer.org Thu Apr 19 21:36:28 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 20:36:28 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works In-Reply-To: <4F903DC1.5070501@schetterer.org> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> <4F90250E.3000200@schetterer.org> <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> <4F903DC1.5070501@schetterer.org> Message-ID: <4F905B2C.8040204@schetterer.org> Am 19.04.2012 18:30, schrieb Robert Schetterer: > Am 19.04.2012 17:41, schrieb Timo Sirainen: >> On 19.4.2012, at 17.45, Robert Schetterer wrote: >> >>> Hi Timo, >>> i did some more tests >>> in shared namespace >>> prefix >>> >>> shared/%%u/ is not working >>> >>> >>> shared/%%n/ is working >>> >>> i cant use that cause of mutiple domains >>> naming convention ,users are user at domain.de etc >> >> I can't really think of why that would make a difference. I did a few tests and couldn't reproduce the problem. Try if changing the location in the shared namespace makes a difference: >> >> namespace { >> list = yes >> location = maildir:%%h:INDEX=~/shared/%%d/%%u > > hi Timo, > >> prefix = shared/%%u/ > is not working !!! under 2.1 / it works under 2.0.20 > > prefix = shared/%%n/ > is working > > > > but then i see only users of my domain acl shared > with their user part of i.e user at domain.de > > --shared > | > user ( of my own domain ) > > i cant use this because i have to share cross domain > and users may have same names in different domains > > > what i need is > > --shared > | > user at domainwhatever.de > >> separator = / >> subscriptions = no >> type = shared >> } >> > >> BTW. You could remove autocreate plugin by moving the rest of the autocreates to mailbox { auto=create } settings. > i know this > > nevertheless i try location change > just for info location change to location = maildir:%%h:INDEX=~/shared/%%d/%%u does not lead to get it work prefix = shared/%%u/ simply dont works i have reverted back to exact namespace config of the 2.0.20 server, same result, so something broken in 2.1 -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Thu Apr 19 23:00:40 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 22:00:40 +0200 Subject: [Dovecot] doveadm delete folders ? In-Reply-To: <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> References: <4F900066.9030808@schetterer.org> <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> Message-ID: <4F906EE8.1030900@schetterer.org> Am 19.04.2012 17:16, schrieb Timo Sirainen: > On 19.4.2012, at 15.09, Robert Schetterer wrote: > >> Hi , >> is there a feature like >> doveadm expunge savedbefore ... >> for imap folder delete ? > > Here: > > http://hg.dovecot.org/dovecot-2.1/rev/2d8bafd11569 > http://hg.dovecot.org/dovecot-2.1/rev/782570f644f7 > thx Timo ! -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Thu Apr 19 23:27:21 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 22:27:21 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works / solved by remove listescape In-Reply-To: <4F905B2C.8040204@schetterer.org> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> <4F90250E.3000200@schetterer.org> <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> <4F903DC1.5070501@schetterer.org> <4F905B2C.8040204@schetterer.org> Message-ID: <4F907529.2020403@schetterer.org> > > prefix = shared/%%u/ simply dont works > > i have reverted back to exact namespace config > of the 2.0.20 server, same result, so something broken in 2.1 > > Hi Timo problem solved by removing listescape plugin the acl paths did / instead of dot in domainname any chance to get listescape fixed ? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From stan at hardwarefreak.com Fri Apr 20 03:31:13 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 19 Apr 2012 19:31:13 -0500 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <20120417200820.GA30743@dibs.tanso.net> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <20120417124241.GA23120@dibs.tanso.net> <4F8D6BAA.7060501@gtdinternet.com> <20120417200820.GA30743@dibs.tanso.net> Message-ID: <4F90AE51.8080407@hardwarefreak.com> On 4/17/2012 3:08 PM, Jan-Frode Myklebust wrote: > Our struggle is the number of > iops we're able to get from the backend storage (IBM DS4800), mostly > a problem when we have storms of incoming marketing messages in addition > to the pop/imap traffic. This issue has come up twice on the Postfix list in less than a month. You can fix this specific problem very easily. Only marketing servers and busy/misconfigured list servers make many parallel connections to your MX hosts. Allowing them to blast all those messages over parallel connections is what bogs down your spool storage. The fix is simple: limit all SMTP clients to a small number of parallel connections. This will slow down marketing and list server blasts without affecting normal sending MTAs. To do so, add this to /etc/postfix/main.cf: smtpd_client_connection_count_limit = 4* The default Postfix process limit is 100. The concurrent connection limit is 1/2 the process limit, so 50 parallel connections per client IP are allowed by default. If remote hosts also do connection caching, they can force feed your MTA many hundreds of messages/sec. Limiting concurrent connections will decrease their mail rate to a small fraction of what you're seeing now, reducing IOPS load on your spool storage significantly. * This is a good starting point. You may need to tweak it up a little bit. Some list servers (such as XFS) will unsub members if their multiple connections keep getting refused, so tweak this value until you find your sweet spot. -- Stan From dchenusa at yahoo.com Fri Apr 20 03:38:01 2012 From: dchenusa at yahoo.com (Dennis Chen) Date: Thu, 19 Apr 2012 17:38:01 -0700 Subject: [Dovecot] Fwd: IMAP SSL incoming test need help ! References: <1334876809.51362.YahooMailNeo@web161602.mail.bf1.yahoo.com> Message-ID: <82BE5266-7A1D-4E1F-9862-F503668D6EAF@yahoo.com> Sent from my iPhone Begin forwarded message: > From: D Chen > Date: April 19, 2012 4:06:49 PM PDT > To: lists-dovecot > Subject: Re: [Dovecot] IMAP SSL incoming test need help ! > Reply-To: D Chen > > Thanks for nice pointing out in my mail.log! > > I could not find where is the procmail log located under /var/log ! > > As procmail is "suspicious", I recalled that I selected either use procmail or not use for local delivery while I re-configure the postfix. I ran "sudo dpkg-reconfigure postfix" again, and selected NOT use "Procmail" for local delivery, restart postfix and ran the tested again, it worked now ! examined the mail.log, it said "delivered to maildir" instead of procmail. > > why works NOT using procmail ? thx. > > From: lists-dovecot > To: D Chen > Sent: Thursday, April 19, 2012 5:22 AM > Subject: Re: [Dovecot] IMAP SSL incoming test need help ! > > > > ------------ Original Message ------------ > > Date: Thursday, April 19, 2012 01:44:35 AM -0700 > > From: D Chen > > To: Dovecot Mailing List > > Subject: [Dovecot] IMAP SSL incoming test need help ! > > > > Ubuntu 11.10 server with postfix/dovecot/squirrelmail configured. > > > > Outgoing SMTP server seemed worked fine, however, the incoming > > IMAP with SSL on port 993 doesn't seem to receive any mail at all > > !? Here is the mail.log during the test session from an external > > yahoo mail (dchenusa at yahoo.com) to the internal server > > testmail.biokeyinc.com (dchen at testmail.biokeyinc.com) The log did > > NOT show any error ! The sender showed sent mail! but no mail > > reach to the destination! Any help would be appreciated. > > > > Apr 19 01:20:02 testmail postfix/smtpd[4103]: connect from > > nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:03 > > testmail postfix/smtpd[4103]: 091ED200973: > > client=nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 > > 01:20:03 testmail postfix/cleanup[4108]: 091ED200973: > > message-id=<32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com> Apr 19 > > 01:20:03 testmail postfix/qmgr[2007]: 091ED200973: > > from=, size=2840, nrcpt=1 (queue active) Apr > > 19 01:20:03 testmail postfix/smtpd[4103]: disconnect from > > nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:12 > > testmail postfix/smtpd[4113]: connect from localhost[127.0.0.1] > > Apr 19 01:20:12 testmail postfix/smtpd[4113]: 33AE9200AE9: > > client=localhost[127.0.0.1] Apr 19 01:20:12 testmail > > postfix/cleanup[4108]: 33AE9200AE9: > > message-id=<32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com> Apr 19 > > 01:20:12 testmail postfix/qmgr[2007]: 33AE9200AE9: > > from=, size=3536, nrcpt=1 (queue active) Apr > > 19 01:20:12 testmail postfix/smtpd[4113]: disconnect from > > localhost[127.0.0.1] Apr 19 01:20:12 testmail amavis[2042]: > > (02042-06) Passed CLEAN, [98.138.91.180] [98.138.87.4] > > -> , > > Message-ID: <32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com>, > > mail_id: rqyVZiQDwSsG, Hits: -0.109, size: 2837, queued_as: > > 33AE9200AE9, dkim_id=dchenusa at yahoo.com, at yahoo.com, 8647 ms Apr 19 > > 01:20:12 testmail postfix/smtp[4109]: 091ED200973: > > to=, > > relay=127.0.0.1[127.0.0.1]:10024, delay=9.5, > > delays=0.87/0.01/0/8.6, dsn=2.0.0, status=sent (250 2.0.0 from > > MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 33AE9200AE9) Apr > > 19 01:20:12 testmail postfix/qmgr[2007]: 091ED200973: removed Apr > > 19 01:20:12 testmail postfix/local[4114]: 33AE9200AE9: > > to=, relay=local, delay=0.18, > > delays=0.1/0/0/0.07, dsn=2.0.0, status=sent (delivered to command: > > procmail -a "$EXTENSION") Apr 19 01:20:12 testmail > > postfix/qmgr[2007]: 33AE9200AE9: removed Apr 19 01:21:30 testmail > > dovecot: imap-login: Login: user=, method=PLAIN, > > rip=63.195.90.22, lip=192.168.20.100, mpid=4122, TLS Apr 19 > > 01:23:13 testmail dovecot: imap(dchen): Disconnected: Disconnected > > in IDLE bytes=99/708 Apr 19 01:23:23 testmail postfix/anvil[4105]: > > statistics: max connection rate 1/60s for (smtp:98.138.91.180) at > > Apr 19 01:20:02 Apr 19 01:23:23 testmail postfix/anvil[4105]: > > statistics: max connection count 1 for (smtp:98.138.91.180) at Apr > > 19 01:20:02 Apr 19 01:23:23 testmail postfix/anvil[4105]: > > statistics: max cache size 1 at Apr 19 01:20:02 > > ------------ End Original Message ------------ > > > I'm going to bet that procmail is eating the inbound message: > > Apr 19 01:20:12 testmail postfix/local[4114]: 33AE9200AE9: > to=, relay=local, delay=0.18, > delays=0.1/0/0/0.07, dsn=2.0.0, status=sent (delivered to command: > > procmail -a "$EXTENSION") > > and it's not getting to the user's inbox. If a message doesn't get > to the user's inbox then it's not dovecot's issue that it can't be > retrieved. > > fyi -- imap/dovecot have nothing to do your inbound delivery. > > > however, the incoming IMAP with SSL on port 993 doesn't > > seem to receive any mail > > that's handled by your MTA, which in your case is postfix, with > procmail in the mix. > > I would try looking at the procmail log to see what's happening > there. > > - Richard > > > > > > > > > > > From jtam.home at gmail.com Fri Apr 20 04:45:38 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Thu, 19 Apr 2012 18:45:38 -0700 (PDT) Subject: [Dovecot] dovecot LDA with sendmail In-Reply-To: References: Message-ID: Hadi Salem writes: > Im configuring sendmail with dovecot for virtual users using > password file and file for user name. on centos. > > dovecot-1.0.7-7.el5 Maybe I can beat Charles Marcus to the punch and recommend you upgrade to the latest version, otherwise you'll run into bugs that have already been fixed. > Mdovecot, > P=/usr/libexec/dovecot/deliver, F=DFMPhnu9,S=EnvFromSMTP/HdrFromSMTP, > R=EnvToSMTP/HdrFromSMTP,T=DNS/RFC822/X-Unix,A=/usr/libexec/dovecot/deliver -d > $u > ... > Apr 19 20:52:13 host1 sendmail[22311]: q3JHqDAR022309: > to=, ctladdr= (0/0), > delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120292, relay=dovecot, > dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/libexec/dovecot/deliver) exited > with EX_TEMPFAIL Is there any logs from deliver? If not, look at this page to configure logging http://wiki.dovecot.org/LDA That's your best chance at finding out the problem. I don't have virtual users, so I can't venture to say whether this is part of the problem. I know I had to fiddle with the mailer flags (you have "DFMPhnu9") to get my setup working, but I don't remember what I did or why. Try adding the "S" flag and see if that helps. Joseph Tam From dovecot-user at spambox.dk Fri Apr 20 10:40:02 2012 From: dovecot-user at spambox.dk (Henrik Larsson) Date: Fri, 20 Apr 2012 08:40:02 +0100 Subject: [Dovecot] 2.1.3 Corrupted squat uidlist In-Reply-To: <4F8A97F8.70808@spambox.dk> References: <4F7164DC.7010706@unict.it> <4F8A97F8.70808@spambox.dk> Message-ID: <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> This problem is still showing up every day in the logs. I tried to delete the dovecot.index.search and dovecot.index.search.uids files for all users, but the errors are still there after a re-build of the search index with "/usr/local/bin/doveadm search -A TEXT xyzabczzz". If you need any further details, please let me know. Best regards Henrik Larsson On 15-04-2012 10:42, Henrik Larsson wrote: > On 27-03-2012 08:57, Luca Palazzo wrote: >> Hi Timo and All, >> after upgrading to 2.1.2 i'm getting a lot of these messages: >> Error: Corrupted squat uidlist file XXXXXX wrong indexid > After an upgrade to 2.1.3 i see this as well. Any thoughts? > > Apr 15 03:43:43 imap(xxxx): Error: Corrupted squat uidlist file > /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong > indexid > Apr 15 06:30:27 imap(xxxx): Error: Corrupted squat uidlist file > /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong > indexid > Apr 15 07:43:55 pop3(xxxx): Error: Corrupted squat uidlist file > /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong > indexid > Apr 15 09:00:01 imap(xxxx): Error: Corrupted squat uidlist file > /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong > indexid > > # doveconf -n > # 2.1.3: /usr/local/etc/dovecot/dovecot.conf > # OS: FreeBSD 8.2-STABLE amd64 > auth_mechanisms = plain login digest-md5 cram-md5 > first_valid_uid = 125 > listen = * > log_path = /var/log/dovecot > mail_plugins = fts fts_squat zlib > mail_privileged_group = postfix > mail_temp_dir = /var/db/dovecot > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date ihave > namespace { > inbox = yes > location = > prefix = > separator = . > type = private > } > namespace { > hidden = yes > inbox = no > list = no > location = > prefix = INBOX. > separator = . > type = private > } > passdb { > args = /usr/local/etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > plugin { > fts = squat > fts_squat = partial=4 full=10 > sieve = ~/.dovecot.sieve > sieve_dir = ~/sieve > } > protocols = imap pop3 lmtp sieve > service auth-worker { > user = $default_internal_user > } > service auth { > unix_listener /home/mail/postfix/private/dovecot-auth { > group = postfix > mode = 0660 > user = postfix > } > } > service lmtp { > executable = lmtp -L > unix_listener /home/mail/postfix/private/dovecot-lmtp { > group = postfix > mode = 0660 > user = postfix > } > } > ssl_cert = ssl_key = userdb { > args = /usr/local/etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > protocol lmtp { > log_path = /var/log/dovecot-deliver > mail_plugins = fts fts_squat zlib sieve > } > protocol lda { > mail_plugins = fts fts_squat zlib sieve > } > protocol imap { > mail_plugins = fts fts_squat zlib imap_zlib > } > > > Best regards > Henrik Larsson From dchenusa at yahoo.com Fri Apr 20 11:29:31 2012 From: dchenusa at yahoo.com (Dennis Chen) Date: Fri, 20 Apr 2012 01:29:31 -0700 Subject: [Dovecot] Fwd: IMAP SSL incoming test need help ! References: <82BE5266-7A1D-4E1F-9862-F503668D6EAF@yahoo.com> Message-ID: <485DD6C9-EFCB-4644-9BFA-D0C0CCA224EA@yahoo.com> Sent from my iPhone Begin forwarded message: > From: Dennis Chen > Date: April 19, 2012 5:38:01 PM PDT > To: dovecot at dovecot.org > Subject: Fwd: [Dovecot] IMAP SSL incoming test need help ! > > > > Sent from my iPhone > > Begin forwarded message: > >> From: D Chen >> Date: April 19, 2012 4:06:49 PM PDT >> To: lists-dovecot >> Subject: Re: [Dovecot] IMAP SSL incoming test need help ! >> Reply-To: D Chen >> > >> Thanks for nice pointing out in my mail.log! >> >> I could not find where is the procmail log located under /var/log ! >> >> As procmail is "suspicious", I recalled that I selected either use procmail or not use for local delivery while I re-configure the postfix. I ran "sudo dpkg-reconfigure postfix" again, and selected NOT use "Procmail" for local delivery, restart postfix and ran the tested again, it worked now ! examined the mail.log, it said "delivered to maildir" instead of procmail. >> >> why works NOT using procmail ? thx. >> >> From: lists-dovecot >> To: D Chen >> Sent: Thursday, April 19, 2012 5:22 AM >> Subject: Re: [Dovecot] IMAP SSL incoming test need help ! >> >> >> >> ------------ Original Message ------------ >> > Date: Thursday, April 19, 2012 01:44:35 AM -0700 >> > From: D Chen >> > To: Dovecot Mailing List >> > Subject: [Dovecot] IMAP SSL incoming test need help ! >> > >> > Ubuntu 11.10 server with postfix/dovecot/squirrelmail configured. >> > >> > Outgoing SMTP server seemed worked fine, however, the incoming >> > IMAP with SSL on port 993 doesn't seem to receive any mail at all >> > !? Here is the mail.log during the test session from an external >> > yahoo mail (dchenusa at yahoo.com) to the internal server >> > testmail.biokeyinc.com (dchen at testmail.biokeyinc.com) The log did >> > NOT show any error ! The sender showed sent mail! but no mail >> > reach to the destination! Any help would be appreciated. >> > >> > Apr 19 01:20:02 testmail postfix/smtpd[4103]: connect from >> > nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:03 >> > testmail postfix/smtpd[4103]: 091ED200973: >> > client=nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 >> > 01:20:03 testmail postfix/cleanup[4108]: 091ED200973: >> > message-id=<32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com> Apr 19 >> > 01:20:03 testmail postfix/qmgr[2007]: 091ED200973: >> > from=, size=2840, nrcpt=1 (queue active) Apr >> > 19 01:20:03 testmail postfix/smtpd[4103]: disconnect from >> > nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:12 >> > testmail postfix/smtpd[4113]: connect from localhost[127.0.0.1] >> > Apr 19 01:20:12 testmail postfix/smtpd[4113]: 33AE9200AE9: >> > client=localhost[127.0.0.1] Apr 19 01:20:12 testmail >> > postfix/cleanup[4108]: 33AE9200AE9: >> > message-id=<32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com> Apr 19 >> > 01:20:12 testmail postfix/qmgr[2007]: 33AE9200AE9: >> > from=, size=3536, nrcpt=1 (queue active) Apr >> > 19 01:20:12 testmail postfix/smtpd[4113]: disconnect from >> > localhost[127.0.0.1] Apr 19 01:20:12 testmail amavis[2042]: >> > (02042-06) Passed CLEAN, [98.138.91.180] [98.138.87.4] >> > -> , >> > Message-ID: <32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com>, >> > mail_id: rqyVZiQDwSsG, Hits: -0.109, size: 2837, queued_as: >> > 33AE9200AE9, dkim_id=dchenusa at yahoo.com, at yahoo.com, 8647 ms Apr 19 >> > 01:20:12 testmail postfix/smtp[4109]: 091ED200973: >> > to=, >> > relay=127.0.0.1[127.0.0.1]:10024, delay=9.5, >> > delays=0.87/0.01/0/8.6, dsn=2.0.0, status=sent (250 2.0.0 from >> > MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 33AE9200AE9) Apr >> > 19 01:20:12 testmail postfix/qmgr[2007]: 091ED200973: removed Apr >> > 19 01:20:12 testmail postfix/local[4114]: 33AE9200AE9: >> > to=, relay=local, delay=0.18, >> > delays=0.1/0/0/0.07, dsn=2.0.0, status=sent (delivered to command: >> > procmail -a "$EXTENSION") Apr 19 01:20:12 testmail >> > postfix/qmgr[2007]: 33AE9200AE9: removed Apr 19 01:21:30 testmail >> > dovecot: imap-login: Login: user=, method=PLAIN, >> > rip=63.195.90.22, lip=192.168.20.100, mpid=4122, TLS Apr 19 >> > 01:23:13 testmail dovecot: imap(dchen): Disconnected: Disconnected >> > in IDLE bytes=99/708 Apr 19 01:23:23 testmail postfix/anvil[4105]: >> > statistics: max connection rate 1/60s for (smtp:98.138.91.180) at >> > Apr 19 01:20:02 Apr 19 01:23:23 testmail postfix/anvil[4105]: >> > statistics: max connection count 1 for (smtp:98.138.91.180) at Apr >> > 19 01:20:02 Apr 19 01:23:23 testmail postfix/anvil[4105]: >> > statistics: max cache size 1 at Apr 19 01:20:02 >> >> ------------ End Original Message ------------ >> >> >> I'm going to bet that procmail is eating the inbound message: >> >> Apr 19 01:20:12 testmail postfix/local[4114]: 33AE9200AE9: >> to=, relay=local, delay=0.18, >> delays=0.1/0/0/0.07, dsn=2.0.0, status=sent (delivered to command: >> >> procmail -a "$EXTENSION") >> >> and it's not getting to the user's inbox. If a message doesn't get >> to the user's inbox then it's not dovecot's issue that it can't be >> retrieved. >> >> fyi -- imap/dovecot have nothing to do your inbound delivery. >> >> > however, the incoming IMAP with SSL on port 993 doesn't >> > seem to receive any mail >> >> that's handled by your MTA, which in your case is postfix, with >> procmail in the mix. >> >> I would try looking at the procmail log to see what's happening >> there. >> >> - Richard >> >> >> >> >> >> >> >> >> >> >> From john.robinson at anonymous.org.uk Fri Apr 20 12:01:45 2012 From: john.robinson at anonymous.org.uk (John Robinson) Date: Fri, 20 Apr 2012 10:01:45 +0100 Subject: [Dovecot] mbox folders on-disc layout In-Reply-To: <35C2B470-438E-4CEC-A42A-0BC2C39A0159@iki.fi> References: <4F8EA753.4070605@anonymous.org.uk> <35C2B470-438E-4CEC-A42A-0BC2C39A0159@iki.fi> Message-ID: <4F9125F9.6060601@anonymous.org.uk> On 18/04/2012 17:45, Timo Sirainen wrote: > On 18.4.2012, at 14.36, John Robinson wrote: > >> I'd like to change the on-disk layout for my mboxes from being >> ~/mail/foo/bar >> to >> ~/mail/foo.bar >> so that I can have folders containing both messages and subfolders, without having subfolders beginning with . and then having to do all the other fiddling with locations of index files etc. that goes along with LAYOUT=maildir++, and also leaving the filesystem layout looking sensible so my users see their mailboxes instead of the index and control files. >> >> Is this possible or do I need to patch something? > > Not possible without patching. There's also the other possibility of doing something like: > > # Trick mbox configuration which allows a mail folder which contains both > # messages and sub-folders > mail_location = mbox:~/mail/mailboxes:DIRNAME=mBoX-MeSsAgEs:INDEX=~/mail/index:CONTROL=~/mail/control > > from http://wiki2.dovecot.org/MboxChildFolders Yes, I read all that, and thought all the options were pretty ugly - and the comment that it's a "trick" configuration would seem to suggest that the original author thought so to. And I think Maildir++ is pretty ugly too. If I wanted to try writing a new LAYOUT, where should I begin? Cheers, John. From CMarcus at Media-Brokers.com Fri Apr 20 14:07:57 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 20 Apr 2012 07:07:57 -0400 Subject: [Dovecot] dovecot LDA with sendmail In-Reply-To: References: Message-ID: <4F91438D.3040200@Media-Brokers.com> On 2012-04-19 9:45 PM, Joseph Tam wrote: > Hadi Salem writes: >> Im configuring sendmail with dovecot for virtual users using >> password file and file for user name. on centos. >> dovecot-1.0.7-7.el5 > Maybe I can beat Charles Marcus to the punch and recommend you upgrade > to the latest version, otherwise you'll run into bugs that have already > been fixed. Lol... I stopped reading at '...sendmail with dovecot...', otherwise I'd have beat you to it... ;) -- Best regards, Charles From news1204.10.weezy at spamgourmet.com Fri Apr 20 14:11:09 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Fri, 20 Apr 2012 13:11:09 +0200 Subject: [Dovecot] default mail quota when using per user quota Message-ID: Hello, using dovecot 2.1 and per user mail quota via ldap is there a way to have a default quota which gets used, if the "quota" field in ldap is not set? I tried with: plugin { quota = maildir:User quota quota_rule = *:storage=3G } userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } and in dovecot-ldap.conf: user_attrs = dcMailQuota=quota_rule=*:storage=%$ but with this configuration the mail quota /allways/ gets loaded from ldap resulting in users without dcMailQuota attribute having no quota at all. Greetings Andreas From nmilas at noa.gr Fri Apr 20 15:26:51 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Fri, 20 Apr 2012 15:26:51 +0300 Subject: [Dovecot] default mail quota when using per user quota In-Reply-To: References: Message-ID: <4F91560B.5040009@noa.gr> On 20/4/2012 2:11 ??, Andreas Helmcke wrote: > using dovecot 2.1 and per user mail quota via ldap is there a way to have a default quota which gets used, if the > "quota" field in ldap is not set? You may see: http://www.mail-archive.com/dovecot at dovecot.org/msg44010.html Regards, Nick From campbell at cnpapers.com Fri Apr 20 15:30:27 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Fri, 20 Apr 2012 08:30:27 -0400 Subject: [Dovecot] Error in logs indicating broken files Message-ID: <4F9156E3.4050801@cnpapers.com> Still new to dovecot, I'm seeing a couple of errors in my log file indicating corruption of index files. Error: Corrupted index cache file/home/xxxxxx/mail/.imap/sent-mail/dovecot.index.cache: Broken virtual size for mail UID 67: 1 Time(s) Error: FETCH [] for mailbox ~/mail/sent-mail UID 67 got too little data: 6203 vs 7478: 1 Time(s) This account is used by multiple users, but it is not a shared account (haven't set any of those up yet - still new to me), so I would expect corruption possibilities when multiple users modify the account folders in some manner. Until I get some experience under my belt and make this a truly "shared" account, is there any way to resolve the corruption of the files mentioned above? Maybe just delete them? thanks steve campbell From ml at smtp.fakessh.eu Fri Apr 20 15:25:01 2012 From: ml at smtp.fakessh.eu (ml) Date: Fri, 20 Apr 2012 14:25:01 +0200 Subject: [Dovecot] 2.1.3 Corrupted squat uidlist In-Reply-To: <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> References: <4F7164DC.7010706@unict.it> <4F8A97F8.70808@spambox.dk> <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> Message-ID: Le 2012-04-20 09:40, Henrik Larsson a ?crit?: > This problem is still showing up every day in the logs. > > I tried to delete the dovecot.index.search and > dovecot.index.search.uids files for all users, but the errors are > still there after a re-build of the search index with > "/usr/local/bin/doveadm search -A TEXT xyzabczzz". > > If you need any further details, please let me know. > > > Best regards > Henrik Larsson > > > On 15-04-2012 10:42, Henrik Larsson wrote: >> On 27-03-2012 08:57, Luca Palazzo wrote: >>> Hi Timo and All, >>> after upgrading to 2.1.2 i'm getting a lot of these messages: >>> Error: Corrupted squat uidlist file XXXXXX wrong indexid >> After an upgrade to 2.1.3 i see this as well. Any thoughts? >> >> Apr 15 03:43:43 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> Apr 15 06:30:27 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> Apr 15 07:43:55 pop3(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> Apr 15 09:00:01 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> >> # doveconf -n >> # 2.1.3: /usr/local/etc/dovecot/dovecot.conf >> # OS: FreeBSD 8.2-STABLE amd64 >> auth_mechanisms = plain login digest-md5 cram-md5 >> first_valid_uid = 125 >> listen = * >> log_path = /var/log/dovecot >> mail_plugins = fts fts_squat zlib >> mail_privileged_group = postfix >> mail_temp_dir = /var/db/dovecot >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date ihave >> namespace { >> inbox = yes >> location = >> prefix = >> separator = . >> type = private >> } >> namespace { >> hidden = yes >> inbox = no >> list = no >> location = >> prefix = INBOX. >> separator = . >> type = private >> } >> passdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> plugin { >> fts = squat >> fts_squat = partial=4 full=10 >> sieve = ~/.dovecot.sieve >> sieve_dir = ~/sieve >> } >> protocols = imap pop3 lmtp sieve >> service auth-worker { >> user = $default_internal_user >> } >> service auth { >> unix_listener /home/mail/postfix/private/dovecot-auth { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> service lmtp { >> executable = lmtp -L >> unix_listener /home/mail/postfix/private/dovecot-lmtp { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> ssl_cert = > ssl_key = > userdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> protocol lmtp { >> log_path = /var/log/dovecot-deliver >> mail_plugins = fts fts_squat zlib sieve >> } >> protocol lda { >> mail_plugins = fts fts_squat zlib sieve >> } >> protocol imap { >> mail_plugins = fts fts_squat zlib imap_zlib >> } >> >> >> Best regards >> Henrik Larsson comparable error exists for corrupted Maildir with the wrong filename i don't no the exact name of error i do not use zlib for this reason i thinks a patch from timo fix this feature later -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation From news1204.10.weezy at spamgourmet.com Fri Apr 20 17:31:33 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Fri, 20 Apr 2012 16:31:33 +0200 Subject: [Dovecot] default mail quota when using per user quota In-Reply-To: <4F91560B.5040009@noa.gr> References: <4F91560B.5040009@noa.gr> Message-ID: Am 20.04.2012 14:26, schrieb Nikolaos Milas: > > You may see: http://www.mail-archive.com/dovecot at dovecot.org/msg44010.html I do not see any relevant differences to my setup. Do you /really/ have a default quota of 4G if roomNumber is empty or rather have unlimited quota then? Greetings Andreas From cor at xs4all.nl Fri Apr 20 19:05:20 2012 From: cor at xs4all.nl (Cor Bosman) Date: Fri, 20 Apr 2012 18:05:20 +0200 Subject: [Dovecot] quota not being calculated Message-ID: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> It looks like my quota isnt being calculated properly after I started setting quota to a specific folder. The quota in that folder is always starting out at 0, and only new email is being added to the quota. If I remove the maildirsize file, and recalculate, it still starts at 0. Once email arrives in the folder, it gets added to maildirsize. This is with maildirquota, in perhaps a bit of an exotic config. The idea is to set a specific quota to Spam folder, using the trash plugin to keep it at a certain size. . getquotaroot "Spam" * QUOTAROOT "Spam" "User quota" "Spam quota" * QUOTA "User quota" (STORAGE 40848 1228800) * QUOTA "Spam quota" (STORAGE 0 20000) . OK Getquotaroot completed. Once 1 email gets added: .Spam# cat maildirsize 20480000S 0 0 2017 1 namespace { hidden = no ignore_on_failure = no inbox = yes list = yes location = maildir:%h:INDEX=/var/spool/mail/dovecot-control/indexes/%1u/%2u/%u:CONTROL=/var/spool/mail/dovecot-control/%1u/%2u/%u/INBOX prefix = separator = / subscriptions = yes type = private } namespace spam { hidden = yes ignore_on_failure = no inbox = no list = yes location = maildir:%h/.Spam:INDEX=/var/spool/mail/dovecot-control/indexes/%1u/%2u/%u/.Spam:CONTROL=/var/spool/mail/dovecot-control/%1u/%2u/%u/.Spam prefix = Spam/ separator = / subscriptions = no type = private } plugin { quota = fs:User quota quota2 = maildir:Spam quota:ns=Spam/ quota2_rule = *:storage=20000K sieve = /var/spool/mail/dovecot-control/sieve/%1u/%2u/%u/dovecot.sieve sieve_before = /etc/sieve/before sieve_dir = /var/spool/mail/dovecot-control/sieve/%1u/%2u/%u/scripts trash = /etc/dovecot/conf.d/dovecot-trash.conf.ext } (full config: http://pastebin.com/Mui4X7Zh) From tim.ruehsen at gmx.de Fri Apr 20 17:27:54 2012 From: tim.ruehsen at gmx.de (Tim Ruehsen) Date: Fri, 20 Apr 2012 16:27:54 +0200 Subject: [Dovecot] array code issue ? Message-ID: <201204201627.54213.tim.ruehsen@gmx.de> Hi, I just took a look into the dovecot 2.1 sources and just saw a possible issue in array.h. This code snippet as an example: #static inline void * #array_get_modifiable_i(struct array *array, unsigned int *count_r) #{ # *count_r = array->buffer->used / array->element_size; # return buffer_get_modifiable_data(array->buffer, NULL); #} array->buffer->used and array->element_size are of type 'size_t' which is 64bit on amd64 and others while 'count_r' is a 32bit value. At least, I see ugly warnings with -Wconversion (which I personally like to use). I know, it is unlikely that 'array->buffer->used / array->element_size' exceeds 32bit range. But then, dovecot's source is so well written, that the above code seems to disturb dovecot's code aesthetics. And who knows... in a few years (when we have THz and TBytes on our desktops) emails (and array sizes) might exceed everything that we think of today. Tim From mafonso at hangas.net Fri Apr 20 19:19:48 2012 From: mafonso at hangas.net (Miguel Afonso) Date: Fri, 20 Apr 2012 17:19:48 +0100 Subject: [Dovecot] dbox vs. mdbox In-Reply-To: References: <4D69FCB5.3090100@wildgooses.com> <58BED0A6-9EA0-4B4D-9065-69B22033D627@iki.fi> Message-ID: Hi, I finally deployed a migration test plant and done some conversions last night, so I took the chance to collect some data of these tests. I've set up a virtual machine with a dovecot 2.0.13 installation in order test the mailbox conversion. This machine has two adicional disks, one with the source data and other to store the conversion results. The source data is a restore from the latest production Maildir backup and I did a conversion to sdbox, and then to mdbox. I attach DiskIO and CPU plots for both conversions orange Line is read from source disk, blue line is write to destination. sdbox on the left and mdbox on the right. Ok, this is a conversion process and not actual mailbox usage. And lacks info on how does dbox behaves in read operations.. but this is what I got for now. [image: Inline image 1] [image: Inline image 2] Miguel On Thu, Mar 29, 2012 at 12:16 PM, Miguel Afonso wrote: > > > On Thu, Mar 29, 2012 at 12:30 AM, Timo Sirainen wrote: > >> >> The main problem is that it's difficult to do any "real world" tests with >> IMAP, especially when users are using many different kinds of IMAP clients. >> So I'm very interested in hearing some numbers (and disk IO graphs for a >> few weeks would be great) before your migration and after your migration, >> but the numbers for your tests might not mean all that much. > > > I was considering using the imaptest tool to simulate IMAP activity. I > would keep the same machine configuration, only varying the mailbox format > while running imaptest against each setup for a few hours/days. > > I'm now converting the original Maildir format to both dbox formats and > I'll give it a try. I'll share some graphs afterwards. > > -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 24202 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 27261 bytes Desc: not available URL: From tss at iki.fi Fri Apr 20 20:13:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 20 Apr 2012 20:13:29 +0300 Subject: [Dovecot] array code issue ? In-Reply-To: <201204201627.54213.tim.ruehsen@gmx.de> References: <201204201627.54213.tim.ruehsen@gmx.de> Message-ID: On 20.4.2012, at 17.27, Tim Ruehsen wrote: > I just took a look into the dovecot 2.1 sources and just saw a possible issue > in array.h. > > This code snippet as an example: > #static inline void * > #array_get_modifiable_i(struct array *array, unsigned int *count_r) > #{ > # *count_r = array->buffer->used / array->element_size; > # return buffer_get_modifiable_data(array->buffer, NULL); > #} > > array->buffer->used and array->element_size are of type 'size_t' which is > 64bit on amd64 and others while 'count_r' is a 32bit value. At least, I see > ugly warnings with -Wconversion (which I personally like to use). I've been planning on trying out some of clang's warning flags. Last time I used -Wconversion with gcc it was giving way too many warnings to be usable, but clang's -Wconversion looked better when I quickly looked at it. > I know, it is unlikely that 'array->buffer->used / array->element_size' > exceeds 32bit range. But then, dovecot's source is so well written, that the > above code seems to disturb dovecot's code aesthetics. :) Yeah, I intentionally decided to use unsigned int here. It's a bit of wasteful and ugly to use size_t everywhere.. I guess the code could be made something like: size_t count = array->buffer->used / array->element_size; I_assert(count < UINT_MAX); *count_r = (unsigned int)count; Or something like that. Although these array functions are sometimes in performance critical paths, so adding extra code isn't very good either. Perhaps a simple cast to make the warning go away.. Probably the element_size could also be changed to be unsigned int. > And who knows... in a few years (when we have THz and TBytes on our desktops) > emails (and array sizes) might exceed everything that we think of today. The email sizes yes, but probably not the number of emails in a mailbox. From nmilas at noa.gr Fri Apr 20 21:01:39 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Fri, 20 Apr 2012 21:01:39 +0300 Subject: [Dovecot] default mail quota when using per user quota In-Reply-To: References: <4F91560B.5040009@noa.gr> Message-ID: <4F91A483.8000406@noa.gr> On 20/4/2012 5:31 ??, Andreas Helmcke wrote: > Do you/really/ have a default quota of 4G if roomNumber is empty or rather have unlimited quota then? Yes, default quota is 4G for all mailboxes; if roomNumber is defined, then that value is used instead. In practice we are using non-default values for very few maiboxes. We have a relatively small number of users/mailboxes (<300), so we have the luxury of enjoying relatively large mailboxes (with some "overbooking" - because most users rarely use more than 1G). Nick From CMarcus at Media-Brokers.com Fri Apr 20 21:05:42 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 20 Apr 2012 14:05:42 -0400 Subject: [Dovecot] Error in logs indicating broken files In-Reply-To: <4F9156E3.4050801@cnpapers.com> References: <4F9156E3.4050801@cnpapers.com> Message-ID: <4F91A576.1090709@Media-Brokers.com> On 2012-04-20 8:30 AM, Steve Campbell wrote: > Still new to dovecot, > > I'm seeing a couple of errors in my log file indicating corruption of > index files. It is customary to provide at least some bare minimal system details, like, dovecot -n output (which includes the version), platform/OS, etc... -- Best regards, Charles From jeff at j-simmons.net Fri Apr 20 22:26:41 2012 From: jeff at j-simmons.net (Jeff Simmons) Date: Fri, 20 Apr 2012 12:26:41 -0700 Subject: [Dovecot] dsync: convert only mbox in /var/mail to maildir? Message-ID: <201204201226.41710.jeff@j-simmons.net> I'm migrating an old UW pop server to a new Dovecot server. The old server did not allow mail to be stored on the server, so only mbox files exist in /var/mail, with nothing in /home/$user. I'd like to copy the /var/mail directory over to the new server and then use dsync to convert the mbox files and put them in /home/$user/Maildir, but every permutation of the command line I've tried dies with a variation on can't find/write either /var/mail/mail or /home/$user/mail, seems it won't deal with mbox files without an associated mail directory. Can dsync be used for this kind of conversion, or should I look elsewhere for conversion utilities? Thanks for helping a noob who can't traverse a wiki without leaving a trail of breadcrumbs. -- Jeff Simmons jeff at j-simmons.net Simmons Consulting - Network Engineering, Administration, Security From victormanuelo at gmail.com Fri Apr 20 23:40:18 2012 From: victormanuelo at gmail.com (=?UTF-8?Q?Victor_O=C3=B1ate?=) Date: Fri, 20 Apr 2012 16:10:18 -0430 Subject: [Dovecot] sieve against ldap in Dovecot 1.2 vacation messeage Message-ID: Hi. I need configure "sieve" against ldap this is my file configuration dovecot.conf plugin { quota = dirsize:user sieve_dir = ~/sieve } protocol lda { mail_plugins = sieve } auth default { mechanisms = plain passdb pam { } userdb ldap { args = /etc/dovecot/dovecot-ldap.conf } user = nobody } I dont know what I have to put in dovecot-ldap.conf, ofcourse I have a atribut in LDAP by user with a vacation messeage "I'm out of office, please contact Mr. Black" this is dovecot-ldap.conf hosts = server auth_bind = yes ldap_version = 3 base = dc=dominio,dc=uk deref = never scope = subtree user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid,mail=Maildir,mailQuotaSize=quota_rule=*:storage=%$ user_filter = (&(objectClass=posixAccount)(uid=%u)) The principal idea is keep the control the message by user with LDAP and Dovecot(IMAP). without touching the postfix configuration file is that possible, or am I dreaming. Thanks.. Victor O?ate. From tss at iki.fi Sat Apr 21 00:18:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Apr 2012 00:18:24 +0300 Subject: [Dovecot] default mail quota when using per user quota In-Reply-To: References: Message-ID: <553C5242-5931-47B1-9FA9-0B64A7216197@iki.fi> On 20.4.2012, at 14.11, Andreas Helmcke wrote: > using dovecot 2.1 and per user mail quota via ldap is there a way to have a default quota which gets used, if the > "quota" field in ldap is not set? By "not set" you mean it doesn't exist at all, or it exists but is empty? > I tried with: > > plugin { > quota = maildir:User quota > quota_rule = *:storage=3G > } > > userdb { > args = /etc/dovecot/dovecot-ldap.conf > driver = ldap > } > > and in dovecot-ldap.conf: > user_attrs = dcMailQuota=quota_rule=*:storage=%$ If dcMailQuota isn't returned, then the global quota_rule is used. If it is returned as empty, I guess it gets treated as unlimited quota. There's currently no easy way avoid this (a difficult way could be e.g. post-login scripting, but that works only for imap/pop3). From tss at iki.fi Sat Apr 21 00:23:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Apr 2012 00:23:02 +0300 Subject: [Dovecot] mbox folders on-disc layout In-Reply-To: <4F9125F9.6060601@anonymous.org.uk> References: <4F8EA753.4070605@anonymous.org.uk> <35C2B470-438E-4CEC-A42A-0BC2C39A0159@iki.fi> <4F9125F9.6060601@anonymous.org.uk> Message-ID: <73A197D0-6D8A-4C7D-B6DE-F6B89EC40A0B@iki.fi> On 20.4.2012, at 12.01, John Robinson wrote: > Yes, I read all that, and thought all the options were pretty ugly - and the comment that it's a "trick" configuration would seem to suggest that the original author thought so to. And I think Maildir++ is pretty ugly too. If I wanted to try writing a new LAYOUT, where should I begin? src/lib-storage/mailbox-list-private.h has the API you need to implement src/lib-storage/list/mailbox-list-fs* is the "fs" implementation src/lib-storage/list/mailbox-list-maildir* is the "maildir++" implementation So decide which one looks more like the one you want, and either copy&paste it or start extending it. For example "imapdir" is nearly the same implementation as "maildir++" but almost all of the code is shared. From tss at iki.fi Sat Apr 21 00:27:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Apr 2012 00:27:59 +0300 Subject: [Dovecot] dsync: convert only mbox in /var/mail to maildir? In-Reply-To: <201204201226.41710.jeff@j-simmons.net> References: <201204201226.41710.jeff@j-simmons.net> Message-ID: <04D01B3D-CD58-4E0A-8C1A-36B01E4792E7@iki.fi> On 20.4.2012, at 22.26, Jeff Simmons wrote: > I'm migrating an old UW pop server to a new Dovecot server. The old server did > not allow mail to be stored on the server, so only mbox files exist in > /var/mail, with nothing in /home/$user. I'd like to copy the /var/mail > directory over to the new server and then use dsync to convert the mbox files > and put them in /home/$user/Maildir, but every permutation of the command line > I've tried dies with a variation on can't find/write either /var/mail/mail or > /home/$user/mail, seems it won't deal with mbox files without an associated > mail directory. Can dsync be used for this kind of conversion, or should I > look elsewhere for conversion utilities? Assuming in destination server the mbox is in /var/mail/$user and you want the destination to ~/Maildir: in dovecot.conf set: mail_location = maildir.~/Maildir rm -rf /tmp/emptydir dsync mirror mbox:/tmp/emptydir:/var/mail/$user be sure to delete the emptydir between users. From tss at iki.fi Sat Apr 21 00:29:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Apr 2012 00:29:40 +0300 Subject: [Dovecot] quota not being calculated In-Reply-To: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> References: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> Message-ID: <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> On 20.4.2012, at 19.05, Cor Bosman wrote: > It looks like my quota isnt being calculated properly after I started setting quota to a specific folder. The quota in that folder is always starting out at 0, and only new email is being added to the quota. If I remove the maildirsize file, and recalculate, it still starts at 0. Once email arrives in the folder, it gets added to maildirsize. This is with maildirquota, in perhaps a bit of an exotic config. The idea is to set a specific quota to Spam folder, using the trash plugin to keep it at a certain size. I'll try later, but try if it would work with dict quota file? http://wiki2.dovecot.org/Quota/Dict From news1204.10.weezy at spamgourmet.com Sat Apr 21 00:47:44 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Fri, 20 Apr 2012 23:47:44 +0200 Subject: [Dovecot] default mail quota when using per user quota (news1204: message 6 of 20) In-Reply-To: <553C5242-5931-47B1-9FA9-0B64A7216197@iki.fi> References: <553C5242-5931-47B1-9FA9-0B64A7216197@iki.fi> Message-ID: <4F91D980.8070307@spamgourmet.com> Am 20.04.2012 23:18, schrieb Timo Sirainen: > By "not set" you mean it doesn't exist at all, or it exists but is empty? I tried both (at least I do think I tried both). > If dcMailQuota isn't returned, then the global quota_rule is used. If it is returned as empty, I guess it gets treated as unlimited quota. There's currently no easy way avoid this (a difficult way could be e.g. post-login scripting, but that works only for imap/pop3). > Maybe openldap does not distingish between this cases. I will try again on monday, when I am back at work. From tss at iki.fi Sat Apr 21 00:54:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Apr 2012 00:54:01 +0300 Subject: [Dovecot] default mail quota when using per user quota (news1204: message 6 of 20) In-Reply-To: <4F91D980.8070307@spamgourmet.com> References: <553C5242-5931-47B1-9FA9-0B64A7216197@iki.fi> <4F91D980.8070307@spamgourmet.com> Message-ID: On 21.4.2012, at 0.47, Andreas Helmcke wrote: >> If dcMailQuota isn't returned, then the global quota_rule is used. If it is returned as empty, I guess it gets treated as unlimited quota. There's currently no easy way avoid this (a difficult way could be e.g. post-login scripting, but that works only for imap/pop3). >> > > Maybe openldap does not distingish between this cases. I will try again on > monday, when I am back at work. Hm. Or maybe I broke it in v2.1 when I changed this code. I'll also check later. :) From stan at hardwarefreak.com Sat Apr 21 03:22:05 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 20 Apr 2012 19:22:05 -0500 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8FDD9B.6060007@hardwarefreak.com> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <4F8D69AB.5090706@esiee.fr> <4F8FDD9B.6060007@hardwarefreak.com> Message-ID: <4F91FDAD.7050102@hardwarefreak.com> On 4/19/2012 4:40 AM, Stan Hoeppner wrote: > On 4/17/2012 8:01 AM, Frank Bonnet wrote: > >> have 4000/6000 imaps concurent connections during working hours . >>>> for approx 50K "intensives" users. >>>> >>>> The only mandatory thing will be I must use HP proliant servers >>>> >>>> The operating system will be FreeBSD or Linux > I just made the wishlist public so it should be available tomorrow or > Friday. I'll provide the link when it's available. And here it is: http://secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=16797311 Since your requirement is for an HP solution, following is an HP server and storage system solution of roughly identical performance and redundancy to the SuperMicro based system I detailed. The HP system solution is $44,263, almost double the cost at $20,000 more. Due to the stupidity of Newegg requiring all wish lists to be reviewed before going live, I'll simply provide the links to all the products. Yes boys and girls, Newegg isn't just consumer products. They carry nearly the entire line of HP Proliant servers and storage, including the 4-way 48-core Opteron DL585 G7 w/64GB, the P2000 fiber channel array, and much more. In this case they sell every product needed to assemble this complete mail server solution: 1x http://www.newegg.com/Product/Product.aspx?Item=N82E16859105807 8x http://www.newegg.com/Product/Product.aspx?Item=N82E16820326150 3x http://www.newegg.com/Product/Product.aspx?Item=N82E16816401143 80x http://www.newegg.com/Product/Product.aspx?Item=N82E16822332061 3x http://www.newegg.com/Product/Product.aspx?Item=N82E16816118109 3x http://www.newegg.com/Product/Product.aspx?Item=N82E16816118163 2x http://www.newegg.com/Product/Product.aspx?Item=N82E16816133048 2x http://www.newegg.com/Product/Product.aspx?Item=N82E16833106050 The 9280-8e RAID controllers are identical to 9261-8i boards but have 2 external vs internal x4 6Gb SAS ports. I spec them instead of the Smart Array boards as they're far cheaper, easier to work with, and offer equal or superior performance. Thus everything written below is valid for this system as well, with the exception that you would configure 1 global hot spare in each chassis since these units have 25 drive bays instead of 24. The D2700 units come with 20" 8088 cables. I an additional spec'd two 3ft cables to make sure we reach all 3 disk chassis from the server, thinking the sever would be on top with the 3 disk chassis below. I hope this and my previous post are helpful in one aspect or another to Frank and anyone else. I spent more than a few minutes on these designs. ;) Days in fact on the SuperMicro design, only a couple of hours on the HP. It wouldn't have taken quite so long if all PCIe slots were created equal (x8), which they're not, or if modern servers didn't require 4 different types of DIMMs depending on how many slots you want to fill and how much expansion capacity you need without having to throw out all the previous memory, which many folks end up doing out of ignorance. Memory configuration is simply too darn complicated with high cap servers containing 8 channels and 24 slots. > The key to performance, and yielding a single file tree, is once again > using XFS to take advantage of this large spindle count across 3 RAID > controllers. Unlike previous configurations where I recommended using a > straight md concatenation of hardware RAID1 pairs, in this case we're > going to use a concatenation of 6 hardware RAID10 arrays. There are a > couple of reasons for doing so in this case: > > 1. Using 36 device names in a single md command line is less than > intuitive and possibly error prone. Using 6 is more manageable. > > 2. We have 3 BBWC RAID controllers w/24 drives each. This is a high > performance server and will see a high IO load in production. In many > cases one would use an external filesystem journal, which we could > easily do and get great performance with our mirrored SSDs. However, > the SSDs are not backed by BBWC, so a UPS failure or system crash could > hose the log journal. So we'll go with the default internal journal > which will be backed by the BBWC. > > Going internal with the log in this mail scenario can cause a serious > amount of extra IOPS on the filesystem data section, this being > Allocation Group 0. If we did the "normal" RAID1 concat, all the log IO > would hit the first RAID1 pair. On this system, the load may hit that > spindle pretty hard, making access to mailboxes in AG0 slower than > others. With 6 RAID10 arrays in a concat, the internal log writes will > be striped across 6 spindles in the first array. With 512MB BBWC > backing that array and optimizing writeout, and with delaylog, this will > yield optimal log write performance without slowing down mailbox file > access in AG0. To create such a setup we'd do something like this, > assuming the mobo LSI controller yields sd[ab], and the 6 array devices > on the PCIe LSI cards yield sd[cdefgh] > > 1. Create two RAID10 arrays, each of 12 drives, in the WebBIOS GUI of > each LSI card, using a strip size of 32KB which should yield good random > r/w performance for any mailbox format. Use the following policies for > each array: RW, Normal, Wback, Direct, Disable, No, and use the full > size. > > Create the concatenated md device: > $ mdadm -C /dev/md0 -l linear -n 6 /dev/sd[cdefgh] > > Then we format it with XFS, optimizing the AG layout for our mailbox > workload, and allocation write stripe alignment to each hardware array: > $ mkfs.xfs -d agcount=24 su=32k sw=6 /dev/md0 > > This yields 4 AGs per RAID10 array which will minimize the traditional > inode64 head seeking overhead on striped arrays, while still yielding > fantastic allocation parallelism with 24 AGs. > > Optimal fstab for MTA queue/mailbox workload, assuming kernel 2.6.39+: > /dev/md0 /mail xfs defaults,inode64,nobarrier 0 0 > > We disable write barriers as we have BBWC. And that 1.5GB of BBWC will > yield extremely low Dovecot write latency and throughput. > > Given the throughput available, if you're running Postfix on this box, > you will want to create a directory on this filesystem for the Postfix > spool. Postfix puts the spool files in many dozens, hundreds of > subdirectories, so you'll get 100% parallelism across all AGs, thus all > disks. > > It's very likely none of you will decide to build this system. My hope > is that some of the design concepts and components used, along with the > low cost but high performance of this machine, may be educational or > simply give people new ideas, steer them in directions they may not have > previously considered. -- Stan From jtam.home at gmail.com Sat Apr 21 06:32:30 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Fri, 20 Apr 2012 20:32:30 -0700 (PDT) Subject: [Dovecot] Error in logs indicating broken files In-Reply-To: References: Message-ID: Steve Campbell writes: > I'm seeing a couple of errors in my log file indicating corruption of > index files. > > Error: Corrupted index cache file/home/xxxxxx/mail/.imap/sent-mail/dovecot.index.cache: Broken virtual size for mail UID 67: 1 Time(s) > Error: FETCH [] for mailbox ~/mail/sent-mail UID 67 got too little data: 6203 vs 7478: 1 Time(s) > > This account is used by multiple users, but it is not a shared account > (haven't set any of those up yet - still new to me), so I would expect > corruption possibilities when multiple users modify the account folders > in some manner. This is just one dovecot IMAP client process moaning after another user client process pulled the rug out from under it by modifying the mailbox. I get a few dozen log entries of this sort per week with ~200 IMAP users without shared access. Unless your users actually complain about a problem, I think it's safe to ignore them. Joseph Tam From lgruen at gmx.ch Sat Apr 21 09:20:32 2012 From: lgruen at gmx.ch (=?ISO-8859-15?Q?=22L=2E_Gr=FCn=22?=) Date: Sat, 21 Apr 2012 08:20:32 +0200 Subject: [Dovecot] Correction of home and mail directory config Message-ID: <4F9251B0.5030501@gmx.ch> Hello, after running dovecot for a while, I stumbled accross the warning, not to have mail and home directory settings point to one and same directory. Here is the output of dovecot -n: > dovecot -n # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686 i686 Debian 6.0.4 ext3 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imaps imap disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login mail_max_userip_connections: 100 mail_privileged_group: mail mail_uid: 5000 mail_gid: 5000 mail_location: maildir:/var/vmail/%d/%n/Maildir mail_full_filesystem_access: yes mbox_write_locks: fcntl dotlock namespace: type: private separator: . inbox: yes list: yes subscriptions: yes lda: auth_socket_path: /var/run/dovecot/auth-master postmaster_address: xxxx at xxxx.org mail_plugins: sieve log_path: auth default: mechanisms: plain login passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: static args: uid=5000 gid=5000 home=/var/vmail/%d/%n/Maildir allow_all_users=yes socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail In "args: uid=5000 gid=5000 home=/var/vmail/%d/%n/Maildir allow_all_users=yes" I should delete "Maildir". My question is: What files do I have to copy from /Maildir to the home directory /var/vmail/%d/%n/ ? Are there other side effects, I have to consider? Thank you very much for your help! Stefan From cor at xs4all.nl Sat Apr 21 11:01:51 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sat, 21 Apr 2012 10:01:51 +0200 Subject: [Dovecot] quota not being calculated In-Reply-To: <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> References: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> Message-ID: > >> It looks like my quota isnt being calculated properly after I started setting quota to a specific folder. The quota in that folder is always starting out at 0, and only new email is being added to the quota. If I remove the maildirsize file, and recalculate, it still starts at 0. Once email arrives in the folder, it gets added to maildirsize. This is with maildirquota, in perhaps a bit of an exotic config. The idea is to set a specific quota to Spam folder, using the trash plugin to keep it at a certain size. > > I'll try later, but try if it would work with dict quota file? http://wiki2.dovecot.org/Quota/Dict > This almost works, but maybe there's a small bug. Config (2.1.4) namespace spam { type = private separator = / inbox = no prefix = Spam/ location = maildir:%h/.Spam:INDEX=/var/spool/mail/dovecot-control/indexes/%1u/%2u/%u/.Spam:CONTROL=/var/spool/mail/dovecot-control/%1u/%2u/%u/.Spam subscriptions = no list = yes hidden = yes } plugin { quota = fs:User quota quota2 = dict:Spam Quota::ns=spam:file:%h/spam-quota quota2_rule = *:storage=20M } Now I see: . getquotaroot "Spam" * QUOTAROOT "Spam" "User quota" "Spam Quota" * QUOTA "User quota" (STORAGE 10128 1228800) * QUOTA "Spam Quota" (STORAGE 8727 20480) The problem is that the storage calculated for the spam ns is wrong. It is less than 1MB. The 8727 seems to be the whole maildir including all folders. This seems to be caused by: Apr 21 10:00:11 lmtp1 dovecot: imap(cor): Error: quota: Unknown namespace: spam Regards, Cor From janfrode at tanso.net Sat Apr 21 12:52:57 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sat, 21 Apr 2012 11:52:57 +0200 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F90AE51.8080407@hardwarefreak.com> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <20120417124241.GA23120@dibs.tanso.net> <4F8D6BAA.7060501@gtdinternet.com> <20120417200820.GA30743@dibs.tanso.net> <4F90AE51.8080407@hardwarefreak.com> Message-ID: <20120421095257.GA19877@dibs.tanso.net> On Thu, Apr 19, 2012 at 07:31:13PM -0500, Stan Hoeppner wrote: > > This issue has come up twice on the Postfix list in less than a month. Oh, thanks! I'll look into those list posts.. I had mostly given up solving this by rate limits and decided to throw hardware at the problem when I saw the log entries for sender *.anpdm.com.. Seems to be a newsletter sender, which I found as 203 different mailserver ip-addresses in our incoming mailserver logs, from 53 different B-nets and 8 different A-nets. Will give smtpd_client_connection_count_limit a try.. -jf From janfrode at tanso.net Sat Apr 21 13:22:52 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sat, 21 Apr 2012 12:22:52 +0200 Subject: [Dovecot] migrate 15K users to new domain name In-Reply-To: References: <20120419082245.GA4050@dibs.tanso.net> <20120419132803.GA9396@dibs.tanso.net> <20120419141930.GB10069@dibs.tanso.net> Message-ID: <20120421102252.GA20159@dibs.tanso.net> On Thu, Apr 19, 2012 at 05:59:39PM +0300, Timo Sirainen wrote: > > With v2.1.4 you could do something like: > > doveadm -c dummy.conf user -m user at domain > > where dummy.conf contains the minimum configuration needed: > > mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln > ssl = no > Thanks! Works perfect. -jf From tss at iki.fi Sat Apr 21 13:32:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Apr 2012 13:32:21 +0300 Subject: [Dovecot] quota not being calculated In-Reply-To: References: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> Message-ID: <2D57E521-3DB3-41D9-B8AB-F280DE546061@iki.fi> On 21.4.2012, at 11.01, Cor Bosman wrote: > prefix = Spam/ .. > quota2 = dict:Spam Quota::ns=spam:file:%h/spam-quota .. > Apr 21 10:00:11 lmtp1 dovecot: imap(cor): Error: quota: Unknown namespace: spam Oh. It would make more sense to have ns=spam, but unfortunately I hadn't thought of namespace names before v2.1. You need to use ns=Spam/ From cor at xs4all.nl Sat Apr 21 13:42:35 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sat, 21 Apr 2012 12:42:35 +0200 Subject: [Dovecot] quota not being calculated In-Reply-To: <2D57E521-3DB3-41D9-B8AB-F280DE546061@iki.fi> References: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> <2D57E521-3DB3-41D9-B8AB-F280DE546061@iki.fi> Message-ID: On Apr 21, 2012, at 12:32 PM, Timo Sirainen wrote: > On 21.4.2012, at 11.01, Cor Bosman wrote: > >> prefix = Spam/ > . >> quota2 = dict:Spam Quota::ns=spam:file:%h/spam-quota > . >> Apr 21 10:00:11 lmtp1 dovecot: imap(cor): Error: quota: Unknown namespace: spam > > Oh. It would make more sense to have ns=spam, but unfortunately I hadn't thought of namespace names before v2.1. You need to use ns=Spam/ Exact same issue with dict quota. It starts out at 0 (even though there are 116 messages in the Spam folder. Then when 1 email gets saved to the Spam folder, the counter is increased by the size of that 1 email. Cor From nmilas at noa.gr Sat Apr 21 13:53:45 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Sat, 21 Apr 2012 13:53:45 +0300 Subject: [Dovecot] default mail quota when using per user quota (news1204: message 6 of 20) In-Reply-To: References: <553C5242-5931-47B1-9FA9-0B64A7216197@iki.fi> <4F91D980.8070307@spamgourmet.com> Message-ID: <4F9291B9.4080701@noa.gr> On 21/4/2012 12:54 ??, Timo Sirainen wrote: > Hm. Or maybe I broke it in v2.1 when I changed this code. I'll also check later. I'm afraid that's possible. The same configuration in 2.0.13 produces (doveadm quota get -u tester): Quota name Type Value Limit % User quota STORAGE 237803 4194304 5 User quota MESSAGE 531 - 0 and in 2.1.1: Quota name Type Value Limit % User quota STORAGE 0 - 0 User quota MESSAGE 0 - 0 (In the latter test server value should be 0 - because there are no messages there - but limit should be as above. Things do not change after: doveadm quota recalc -u tester) Regards, Nick From almarzuki2001 at hotmail.com Sat Apr 21 17:51:55 2012 From: almarzuki2001 at hotmail.com (Hadi Salem) Date: Sat, 21 Apr 2012 17:51:55 +0300 Subject: [Dovecot] dovecot LDA with sendmail In-Reply-To: <4F91438D.3040200@Media-Brokers.com> References: , , <4F91438D.3040200@Media-Brokers.com> Message-ID: Hi, i installed new version 2.0.20. but its the same problem # 2.0.20: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-164.el5 x86_64 CentOS release 5.4 (Final) auth_debug = yes auth_debug_passwords = yes auth_verbose = yes debug_log_path = /var/log/dovecot.log default_login_user = vmail disable_plaintext_auth = no hostname = host1.bigmama.com info_log_path = /var/log/dovecot.log log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:~/Maildir mail_privileged_group = vmail passdb { args = scheme=PLAIN username_format=%u /etc/dovecot/users driver = passwd-file } postmaster_address = root at host1.bigmama.com service auth { unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } ssl = no userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } Apr 21 17:47:16 host1 sendmail[32561]: q3LElF79032559: to=, ctladdr= (0/0), delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120292, relay=dovecot, dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/local/libexec/dovecot/dovecot-lda) exited with EX_TEMPFAIL > Date: Fri, 20 Apr 2012 07:07:57 -0400 > From: CMarcus at Media-Brokers.com > To: dovecot at dovecot.org > Subject: Re: [Dovecot] dovecot LDA with sendmail > > On 2012-04-19 9:45 PM, Joseph Tam wrote: > > Hadi Salem writes: > >> Im configuring sendmail with dovecot for virtual users using > >> password file and file for user name. on centos. > >> dovecot-1.0.7-7.el5 > > > Maybe I can beat Charles Marcus to the punch and recommend you upgrade > > to the latest version, otherwise you'll run into bugs that have already > > been fixed. > > Lol... I stopped reading at '...sendmail with dovecot...', otherwise I'd > have beat you to it... ;) > > -- > > Best regards, > > Charles From stan at hardwarefreak.com Sat Apr 21 22:59:44 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 21 Apr 2012 14:59:44 -0500 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <20120421095257.GA19877@dibs.tanso.net> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <20120417124241.GA23120@dibs.tanso.net> <4F8D6BAA.7060501@gtdinternet.com> <20120417200820.GA30743@dibs.tanso.net> <4F90AE51.8080407@hardwarefreak.com> <20120421095257.GA19877@dibs.tanso.net> Message-ID: <4F9311B0.7000503@hardwarefreak.com> On 4/21/2012 4:52 AM, Jan-Frode Myklebust wrote: > On Thu, Apr 19, 2012 at 07:31:13PM -0500, Stan Hoeppner wrote: >> >> This issue has come up twice on the Postfix list in less than a month. > > Oh, thanks! I'll look into those list posts.. I had mostly given up > solving this by rate limits and decided to throw hardware at the problem > when I saw the log entries for sender *.anpdm.com.. Seems to be a newsletter > sender, which I found as 203 different mailserver ip-addresses in our > incoming mailserver logs, from 53 different B-nets and 8 different A-nets. Yeah, they're a newsletter service provider. > Will give smtpd_client_connection_count_limit a try.. Setting this to 1 or 2 should severely slow their delivery rate. You can also do rate limiting at a much more fine grained level with a Postfix policy daemon such as postfwd (Postfix firewall daemon), though the setup is a bit more complicated. -- Stan From daniel.parthey at informatik.tu-chemnitz.de Sat Apr 21 23:29:28 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 21 Apr 2012 22:29:28 +0200 Subject: [Dovecot] How to remove duplicate GUID messages from storage? Message-ID: <20120421202928.GA12998@daniel.localdomain> Hi, I tried to import messages like this: doveadm import -u username at example.org mdbox:/mail/dovecot/example.org/username/mail mbox:/root/mail.txt all This seemed to have imported the same messages in the mail storage, as they were already there. And now I have got a lot of duplicates. Then I deleted the mbox:/root/mail mail stuff from "mailboxes" folder and the storage remained, with duplicate GUIDs. The following command doveadm -v force-resync -u username at example.org INBOX outputs lots of messages like this one, with different GUID doveadm(username at example.org): Error: mdbox /mail/dovecot/example.org/username/mail/storage: Duplicate GUID 0b8a032d66a0924fb42c0000de5f8128 in m.55:45484041 and m.14:52173045 The messages at m.55:45484041 and m.14:52173045 have the same content, since they were seemingly imported from the mail store itself, they are in the storage twice now and require twice as much disk space as before. How can I manually remove these identical, duplicate messages from the storage to save space? Dovecot does not do it automatically. Kind regards, Daniel From jtam.home at gmail.com Sun Apr 22 07:36:57 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Sat, 21 Apr 2012 21:36:57 -0700 (PDT) Subject: [Dovecot] dovecot LDA with sendmail In-Reply-To: References: Message-ID: > i installed new version 2.0.20. but its the same problem > > args = username_format=%u /etc/dovecot/users ... > > Apr 21 17:47:16 host1 sendmail[32561]: q3LElF79032559: to=, ctladdr= (0/0), delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120292, relay=dovecot, dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/local/libexec/dovecot/dovecot-lda) exited with EX_TEMPFAIL Any logs for dovecot? Same sendmail configs as before or did you try the extra flag I suggested? Joseph Tam From dovecot at webrz.net Sun Apr 22 11:03:55 2012 From: dovecot at webrz.net (Jos Chrispijn) Date: Sun, 22 Apr 2012 10:03:55 +0200 Subject: [Dovecot] Dovecot upgrade Message-ID: <4F93BB6B.7000501@webrz.net> Can someone tell me how I can upgrade from Dovecot 1.x to 2.x best? thanks for your reply, Jos Chrispijn From cor at xs4all.nl Sun Apr 22 11:08:44 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sun, 22 Apr 2012 10:08:44 +0200 Subject: [Dovecot] Dovecot upgrade In-Reply-To: <4F93BB6B.7000501@webrz.net> References: <4F93BB6B.7000501@webrz.net> Message-ID: <0BC7794E-1F1E-45BF-8539-61D5D8BAEE74@xs4all.nl> On Apr 22, 2012, at 10:03 AM, Jos Chrispijn wrote: > Can someone tell me how I can upgrade from Dovecot 1.x to 2.x best? > thanks for your reply, > Jos Chrispijn Have you read this? http://wiki2.dovecot.org/Upgrading/2.0 Cor From tlx at leuxner.net Sun Apr 22 11:08:52 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sun, 22 Apr 2012 10:08:52 +0200 Subject: [Dovecot] Dovecot upgrade In-Reply-To: <4F93BB6B.7000501@webrz.net> References: <4F93BB6B.7000501@webrz.net> Message-ID: Am 22.04.2012 um 10:03 schrieb Jos Chrispijn: > Can someone tell me how I can upgrade from Dovecot 1.x to 2.x best? > thanks for your reply, > Jos Chrispijn http://wiki2.dovecot.org/Upgrading/2.0 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dg at dguhl.org Sun Apr 22 12:06:20 2012 From: dg at dguhl.org (Dennis Guhl) Date: Sun, 22 Apr 2012 11:06:20 +0200 Subject: [Dovecot] sieve against ldap in Dovecot 1.2 vacation messeage In-Reply-To: References: Message-ID: <20120422090620.GA22644@laptop-dg.leere.eu> On Fri, Apr 20, 2012 at 04:10:18PM -0430, Victor O?ate wrote: > Hi. > I need configure "sieve" against ldap You can't configure sieve against LDAP. Sieve is a filter language, which is called by the LDA and configured by files in your home dir. For more information about sieve (with Dovecot 1) look here: http://wiki.dovecot.org/LDA/Sieve and here: http://wiki.dovecot.org/LDA/Sieve/Dovecot [..] > The principal idea is keep the control the message by user with LDAP > and Dovecot(IMAP). without touching the postfix configuration file > is that possible, or am I dreaming. What you want is ManageSieve: http://wiki.dovecot.org/ManageSieve more specifically: http://wiki.dovecot.org/ManageSieve/Clients Dennis From dg at dguhl.org Sun Apr 22 12:10:18 2012 From: dg at dguhl.org (Dennis Guhl) Date: Sun, 22 Apr 2012 11:10:18 +0200 Subject: [Dovecot] Correction of home and mail directory config In-Reply-To: <4F9251B0.5030501@gmx.ch> References: <4F9251B0.5030501@gmx.ch> Message-ID: <20120422091018.GB22644@laptop-dg.leere.eu> On Sat, Apr 21, 2012 at 08:20:32AM +0200, "L. Gr?n" wrote: > Hello, > after running dovecot for a while, I stumbled accross the warning, > not to have mail and home directory settings point to one and same > directory. > Here is the output of dovecot -n: > > > dovecot -n > # 1.2.15: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-5-686 i686 Debian 6.0.4 ext3 [..] > mail_location: maildir:/var/vmail/%d/%n/Maildir [..] > auth default: > mechanisms: plain login > passdb: > driver: sql > args: /etc/dovecot/dovecot-sql.conf > userdb: > driver: static > args: uid=5000 gid=5000 home=/var/vmail/%d/%n/Maildir [..] > In "args: uid=5000 gid=5000 home=/var/vmail/%d/%n/Maildir > allow_all_users=yes" I should delete "Maildir". Yes, exactly. > My question is: > What files do I have to copy from /Maildir to the home directory > /var/vmail/%d/%n/ ? Everything concerning sieve should be in the home dir. > Are there other side effects, I have to consider? Less warnings ;) Dennis From cor at xs4all.nl Sun Apr 22 12:33:03 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sun, 22 Apr 2012 11:33:03 +0200 Subject: [Dovecot] Correction of home and mail directory config In-Reply-To: <20120422091018.GB22644@laptop-dg.leere.eu> References: <4F9251B0.5030501@gmx.ch> <20120422091018.GB22644@laptop-dg.leere.eu> Message-ID: <98FF7560-C0B6-495E-9E9C-DA9D1C742085@xs4all.nl> > Everything concerning sieve should be in the home dir. Why? It can be anywhere you want as long as it doesnt conflict with the names of your mailstore. Cor From dovecot-user at spambox.dk Mon Apr 23 11:30:06 2012 From: dovecot-user at spambox.dk (Henrik Larsson) Date: Mon, 23 Apr 2012 09:30:06 +0100 Subject: [Dovecot] 2.1.4 Corrupted squat uidlist In-Reply-To: <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> References: <4F7164DC.7010706@unict.it> <4F8A97F8.70808@spambox.dk> <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> Message-ID: <829b37b74c07acbf148f7bee6a1e175e@larsson.as> Just to let you know, 2.1.4 didn't solve this either. If you need any futher info to get this solved, please let me know. Best regards Henrik Larsson > This problem is still showing up every day in the logs. > > I tried to delete the dovecot.index.search and > dovecot.index.search.uids files for all users, but the errors are > still there after a re-build of the search index with > "/usr/local/bin/doveadm search -A TEXT xyzabczzz". > > If you need any further details, please let me know. > > > Best regards > Henrik Larsson > > > On 15-04-2012 10:42, Henrik Larsson wrote: >> On 27-03-2012 08:57, Luca Palazzo wrote: >>> Hi Timo and All, >>> after upgrading to 2.1.2 i'm getting a lot of these messages: >>> Error: Corrupted squat uidlist file XXXXXX wrong indexid >> After an upgrade to 2.1.3 i see this as well. Any thoughts? >> >> Apr 15 03:43:43 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> Apr 15 06:30:27 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> Apr 15 07:43:55 pop3(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> Apr 15 09:00:01 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> >> # doveconf -n >> # 2.1.3: /usr/local/etc/dovecot/dovecot.conf >> # OS: FreeBSD 8.2-STABLE amd64 >> auth_mechanisms = plain login digest-md5 cram-md5 >> first_valid_uid = 125 >> listen = * >> log_path = /var/log/dovecot >> mail_plugins = fts fts_squat zlib >> mail_privileged_group = postfix >> mail_temp_dir = /var/db/dovecot >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date ihave >> namespace { >> inbox = yes >> location = >> prefix = >> separator = . >> type = private >> } >> namespace { >> hidden = yes >> inbox = no >> list = no >> location = >> prefix = INBOX. >> separator = . >> type = private >> } >> passdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> plugin { >> fts = squat >> fts_squat = partial=4 full=10 >> sieve = ~/.dovecot.sieve >> sieve_dir = ~/sieve >> } >> protocols = imap pop3 lmtp sieve >> service auth-worker { >> user = $default_internal_user >> } >> service auth { >> unix_listener /home/mail/postfix/private/dovecot-auth { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> service lmtp { >> executable = lmtp -L >> unix_listener /home/mail/postfix/private/dovecot-lmtp { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> ssl_cert = > ssl_key = > userdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> protocol lmtp { >> log_path = /var/log/dovecot-deliver >> mail_plugins = fts fts_squat zlib sieve >> } >> protocol lda { >> mail_plugins = fts fts_squat zlib sieve >> } >> protocol imap { >> mail_plugins = fts fts_squat zlib imap_zlib >> } >> >> >> Best regards >> Henrik Larsson From tss at iki.fi Mon Apr 23 13:20:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 13:20:24 +0300 Subject: [Dovecot] quota not being calculated In-Reply-To: References: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> <2D57E521-3DB3-41D9-B8AB-F280DE546061@iki.fi> Message-ID: <96E3D11C-8F71-4726-816A-6C448554BC70@iki.fi> On 21.4.2012, at 13.42, Cor Bosman wrote: > On Apr 21, 2012, at 12:32 PM, Timo Sirainen wrote: > >> On 21.4.2012, at 11.01, Cor Bosman wrote: >> >>> prefix = Spam/ >> . >>> quota2 = dict:Spam Quota::ns=spam:file:%h/spam-quota >> . >>> Apr 21 10:00:11 lmtp1 dovecot: imap(cor): Error: quota: Unknown namespace: spam >> >> Oh. It would make more sense to have ns=spam, but unfortunately I hadn't thought of namespace names before v2.1. You need to use ns=Spam/ > > Exact same issue with dict quota. It starts out at 0 (even though there are 116 messages in the Spam folder. Then when 1 email gets saved to the Spam folder, the counter is increased by the size of that 1 email. http://hg.dovecot.org/dovecot-2.1/rev/4c8f79d1f9f1 should fix it with dict quota. From tss at iki.fi Mon Apr 23 13:34:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 13:34:09 +0300 Subject: [Dovecot] default mail quota when using per user quota (news1204: message 6 of 20) In-Reply-To: <4F9291B9.4080701@noa.gr> References: <553C5242-5931-47B1-9FA9-0B64A7216197@iki.fi> <4F91D980.8070307@spamgourmet.com> <4F9291B9.4080701@noa.gr> Message-ID: On 21.4.2012, at 13.53, Nikolaos Milas wrote: > On 21/4/2012 12:54 ??, Timo Sirainen wrote: > >> Hm. Or maybe I broke it in v2.1 when I changed this code. I'll also check later. Just tried, it works as expected. > I'm afraid that's possible. The same configuration in 2.0.13 produces (doveadm quota get -u tester): > > Quota name Type Value Limit % > User quota STORAGE 237803 4194304 5 > User quota MESSAGE 531 - 0 > > and in 2.1.1: > > Quota name Type Value Limit % > User quota STORAGE 0 - 0 > User quota MESSAGE 0 - 0 > > (In the latter test server value should be 0 - because there are no messages there - but limit should be as above. > Things do not change after: doveadm quota recalc -u tester) What do you get in logs with auth_debug=yes? From tss at iki.fi Mon Apr 23 13:35:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 13:35:46 +0300 Subject: [Dovecot] Error in logs indicating broken files In-Reply-To: References: Message-ID: <53563F64-C80D-4E98-B722-E7154BC0FAB3@iki.fi> On 21.4.2012, at 6.32, Joseph Tam wrote: >> Error: Corrupted index cache file/home/xxxxxx/mail/.imap/sent-mail/dovecot.index.cache: Broken virtual size for mail UID 67: 1 Time(s) >> Error: FETCH [] for mailbox ~/mail/sent-mail UID 67 got too little data: 6203 vs 7478: 1 Time(s) > > This is just one dovecot IMAP client process moaning after another user > client process pulled the rug out from under it by modifying the mailbox. > I get a few dozen log entries of this sort per week with ~200 IMAP users > without shared access. Well, it is a bug and it shouldn't happen. But since it happens only with mbox, I haven't bothered to look into it. From tss at iki.fi Mon Apr 23 13:52:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 13:52:49 +0300 Subject: [Dovecot] 2.1.3 Corrupted squat uidlist In-Reply-To: <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> References: <4F7164DC.7010706@unict.it> <4F8A97F8.70808@spambox.dk> <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> Message-ID: <96F0AD8C-9832-4CA5-B1DA-3936F9E2F0FB@iki.fi> So the doveadm search itself succeeds without errors, but if you immediately do another doveadm search it fails? Or do you need to do something else also in the middle? I couldn't reproduce this with a few tests. On 20.4.2012, at 10.40, Henrik Larsson wrote: > This problem is still showing up every day in the logs. > > I tried to delete the dovecot.index.search and dovecot.index.search.uids files for all users, but the errors are still there after a re-build of the search index with "/usr/local/bin/doveadm search -A TEXT xyzabczzz". > > If you need any further details, please let me know. > > > Best regards > Henrik Larsson > > > On 15-04-2012 10:42, Henrik Larsson wrote: >> On 27-03-2012 08:57, Luca Palazzo wrote: >>> Hi Timo and All, >>> after upgrading to 2.1.2 i'm getting a lot of these messages: >>> Error: Corrupted squat uidlist file XXXXXX wrong indexid >> After an upgrade to 2.1.3 i see this as well. Any thoughts? >> >> Apr 15 03:43:43 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid >> Apr 15 06:30:27 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid >> Apr 15 07:43:55 pop3(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid >> Apr 15 09:00:01 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid >> >> # doveconf -n >> # 2.1.3: /usr/local/etc/dovecot/dovecot.conf >> # OS: FreeBSD 8.2-STABLE amd64 >> auth_mechanisms = plain login digest-md5 cram-md5 >> first_valid_uid = 125 >> listen = * >> log_path = /var/log/dovecot >> mail_plugins = fts fts_squat zlib >> mail_privileged_group = postfix >> mail_temp_dir = /var/db/dovecot >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date ihave >> namespace { >> inbox = yes >> location = >> prefix = >> separator = . >> type = private >> } >> namespace { >> hidden = yes >> inbox = no >> list = no >> location = >> prefix = INBOX. >> separator = . >> type = private >> } >> passdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> plugin { >> fts = squat >> fts_squat = partial=4 full=10 >> sieve = ~/.dovecot.sieve >> sieve_dir = ~/sieve >> } >> protocols = imap pop3 lmtp sieve >> service auth-worker { >> user = $default_internal_user >> } >> service auth { >> unix_listener /home/mail/postfix/private/dovecot-auth { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> service lmtp { >> executable = lmtp -L >> unix_listener /home/mail/postfix/private/dovecot-lmtp { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> ssl_cert = > ssl_key = > userdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> protocol lmtp { >> log_path = /var/log/dovecot-deliver >> mail_plugins = fts fts_squat zlib sieve >> } >> protocol lda { >> mail_plugins = fts fts_squat zlib sieve >> } >> protocol imap { >> mail_plugins = fts fts_squat zlib imap_zlib >> } >> >> >> Best regards >> Henrik Larsson > From tss at iki.fi Mon Apr 23 13:59:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 13:59:44 +0300 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: <2vp569xe2o.ln2@mail.ela-soft.com> References: <2vp569xe2o.ln2@mail.ela-soft.com> Message-ID: <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> On 19.4.2012, at 19.09, Andreas Helmcke wrote: > Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace > dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, > subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ Here's the dummy namespace with prefix="". > Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace : > type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, > subscriptions=yes location=maildir:/home/mail/user/ahelmcke But what is this? You appear to have an empty namespace {} in your configs? > Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Error: user > ahelmcke: Initialization failed: namespace configuration error: Duplicate > namespace prefix: "" That's why it fails with this. From tss at iki.fi Mon Apr 23 14:12:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 14:12:25 +0300 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works / solved by remove listescape In-Reply-To: <4F907529.2020403@schetterer.org> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> <4F90250E.3000200@schetterer.org> <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> <4F903DC1.5070501@schetterer.org> <4F905B2C.8040204@schetterer.org> <4F907529.2020403@schetterer.org> Message-ID: On 19.4.2012, at 23.27, Robert Schetterer wrote: >> prefix = shared/%%u/ simply dont works >> >> i have reverted back to exact namespace config >> of the 2.0.20 server, same result, so something broken in 2.1 >> >> > Hi Timo > problem solved by removing listescape plugin > the acl paths did / instead of dot in domainname > > any chance to get listescape fixed ? Ah, that makes it easy to fix: http://hg.dovecot.org/dovecot-2.1/rev/63af3274fb6f From tss at iki.fi Mon Apr 23 14:16:41 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 14:16:41 +0300 Subject: [Dovecot] imapc / namespace question In-Reply-To: <4F888F5E.5030709@necoro.eu> References: <4F888F5E.5030709@necoro.eu> Message-ID: <23ABA092-7511-4DF5-98AA-932E7338A994@iki.fi> On 13.4.2012, at 23.41, Ren? Neumann wrote: > I've got two questions regarding imapc and namespaces: > > 1) Is there a way of stating, that I only want part of the hierarchy on > the end of an imapc connection to be available here? My "problem": hg has now imapc_list_prefix. > 2) "Private" namespaces only make sense, if location contains something > user-specific like "%u" or "~". Is this correct? > If yes: Is there some other way to make something like an imapc > connection only appear for one user, if there is nothing in the location > or imapc_user to denote the user it belongs to and hence I cannot use > "private"? > The only way I see currently is to use "shared" and set up ACLs to > forbid all users except one to access the namespace (which gets slightly > ugly as I have to use global ACLs, as -- I think -- there is no way to > store ACLs for an imapc storage). Well, it's not necessarily a problem to use a private namespace for anything you want. If you set it hidden=yes (probably should do anyway) then IMAP clients won't even know that you've set it as private. Internally the main difference between private and public namespaces is the default ACLs. From tss at iki.fi Mon Apr 23 14:18:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 14:18:20 +0300 Subject: [Dovecot] French characters from 2.0.x to 2.1.3 In-Reply-To: <4F883FE7.9080401@esiee.fr> References: <4F883FE7.9080401@esiee.fr> Message-ID: <912BFCAA-3ECE-42AA-884F-0FE0BEDB4BBC@iki.fi> On 13.4.2012, at 18.01, Frank Bonnet wrote: > Maybe it's a coincidence but it "seems" > since I upgrade from 2.0.18 to 2.1.3 > that some French characters ( a with accents by example ) > are not well displayed anymore using thunderbird and are > replaced by a "?" character Dovecot doesn't send decoded characters to IMAP clients. Of course there might be something else going on, but I can't really guess what it could be. From news1204.10.weezy at spamgourmet.com Mon Apr 23 14:40:11 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Mon, 23 Apr 2012 13:40:11 +0200 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> References: <2vp569xe2o.ln2@mail.ela-soft.com> <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> Message-ID: <4F953F9B.10307@spamgourmet.com> Am 23.04.2012 12:59, schrieb Timo Sirainen: > On 19.4.2012, at 19.09, Andreas Helmcke wrote: > >> Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace : >> type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, >> subscriptions=yes location=maildir:/home/mail/user/ahelmcke > > But what is this? You appear to have an empty namespace {} in your configs? > nope. You might check yourself; complete dovecot -n: # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) ext4 auth_master_user_separator = * auth_verbose = yes first_valid_uid = 400 mail_debug = yes mail_gid = vmail mail_home = /home/mail/user/%u mail_location = maildir:/home/mail/user/%u mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl namespace Allgemeines { location = maildir:/home/mail/Allgemeines:INDEX=/home/mail/user/%u/SEEN.Allgemeines prefix = Allgemeines/ separator = / subscriptions = no type = public } namespace dummy { alias_for = INBOX/ hidden = yes list = no location = maildir:/home/mail/user/%u/public/ prefix = separator = / type = private } namespace inbox { inbox = yes location = prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { quota = maildir:User quota quota_rule = *:storage=3G } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = * } inet_listener imaps { address = * } } ssl = required ssl_cert = References: <4F883FE7.9080401@esiee.fr> <912BFCAA-3ECE-42AA-884F-0FE0BEDB4BBC@iki.fi> Message-ID: <4F954017.50702@esiee.fr> On 04/23/2012 01:18 PM, Timo Sirainen wrote: > On 13.4.2012, at 18.01, Frank Bonnet wrote: > >> Maybe it's a coincidence but it "seems" >> since I upgrade from 2.0.18 to 2.1.3 >> that some French characters ( a with accents by example ) >> are not well displayed anymore using thunderbird and are >> replaced by a "?" character > Dovecot doesn't send decoded characters to IMAP clients. Of course there might be something else going on, but I can't really guess what it could be. > OK thanks , maybe an update of thunderbird ? From tss at iki.fi Mon Apr 23 14:51:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 14:51:50 +0300 Subject: [Dovecot] Bug in HIGHESTMODSEQ when MODSEQ tracking is not enabled for version 2.0.18. In-Reply-To: <48A61B0976102F488C64EF8A8F1E929D074E9F@XMB118CNC.rim.net> References: <48A61B0976102F488C64EF8A8F1E929D074BBA@XMB118CNC.rim.net> <11BD08AA-2E29-48DF-973D-4D5B4CCE2FF7@iki.fi> <48A61B0976102F488C64EF8A8F1E929D074E9F@XMB118CNC.rim.net> Message-ID: <8A3B9DEC-6E45-4A2B-82FB-1A39BABF2247@iki.fi> On 10.4.2012, at 23.50, Glenn Wurster wrote: >> Yes, modseqs aren't tracked in a mailbox until client expresses an >> interest for them. It would be a waste of disk space to save them >> since 99% of users don't need them. > > Makes sense, our mail client gets caught in the middle though, because it uses HIGHESTMODSEQ to track mailbox updates without using MODSEQ options on SELECT/FETCH to track message updates. It would be actually possible for Dovecot to always keep track of highestmodseq, even if individual modseqs weren't tracked. I almost implemented it, but keeping it backwards compatible with old versions would have needed to make it more complex. Maybe v2.2 could do this. >> 2) If a mailbox doesn't have modseqs enabled, return NOMODSEQ. This >> isn't ideal, but seems like the only possibility. > > The RFC also states that if we return NOMODSEQ we'd have to return a tagged BAD response to "UID FETCH 1 MODSEQ", which appears to one of the commands that enables MODSEQ for Dovecot ("SELECT INBOX (CONDSTORE)" also enables it...). What about returning a BAD response and at the same time start tracking MODSEQ so that future SELECT commands would return HIGHESTMODSEQ? Do we know what email clients are using CONDSTORE options and how they'd react to a mailbox suddenly having MODSEQ capabilities after we just told them it didn't? That's kind of an annoying part of the RFC that it says the commands MUST fail with BAD.. I don't think there was really any good reason to add that text. Also Dovecot hasn't failed those commands earlier also with mailbox formats that don't support modseqs at all. So at least for now I simply made it return NOMODSEQ when modseqs aren't enabled, and the rest of the behavior is the same. From tss at iki.fi Mon Apr 23 14:55:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 14:55:21 +0300 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: <4F953F9B.10307@spamgourmet.com> References: <2vp569xe2o.ln2@mail.ela-soft.com> <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> <4F953F9B.10307@spamgourmet.com> Message-ID: On 23.4.2012, at 14.40, Andreas Helmcke wrote: > Am 23.04.2012 12:59, schrieb Timo Sirainen: >> On 19.4.2012, at 19.09, Andreas Helmcke wrote: >> >>> Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace : >>> type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, >>> subscriptions=yes location=maildir:/home/mail/user/ahelmcke >> >> But what is this? You appear to have an empty namespace {} in your configs? >> > nope. > > You might check yourself; complete dovecot -n: See if this changes the behavior: service config { unix_listener config { mode = 0666 } } I'd guess that makes the vmail work also? Although it still doesn't explain the difference between asking config from config process vs. getting it by executing doveconf. From dovecot-user at spambox.dk Mon Apr 23 14:56:13 2012 From: dovecot-user at spambox.dk (Henrik Larsson) Date: Mon, 23 Apr 2012 12:56:13 +0100 Subject: [Dovecot] 2.1.3 Corrupted squat uidlist In-Reply-To: <96F0AD8C-9832-4CA5-B1DA-3936F9E2F0FB@iki.fi> References: <4F7164DC.7010706@unict.it> <4F8A97F8.70808@spambox.dk> <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> <96F0AD8C-9832-4CA5-B1DA-3936F9E2F0FB@iki.fi> Message-ID: <9ca41dc4b98ee3c8d0e38f4c2f5d187d@larsson.as> > So the doveadm search itself succeeds without errors, but if you > immediately do another doveadm search it fails? Or do you need to do > something else also in the middle? I couldn't reproduce this with a > few tests. Is there any way to produce debug logs that will show what goes wrong. It is consistent in such a way that I will have several errors within a 24 hour period. I will be willing to add code and rebuild dovecot to produce the logs you need. Best regards Henrik Larsson From tss at iki.fi Mon Apr 23 15:02:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 15:02:37 +0300 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: <4F953F9B.10307@spamgourmet.com> References: <2vp569xe2o.ln2@mail.ela-soft.com> <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> <4F953F9B.10307@spamgourmet.com> Message-ID: <2F3EF8E1-2851-418A-8B43-EE0D91A321C8@iki.fi> Oh, right: On 23.4.2012, at 14.40, Andreas Helmcke wrote: > namespace Allgemeines { Lowercase it: namespace allgemeines { This is a bit annoying bug, but I'm not entirely sure how to fix it properly. From tss at iki.fi Mon Apr 23 15:05:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 15:05:58 +0300 Subject: [Dovecot] 2.1.3 Corrupted squat uidlist In-Reply-To: <9ca41dc4b98ee3c8d0e38f4c2f5d187d@larsson.as> References: <4F7164DC.7010706@unict.it> <4F8A97F8.70808@spambox.dk> <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> <96F0AD8C-9832-4CA5-B1DA-3936F9E2F0FB@iki.fi> <9ca41dc4b98ee3c8d0e38f4c2f5d187d@larsson.as> Message-ID: <3B05C40A-4FC7-479D-8EF3-F21C5DBEEA90@iki.fi> On 23.4.2012, at 14.56, Henrik Larsson wrote: >> So the doveadm search itself succeeds without errors, but if you >> immediately do another doveadm search it fails? Or do you need to do >> something else also in the middle? I couldn't reproduce this with a >> few tests. > > Is there any way to produce debug logs that will show what goes wrong. It is consistent in such a way that I will have several errors within a 24 hour period. > > I will be willing to add code and rebuild dovecot to produce the logs you need. Well, Squat is currently rather low priority to me .. I even considered removing it entirely, since I don't think it's very usable. fts-lucene would work much better. From l.chelchowski at eurocar.pl Mon Apr 23 15:27:08 2012 From: l.chelchowski at eurocar.pl (l.chelchowski at eurocar.pl) Date: Mon, 23 Apr 2012 14:27:08 +0200 Subject: [Dovecot] questions about SPECIAL-USE and virtual Message-ID: <4F954A9C.6000604@eurocar.pl> Hi, I have some questions about Special-Use and virtual namespace: I want to create a virtual namespace with mailboxes: Sent, Unread, Flagged, All. 1. Is it enough to put in the config file: mailbox virtual/Sent { auto = subscribe special_use = \Sent } mailbox virtual/Unread { auto = subscribe special_use = \Unread #Is it correct? } mailbox virtual/Flagged { auto = subscribe special_use = \Flagged } mailbox virtual/All { auto = subscribe special_use = \All } and it would displays mail from every namespace I have subscribed (Shared, Public)? OR I have to make also dovecot-virtual files? 2. In dovecot-virtual file can I put for mailbox virtual/Sent something like: * \Sent 3. Is the http://wiki2.dovecot.org/Plugins/Virtual full documented, if not , where can I find it? 4. Is there a documentation for Special-Use? -- ?ukasz From tss at iki.fi Mon Apr 23 15:33:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 15:33:05 +0300 Subject: [Dovecot] questions about SPECIAL-USE and virtual In-Reply-To: <4F954A9C.6000604@eurocar.pl> References: <4F954A9C.6000604@eurocar.pl> Message-ID: On 23.4.2012, at 15.27, l.chelchowski at eurocar.pl wrote: > I have some questions about Special-Use and virtual namespace: > > I want to create a virtual namespace with mailboxes: Sent, Unread, Flagged, All. > > 1. Is it enough to put in the config file: > mailbox virtual/Sent { > auto = subscribe > special_use = \Sent > } You have to put the mailbox {}es inside the namespace {} you want. Otherwise it just gives "unknown section" error. > OR I have to make also dovecot-virtual files? You have to do this also. The mailbox {} sections don't do anything magically different. They simply add the \Sent flag. The auto=subscribe can automatically subscribe the virtual mailbox, but it can't automatically create it, because new virtual mailboxes can't be created by IMAP clients. > 2. In dovecot-virtual file can I put for mailbox virtual/Sent something like: > * > \Sent No. I'm not sure what you'd want that to do. There is no \Sent message flag. > 3. Is the http://wiki2.dovecot.org/Plugins/Virtual full documented, if not , where can I find it? The only other documentation is the code. > 4. Is there a documentation for Special-Use? http://tools.ietf.org/html/rfc6154 From CMarcus at Media-Brokers.com Mon Apr 23 15:47:04 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 23 Apr 2012 08:47:04 -0400 Subject: [Dovecot] questions about SPECIAL-USE and virtual In-Reply-To: References: <4F954A9C.6000604@eurocar.pl> Message-ID: <4F954F48.7010409@Media-Brokers.com> On 2012-04-23 8:33 AM, Timo Sirainen wrote: > The auto=subscribe can automatically subscribe the virtual mailbox, > but it can't automatically create it, because new virtual mailboxes > can't be created by IMAP clients. Hmmm... Thunderbird has the ability to create virtual mailboxes too, but obviously they are created and handled locally, not on the server... How much trouble would it be to add an API so that an IMAP client *could* create and manage them on a dovecot server? And Maybe this would be a good candidate for an RFC spec? -- Best regards, Charles From tss at iki.fi Mon Apr 23 15:50:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 15:50:36 +0300 Subject: [Dovecot] questions about SPECIAL-USE and virtual In-Reply-To: <4F954F48.7010409@Media-Brokers.com> References: <4F954A9C.6000604@eurocar.pl> <4F954F48.7010409@Media-Brokers.com> Message-ID: <8D2C7878-B981-4DC9-8091-E0058BD361BA@iki.fi> On 23.4.2012, at 15.47, Charles Marcus wrote: > On 2012-04-23 8:33 AM, Timo Sirainen wrote: >> The auto=subscribe can automatically subscribe the virtual mailbox, >> but it can't automatically create it, because new virtual mailboxes >> can't be created by IMAP clients. > > Hmmm... > > Thunderbird has the ability to create virtual mailboxes too, but obviously they are created and handled locally, not on the server... > > How much trouble would it be to add an API so that an IMAP client *could* create and manage them on a dovecot server? And Maybe this would be a good candidate for an RFC spec? There have been a few drafts about virtual mailboxes, but they died at some point. I guess it could be implemented, but I'm less certain that there's actual need for it. From news1204.10.weezy at spamgourmet.com Mon Apr 23 16:30:39 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Mon, 23 Apr 2012 15:30:39 +0200 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: References: <2vp569xe2o.ln2@mail.ela-soft.com> <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> <4F953F9B.10307@spamgourmet.com> Message-ID: <4F95597F.2080908@spamgourmet.com> Am 23.04.2012 13:55, schrieb Timo Sirainen: > > See if this changes the behavior: > > service config { > unix_listener config { > mode = 0666 > } > } > > I'd guess that makes the vmail work also? Although it still doesn't explain the difference between asking config from config process vs. getting it by executing doveconf. Yes, with this entry sending as user vmail is working. Thanks. From news1204.10.weezy at spamgourmet.com Mon Apr 23 16:39:08 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Mon, 23 Apr 2012 15:39:08 +0200 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: <2F3EF8E1-2851-418A-8B43-EE0D91A321C8@iki.fi> References: <2vp569xe2o.ln2@mail.ela-soft.com> <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> <4F953F9B.10307@spamgourmet.com> <2F3EF8E1-2851-418A-8B43-EE0D91A321C8@iki.fi> Message-ID: <4F955B7C.2070009@spamgourmet.com> Am 23.04.2012 14:02, schrieb Timo Sirainen: > On 23.4.2012, at 14.40, Andreas Helmcke wrote: > >> namespace Allgemeines { > > Lowercase it: namespace allgemeines { > Did the trick, also. Sending as user vmail without the service config entry, which you recommended in the other mail, is working. From tss at iki.fi Mon Apr 23 16:49:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 16:49:12 +0300 Subject: [Dovecot] v2.1.5 released Message-ID: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz.sig A few announcements first: I'm now going to start being more strict about not answering Dovecot questions sent to me privately (especially support requests, but also bug reports, feature requests, etc). Often those questions could be answered by other people in the mailing list as well, so they just waste my time. Even if you're sure that I'm the only person who can possibly answer it, it may be something that others are interested of as well and replying privately denies them the possibility of reading about it. If you really do not want to send something publicly, you still have an option for private communication: just buy one of the Dovecot support packages. This is also the first Dovecot release that has run through automated release testing. I've been planning on doing that for last 5 years at least, and finally it's done. The testing includes running a lot of tests to verify that the IMAP server gives proper responses, stress testing doesn't break it, memory isn't leaked and several other things. The tests are run for many different kinds of configurations. Some bugs show up only for some specific combination of settings, but unfortunately because there's such a large number of them it's impossible to test every combination. So here's another advantage that Dovecot support customers have: they can have their specific configuration included in the automated tests. This release already fixes one bug I found only after testing a customer's config. Oh, and looks like I got accepted as a speaker to LinuxTag 2012. The plan is to talk about some new things in Dovecot and dsync replication (looks like I have to hurry up with the dsync rewrite). And finally, the most important changes since v2.1.4: * IMAP: When neither the session nor the mailbox has modseq tracking enabled, return the mailbox as having NOMODSEQ in SELECT/EXAMINE reply. Old versions in this situation always simply returned HIGHESTMODSEQ as 1, which could have broken some clients. + dict file: Added optional fcntl/flock locking (default is dotlock) + fts-solr: doveadm fts rescan now resets indexes, which allows reindexing mails. (This isn't a full rescan implementation like fts-lucene has.) + doveadm expunge: Added -d parameter to delete mailbox if it's empty after expunging. - IMAP: Several fixes related to mailbox listing in some configs - director: A lot of fixes and performance improvements - v2.1.4 didn't work without a mail home directory set - mbox: Deleting a mailbox didn't delete its index files. - pop3c: TOP command was sent incorrectly - trash plugin didn't work properly - LMTP: Don't add a duplicate Return-Path: header when proxying. - listescape: Don't unescape namespace prefixes. From stsiol at yahoo.co.uk Mon Apr 23 17:09:17 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 23 Apr 2012 15:09:17 +0100 (BST) Subject: [Dovecot] v2.1.5 released In-Reply-To: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <1335190157.12813.YahooMailNeo@web132202.mail.ird.yahoo.com> >Oh, and looks like I got accepted as a speaker to LinuxTag 2012. The plan is to talk about some new things in Dovecot and dsync replication (looks like I have to hurry up with the dsync rewrite). > I am quite amazed they haven't called you earlier. From what it seems you are doing a great job and dovecot is a rock-solid piece of software. Just my thoughts, s. ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From giles at coochey.net Mon Apr 23 17:12:50 2012 From: giles at coochey.net (Giles Coochey) Date: Mon, 23 Apr 2012 15:12:50 +0100 Subject: [Dovecot] v2.1.5 released In-Reply-To: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <4F956362.7010507@coochey.net> On 23/04/2012 14:49, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz > http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz.sig > > A few announcements first: > > I'm now going to start being more strict about not answering Dovecot questions sent to me privately (especially support requests, but also bug reports, feature requests, etc). Often those questions could be answered by other people in the mailing list as well, so they just waste my time. Even if you're sure that I'm the only person who can possibly answer it, it may be something that others are interested of as well and replying privately denies them the possibility of reading about it. If you really do not want to send something publicly, you still have an option for private communication: just buy one of the Dovecot support packages. > I'd just like to say thanks for the amount of time you put in not just developing the software, but supporting your userbase. I am sure everyone here is well aware of when Timo signs in each day as a flood of replies to queries on the dovecot mailing list starts to arrive to our mailboxes. It is very much appreciated, there are many OSS mailing-lists out there, but not many as vibrant as dovecots! -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4928 bytes Desc: S/MIME Cryptographic Signature URL: From l.chelchowski at eurocar.pl Mon Apr 23 17:27:54 2012 From: l.chelchowski at eurocar.pl (l.chelchowski at eurocar.pl) Date: Mon, 23 Apr 2012 16:27:54 +0200 Subject: [Dovecot] virtual plugin and shared namespace In-Reply-To: References: <4F4235B8.6090107@eurocar.pl> Message-ID: <4F9566EA.6040109@eurocar.pl> W dniu 2012-02-21 01:44, Timo Sirainen pisze: > On 20.2.2012, at 13.59, l.chelchowski at eurocar.pl wrote: > >> I have a problem with virtual plugin and geting mails form shared namespace. >> dovecot-virtual: >> ---- >> Shared/* >> Public/* >> * > .. >> When I enter the virtual mailbox Unread I get this debug info: >> virtual mailbox Virtual/Unread: Skipping non-existing mailbox Shared/test at domain.pl/ > * matches only within namespace. Shared/* matches only the entries directly under Shared/, i.e. Shared/test at domain.pl itself. Shared/test at domain.pl/* would again be a separate namespace. > >> Is it a bug or misconfig? > Maybe a "missing feature" :) I guess Shared/* should match all autocreated child namespaces.. This should help: > http://hg.dovecot.org/dovecot-2.1/rev/10994a5b64e1 > I have installed version 2.1.3 and this patch doesn't work as expected. I can't use Shared/*. Only Shared/test at domain.pl/* works. -- Pozdrawiam ?ukasz Che?chowski Informatyk Autoryzowany Dealer Ford Euro-Car Sp. z o.o. tel. 58 667 327 mailto: l.chelchowski at eurocar.pl -------------- next part -------------- A non-text attachment was scrubbed... Name: stopka2.gif Type: image/gif Size: 17309 bytes Desc: not available URL: From andrei at lctax.ro Mon Apr 23 19:17:35 2012 From: andrei at lctax.ro (Michescu Andrei) Date: Mon, 23 Apr 2012 12:17:35 -0400 Subject: [Dovecot] v2.1.5 released In-Reply-To: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <26fed0264da5829b3376056abd7dd1fd.squirrel@web.miau.ca> > Oh, and looks like I got accepted as a speaker to LinuxTag 2012. The plan > is to talk about some new things in Dovecot and dsync replication (looks > like I have to hurry up with the dsync rewrite). > Hey Timo, Thank you very much for your hard work ;) Please let us know if there is anything that we can help you clarify / design for the dsync rewrite project. Best regards, Andrei From cor at xs4all.nl Mon Apr 23 21:34:39 2012 From: cor at xs4all.nl (Cor Bosman) Date: Mon, 23 Apr 2012 20:34:39 +0200 Subject: [Dovecot] quota not being calculated In-Reply-To: <96E3D11C-8F71-4726-816A-6C448554BC70@iki.fi> References: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> <2D57E521-3DB3-41D9-B8AB-F280DE546061@iki.fi> <96E3D11C-8F71-4726-816A-6C448554BC70@iki.fi> Message-ID: > > http://hg.dovecot.org/dovecot-2.1/rev/4c8f79d1f9f1 should fix it with dict quota. Thank you, this fixed it with dict quota. Cor From dovecot at allycomm.com Mon Apr 23 22:56:13 2012 From: dovecot at allycomm.com (Jeff Kletsky) Date: Mon, 23 Apr 2012 12:56:13 -0700 Subject: [Dovecot] dsync on large mailbox "fails" Message-ID: <4F95B3DD.1070609@allycomm.com> I've been trying to convert an existing set of mbox mail to maildir using dsync, but it seems to fail when it gets to the large boxes. $ dsync -Dv mirror maildir:~/Maildir Everything seems to work reasonably well until the first of the large boxes hits: dsync(jeff): Warning: Maildir /home/jeff/Maildir/.cron.2010-07-10: Synchronization took 1210 seconds (102289 new msgs, 0 flag change attempts, 0 expunge attempts) dsync(jeff): Warning: Transaction log file /home/jeff/Maildir/.cron.2010-07-10/dovecot.index.log was locked for 1210 seconds After this, the doveadm process goes from ~75% of the CPU core utilization to 100% and stays there, even overnight. I've also tried $ dsync -Dv backup maildir:~/Maildir with similar results. While I did see some threads on dsync speed, they seemed to be minutes for a few GB of data, not hours. There is plenty of space on the drive, and, from what I understand, there is not a file-count limit on FreeBSD's ufs (at least not one that I'm anywhere near) Any suggestions? Thanks! Jeff [jeff at mail ~]$ du -h -d 0 ~/mail 3.9G /home/jeff/mail [jeff at mail ~]$ ls -lh /var/mail/jeff -rw------- 1 jeff jeff 554M Apr 23 11:27 /var/mail/jeff [jeff at mail ~]$ df -h Filesystem Size Used Avail Capacity Mounted on /dev/ada0s2g 38G 18G 16G 53% / (Note that the hostname and the location of the ssl certificate has been manually edited for this message in the following output) From dmesg.boot: Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-STABLE #0: Sat Apr 21 17:13:07 PDT 2012 root at my.host.name:/usr/obj/usr/src/sys/GENERIC i386 CPU: Intel(R) Atom(TM) CPU 330 @ 1.60GHz (1600.03-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x106c2 Family = 6 Model = 1c Stepping = 2 Features=0xbfe9fbff Features2=0x40e31d AMD Features=0x20100000 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 2147483648 (2048 MB) avail memory = 2077757440 (1981 MB) [jeff at mail ~]$ doveconf -n # 2.1.3: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 9.0-STABLE i386 auth_mechanisms = plain login disable_plaintext_auth = no first_valid_gid = 1000 first_valid_uid = 1000 listen = * mail_location = mbox:~/mail/:INBOX=/var/mail/%u mail_privileged_group = mail namespace { inbox = yes location = prefix = separator = / } passdb { args = session=yes dovecot driver = pam } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { mode = 0600 } user = root } service imap-login { inet_listener imap { port = 0 } process_limit = 200 } ssl = required ssl_cert = Hey all, has anyone ever tried turning on the trash plugin when the directory is already over quota? I see some messages being deleted, but it seems it just deletes enough to fit the new email, not enough to go below quota. Example: . getquotaroot "Spam" * QUOTAROOT "Spam" "User quota" "Spam Quota" * QUOTA "User quota" (STORAGE 591036 5242880) * QUOTA "Spam Quota" (STORAGE 21173 20480) I now move a 5K message into this folder, and I see: Apr 23 22:26:43 lmtp1 dovecot: imap(cor): expunge: box=Spam, uid=226337, msgid=<3YVKIIE4II6R0064BA7IM55C2Q4MD01VX12U1I at shedevils.com>, size=12984 Apr 23 22:26:43 lmtp1 dovecot: imap(cor): expunge: box=Spam, uid=226338, msgid=, size=5623 Apr 23 22:26:43 lmtp1 dovecot: imap(cor): save: box=Spam, uid=227759, msgid=<20120423191509.F3B584A355B at signal.groundspeak.com>, size=2694 And now: . getquotaroot "Spam" * QUOTAROOT "Spam" "User quota" "Spam Quota" * QUOTA "User quota" (STORAGE 591016 5242880) * QUOTA "Spam Quota" (STORAGE 21158 20480) . OK Getquotaroot completed. This seems like a possible bug. For one, it shouldnt be allowed to actually save a message into that folder, but it does. The email does end up in the Spam folder even though the folder is over quota. And of course it doesnt clear up enough space to have the new email fit, it just deleted enough to match the 5K. Without the trash plugin quota is being properly enforced. Everything works fine if you start out below the quota limit btw. Regards, Cor ps: this is 2.1.4, but with the recent trash and quota patches applied. From asai at globalchangemusic.org Tue Apr 24 00:50:36 2012 From: asai at globalchangemusic.org (Asai) Date: Mon, 23 Apr 2012 14:50:36 -0700 Subject: [Dovecot] v2.1.5 released In-Reply-To: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <4F95CEAC.10103@globalchangemusic.org> Let me also thank you, Timo, for your selfless excellence in developing Dovecot. --asai On 4/23/2012 6:49 AM, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz > http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz.sig > > A few announcements first: > > I'm now going to start being more strict about not answering Dovecot questions sent to me privately (especially support requests, but also bug reports, feature requests, etc). Often those questions could be answered by other people in the mailing list as well, so they just waste my time. Even if you're sure that I'm the only person who can possibly answer it, it may be something that others are interested of as well and replying privately denies them the possibility of reading about it. If you really do not want to send something publicly, you still have an option for private communication: just buy one of the Dovecot support packages. > > This is also the first Dovecot release that has run through automated release testing. I've been planning on doing that for last 5 years at least, and finally it's done. The testing includes running a lot of tests to verify that the IMAP server gives proper responses, stress testing doesn't break it, memory isn't leaked and several other things. The tests are run for many different kinds of configurations. > > Some bugs show up only for some specific combination of settings, but unfortunately because there's such a large number of them it's impossible to test every combination. So here's another advantage that Dovecot support customers have: they can have their specific configuration included in the automated tests. This release already fixes one bug I found only after testing a customer's config. > > Oh, and looks like I got accepted as a speaker to LinuxTag 2012. The plan is to talk about some new things in Dovecot and dsync replication (looks like I have to hurry up with the dsync rewrite). > > And finally, the most important changes since v2.1.4: > > * IMAP: When neither the session nor the mailbox has modseq tracking > enabled, return the mailbox as having NOMODSEQ in SELECT/EXAMINE > reply. Old versions in this situation always simply returned > HIGHESTMODSEQ as 1, which could have broken some clients. > > + dict file: Added optional fcntl/flock locking (default is dotlock) > + fts-solr: doveadm fts rescan now resets indexes, which allows > reindexing mails. (This isn't a full rescan implementation like > fts-lucene has.) > + doveadm expunge: Added -d parameter to delete mailbox if it's > empty after expunging. > - IMAP: Several fixes related to mailbox listing in some configs > - director: A lot of fixes and performance improvements > - v2.1.4 didn't work without a mail home directory set > - mbox: Deleting a mailbox didn't delete its index files. > - pop3c: TOP command was sent incorrectly > - trash plugin didn't work properly > - LMTP: Don't add a duplicate Return-Path: header when proxying. > - listescape: Don't unescape namespace prefixes. > From dovecot at allycomm.com Tue Apr 24 06:38:16 2012 From: dovecot at allycomm.com (Jeff Kletsky) Date: Mon, 23 Apr 2012 20:38:16 -0700 Subject: [Dovecot] dsync on large mailbox "fails" In-Reply-To: <4F95B3DD.1070609@allycomm.com> References: <4F95B3DD.1070609@allycomm.com> Message-ID: <4F962028.3060501@allycomm.com> On 4/23/12 12:56 PM, Jeff Kletsky wrote: > I've been trying to convert an existing set of mbox mail to maildir > using dsync, but it seems to fail when it gets to the large boxes. > > $ dsync -Dv mirror maildir:~/Maildir > > Everything seems to work reasonably well until the first of the large > boxes hits: > > dsync(jeff): Warning: Maildir /home/jeff/Maildir/.cron.2010-07-10: > Synchronization took 1210 seconds (102289 new msgs, 0 flag change > attempts, 0 expunge attempts) > dsync(jeff): Warning: Transaction log file > /home/jeff/Maildir/.cron.2010-07-10/dovecot.index.log was locked for > 1210 seconds > > After this, the doveadm process goes from ~75% of the CPU core > utilization to 100% and stays there, even overnight. > I believe that some of problem may be that there were a handful of boxes named with "." in them. These were not "flagged" in the conversion process in any way. http://wiki2.dovecot.org/Migration/MailFormat indicates that you can't have a "." in the name of a Maildir box (I dutifully stopped reading that page at "If you can successfully use dsync, you can skip the rest of this page.") In my case, it may be a contributing factor that there were boxes that ended up "stacking" inside each other: * cron * cron.2010-07-10 * cron.2010-07-10.remaining Still not blisteringly fast, but dsync(jeff): Warning: Maildir /home/jeff/Maildir/.cron_2010-07-10: Synchronization took 686 seconds (102289 new msgs, 0 flag change attempts, 0 expunge attempts) dsync(jeff): Warning: Transaction log file /home/jeff/Maildir/.cron_2010-07-10/dovecot.index.log was locked for 686 seconds is certainly better. I'm still seeing 100% CPU utilization, with the tail of ls -Altr in ~/Maildir showing drwx------ 5 jeff jeff 512 Apr 23 18:19 .XDA drwx------ 5 jeff jeff 512 Apr 23 18:19 .Spam drwx------ 5 jeff jeff 512 Apr 23 18:42 .Health drwx------ 5 jeff jeff 512 Apr 23 18:42 .cron_2010-07-10_remaining drwx------ 5 jeff jeff 512 Apr 23 18:58 .cron_2010-07-10 It appears from looking in the .cron_2010-07-10 directory that indexing is the action underway [jeff at mail ~/Maildir]$ ls -Altr .cron_2010-07-10 total 27 drwx------ 2 jeff jeff 512 Apr 23 18:15 new -rw------- 1 jeff jeff 0 Apr 23 18:15 maildirfolder drwx------ 2 jeff jeff 5819392 Apr 23 18:42 tmp drwx------ 2 jeff jeff 5237248 Apr 23 18:42 cur -rw------- 1 jeff jeff 4122971 Apr 23 18:42 dovecot-uidlist -rw------- 1 jeff jeff 36 Apr 23 18:42 dovecot-keywords -rw------- 1 jeff jeff 1650764 Apr 23 18:42 dovecot.index.log.2 -rw------- 1 jeff jeff 1637008 Apr 23 18:42 dovecot.index -rw------- 1 jeff jeff 56124 Apr 23 19:04 dovecot.index.log -rw------- 1 jeff jeff 9442304 Apr 23 19:04 dovecot.index.cache -rw------- 1 jeff jeff 24 Apr 23 19:04 dovecot-uidlist.lock (time passes) It is now 20:35 and it has not advanced. I now no longer believe the problem is the mailbox naming, but the indexing. PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 25934 jeff 1 103 0 135M 102M CPU2 2 128:47 100.00% doveadm [jeff at mail ~/Maildir]$ ls -Altr .cron_2010-07-10 total 28 drwx------ 2 jeff jeff 512 Apr 23 18:15 new -rw------- 1 jeff jeff 0 Apr 23 18:15 maildirfolder drwx------ 2 jeff jeff 5819392 Apr 23 18:42 tmp drwx------ 2 jeff jeff 5237248 Apr 23 18:42 cur -rw------- 1 jeff jeff 4122971 Apr 23 18:42 dovecot-uidlist -rw------- 1 jeff jeff 36 Apr 23 18:42 dovecot-keywords -rw------- 1 jeff jeff 1650764 Apr 23 18:42 dovecot.index.log.2 -rw------- 1 jeff jeff 1637008 Apr 23 20:26 dovecot.index -rw------- 1 jeff jeff 289120 Apr 23 20:36 dovecot.index.log -rw------- 1 jeff jeff 9442304 Apr 23 20:36 dovecot.index.cache -rw------- 1 jeff jeff 24 Apr 23 20:36 dovecot-uidlist.lock Any suggestions as to how to proceed? Jeff From robert at schetterer.org Tue Apr 24 09:56:26 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 24 Apr 2012 08:56:26 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works / solved by remove listescape In-Reply-To: References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> <4F90250E.3000200@schetterer.org> <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> <4F903DC1.5070501@schetterer.org> <4F905B2C.8040204@schetterer.org> <4F907529.2020403@schetterer.org> Message-ID: <4F964E9A.2050401@schetterer.org> Am 23.04.2012 13:12, schrieb Timo Sirainen: > On 19.4.2012, at 23.27, Robert Schetterer wrote: > >>> prefix = shared/%%u/ simply dont works >>> >>> i have reverted back to exact namespace config >>> of the 2.0.20 server, same result, so something broken in 2.1 >>> >>> >> Hi Timo >> problem solved by removing listescape plugin >> the acl paths did / instead of dot in domainname >> >> any chance to get listescape fixed ? > > Ah, that makes it easy to fix: http://hg.dovecot.org/dovecot-2.1/rev/63af3274fb6f > Hi Timo, for small tests problem seems to be fixed, thx youre great -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Tue Apr 24 10:50:06 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 24 Apr 2012 09:50:06 +0200 Subject: [Dovecot] doveadm delete folders ? / unsubscribe ? In-Reply-To: <4F906EE8.1030900@schetterer.org> References: <4F900066.9030808@schetterer.org> <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> <4F906EE8.1030900@schetterer.org> Message-ID: <4F965B2E.2010103@schetterer.org> Am 19.04.2012 22:00, schrieb Robert Schetterer: > Am 19.04.2012 17:16, schrieb Timo Sirainen: >> On 19.4.2012, at 15.09, Robert Schetterer wrote: >> >>> Hi , >>> is there a feature like >>> doveadm expunge savedbefore ... >>> for imap folder delete ? >> >> Here: >> >> http://hg.dovecot.org/dovecot-2.1/rev/2d8bafd11569 >> http://hg.dovecot.org/dovecot-2.1/rev/782570f644f7 >> > > thx Timo ! > Hi Timo doveadm expunge -d -u *@domain.example mailbox exampelfolder\* savedbefore 10d works like charm, but the folders do not get unsubscribed any chance to have i.e another add parameter ,or do it auto cause empty and deleted folders should not stay subscribed in my eyes -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From lists at wiesinger.com Tue Apr 24 11:59:20 2012 From: lists at wiesinger.com (Gerhard Wiesinger) Date: Tue, 24 Apr 2012 10:59:20 +0200 (CEST) Subject: [Dovecot] v2.1.5 released In-Reply-To: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: Hello Timo, Dovecot 2.1.5 seems to be broken. After upgrade from 2.1.3 + patch http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e I only see Inbox and Trash folder. Reverting => OK. Same config as before and disucssed per private E-mail and on the list. Guessing this comes again from: mail_full_filesystem_access=yes Ciao, Gerhard -- http://www.wiesinger.com/ On Mon, 23 Apr 2012, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz > http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz.sig > > A few announcements first: > > I'm now going to start being more strict about not answering Dovecot questions sent to me privately (especially support requests, but also bug reports, feature requests, etc). Often those questions could be answered by other people in the mailing list as well, so they just waste my time. Even if you're sure that I'm the only person who can possibly answer it, it may be something that others are interested of as well and replying privately denies them the possibility of reading about it. If you really do not want to send something publicly, you still have an option for private communication: just buy one of the Dovecot support packages. > > This is also the first Dovecot release that has run through automated release testing. I've been planning on doing that for last 5 years at least, and finally it's done. The testing includes running a lot of tests to verify that the IMAP server gives proper responses, stress testing doesn't break it, memory isn't leaked and several other things. The tests are run for many different kinds of configurations. > > Some bugs show up only for some specific combination of settings, but unfortunately because there's such a large number of them it's impossible to test every combination. So here's another advantage that Dovecot support customers have: they can have their specific configuration included in the automated tests. This release already fixes one bug I found only after testing a customer's config. > > Oh, and looks like I got accepted as a speaker to LinuxTag 2012. The plan is to talk about some new things in Dovecot and dsync replication (looks like I have to hurry up with the dsync rewrite). > > And finally, the most important changes since v2.1.4: > > * IMAP: When neither the session nor the mailbox has modseq tracking > enabled, return the mailbox as having NOMODSEQ in SELECT/EXAMINE > reply. Old versions in this situation always simply returned > HIGHESTMODSEQ as 1, which could have broken some clients. > > + dict file: Added optional fcntl/flock locking (default is dotlock) > + fts-solr: doveadm fts rescan now resets indexes, which allows > reindexing mails. (This isn't a full rescan implementation like > fts-lucene has.) > + doveadm expunge: Added -d parameter to delete mailbox if it's > empty after expunging. > - IMAP: Several fixes related to mailbox listing in some configs > - director: A lot of fixes and performance improvements > - v2.1.4 didn't work without a mail home directory set > - mbox: Deleting a mailbox didn't delete its index files. > - pop3c: TOP command was sent incorrectly > - trash plugin didn't work properly > - LMTP: Don't add a duplicate Return-Path: header when proxying. > - listescape: Don't unescape namespace prefixes. > > > From alessio at skye.it Tue Apr 24 12:15:08 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 24 Apr 2012 11:15:08 +0200 Subject: [Dovecot] v2.1.5 released In-Reply-To: References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <4F966F1C.2080204@skye.it> Il 24/04/2012 10:59, Gerhard Wiesinger ha scritto: > Hello Timo, > > Dovecot 2.1.5 seems to be broken. After upgrade from > 2.1.3 + patch http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e > I only see Inbox and Trash folder. > > Reverting => OK. > > Same config as before and disucssed per private E-mail and on the list. > Guessing this comes again from: > mail_full_filesystem_access=yes > > Ciao, > Gerhard Hi Gerhard, for us 2.1.5 works fine. We are using Maildir as mail storage type. Ciao -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From joseba.torre at ehu.es Tue Apr 24 12:25:38 2012 From: joseba.torre at ehu.es (Joseba Torre) Date: Tue, 24 Apr 2012 11:25:38 +0200 Subject: [Dovecot] Error compiling pigeonhole 0.3.0 with dovecot-2.1.5 in openbsd 5.0 Message-ID: <4F967192.2060701@ehu.es> Hi, I've just tried to compile dovecot & pigeonhole in my openbsd box, and got this: gcc -DHAVE_CONFIG_H -I. -I../.. -I/home/seajob/build/dovecot-2.1.5 -I/home/seajob/build/dovecot-2.1.5/src/lib -I/home/seajob/build/dovecot-2.1.5/src/lib-dict -I/home/seajob/build/dovecot-2.1.5/src/lib-dns -I/home/seajob/build/dovecot-2.1.5/src/lib-mail -I/home/seajob/build/dovecot-2.1.5/src/lib-imap -I/home/seajob/build/dovecot-2.1.5/src/lib-fs -I/home/seajob/build/dovecot-2.1.5/src/lib-charset -I/home/seajob/build/dovecot-2.1.5/src/lib-index -I/home/seajob/build/dovecot-2.1.5/src/lib-storage -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index/raw -I../.. -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/include -MT managesieve-quote.o -MD -MP -MF .deps/managesieve-quote.Tpo -c -o managesieve-quote.o managesieve-quote.c mv -f .deps/managesieve-quote.Tpo .deps/managesieve-quote.Po gcc -DHAVE_CONFIG_H -I. -I../.. -I/home/seajob/build/dovecot-2.1.5 -I/home/seajob/build/dovecot-2.1.5/src/lib -I/home/seajob/build/dovecot-2.1.5/src/lib-dict -I/home/seajob/build/dovecot-2.1.5/src/lib-dns -I/home/seajob/build/dovecot-2.1.5/src/lib-mail -I/home/seajob/build/dovecot-2.1.5/src/lib-imap -I/home/seajob/build/dovecot-2.1.5/src/lib-fs -I/home/seajob/build/dovecot-2.1.5/src/lib-charset -I/home/seajob/build/dovecot-2.1.5/src/lib-index -I/home/seajob/build/dovecot-2.1.5/src/lib-storage -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index/raw -I../.. -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/include -MT managesieve-parser.o -MD -MP -MF .deps/managesieve-parser.Tpo -c -o managesieve-parser.o managesieve-parser.c managesieve-parser.c: In function 'quoted_string_istream_read': managesieve-parser.c:639: error: 'EPROTO' undeclared (first use in this function) managesieve-parser.c:639: error: (Each undeclared identifier is reported only once managesieve-parser.c:639: error: for each function it appears in.) *** Error code 1 Stop in /home/seajob/build/dovecot-2.1-pigeonhole-0.3.0/src/lib-managesieve (line 92 of /usr/share/mk/sys.mk). *** Error code 1 Stop in /home/seajob/build/dovecot-2.1-pigeonhole-0.3.0/src (line 289 of Makefile). *** Error code 1 Stop in /home/seajob/build/dovecot-2.1-pigeonhole-0.3.0 (line 474 of Makefile). *** Error code 1 Stop in /home/seajob/build/dovecot-2.1-pigeonhole-0.3.0 (line 353 of Makefile). From piotr.maier at gmail.com Tue Apr 24 12:51:43 2012 From: piotr.maier at gmail.com (Piotr Maier) Date: Tue, 24 Apr 2012 11:51:43 +0200 Subject: [Dovecot] Dovecot Virtual Users with Users Domain Mapping Message-ID: I have successfully compiled, configured and ran Dovecot with virtual users feature. Here's part of my /etc/dovecot.conf configuration file: mail_location = maildir:~/Maildir auth default { mechanisms = plain login userdb passwd-file { args = /home/%d/etc/passwd } passdb passwd-file { args = /home/%d/etc/shadow } socket listen { master { path = /var/run/dovecot/auth-worker mode = 0600 } } } I faced one issue I can't resolve myself. Is there anyway to create users' domains mapping and provide username in mail_location? Example of usersdomains file: domain.com:user1 domain2.org:user3 Examples: 1. currently I have /home/domain.com/user/Maildir 2. I'd like to have /home/SYSTEM_USER/domain.com/user/Maildir Can I achieve this somehow? From tss at iki.fi Tue Apr 24 13:26:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 24 Apr 2012 13:26:32 +0300 Subject: [Dovecot] dsync on large mailbox "fails" In-Reply-To: <4F962028.3060501@allycomm.com> References: <4F95B3DD.1070609@allycomm.com> <4F962028.3060501@allycomm.com> Message-ID: <4EE7C62E-928C-499D-BE4E-9944D6013198@iki.fi> On 24.4.2012, at 6.38, Jeff Kletsky wrote: > 25934 jeff 1 103 0 135M 102M CPU2 2 128:47 100.00% doveadm Looks like it could be in an infinite loop. To find out where: 1) strace for a few seconds: strace -tt -p 25934 2>log 2) get gdb backtrace a couple of times (2-5 should be enough, preferably slightly different ones): gdb /usr/local/bin/doveadm 25934 bt full cont bt full cont ..etc.. From markus at mpetri.org Tue Apr 24 14:00:05 2012 From: markus at mpetri.org (Markus Petri) Date: Tue, 24 Apr 2012 13:00:05 +0200 Subject: [Dovecot] Permission problems when trying to create new folders with ACL plugin Message-ID: <20120424130005.0db3d425@legolas.home.ceotex.de> Hi, I have the a problem with shared folders and the ACL plugin. The user "markus" has full access to "Shared/shared-user/Data": 3 GETACL "Shared/shared-user/Data" * ACL "Shared/shared-user/Data" "markus" akxeilprwtscd 3 OK Getacl completed. but creating a new subfolder under "Data" fails: 4 CREATE "Shared/shared-user/Data/New1" 4 NO [NOPERM] Permission denied This used to work with the same configuration and Dovecot 2.0. Cheers, Markus dovecot -n output: # 2.1.5: /opt/dovecot-2.1/etc/dovecot/dovecot.conf # OS: Linux 3.2.0-2-amd64 x86_64 Debian wheezy/sid auth_mechanisms = plain login disable_plaintext_auth = no listen = 192.168.56.11 mail_location = maildir:~/Maildir mail_plugins = acl managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = prefix = separator = / subscriptions = yes type = private } namespace { inbox = no list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/index/shared/%%u prefix = Shared/%%u/ separator = / subscriptions = no type = shared } passdb { args = /opt/dovecot-2.1/etc/dovecot/passwd driver = passwd-file } plugin { acl = vfile acl_anyone = allow acl_shared_dict = file:/var/lib/vdovecot/shared-mailboxes.db } protocols = imap service auth { unix_listener auth-userdb { mode = 0600 user = vdovecot } } ssl = no userdb { args = /opt/dovecot-2.1/etc/dovecot/passwd driver = passwd-file } verbose_proctitle = yes protocol imap { mail_plugins = acl imap_acl } From stephan at rename-it.nl Tue Apr 24 14:13:54 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 24 Apr 2012 13:13:54 +0200 Subject: [Dovecot] Error compiling pigeonhole 0.3.0 with dovecot-2.1.5 in openbsd 5.0 In-Reply-To: <4F967192.2060701@ehu.es> References: <4F967192.2060701@ehu.es> Message-ID: <4F968AF2.4090807@rename-it.nl> Op 4/24/2012 11:25 AM, Joseba Torre schreef: > Hi, > > I've just tried to compile dovecot & pigeonhole in my openbsd box, and > got this: > > gcc -DHAVE_CONFIG_H -I. -I../.. -I/home/seajob/build/dovecot-2.1.5 > -I/home/seajob/build/dovecot-2.1.5/src/lib > -I/home/seajob/build/dovecot-2.1.5/src/lib-dict > -I/home/seajob/build/dovecot-2.1.5/src/lib-dns > -I/home/seajob/build/dovecot-2.1.5/src/lib-mail > -I/home/seajob/build/dovecot-2.1.5/src/lib-imap > -I/home/seajob/build/dovecot-2.1.5/src/lib-fs > -I/home/seajob/build/dovecot-2.1.5/src/lib-charset > -I/home/seajob/build/dovecot-2.1.5/src/lib-index > -I/home/seajob/build/dovecot-2.1.5/src/lib-storage > -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index > -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index/raw -I../.. > -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes > -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 > -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 > -I/usr/include -MT managesieve-quote.o -MD -MP -MF > .deps/managesieve-quote.Tpo -c -o managesieve-quote.o managesieve-quote.c > > mv -f .deps/managesieve-quote.Tpo .deps/managesieve-quote.Po > gcc -DHAVE_CONFIG_H -I. -I../.. -I/home/seajob/build/dovecot-2.1.5 > -I/home/seajob/build/dovecot-2.1.5/src/lib > -I/home/seajob/build/dovecot-2.1.5/src/lib-dict > -I/home/seajob/build/dovecot-2.1.5/src/lib-dns > -I/home/seajob/build/dovecot-2.1.5/src/lib-mail > -I/home/seajob/build/dovecot-2.1.5/src/lib-imap > -I/home/seajob/build/dovecot-2.1.5/src/lib-fs > -I/home/seajob/build/dovecot-2.1.5/src/lib-charset > -I/home/seajob/build/dovecot-2.1.5/src/lib-index > -I/home/seajob/build/dovecot-2.1.5/src/lib-storage > -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index > -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index/raw -I../.. > -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes > -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 > -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 > -I/usr/include -MT managesieve-parser.o -MD -MP -MF > .deps/managesieve-parser.Tpo -c -o managesieve-parser.o > managesieve-parser.c > managesieve-parser.c: In function 'quoted_string_istream_read': > managesieve-parser.c:639: error: 'EPROTO' undeclared (first use in > this function) > managesieve-parser.c:639: error: (Each undeclared identifier is > reported only once > managesieve-parser.c:639: error: for each function it appears in.) Looks like BSD doesn't know EPROTO (although it is POSIX). Dovecot uses EIO elsewhere for such stream errors, so I adopted that for ManageSieve as well. This should fix it: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/ef5045159e6b Regards, Stephan. From ah-news-1204 at freenet.de Tue Apr 24 17:09:58 2012 From: ah-news-1204 at freenet.de (Andreas Helmcke) Date: Tue, 24 Apr 2012 16:09:58 +0200 Subject: [Dovecot] acls not copied when creating subfolder of private INBOX Message-ID: <4F96B436.9090402@freenet.de> Using brand new dovecot 2.1.5 I still have a problem with the inheritance of acls. Setting: - maildir with private INBOX and public folders. - users maildir directory (/home/mail/user/ahelmcke) contains dovecot-acl file - public folders root directory (/home/mail/Fax) contains dovecot-acl file When creating a subfolder of the public folder the dovecot-acl files gets copied to the subfolders directory as expected. When creating a subfolder of the users INBOX the dovecot-acl files does /not/ get copied to the subfolders directory. Debug output: doveadm -Dv mailbox create -u ahelmcke Fax/Buhhhh doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib01_acl_plugin.so doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib10_quota_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_lookup (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_zlib_plugin.so: undefined symbol: i_stream_create_deflate (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_list_backend (this is usually intentional, so just ignore this message) doveadm(ahelmcke): Debug: auth input: ahelmcke quota_rule=*:storage=2000M doveadm(ahelmcke): Debug: Added userdb setting: plugin/quota_rule=*:storage=2000M doveadm(ahelmcke): Debug: Effective uid=494, gid=491, home=/home/mail/user/ahelmcke doveadm(ahelmcke): Debug: Quota root: name=User quota backend=maildir args= doveadm(ahelmcke): Debug: Quota rule: root=User quota mailbox=* bytes=2097152000 messages=0 doveadm(ahelmcke): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke doveadm(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=/home/mail/user/ahelmcke, alt= doveadm(ahelmcke): Debug: acl: initializing backend with data: vfile doveadm(ahelmcke): Debug: acl: acl username = ahelmcke doveadm(ahelmcke): Debug: acl: owner = 1 doveadm(ahelmcke): Debug: acl vfile: Global ACL directory: (none) doveadm(ahelmcke): Debug: Namespace dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ doveadm(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke/public, index=, control=, inbox=, alt= doveadm(ahelmcke): Debug: acl: initializing backend with data: vfile doveadm(ahelmcke): Debug: acl: acl username = ahelmcke doveadm(ahelmcke): Debug: acl: owner = 1 doveadm(ahelmcke): Debug: acl vfile: Global ACL directory: (none) doveadm(ahelmcke): Debug: Namespace fax: type=public, prefix=Fax/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no location=maildir:/home/mail/Fax:INDEX=/home/mail/user/ahelmcke/SEEN.Fax doveadm(ahelmcke): Debug: maildir++: root=/home/mail/Fax, index=/home/mail/user/ahelmcke/SEEN.Fax, control=, inbox=, alt= doveadm(ahelmcke): Debug: acl: initializing backend with data: vfile doveadm(ahelmcke): Debug: acl: acl username = ahelmcke doveadm(ahelmcke): Debug: acl: owner = 0 doveadm(ahelmcke): Debug: acl vfile: Global ACL directory: (none) doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/dovecot-acl doveadm(ahelmcke): Debug: Namespace Fax/: Using permissions from /home/mail/Fax: mode=0700 gid=-1 doveadm(ahelmcke): Debug: Namespace INBOX/: Using permissions from /home/mail/user/ahelmcke: mode=0700 gid=-1 doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/user/ahelmcke/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/user/ahelmcke/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/user/ahelmcke/dovecot-acl doveadm(ahelmcke): Debug: Namespace Fax/: /home/mail/Fax/.Buhhhh doesn't exist yet, using default permissions doveadm(ahelmcke): Debug: Namespace Fax/: Using permissions from /home/mail/Fax: mode=0700 gid=-1 doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: file /home/mail/Fax/.Buhhhh/dovecot-acl not found doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/.Buhhhh/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/user/ahelmcke/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/.Buhhhh/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/.Buhhhh/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/user/ahelmcke/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/.Buhhhh/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/.Buhhhh/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/user/ahelmcke/dovecot-acl doveadm -Dv mailbox create -u ahelmcke INBOX/Buhhhh doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib01_acl_plugin.so doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib10_quota_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_lookup (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_zlib_plugin.so: undefined symbol: i_stream_create_deflate (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_list_backend (this is usually intentional, so just ignore this message) doveadm(ahelmcke): Debug: auth input: ahelmcke quota_rule=*:storage=2000M doveadm(ahelmcke): Debug: Added userdb setting: plugin/quota_rule=*:storage=2000M doveadm(ahelmcke): Debug: Effective uid=494, gid=491, home=/home/mail/user/ahelmcke doveadm(ahelmcke): Debug: Quota root: name=User quota backend=maildir args= doveadm(ahelmcke): Debug: Quota rule: root=User quota mailbox=* bytes=2097152000 messages=0 doveadm(ahelmcke): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke doveadm(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=/home/mail/user/ahelmcke, alt= doveadm(ahelmcke): Debug: acl: initializing backend with data: vfile doveadm(ahelmcke): Debug: acl: acl username = ahelmcke doveadm(ahelmcke): Debug: acl: owner = 1 doveadm(ahelmcke): Debug: acl vfile: Global ACL directory: (none) doveadm(ahelmcke): Debug: Namespace dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ doveadm(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke/public, index=, control=, inbox=, alt= doveadm(ahelmcke): Debug: acl: initializing backend with data: vfile doveadm(ahelmcke): Debug: acl: acl username = ahelmcke doveadm(ahelmcke): Debug: acl: owner = 1 doveadm(ahelmcke): Debug: acl vfile: Global ACL directory: (none) doveadm(ahelmcke): Debug: Namespace fax: type=public, prefix=Fax/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no location=maildir:/home/mail/Fax:INDEX=/home/mail/user/ahelmcke/SEEN.Fax doveadm(ahelmcke): Debug: maildir++: root=/home/mail/Fax, index=/home/mail/user/ahelmcke/SEEN.Fax, control=, inbox=, alt= doveadm(ahelmcke): Debug: acl: initializing backend with data: vfile doveadm(ahelmcke): Debug: acl: acl username = ahelmcke doveadm(ahelmcke): Debug: acl: owner = 0 doveadm(ahelmcke): Debug: acl vfile: Global ACL directory: (none) doveadm(ahelmcke): Debug: Namespace INBOX/: /home/mail/user/ahelmcke/.Buhhhh doesn't exist yet, using default permissions doveadm(ahelmcke): Debug: Namespace INBOX/: Using permissions from /home/mail/user/ahelmcke: mode=0700 gid=-1 doveconf -n # 2.1.5: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) ext4 auth_master_user_separator = * auth_verbose = yes first_valid_uid = 400 mail_debug = yes mail_gid = vmail mail_home = /home/mail/user/%u mail_location = maildir:/home/mail/user/%u mail_plugins = " quota acl" mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl namespace dummy { alias_for = INBOX/ hidden = yes list = no location = maildir:/home/mail/user/%u/public/ prefix = separator = / type = private } namespace fax { location = maildir:/home/mail/Fax:INDEX=/home/mail/user/%u/SEEN.Fax prefix = Fax/ separator = / subscriptions = no type = public } namespace inbox { inbox = yes location = prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile acl_shared_dict = file:/home/mail/shared-mailboxes quota = maildir:User quota quota_rule = *:storage=3G sieve = /home/mail/user/%u/sieve.active sieve_dir = /home/mail/user/%u/sieve.scripts } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = * } inet_listener imaps { address = * } } service managesieve-login { inet_listener sieve { address = * port = 4190 } } ssl = required ssl_cert = References: <4F96B436.9090402@freenet.de> Message-ID: <4F96BE96.9080505@schetterer.org> Am 24.04.2012 16:09, schrieb Andreas Helmcke: > Setting: > - maildir with private INBOX and public folders. > - users maildir directory (/home/mail/user/ahelmcke) contains dovecot-acl file > - public folders root directory (/home/mail/Fax) contains dovecot-acl file > > When creating a subfolder of the public folder the dovecot-acl files gets copied to the subfolders directory as expected. > When creating a subfolder of the users INBOX the dovecot-acl files does /not/ get copied to the subfolders directory. i am not sure ,but but i think that works like designed http://wiki2.dovecot.org/ACL .. ACL Inheritance Every time you create a new mailbox, it gets its ACLs from the parent mailbox. If you're creating a root-level mailbox, it uses the namespace's default ACLs. There is no actual inheritance, however: If you modify parent's ACLs, the child's ACLs stay the same. There is currently no support for ACL inheritance. Namespace's default ACLs are read from "dovecot-acl" file in the namespace's mail root directory (e.g. /var/public/Maildir). Note that currently these default ACLs are used only when creating new mailboxes, they aren't used for mailboxes without ACLs. .... -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From ah-news-1204 at freenet.de Tue Apr 24 18:17:39 2012 From: ah-news-1204 at freenet.de (Andreas Helmcke) Date: Tue, 24 Apr 2012 17:17:39 +0200 Subject: [Dovecot] acls not copied when creating subfolder of private INBOX In-Reply-To: <4F96BE96.9080505@schetterer.org> References: <4F96B436.9090402@freenet.de> <4F96BE96.9080505@schetterer.org> Message-ID: <4F96C413.7080709@freenet.de> Am 24.04.2012 16:54, schrieb Robert Schetterer: > Am 24.04.2012 16:09, schrieb Andreas Helmcke: >> When creating a subfolder of the public folder the dovecot-acl files gets copied to the subfolders directory as expected. >> When creating a subfolder of the users INBOX the dovecot-acl files does /not/ get copied to the subfolders directory. > > i am not sure ,but but i think that works like designed > > http://wiki2.dovecot.org/ACL > .. > ACL Inheritance > .... I do read this as: when creating a new mailbox; acls are copied from parent. This it what it does for public folders but /not/ for the private ones. So in my opinion at least it is not working as documented. From robert at schetterer.org Tue Apr 24 18:31:08 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 24 Apr 2012 17:31:08 +0200 Subject: [Dovecot] acls not copied when creating subfolder of private INBOX In-Reply-To: <4F96C413.7080709@freenet.de> References: <4F96B436.9090402@freenet.de> <4F96BE96.9080505@schetterer.org> <4F96C413.7080709@freenet.de> Message-ID: <4F96C73C.5010202@schetterer.org> Am 24.04.2012 17:17, schrieb Andreas Helmcke: > Am 24.04.2012 16:54, schrieb Robert Schetterer: >> Am 24.04.2012 16:09, schrieb Andreas Helmcke: >>> When creating a subfolder of the public folder the dovecot-acl files gets copied to the subfolders directory as expected. >>> When creating a subfolder of the users INBOX the dovecot-acl files does /not/ get copied to the subfolders directory. >> >> i am not sure ,but but i think that works like designed >> >> http://wiki2.dovecot.org/ACL >> .. >> ACL Inheritance >> .... > > I do read this as: when creating a new mailbox; acls are copied from parent. > This it what it does for public folders but /not/ for the private ones. So in my opinion at least it is not working as > documented. i think its missunderstandable written this is what counts i think http://wiki2.dovecot.org/ACL ... "There is currently no support for ACL inheritance" .. public folders are special cases.... it makes sense to have inheritance as default acl but this may not be a good idea as default at users folders read the list archive about this stuff, i cant remember all but Timo wrote some about it, at last ,develop of that feature may be still be not finshed -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From l.rame at griensu.com Tue Apr 24 19:07:12 2012 From: l.rame at griensu.com (Leonardo =?iso-8859-1?Q?M=2E_Ram=E9?=) Date: Tue, 24 Apr 2012 13:07:12 -0300 Subject: [Dovecot] Reading Mutt mboxes from Thunderbird Message-ID: <20120424160712.GB5048@leonardo-laptop> Hi, I'm a user of Mutt, and before moving to Thunderbird I'm trying to share mboxes between them. To do this, I installed Dovecot to create an IMAP server in my local machine, to serve mutt mboxes to be read by Thunderbird. So far, I can read my inbox from /var/mail/%u, but in mutt, I have many folders in ~/mail, like 2010-mails, 2011-mails, mailing-list-x, mailing-list-y, and so on. How can I configure Dovecot to read them?. In /etc/dovecot/conf.d/10-mail.conf I have this: mail_location = mbox:~/Mail:INBOX=/var/mail/%u I'm using Ubuntu 11.10, and Dovecot 2.0.13. Regards, -- Leonardo M. Ram? Medical IT - Griensu S.A. Av. Col?n 636 - Piso 8 Of. A X5000EPT -- C?rdoba Tel.: +54(351)4246924 +54(351)4247788 +54(351)4247979 int. 19 Cel.: +54(351)156629292 From interfasys at gmail.com Tue Apr 24 19:31:38 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Tue, 24 Apr 2012 17:31:38 +0100 Subject: [Dovecot] sievec segfaults Message-ID: <4F96D56A.60907@gmail.com> Hello, sievec segfaults when it reads a file. I've tried with both the standard and gold linker. I've tried with various sieve files and they all make it crash. I've compiled pigeonhole with -ggdb, but I'm not getting much info Program received signal SIGSEGV, Segmentation fault. 0x0000000800b8e587 in strlen () from /lib/libc.so.7 (gdb) bt full #0 0x0000000800b8e587 in strlen () from /lib/libc.so.7 No symbol table info available. #1 0x0000000800a12e8c in str_append () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #2 0x000000080077f02d in sieve_binary_save () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #3 0x0000000000605bf0 in environ () No symbol table info available. #4 0x0000000000000000 in ?? () No symbol table info available. #5 0x00007fffffffeb6e in ?? () No symbol table info available. #6 0x0000000000000001 in ?? () No symbol table info available. #7 0x0000000000000000 in ?? () No symbol table info available. #8 0x000000080060878d in ?? () from /libexec/ld-elf.so.1 No symbol table info available. #9 0x0000000000605cd9 in ?? () No symbol table info available. #10 0x0000000801000ea8 in ?? () No symbol table info available. #11 0x0000000000005008 in ?? () No symbol table info available. #12 0x0000000000000000 in ?? () No symbol table info available. #13 0x0000000801099040 in ?? () No symbol table info available. #14 0x0000000000000000 in ?? () No symbol table info available. #15 0x0000000000000180 in ?? () No symbol table info available. #16 0x0000000000605bf0 in environ () No symbol table info available. #17 0x0000000000000000 in ?? () No symbol table info available. #18 0x00007fffffffeb6e in ?? () No symbol table info available. #19 0x0000000000000001 in ?? () No symbol table info available. #20 0x0000000000000000 in ?? () No symbol table info available. #21 0x0000000000000000 in ?? () No symbol table info available. #22 0x0000000000402f93 in main () No symbol table info available. Cheers, Olivier From dovecot at allycomm.com Tue Apr 24 20:05:07 2012 From: dovecot at allycomm.com (Jeff Kletsky) Date: Tue, 24 Apr 2012 10:05:07 -0700 Subject: [Dovecot] dsync on large mailbox "fails" In-Reply-To: <4EE7C62E-928C-499D-BE4E-9944D6013198@iki.fi> References: <4F95B3DD.1070609@allycomm.com> <4F962028.3060501@allycomm.com> <4EE7C62E-928C-499D-BE4E-9944D6013198@iki.fi> Message-ID: <4F96DD43.3070700@allycomm.com> On 4/24/12 3:26 AM, Timo Sirainen wrote: > On 24.4.2012, at 6.38, Jeff Kletsky wrote: > >> 25934 jeff 1 103 0 135M 102M CPU2 2 128:47 100.00% doveadm > Looks like it could be in an infinite loop. To find out where: > > 1) strace for a few seconds: > > strace -tt -p 25934 2>log > > 2) get gdb backtrace a couple of times (2-5 should be enough, preferably slightly different ones): > > gdb /usr/local/bin/doveadm 25934 > bt full > > cont > > bt full > > cont > > ..etc.. Rather than clutter everyone's inbox, even if we know they have a great IMAP server... http://wildside.wagsky.com/dovecot/2012-04-24/54097/doveadm.gdb.log.bz2 http://wildside.wagsky.com/dovecot/2012-04-24/54097/doveadm.truss.log.bz2 Thanks for looking into this! Jeff From tss at iki.fi Tue Apr 24 20:08:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 24 Apr 2012 20:08:38 +0300 Subject: [Dovecot] dsync on large mailbox "fails" In-Reply-To: <4F96DD43.3070700@allycomm.com> References: <4F95B3DD.1070609@allycomm.com> <4F962028.3060501@allycomm.com> <4EE7C62E-928C-499D-BE4E-9944D6013198@iki.fi> <4F96DD43.3070700@allycomm.com> Message-ID: <78F42D61-1402-408B-953B-F85254479761@iki.fi> On 24.4.2012, at 20.05, Jeff Kletsky wrote: > On 4/24/12 3:26 AM, Timo Sirainen wrote: >> On 24.4.2012, at 6.38, Jeff Kletsky wrote: >> >>> 25934 jeff 1 103 0 135M 102M CPU2 2 128:47 100.00% doveadm >> Looks like it could be in an infinite loop. To find out where: > http://wildside.wagsky.com/dovecot/2012-04-24/54097/doveadm.truss.log.bz2 That's a lot of getdirentries() calls in there, repeating all the time. So I don't think it's an infinite loop, just a lot of slowness repeating the same work over and over again. See if it helps if you add: maildir_very_dirty_syncs = yes From stephan at rename-it.nl Tue Apr 24 20:10:21 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 24 Apr 2012 19:10:21 +0200 Subject: [Dovecot] sievec segfaults In-Reply-To: <4F96D56A.60907@gmail.com> References: <4F96D56A.60907@gmail.com> Message-ID: <4F96DE7D.6030807@rename-it.nl> Op 4/24/2012 6:31 PM, interfaSys s?rl schreef: > Hello, > sievec segfaults when it reads a file. > I've tried with both the standard and gold linker. > I've tried with various sieve files and they all make it crash. What version? Regards, Stephan. From interfasys at gmail.com Tue Apr 24 20:14:13 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Tue, 24 Apr 2012 18:14:13 +0100 Subject: [Dovecot] sievec segfaults In-Reply-To: <4F96DE7D.6030807@rename-it.nl> References: <4F96D56A.60907@gmail.com> <4F96DE7D.6030807@rename-it.nl> Message-ID: <4F96DF65.90909@gmail.com> I pulled the latest code from the dovecot-2.1-pigeonhole repository. Cheers, Olivier On 24/04/2012 18:10, Stephan Bosch wrote: > Op 4/24/2012 6:31 PM, interfaSys s?rl schreef: >> Hello, >> sievec segfaults when it reads a file. >> I've tried with both the standard and gold linker. >> I've tried with various sieve files and they all make it crash. > > What version? > > Regards, > > Stephan. From interfasys at gmail.com Tue Apr 24 20:27:51 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Tue, 24 Apr 2012 18:27:51 +0100 Subject: [Dovecot] sievec segfaults In-Reply-To: <4F96DE7D.6030807@rename-it.nl> References: <4F96D56A.60907@gmail.com> <4F96DE7D.6030807@rename-it.nl> Message-ID: <4F96E297.50300@gmail.com> I've recompiled Dovecot with ggdb and here is a more meaningful trace. Program received signal SIGSEGV, Segmentation fault. 0x000000080067796b in str_append () from /usr/lib/dovecot/libdovecot.so.0 (gdb) bt full #0 0x000000080067796b in str_append () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #1 0x000000080047b672 in sieve_binary_save () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #2 0x0000000800461b04 in sieve_save_as () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #3 0x00000000004032e6 in main (argc=2, argv=0x7fffffffe948) at sievec.c:147 svinst = (struct sieve_instance *) 0x800c90040 st = {st_dev = 1288178348, st_ino = 6479, st_mode = 33261, st_nlink = 1, st_uid = 12, st_gid = 6, st_rdev = 4294967295, st_atimespec = { tv_sec = 1335288301, tv_nsec = 418632262}, st_mtimespec = {tv_sec = 1285859319, tv_nsec = 0}, st_ctimespec = {tv_sec = 1297096638, tv_nsec = 658928714}, st_size = 1197, st_blocks = 4, st_blksize = 4096, st_flags = 0, st_gen = 8013395, st_lspare = 0, st_birthtimespec = { tv_sec = 1297096638, tv_nsec = 658928714}} sbin = (struct sieve_binary *) 0x800c99040 dump = false scriptfile = (const __unknown__ *) 0x7fffffffec56 "/etc/test.sieve" outfile = (const __unknown__ *) 0x0 exit_status = 0 c = -1 Cheers, Olivier On 24/04/2012 18:10, Stephan Bosch wrote: > Op 4/24/2012 6:31 PM, interfaSys s?rl schreef: >> Hello, >> sievec segfaults when it reads a file. >> I've tried with both the standard and gold linker. >> I've tried with various sieve files and they all make it crash. > > What version? > > Regards, > > Stephan. From stephan at rename-it.nl Tue Apr 24 20:34:39 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 24 Apr 2012 19:34:39 +0200 Subject: [Dovecot] sievec segfaults In-Reply-To: <4F96DF65.90909@gmail.com> References: <4F96D56A.60907@gmail.com> <4F96DE7D.6030807@rename-it.nl> <4F96DF65.90909@gmail.com> Message-ID: <4F96E42F.4090206@rename-it.nl> Op 4/24/2012 7:14 PM, interfaSys s?rl schreef: > I pulled the latest code from the dovecot-2.1-pigeonhole repository. This is caused by the recent addition of dict database support for storing Sieve scripts. This should fix it: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/93bfa75d9ab9 Regards, Stephan. From almarzuki2001 at hotmail.com Tue Apr 24 20:41:00 2012 From: almarzuki2001 at hotmail.com (Hadi Salem) Date: Tue, 24 Apr 2012 20:41:00 +0300 Subject: [Dovecot] dovecot LDA error Message-ID: Hi, My dovecot version is , # 2.0.20: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-164.el5 x86_64 CentOS release 5.4 (Final) auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_verbose_passwords = plain debug_log_path = /var/log/dovecot.log disable_plaintext_auth = no hostname = host1.bigmama.com info_log_path = /var/log/dovecot.log log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:~/Maildir passdb { args = scheme=PLAIN username_format=%u /etc/dovecot/users driver = passwd-file } postmaster_address = root at host1.bigmama.com service auth { unix_listener auth-userdb { group = mail mode = 0660 user = mail } } ssl = no userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol lda { info_log_path = /var/log/dovecot-lda.log log_path = /var/log/dovecot-lda-errors.log } Im getting error from dovecot LDA logs log_path = /var/log/dovecot-lda-errors.log Apr 24 20:25:46 lda(jamal at example.com): Fatal: setgid(5000(vmail) from userdb lo okup) failed with euid=8(mail), gid=12(mail), egid=12(mail): Operation not permi tted (This binary should probably be called with process group set to 5000(vmail ) instead of 12(mail)) My usersdb config jamal at example.com:{PLAIN}123:5000:5000::/var/vmail/example.com/jamal/ user 5000 system user as vmail Permissions for the /var/vmail directory as follow drwxr-xr-x 3 vmail vmail 4096 Apr 24 20:18 /var/vmail/ drwx------ 3 vmail vmail 4096 Apr 24 20:18 /var/vmail/example.com/ drwx------ 3 vmail vmail 4096 Apr 24 20:18 /var/vmail/example.com/jamal/ drwx------ 5 vmail vmail 4096 Apr 24 20:20 /var/vmail/example.com/jamal/Maildir/ From interfasys at gmail.com Tue Apr 24 20:42:29 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Tue, 24 Apr 2012 18:42:29 +0100 Subject: [Dovecot] sievec segfaults In-Reply-To: <4F96E42F.4090206@rename-it.nl> References: <4F96D56A.60907@gmail.com> <4F96DE7D.6030807@rename-it.nl> <4F96DF65.90909@gmail.com> <4F96E42F.4090206@rename-it.nl> Message-ID: <4F96E605.6080506@gmail.com> It worked! :) Thank you. Olivier On 24/04/2012 18:34, Stephan Bosch wrote: > Op 4/24/2012 7:14 PM, interfaSys s?rl schreef: >> I pulled the latest code from the dovecot-2.1-pigeonhole repository. > > This is caused by the recent addition of dict database support for > storing Sieve scripts. > > This should fix it: > > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/93bfa75d9ab9 > > Regards, > > Stephan. From tss at iki.fi Tue Apr 24 21:10:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 24 Apr 2012 21:10:17 +0300 Subject: [Dovecot] trash plugin bug? In-Reply-To: <932BA6EA-2884-4847-B66A-E056C8068B28@xs4all.nl> References: <932BA6EA-2884-4847-B66A-E056C8068B28@xs4all.nl> Message-ID: <69567A90-450A-457A-BC38-C0DE235156D9@iki.fi> On 23.4.2012, at 23.40, Cor Bosman wrote: > Hey all, has anyone ever tried turning on the trash plugin when the directory is already over quota? I see some messages being deleted, but it seems it just deletes enough to fit the new email, not enough to go below quota. Well, normally you shouldn't be over quota I guess.. :) Anyway, http://hg.dovecot.org/dovecot-2.1/rev/ec8564741aa8 http://hg.dovecot.org/dovecot-2.1/rev/dd3798681283 From tss at iki.fi Tue Apr 24 21:13:23 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 24 Apr 2012 21:13:23 +0300 Subject: [Dovecot] doveadm delete folders ? / unsubscribe ? In-Reply-To: <4F965B2E.2010103@schetterer.org> References: <4F900066.9030808@schetterer.org> <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> <4F906EE8.1030900@schetterer.org> <4F965B2E.2010103@schetterer.org> Message-ID: <283E4240-57E8-4BF2-8DCA-6DCC78C1D5A8@iki.fi> On 24.4.2012, at 10.50, Robert Schetterer wrote: > doveadm expunge -d -u *@domain.example mailbox exampelfolder\* > savedbefore 10d > > works like charm, but the folders do not get unsubscribed > any chance to have i.e another add parameter ,or do it auto > cause empty and deleted folders should not stay subscribed in my eyes Yeah, I guess unsubscribing is always wanted: http://hg.dovecot.org/dovecot-2.1/rev/b1653b7bd165 From cor at xs4all.nl Tue Apr 24 21:23:54 2012 From: cor at xs4all.nl (Cor Bosman) Date: Tue, 24 Apr 2012 20:23:54 +0200 Subject: [Dovecot] trash plugin bug? In-Reply-To: <69567A90-450A-457A-BC38-C0DE235156D9@iki.fi> References: <932BA6EA-2884-4847-B66A-E056C8068B28@xs4all.nl> <69567A90-450A-457A-BC38-C0DE235156D9@iki.fi> Message-ID: <6B17E4EF-17E9-4BD1-95DE-93CC0E0316AD@xs4all.nl> > >> Hey all, has anyone ever tried turning on the trash plugin when the directory is already over quota? I see some messages being deleted, but it seems it just deletes enough to fit the new email, not enough to go below quota. > > Well, normally you shouldn't be over quota I guess.. :) Anyway, :) It's hard to initialize new quota rules on a large userbase without some users falling outside the expected parameters :) Sucks when you have lots of users, you always always hit edge cases. Thanks! Cor From daniel.parthey at informatik.tu-chemnitz.de Wed Apr 25 05:13:55 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 25 Apr 2012 04:13:55 +0200 Subject: [Dovecot] How to remove duplicate GUID messages from storage? In-Reply-To: <20120421202928.GA12998@daniel.localdomain> References: <20120421202928.GA12998@daniel.localdomain> Message-ID: <20120425021355.GA10493@daniel.localdomain> Daniel Parthey schrieb: > I tried to import messages like this: > doveadm import -u username at example.org mdbox:/mail/dovecot/example.org/username/mail mbox:/root/mail.txt all > > This seemed to have imported the same messages in the mail storage, > as they were already there. And now I have got a lot of duplicates. > > Then I deleted the mbox:/root/mail mail stuff from "mailboxes" folder > and the storage remained, with duplicate GUIDs. > > The following command > doveadm -v force-resync -u username at example.org INBOX > outputs lots of messages like this one, with different GUID > doveadm(username at example.org): Error: mdbox /mail/dovecot/example.org/username/mail/storage: Duplicate GUID 0b8a032d66a0924fb42c0000de5f8128 in m.55:45484041 and m.14:52173045 > > The messages at m.55:45484041 and m.14:52173045 have the same content, > since they were seemingly imported from the mail store itself, > they are in the storage twice now and require twice as much disk > space as before. > > How can I manually remove these identical, duplicate messages > from the storage to save space? Dovecot does not do it > automatically. Should I edit the mdbox storage files directly using vim in order to remove the duplicate messages which I imported by accident, or is there any dovecot mdbox "repair toolkit" with removes duplicate messages? I attached the dovecot version and config. Regards, Daniel # doveconf -n # 2.0.20: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-40-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_verbose = yes auth_verbose_passwords = sha1 deliver_log_format = mailbox: deliver: msgid=%m from=%f: %$ dict { quota = mysql:/etc/dovecot/conf.d/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = Mailbox login_log_format = mailbox: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "mailbox: mail: %s(%u): " mail_plugins = quota mail_privileged_group = vmail mail_uid = vmail managesieve_implementation_string = Sieve managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 50 M mmap_disable = yes passdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve service dict { unix_listener dict { group = vmail mode = 0660 } } service imap-login { inet_listener imap { port = 19143 } } service lmtp { inet_listener lmtp { address = * port = 19024 } } service managesieve-login { inet_listener sieve { port = 19200 } } service pop3-login { inet_listener pop3 { port = 19110 } } service quota-warning { executable = script /usr/local/bin/quota-warning unix_listener quota-warning { user = vmail } user = dovecot } ssl = no userdb { driver = prefetch } userdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_plugins = quota imap_quota } protocol lmtp { mail_plugins = quota sieve } From robert at schetterer.org Wed Apr 25 09:13:02 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 08:13:02 +0200 Subject: [Dovecot] doveadm delete folders ? / unsubscribe ? In-Reply-To: <283E4240-57E8-4BF2-8DCA-6DCC78C1D5A8@iki.fi> References: <4F900066.9030808@schetterer.org> <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> <4F906EE8.1030900@schetterer.org> <4F965B2E.2010103@schetterer.org> <283E4240-57E8-4BF2-8DCA-6DCC78C1D5A8@iki.fi> Message-ID: <4F9795EE.4030604@schetterer.org> Am 24.04.2012 20:13, schrieb Timo Sirainen: > On 24.4.2012, at 10.50, Robert Schetterer wrote: > >> doveadm expunge -d -u *@domain.example mailbox exampelfolder\* >> savedbefore 10d >> >> works like charm, but the folders do not get unsubscribed >> any chance to have i.e another add parameter ,or do it auto >> cause empty and deleted folders should not stay subscribed in my eyes > > Yeah, I guess unsubscribing is always wanted: > > http://hg.dovecot.org/dovecot-2.1/rev/b1653b7bd165 > Hi Timo, i will try after update and report, jep unsubscribing should be the uni wanted behave thx again ! -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From weston.kris at ymail.com Wed Apr 25 09:17:18 2012 From: weston.kris at ymail.com (Kris) Date: Wed, 25 Apr 2012 07:17:18 +0100 Subject: [Dovecot] dovecot %u variable problem.... Message-ID: <4F9796EE.5070907@ymail.com> Hi guys n gals, im on an ubuntu 11 VPS on linode (dovecot version 2.013). ive spent a lot of time and googling trying to get postfix, dovecot, ssl, saslauthd and mysql to all play nice together and im quite close now but theres something going on i cant quite explain and id really appreciate some help. I have been going for ten hours on this today and it seems in direct conflict with what dovecot themselves say... the problem is a variable in dovecot (%u) which is normally used as a full username in sql query strings (i.e. user at domain) and is simply dropping the domain for some reason... From dovecot docs (http://wiki2.dovecot.org/Variables): The variables that work everywhere are: %u user full username (e.g. user at domain) so i have postfix and stuff set up ok to send at least and i know this because ive tested it by hard-coding the string in the sql query myself with a successful send... but, anyway, the problem with the '%u' variable is it is doing stuff like this: (from mail.log and mysql.log) localhost dovecot: auth: Debug: sql(admin at microhard.com,xx.xx.xx.xx): SELECT home,uid,gid FROM users WHERE id = 'admin at microhard.com' this was derived from the following query in /etc/dovecot/dovecot-sql.conf.ext user_query = SELECT home,uid,gid FROM users WHERE id = '%u' so no problem there.... but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user at domain): localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' Query SELECT id as user, crypt as password FROM users WHERE id= 'admin' which is derived from the following query also in /etc/dovecot/dovecot-sql.conf.ext password_query = SELECT id as user, crypt as password FROM users WHERE id= '%u' and to further confuse me, this 'unchangeable' variable sometimes decides to right itself without any obvious reason... localhost dovecot: auth: Debug: sql(admin at microhard.com,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin at microhard.com' im wondering if its something to do with my client dropping the domain, or whether dovecot has an error, or some other thing ive overlooked... my dovecot.conf looks a lot like this: # 2.0.13: dovecot.conf # OS: Linux 3.0.18-x86_64-linode24 x86_64 Ubuntu 11.10 ext3 first_valid_uid = 5000 last_valid_uid = 5000 login_greeting = I likes my ducks n geese I do. mail_location = maildir:/var/spool/mail/virtual/%u #enables logging all failed authentication attempts. auth_verbose=yes #enables all authentication debug logging (also enables auth_verbose). Passwords are logged as . auth_debug=yes #does everything that auth_debug=yes does, but it also removes password hiding. auth_debug_passwords=yes #enables all kinds of mail related debug logging, such as showing where Dovecot is looking for mails. mail_debug=yes #enables logging SSL errors and warnings. Even without this setting if connection is closed because of an SSL error, the error is logged as the disconnection reason (v1.1+). verbose_ssl=yes passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/etc/dovecot/acls #quota = maildir:storage=10240:messages=1000 trash = /etc/dovecot/trash.conf } protocols = " imap" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = dovecot mode = 0660 user = dovecot } user = dovecot } ssl_ca = /etc/pki/dovecot/certs/ca-bundle.crt ssl_cert = References: <4F9796EE.5070907@ymail.com> Message-ID: <4F979E5E.5000409@schetterer.org> Am 25.04.2012 08:17, schrieb Kris: > this was derived from the following query in > /etc/dovecot/dovecot-sql.conf.ext > user_query = SELECT home,uid,gid FROM users WHERE id = '%u' i use like this user_query = SELECT concat('/usr/local/virtual/', maildir) AS home, \ concat('*:bytes=', mailbox.quota) AS quota_rule, \ #when saving to Trash mailbox the user gets additional 50MB "Trash:storage=+50240" AS quota_rule2, \ #when saving to Sent mailbox the user gets additional 50MB "Sent:storage=+50240" AS quota_rule3, \ #when saving to Drafts mailbox the user gets additional 50MB "Drafts:storage=+50240" AS quota_rule4, \ #when saving to Templates mailbox the user gets additional 50MB "Templates:storage=+50240" AS quota_rule5, \ #when saving to Junk mailbox the user gets additional 50MB "Junk:storage=+50240" AS quota_rule6, \ #when saving to Archives mailbox the user gets additional 50MB "Archives:storage=+50240" AS quota_rule7, \ ########### concat('maildir:/usr/local/virtual/', maildir) AS mail, \ CASE '%s' WHEN 'pop3' THEN NULL ELSE 'yes' END AS 'namespace/inbox/inbox', \ CASE '%s' WHEN 'pop3' THEN 'yes' ELSE NULL END AS 'namespace/virtual/inbox', \ 1001 AS uid, 1001 AS gid FROM mailbox WHERE username = '%u' AND active = '1' > > so no problem there.... > > but then this happens when i attempt to send mail and this means the > user gets rejected because dovecot cant find the user id in the db > without the correct string (i.e. user at domain): > localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id > as user, crypt as password FROM users WHERE id= 'admin' > Query SELECT id as user, crypt as password FROM users WHERE id= 'admin' > which is derived from the following query also in > /etc/dovecot/dovecot-sql.conf.ext > password_query = SELECT id as user, crypt as password FROM users WHERE > id= '%u' > > and to further confuse me, this 'unchangeable' variable sometimes > decides to right itself without any obvious reason... > localhost dovecot: auth: Debug: sql(admin at microhard.com,xx.xx.xx.xx): > query: SELECT id as user, crypt as password FROM users WHERE id= > 'admin at microhard.com' > > im wondering if its something to do with my client dropping the domain, > or whether dovecot has an error, or some other thing ive overlooked... password_query = SELECT username as user, password, \ 1001 as userdb_uid, \ 1001 as userdb_gid, \ "/usr/local/virtual/%d/%u/" AS userdb_home, \ "maildir:/usr/local/virtual/%d/%u/" AS userdb_mail \ FROM mailbox WHERE username = '%u' AND active = '1' AND (imap_allowed = '1' or '%Ls' = 'pop3') this should not fit to you ( postfixadmin full domain layout with virtual plugin and quota additions ,clear text passwords etc), but perhaps gives you ideas by compare, what to fix at your side, however upgrade to dove latest 2.0.20 -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From dg at dguhl.org Wed Apr 25 11:26:15 2012 From: dg at dguhl.org (Dennis Guhl) Date: Wed, 25 Apr 2012 10:26:15 +0200 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F9796EE.5070907@ymail.com> References: <4F9796EE.5070907@ymail.com> Message-ID: <20120425082615.GA14759@laptop-dg.leere.eu> On Wed, Apr 25, 2012 at 07:17:18AM +0100, Kris wrote: [..] > the problem is a variable in dovecot (%u) which is normally used as > a full username in sql query strings (i.e. user at domain) and is > simply dropping the domain for some reason... [..] > and to further confuse me, this 'unchangeable' variable sometimes > decides to right itself without any obvious reason... > localhost dovecot: auth: Debug: > sql(admin at microhard.com,xx.xx.xx.xx): query: SELECT id as user, > crypt as password FROM users WHERE id= 'admin at microhard.com' > > im wondering if its something to do with my client dropping the > domain, or whether dovecot has an error, or some other thing ive > overlooked... Indeed, it seems some of your users try to authenticate with a username without the '@domain' part. You can either try to make your users comply or you can use auth_default_realm in your config: # Default realm/domain to use if none was specified. This is # used for both SASL realms and appending @domain to username # in plaintext logins. # #auth_default_realm = Dennis [..] From cor at xs4all.nl Wed Apr 25 11:27:18 2012 From: cor at xs4all.nl (Cor Bosman) Date: Wed, 25 Apr 2012 10:27:18 +0200 Subject: [Dovecot] trash plugin bug? In-Reply-To: <69567A90-450A-457A-BC38-C0DE235156D9@iki.fi> References: <932BA6EA-2884-4847-B66A-E056C8068B28@xs4all.nl> <69567A90-450A-457A-BC38-C0DE235156D9@iki.fi> Message-ID: <0D69989C-00E4-48CC-AF08-B58C52EA3A12@xs4all.nl> > > Well, normally you shouldn't be over quota I guess.. :) Anyway, > > http://hg.dovecot.org/dovecot-2.1/rev/ec8564741aa8 > http://hg.dovecot.org/dovecot-2.1/rev/dd3798681283 This indeed fixed the problem. Thank you, Cor From weston.kris at ymail.com Wed Apr 25 11:50:20 2012 From: weston.kris at ymail.com (Kris Weston) Date: Wed, 25 Apr 2012 09:50:20 +0100 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <20120425082615.GA14759@laptop-dg.leere.eu> References: <4F9796EE.5070907@ymail.com> <20120425082615.GA14759@laptop-dg.leere.eu> Message-ID: <4F97BACC.4080902@ymail.com> thanks for your help dennis... > Indeed, it seems some of your users try to authenticate with a > username without the '@domain' part. > > You can either try to make your users comply or you can use > auth_default_realm in your config: in this case the users are me :) same username every time i connect, but each time i see a load of db lookups that are different but using this same u variable... > # Default realm/domain to use if none was specified. This is > # used for both SASL realms and appending @domain to username > # in plaintext logins. > # > #auth_default_realm = > ahhh, ok. only i have multiple domains, so how does that werk ? just put em all in ? cheerz Kris From dg at dguhl.org Wed Apr 25 11:58:05 2012 From: dg at dguhl.org (Dennis Guhl) Date: Wed, 25 Apr 2012 10:58:05 +0200 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F97BACC.4080902@ymail.com> References: <4F9796EE.5070907@ymail.com> <20120425082615.GA14759@laptop-dg.leere.eu> <4F97BACC.4080902@ymail.com> Message-ID: <20120425085805.GB14759@laptop-dg.leere.eu> On Wed, Apr 25, 2012 at 09:50:20AM +0100, Kris Weston wrote: > > thanks for your help dennis... > >Indeed, it seems some of your users try to authenticate with a > >username without the '@domain' part. > > > >You can either try to make your users comply or you can use > >auth_default_realm in your config: > > in this case the users are me :) > same username every time i connect, but each time i see a load of db > lookups that are different but using this same u variable... If it's no bug, I doubt it randomly changes the value for %u while the input is the same. So it seems the problem lies somewhere else. And if it is no layer 8 problem it must be the client ;) > > # Default realm/domain to use if none was specified. This is > > # used for both SASL realms and appending @domain to username > > # in plaintext logins. > > # > > #auth_default_realm = > > > > ahhh, ok. only i have multiple domains, so how does that werk ? > just put em all in ? IIRC no, you can only name one default. Dennis From tss at iki.fi Wed Apr 25 13:45:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 13:45:08 +0300 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F979E5E.5000409@schetterer.org> References: <4F9796EE.5070907@ymail.com> <4F979E5E.5000409@schetterer.org> Message-ID: <629E1A55-910F-48B9-8986-C87A34852F24@iki.fi> On 25.4.2012, at 9.49, Robert Schetterer wrote: > #when saving to Trash mailbox the user gets additional 50MB > "Trash:storage=+50240" AS quota_rule2, \ > #when saving to Sent mailbox the user gets additional 50MB > "Sent:storage=+50240" AS quota_rule3, \ > #when saving to Drafts mailbox the user gets additional 50MB > "Drafts:storage=+50240" AS quota_rule4, \ > #when saving to Templates mailbox the user gets additional 50MB > "Templates:storage=+50240" AS quota_rule5, \ > #when saving to Junk mailbox the user gets additional 50MB > "Junk:storage=+50240" AS quota_rule6, \ > #when saving to Archives mailbox the user gets additional 50MB > "Archives:storage=+50240" AS quota_rule7, \ > ########### BTW. These are all static values and don't really need to be in SQL query: plugin { quota_rule2 = Trash:... quota_rule3 = ... ... } > 1001 as userdb_uid, \ > 1001 as userdb_gid, \ > "/usr/local/virtual/%d/%u/" AS userdb_home, \ > "maildir:/usr/local/virtual/%d/%u/" AS userdb_mail \ Same for these. Also slightly confusing that your passdb and userdb lookups return mail setting differently. From tss at iki.fi Wed Apr 25 13:48:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 13:48:45 +0300 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F9796EE.5070907@ymail.com> References: <4F9796EE.5070907@ymail.com> Message-ID: <5B92F036-DDBE-42AB-94A1-DBF21F11DE14@iki.fi> On 25.4.2012, at 9.17, Kris wrote: > but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user at domain): > localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' What exactly is doing this lookup? SMTP AUTH? Maybe the client's SMTP authentication is configured without the @domain part in username? From tss at iki.fi Wed Apr 25 13:52:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 13:52:26 +0300 Subject: [Dovecot] How to remove duplicate GUID messages from storage? In-Reply-To: <20120421202928.GA12998@daniel.localdomain> References: <20120421202928.GA12998@daniel.localdomain> Message-ID: On 21.4.2012, at 23.29, Daniel Parthey wrote: > The following command > doveadm -v force-resync -u username at example.org INBOX > outputs lots of messages like this one, with different GUID > doveadm(username at example.org): Error: mdbox /mail/dovecot/example.org/username/mail/storage: Duplicate GUID 0b8a032d66a0924fb42c0000de5f8128 in m.55:45484041 and m.14:52173045 I think this is a force-resync bug and it shouldn't really complain about duplicates. Although I'm not entirely sure why with you it's complaining about them at all. I have anyway in TODO to look into this.. > How can I manually remove these identical, duplicate messages > from the storage to save space? Dovecot does not do it > automatically. Perhaps force-resync + purge should do that, but currently it doesn't. Also it would be nice if doveadm import didn't add duplicates in the first place. This is also something for which I have vague plans, because it would help dsync as well. From robert at schetterer.org Wed Apr 25 14:07:56 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 13:07:56 +0200 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <629E1A55-910F-48B9-8986-C87A34852F24@iki.fi> References: <4F9796EE.5070907@ymail.com> <4F979E5E.5000409@schetterer.org> <629E1A55-910F-48B9-8986-C87A34852F24@iki.fi> Message-ID: <4F97DB0C.4040901@schetterer.org> Am 25.04.2012 12:45, schrieb Timo Sirainen: > On 25.4.2012, at 9.49, Robert Schetterer wrote: > >> #when saving to Trash mailbox the user gets additional 50MB >> "Trash:storage=+50240" AS quota_rule2, \ >> #when saving to Sent mailbox the user gets additional 50MB >> "Sent:storage=+50240" AS quota_rule3, \ >> #when saving to Drafts mailbox the user gets additional 50MB >> "Drafts:storage=+50240" AS quota_rule4, \ >> #when saving to Templates mailbox the user gets additional 50MB >> "Templates:storage=+50240" AS quota_rule5, \ >> #when saving to Junk mailbox the user gets additional 50MB >> "Junk:storage=+50240" AS quota_rule6, \ >> #when saving to Archives mailbox the user gets additional 50MB >> "Archives:storage=+50240" AS quota_rule7, \ >> ########### > > BTW. These are all static values and don't really need to be in SQL query: > > plugin { > quota_rule2 = Trash:... > quota_rule3 = ... > ... > } > >> 1001 as userdb_uid, \ >> 1001 as userdb_gid, \ >> "/usr/local/virtual/%d/%u/" AS userdb_home, \ >> "maildir:/usr/local/virtual/%d/%u/" AS userdb_mail \ > > Same for these. Also slightly confusing that your passdb and userdb lookups return mail setting differently. thx timo, i will look at this, thats all historic but ever worked currently i am still fighting with some upgrade issues to 2.1 just now some sieve experimental stuff failed , i will post about it -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Apr 25 16:06:29 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 15:06:29 +0200 Subject: [Dovecot] sieve vacation redirect problem Message-ID: <4F97F6D5.2080104@schetterer.org> Hi , since upgrade sieve vacation, redirect stopped working getting postfix error while sending end of data -- message may be sent more than once i played a lot about sieve , lda submission host conf entries etc but all failed , i am not seeing any heavy bug in the logs -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Apr 25 16:29:47 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 15:29:47 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F97F6D5.2080104@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> Message-ID: <4F97FC4B.6060803@schetterer.org> Am 25.04.2012 15:06, schrieb Robert Schetterer: > Hi , since upgrade > sieve vacation, redirect stopped working getting > > postfix error > while sending end of data -- message may be sent more than once > > i played a lot about sieve , lda submission host conf entries etc but > all failed , i am not seeing any heavy bug in the logs > that not full truth i am seeing master: service(lmtp): child 31952 killed with signal 11 (core not dumped) -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Wed Apr 25 16:46:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 16:46:31 +0300 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F97FC4B.6060803@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> Message-ID: <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> On 25.4.2012, at 16.29, Robert Schetterer wrote: > master: service(lmtp): child 31952 killed with signal 11 (core not dumped) Can you get a gdb backtrace? Might be a bit difficult to get a core dump from lmtp, but you should be able to attach gdb into it: service lmtp { process_min_avail = 1 } gdb -p `pidof lmtp` cont bt full From stephan at rename-it.nl Wed Apr 25 16:51:29 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 25 Apr 2012 15:51:29 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F97FC4B.6060803@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> Message-ID: <4F980161.9080007@rename-it.nl> Op 4/25/2012 3:29 PM, Robert Schetterer schreef: > Am 25.04.2012 15:06, schrieb Robert Schetterer: >> Hi , since upgrade >> sieve vacation, redirect stopped working getting >> >> postfix error >> while sending end of data -- message may be sent more than once >> >> i played a lot about sieve , lda submission host conf entries etc but >> all failed , i am not seeing any heavy bug in the logs >> > that not full truth i am seeing > > master: service(lmtp): child 31952 killed with signal 11 (core not dumped) Please try to obtain a gdb backtrace: http://dovecot.org/bugreport.html And provide dovecot -n output Regards, Stephan. From robert at schetterer.org Wed Apr 25 17:03:49 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 16:03:49 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> Message-ID: <4F980445.4090203@schetterer.org> Am 25.04.2012 15:46, schrieb Timo Sirainen: > bt full Program received signal SIGSEGV, Segmentation fault. smtp_client_open (set=0x252b0b0, destination=0x254e410 "ncitester2 at onmotive.de", return_path=0x24e4148 "root at mail02.mailspooler.com", file_r=0x7fff2040a748) at smtp-client.c:146 146 smtp-client.c: Datei oder Verzeichnis nicht gefunden. in smtp-client.c (gdb) Undefined command: " path = 0x180000

fd = #1 0x00007fd4e58b57a5 in act_redirect_send (action=, aenv=0x254e2c0, tr_context=, keep=) at cmd-redirect.c:339 size = msgctx = crlf_input = smtp_handle = 0x24e4148 data = ret = sender = 0x24e4148 "root at mail02.mailspooler.com" recipient = 0x24e4168 "robert at schetterer.name" input = f = hide_headers = {0x7fd4e58d2ee7 "Return-Path", 0x7fd4e58d6d3c "X-Sieve", 0x7fd4e58d7929 "X-Sieve-Redirected-From"} #2 act_redirect_commit (action=, aenv=0x254e2c0, tr_context=, keep=) at cmd-redirect.c:402 ctx = 0x254e408 mail = 0x252ba90 senv = 0x7fff2040a8c0 dupeid = #3 0x00007fd4e58a7eeb in sieve_result_execute (result=, keep=) at sieve-result.c:1156 impl_keep = true rsef = 0x0 implicit_keep = true success = true commit_ok = rac = 0x254e428 first_action = #4 0x00007fd4e58b8418 in sieve_multiscript_execute (mscript=0x254e3d8, sbin=0x25516c0, ehandler=0x254c100, flags=, final=true) at sieve.c:601 No locals. #5 sieve_multiscript_run (mscript=0x254e3d8, sbin=0x25516c0, ehandler=0x254c100, flags=, final=true) at sieve.c:630 No locals. #6 0x00007fd4e5af47e6 in lda_sieve_multiscript_execute (mdctx=, storage_r=) at lda-sieve-plugin.c:514 sbin = 0x25516c0 rtflags = SIEVE_RUNTIME_FLAG_NOGLOBAL cpflags = SIEVE_COMPILE_FLAG_NOGLOBAL mscript = 0x254e3d8 svinst = 0x2546e80 last_script = 0x254b8e0 ret = scripts = ehandler = 0x254c100 i = 1 more = true error = SIEVE_ERROR_NONE #7 lda_sieve_deliver_mail (mdctx=, storage_r=) at lda-sieve-plugin.c:794 scripts = i = _data_stack_cur_id = 3 srctx = {svinst = 0x2546e80, mdctx = 0x7fff2040aac0, scripts = 0x24e3340, script_count = 2, user_script = 0x254b8e0, main_script = 0x254b8e0, msgdata = 0x7fff2040a990, scriptenv = 0x7fff2040a8c0, user_ehandler = 0x254c100, master_ehandler = 0x254b7d0, userlog = 0x24e3658 "/usr/local/virtual/schetterer.name/robert at schetterer.name/dovecot.sieve.log"} svenv = {hostname = 0x250aba0 "mail02.mailspooler.com", base_dir = 0x2536a78 "/var/run/dovecot", username = 0x25354c8 "robert at schetterer.name", home_dir = 0x2537c40 "/usr/local/virtual/schetterer.name/robert at schetterer.name/", flags = SIEVE_FLAG_HOME_RELATIVE} svinst = 0x2546e80 msgdata = {mail = 0x252ba90, return_path = 0x24f0c68 "root at mail02.mailspooler.com", orig_envelope_to = 0x24f12c0 "robert at schetterer.name", final_envelope_to = 0x24f12c0 "robert at schetterer.name", auth_user = 0x25354c8 "robert at schetterer.name", id = 0x252c410 "<20120425140007.D3556840227 at mail02.mailspooler.com>"} scriptenv = {action_log_format = 0x250ac28 "msgid=%m: %$", user = 0x2535410, default_mailbox = 0x4077ac "INBOX", postmaster_address = 0x250ab78 "hostmaster at mail02.mailspooler.com", mailbox_autocreate = false, mailbox_autosubscribe = false, script_context = 0x7fff2040aac0, smtp_open = 0x7fd4e5af4180 , smtp_close = 0x7fd4e5af4160 , duplicate_check = 0x7fd4e5af4110 , duplicate_mark = 0x7fd4e5af4140 , reject_mail = 0x7fd4e5af4100 , exec_status = 0x7fff2040aa00, trace_stream = 0x0, trace_config = {level = SIEVE_TRLVL_NONE, flags = 0}} estatus = {message_saved = false, message_forwarded = false, tried_default_save = false, keep_original = false, last_storage = 0x0} master_ehandler = 0x254b7d0 user_location = default_location = sieve_before = sieve_after = script_sequence = {arr = {buffer = 0x24e3308, element_size = 8}, v = 0x24e3308, v_modifiable = 0x24e3308} debug = error = SIEVE_ERROR_NONE ret = #8 0x00007fd4e888a445 in mail_deliver (ctx=0x7fff2040aac0, storage_r=0x7fff2040ab50) at mail-deliver.c:389 ret = #9 0x0000000000405a33 in client_deliver (client=0x24f00d0) at commands.c:538 ns = storage = 0x0 sets = mail_error = MAIL_ERROR_NONE dctx = {pool = 0x252aa90, set = 0x250ab28, session = 0x252aab0, dup_ctx = 0x2546d90, session_id = 0x24f0c50 "ma3NNmcDmE/bDAAAAfUEdQ", src_mail = 0x252ba90, src_envelope_sender = 0x24f0c68 "root at mail02.mailspooler.com", dest_user = 0x2535410, dest_addr = 0x24f12c0 "robert at schetterer.name", final_dest_addr = 0x24f12c0 "robert at schetterer.name", dest_mailbox_name = 0x4077ac "INBOX", dest_mail = 0x0, var_expand_table = 0x0, tried_default_save = false, saved_mail = false, save_dest_mail = false, mailbox_full = false, dsn = false} input = error = #10 client_deliver_next (client=0x24f00d0) at commands.c:577 rcpts = 0x24f0cc0 #11 client_input_data_write_local (client=0x24f00d0) at commands.c:671 src_mail = 0x252ba90 first_uid = 4294967295 session = 0x252aab0 old_uid = 0 #12 client_input_data_write (client=0x24f00d0) at commands.c:779 input = 0x2513530 #13 client_input_data_handle (client=0x24f00d0) at commands.c:873 data = size = 861 ---Type to continue, or q to quit--- -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Apr 25 17:10:25 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 16:10:25 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F980161.9080007@rename-it.nl> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4F980161.9080007@rename-it.nl> Message-ID: <4F9805D1.8050101@schetterer.org> Am 25.04.2012 15:51, schrieb Stephan Bosch: > Op 4/25/2012 3:29 PM, Robert Schetterer schreef: >> Am 25.04.2012 15:06, schrieb Robert Schetterer: >>> Hi , since upgrade >>> sieve vacation, redirect stopped working getting >>> >>> postfix error >>> while sending end of data -- message may be sent more than once >>> >>> i played a lot about sieve , lda submission host conf entries etc but >>> all failed , i am not seeing any heavy bug in the logs >>> >> that not full truth i am seeing >> >> master: service(lmtp): child 31952 killed with signal 11 (core not >> dumped) > > Please try to obtain a gdb backtrace: > > http://dovecot.org/bugreport.html > > And provide dovecot -n output > > Regards, > > Stephan. did test a few sieve related stuff cause the working 2.0.20 didnt worked but now my ideas are running out # 2.1.5 (d2b01dcce79f): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-37-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 50 M auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes auth_verbose_passwords = plain auth_worker_max_count = 300 default_process_limit = 250 dict { acl = mysql:/etc/dovecot/dovecot-dict-acl-sql.conf.ext expire = mysql:/etc/dovecot/dovecot-dict-expire-sql.conf.ext quotadict = mysql:/etc/dovecot/dovecot-dict-quota-sql.conf.ext } disable_plaintext_auth = no first_valid_gid = 1001 first_valid_uid = 1001 hostname = mail02.mailspooler.com last_valid_gid = 1001 last_valid_uid = 1001 listen = * login_greeting = imap, pop ready mail_access_groups = vmail mail_debug = yes mail_fsync = always mail_gid = 1001 mail_location = maildir:~/ mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = " zlib expire virtual fts fts_squat mail_log autocreate notify acl listescape" mail_privileged_group = vmail mail_uid = 1001 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags notify mmap_disable = yes namespace { inbox = no list = children location = maildir:/usr/local/virtual/%%d/%%u:INDEX=~/shared/%%d/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { hidden = no list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } mailbox name { special_use = \Drafts \Junk \Sent \Trash } prefix = separator = / subscriptions = yes } namespace real { hidden = yes list = no location = prefix = RealMails/ separator = / } namespace virtual { hidden = yes list = no location = virtual:/etc/dovecot/virtual:LAYOUT=maildir++:INDEX=~/virtual prefix = virtual/ separator = / subscriptions = no } passdb { args = /etc/dovecot/dovecot-sql-master.conf.ext driver = sql master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl autocreate = Trash autocreate2 = Junk autocreate3 = Sent autocreate4 = Drafts autocreate5 = Templates autocreate6 = Hostmaster-Backup autocreate7 = archiv-backup-mailspooler autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Drafts autosubscribe5 = Templates expire = Trash expire2 = Trash/* expire3 = Junk expire4 = Junk/* expire5 = Hostmaster-Backup expire6 = Hostmaster-Backup/* expire7 = archiv-backup-mailspooler/* expire_dict = proxy::expire fts = squat home = /usr/local/virtual/%d/%u mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size mail_log_group_events = yes quota = dict:::proxy::quotadict quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /usr/local/virtual/%d/%u/dovecot.sieve sieve_before = /etc/dovecot/sieve/global.sieve sieve_dir = /usr/local/virtual/%d/%u/sieve sieve_extensions = +fileinto +reject +envelope +encoded-character +vacation +subaddress +relational +regex +imap4flags +copy +include +variables +body +enotify +environment +mailbox +date +ihave +imapflags +notify sieve_global_dir = /etc/dovecot/sieve/ sieve_global_path = /etc/dovecot/sieve/global.sieve zlib_save = gz zlib_save_level = 6 } postmaster_address = hostmaster at mail02.mailspooler.com protocols = imap pop3 lmtp sieve service anvil { client_limit = 1000 } service auth-worker { user = root } service auth { client_limit = 1000 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service dict { extra_groups = vmail unix_listener dict { group = vmail mode = 0660 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 8 service_count = 1 vsz_limit = 256 M } service imap-postlogin { executable = script-login /usr/local/bin/postlogin_imap.sh } service imap { executable = imap imap-postlogin process_limit = 1024 vsz_limit = 256 M } service lmtp { process_min_avail = 1 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } vsz_limit = 256 M } service managesieve-login { inet_listener { address = 127.0.0.1 212.52.224.210 212.52.224.206 port = 2000 } vsz_limit = 256 M } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } process_min_avail = 8 } service pop3-postlogin { executable = script-login /usr/local/bin/postlogin_pop3.sh } service pop3 { executable = pop3 pop3-postlogin process_limit = 1024 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { mode = 0666 user = vmail } user = vmail } ssl_ca = References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> Message-ID: <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> On 25.4.2012, at 17.03, Robert Schetterer wrote: > Program received signal SIGSEGV, Segmentation fault. > smtp_client_open (set=0x252b0b0, destination=0x254e410 > "ncitester2 at onmotive.de", return_path=0x24e4148 > "root at mail02.mailspooler.com", file_r=0x7fff2040a748) at smtp-client.c:146 > 146 smtp-client.c: Datei oder Verzeichnis nicht gefunden. > in smtp-client.c If you do the same and get here again, what does it say with: p *set From robert at schetterer.org Wed Apr 25 17:16:28 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 16:16:28 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> Message-ID: <4F98073C.3080406@schetterer.org> Am 25.04.2012 16:11, schrieb Timo Sirainen: > p *set hope doing it right (gdb) p *set $1 = {postmaster_address = 0xaa33d0 "P6\252", hostname = 0x0, submission_host = 0x0, sendmail_path = 0x7fbaf6e41f50 "H\211\\$\360H\211l$\370H\203\354\030\200=\207A%", rejection_subject = 0x7fbaf6e42b90 "H\211\\$\350L\211d$\370H\211\373H\211l$\360H\203\354(\200=?5%", rejection_reason = 0x7fbaf6e41ed0 "H\211\\$\360H\211l$\370H\203\354(\200=\aB%", deliver_log_format = 0x1
, recipient_delimiter = 0xa9c910 "\003", lda_original_recipient_header = 0xabd490 "\001", quota_full_tempfail = 16, lda_mailbox_autocreate = 208, lda_mailbox_autosubscribe = 171} -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From warden at geneseo.edu Wed Apr 25 17:20:41 2012 From: warden at geneseo.edu (David Warden) Date: Wed, 25 Apr 2012 10:20:41 -0400 Subject: [Dovecot] stats + fts squat plugins, indexer-worker error on message indexing Message-ID: On RHEL 6.2 + dovecot 2.1.5 with the stats and fts (squat) plugins enabled and each user having their own uid on the system I get the following error in the logs when the indexer-worker process indexes some messages: Apr 25 09:56:19 wardentest3 dovecot: imap-login: Login: user=warden, method=PLAIN, rip=137.238.60.164, lip=137.238.2.240, lport=1993, mpid=19464, encryption=TLS Apr 25 09:56:26 wardentest3 dovecot: service=indexer-worker user=warden rip= Indexed 12 messages in INBOX Apr 25 09:56:26 wardentest3 dovecot: service=indexer-worker user=warden rip= Error: open(/proc/self/io) failed: Permission denied Apr 25 09:56:29 wardentest3 dovecot: service=imap user=warden rip=137.238.60.164 Disconnected: Logged out bytes(in/out)=46/780 What I did to generate the above was remove the squat indexes for my Inbox then log in via telnet and search for a string in my Inbox, which forced an index of the messages in my Inbox. Based on http://wiki2.dovecot.org/Services#indexer-worker it seems like this is because the indexer-worker process is created as root but at the time of indexing the indexer-worker drops privs to the mail user. In /proc//, io is mode 0400 and owned by root, so it makes sense that the mail user would be unable to read that file. The strange thing is it looks like the stats are still collected (unless I am reading the output of the stats dump incorrectly): [root at wardentest3 mailtest]# doveadm stats dump -s /var/run/dovecot/mailtest/stats command user=warden cmd args session user last_update user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits select inbox 27fb87218302984f084c0000ae899e04 warden 1335362182.950250 0.000000 0.000000 172 0 46 0 0 12288 13 7883 4 381 0 0 0 0 0 search text "warden" 27fb87218302984f084c0000ae899e04 warden 1335362186.588023 0.001000 0.000000 32 0 4 1 0 0 4 108 1 43 0 0 0 0 0 logout 27fb87218302984f084c0000ae899e04 warden 1335362189.161790 0.000000 0.000000 1 0 0 3 0 0 1 108 3 157 0 0 0 0 0 This obviously isn't a show-stopper but if this is causing stats to be incorrect it would be great if this could be fixed. -David Warden My doveconf -n: # 2.1.5: /etc/dovecot/mailtest/dovecot.conf doveconf: Warning: service auth { client_limit=4096 } is lower than required under max. load (12288) doveconf: Warning: service anvil { client_limit=4096 } is lower than required under max. load (12291) # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.2 (Santiago) nfs auth_cache_negative_ttl = 0 auth_cache_size = 16 M auth_gssapi_hostname = $ALL auth_krb5_keytab = /etc/dovecot/mailtest.combined.keytab auth_master_user_separator = * auth_mechanisms = plain login gssapi auth_username_format = %Ln base_dir = /var/run/dovecot/mailtest/ default_client_limit = 4096 default_process_limit = 4096 deliver_log_format = msgid="%m" subject="%s" from="%f" size=%p result="%$" first_valid_uid = 0 hostname = mailtest.geneseo.edu instance_name = mailtest lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_log_format_elements = user=%u method=%m rip=%r lip=%l lport=%a mpid=%e encryption=%c mail_fsync = always mail_location = maildir:/Mail/mailhome/%Ln/mailtest/Maildir:CONTROL=/Mail/mailhome/%Ln/mailtest/.dovecot-control:INDEX=/Mail/mailhome/%Ln/mailtest/.dovecot-index mail_log_prefix = "service=%s user=%u rip=%r " mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = zlib quota mail_log notify fts fts_squat stats maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl mmap_disable = yes namespace default { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = no special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = . type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes } passdb { args = cache_key=%u dovecot driver = pam } plugin { fts = squat mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_fields = uid box msgid size from subject flags mail_log_group_events = yes quota = maildir:User quota quota_exceeded_message = Quota exceeded (mailbox for user is full). Please see http://go.geneseo.edu/emailoverquota for help deleting messages while over quota. quota_rule = *:storage=200M quota_rule2 = Trash:storage=+50M sieve = /Mail/mailhome/%Ln/mailtest/.filter.sieve sieve_dir = /Mail/mailhome/%Ln/mailtest/.sievedir sieve_max_redirects = 25 stats_memory_limit = 32 M stats_refresh = 30 secs stats_track_cmds = yes } postmaster_address = postmaster at geneseo.edu protocols = imap sieve lmtp quota_full_tempfail = yes service auth { unix_listener auth-exim { group = exim mode = 0660 } } service imap-login { inet_listener imap { port = 1143 } inet_listener imaps { port = 1993 } service_count = 0 vsz_limit = 256 M } service imap { process_limit = 4096 } service lmtp { inet_listener lmtp { port = 124 } } service managesieve-login { inet_listener sieve { port = 14190 } inet_listener sieve_deprecated { port = 12000 } } service pop3 { process_limit = 4096 } service stats { fifo_listener stats-mail { mode = 0666 } } ssl_ca = References: <4F95B3DD.1070609@allycomm.com> <4F962028.3060501@allycomm.com> <4EE7C62E-928C-499D-BE4E-9944D6013198@iki.fi> <4F96DD43.3070700@allycomm.com> <78F42D61-1402-408B-953B-F85254479761@iki.fi> Message-ID: <4F98091D.5070902@allycomm.com> On 04/24/2012 10:08 AM, Timo Sirainen wrote: > >>>> 25934 jeff 1 103 0 135M 102M CPU2 2 128:47 100.00% doveadm >>> Looks like it could be in an infinite loop. To find out where: >>> > That's a lot of getdirentries() calls in there, repeating all the time. So I don't think it's an infinite loop, just a lot of slowness repeating the same work over and over again. See if it helps if you add: > > maildir_very_dirty_syncs = yes I hope I understood correctly when I tried $ dsync -Dv -o maildir_very_dirty_syncs=yes backup maildir:~/Maildir into fresh directories (moving the previous Maildir aside). I didn't see any significant change (and it did not complete in several hours) on the usual UFS directory, and have also tried ZFS to see if it might be a filesystem issue. That has not completed either after over nine hours. From what I can see, I am guessing that it is "hanging" in index creation. Below, "mail" is the source directory, and "Maildir" is the destination of the sync. This is about nine hours of run time. [jeff at mail ~]$ ls -Al mail/.imap/cron_2010-07-10 total 9 -rw------- 1 jeff jeff 4910320 Feb 22 2011 dovecot.index -rw------- 1 jeff jeff 3600384 Apr 25 01:18 dovecot.index.cache -rw------- 1 jeff jeff 56 Feb 22 2011 dovecot.index.log [jeff at mail ~]$ ls -Al Maildir/.cron_2010-07-10 total 26 drwx------ 2 jeff jeff 102291 Apr 25 01:17 cur -rw------- 1 jeff jeff 36 Apr 25 01:18 dovecot-keywords -rw------- 1 jeff jeff 4122971 Apr 25 01:17 dovecot-uidlist -rw------- 1 jeff jeff 24 Apr 25 07:19 dovecot-uidlist.lock -rw------- 1 jeff jeff 1637008 Apr 25 07:03 dovecot.index -rw------- 1 jeff jeff 9442304 Apr 25 07:19 dovecot.index.cache -rw------- 1 jeff jeff 87984 Apr 25 07:19 dovecot.index.log -rw------- 1 jeff jeff 1048600 Apr 25 07:03 dovecot.index.log.2 -rw------- 1 jeff jeff 0 Apr 24 22:12 maildirfolder drwx------ 2 jeff jeff 2 Apr 24 22:12 new drwx------ 2 jeff jeff 2 Apr 25 01:17 tmp From tss at iki.fi Wed Apr 25 17:30:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 17:30:27 +0300 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F98073C.3080406@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> Message-ID: On 25.4.2012, at 17.16, Robert Schetterer wrote: > Am 25.04.2012 16:11, schrieb Timo Sirainen: >> p *set > > hope doing it right > > (gdb) p *set > $1 = {postmaster_address = 0xaa33d0 "P6\252", hostname = 0x0, > submission_host = 0x0, sendmail_path = 0x7fbaf6e41f50 > "H\211\\$\360H\211l$\370H\203\354\030\200=\207A%", > rejection_subject = 0x7fbaf6e42b90 > "H\211\\$\350L\211d$\370H\211\373H\211l$\360H\203\354(\200=?5%", > rejection_reason = 0x7fbaf6e41ed0 That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? From robert at schetterer.org Wed Apr 25 17:38:56 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 16:38:56 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> Message-ID: <4F980C80.2030908@schetterer.org> Am 25.04.2012 16:30, schrieb Timo Sirainen: > On 25.4.2012, at 17.16, Robert Schetterer wrote: > >> Am 25.04.2012 16:11, schrieb Timo Sirainen: >>> p *set >> >> hope doing it right >> >> (gdb) p *set >> $1 = {postmaster_address = 0xaa33d0 "P6\252", hostname = 0x0, >> submission_host = 0x0, sendmail_path = 0x7fbaf6e41f50 >> "H\211\\$\360H\211l$\370H\203\354\030\200=\207A%", >> rejection_subject = 0x7fbaf6e42b90 >> "H\211\\$\350L\211d$\370H\211\373H\211l$\360H\203\354(\200=?5%", >> rejection_reason = 0x7fbaf6e41ed0 > > That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? > its from latest http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ recompiled the debain way , never had bugged -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Wed Apr 25 17:43:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 17:43:06 +0300 Subject: [Dovecot] stats + fts squat plugins, indexer-worker error on message indexing In-Reply-To: References: Message-ID: <8F39E28F-3079-4EB1-BCAD-0A5C78DA05DC@iki.fi> On 25.4.2012, at 17.20, David Warden wrote: > Apr 25 09:56:26 wardentest3 dovecot: service=indexer-worker user=warden rip= Error: open(/proc/self/io) failed: Permission .. > Based on http://wiki2.dovecot.org/Services#indexer-worker it seems like this is because the indexer-worker process is created as root but at the time of indexing the indexer-worker drops privs to the mail user. In /proc//, io is mode 0400 and owned by root, so it makes sense that the mail user would be unable to read that file. Does the attached patch fix it? -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 2639 bytes Desc: not available URL: -------------- next part -------------- > The strange thing is it looks like the stats are still collected (unless I am reading the output of the stats dump incorrectly): > > [root at wardentest3 mailtest]# doveadm stats dump -s /var/run/dovecot/mailtest/stats command user=warden > cmd args session user last_update user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits > select inbox 27fb87218302984f084c0000ae899e04 warden 1335362182.950250 0.000000 0.000000 172 0 46 0 0 12288 13 7883 4 381 0 0 0 0 0 > search text "warden" 27fb87218302984f084c0000ae899e04 warden 1335362186.588023 0.001000 0.000000 32 0 4 1 0 0 4 108 1 43 0 0 0 0 0 > logout 27fb87218302984f084c0000ae899e04 warden 1335362189.161790 0.000000 0.000000 1 0 0 3 0 0 1 108 3 157 0 0 0 0 0 These are IMAP command statistics. The problem was only with indexer-worker. From xchris89x at googlemail.com Wed Apr 25 17:47:16 2012 From: xchris89x at googlemail.com (Chris) Date: Wed, 25 Apr 2012 16:47:16 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F980C80.2030908@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> <4F980C80.2030908@schetterer.org> Message-ID: 2012/4/25 Robert Schetterer : >> That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? >> > > its from latest > http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ > recompiled the debain way , never had bugged You are using packages compiled against debian sources under ubuntu??? -- Chris From robert at schetterer.org Wed Apr 25 17:50:16 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 16:50:16 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> <4F980C80.2030908@schetterer.org> Message-ID: <4F980F28.90205@schetterer.org> Am 25.04.2012 16:47, schrieb Chris: > 2012/4/25 Robert Schetterer : >>> That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? >>> >> >> its from latest >> http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ >> recompiled the debain way , never had bugged > > You are using packages compiled against debian sources under ubuntu??? > > -- > Chris jep that works since years -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From almarzuki2001 at hotmail.com Wed Apr 25 17:52:39 2012 From: almarzuki2001 at hotmail.com (Hadi Salem) Date: Wed, 25 Apr 2012 17:52:39 +0300 Subject: [Dovecot] dovecot-lda error Message-ID: Hi, im getting the fallowing error. from dovecot-lda log Apr 25 17:30:41 lda(jamal at example.com): Fatal: setgid(5003(vmail) from userdb lookup) failed with euid=8(mail), gid=12(mail), egid=12(mail): Operation not permitted (This binary should probably be called with process group set to 5003(vmail) instead of 12(mail)) this error occur if im using different uid and gid part from mail uid.. jamal at example.com:{PLAIN}123:5003:5003::/var/vmail/example.com/jamal/::userdb_mail=maildir:~/Maildir but if i use mail uid it work fine.. jamal at example.com:{PLAIN}123:8:12::/var/vmail/example.com/jamal/::userdb_mail=maildir:~/Maildir i tried using multiple uids as mention in wiki2.dovecot.org/LDA bye setting up setuid-root for dovecot-lda but im getting on-there error from sendmail :- Apr 25 17:30:41 host1 sendmail[31024]: q3PEUen2031022: to=, delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120359, relay=dovecot, dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/local/libexec/dovecot/dovecot-lda) exited with EX_TEMPFAIL im using sendmail. [root at host1 conf.d]# doveconf -n # 2.0.20: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-164.el5 x86_64 CentOS release 5.4 (Final) auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_verbose_passwords = plain debug_log_path = /var/log/dovecot.log disable_plaintext_auth = no first_valid_gid = 12 first_valid_uid = 8 hostname = host1.bigmama.com info_log_path = /var/log/dovecot.log last_valid_gid = 50000 last_valid_uid = 50000 log_path = /var/log/dovecot.log mail_debug = yes mail_gid = mail mail_location = maildir:~/Maildir mail_uid = mail passdb { args = scheme=PLAIN username_format=%u /etc/dovecot/users driver = passwd-file } postmaster_address = root at host1.bigmama.com service auth { unix_listener auth-userdb { group = mail mode = 0666 user = mail } } ssl = no userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol lda { info_log_path = /var/log/dovecot-lda.log log_path = /var/log/dovecot-lda-errors.log } From xchris89x at googlemail.com Wed Apr 25 17:54:03 2012 From: xchris89x at googlemail.com (Chris) Date: Wed, 25 Apr 2012 16:54:03 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F980F28.90205@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> <4F980C80.2030908@schetterer.org> <4F980F28.90205@schetterer.org> Message-ID: 2012/4/25 Robert Schetterer : > Am 25.04.2012 16:47, schrieb Chris: >> 2012/4/25 Robert Schetterer : >>>> That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? >>>> >>> >>> its from latest >>> http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ >>> recompiled the debain way , never had bugged >> >> You are using packages compiled against debian sources under ubuntu??? >> >> -- >> Chris > > jep that works since years Thats really a very bad idea. Please recompile it under ubuntu 10.04. -- Chris From robert at schetterer.org Wed Apr 25 17:59:21 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 16:59:21 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> <4F980C80.2030908@schetterer.org> <4F980F28.90205@schetterer.org> Message-ID: <4F981149.4070102@schetterer.org> Am 25.04.2012 16:54, schrieb Chris: > 2012/4/25 Robert Schetterer : >> Am 25.04.2012 16:47, schrieb Chris: >>> 2012/4/25 Robert Schetterer : >>>>> That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? >>>>> >>>> >>>> its from latest >>>> http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ >>>> recompiled the debain way , never had bugged >>> >>> You are using packages compiled against debian sources under ubuntu??? >>> >>> -- >>> Chris >> >> jep that works since years > > Thats really a very bad idea. Please recompile it under ubuntu 10.04. you miss understand i have it compiled under ubuntu 10.04 > > -- > Chris -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From joseba.torre at ehu.es Wed Apr 25 18:27:44 2012 From: joseba.torre at ehu.es (Joseba Torre) Date: Wed, 25 Apr 2012 17:27:44 +0200 Subject: [Dovecot] Error compiling pigeonhole 0.3.0 with dovecot-2.1.5 in openbsd 5.0 In-Reply-To: <4F968AF2.4090807@rename-it.nl> References: <4F967192.2060701@ehu.es> <4F968AF2.4090807@rename-it.nl> Message-ID: <4F9817F0.7030908@ehu.es> > > Looks like BSD doesn't know EPROTO (although it is POSIX). Dovecot uses > EIO elsewhere for such stream errors, so I adopted that for ManageSieve > as well. This should fix it: > > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/ef5045159e6b > > Regards, Perfect, thank you From robert at schetterer.org Wed Apr 25 18:48:20 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 17:48:20 +0200 Subject: [Dovecot] sieve vacation redirect problem / solved In-Reply-To: References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> Message-ID: <4F981CC4.3010600@schetterer.org> Am 25.04.2012 16:30, schrieb Timo Sirainen: > On 25.4.2012, at 17.16, Robert Schetterer wrote: > >> Am 25.04.2012 16:11, schrieb Timo Sirainen: >>> p *set >> >> hope doing it right >> >> (gdb) p *set >> $1 = {postmaster_address = 0xaa33d0 "P6\252", hostname = 0x0, >> submission_host = 0x0, sendmail_path = 0x7fbaf6e41f50 >> "H\211\\$\360H\211l$\370H\203\354\030\200=\207A%", >> rejection_subject = 0x7fbaf6e42b90 >> "H\211\\$\350L\211d$\370H\211\373H\211l$\360H\203\354(\200=?5%", >> rejection_reason = 0x7fbaf6e41ed0 > > That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? > jep it was broken , no idea why serveral new downloads give same result using http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz with recompile debian way ( same scripts etc ) to recent downloaded dovecot sources http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ works now no idea whats going on , but works now -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From weston.kris at ymail.com Wed Apr 25 18:52:12 2012 From: weston.kris at ymail.com (Kris Weston) Date: Wed, 25 Apr 2012 16:52:12 +0100 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <5B92F036-DDBE-42AB-94A1-DBF21F11DE14@iki.fi> References: <4F9796EE.5070907@ymail.com> <5B92F036-DDBE-42AB-94A1-DBF21F11DE14@iki.fi> Message-ID: <4F981DAC.8000106@ymail.com> On 25/04/12 11:48, Timo Sirainen wrote: > On 25.4.2012, at 9.17, Kris wrote: > >> but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user at domain): >> localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' > What exactly is doing this lookup? SMTP AUTH? Maybe the client's SMTP authentication is configured without the @domain part in username? > saslauthd via a query in dovecot-sql.conf.ext? after these suggestions im pretty sure nobody here has a clue about this, and this is not a known problem, i cant find anyone on the internet with it and the query is quite clearly coming up wrong and the unchangeable variable is quite clearly changing. the client can not be sending out different usernames every second surely. i put the username in myself in thunderbird, double, triple checked, made other accounts etc, the username changes by the second, ive watched the sql queries go in and they change for no obvious reason. From tss at iki.fi Wed Apr 25 18:55:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 18:55:40 +0300 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F981DAC.8000106@ymail.com> References: <4F9796EE.5070907@ymail.com> <5B92F036-DDBE-42AB-94A1-DBF21F11DE14@iki.fi> <4F981DAC.8000106@ymail.com> Message-ID: <8B6B7B68-278A-44C5-9013-AA14CCA991CD@iki.fi> On 25.4.2012, at 18.52, Kris Weston wrote: > On 25/04/12 11:48, Timo Sirainen wrote: >> On 25.4.2012, at 9.17, Kris wrote: >> >>> but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user at domain): >>> localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' >> What exactly is doing this lookup? SMTP AUTH? Maybe the client's SMTP authentication is configured without the @domain part in username? >> > saslauthd via a query in dovecot-sql.conf.ext? There is no saslauthd in Dovecot. > after these suggestions im pretty sure nobody here has a clue about this, and this is not a known problem, i cant find anyone on the internet with it and the query is quite clearly coming up wrong and the unchangeable variable is quite clearly changing. To me it looks like the client isn't sending the domain. > the client can not be sending out different usernames every second surely. i put the username in myself in thunderbird, double, triple checked, made other accounts etc, the username changes by the second, ive watched the sql queries go in and they change for no obvious reason. Show the full logs for a failing session, not just this one line. Even better would be full logs with auth_debug_passwords=yes, but in that case you'd better be using an unimportant password (some of the base64 encoded strings contain the password). From postmaster at mathiesen.info Wed Apr 25 20:56:18 2012 From: postmaster at mathiesen.info (postmaster) Date: Wed, 25 Apr 2012 19:56:18 +0200 Subject: [Dovecot] 2.1.5 problem with mail_location Message-ID: <4F983AC2.5010007@mathiesen.info> I've been advised to set : mail_location = maildir:~/ as the home directory is found through a sql request. this has been working perfectly up to and including 2.1.4 from 2.1.5 however this setting gives cause to a problem : instead of looking in the mail home eg /var/mail/domain/user\@domain/ dovecot is looking for mail in /var/mail/domain/user\@domain/~ the '~' directory is created upon the first request for mail and all future mail transactions in dovecot is done relative to this folder. is this behavior a bug or a feature ??? -- Bjarne D Mathiesen K?benhavn N ; Danmark ; Europa ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ denne besked er skrevet i et totalt M$-frit milj? MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3 From robert at schetterer.org Wed Apr 25 21:02:28 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 20:02:28 +0200 Subject: [Dovecot] sieve vacation redirect problem / solved / restested with lates hg pigeonhole broken In-Reply-To: <4F981CC4.3010600@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> <4F981CC4.3010600@schetterer.org> Message-ID: <4F983C34.4030605@schetterer.org> Am 25.04.2012 17:48, schrieb Robert Schetterer: > Am 25.04.2012 16:30, schrieb Timo Sirainen: >> On 25.4.2012, at 17.16, Robert Schetterer wrote: >> >>> Am 25.04.2012 16:11, schrieb Timo Sirainen: >>>> p *set >>> >>> hope doing it right >>> >>> (gdb) p *set >>> $1 = {postmaster_address = 0xaa33d0 "P6\252", hostname = 0x0, >>> submission_host = 0x0, sendmail_path = 0x7fbaf6e41f50 >>> "H\211\\$\360H\211l$\370H\203\354\030\200=\207A%", >>> rejection_subject = 0x7fbaf6e42b90 >>> "H\211\\$\350L\211d$\370H\211\373H\211l$\360H\203\354(\200=?5%", >>> rejection_reason = 0x7fbaf6e41ed0 >> >> That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? >> > > jep it was broken , no idea why serveral new downloads give same result > using > http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz > with recompile debian way ( same scripts etc ) to recent downloaded > dovecot sources > http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ > > works now > > no idea whats going on , but works now > just for info, i retested again with hg code latest hg clone http://hg.rename-it.nl/dovecot-2.1-pigeonhole/ compile runs ok... dpkg-buildpackage -rfakeroot -b ( on ubuntu 64 lucid latest patchlevel from today ) but at dpkg install it gives warning /usr/lib/dovecot/managesieve: symbol lookup error: /usr/lib/dovecot/managesieve: undefined symbol: sieve_file_script doveconf: Error: managesieve-login: dump-capability process returned 127 dont know if there is the bug reason however my test showes up described failures with latest hg code then sieve vacation redirect got broken it works if i use http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz for compile thx for your help and code anywhere, hope this results may help for debug -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Apr 25 21:16:58 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 20:16:58 +0200 Subject: [Dovecot] doveadm delete folders ? / unsubscribe ? In-Reply-To: <4F9795EE.4030604@schetterer.org> References: <4F900066.9030808@schetterer.org> <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> <4F906EE8.1030900@schetterer.org> <4F965B2E.2010103@schetterer.org> <283E4240-57E8-4BF2-8DCA-6DCC78C1D5A8@iki.fi> <4F9795EE.4030604@schetterer.org> Message-ID: <4F983F9A.5040900@schetterer.org> Am 25.04.2012 08:13, schrieb Robert Schetterer: > Am 24.04.2012 20:13, schrieb Timo Sirainen: >> On 24.4.2012, at 10.50, Robert Schetterer wrote: >> >>> doveadm expunge -d -u *@domain.example mailbox exampelfolder\* >>> savedbefore 10d >>> >>> works like charm, but the folders do not get unsubscribed >>> any chance to have i.e another add parameter ,or do it auto >>> cause empty and deleted folders should not stay subscribed in my eyes >> >> Yeah, I guess unsubscribing is always wanted: >> >> http://hg.dovecot.org/dovecot-2.1/rev/b1653b7bd165 >> > > Hi Timo, i will try after update and report, jep > unsubscribing should be the uni wanted behave > thx again ! tested works nice thx Timo ! -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From weston.kris at ymail.com Wed Apr 25 21:28:42 2012 From: weston.kris at ymail.com (Kris Weston) Date: Wed, 25 Apr 2012 19:28:42 +0100 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <8B6B7B68-278A-44C5-9013-AA14CCA991CD@iki.fi> References: <4F9796EE.5070907@ymail.com> <5B92F036-DDBE-42AB-94A1-DBF21F11DE14@iki.fi> <4F981DAC.8000106@ymail.com> <8B6B7B68-278A-44C5-9013-AA14CCA991CD@iki.fi> Message-ID: <4F98425A.6020908@ymail.com> >>>> but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user at domain): >>>> localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' >>> What exactly is doing this lookup? SMTP AUTH? Maybe the client's SMTP authentication is configured without the @domain part in username? >>> >> saslauthd via a query in dovecot-sql.conf.ext? > There is no saslauthd in Dovecot. thats correct, its not in dovecot, its installed on the same machine. but regardless, its salsauthd. >> after these suggestions im pretty sure nobody here has a clue about this, and this is not a known problem, i cant find anyone on the internet with it and the query is quite clearly coming up wrong and the unchangeable variable is quite clearly changing. > To me it looks like the client isn't sending the domain. so the client is changing its domain every coupla seconds for what reason, surely the thunderbird devs would have found this error? and even if thunderbird is sending it without the domain, the server should deal with thunderbird correctly, the user should not have to configure her thunderbird to work with dovecot rather than the other way round ? i dont believe this is the problem at all. >> the client can not be sending out different usernames every second surely. i put the username in myself in thunderbird, double, triple checked, made other accounts etc, the username changes by the second, ive watched the sql queries go in and they change for no obvious reason. > Show the full logs for a failing session, not just this one line. Even better would be full logs with auth_debug_passwords=yes, but in that case you'd better be using an unimportant password (some of the base64 encoded strings contain the password). thanks for trying, but forget it, its clear to me from answers ive received that nobody has a clue about this problem. guess im on me own. cheers bye! From warden at geneseo.edu Wed Apr 25 21:30:06 2012 From: warden at geneseo.edu (David Warden) Date: Wed, 25 Apr 2012 14:30:06 -0400 Subject: [Dovecot] stats + fts squat plugins, indexer-worker error on message indexing In-Reply-To: <8F39E28F-3079-4EB1-BCAD-0A5C78DA05DC@iki.fi> References: <8F39E28F-3079-4EB1-BCAD-0A5C78DA05DC@iki.fi> Message-ID: <86EED9D8-8445-49B4-9D72-C77883F926F9@geneseo.edu> On Apr 25, 2012, at 10:43 AM, Timo Sirainen wrote: > On 25.4.2012, at 17.20, David Warden wrote: > >> Apr 25 09:56:26 wardentest3 dovecot: service=indexer-worker user=warden rip= Error: open(/proc/self/io) failed: Permission > .. >> Based on http://wiki2.dovecot.org/Services#indexer-worker it seems like this is because the indexer-worker process is created as root but at the time of indexing the indexer-worker drops privs to the mail user. In /proc//, io is mode 0400 and owned by root, so it makes sense that the mail user would be unable to read that file. > > Does the attached patch fix it? > > The error is still logged, but is that intentional on your part? It looks like the patch logs the failure to open() when the error returned is anything other than ENOENT and in my case the error is EACCESS. Other than that it looks like it correctly disables accessing that file. >> The strange thing is it looks like the stats are still collected (unless I am reading the output of the stats dump incorrectly): >> >> [root at wardentest3 mailtest]# doveadm stats dump -s /var/run/dovecot/mailtest/stats command user=warden >> cmd args session user last_update user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits >> select inbox 27fb87218302984f084c0000ae899e04 warden 1335362182.950250 0.000000 0.000000 172 0 46 0 0 12288 13 7883 4 381 0 0 0 0 0 >> search text "warden" 27fb87218302984f084c0000ae899e04 warden 1335362186.588023 0.001000 0.000000 32 0 4 1 0 0 4 108 1 43 0 0 0 0 0 >> logout 27fb87218302984f084c0000ae899e04 warden 1335362189.161790 0.000000 0.000000 1 0 0 3 0 0 1 108 3 157 0 0 0 0 0 > > These are IMAP command statistics. The problem was only with indexer-worker. From tss at iki.fi Wed Apr 25 21:55:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 21:55:12 +0300 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F98425A.6020908@ymail.com> References: <4F9796EE.5070907@ymail.com> <5B92F036-DDBE-42AB-94A1-DBF21F11DE14@iki.fi> <4F981DAC.8000106@ymail.com> <8B6B7B68-278A-44C5-9013-AA14CCA991CD@iki.fi> <4F98425A.6020908@ymail.com> Message-ID: On 25.4.2012, at 21.28, Kris Weston wrote: >>>>> but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user at domain): >>>>> localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' >>>> What exactly is doing this lookup? SMTP AUTH? Maybe the client's SMTP authentication is configured without the @domain part in username? >>>> >>> saslauthd via a query in dovecot-sql.conf.ext? >> There is no saslauthd in Dovecot. > > thats correct, its not in dovecot, its installed on the same machine. > but regardless, its salsauthd. saslauthd doesn't authenticate via Dovecot. It might authenticate via IMAP though. > thanks for trying, but forget it, its clear to me from answers ive received that nobody has a clue about this problem. guess im on me own. cheers bye! The reason why nobody has a clue is because you haven't provided the full logs. We can only guess what the problem is. With full logs there would be no guessing necessary. From tss at iki.fi Wed Apr 25 22:09:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 22:09:09 +0300 Subject: [Dovecot] stats + fts squat plugins, indexer-worker error on message indexing In-Reply-To: <86EED9D8-8445-49B4-9D72-C77883F926F9@geneseo.edu> References: <8F39E28F-3079-4EB1-BCAD-0A5C78DA05DC@iki.fi> <86EED9D8-8445-49B4-9D72-C77883F926F9@geneseo.edu> Message-ID: On 25.4.2012, at 21.30, David Warden wrote: >>> Apr 25 09:56:26 wardentest3 dovecot: service=indexer-worker user=warden rip= Error: open(/proc/self/io) failed: Permission >> >> Does the attached patch fix it? > > The error is still logged, but is that intentional on your part? It looks like the patch logs the failure to open() when the error returned is anything other than ENOENT and in my case the error is EACCESS. Other than that it looks like it correctly disables accessing that file. The patch intended to open the /proc/self/io immediately at startup while still running as root. Anyway, now that I'm testing it again myself, I see that it's supposed to work even without the patch, because /proc/self/io's permissions change after seteuid(). I noticed similar errors happening somewhat randomly in another CentOS 6.2 system. Changing the kernel fixed it. (Downgrading at least worked, possibly upgrading might also.) From tss at iki.fi Wed Apr 25 23:46:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 23:46:08 +0300 Subject: [Dovecot] 2.1.5 problem with mail_location In-Reply-To: <4F983AC2.5010007@mathiesen.info> References: <4F983AC2.5010007@mathiesen.info> Message-ID: <1B191C98-90A2-40B6-ACDF-2420E9A5161B@iki.fi> On 25.4.2012, at 20.56, postmaster wrote: > I've been advised to set : > mail_location = maildir:~/ > as the home directory is found through a sql request. > > this has been working perfectly up to and including 2.1.4 > > from 2.1.5 however this setting gives cause to a problem : > instead of looking in the mail home eg /var/mail/domain/user\@domain/ dovecot is looking for mail in /var/mail/domain/user\@domain/~ > > the '~' directory is created upon the first request for mail and all future mail transactions in dovecot is done relative to this folder. Yeah, it's a bug. Fixed and added to tests so it won't break in future: http://hg.dovecot.org/dovecot-2.1/rev/249c1c89d9d3 From tss at iki.fi Thu Apr 26 00:16:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:16:45 +0300 Subject: [Dovecot] v2.1.5 released In-Reply-To: References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: On 24.4.2012, at 11.59, Gerhard Wiesinger wrote: > Hello Timo, > > Dovecot 2.1.5 seems to be broken. After upgrade from > 2.1.3 + patch http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e > I only see Inbox and Trash folder. > > Reverting => OK. > > Same config as before and disucssed per private E-mail and on the list. Guessing this comes again from: > mail_full_filesystem_access=yes I guess the problem was this: http://hg.dovecot.org/dovecot-2.1/rev/249c1c89d9d3 From Ralf.Hildebrandt at charite.de Thu Apr 26 00:26:10 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 25 Apr 2012 23:26:10 +0200 Subject: [Dovecot] v2.1.5 released In-Reply-To: References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <20120425212610.GA5653@charite.de> * Timo Sirainen : > On 24.4.2012, at 11.59, Gerhard Wiesinger wrote: > > > Hello Timo, > > > > Dovecot 2.1.5 seems to be broken. After upgrade from > > 2.1.3 + patch http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e > > I only see Inbox and Trash folder. > > > > Reverting => OK. > > > > Same config as before and disucssed per private E-mail and on the list. Guessing this comes again from: > > mail_full_filesystem_access=yes > > I guess the problem was this: http://hg.dovecot.org/dovecot-2.1/rev/249c1c89d9d3 2.1.5 doesn't seem to build with pigeonhole-sieve (or it's pigeonhole-sieve itself?): /bin/bash ../../libtool --tag=CC --mode=link gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -export-dynamic -g -Wl,--as-needed -o sievec sievec.o ../../src/lib-sieve/libdovecot-sieve.la ../../src/lib-sieve-tool/libsieve-tool.la /usr/src/dovecot-2.1/dovecot-2.1/src/lib-storage/libdovecot-storage.la -lssl -lcrypto /usr/src/dovecot-2.1/dovecot-2.1/src/lib-lda/libdovecot-lda.la /usr/src/dovecot-2.1/dovecot-2.1/src/lib-dovecot/libdovecot.la -export-dynamic -ldl -lrt libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -g -Wl,--as-needed -o .libs/sievec sievec.o -Wl,--export-dynamic ../../src/lib-sieve/.libs/libdovecot-sieve.so ../../src/lib-sieve-tool/.libs/libsieve-tool.a /usr/src/dovecot-2.1/dovecot-2.1/src/lib-storage/.libs/libdovecot-storage.so -lssl -lcrypto /usr/src/dovecot-2.1/dovecot-2.1/src/lib-lda/.libs/libdovecot-lda.so /usr/src/dovecot-2.1/dovecot-2.1/src/lib-dovecot/.libs/libdovecot.so -ldl -lrt -Wl,-rpath -Wl,/usr/local/dovecot-2.1/lib/dovecot ../../src/lib-sieve/.libs/libdovecot-sieve.so: error: undefined reference to 'sieve_file_script' ../../src/lib-sieve/.libs/libdovecot-sieve.so: error: undefined reference to 'sieve_dict_script' sievec.c:116: error: undefined reference to 'sieve_scriptfile_has_extension' collect2: ld returned 1 exit status make[3]: *** [sievec] Error 1 make[3]: Leaving directory /usr/src/dovecot-2.1/dovecot-2.1-pigeonhole/src/sieve-tools' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory /usr/src/dovecot-2.1/dovecot-2.1-pigeonhole/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory /usr/src/dovecot-2.1/dovecot-2.1-pigeonhole' make: *** [all] Error 2 Making install in . -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Thu Apr 26 00:27:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:27:25 +0300 Subject: [Dovecot] Permission problems when trying to create new folders with ACL plugin In-Reply-To: <20120424130005.0db3d425@legolas.home.ceotex.de> References: <20120424130005.0db3d425@legolas.home.ceotex.de> Message-ID: <7D3AF25B-FAE0-4187-B53C-DBB477A4AE79@iki.fi> On 24.4.2012, at 14.00, Markus Petri wrote: > I have the a problem with shared folders and the ACL plugin. The user > "markus" has full access to "Shared/shared-user/Data": > > 3 GETACL "Shared/shared-user/Data" > * ACL "Shared/shared-user/Data" "markus" akxeilprwtscd > 3 OK Getacl completed. > > but creating a new subfolder under "Data" fails: > > 4 CREATE "Shared/shared-user/Data/New1" > 4 NO [NOPERM] Permission denied Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e8b80e0767ac From tss at iki.fi Thu Apr 26 00:27:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:27:45 +0300 Subject: [Dovecot] acls not copied when creating subfolder of private INBOX In-Reply-To: <4F96B436.9090402@freenet.de> References: <4F96B436.9090402@freenet.de> Message-ID: <06DE7611-7783-4E5A-B58A-1448115802F5@iki.fi> On 24.4.2012, at 17.09, Andreas Helmcke wrote: > Using brand new dovecot 2.1.5 I still have a problem with the inheritance of acls. Maybe http://hg.dovecot.org/dovecot-2.1/rev/e8b80e0767ac fixes this as well? From tss at iki.fi Thu Apr 26 00:28:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:28:55 +0300 Subject: [Dovecot] Reading Mutt mboxes from Thunderbird In-Reply-To: <20120424160712.GB5048@leonardo-laptop> References: <20120424160712.GB5048@leonardo-laptop> Message-ID: On 24.4.2012, at 19.07, Leonardo M. Ram? wrote: > Hi, I'm a user of Mutt, and before moving to Thunderbird I'm trying to > share mboxes between them. To do this, I installed Dovecot to create an > IMAP server in my local machine, to serve mutt mboxes to be read by > Thunderbird. > > So far, I can read my inbox from /var/mail/%u, but in mutt, I have many > folders in ~/mail, like 2010-mails, 2011-mails, mailing-list-x, > mailing-list-y, and so on. How can I configure Dovecot to read them?. > > In /etc/dovecot/conf.d/10-mail.conf I have this: > > mail_location = mbox:~/Mail:INBOX=/var/mail/%u ~/mail vs. ~/Mail? From tss at iki.fi Thu Apr 26 00:29:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:29:51 +0300 Subject: [Dovecot] dovecot LDA error In-Reply-To: References: Message-ID: On 24.4.2012, at 20.41, Hadi Salem wrote: > Apr 24 20:25:46 lda(jamal at example.com): Fatal: > setgid(5000(vmail) from userdb lo > okup) failed with > euid=8(mail), gid=12(mail), egid=12(mail): Operation not permi > tted (This binary should probably be called with process group set to 5000(vmail > ) instead of 12(mail)) Configure your MTA to run dovecot-lda as vmail instead of as mail. From tss at iki.fi Thu Apr 26 00:31:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:31:29 +0300 Subject: [Dovecot] dovecot-lda error In-Reply-To: References: Message-ID: <1353D406-AEF1-46A1-ADB8-DFC159EC9946@iki.fi> On 25.4.2012, at 17.52, Hadi Salem wrote: > i tried using multiple uids as mention in wiki2.dovecot.org/LDA bye setting up setuid-root for dovecot-lda but im getting on-there error from sendmail :- > > Apr 25 17:30:41 host1 sendmail[31024]: q3PEUen2031022: to=, delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120359, relay=dovecot, dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/local/libexec/dovecot/dovecot-lda) exited with EX_TEMPFAIL What do the Dovecot logs say about this? > protocol lda { > info_log_path = /var/log/dovecot-lda.log > log_path = /var/log/dovecot-lda-errors.log > } If there is nothing in dovecot-lda-errors.log, make sure dovecot-lda has permissions to write there (e.g. temporarily make it world-writable). From tss at iki.fi Thu Apr 26 00:33:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:33:13 +0300 Subject: [Dovecot] v2.1.5 released In-Reply-To: <20120425212610.GA5653@charite.de> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> <20120425212610.GA5653@charite.de> Message-ID: On 26.4.2012, at 0.26, Ralf Hildebrandt wrote: > 2.1.5 doesn't seem to build with pigeonhole-sieve (or it's > pigeonhole-sieve itself?): Builds against Pigeonhole 0.3.0 tarball at least. From Ralf.Hildebrandt at charite.de Thu Apr 26 00:35:20 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 25 Apr 2012 23:35:20 +0200 Subject: [Dovecot] v2.1.5 released In-Reply-To: References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> <20120425212610.GA5653@charite.de> Message-ID: <20120425213520.GB5653@charite.de> * Timo Sirainen : > On 26.4.2012, at 0.26, Ralf Hildebrandt wrote: > > > 2.1.5 doesn't seem to build with pigeonhole-sieve (or it's > > pigeonhole-sieve itself?): > > Builds against Pigeonhole 0.3.0 tarball at least. Thought so. I'm rebuilding against that one while I write this :O -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Thu Apr 26 00:58:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:58:13 +0300 Subject: [Dovecot] v2.1.5 released In-Reply-To: <20120425212610.GA5653@charite.de> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> <20120425212610.GA5653@charite.de> Message-ID: <05BAF7A2-FFEC-4684-BEEC-B4B22F684114@iki.fi> On 26.4.2012, at 0.26, Ralf Hildebrandt wrote: > * Timo Sirainen : > 2.1.5 doesn't seem to build with pigeonhole-sieve (or it's > pigeonhole-sieve itself?): > > ../../src/lib-sieve/.libs/libdovecot-sieve.so: error: undefined reference to 'sieve_file_script' > ../../src/lib-sieve/.libs/libdovecot-sieve.so: error: undefined reference to 'sieve_dict_script' That's from hg, and you didn't run autogen.sh? ./configure --enable-maintainer-mode also helps avoiding this. From Ralf.Hildebrandt at charite.de Thu Apr 26 01:02:42 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Thu, 26 Apr 2012 00:02:42 +0200 Subject: [Dovecot] v2.1.5 released In-Reply-To: <05BAF7A2-FFEC-4684-BEEC-B4B22F684114@iki.fi> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> <20120425212610.GA5653@charite.de> <05BAF7A2-FFEC-4684-BEEC-B4B22F684114@iki.fi> Message-ID: <20120425220242.GY5653@charite.de> * Timo Sirainen : > On 26.4.2012, at 0.26, Ralf Hildebrandt wrote: > > > * Timo Sirainen : > > 2.1.5 doesn't seem to build with pigeonhole-sieve (or it's > > pigeonhole-sieve itself?): > > > > ../../src/lib-sieve/.libs/libdovecot-sieve.so: error: undefined reference to 'sieve_file_script' > > ../../src/lib-sieve/.libs/libdovecot-sieve.so: error: undefined reference to 'sieve_dict_script' > > That's from hg, yes > and you didn't run autogen.sh? ./configure --enable-maintainer-mode > also helps avoiding this. Damn, forgot that in the pigeonhole dir (I have that for the dovecot sources, but not for the pigeonhole sources)... -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From daniel.parthey at informatik.tu-chemnitz.de Thu Apr 26 01:51:16 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 26 Apr 2012 00:51:16 +0200 Subject: [Dovecot] How to remove duplicate GUID messages from storage? In-Reply-To: References: <20120421202928.GA12998@daniel.localdomain> Message-ID: <20120425225116.GB10062@daniel.localdomain> Timo Sirainen schrieb: > On 21.4.2012, at 23.29, Daniel Parthey wrote: > > > The following command > > doveadm -v force-resync -u username at example.org INBOX > > outputs lots of messages like this one, with different GUID > > doveadm(username at example.org): Error: mdbox /mail/dovecot/example.org/username/mail/storage: Duplicate GUID 0b8a032d66a0924fb42c0000de5f8128 in m.55:45484041 and m.14:52173045 > > I think this is a force-resync bug and it shouldn't really complain about duplicates. > Although I'm not entirely sure why with you it's complaining about them at all. I have anyway in TODO to look into this.. Maybe this is because I "manually" deleted all the mailboxes directories (rm -rf) containing the duplicates, which one should better avoid with mdbox mailboxes... :( So there is no "meta information" anymore, just the duplicate messages in the storage. and dovecot would need to "guess" where these messages from the store belong. > > How can I manually remove these identical, duplicate messages > > from the storage to save space? Dovecot does not do it > > automatically. > > Perhaps force-resync + purge should do that, but currently it doesn't. I already tried both, and purge is even running as a nightly cronjob. > Also it would be nice if doveadm import didn't add duplicates in the first place. > This is also something for which I have vague plans, because it would help dsync as well. Thanks for looking into this. I would really appreciate this idea of "duplicate GUID prevention" while duplicate messages (with different GUIDs) should probably be allowed. Maybe someone wants to store the same message several times... Regards, Daniel From daniel.parthey at informatik.tu-chemnitz.de Thu Apr 26 01:56:04 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 26 Apr 2012 00:56:04 +0200 Subject: [Dovecot] Reading Mutt mboxes from Thunderbird In-Reply-To: References: <20120424160712.GB5048@leonardo-laptop> Message-ID: <20120425225604.GA10291@daniel.localdomain> Timo Sirainen schrieb: > On 24.4.2012, at 19.07, Leonardo M. Ram? wrote: > > > Hi, I'm a user of Mutt, and before moving to Thunderbird I'm trying to > > share mboxes between them. To do this, I installed Dovecot to create an > > IMAP server in my local machine, to serve mutt mboxes to be read by > > Thunderbird. > > > > So far, I can read my inbox from /var/mail/%u, but in mutt, I have many > > folders in ~/mail, like 2010-mails, 2011-mails, mailing-list-x, > > mailing-list-y, and so on. How can I configure Dovecot to read them?. > > > > In /etc/dovecot/conf.d/10-mail.conf I have this: > > > > mail_location = mbox:~/Mail:INBOX=/var/mail/%u > > ~/mail vs. ~/Mail? On 2012-04-25 04:26:27 +0200, Daniel Parthey wrote: > Are you sure, dovecot does not read them, or are you just not "seeing" > your folders in thunderbird? Please try to right click your INBOX and > "subscribe" to the folders. On serverside, the subscribed folders are > listed in $mail_location/subscriptions, so you might auto-generate > this file by listing your mutt mailboxes. > > Regards, > Daniel You are right!, I just had to subscribe to the folders. Regards, Leonardo From dovecot at allycomm.com Thu Apr 26 02:16:29 2012 From: dovecot at allycomm.com (Jeff Kletsky) Date: Wed, 25 Apr 2012 16:16:29 -0700 Subject: [Dovecot] 2.1.x changes and dsync -- Wiki suggestion Message-ID: <4F9885CD.2000800@allycomm.com> There are a couple things that show up on the Upgrading/2.1 wiki page that would be useful to have on pages describing dsync (Tools/Dsync, Migration/MailFormat, Tools, perhaps others): * dsync was merged into doveadm * You can't dsync 2.0.x to/from 2.1.x That second point is especially "interesting" to me as it seems that Apple MacOS X Server is still on 2.0.16, at least from what I see on opensource.apple.com. I was planning on migrating mail off the current server onto my friend's new MacMini, but I'll have to figure out how to "downgrade" the mail jail to a 2.0 version, it seems. Thanks for all the great work, Jeff From jeff at j-simmons.net Thu Apr 26 04:34:49 2012 From: jeff at j-simmons.net (Jeff Simmons) Date: Wed, 25 Apr 2012 18:34:49 -0700 Subject: [Dovecot] dsync: convert only mbox in /var/mail to maildir? In-Reply-To: <04D01B3D-CD58-4E0A-8C1A-36B01E4792E7@iki.fi> References: <201204201226.41710.jeff@j-simmons.net> <04D01B3D-CD58-4E0A-8C1A-36B01E4792E7@iki.fi> Message-ID: <201204251834.50001.jeff@j-simmons.net> On Friday, April 20, 2012 02:27:59 pm Timo Sirainen wrote: > On 20.4.2012, at 22.26, Jeff Simmons wrote: > > I'm migrating an old UW pop server to a new Dovecot server. The old > > server did not allow mail to be stored on the server, so only mbox files > > exist in /var/mail, with nothing in /home/$user. I'd like to copy the > > /var/mail directory over to the new server and then use dsync to convert > > the mbox files and put them in /home/$user/Maildir, but every > > permutation of the command line I've tried dies with a variation on > > can't find/write either /var/mail/mail or /home/$user/mail, seems it > > won't deal with mbox files without an associated mail directory. Can > > dsync be used for this kind of conversion, or should I look elsewhere > > for conversion utilities? > > Assuming in destination server the mbox is in /var/mail/$user and you want > the destination to ~/Maildir: > > in dovecot.conf set: > mail_location = maildir.~/Maildir > > rm -rf /tmp/emptydir > dsync mirror mbox:/tmp/emptydir:/var/mail/$user Actually: dsync mirror mbox:/tmp/emptydir:INBOX=/var/mail/$user :) And be sure that /var/mail/$user has the same ownership:group as the user (i.e. $user:$user NOT $user:mail). > be sure to delete the emptydir between users. -- Jeff Simmons jeff at j-simmons.net Simmons Consulting - Network Engineering, Administration, Security From me at junc.org Thu Apr 26 04:53:08 2012 From: me at junc.org (Benny Pedersen) Date: Thu, 26 Apr 2012 03:53:08 +0200 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F97BACC.4080902@ymail.com> References: <4F9796EE.5070907@ymail.com> <20120425082615.GA14759@laptop-dg.leere.eu> <4F97BACC.4080902@ymail.com> Message-ID: <9d5bd8f6f2e4efc1c576f4a0246e03fb@junc.org> Den 2012-04-25 10:50, Kris Weston skrev: >> # auth_default_realm = > ahhh, ok. only i have multiple domains, so how does that werk ? > just put em all in ? if you use @ in imap/pop3 logins then the default for this setting is fine, if logins is without @ then you need to set it to the hostname of the machine where users is on, it cant be multiple hostnames on 127.0.0.1 :=) use localhost.example.org if unsure saslauthtest -u user -p passwd -r example.org OK or saslauthtest -u user at example.org -p passwd OK From jtam.home at gmail.com Thu Apr 26 05:14:38 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 25 Apr 2012 19:14:38 -0700 (PDT) Subject: [Dovecot] dovecot LDA with sendmail In-Reply-To: References: Message-ID: Timo, in response to Hadi, wrote: >> Apr 24 20:25:46 lda(jamal at example.com): Fatal: >> setgid(5000(vmail) from userdb lo >> okup) failed with >> euid=8(mail), gid=12(mail), egid=12(mail): Operation not permi >> tted (This binary should probably be called with process group set to 5000(vmail >> ) instead of 12(mail)) > > Configure your MTA to run dovecot-lda as vmail instead of as mail. Yeah, what he said. Someone helpfully added documentation on how to do this at the end of Dovecot1's wiki (http://wiki.dovecot.org/LDA/Sendmail), which probably should be copied to Dovecot 2's wiki. So that's what I did. Joseph Tam From abruce at tumnus.co.nz Thu Apr 26 08:35:00 2012 From: abruce at tumnus.co.nz (Bruce, Andrew) Date: Thu, 26 Apr 2012 17:35:00 +1200 Subject: [Dovecot] LDAP Lookup not returning value in maxStorage In-Reply-To: <4F74B274.8000600@tumnus.co.nz> References: <20120327091425.73963576@jimbo> <20120328092534.5690fa40@jimbo> <4F74B274.8000600@tumnus.co.nz> Message-ID: On 30 March 2012 08:05, Andrew Bruce wrote: > > On 28/03/2012 19:25, Nikita Koshikov wrote: >> >> On Wed, 28 Mar 2012 09:39:37 +1300 >> Bruce, Andrew wrote: >> >>> On 28 March 2012 09:36, Bruce, Andrew ?wrote: >>>> >>>> On 27 March 2012 19:14, Nikita Koshikov ?wrote: >>>>> >>>>> On Tue, 27 Mar 2012 13:57:04 +1300 >>>>> Bruce, Andrew wrote: >>>>> >>>>> Hi there, >>>>> >>>>> We're setting up a Dovecot virtual email setup - we've got everything >>>>> working perfect with LDAP logins authenticating against AD and so >>>>> forth, but we're having issues with retrieving the maxStorage value >>>>> from AD (this is a pre-setup field in AD that we'd like to use to set >>>>> per user quotas). >>>>> >>>>> In our LDAP lookup, we have the maxStorage entry listed under >>>>> user_attrs for the quota (user_attrs = >>>>> maxStorage=quota_rule=*:storage=%$M), and in the debug logs, can see >>>>> it trying to get the entry, but it fails with: >>>>> Mar 27 13:19:27 auth: Debug: ldap(username at site,192.168.1.5): user >>>>> search: base=dc=site,dc=local scope=subtree >>>>> filter=(&(objectClass=person)(| (userPrincipalName=username at site) >>>>> (|(mail=username at site)(samAccountName=username at site)))) >>>>> fields=maxStorage >>>>> Mar 27 13:19:27 auth: Debug: ldap(username at site,192.168.1.5): no >>>>> fields returned by the server >>>>> >>>>> At this point, we then see the default quota applied. >>>>> >>>> Try to change your quota rule to be like: >>>> maxStorage=quota_rule=*:bytes=%$ >>>> ? ? ? ? ? ? ? ? ? ? ? ?^^^^^^^^^ >>>> And put the value in bytes to maxStorage - if I remember correct - this >>>> is integer field and no K\M\G values is valid here. >>>> >>>> PS We successfully using maxStorage field to obtain non-default quota >>>> from AD, dovecot version 2.0.x >>>>> >>>>> If we change the name of the field from maxStorage to instanceType we >>>>> see the value show up in the logs and passed through to the quota >>>>> system and applied successfully: >>>>> Mar 27 11:09:01 auth: Debug: ldap(username at site,192.168.1.5): user >>>>> search: base=dc=site,dc=local scope=subtree >>>>> filter=(&(objectClass=person)(| (userPrincipalName=username at site) >>>>> (|(mail=username at site)(samAccountName=username at site)))) >>>>> fields=instanceType >>>>> Mar 27 11:09:01 auth: Debug: ldap(username at site,192.168.1.5): result: >>>>> instanceType(quota_rule=*:storage=%$M)=*:storage=4M >>>>> Mar 27 11:09:01 auth: Debug: master out: USER ? 3901227009 >>>>> username at site ? ?quota_rule=*:storage=4M >>>>> >>>>> >>>>> Which seems a bit weird. >>>>> >>>>> If we use ldapsearch and pass it the same search string and look for >>>>> the field maxStorage, we clearly see the field and the value being >>>>> returned. ?The result looks the same if we also lookup instanceType. >>>>> >>>>> We're using Dovecot 2.0.9. >>>>> >>>>> Does anyone have any idea as to why we can't use this field? >>>>> >>>>> Thanks, >>>>> >>>>> Andrew >>> >>> Tried your suggestion Nikita, no joy unfortunately. ?It still looks >>> like the value never gets returned from the LDAP server to Dovecot. >>> It definitely has something in the field (equivalent of 10GB, but in >>> bytes as suggested) and I changed the user_attrs also, but still get >>> the same "no fields returned by the server" error message. >>> >>> Modifying the user_attrs to lookup from a different field >>> (instanceType) definitely works. >>> >>> What exact version are you using - perhaps it's a problem with our >>> copy of 2.0.9. >>> >>> Thanks, >>> >>> Andrew >> >> maybe you met restriction of ldap port >> 3268?(http://wiki2.dovecot.org/AuthDatabase/LDAP) >> > Dead on - it was a restriction of ldap port 3268 - as soon as we pointed > ldapsearch at the same port, we got the same result - some of the fields > were missing. ?It all makes perfect sense and I wish I noticed that earlier. > > Now need to work out why Dovecot can get the fields and username back from > ldap on port 389, but it can't do the auth through it like it could with > 3268. > > Thanks Nikita for your help. > > Andrew Just to update the solution for us in the end... We couldn't get auth to work on port 389, but in the end we managed to get the maxStorage field to be presented via the query on port 3268. What we needed to do was to ensure that it was added as an attribute to the Global Catalog on our Windows Servers. We could then get this entry back via our query. There's instructions on how to do this on the following page: http://technet.microsoft.com/en-us/library/cc737521(v=ws.10).aspx Thanks for the help. Regards, Andrew From lists at wiesinger.com Thu Apr 26 09:26:34 2012 From: lists at wiesinger.com (Gerhard Wiesinger) Date: Thu, 26 Apr 2012 08:26:34 +0200 Subject: [Dovecot] v2.1.5 released In-Reply-To: References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <4F98EA9A.9080909@wiesinger.com> Hello Timo, Yes, fixed it. Thnx. I also had a coredump with 2.1.3 + http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e A02 LIST "" "*" imap(username: Panic: file mail-storage.c: line 628 (mailbox_alloc): assertion failed: (uni_utf8_str_is_valid(vname)) Also fixed with 2.1.5 + http://hg.dovecot.org/dovecot-2.1/rev/249c1c89d9d3 Ciao, Gerhard -- http://www.wiesinger.com/ On 25.04.2012 23:16, Timo Sirainen wrote: > On 24.4.2012, at 11.59, Gerhard Wiesinger wrote: > >> Hello Timo, >> >> Dovecot 2.1.5 seems to be broken. After upgrade from >> 2.1.3 + patch http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e >> I only see Inbox and Trash folder. >> >> Reverting => OK. >> >> Same config as before and disucssed per private E-mail and on the list. Guessing this comes again from: >> mail_full_filesystem_access=yes > I guess the problem was this: http://hg.dovecot.org/dovecot-2.1/rev/249c1c89d9d3 > > From markus at mpetri.org Thu Apr 26 09:34:44 2012 From: markus at mpetri.org (Markus Petri) Date: Thu, 26 Apr 2012 08:34:44 +0200 Subject: [Dovecot] Permission problems when trying to create new folders with ACL plugin In-Reply-To: <7D3AF25B-FAE0-4187-B53C-DBB477A4AE79@iki.fi> References: <20120424130005.0db3d425@legolas.home.ceotex.de> <7D3AF25B-FAE0-4187-B53C-DBB477A4AE79@iki.fi> Message-ID: <20120426083444.50be1185@legolas.home.ceotex.de> On Thu, 26 Apr 2012 00:27:25 +0300 Timo Sirainen wrote: > On 24.4.2012, at 14.00, Markus Petri wrote: > > > I have the a problem with shared folders and the ACL plugin. The > > user "markus" has full access to "Shared/shared-user/Data": > > > > 3 GETACL "Shared/shared-user/Data" > > * ACL "Shared/shared-user/Data" "markus" akxeilprwtscd > > 3 OK Getacl completed. > > > > but creating a new subfolder under "Data" fails: > > > > 4 CREATE "Shared/shared-user/Data/New1" > > 4 NO [NOPERM] Permission denied > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e8b80e0767ac > That fixed it. Thanks. From tomislav.mihalicek at gmail.com Thu Apr 26 13:37:32 2012 From: tomislav.mihalicek at gmail.com (Tomislav Mihalicek) Date: Thu, 26 Apr 2012 03:37:32 -0700 (PDT) Subject: [Dovecot] dovecot 2:2.1.5-0~auto+6 on debian squeeze Message-ID: <33752072.post@talk.nabble.com> After upgrade from latest 2.1.4 to 2.1.5 occasional coredumps on lmtp/lda daemon happens. It results in deffering mail in previous hop server and mail delay [1383350.925251] lmtp[22130]: segfault at 0 ip 00007f8829e2b4c5 sp 00007fff29fd37e0 error 4 in libdovecot-lda.so.0.0.0[7f8829e23000+b000] [1383351.038095] lmtp[22121]: segfault at 0 ip 00007f18863084c5 sp 00007fff65d87b20 error 4 in libdovecot-lda.so.0.0.0[7f1886300000+b000] [1383366.682285] lmtp[22101]: segfault at 0 ip 00007fe87a36a4c5 sp 00007fffb391b430 error 4 in libdovecot-lda.so.0.0.0[7fe87a362000+b000] [1383530.705974] lmtp[22126]: segfault at 0 ip 00007f0a7caa74c5 sp 00007ffff5bd0050 error 4 in libdovecot-lda.so.0.0.0[7f0a7ca9f000+b000] [1383530.844101] lmtp[17613]: segfault at 0 ip 00007f43aa1e84c5 sp 00007fff977ffd70 error 4 in libdovecot-lda.so.0.0.0[7f43aa1e0000+b000] [1383544.463542] lmtp[17576]: segfault at 0 ip 00007fbec6bc64c5 sp 00007fff3e347910 error 4 in libdovecot-lda.so.0.0.0[7fbec6bbe000+b000] -- View this message in context: http://old.nabble.com/dovecot-2%3A2.1.5-0%7Eauto%2B6-on-debian-squeeze-tp33752072p33752072.html Sent from the Dovecot mailing list archive at Nabble.com. From tomislav.mihalicek at gmail.com Thu Apr 26 13:39:22 2012 From: tomislav.mihalicek at gmail.com (Tomislav Mihalicek) Date: Thu, 26 Apr 2012 03:39:22 -0700 (PDT) Subject: [Dovecot] segfaults with dovecot 2:2.1.5-0~auto+6 on debian squeeze Message-ID: <33752072.post@talk.nabble.com> After upgrade from latest 2.1.4 to 2.1.5 occasional coredumps on lmtp/lda daemon happens. It results in deffering mail in previous hop server and mail delay [1383350.925251] lmtp[22130]: segfault at 0 ip 00007f8829e2b4c5 sp 00007fff29fd37e0 error 4 in libdovecot-lda.so.0.0.0[7f8829e23000+b000] [1383351.038095] lmtp[22121]: segfault at 0 ip 00007f18863084c5 sp 00007fff65d87b20 error 4 in libdovecot-lda.so.0.0.0[7f1886300000+b000] [1383366.682285] lmtp[22101]: segfault at 0 ip 00007fe87a36a4c5 sp 00007fffb391b430 error 4 in libdovecot-lda.so.0.0.0[7fe87a362000+b000] [1383530.705974] lmtp[22126]: segfault at 0 ip 00007f0a7caa74c5 sp 00007ffff5bd0050 error 4 in libdovecot-lda.so.0.0.0[7f0a7ca9f000+b000] [1383530.844101] lmtp[17613]: segfault at 0 ip 00007f43aa1e84c5 sp 00007fff977ffd70 error 4 in libdovecot-lda.so.0.0.0[7f43aa1e0000+b000] [1383544.463542] lmtp[17576]: segfault at 0 ip 00007fbec6bc64c5 sp 00007fff3e347910 error 4 in libdovecot-lda.so.0.0.0[7fbec6bbe000+b000] -- View this message in context: http://old.nabble.com/segfaults-with-dovecot-2%3A2.1.5-0%7Eauto%2B6-on-debian-squeeze-tp33752072p33752072.html Sent from the Dovecot mailing list archive at Nabble.com. From interfasys at gmail.com Thu Apr 26 17:23:25 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 26 Apr 2012 15:23:25 +0100 Subject: [Dovecot] LDA crashes on delivery (using sieve) Message-ID: <4F995A5D.7070708@gmail.com> Hello, The current version of LDA can crash (seg fault 11) under certain circumstances. I don't know why it happens only with some accounts, but here is a backtrace. The user has a sieve filter in place : (gdb) bt full #0 0x0000000800443434 in smtp_client_open () from /usr/lib/dovecot/libdovecot-lda.so.0 No symbol table info available. #1 0x00000008009943ab in lda_sieve_smtp_open.2406.2210 () from /usr/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #2 0x0000000800e80532 in sieve_smtp_open () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #3 0x0000000800e6157d in act_redirect_send.23018 () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #4 0x0000000800e7e175 in act_redirect_commit.23025 () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #5 0x0000000800e5f2bd in sieve_result_execute () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #6 0x0000000800e5ee7b in sieve_multiscript_execute.25305 () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #7 0x0000000800e496f8 in sieve_multiscript_run () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #8 0x0000000800994652 in lda_sieve_multiscript_execute.2399.2297 () from /usr/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #9 0x00000008009942c7 in lda_sieve_deliver_mail.2402.2206 () from /usr/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #10 0x00000008004435d8 in mail_deliver () from /usr/lib/dovecot/libdovecot-lda.so.0 No symbol table info available. #11 0x00000000004029f7 in main (argc=7, argv=0x7fffffffe830) at main.c:434 set_roots = {0x404520, 0x0} ctx = {pool = 0x800a0d820, set = 0x800a3cc98, session = 0x800a0d840, dup_ctx = 0x800abf0c0, session_id = 0x0, src_mail = 0x800a34840, src_envelope_sender = 0x800a0d858 "mailtest at example.com", dest_user = 0x800a89040, dest_addr = 0x7fffffffebfd "test at example.com", final_dest_addr = 0x7fffffffebfd "test at example.com", dest_mailbox_name = 0x40356b "INBOX", dest_mail = 0x0, var_expand_table = 0x0, tried_default_save = false, saved_mail = false, save_dest_mail = false, mailbox_full = false, dsn = false} service_flags = 1027 user = (const __unknown__ *) 0x7fffffffebcc "test at example.com" errstr = (const __unknown__ *) 0x0 path = (const __unknown__ *) 0x0 storage_service = (struct mail_storage_service_ctx *) 0x800a30840 service_user = (struct mail_storage_service_user *) 0x800a3c0a0 service_input = {module = 0x403567 "lda", service = 0x403567 "lda", username = 0x7fffffffebcc "test at example.com", session_id = 0x0, local_ip = {family = 0, u = {ip6 = {__u6_addr = {__u6_addr8 = '\0' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__u6_addr = {__u6_addr8 = '\0' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0} storage = (struct mail_storage *) 0x0 user_source = (const __unknown__ *) 0x4034c0 "" destaddr_source = (const __unknown__ *) 0x403576 "-a parameter" process_euid = 0 stderr_rejection = false ret = 1 c = -1 error = MAIL_ERROR_NONE Cheers, Olivier From interfasys at gmail.com Thu Apr 26 18:32:23 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 26 Apr 2012 16:32:23 +0100 Subject: [Dovecot] LDA crashes on delivery (using sieve) Message-ID: <4F996A87.90701@gmail.com> Hello, The current version of LDA can crash (seg fault 11) under certain circumstances. I don't know why it happens only with some accounts, but here is a backtrace. The user has a sieve filter in place : (gdb) bt full #0 0x0000000800443434 in smtp_client_open () from /usr/lib/dovecot/libdovecot-lda.so.0 No symbol table info available. #1 0x00000008009943ab in lda_sieve_smtp_open.2406.2210 () from /usr/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #2 0x0000000800e80532 in sieve_smtp_open () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #3 0x0000000800e6157d in act_redirect_send.23018 () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #4 0x0000000800e7e175 in act_redirect_commit.23025 () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #5 0x0000000800e5f2bd in sieve_result_execute () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #6 0x0000000800e5ee7b in sieve_multiscript_execute.25305 () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #7 0x0000000800e496f8 in sieve_multiscript_run () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #8 0x0000000800994652 in lda_sieve_multiscript_execute.2399.2297 () from /usr/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #9 0x00000008009942c7 in lda_sieve_deliver_mail.2402.2206 () from /usr/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #10 0x00000008004435d8 in mail_deliver () from /usr/lib/dovecot/libdovecot-lda.so.0 No symbol table info available. #11 0x00000000004029f7 in main (argc=7, argv=0x7fffffffe830) at main.c:434 set_roots = {0x404520, 0x0} ctx = {pool = 0x800a0d820, set = 0x800a3cc98, session = 0x800a0d840, dup_ctx = 0x800abf0c0, session_id = 0x0, src_mail = 0x800a34840, src_envelope_sender = 0x800a0d858 "mailtest at example.com", dest_user = 0x800a89040, dest_addr = 0x7fffffffebfd "test at example.com", final_dest_addr = 0x7fffffffebfd "test at example.com", dest_mailbox_name = 0x40356b "INBOX", dest_mail = 0x0, var_expand_table = 0x0, tried_default_save = false, saved_mail = false, save_dest_mail = false, mailbox_full = false, dsn = false} service_flags = 1027 user = (const __unknown__ *) 0x7fffffffebcc "test at example.com" errstr = (const __unknown__ *) 0x0 path = (const __unknown__ *) 0x0 storage_service = (struct mail_storage_service_ctx *) 0x800a30840 service_user = (struct mail_storage_service_user *) 0x800a3c0a0 service_input = {module = 0x403567 "lda", service = 0x403567 "lda", username = 0x7fffffffebcc "test at example.com", session_id = 0x0, local_ip = {family = 0, u = {ip6 = {__u6_addr = {__u6_addr8 = '\0' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__u6_addr = {__u6_addr8 = '\0' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0} storage = (struct mail_storage *) 0x0 user_source = (const __unknown__ *) 0x4034c0 "" destaddr_source = (const __unknown__ *) 0x403576 "-a parameter" process_euid = 0 stderr_rejection = false ret = 1 c = -1 error = MAIL_ERROR_NONE Cheers, Olivier From tss at iki.fi Thu Apr 26 18:35:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 18:35:17 +0300 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <4F996A87.90701@gmail.com> References: <4F996A87.90701@gmail.com> Message-ID: <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> On 26.4.2012, at 18.32, interfaSys s?rl wrote: > The current version of LDA can crash (seg fault 11) under certain > circumstances. "The current" means what exactly? (Stephan's autobuilt .debs?) From feltrin at gmail.com Thu Apr 26 18:40:30 2012 From: feltrin at gmail.com (Jean Michel) Date: Thu, 26 Apr 2012 12:40:30 -0300 Subject: [Dovecot] Troubles with /.dovecot.lda-dupe on dovecot 2.1.5 Message-ID: Seems that is something with vacation message, Apr 26 12:20:02 server dovecot: lmtp(26937, user at domain): Error: file_dotlock_create(/storage/domain/users/user/.dovecot.lda-dupes) failed: Resource temporarily unavailable Apr 26 12:20:02 server dovecot: lmtp(26937, user at domain): Fatal: master: service(lmtp): child 26937 killed with signal 11 (core dumps disabled) And its created a .dovecot.lda-dupes.lock Any idea ? -- ------------------------------ Jean Michel Feltrin From interfasys at gmail.com Thu Apr 26 18:47:08 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 26 Apr 2012 16:47:08 +0100 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> References: <4F996A87.90701@gmail.com> <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> Message-ID: <4F996DFC.8070607@gmail.com> Latest revision from these repositories: dovecot-2.1 dovecot-2.1-pigeonhole On 26/04/2012 16:35, Timo Sirainen wrote: > On 26.4.2012, at 18.32, interfaSys s?rl wrote: > >> The current version of LDA can crash (seg fault 11) under certain >> circumstances. > > > "The current" means what exactly? (Stephan's autobuilt .debs?) > > > From goetz.reinicke at filmakademie.de Thu Apr 26 18:48:36 2012 From: goetz.reinicke at filmakademie.de (=?ISO-8859-15?Q?G=F6tz_Reinicke?=) Date: Thu, 26 Apr 2012 17:48:36 +0200 Subject: [Dovecot] OT but E-Mail related. save a copy off all mail as eml Message-ID: <4F996E54.3020706@filmakademie.de> Hi, I'm sorry to bug you with an not directly dovecot related question, but this is for me right now the place with the best mailsystem admins I know. I'm asked if there is a way to save a copy of all mail we send trough our sendmail server (Red Hat EL 5.x) in and out as an .eml file for further processing by a DMS system. The files should be saved on some network share. Any suggestion are welcome and thanks. Best regards G?tz -- G?tz Reinicke IT-Koordinator Tel. +49 7141 969 82 420 Fax +49 7141 969 55 420 E-Mail goetz.reinicke at filmakademie.de Filmakademie Baden-W?rttemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: J?rgen Walter MdL Staatssekret?r im Ministerium f?r Wissenschaft, Forschung und Kunst Baden-W?rttemberg Gesch?ftsf?hrer: Prof. Thomas Schadt -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5161 bytes Desc: S/MIME Kryptografische Unterschrift URL: From robert at schetterer.org Thu Apr 26 19:02:36 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 26 Apr 2012 18:02:36 +0200 Subject: [Dovecot] Troubles with /.dovecot.lda-dupe on dovecot 2.1.5 In-Reply-To: References: Message-ID: <4F99719C.40500@schetterer.org> Am 26.04.2012 17:40, schrieb Jean Michel: > Seems that is something with vacation message, > > Apr 26 12:20:02 server dovecot: lmtp(26937, user at domain): Error: > file_dotlock_create(/storage/domain/users/user/.dovecot.lda-dupes) failed: > Resource temporarily unavailable > Apr 26 12:20:02 server dovecot: lmtp(26937, user at domain): Fatal: master: > service(lmtp): child 26937 killed with signal 11 (core dumps disabled) > And its created a .dovecot.lda-dupes.lock > > Any idea ? jep thats from sieve ,see my my mails from yesterday -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From interfasys at gmail.com Thu Apr 26 19:03:31 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 26 Apr 2012 17:03:31 +0100 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> References: <4F996A87.90701@gmail.com> <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> Message-ID: <4F9971D3.4020107@gmail.com> And it's on FreeBSD 8.2, compiled from source, using GCC 4.6 and the gold linker. On 26/04/2012 16:35, Timo Sirainen wrote: > On 26.4.2012, at 18.32, interfaSys s?rl wrote: > >> The current version of LDA can crash (seg fault 11) under certain >> circumstances. > > > "The current" means what exactly? (Stephan's autobuilt .debs?) > > > From robert at schetterer.org Thu Apr 26 19:04:58 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 26 Apr 2012 18:04:58 +0200 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <4F996DFC.8070607@gmail.com> References: <4F996A87.90701@gmail.com> <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> <4F996DFC.8070607@gmail.com> Message-ID: <4F99722A.7080300@schetterer.org> Am 26.04.2012 17:47, schrieb interfaSys s?rl: > Latest revision from these repositories: > dovecot-2.1 > dovecot-2.1-pigeonhole dovecot-2.1-pigeonhole from hg ,is your problem use i.e http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz to recompile , this works > > > On 26/04/2012 16:35, Timo Sirainen wrote: >> On 26.4.2012, at 18.32, interfaSys s?rl wrote: >> >>> The current version of LDA can crash (seg fault 11) under certain >>> circumstances. >> >> >> "The current" means what exactly? (Stephan's autobuilt .debs?) >> >> >> -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Thu Apr 26 19:09:01 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 26 Apr 2012 18:09:01 +0200 Subject: [Dovecot] OT but E-Mail related. save a copy off all mail as eml In-Reply-To: <4F996E54.3020706@filmakademie.de> References: <4F996E54.3020706@filmakademie.de> Message-ID: <4F99731D.3010405@schetterer.org> Am 26.04.2012 17:48, schrieb G?tz Reinicke: > Hi, > > I'm sorry to bug you with an not directly dovecot related question, but > this is for me right now the place with the best mailsystem admins I know. > > I'm asked if there is a way to save a copy of all mail we send trough > our sendmail server (Red Hat EL 5.x) in and out as an .eml file for > further processing by a DMS system. The files should be saved on some > network share. you need archive solution, in postfix its done like i.e.this recipient_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre sender_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre then use i.e procmail for filter , but i wouldnt recommend .eml i may be not easy to build uniq file names from i.e mail subject storing in maildir should be good enough ( each mail is a file ) > > Any suggestion are welcome and thanks. Best regards G?tz -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Thu Apr 26 19:10:59 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 26 Apr 2012 18:10:59 +0200 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <4F9971D3.4020107@gmail.com> References: <4F996A87.90701@gmail.com> <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> <4F9971D3.4020107@gmail.com> Message-ID: <4F997393.5050306@schetterer.org> Am 26.04.2012 18:03, schrieb interfaSys s?rl: > And it's on FreeBSD 8.2, compiled from source, using GCC 4.6 and the > gold linker. > > On 26/04/2012 16:35, Timo Sirainen wrote: >> On 26.4.2012, at 18.32, interfaSys s?rl wrote: >> >>> The current version of LDA can crash (seg fault 11) under certain >>> circumstances. >> >> >> "The current" means what exactly? (Stephan's autobuilt .debs?) >> >> >> oh ,may be more complicated with bsd i think i saw some late patches to compile -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From interfasys at gmail.com Thu Apr 26 22:18:10 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 26 Apr 2012 20:18:10 +0100 Subject: [Dovecot] LDA crashes on delivery (using sieve) Message-ID: <4F999F72.1060603@gmail.com> I can confirm that the issues are gone when using pigeonhole-0.3.0 I thought the sievec crashes might re-appear, but this isn't the case. Thank you Robert. Olivier > Am 26.04.2012 17:47, schrieb interfaSys s?rl: > >/ Latest revision from these repositories: > />/ dovecot-2.1 > />/ dovecot-2.1-pigeonhole > / > dovecot-2.1-pigeonhole from hg ,is your problem > use i.e > > http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz > to recompile , this works > > >/ > />/ > />/ On 26/04/2012 16:35, Timo Sirainen wrote: > />>/ On 26.4.2012, at 18.32, interfaSys s?rl wrote: > />>/ > />>>/ The current version of LDA can crash (seg fault 11) under certain > />>>/ circumstances. > />>/ > />>/ > />>/ "The current" means what exactly? (Stephan's autobuilt .debs?) > />>/ > />>/ > />>/ > / > > -- > Best Regards > > MfG Robert Schetterer > > Germany/Munich/Bavaria From stephan at rename-it.nl Fri Apr 27 03:28:04 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 27 Apr 2012 02:28:04 +0200 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <4F996A87.90701@gmail.com> References: <4F996A87.90701@gmail.com> Message-ID: <4F99E814.3050509@rename-it.nl> On 4/26/2012 5:32 PM, interfaSys s?rl wrote: > Hello, > > The current version of LDA can crash (seg fault 11) under certain > circumstances. > > I don't know why it happens only with some accounts, but here is a > backtrace. The user has a sieve filter in place : Fixed: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/f146790b5db3 Regards, Stephan. From nick.urbanik at optusnet.com.au Fri Apr 27 09:34:31 2012 From: nick.urbanik at optusnet.com.au (Nick Urbanik) Date: Fri, 27 Apr 2012 16:34:31 +1000 Subject: [Dovecot] 2.1.5: Panic: file mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: (mailbox_list_is_valid_pattern(_list, name)) Message-ID: <20120427063431.GA12180@nick.optusnet.com.au> Dear Folks, We had this assertion failure on Dovecot 2.1.1 on a particular mailbox. I upgraded to 2.1.5, and we have the same assertion failure, on the same mailbox: Apr 27 16:02:03 imap(nu-imaptest00258 at example.com): Panic: file mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: (mailbox_list_is_valid_pattern(_list, name)) Apr 27 16:02:03 imap(nu-imaptest00258 at example.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x4246a) [0x7f310c28146a] -> /usr/lib64/dovecot/libdovecot.so.0(+0x424b6) +[0x7f310c2814b6] -> /usr/lib64/dovecot/libdovecot.so.0(+0x1950a) [0x7f310c25850a] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x889b5) [0x7f310c5539b5] -> +/usr/lib64/dovecot/lib10_quota_plugin.so(+0x8a30) [0x7f3109ccea30] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0x8cdf) [0x7f3109ccecdf] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0x93c8) +[0x7f3109ccf3c8] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0x9436) [0x7f3109ccf436] -> /usr/lib64/dovecot/lib10_quota_plugin.so(quota_get_resource+0x72) [0x7f3109cca792] -> +/usr/lib64/dovecot/lib10_quota_plugin.so(quota_test_alloc+0x1bd) [0x7f3109ccaaed] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0xa870) [0x7f3109cd0870] -> +/usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_save_begin+0x4d) [0x7f310c544b9d] -> dovecot/imap [nu-imaptest00258 at example.com 127.0.0.1 APPEND]() [0x408ac4] -> dovecot/imap +[nu-imaptest00258 at example.com 127.0.0.1 APPEND](cmd_append+0x139) [0x408d69] -> dovecot/imap [nu-imaptest00258 at example.com 127.0.0.1 APPEND](command_exec+0x3d) [0x410aed] -> dovecot/imap +[nu-imaptest00258 at example.com 127.0.0.1 APPEND]() [0x40fa2e] -> dovecot/imap [nu-imaptest00258 at example.com 127.0.0.1 APPEND]() [0x40fb1a] -> dovecot/imap [nu-imaptest00258 at example.com +127.0.0.1 APPEND](client_handle_input+0x135) [0x40fd45] -> dovecot/imap [nu-imaptest00258 at example.com 127.0.0.1 APPEND](client_input+0x5f) [0x41066f] -> +/usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f310c28d9d6] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) [0x7f310c28ea5f] -> +/usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f310c28d978] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f310c279c73] -> dovecot/imap [nu-imaptest00258 at example.com +127.0.0.1 APPEND](main+0x29d) [0x418b3d] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f310bebccdd] -> dovecot/imap [nu-imaptest00258 at example.com 127.0.0.1 APPEND]() [0x408449] Apr 27 16:02:03 imaptest01 perdition.imap4[15893]: Closing session: 211.29.132.209:42922->211.29.132.245:143 authorisation_id=NONE authentication_id="nu-imaptest00258" received=67 sent=53 Apr 27 16:02:03 imap(nu-imaptest00258 at example.com): Fatal: master: service(imap): child 15894 killed with signal 6 (core not dumped) Any suggestions welcome. I guess enabling core dumps is a good start, I'll do that, but has anyone seen this before? -- Nick Urbanik http://nicku.org 808-71011 nick.urbanik at optusnet.com.au GPG: 7FFA CDC7 5A77 0558 DC7A 790A 16DF EC5B BB9D 2C24 ID: BB9D2C24 I disclaim, therefore I am. From robert at schetterer.org Fri Apr 27 09:50:24 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 27 Apr 2012 08:50:24 +0200 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <4F99E814.3050509@rename-it.nl> References: <4F996A87.90701@gmail.com> <4F99E814.3050509@rename-it.nl> Message-ID: <4F9A41B0.8040302@schetterer.org> Am 27.04.2012 02:28, schrieb Stephan Bosch: > On 4/26/2012 5:32 PM, interfaSys s?rl wrote: >> Hello, >> >> The current version of LDA can crash (seg fault 11) under certain >> circumstances. >> >> I don't know why it happens only with some accounts, but here is a >> backtrace. The user has a sieve filter in place : > > Fixed: > > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/f146790b5db3 > > Regards, > > Stephan. Hi Stephan, i can confirm by small tests using deb source latest from http://hg.rename-it.nl your patch fixed the lmtp crash with sieve and redirect/vacation Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From fumiyas at osstech.jp Fri Apr 27 10:20:28 2012 From: fumiyas at osstech.jp (SATOH Fumiyasu) Date: Fri, 27 Apr 2012 16:20:28 +0900 Subject: [Dovecot] [PATCH] Dovecot 2.1 compiling fix for AIX Message-ID: <87vcklfxoj.wl%fumiyas@osstech.jp> Hi, I'm trying to build Dovecot 2.1.5 on AIX 6.1 and got the following build error: $ ./configure ... $ make ... In file included from stats-connection.c:9: stats-plugin.h:14: error: field 'user_cpu' has incomplete type stats-plugin.h:14: error: field 'sys_cpu' has incomplete type ... The attached patch fixes this problem. -- -- Name: SATOH Fumiyasu (fumiyas @ osstech co jp) -- Business Home: http://www.OSSTech.co.jp/ -- Personal Home: https://github.com/fumiyas/ -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-2.1-stats-plugin-build-on-AIX.patch Type: application/octet-stream Size: 821 bytes Desc: not available URL: From fumiyas at osstech.jp Fri Apr 27 10:49:51 2012 From: fumiyas at osstech.jp (SATOH Fumiyasu) Date: Fri, 27 Apr 2012 16:49:51 +0900 Subject: [Dovecot] [PATCH] Dovecot 2.1 compiling fix for AIX In-Reply-To: <87vcklfxoj.wl%fumiyas@osstech.jp> References: <87vcklfxoj.wl%fumiyas@osstech.jp> Message-ID: <87sjfpfwbk.wl%fumiyas@osstech.jp> Revised patch. Sorry. At Fri, 27 Apr 2012 16:20:28 +0900, SATOH Fumiyasu wrote: > I'm trying to build Dovecot 2.1.5 on AIX 6.1 and got the following > build error: > > $ ./configure > ... > $ make > ... > In file included from stats-connection.c:9: > stats-plugin.h:14: error: field 'user_cpu' has incomplete type > stats-plugin.h:14: error: field 'sys_cpu' has incomplete type > ... > > The attached patch fixes this problem. -- -- Name: SATOH Fumiyasu (fumiyas @ osstech co jp) -- Business Home: http://www.OSSTech.co.jp/ -- Personal Home: https://github.com/fumiyas/ -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-2.1-include-sys-time.h-to-build-on-AIX.patch Type: application/octet-stream Size: 1230 bytes Desc: not available URL: From interfasys at gmail.com Fri Apr 27 20:11:14 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Fri, 27 Apr 2012 18:11:14 +0100 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <4F99E814.3050509@rename-it.nl> References: <4F996A87.90701@gmail.com> <4F99E814.3050509@rename-it.nl> Message-ID: <4F9AD332.4080803@gmail.com> Cheers Stephan! :) On 27/04/2012 01:28, Stephan Bosch wrote: > On 4/26/2012 5:32 PM, interfaSys s?rl wrote: >> Hello, >> >> The current version of LDA can crash (seg fault 11) under certain >> circumstances. >> >> I don't know why it happens only with some accounts, but here is a >> backtrace. The user has a sieve filter in place : > > Fixed: > > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/f146790b5db3 > > Regards, > > Stephan. From almarzuki2001 at hotmail.com Sat Apr 28 03:45:08 2012 From: almarzuki2001 at hotmail.com (Hadi Salem) Date: Sat, 28 Apr 2012 03:45:08 +0300 Subject: [Dovecot] dovecot LDA error In-Reply-To: References: , Message-ID: Hi Timo Sirainen, It?s working with me fine. With no problem. After I configured my MTA (sendmail) to run dovecot-lda as root with chmod 4750 /local/user/libexce/dovecot/dovecot-lda. And create group called secmail. I had to change dovecot-lda group to secmail. And then added mail to secmail group. It works with me. Before when I changed dovecot-lda group direct to mail doesn?t work. But it should to work.!!! I think that was the error. Later after creating separate group and adding mail to it, it?s working fine.. Any idea here? why doesn?t work if I changed dovecot-lda direct to mail group instead of creating new group and adding mail to it???. Hadi- > From: tss at iki.fi > Date: Thu, 26 Apr 2012 00:29:51 +0300 > To: almarzuki2001 at hotmail.com > CC: dovecot at dovecot.org > Subject: Re: [Dovecot] dovecot LDA error > > On 24.4.2012, at 20.41, Hadi Salem wrote: > > > Apr 24 20:25:46 lda(jamal at example.com): Fatal: > > setgid(5000(vmail) from userdb lo > > okup) failed with > > euid=8(mail), gid=12(mail), egid=12(mail): Operation not permi > > tted (This binary should probably be called with process group set to 5000(vmail > > ) instead of 12(mail)) > > Configure your MTA to run dovecot-lda as vmail instead of as mail. > From daniel.parthey at informatik.tu-chemnitz.de Sat Apr 28 14:00:23 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 28 Apr 2012 13:00:23 +0200 Subject: [Dovecot] lmtp proxy timeout while waiting for reply to DATA reply Message-ID: <20120428110023.GA9236@daniel.localdomain> Hi, we are experiencing similar sporadic data timeout issues with dovecot 2.0.20 as in http://dovecot.org/pipermail/dovecot/2011-June/059807.html at least once a week. Some mails get temporarily deferred in the postfix queue since dovecot director lmtp refuses them and the mails are delivered at a later time. The problem applies to mails delivered to single users as well as distribution lists where the mails gets delivered to several mailboxes by postfix. The postfix log shows the following output: postfix.log.7.gz:Apr 21 18:48:46 10.129.3.233 postfix/lmtp[15868]: 88062C08E: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=35, delays=5.4/0.01/0/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) The dovecot.log (mailbox and director log there via syslog) shows the following output: Apr 21 18:46:53 10.129.3.233 dovecot: lmtp(15869): Debug: none: root=, index=, control=, inbox=, alt= Apr 21 18:46:53 10.129.3.233 dovecot: lmtp(15869): Connect from 127.0.0.1 Apr 21 18:46:53 10.129.3.233 dovecot: lmtp(15869): Debug: auth input: user=username at example.org proxy port=19024 host=10.129.3.190 proxy_refresh=450 Apr 21 18:46:53 10.129.3.233 dovecot: lmtp(15869): Disconnect from 127.0.0.1: Client quit (in reset) Apr 21 18:47:29 10.129.3.233 dovecot: lmtp(15869): Debug: none: root=, index=, control=, inbox=, alt= Apr 21 18:47:29 10.129.3.233 dovecot: lmtp(15869): Connect from 127.0.0.1 Apr 21 18:47:29 10.129.3.233 dovecot: lmtp(15869): Debug: auth input: user=username at example.org proxy port=19024 host=10.129.3.190 proxy_refresh=450 Apr 21 18:47:29 10.129.3.233 dovecot: lmtp(15869): Disconnect from 127.0.0.1: Client quit (in reset) Apr 21 18:48:15 10.129.3.233 dovecot: lmtp(15869): Debug: none: root=, index=, control=, inbox=, alt= Apr 21 18:48:15 10.129.3.233 dovecot: lmtp(15869): Connect from 127.0.0.1 Apr 21 18:48:15 10.129.3.233 dovecot: lmtp(15869): Debug: auth input: user=username at example.org proxy port=19024 host=10.129.3.190 proxy_refresh=450 Apr 21 18:48:46 10.129.3.233 dovecot: lmtp(15869): Disconnect from 127.0.0.1: Client quit (in reset) One cannot identify the lmtp log origin from the log_format prefixes, but it seems that the "lmtp($PID)" log originated from the director, since proxy port 19024 is mentioned in the logs, which is our dovecot mailbox backend. Config for dovecot (mailbox) and director (proxy) is attached. mail_debug is yes for both dovecot and director. How can we find out more information? Shall I apply the following patches from http://dovecot.org/pipermail/dovecot/2011-June/059987.html or are they already included with version 2.0.20? Regards, Daniel -------------- next part -------------- # 2.0.20: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-40-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_cache_ttl = 1 mins auth_verbose = yes auth_verbose_passwords = sha1 deliver_log_format = mailbox: deliver: msgid=%m from=%f: %$ dict { quota = mysql:/etc/dovecot/conf.d/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = Mailbox login_log_format = mailbox: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "mailbox: mail: %s(%u): " mail_plugins = quota mail_privileged_group = vmail mail_uid = vmail managesieve_implementation_string = Sieve managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 50 M mmap_disable = yes passdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve service dict { unix_listener dict { group = vmail mode = 0660 } } service imap-login { inet_listener imap { port = 19143 } } service lmtp { inet_listener lmtp { address = * port = 19024 } } service managesieve-login { inet_listener sieve { port = 19200 } } service pop3-login { inet_listener pop3 { port = 19110 } } service quota-warning { executable = script /usr/local/bin/quota-warning unix_listener quota-warning { user = vmail } user = dovecot } ssl = no userdb { driver = prefetch } userdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_plugins = quota imap_quota } protocol lmtp { mail_plugins = quota sieve } -------------- next part -------------- # 2.0.20: /etc/dovecot-director/dovecot-director.conf # OS: Linux 2.6.32-40-server x86_64 Ubuntu 10.04.4 LTS auth_verbose = yes auth_verbose_passwords = sha1 base_dir = /var/run/dovecot-director deliver_log_format = director: deliver: msgid=%m from=%f: %$ director_mail_servers = 10.129.3.193 10.129.3.192 10.129.3.191 10.129.3.190 director_servers = 10.129.3.193 10.129.3.192 10.129.3.191 10.129.3.190 lmtp_proxy = yes login_greeting = Mail Balancer login_log_format = director: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "director: mail: %s(%u): " mail_privileged_group = vmail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes passdb { args = proxy=y nopassword=y user=%n at dovecotmail.%d driver = static } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director inet_listener imap { port = 20143 } inet_listener imaps { port = 20993 ssl = yes } } service lmtp { inet_listener lmtp { address = * port = 20024 } } service managesieve-login { inet_listener sieve { port = 20200 } } service pop3-login { executable = pop3-login director inet_listener pop3 { port = 20110 } inet_listener pop3s { port = 20995 ssl = yes } } ssl_cert = -- View this message in context: http://old.nabble.com/ERR-Attentication-error-dovecot-tp33763251p33763251.html Sent from the Dovecot mailing list archive at Nabble.com. From tss at iki.fi Sat Apr 28 17:53:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Apr 2012 17:53:12 +0300 Subject: [Dovecot] Session IDs Message-ID: <3AFABAEC-9FB7-449D-9473-64714C2038A7@iki.fi> v2.1.2 added support for "session IDs", which is a 9 year unique identifier for that specific IMAP/POP3 connection. Initially I thought it would only be useful for tracking connections going through Dovecot proxies to backends (the session ID is forwarded), but then I thought it could be useful for everyone if auth process also logs the session ID. So I thought for v2.1.6 I'd change the defaults: a) Failed login: Apr 28 17:37:23 auth: Debug: client in: AUTH 1 PLAIN service=imap secured session=XtG7KL6+jQB/AAAB lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=42125 resp= Apr 28 17:37:23 auth: Debug: passwd-file(foo,127.0.0.1,): lookup: user=foo file=/usr/local/etc/passwd.imap Apr 28 17:37:23 auth: Info: passwd-file(foo,127.0.0.1,): unknown user Apr 28 17:37:25 auth: Debug: client out: FAIL 1 user=foo Apr 28 17:37:50 imap-login: Info: Aborted login (auth failed, 1 attempts in 27 secs): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, session= b) Successful login: Apr 28 17:38:27 auth: Debug: client in: AUTH 2 PLAIN service=imap secured session=rNuHLL6+jgB/AAAB lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=42126 resp= Apr 28 17:38:27 auth: Debug: passwd-file(tss,127.0.0.1,): lookup: user=tss file=/usr/local/etc/passwd.imap Apr 28 17:38:27 auth: Debug: client out: OK 2 user=tss Apr 28 17:38:27 auth: Debug: master in: REQUEST 1375993857 14321 2 0f2f78deedcb7e38a278cf79819360c5 Apr 28 17:38:27 auth: Debug: passwd-file(tss,127.0.0.1,): lookup: user=tss file=/usr/local/etc/passwd.imap Apr 28 17:38:27 auth: Debug: master out: USER 1375993857 tss uid=1000 gid=1000 home=/home/tss Apr 28 17:38:27 imap-login: Info: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=14338, secured, session= .. Apr 28 17:47:15 imap(tss,): Info: Disconnected: Logged out in=8 out=369 The last line requires a modified mail_log_prefix, I wasn't planning on changing its default. I'm mainly wondering: 1) Does this change break someone's auth log line parsing? 2) Does the auth prefix look prettier with or without <> around session id? :) 3) Anything else I should do about this now that I'm changing it anyway? From dieterknopf at googlemail.com Sat Apr 28 19:07:08 2012 From: dieterknopf at googlemail.com (Dieter Knopf) Date: Sat, 28 Apr 2012 18:07:08 +0200 Subject: [Dovecot] Refiltering mails without loosing flags In-Reply-To: <4EE5E8FF.8080306@rename-it.nl> References: <4EE5E8FF.8080306@rename-it.nl> Message-ID: 2011/12/12 Stephan Bosch : > Since last week, the new sieve-filter command line tool, which is intended > for just such a task, is part of the normal compile output for Pigeonhole > 0.3.x for Dovecot v2.1. Unfortunately, it is not released yet, so I guess > you're stuck with your old getmail-based solution for now. > > The tool is still experimental and I must say that I have not tested the > preservation of flags for some time and many changes were performed since > then. I will verify that later today. Also, the interaction with the new > editheader extension needs attention. Is there any update about that? Would be really nice to have an nice and easy way like with procmail. Just via command line not via getmail/imap/.... Thanks From dieterknopf at googlemail.com Sat Apr 28 19:09:10 2012 From: dieterknopf at googlemail.com (Dieter Knopf) Date: Sat, 28 Apr 2012 18:09:10 +0200 Subject: [Dovecot] SETANNOTATION for Dovecot Message-ID: Hello, i'm using kmail2 and for Folder Settings the SETANNOATION command is needed. Is there any way to add this command to Dovecot? I already found a dovecot-metadata-plugin but there is no Package available for Debian. Thanks Dieter From daniel.parthey at informatik.tu-chemnitz.de Sat Apr 28 20:16:54 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 28 Apr 2012 19:16:54 +0200 Subject: [Dovecot] Session IDs In-Reply-To: <3AFABAEC-9FB7-449D-9473-64714C2038A7@iki.fi> References: <3AFABAEC-9FB7-449D-9473-64714C2038A7@iki.fi> Message-ID: <20120428171654.GA19481@daniel.localdomain> Timo Sirainen wrote: > v2.1.2 added support for "session IDs", which is a 9 year unique identifier > for that specific IMAP/POP3 connection. Initially I thought it would only be > useful for tracking connections going through Dovecot proxies to backends > (the session ID is forwarded) Sounds like an interesting debugging feature. > Apr 28 17:38:27 imap-login: Info: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=14338, secured, session= > The last line requires a modified mail_log_prefix, I wasn't planning on changing its default. What does your mail_log_prefix look like? Can you document the needed variable at http://wiki2.dovecot.org/Variables please? > 1) Does this change break someone's auth log line parsing? We don't do any parsing, just sending logs to a central syslog server for debugging purposes. > 2) Does the auth prefix look prettier with or without <> around session id? :) It should be separated using a fixed name like session=... for easy parsing. Regards, Daniel From jeep at rahul.net Sat Apr 28 23:54:45 2012 From: jeep at rahul.net (Jeff Lacki) Date: Sat, 28 Apr 2012 13:54:45 -0700 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query Message-ID: <20120428205446.0C08D1298B0@aqua.rahul.net> Hello- Sorry if this is a noob question, but I cant seem to get my password_query to work with dovecot + mysql. Im using 'Password verification by SQL server' at: http://wiki2.dovecot.org/AuthDatabase/SQL trying to modify it to work with my encrypted passwords in the DB. Im using the following which isnt working: password_query = SELECT NULL AS password, \ 'Y' as nopassword, userid AS user \ FROM users WHERE userid='%u' AND AES_DECRYPT(password, 'mykey')=password 1. Is it even possible to do this via 'password_query'? 2. If so, what am I doing wrong? Thanks, Jeff /mf/home/jeep/shell/.signature From gedalya at gedalya.net Sun Apr 29 00:16:59 2012 From: gedalya at gedalya.net (Gedalya) Date: Sat, 28 Apr 2012 17:16:59 -0400 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <20120428205446.0C08D1298B0@aqua.rahul.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> Message-ID: <4F9C5E4B.5080704@gedalya.net> On 04/28/2012 04:54 PM, Jeff Lacki wrote: > Hello- > > Sorry if this is a noob question, but I cant seem to get my password_query to > work with dovecot + mysql. Im using 'Password verification by SQL server' at: > > http://wiki2.dovecot.org/AuthDatabase/SQL > > trying to modify it to work with my encrypted passwords in the DB. > > Im using the following which isnt working: > > password_query = SELECT NULL AS password, \ > 'Y' as nopassword, userid AS user \ > FROM users WHERE userid='%u' AND AES_DECRYPT(password, 'mykey')=password > > 1. Is it even possible to do this via 'password_query'? > > 2. If so, what am I doing wrong? > > Thanks, > Jeff > > /mf/home/jeep/shell/.signature The wiki says: "The password is in %w variable" Maybe you meant password_query = SELECT NULL AS password, \ 'Y' as nopassword, userid AS user \ FROM users WHERE userid='%u' AND AES_DECRYPT(password, 'mykey')='%w' From jeep at rahul.net Sun Apr 29 00:25:36 2012 From: jeep at rahul.net (Jeff Lacki) Date: Sat, 28 Apr 2012 14:25:36 -0700 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <4F9C5E4B.5080704@gedalya.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <4F9C5E4B.5080704@gedalya.net> Message-ID: <20120428212537.20EDF12954C@aqua.rahul.net> > > > > /mf/home/jeep/shell/.signature > The wiki says: "The password is in %w variable" > Maybe you meant > > password_query = SELECT NULL AS password, \ > 'Y' as nopassword, userid AS user \ > FROM users WHERE userid='%u' AND AES_DECRYPT(password, 'mykey')='%w' > > Thanks, but when I do that, I get an empty value for '%w', not sure why? Apr 28 14:23:48 mydomain dovecot: auth-worker(13349): Debug: sql(jeff,127.0.0.1): query: SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users WHERE userid='jeff' AND AES_DECRYPT(password, 'mykey')='' /mf/home/jeep/shell/.signature From daniel.parthey at informatik.tu-chemnitz.de Sun Apr 29 01:11:29 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 29 Apr 2012 00:11:29 +0200 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <20120428205446.0C08D1298B0@aqua.rahul.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> Message-ID: <20120428221129.GA24819@daniel.localdomain> Hi Jeff, Jeff Lacki wrote: > Sorry if this is a noob question, but I cant seem to get my password_query to > work with dovecot + mysql. Im using 'Password verification by SQL server' at: > > http://wiki2.dovecot.org/AuthDatabase/SQL > > trying to modify it to work with my encrypted passwords in the DB. > > Im using the following which isn't working: > > password_query = SELECT NULL AS password, \ > 'Y' as nopassword, userid AS user \ > FROM users WHERE userid='%u' AND AES_DECRYPT(password, 'mykey')=password > > 1. Is it even possible to do this via 'password_query'? Please provide your dovecot version and output of the following command: doveconf -n and the complete external sql query files without passwords. You might alsolet the SQL server compare the encrypted password in the database with the encrypted string: password_query = SELECT NULL AS password, \ 'Y' as nopassword, userid AS user \ FROM users WHERE userid='%u' AND AES_ENCRYPT('%w','mykey')=password Regards, Daniel From gedalya at gedalya.net Sun Apr 29 01:25:18 2012 From: gedalya at gedalya.net (Gedalya) Date: Sat, 28 Apr 2012 18:25:18 -0400 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <20120428212537.20EDF12954C@aqua.rahul.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <4F9C5E4B.5080704@gedalya.net> <20120428212537.20EDF12954C@aqua.rahul.net> Message-ID: <4F9C6E4E.6030608@gedalya.net> On 04/28/2012 05:25 PM, Jeff Lacki wrote: >>> /mf/home/jeep/shell/.signature >> The wiki says: "The password is in %w variable" >> Maybe you meant >> >> password_query = SELECT NULL AS password, \ >> 'Y' as nopassword, userid AS user \ >> FROM users WHERE userid='%u' AND AES_DECRYPT(password, 'mykey')='%w' >> >> > Thanks, but when I do that, I get an empty value for '%w', not sure why? > > Apr 28 14:23:48 mydomain dovecot: auth-worker(13349): Debug: sql(jeff,127.0.0.1): query: SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users WHERE userid='jeff' AND AES_DECRYPT(password, 'mykey')='' > > > /mf/home/jeep/shell/.signature Are you using a plaintext authentication mechanism? From jeep at rahul.net Sun Apr 29 01:28:43 2012 From: jeep at rahul.net (Jeff Lacki) Date: Sat, 28 Apr 2012 15:28:43 -0700 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <20120428221129.GA24819@daniel.localdomain> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <20120428221129.GA24819@daniel.localdomain> Message-ID: <20120428222843.C0F44129904@aqua.rahul.net> > > > > 1. Is it even possible to do this via 'password_query'? > > Please provide your dovecot version and output of the following command: > doveconf -n > and the complete external sql query files without passwords. > > You might alsolet the SQL server compare the encrypted > password in the database with the encrypted string: > > password_query = SELECT NULL AS password, \ > 'Y' as nopassword, userid AS user \ > FROM users WHERE userid='%u' AND AES_ENCRYPT('%w','mykey')=password > > Regards, > Daniel Thank you Daniel. I downloaded and compiled 2.1.5 yesterday. The problem seems to be that '%w' evaulates to an empty string: Debug: sql(jeff,127.0.0.1): query: SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users WHERE userid='jeff' AND AES_DECRYPT('', 'key')=password I also just noticed that version 2.0.15 in my output below is coming from somewhere? I did try setting things up under 2.0.15 initially last week, but wanted to be up to date so downloaded the latest yesterday. I never did get it all working under 2.0.15 either btw. dovecot -n -c /opt/dovecot/etc/dovecot/dovecot.conf # 2.0.15: /opt/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.35.14-106.fc14.x86_64 x86_64 Fedora release 14 (Laughlin) ext4 auth_debug = yes auth_debug_passwords = yes auth_mechanisms = cram-md5 auth_verbose = yes auth_verbose_passwords = plain default_client_limit = 225 first_valid_uid = 1000 listen = * lock_method = flock mail_location = mbox:/var/mail/%d/%1n/%n:INDEX=/var/indexes/%d/%1n/%n mail_privileged_group = mail mbox_lock_timeout = 1 mins mbox_write_locks = fcntl namespace { inbox = yes location = prefix = separator = . type = private } passdb { args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } protocols = imap service auth { inet_listener { port = 12345 } unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } user = $default_internal_user } service imap-login { inet_listener imap { port = 143 } service_count = 1 } ssl_cert = References: <20120428205446.0C08D1298B0@aqua.rahul.net> <20120428221129.GA24819@daniel.localdomain> <20120428222843.C0F44129904@aqua.rahul.net> Message-ID: <4F9C7160.6030904@gedalya.net> On 04/28/2012 06:28 PM, Jeff Lacki wrote: >>> 1. Is it even possible to do this via 'password_query'? >> Please provide your dovecot version and output of the following command: >> doveconf -n >> and the complete external sql query files without passwords. >> >> You might alsolet the SQL server compare the encrypted >> password in the database with the encrypted string: >> >> password_query = SELECT NULL AS password, \ >> 'Y' as nopassword, userid AS user \ >> FROM users WHERE userid='%u' AND AES_ENCRYPT('%w','mykey')=password >> >> Regards, >> Daniel > Thank you Daniel. I downloaded and compiled 2.1.5 yesterday. > The problem seems to be that '%w' evaulates to an empty string: > > Debug: sql(jeff,127.0.0.1): query: SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users WHERE userid='jeff' AND AES_DECRYPT('', 'key')=password > > I also just noticed that version 2.0.15 in my output below is coming from > somewhere? I did try setting things up under 2.0.15 initially last week, > but wanted to be up to date so downloaded the latest yesterday. I never did > get it all working under 2.0.15 either btw. > > dovecot -n -c /opt/dovecot/etc/dovecot/dovecot.conf > # 2.0.15: /opt/dovecot/etc/dovecot/dovecot.conf > # OS: Linux 2.6.35.14-106.fc14.x86_64 x86_64 Fedora release 14 (Laughlin) ext4 > auth_debug = yes > auth_debug_passwords = yes > auth_mechanisms = cram-md5 > auth_verbose = yes > auth_verbose_passwords = plain > default_client_limit = 225 > first_valid_uid = 1000 > listen = * > lock_method = flock > mail_location = mbox:/var/mail/%d/%1n/%n:INDEX=/var/indexes/%d/%1n/%n > mail_privileged_group = mail > mbox_lock_timeout = 1 mins > mbox_write_locks = fcntl > namespace { > inbox = yes > location = > prefix = > separator = . > type = private > } > passdb { > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > driver = sql > } > protocols = imap > service auth { > inet_listener { > port = 12345 > } > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0666 > user = postfix > } > user = $default_internal_user > } > service imap-login { > inet_listener imap { > port = 143 > } > service_count = 1 > } > ssl_cert = ssl_key = userdb { > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > driver = sql > } > userdb { > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > driver = sql > } > userdb { > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > driver = sql > } > protocol imap { > imap_idle_notify_interval = 1 mins > imap_max_line_length = 64 k > mail_max_userip_connections = 5 > } > > > /mf/home/jeep/shell/.signature Yeap, you seem to only allow cram-md5. In this case, you client isn't transmitting the actual password that the user is typing, so dovecot simply doesn't have the password you want it to put in %w. It rather has a digest of it. The only way to use a non-plaintext auth mechanism is to provide dovecot the correct password from the database in plaintext. http://wiki2.dovecot.org/Authentication/Mechanisms From jeep at rahul.net Sun Apr 29 02:02:42 2012 From: jeep at rahul.net (Jeff Lacki) Date: Sat, 28 Apr 2012 16:02:42 -0700 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <4F9C7160.6030904@gedalya.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <20120428221129.GA24819@daniel.localdomain> <20120428222843.C0F44129904@aqua.rahul.net> <4F9C7160.6030904@gedalya.net> Message-ID: <20120428230243.BCF50129947@aqua.rahul.net> Gedalya wrote: > On 04/28/2012 06:28 PM, Jeff Lacki wrote: > >>> 1. Is it even possible to do this via 'password_query'? > >> Please provide your dovecot version and output of the following command: > >> doveconf -n > >> and the complete external sql query files without passwords. > >> > >> You might alsolet the SQL server compare the encrypted > >> password in the database with the encrypted string: > >> > >> password_query = SELECT NULL AS password, \ > >> 'Y' as nopassword, userid AS user \ > >> FROM users WHERE userid='%u' AND AES_ENCRYPT('%w','mykey')=password > >> > >> Regards, > >> Daniel > > Thank you Daniel. I downloaded and compiled 2.1.5 yesterday. > > The problem seems to be that '%w' evaulates to an empty string: > > > > Debug: sql(jeff,127.0.0.1): query: SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users WHERE userid='jeff' AND AES_DECRYPT('', 'key')=password > > > > I also just noticed that version 2.0.15 in my output below is coming from > > somewhere? I did try setting things up under 2.0.15 initially last week, > > but wanted to be up to date so downloaded the latest yesterday. I never did > > get it all working under 2.0.15 either btw. > > > > dovecot -n -c /opt/dovecot/etc/dovecot/dovecot.conf > > # 2.0.15: /opt/dovecot/etc/dovecot/dovecot.conf > > # OS: Linux 2.6.35.14-106.fc14.x86_64 x86_64 Fedora release 14 (Laughlin) ext4 > > auth_debug = yes > > auth_debug_passwords = yes > > auth_mechanisms = cram-md5 > > auth_verbose = yes > > auth_verbose_passwords = plain > > default_client_limit = 225 > > first_valid_uid = 1000 > > listen = * > > lock_method = flock > > mail_location = mbox:/var/mail/%d/%1n/%n:INDEX=/var/indexes/%d/%1n/%n > > mail_privileged_group = mail > > mbox_lock_timeout = 1 mins > > mbox_write_locks = fcntl > > namespace { > > inbox = yes > > location = > > prefix = > > separator = . > > type = private > > } > > passdb { > > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > > driver = sql > > } > > protocols = imap > > service auth { > > inet_listener { > > port = 12345 > > } > > unix_listener /var/spool/postfix/private/auth { > > group = postfix > > mode = 0666 > > user = postfix > > } > > user = $default_internal_user > > } > > service imap-login { > > inet_listener imap { > > port = 143 > > } > > service_count = 1 > > } > > ssl_cert = > ssl_key = > userdb { > > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > > driver = sql > > } > > userdb { > > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > > driver = sql > > } > > userdb { > > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > > driver = sql > > } > > protocol imap { > > imap_idle_notify_interval = 1 mins > > imap_max_line_length = 64 k > > mail_max_userip_connections = 5 > > } > > > > > > /mf/home/jeep/shell/.signature > > Yeap, you seem to only allow cram-md5. In this case, you client isn't > transmitting the actual password that the user is typing, so dovecot > simply doesn't have the password you want it to put in %w. It rather has > a digest of it. > The only way to use a non-plaintext auth mechanism is to provide dovecot > the correct password from the database in plaintext. > > http://wiki2.dovecot.org/Authentication/Mechanisms Ok thank you both for your help. I see why I didnt get anything for %w now. After reading the docs the past few days Im a little unsure of what setup I should use then for security purposes (primarily potential sniffing). Obviously using DIGEST-MD5 (per the docs) is better than CRAM-MD5, but is that possible to use here if I am trying to match passwords out of my mysql DB? I certainly do not want to use plaintext (and quite honestly Im not sure what 'login' really means either, but thats another issue). My desire is: - Only virtual users - All users info stored in mysql - login is via squirrelmail ATM, and later imap/iphone or other. Security is my #1 focus right now. Can someone explain the best solution? Or is the best solution to just get an SSL cert and use plaintext? (which is actually my future plan). Thanks! Jeff /mf/home/jeep/shell/.signature From gedalya at gedalya.net Sun Apr 29 02:31:07 2012 From: gedalya at gedalya.net (Gedalya) Date: Sat, 28 Apr 2012 19:31:07 -0400 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <20120428230243.BCF50129947@aqua.rahul.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <20120428221129.GA24819@daniel.localdomain> <20120428222843.C0F44129904@aqua.rahul.net> <4F9C7160.6030904@gedalya.net> <20120428230243.BCF50129947@aqua.rahul.net> Message-ID: <4F9C7DBB.2000604@gedalya.net> On 04/28/2012 07:02 PM, Jeff Lacki wrote: > Security is my #1 focus right now. > > Can someone explain the best solution? Or is the best solution to just get > an SSL cert and use plaintext? (which is actually my future plan). You absolutely must use SSL if you want security. A non-plaintext authentication mechanism only obfuscates the password itself during the login stage. The IMAP session itself (email content) needs to be secured and that can be more important than the email password (people emailing to each other passwords to more interesting things). Getting your certificate signed by a recognized CA helps your clients to verify that the server they are talking to is the server they want to be talking to. It doesn't make the encryption any stronger. If your clients are willing to click "I know what I'm doing, I trust this certificate", then you have the same results. You can try to get a free certificate here - http://www.startssl.com/ - their certificates are trusted by Mozilla and Microsoft products but not by RIM (blackberry) or java. Anyway, given your current setup: you're not using SSL, you want to AES-encrypt your passwords in mysql (you don't trust your database server) and keep your encryption key in the dovecot configuration (you do trust your dovecot server), you can just do: password_query = SELECT AES_DECRYPT(password, 'mykey') AS password, \ userid AS user \ FROM users WHERE userid='%u' This would allow you to use a digest-based authentication mechanism. However, you still have the liability of having your users' passwords in a reversibly encrypted format, with the key available nearby. Once you get SSL set up, it would be better to store the passwords in a salted hash format such as SSHA, and use plaintext auth (over SSL, of course). From jeep at rahul.net Sun Apr 29 05:29:49 2012 From: jeep at rahul.net (Jeff Lacki) Date: Sat, 28 Apr 2012 19:29:49 -0700 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <4F9C7DBB.2000604@gedalya.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <20120428221129.GA24819@daniel.localdomain> <20120428222843.C0F44129904@aqua.rahul.net> <4F9C7160.6030904@gedalya.net> <20120428230243.BCF50129947@aqua.rahul.net> <4F9C7DBB.2000604@gedalya.net> Message-ID: <20120429022949.9559D129606@aqua.rahul.net> > salted hash format such as SSHA, and use plaintext auth (over SSL, of > course). Thank you so much for your in depth reponse Gedalya, I appreciate it! So now that I changed things up and my conf is: auth_mechanisms = plain login disable_plaintext_auth = no I get the following, which I dont get, because now it seems to be telling me that my decrypted AES password needs to be MD5-CRYPT or PLAIN-MD5? Debug: sql(jeff,127.0.0.1): query: SELECT AES_DECRYPT(password, 'mykey') AS password, userid AS user FROM users WHERE userid='jeff' AND AES_DECRYPT(password, 'mykey')='mypass' Error: sql(jeff,127.0.0.1): Invalid password 'mypass' in passdb: Not a valid MD5-CRYPT or PLAIN-MD5 password Debug: sql(jeff,127.0.0.1): MD5(mypass) != 'mypass', try PLAIN scheme instead Apr 28 19:18:56 mydomain dovecot: auth: Debug: client out: FAIL#0111#011user=jeff This is even more confusing as my mechanisms say plain, not any type of MD5? Thanks again! /mf/home/jeep/shell/.signature From gedalya at gedalya.net Sun Apr 29 05:53:20 2012 From: gedalya at gedalya.net (Gedalya) Date: Sat, 28 Apr 2012 22:53:20 -0400 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <20120429022949.9559D129606@aqua.rahul.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <20120428221129.GA24819@daniel.localdomain> <20120428222843.C0F44129904@aqua.rahul.net> <4F9C7160.6030904@gedalya.net> <20120428230243.BCF50129947@aqua.rahul.net> <4F9C7DBB.2000604@gedalya.net> <20120429022949.9559D129606@aqua.rahul.net> Message-ID: <4F9CAD20.9020706@gedalya.net> On 4/28/2012 10:29 PM, Jeff Lacki wrote: > Error: sql(jeff,127.0.0.1): Invalid password 'mypass' in passdb: Not a valid MD5-CRYPT or PLAIN-MD5 password > Debug: sql(jeff,127.0.0.1): MD5(mypass) != 'mypass', try PLAIN scheme instead In dovecot-sql.conf.ext, set: default_pass_scheme = PLAIN Dovecot needs to be told what format the stored password is in. From ml at well-adjusted.de Sun Apr 29 14:04:51 2012 From: ml at well-adjusted.de (Jochen Spieker) Date: Sun, 29 Apr 2012 13:04:51 +0200 Subject: [Dovecot] Sieve doesn't find user scripts Message-ID: <20120429110451.GF12090@wasteland.homelinux.net> Hi, I want to use Sieve filtering with my Dovecot 1.2 installation on Debian squeeze. I have a virtual domain setup using Portgresql. ManageSieve works fine so far, I can edit and activate/deactive scripts (using Thunderbird + Plugin) and they show up in the filesystem where I expect them to be, see below. The problem is that LDA doesn't find the script. From /var/log/dovecot-deliver.log: | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Loading modules from directory: /usr/lib/dovecot/modules/lda | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: uid=105 | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: gid=109 | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Quota root: name=storage=512000 backend=maildir args= | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: maildir: data=/srv/mail/well-adjusted.de/jrspieker | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: maildir++: root=/srv/mail/well-adjusted.de/jrspieker, index=, control=, inbox= | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: sieve: user has no valid personal script | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: sieve: no scripts to execute: reverting to default delivery. AFAICS, the sieve module is loaded and the home directory is correct. The content of this directory looks like this: | $ ls -ltRa /srv/mail/home/well-adjusted.de/jrspieker/ | /srv/mail/home/well-adjusted.de/jrspieker/: | total 12 | drwx------ 3 vmail vmail 4096 Apr 29 12:22 . | lrwxrwxrwx 1 vmail vmail 16 Apr 29 12:22 .dovecot.sieve -> sieve/test.sieve | drwx------ 3 vmail vmail 4096 Apr 28 22:31 sieve | drwx------ 3 vmail vmail 4096 Apr 27 20:55 .. | | /srv/mail/home/well-adjusted.de/jrspieker/sieve: | total 16 | drwx------ 3 vmail vmail 4096 Apr 29 12:22 .. | drwx------ 3 vmail vmail 4096 Apr 28 22:31 . | drwx------ 2 vmail vmail 4096 Apr 28 22:31 tmp | -rw------- 1 vmail vmail 338 Apr 28 22:31 test.sieve | ? dovecot -n shows this for the plugin section: | plugin: | quota: maildir:storage=512000 | sieve_dir: ~/sieve/ | sieve: ~/.dovecot.sieve And finally, the content of the script: | require["fileinto"]; | | if header :is "subject" "test" { | fileinto "Test"; | } else { | keep; | } What am I missing here? Thanks, Jochen. -- Fashion is more important to me than war, famine, disease or art. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From giles at coochey.net Sun Apr 29 14:47:04 2012 From: giles at coochey.net (Giles Coochey) Date: Sun, 29 Apr 2012 12:47:04 +0100 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <20120429110451.GF12090@wasteland.homelinux.net> References: <20120429110451.GF12090@wasteland.homelinux.net> Message-ID: <4F9D2A38.1010400@coochey.net> On 29/04/2012 12:04, Jochen Spieker wrote: > Hi, > > I want to use Sieve filtering with my Dovecot 1.2 installation on Debian > squeeze. I have a virtual domain setup using Portgresql. > > ManageSieve works fine so far, I can edit and activate/deactive scripts (using > Thunderbird + Plugin) and they show up in the filesystem where I expect them to > be, see below. > > The problem is that LDA doesn't find the script. From > /var/log/dovecot-deliver.log: > > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Loading modules from directory: /usr/lib/dovecot/modules/lda > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: uid=105 > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: gid=109 > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Quota root: name=storage=512000 backend=maildir args= > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: maildir: data=/srv/mail/well-adjusted.de/jrspieker > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: maildir++: root=/srv/mail/well-adjusted.de/jrspieker, index=, control=, inbox= > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: sieve: user has no valid personal script > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: sieve: no scripts to execute: reverting to default delivery. > > AFAICS, the sieve module is loaded and the home directory is correct. The > content of this directory looks like this: > > | $ ls -ltRa /srv/mail/home/well-adjusted.de/jrspieker/ > | /srv/mail/home/well-adjusted.de/jrspieker/: > | total 12 > | drwx------ 3 vmail vmail 4096 Apr 29 12:22 . > | lrwxrwxrwx 1 vmail vmail 16 Apr 29 12:22 .dovecot.sieve -> sieve/test.sieve > | drwx------ 3 vmail vmail 4096 Apr 28 22:31 sieve > | drwx------ 3 vmail vmail 4096 Apr 27 20:55 .. > | > | /srv/mail/home/well-adjusted.de/jrspieker/sieve: > | total 16 > | drwx------ 3 vmail vmail 4096 Apr 29 12:22 .. > | drwx------ 3 vmail vmail 4096 Apr 28 22:31 . > | drwx------ 2 vmail vmail 4096 Apr 28 22:31 tmp > | -rw------- 1 vmail vmail 338 Apr 28 22:31 test.sieve > | ? > What user is deliver running as? -- Best Regards, Giles Coochey, CCNA Security, CCNA NetSecSpec Ltd giles.coochey at netsecspec.co.uk Tel: +44 (0) 7983 877 438 Live Messenger: giles at coochey.net http://www.netsecspec.co.uk http://www.coochey.net -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4928 bytes Desc: S/MIME Cryptographic Signature URL: From ml at well-adjusted.de Sun Apr 29 16:15:08 2012 From: ml at well-adjusted.de (Jochen Spieker) Date: Sun, 29 Apr 2012 15:15:08 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <4F9D2A38.1010400@coochey.net> References: <20120429110451.GF12090@wasteland.homelinux.net> <4F9D2A38.1010400@coochey.net> Message-ID: <20120429131508.GG12090@wasteland.homelinux.net> Giles Coochey: > On 29/04/2012 12:04, Jochen Spieker wrote: >> >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Loading modules from directory: /usr/lib/dovecot/modules/lda >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: uid=105 >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: gid=109 These IDs translate to user/group vmail: | $ grep 105 /etc/passwd | vmail:x:105:109::/home/vmail:/bin/false | | $ grep 109 /etc/group | vmail:x:109: > What user is deliver running as? Should be vmail: /etc/postfix/master.cf: | dovecot unix - n n - 20 pipe | flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} All mailboxes and home directories belong to this user and group. J. -- If nightclub doormen recognised me I would be more fulfilled. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From ml at well-adjusted.de Sun Apr 29 16:44:12 2012 From: ml at well-adjusted.de (Jochen Spieker) Date: Sun, 29 Apr 2012 15:44:12 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <20120429110451.GF12090@wasteland.homelinux.net> References: <20120429110451.GF12090@wasteland.homelinux.net> Message-ID: <20120429134411.GH12090@wasteland.homelinux.net> Jochen Spieker: > > The problem is that LDA doesn't find the script. Ok, using a global script works: deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: auth input: uid=105 deliver(jrspieker at well-adjusted.de): Info: auth input: gid=109 deliver(jrspieker at well-adjusted.de): Info: Quota root: name=storage=512000 backend=maildir args= deliver(jrspieker at well-adjusted.de): Info: maildir: data=/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: maildir++: root=/srv/mail/well-adjusted.de/jrspieker, index=, control=, inbox=/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: sieve: using sieve path for user's script: /srv/mail/home/global/sieve/global.sieve deliver(jrspieker at well-adjusted.de): Info: sieve: opening script /srv/mail/home/global/sieve/global.sieve deliver(jrspieker at well-adjusted.de): Info: sieve: executing compiled script /srv/mail/home/global/sieve/global.sieve deliver(jrspieker at well-adjusted.de): Info: Namespace : Using permissions from /srv/mail/well-adjusted.de/jrspieker: mode=0755 gid=-1 deliver(jrspieker at well-adjusted.de): Info: sieve: msgid=<20120429132328.D581B1E9 at manowar.wasteland.homelinux.net>: stored mail into mailbox 'INBOX' I suspect LDA doesn't look for the script where I expect it to look. Again, dovecot -n show this (now including the global settings): plugin: quota: maildir:storage=512000 sieve_dir: ~/sieve/ sieve: ~/.dovecot.sieve sieve_global_path: /srv/mail/home/global/sieve/global.sieve sieve_global_dir: /srv/mail/home/global/sieve/ I would expect ~ to translate to the userdb_home from the log entry above (/srv/mail/home/well-adjusted.de/jrspieker). ManageSieve's behaviour confirms that expectation. The wiki says "A relative path (or just a filename) will be interpreted to point under the user's home directory."? This works for ManageSieve just like the ~-notation, but LDA still doesn't find the script. J. ? http://wiki.dovecot.org/LDA/Sieve/Dovecot#Per-user_Sieve_script_location -- When I get home from the supermarket I don't know what to do with all the plastic. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From ml at well-adjusted.de Sun Apr 29 16:59:46 2012 From: ml at well-adjusted.de (Jochen Spieker) Date: Sun, 29 Apr 2012 15:59:46 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <20120429134411.GH12090@wasteland.homelinux.net> References: <20120429110451.GF12090@wasteland.homelinux.net> <20120429134411.GH12090@wasteland.homelinux.net> Message-ID: <20120429135945.GI12090@wasteland.homelinux.net> Someone asked for the complete dovecot -n: # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.34-xen-r4 i686 Debian 6.0.4 rootfs log_timestamp: %Y-%m-%d %H:%M:%S protocols: pop3s imaps managesieve ssl_cert_file: /etc/ssl/certs/mail.well-adjusted.de.cert ssl_key_file: /etc/ssl/private/mail.well-adjusted.de.key verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login login_processes_count: 1 login_max_processes_count: 32 first_valid_uid: 1 mail_privileged_group: vmail mail_location: maildir:/srv/mail/%Ld/%Ln fsync_disable: yes mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve lda: fsync_disable: no postmaster_address: postmaster at wasteland.xen-host.de log_path: /var/log/dovecot-deliver.log info_log_path: /var/log/dovecot-deliver.log mail_plugins: quota sieve auth_socket_path: /var/run/dovecot/auth-master mail_debug: yes auth default: mechanisms: plain login user: postgres username_chars: +*abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ username_format: %Lu passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: prefetch userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix master: path: /var/run/dovecot/auth-master mode: 432 user: vmail group: vmail plugin: quota: maildir:storage=512000 sieve_dir: sieve/ sieve: .dovecot.sieve sieve_global_path: /srv/mail/home/global/sieve/global.sieve sieve_global_dir: /srv/mail/home/global/sieve/ J. -- Tony Blair is a hypnotised self-seeking scarecrow just like all the rest. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From daniel.parthey at informatik.tu-chemnitz.de Mon Apr 30 01:49:28 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Mon, 30 Apr 2012 00:49:28 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <20120429135945.GI12090@wasteland.homelinux.net> References: <20120429110451.GF12090@wasteland.homelinux.net> <20120429134411.GH12090@wasteland.homelinux.net> <20120429135945.GI12090@wasteland.homelinux.net> Message-ID: <20120429224927.GA9001@daniel.localdomain> Hi Jochen, Jochen Spieker wrote: > mail_location: maildir:/srv/mail/%Ld/%Ln > plugin: > sieve_dir: sieve/ > sieve: .dovecot.sieve Sieve requires a mail_home directory. The sieve_dir is searched below mail_home, not mail_location where the actual mailboxes are stored. http://wiki.dovecot.org/VirtualUsers/Home Home vs. mail directory Regards, Daniel From snabb at epipe.com Sun Apr 29 21:09:43 2012 From: snabb at epipe.com (Janne Snabb) Date: Sun, 29 Apr 2012 18:09:43 +0000 (UTC) Subject: [Dovecot] setacl fails - does not find dovecot-acl file In-Reply-To: <1320442758.21919.222.camel@hurina> References: <4EA2C213.7030204@mike2k.de> <1320436059.21919.152.camel@hurina> <4EB44B1C.90505@mike2k.de> <1320442758.21919.222.camel@hurina> Message-ID: On Fri, 4 Nov 2011, Timo Sirainen wrote: > On Fri, 2011-11-04 at 21:29 +0100, Michael Stilkerich wrote: > > > Nov 4 16:29:03 keira dovecot: imap(isa): Error: fcntl(unlock) locking > > failed for file /home/dovecot/isa/dovecot.index.log: No such file or > > directory > > Nov 4 16:29:03 keira dovecot: imap(isa): Error: fstat() failed with > > file /home/dovecot/isa/dovecot.index.log: No such file or directory > > These simply shouldn't happen. I'd say it's a kernel bug. You're running > a default Ubuntu kernel? I wonder if other Ubuntu users have this > problem. I am seeing this same problem on Ubuntu 11.10 and 12.04 with stock kernels. The problem is clearly AppArmor related. The imap process seems to be using the "usr.sbin.dovecot" profile which prevents access to these files. There is a separate profile "usr.lib.dovecot.imap" but it seems that it does not get applied to the imap process for some odd reason. This is especially strange beacuse both profiles are enabled in "complain" and not in "enforce" mode, thus they should not enforce any of the rules. I am simultaneously getting messages similar to the following in my audit log: type=AVC msg=audit(1335712674.515:655016): apparmor="ALLOWED" operation="getattr" parent=10922 profile="/usr/sbin/dovecot//null-107//null-10b//null-118" name="/home/foobar/Maildir/.foobar/dovecot.index.log" pid=10937 comm="imap" requested_mask="r" denied_mask="r" fsuid=1000 ouid=1000 After disabling the "usr.sbin.dovecot" profile everything seems fine. Other dovecot related AppArmor profiles do not seem to cause problems. This looks like an issue in AppArmor to me... -- Janne Snabb / EPIPE Communications snabb at epipe.com - http://epipe.com/ From sdavies at sdc.com.au Mon Apr 30 04:08:37 2012 From: sdavies at sdc.com.au (Stephen Davies) Date: Mon, 30 Apr 2012 10:38:37 +0930 Subject: [Dovecot] Log messages Message-ID: <201204301038.37694.sdavies@sdc.com.au> Hello. I have been on leave for several weeks and have managed to lose all emails received since December due to a server crash. I asked about log synch error back in March and have (repeatedly) deleted all .imap files but the errors continue. # 2.1.1: /usr/etc/dovecot/dovecot.conf # OS: Linux 2.6.33.5-desktop-2mnb i686 Mandriva Linux 2010.2 mail_access_groups = mail mail_location = mbox:~/Mail:INBOX=/var/spool/mail/%u managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve sieve ssl_cert = References: <20120429110451.GF12090@wasteland.homelinux.net> <20120429134411.GH12090@wasteland.homelinux.net> <20120429135945.GI12090@wasteland.homelinux.net> <20120429224927.GA9001@daniel.localdomain> Message-ID: <20120430053946.GK12090@wasteland.homelinux.net> Daniel Parthey: > Jochen Spieker wrote: > >> mail_location: maildir:/srv/mail/%Ld/%Ln >> plugin: >> sieve_dir: sieve/ >> sieve: .dovecot.sieve > > Sieve requires a mail_home directory. Is this different from userdb_home? Both mail and home directory are returned from my user/password_queries and LDA reports exactly what I expect: deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker ManageSieve puts scripts under the directory reported as userdb_home: $ ls -ltra /srv/mail/home/well-adjusted.de/jrspieker total 12 drwx------ 3 vmail vmail 4096 Apr 27 20:55 .. drwx------ 3 vmail vmail 4096 Apr 28 22:31 sieve lrwxrwxrwx 1 vmail vmail 16 Apr 29 15:00 .dovecot.sieve -> sieve/test.sieve drwx------ 3 vmail vmail 4096 Apr 29 15:00 . J. -- I feel yawning hollowness whilst talking to people at parties. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From oni-neko at gmx.net Mon Apr 30 11:59:32 2012 From: oni-neko at gmx.net (oni-neko at gmx.net) Date: Mon, 30 Apr 2012 10:59:32 +0200 Subject: [Dovecot] restoring mails and directories Message-ID: <20120430085932.267600@gmx.net> Good day! is there an easy way to restore e.g. deleted mails and/or mail directories? let me specify: I'm running dovecot 1.2.9 on a current ubuntu lts. It is set to Maildir and I backup the folder with the users maildata nightly via rsync. now, when someone deletes a mail that they still need, and I copy it back from the backup, it will not be visible to mail clients. When I change the mailfile by a letter or so, it shows. that's easy enough, if not exactly awesome ;-) But, if said person deletes a whole folder of mails, this way to make restored mails show up becomes rather unreasonable. surely there is a better way to do this? I'd be glad for any and all hints! =) best regards silvana -- Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de From reuben-dovecot at reub.net Mon Apr 30 13:43:10 2012 From: reuben-dovecot at reub.net (Reuben Farrelly) Date: Mon, 30 Apr 2012 20:43:10 +1000 Subject: [Dovecot] Dovecot dsync mail replication issues Message-ID: <4F9E6CBE.4070609@reub.net> Hi, I'm trying to set up some dsync based replication between two hosts on my network. The current topology is a single server running Postfix with a single dovecot installation with a Maildir per user (only 4 users including myself). No NFS, just local system users on ext4. I am only using system users, ie no virtual users. I am using dovecot deliver to deliver mails into the Maildir's. This - being a very standard Maildir installation - all works just fine. I'm running dovecot-2.1.5 (release versions) x86_64 with Gentoo on both hosts and using key based ssh to transport the data using the root login. What I would like to do is extend the design so as to replicate the Maildirs across a second machine and in the future be able to connect via IMAP into either machine to sync mail. Two way dsync would be rather cool because I could then deliver via SMTP to either system and have changes automagically propagate - but initially just simple replication would be a good start. I was hoping that dsync would allow me to do this, but I've run into quite a number of problems while getting this to work. http://dovecot.org/list/dovecot/2012-March/064243.html ...was very useful and I've based my config on that. Initially I've tried to sync up the user Maildirs, and this has more or less worked: doveadm sync -u lyn remote:root at dustbowl.reub.net This succeeds without error on the initial sync. However if I try to run the re-sync again (such as a use case of if the sync of another Maildir takes 4 hours so I want to resync up the earlier ones again) I end up with a mysteriously named INBOX folder in both the source and destination Maildirs: drwx------ 5 lyn lyn 4096 Apr 30 19:32 .INBOX_7a86a62d465a974fb92f00003b258734 It has the basic structure of a Maildir but is empty in terms of mails: drwx------ 2 lyn lyn 4096 Apr 30 19:32 cur -rw------- 1 lyn lyn 220 Apr 30 19:32 dovecot.index.log -rw------- 1 lyn lyn 51 Apr 30 19:32 dovecot-uidlist -rw------- 1 lyn lyn 0 Apr 30 19:32 maildirfolder drwx------ 2 lyn lyn 4096 Apr 30 19:32 new drwx------ 2 lyn lyn 4096 Apr 30 19:32 tmp First question: why is this random named directory being created in the origin Maildir? Shouldn't the replication be more or less read-only in the origin Maildir? Second question: If I re-attempt a doveadm sync a second time I get this error: tornado Maildir # doveadm sync -u lyn remote:root at dustbowl.reub.net dsync-local(lyn): Error: Can't rename mailbox INBOX_7a86a62d465a974fb92f00003b258734 to INBOX: Target mailbox already exists dsync-local(lyn): Error: Can't rename mailbox INBOX to INBOX_eb15f30ea563be4b703200002bd68bb1: Renaming INBOX isn't supported. tornado Maildir # It's not clear if the second attempt has failed or succeeded, and it's a bit odd that it errors out on a directory that the dovecot sync process itself has created. Third question: Upon starting Dovecot lots of ugliness is logged in the mail log: Apr 30 19:44:59 tornado dovecot: master: Dovecot v2.1.5 starting up (core dumps disabled) Apr 30 19:44:59 tornado dovecot: doveadm(mozsync): Error: user mozsync: Initialization failed: Namespace '': mkdir(/var/www/xxx/server-full/Maildir) failed: Permission denied (euid=1016(mozsync) egid=1016() missing +w perm: /var/www/xxx/server-full, dir owned by 0:0 mode=0755) Apr 30 19:44:59 tornado dovecot: doveadm(mozsync): Error: sync: User init failed Apr 30 19:44:59 tornado dovecot: doveadm(mozsync): Warning: I/O leak: 0x414190 (line 102, fd 16) Apr 30 19:44:59 tornado dovecot: dsync-local(cisco): Error: remote: doveadm(cisco): Fatal: User doesn't exist Apr 30 19:44:59 tornado dovecot: dsync-local(cisco): Error: read() from worker server failed: EOF Users mozsync and cisco are not valid mail users and it's not appropriate that Dovecot tries to create a Maildir for either of them. The users are system unprivileged users only, and do not ever send or receive mail. And - I/O leak? ;) Also, user cisco is local to one box only, and does not exist (nor does it need to) on the remote host. So any complaints about this user are invalid and dovecot needs to ignore replication for this user anyway. Fourth question, upon starting dovecot again, mail.log then spews several hundred of these messages: Apr 30 19:45:06 tornado dovecot: dsync-local(reuben): Error: msg-get failed: box=Trash uid=114863 guid=1335382569.M98089P29952.tornado,S=6479,W=6625 Before aborting entirely with: Apr 30 19:45:09 tornado dovecot: imap: Server shutting down. in=328 out=2042 It seems to me that a a few of those problems logged could be solved by being able to specify which system users to synchronise, rather than Dovecot making a blind assumption that all users actually have valid Maildirs that need to be created, and all need to be sync'd between two hosts. Subsequent delivery based sync'ing fails silently (pending more investigation) but I'd like to try and fix the above listed problems first before worrying about the later stages of the sync not working. Has anyone else run into these problems or found better ways of making this type of setup all work? Thanks, Reuben From stephan at rename-it.nl Mon Apr 30 16:06:07 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 30 Apr 2012 15:06:07 +0200 Subject: [Dovecot] Refiltering mails without loosing flags In-Reply-To: References: <4EE5E8FF.8080306@rename-it.nl> Message-ID: <4F9E8E3F.1010004@rename-it.nl> On 4/28/2012 6:07 PM, Dieter Knopf wrote: > 2011/12/12 Stephan Bosch: > >> Since last week, the new sieve-filter command line tool, which is intended >> for just such a task, is part of the normal compile output for Pigeonhole >> 0.3.x for Dovecot v2.1. Unfortunately, it is not released yet, so I guess >> you're stuck with your old getmail-based solution for now. >> >> The tool is still experimental and I must say that I have not tested the >> preservation of flags for some time and many changes were performed since >> then. I will verify that later today. Also, the interaction with the new >> editheader extension needs attention. > Is there any update about that? Would be really nice to have an nice > and easy way > like with procmail. Just via command line not via getmail/imap/.... I didn't receive much feedback so far, so you may very well encounter some bugs. But I did test flag preservation just now and that just works. Pigeonhole v0.3.0 is released and contains all you need. Just experiment with it a bit before you start using it for real mail. Regards, Stephan. From thierry at odry.net Mon Apr 30 17:07:33 2012 From: thierry at odry.net (Thierry de Montaudry) Date: Mon, 30 Apr 2012 16:07:33 +0200 Subject: [Dovecot] restoring mails and directories In-Reply-To: <20120430085932.267600@gmx.net> References: <20120430085932.267600@gmx.net> Message-ID: <98F4E6D0-A89B-408A-ACC1-7BC15FAC5231@odry.net> On 30 Apr 2012, at 10:59, oni-neko at gmx.net wrote: > Good day! > > is there an easy way to restore e.g. deleted mails and/or mail directories? > let me specify: I'm running dovecot 1.2.9 on a current ubuntu lts. It is set to Maildir and I backup the folder with the users maildata nightly via rsync. > > now, when someone deletes a mail that they still need, and I copy it back from the backup, it will not be visible to mail clients. When I change the mailfile by a letter or so, it shows. that's easy enough, if not exactly awesome ;-) > > But, if said person deletes a whole folder of mails, this way to make restored mails show up becomes rather unreasonable. > > surely there is a better way to do this? I'd be glad for any and all hints! =) > > best regards > silvana > -- > Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir > belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de Hi, Not sure about the availability on version 1.2.9, but the following command should do the trick: doveadm force-resync -u Regards, Thierry From stephan at rename-it.nl Mon Apr 30 17:12:03 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 30 Apr 2012 16:12:03 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <20120429110451.GF12090@wasteland.homelinux.net> References: <20120429110451.GF12090@wasteland.homelinux.net> Message-ID: <4F9E9DB3.1000707@rename-it.nl> On 4/29/2012 1:04 PM, Jochen Spieker wrote: > Hi, > > I want to use Sieve filtering with my Dovecot 1.2 installation on Debian > squeeze. I have a virtual domain setup using Portgresql. > > ManageSieve works fine so far, I can edit and activate/deactive scripts (using > Thunderbird + Plugin) and they show up in the filesystem where I expect them to > be, see below. > > The problem is that LDA doesn't find the script. From > /var/log/dovecot-deliver.log: > > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Loading modules from directory: /usr/lib/dovecot/modules/lda > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: uid=105 > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: gid=109 > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Quota root: name=storage=512000 backend=maildir args= > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: maildir: data=/srv/mail/well-adjusted.de/jrspieker > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: maildir++: root=/srv/mail/well-adjusted.de/jrspieker, index=, control=, inbox= > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: sieve: user has no valid personal script > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: sieve: no scripts to execute: reverting to default delivery. > > AFAICS, the sieve module is loaded and the home directory is correct. The > content of this directory looks like this: Could you enable mail_debug in your configuration? This will provide more detail on why it is not finding a script. Regards, Stephan. From ml at well-adjusted.de Mon Apr 30 17:43:58 2012 From: ml at well-adjusted.de (Jochen Spieker) Date: Mon, 30 Apr 2012 16:43:58 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <4F9E9DB3.1000707@rename-it.nl> References: <20120429110451.GF12090@wasteland.homelinux.net> <4F9E9DB3.1000707@rename-it.nl> Message-ID: <20120430144358.GL12090@wasteland.homelinux.net> Stephan Bosch: >> >> AFAICS, the sieve module is loaded and the home directory is correct. The >> content of this directory looks like this: > > Could you enable mail_debug in your configuration? This will provide > more detail on why it is not finding a script. With globally enabled mail_debug and auth_debug I get this in mail.log: postfix/qmgr[30005]: E306113C1E1: from=, size=1987, nrcpt=1 (queue active) dovecot: auth(default): master in: USER 1 jrspieker at well-adjusted.de service=deliver dovecot: auth(default): prefetch(jrspieker at well-adjusted.de): passdb didn't return userdb entries, trying the next userdb dovecot: auth(default): sql(jrspieker at well-adjusted.de): SELECT u.storage || ':' || d.basedir || u.userid as mail, '/srv/mail/home/well-adjusted.de/jrspieker' as userdb_home, 105 as uid, 109 as gid FROM mailuser u, domain d WHERE u.userid = 'jrspieker' AND u.domain = 'well-adjusted.de' AND u.domain = d.name dovecot: auth(default): master out: USER 1 jrspieker at well-adjusted.de mail=maildir:/srv/mail/well-adjusted.de/jrspieker userdb_home=/srv/mail/home/well-adjusted.de/jrspieker uid=105 gid=109 postfix/smtpd[5421]: disconnect from e177130134.adsl.alicedsl.de[85.177.130.134] postfix/pipe[5428]: E306113C1E1: to=, relay=dovecot, delay=1.1, delays=0.6/0/0/0.5, dsn=2.0.0, status=sent (delivered via dovecot service) postfix/qmgr[30005]: E306113C1E1: removed and this in dovecot-deliver.log: deliver(jrspieker at well-adjusted.de): Info: Loading modules from directory: /usr/lib/dovecot/modules/lda deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: auth input: uid=105 deliver(jrspieker at well-adjusted.de): Info: auth input: gid=109 deliver(jrspieker at well-adjusted.de): Info: Quota root: name=storage=512000 backend=maildir args= deliver(jrspieker at well-adjusted.de): Info: maildir: data=/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: maildir++: root=/srv/mail/well-adjusted.de/jrspieker, index=, control=, inbox=/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: sieve: user has no valid personal script deliver(jrspieker at well-adjusted.de): Info: sieve: no scripts to execute: reverting to default delivery. deliver(jrspieker at well-adjusted.de): Info: Namespace : Using permissions from /srv/mail/well-adjusted.de/jrspieker: mode=0755 gid=-1 deliver(jrspieker at well-adjusted.de): Info: msgid=<20120430142818.354B31FE at manowar.wasteland.homelinux.net>: saved mail to INBOX Doesn't look different than before, I had already enabled mail_debug in the "protocol lda" section. J. -- I am not scared of death but terrified of people in Tommy Hilfiger sweatshirts. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From tss at iki.fi Mon Apr 30 18:18:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 30 Apr 2012 18:18:03 +0300 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <20120430144358.GL12090@wasteland.homelinux.net> References: <20120429110451.GF12090@wasteland.homelinux.net> <4F9E9DB3.1000707@rename-it.nl> <20120430144358.GL12090@wasteland.homelinux.net> Message-ID: <1B683452-BBC7-4D97-ACCF-CC1A6568F1A3@iki.fi> On 30.4.2012, at 17.43, Jochen Spieker wrote: > deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker It should say "home" here, not "userdb_home". From ml at well-adjusted.de Mon Apr 30 18:40:52 2012 From: ml at well-adjusted.de (Jochen Spieker) Date: Mon, 30 Apr 2012 17:40:52 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <1B683452-BBC7-4D97-ACCF-CC1A6568F1A3@iki.fi> References: <20120429110451.GF12090@wasteland.homelinux.net> <4F9E9DB3.1000707@rename-it.nl> <20120430144358.GL12090@wasteland.homelinux.net> <1B683452-BBC7-4D97-ACCF-CC1A6568F1A3@iki.fi> Message-ID: <20120430154052.GM12090@wasteland.homelinux.net> Timo Sirainen: > On 30.4.2012, at 17.43, Jochen Spieker wrote: > >> deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker > > It should say "home" here, not "userdb_home". Thanks, that's it! dovecot-deliver.log now reads: deliver(jrspieker at well-adjusted.de): Info: auth input: home=/srv/mail/home/well-adjusted.de/jrspieker ? deliver(jrspieker at well-adjusted.de): Info: sieve: using sieve path for user's script: /srv/mail/home/well-adjusted.de/jrspieker/.dovecot.sieve deliver(jrspieker at well-adjusted.de): Info: sieve: opening script /srv/mail/home/well-adjusted.de/jrspieker/.dovecot.sieve deliver(jrspieker at well-adjusted.de): Info: sieve: executing compiled script /srv/mail/home/well-adjusted.de/jrspieker/.dovecot.sieve Now I can finally start to migrate my .procmailrc. J. -- Driving behind lorries carrying hazardous chemicals makes me wish for a simpler life. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From andrei at lctax.ro Mon Apr 30 19:25:43 2012 From: andrei at lctax.ro (Michescu Andrei) Date: Mon, 30 Apr 2012 12:25:43 -0400 Subject: [Dovecot] Dovecot dsync mail replication issues In-Reply-To: <4F9E6CBE.4070609@reub.net> References: <4F9E6CBE.4070609@reub.net> Message-ID: Hello Reuben, I'm having a very similar setup. The 2 main differences: all my users are virtual and the "2nd server" is on a different continent (high latency sync). Unfortunately the dsync is not working for the moment. Timo is in the process of redesigning it. So once it is release will know about it. > > drwx------ 5 lyn lyn 4096 Apr 30 19:32 > .INBOX_7a86a62d465a974fb92f00003b258734 > > First question: why is this random named directory being created in the > origin Maildir? Shouldn't the replication be more or less read-only in > the origin Maildir? - the number it is not random, but rather it is the GUID of the folder on the other server. To get rid of this annoying problem you need to clean your source of all these newly created folders, rsync your folders in between the 2 machines, run dsync again (this time it will not mess up with your folder structure) > Second question: If I re-attempt a doveadm sync a second time I get > this error: > > tornado Maildir # doveadm sync -u lyn remote:root at dustbowl.reub.net > dsync-local(lyn): Error: Can't rename mailbox > INBOX_7a86a62d465a974fb92f00003b258734 to INBOX: Target mailbox already > exists > dsync-local(lyn): Error: Can't rename mailbox INBOX to > INBOX_eb15f30ea563be4b703200002bd68bb1: Renaming INBOX isn't supported. > tornado Maildir # > > It's not clear if the second attempt has failed or succeeded, and it's a > bit odd that it errors out on a directory that the dovecot sync process > itself has created. > do the fix at Q1 and you will not run into this... it is not a permission problem but rather a meta-info problem. The setup will run fine as long as you only update 1 server and the other one is backup. The current release does not handle well the master-master model (you'll endup with emails like the folders above: duplicated, with GUID appended to them etc etc)... Wish Timo good luck and inspiration! Best regards, Andrei From robert at schetterer.org Mon Apr 30 21:55:11 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 30 Apr 2012 20:55:11 +0200 Subject: [Dovecot] question dovecot Inheritance global acl vs userfolder acl Message-ID: <4F9EE00F.5050308@schetterer.org> Hi Timo my tests resulted in inheritance is given if a userfolder has set some acl to its new created subfolder , which is nice if some userfolder has its acl from global acl there is no inheritance to its new created subfolders, that subfolders will always created with full owner rights i am not really sure if its a good idea to have inheritance from global acl and if its hackabel what is your idea to this ? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From dmiller at amfes.com Mon Apr 30 22:42:00 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 30 Apr 2012 12:42:00 -0700 Subject: [Dovecot] Single Instance Storage management Message-ID: Was there a doveadm command that checks the SIS files and removes any not referenced? Will the "doveadm purge -A" catch that? -- Daniel From dmiller at amfes.com Mon Apr 30 22:43:22 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 30 Apr 2012 12:43:22 -0700 Subject: [Dovecot] mdbox packing Message-ID: Is there a way to manually force an mdbox storage to be rebuilt into new files? Particularly files of the maximum size? -- Daniel From dmiller at amfes.com Mon Apr 30 23:16:01 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 30 Apr 2012 13:16:01 -0700 Subject: [Dovecot] Dovecot/doveadm crash Message-ID: Having a problem with a mailbox. I've been trying to rebuild - but doveadm force-resync crashes. This is mdbox with sis. doveadm purge -u dmiller at amfes.com doveadm(dmiller at amfes.com): Panic: file istream.c: line 466 (i_stream_grow_buffer): assertion failed: (stream->max_buffer_size > 0) doveadm(dmiller at amfes.com): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x4167a) [0x7fb73465467a] -> /usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7fb734654762] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fb73462c2a0] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4949b) [0x7fb73465c49b] -> /usr/local/lib/dovecot/libdovecot.so.0(i_stream_get_buffer_space+0x82) [0x7fb73465c522] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4b1c8) [0x7fb73465e1c8] -> /usr/local/lib/dovecot/libdovecot.so.0(i_stream_read+0x69) [0x7fb73465bb69] -> /usr/local/lib/dovecot/libdovecot.so.0(i_stream_read_next_line+0x1d) [0x7fb73465c04d] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mdbox_purge+0x921) [0x7fb7348d7d71] -> doveadm() [0x40e1e6] -> doveadm() [0x40e018] -> doveadm() [0x40ea83] -> doveadm(doveadm_mail_try_run+0x141) [0x40eef1] -> doveadm(main+0x3c1) [0x415911] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xff) [0x7fb73429deff] -> doveadm() [0x40de39] -- Daniel From marcin at mejor.pl Mon Apr 30 23:48:50 2012 From: marcin at mejor.pl (=?ISO-8859-2?Q?Marcin_Miros=B3aw?=) Date: Mon, 30 Apr 2012 22:48:50 +0200 Subject: [Dovecot] mdbox packing In-Reply-To: References: Message-ID: <4F9EFAB2.8040303@mejor.pl> W dniu 2012-04-30 21:43, Daniel L. Miller pisze: > Is there a way to manually force an mdbox storage to be rebuilt into new > files? Particularly files of the maximum size? > Hello, probably doveadm purge should do it. Regards, Marcin From tss at iki.fi Sun Apr 1 00:28:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 1 Apr 2012 00:28:51 +0300 Subject: [Dovecot] Dovecot stones Message-ID: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> For the last few days I've been thinking about my company and what it really should do, and as much as the current plan seems reasonable, I think in good conscience I really can't help but to bring up an alternative plan: The support for Dovecot is quite good already, but wouldn't it be better if it was GUARANTEED that your email server was ALWAYS working with zero downtime or other problems? After some thinking, planning and consulting with my spiritual guides I realized I really should just go ahead and do it: From now on you can buy blessed Dovecot stones (http://www.dovecot.org/tmp/stone1.jpg), which are guaranteed to keep your email server running. I don't think there's really any further reason for me to provide commercial support. Probably the whole Dovecot mailing list will be pretty much useless after this as well, so I might as well stop posting there. If you have any problems, you'll just buy a Dovecot stone and the problem will go away. If for any reason that doesn't work, it means that your problem is more difficult than most and you need to buy a few more stones. Please remember that each shipment may take up to 3 weeks to reach you, so get your stones early and in sufficient quantities. A small email server will be happy enough with a single small Dovecot stone, but the more users you have the more stones you'll likely need. This is because most people are rather unhappy with their lives, and they project their negative energies towards their surroundings, including their email servers. To overcome this negative energy you need to have a counterbalancing positive energy, which the Dovecot stones will provide you. The Dovecot stones are hand painted from the finest materials available to the sister of the author of Dovecot that she could find in the local shops. Each stone contains such love and dedication towards the well being of Dovecot that you will shed tears when you lay your eyes upon their physical being. The images provided here unfortunately don't do justice to them. http://www.dovecot.org/tmp/stone2.jpg - This is how i keep my home desktop running all day every day. I have NEVER broken a single hard drive in my life. I don't even bother to keep backups, because I KNOW it is useless for me. A small Dovecot stone can be bought for 10 euros, a medium sized stone for 100 euros and a large stone for 500 euros. Each large stone can usually handle around 1000-2000 users before you need to buy more. If you have millions of users we'll need to find you a larger rock from my parents' backyard. There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago), so their prices need to be negotiated in advance. From pw at wk-serv.de Sun Apr 1 02:18:28 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Sun, 01 Apr 2012 01:18:28 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <4F7790C4.5080002@wk-serv.de> Timo Sirainen schrieb: > A small Dovecot stone can be bought for 10 euros, a medium sized stone for 100 euros and a large stone for 500 euros. Each large stone can usually handle around 1000-2000 users before you need to buy more. If you have millions of users we'll need to find you a larger rock from my parents' backyard. There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago), so their prices need to be negotiated in advance. =) From pw at wk-serv.de Sun Apr 1 02:21:45 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Sun, 01 Apr 2012 01:21:45 +0200 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F760B61.3090209@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> Message-ID: <4F779189.8000705@wk-serv.de> Patrick Westenberg schrieb: > Nick Warr schrieb: > >> I think some of the new Dovecot (director?) software is user aware, but >> I don't know if it's quite ready for production. > > Yes, with director it should be something like that: > > MTA --(lmtp)--\ /--(lmtp)--> backend1 --\ > -- director -- -- NFS > MTA --(lmtp)--/ \--(lmtp)--> backend2 --/ > > > IMAP-User --> frontend1 --\ /--(imap)--> backend1 --\ > -- director -- -- NFS > IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/ > > > So what happens if user1 at example.tld receives a mail? > > - The director decides to connect to backend1 which in turn stores > the mail on the NFS share and the index file locally? > > - Then, user1 at example.tld connects to one of the frontends. Does the > director know that, earlier, this user received a mail and proxies him > to backend1 too? Timo? This post is from March, right before you introduced the stones :D From robert at schetterer.org Sun Apr 1 09:43:13 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 01 Apr 2012 08:43:13 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <4F77F901.30605@schetterer.org> Am 31.03.2012 23:28, schrieb Timo Sirainen: > For the last few days I've been thinking about my company and what it really should do, and as much as the current plan seems reasonable, I think in good conscience I really can't help but to bring up an alternative plan: > > The support for Dovecot is quite good already, but wouldn't it be better if it was GUARANTEED that your email server was ALWAYS working with zero downtime or other problems? After some thinking, planning and consulting with my spiritual guides I realized I really should just go ahead and do it: > > From now on you can buy blessed Dovecot stones (http://www.dovecot.org/tmp/stone1.jpg), which are guaranteed to keep your email server running. I don't think there's really any further reason for me to provide commercial support. Probably the whole Dovecot mailing list will be pretty much useless after this as well, so I might as well stop posting there. If you have any problems, you'll just buy a Dovecot stone and the problem will go away. If for any reason that doesn't work, it means that your problem is more difficult than most and you need to buy a few more stones. Please remember that each shipment may take up to 3 weeks to reach you, so get your stones early and in sufficient quantities. > > A small email server will be happy enough with a single small Dovecot stone, but the more users you have the more stones you'll likely need. This is because most people are rather unhappy with their lives, and they project their negative energies towards their surroundings, including their email servers. To overcome this negative energy you need to have a counterbalancing positive energy, which the Dovecot stones will provide you. > > The Dovecot stones are hand painted from the finest materials available to the sister of the author of Dovecot that she could find in the local shops. Each stone contains such love and dedication towards the well being of Dovecot that you will shed tears when you lay your eyes upon their physical being. The images provided here unfortunately don't do justice to them. > > http://www.dovecot.org/tmp/stone2.jpg - This is how i keep my home desktop running all day every day. I have NEVER broken a single hard drive in my life. I don't even bother to keep backups, because I KNOW it is useless for me. > > A small Dovecot stone can be bought for 10 euros, a medium sized stone for 100 euros and a large stone for 500 euros. Each large stone can usually handle around 1000-2000 users before you need to buy more. If you have millions of users we'll need to find you a larger rock from my parents' backyard. There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago), so their prices need to be negotiated in advance. Hi Timo, perhaps the ghosts in the machines will now be very happy starting eat such bricks starting in the magical Ides of April so lets pray you ll never get out of Jedi power to bless the stones finally i am looking for a stone interface , meanwhile holy rays might have power enough, preparing them near the servers in the datacenter where i allready have prisoned evil trolls -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From anyaddress at gmx.net Sun Apr 1 10:28:40 2012 From: anyaddress at gmx.net (Tom Fernandes) Date: Sun, 1 Apr 2012 09:28:40 +0200 Subject: [Dovecot] namespace hierarchy Message-ID: <201204010928.41017.anyaddress@gmx.net> Hi, I'm migrating from Courier to Dovecot and would like to keep the original namespace and add two new ones on the new server. There's only one namespace in Courier which is "INBOX." . The seperator is "." In the new server I would like to have 3 namespaces (private, public, shared) and use "/" as separator. Is it possible to keep "INBOX." for private, so that the users mailboxes appear below the Inbox, but have public and shared on the same hierarchy-level like the Inbox? When I set prefix to "INBOX/" for private I get the users mailboxes below the Inbox, like I have it Courier now. But then at least the public, probably also the shared namespace doesn't work any more. I get "unknown subscription namespace" when trying to subscribe to some of the folders. My current configuration is: namespace: type: private separator: / inbox: yes list: yes subscriptions: yes namespace: type: public separator: / prefix: public/ location: maildir:/var/vmail/example.com/public list: yes namespace: type: shared separator: / prefix: shared/%%u/ location: maildir:%%h/Maildir list: yes warm regards, Tom From trashcan at odo.in-berlin.de Sun Apr 1 10:36:22 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sun, 1 Apr 2012 09:36:22 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: Nice ;-) On 31.03.2012, at 23:28, Timo Sirainen wrote: > A small Dovecot stone can be bought for 10 euros, a medium sized stone for 100 euros and a large stone for 500 euros. Each large stone can usually handle around 1000-2000 users before you need to buy more. If you have millions of users we'll need to find you a larger rock from my parents' backyard. There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago), so their prices need to be negotiated in advance. And sorry for the previous Mail. Mail.app from Apple isn't the best program ;-) Regards, Michael From marcin at mejor.pl Sun Apr 1 12:13:36 2012 From: marcin at mejor.pl (=?ISO-8859-2?Q?Marcin_Miros=B3aw?=) Date: Sun, 01 Apr 2012 11:13:36 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <4F781C40.40102@mejor.pl> W dniu 2012-03-31 23:28, Timo Sirainen pisze: > For the last few days I've been thinking about my company and what it really should do, and as much as the current plan seems reasonable, I think in good conscience I really can't help but to bring up an alternative plan: > > The support for Dovecot is quite good already, but wouldn't it be better if it was GUARANTEED that your email server was ALWAYS working with zero downtime or other problems? After some thinking, planning and consulting with my spiritual guides I realized I really should just go ahead and do it: > > From now on you can buy blessed Dovecot stones (http://www.dovecot.org/tmp/stone1.jpg), which are guaranteed to keep your email server running. I don't think there's really any further reason for me to provide commercial support. Probably the whole Dovecot mailing list will be pretty much useless after this as well, so I might as well stop posting there. If you have any problems, you'll just buy a Dovecot stone and the problem will go away. If for any reason that doesn't work, it means that your problem is more difficult than most and you need to buy a few more stones. Please remember that each shipment may take up to 3 weeks to reach you, so get your stones early and in sufficient quantities. > > A small email server will be happy enough with a single small Dovecot stone, but the more users you have the more stones you'll likely need. This is because most people are rather unhappy with their lives, and they project their negative energies towards their surroundings, including their email servers. To overcome this negative energy you need to have a counterbalancing positive energy, which the Dovecot stones will provide you. > > The Dovecot stones are hand painted from the finest materials available to the sister of the author of Dovecot that she could find in the local shops. Each stone contains such love and dedication towards the well being of Dovecot that you will shed tears when you lay your eyes upon their physical being. The images provided here unfortunately don't do justice to them. > > http://www.dovecot.org/tmp/stone2.jpg - This is how i keep my home desktop running all day every day. I have NEVER broken a single hard drive in my life. I don't even bother to keep backups, because I KNOW it is useless for me. > > A small Dovecot stone can be bought for 10 euros, a medium sized stone for 100 euros and a large stone for 500 euros. Each large stone can usually handle around 1000-2000 users before you need to buy more. If you have millions of users we'll need to find you a larger rock from my parents' backyard. There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago), so their prices need to be negotiated in advance. Hello! I'm using dovecot on vps, how can i use dovecot stone? Is it enough to use photo of stone as boot splash or should i start dovecot with LD_PRELOAD="/path_to_photo.jpg"? From patrickdk at patrickdk.com Sun Apr 1 17:22:30 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Sun, 01 Apr 2012 10:22:30 -0400 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <20120401102230.Horde.3VgjapLnE6FPeGSmb70RjgA@mail.patrickdk.com> Quoting Timo Sirainen : > For the last few days I've been thinking about my company and what > it really should do, and as much as the current plan seems > reasonable, I think in good conscience I really can't help but to > bring up an alternative plan: > > The support for Dovecot is quite good already, but wouldn't it be > better if it was GUARANTEED that your email server was ALWAYS > working with zero downtime or other problems? After some thinking, > planning and consulting with my spiritual guides I realized I really > should just go ahead and do it: > > From now on you can buy blessed Dovecot stones > (http://www.dovecot.org/tmp/stone1.jpg), which are guaranteed to > keep your email server running. I don't think there's really any > further reason for me to provide commercial support. Probably the > whole Dovecot mailing list will be pretty much useless after this as > well, so I might as well stop posting there. If you have any > problems, you'll just buy a Dovecot stone and the problem will go > away. If for any reason that doesn't work, it means that your > problem is more difficult than most and you need to buy a few more > stones. Please remember that each shipment may take up to 3 weeks > to reach you, so get your stones early and in sufficient quantities. > > A small email server will be happy enough with a single small > Dovecot stone, but the more users you have the more stones you'll > likely need. This is because most people are rather unhappy with > their lives, and they project their negative energies towards their > surroundings, including their email servers. To overcome this > negative energy you need to have a counterbalancing positive energy, > which the Dovecot stones will provide you. > > The Dovecot stones are hand painted from the finest materials > available to the sister of the author of Dovecot that she could find > in the local shops. Each stone contains such love and dedication > towards the well being of Dovecot that you will shed tears when you > lay your eyes upon their physical being. The images provided here > unfortunately don't do justice to them. > > http://www.dovecot.org/tmp/stone2.jpg - This is how i keep my home > desktop running all day every day. I have NEVER broken a single > hard drive in my life. I don't even bother to keep backups, because > I KNOW it is useless for me. > > A small Dovecot stone can be bought for 10 euros, a medium sized > stone for 100 euros and a large stone for 500 euros. Each large > stone can usually handle around 1000-2000 users before you need to > buy more. If you have millions of users we'll need to find you a > larger rock from my parents' backyard. There are only a limited > number of such rocks available (we disposed of most of them to our > neighbors' yards years ago), so their prices need to be negotiated > in advance. I'm unable to locate the order page. Or should I reply to the maillist with my creditcard info? From christian.rohmann at frittentheke.de Sun Apr 1 19:38:07 2012 From: christian.rohmann at frittentheke.de (Christian Rohmann) Date: Sun, 01 Apr 2012 18:38:07 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <4F781C40.40102@mejor.pl> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> <4F781C40.40102@mejor.pl> Message-ID: <4F78846F.5050909@frittentheke.de> Hey all, On 04/01/2012 11:13 AM, Marcin Miros?aw wrote: > I'm using dovecot on vps, how can i use dovecot stone? Is it enough to > use photo of stone as boot splash or should i start dovecot with > LD_PRELOAD="/path_to_photo.jpg"? Valid questions, what about Dovecot vStones? Apart from this practical issue, who is your marketing advisor? Timo in cloudy times like these, one needs product names that assure customers it's compatible to the Cloud: I suggest something like: Dovecot CloudProtector Stone 2000 S Dovecot CloudProtector Stone 4500 GT Dovecot CloudProtector Stone 6900 GTX turbo Also for such sheer value in less nightly calls on broken servers, you need to price the things higher. This will underline that the stones are of exceptional quality and energy. Regards Christian From robert at schetterer.org Sun Apr 1 20:51:13 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 01 Apr 2012 19:51:13 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <4F78846F.5050909@frittentheke.de> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> <4F781C40.40102@mejor.pl> <4F78846F.5050909@frittentheke.de> Message-ID: <4F789591.1000502@schetterer.org> Am 01.04.2012 18:38, schrieb Christian Rohmann: > Hey all, > > On 04/01/2012 11:13 AM, Marcin Miros?aw wrote: >> I'm using dovecot on vps, how can i use dovecot stone? Is it enough to >> use photo of stone as boot splash or should i start dovecot with >> LD_PRELOAD="/path_to_photo.jpg"? > > > Valid questions, what about Dovecot vStones? > > Apart from this practical issue, who is your marketing advisor? Timo in > cloudy times like these, one needs product names that assure customers > it's compatible to the Cloud: > > I suggest something like: > Dovecot CloudProtector Stone 2000 S > Dovecot CloudProtector Stone 4500 GT > Dovecot CloudProtector Stone 6900 GTX turbo > > > Also for such sheer value in less nightly calls on broken servers, you > need to price the things higher. This will underline that the stones are > of exceptional quality and energy. > > > > Regards > > Christian > by the way, are they kryptonite checked ? I dont wanna get in problems with Mr Kent ? http://en.wikipedia.org/wiki/Kryptonite -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From stsiol at yahoo.co.uk Mon Apr 2 00:54:40 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Sun, 1 Apr 2012 22:54:40 +0100 (BST) Subject: [Dovecot] Mail migration troubles Message-ID: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> Hello all, OK, I would really appreciate your help on this. I managed to do a pretty straight-forward and flawless migration on dovecot from one domain to another. However, I have this nagging problem with most of the thunderbird MTAs. When I try to delete some "test" messages I sent to make sure that everything works, it comes back with the following message on the bottom of the Thunderbird window : "The current operation on InBox did not succeed.? The mail server for account "Account at bloomingdomain.gr"? responded [TRYCREATE] mailbox doesn't exist : Trash" Now, I know it has to do something with automatically creating a? trash folder. But don't know anything more. Could someone shed some light on this ? Dovecot v1.2.15 AFAIK CentOS 5.5 x86 (32-bit) other stuff On Clients : Thunderbird v11.0.1 (XP Pro / 32-bit) s. ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis? From stsiol at yahoo.co.uk Mon Apr 2 00:59:25 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Sun, 1 Apr 2012 22:59:25 +0100 (BST) Subject: [Dovecot] strike my last Message-ID: <1333317565.13835.YahooMailNeo@web132204.mail.ird.yahoo.com> strike my last, looks like its a thunderbird issue Apologies... s. ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From jeetuindian at gmail.com Mon Apr 2 10:31:20 2012 From: jeetuindian at gmail.com (Jitendra Bhaskar) Date: Mon, 2 Apr 2012 13:01:20 +0530 Subject: [Dovecot] Dsync automation Message-ID: Hi, I am using dovecot-2.1.3 on centos 5.7. Its running fine. I have two systems for mail server one is for backup. Currently I am running rsync daily. But I want to use dsync. I went through all the http://wiki2.dovecot.org/Tools/Dsync and http://blog.dovecot.org/2012/02/dovecot-clustering-with-dsync-based.htmlbut I couldn't understood how to automatize dsync. Can anybody give me any idea ho to do it ? -- * Thanks & Regards * *Jitendra Kumar Bhaskar* Cell:- +91 7306311531 +91 8102997821 From stsiol at yahoo.co.uk Mon Apr 2 13:19:47 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 2 Apr 2012 11:19:47 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> Message-ID: <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> I have a suspicion that, as usual, no one is going to? answer my message. Because of the fact that this is a call for help and no-one ever bothers, I 'll just have to re-phrase my question : Here's what I found on the dovecot wiki : imapsync Here's an example of how to run imapsync for a single user: imapsync --syncinternaldates \??--host1 192.168.1.57 --authmech1 LOGIN --user1 leah at example.com --password1 secret \??--host2 127.0.0.1????--authmech2 LOGIN --user2 leah at example.com --password2 secret It is quite easy to script this for a number of users, assuming you have their passwords. Even if you do not, imapsync also supports logging in as an admin user that has the ability to copy message for sub users, and a variety of other authentication options. Is the above what I need if I migrate mailboxes FROM dovecot TO dovecot ? E.g. in my example, I only copy mailboxes from an old "user at domainA.gr" to "user at domainB.gr". Is the above all I need ? Is there a native dovecot tool for this or should I just use imapsync ? Thank you again, s. ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis ----- Original Message ----- > From: Spyros Tsiolis > To: Spyros Tsiolis > Cc: > Sent: Monday, 2 April 2012, 11:24 > Subject: Re: [Dovecot] Mail migration troubles > > Hello again, > > Well, it seems I have trouble with the migration after all. > After the migration and eventhough all folders and messages > are on their respective places, any new mail users get > inside their mailbox cannot be dragged-and-dropped to > their respective folders. > > So, if a user has a folder with my name on it and I send > him an e-mail, the user will try to drag-and-drop that > message to the folder with my name, but the message won't > move. > > Any gurus out there on this ? > Any help would be greatly appreciated. > > spyros > > > ? > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis > ? From CMarcus at Media-Brokers.com Mon Apr 2 13:30:06 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 02 Apr 2012 06:30:06 -0400 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> Message-ID: <4F797FAE.7030104@Media-Brokers.com> On 2012-04-02 6:19 AM, Spyros Tsiolis wrote: > Is the above what I need if I migrate mailboxes FROM dovecot TO dovecot ? > E.g. in my example, I only copy mailboxes from an old "user at domainA.gr" to > "user at domainB.gr". > > Is the above all I need ? > > Is there a native dovecot tool for this or should I just use imapsync ? Are these on ths same box? If so, why not just mv them?? -- Best regards, Charles From CMarcus at Media-Brokers.com Mon Apr 2 13:31:56 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 02 Apr 2012 06:31:56 -0400 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <4F79801C.8020804@Media-Brokers.com> On 2012-03-31 5:28 PM, Timo Sirainen wrote: > For the last few days I've been thinking about my company and what it > really should do, and as much as the current plan seems reasonable, I > think in good conscience I really can't help but to bring up an > alternative plan: > > The support for Dovecot is quite good already, but wouldn't it be > better if it was GUARANTEED that your email server was ALWAYS working > with zero downtime or other problems? After some thinking, planning > and consulting with my spiritual guides I realized I really should > just go ahead and do it: > > From now on you can buy blessed Dovecot stones > (http://www.dovecot.org/tmp/stone1.jpg), which are guaranteed to keep > your email server running. I don't think there's really any further > reason for me to provide commercial support. Alright Timo, where do I send the bill for my new keyboard? Mine is no ruined from the coffee I just spilled all over it... Rotflmao! Thanks, I needed that... -- Best regards, Charles From arne at drlinux.no Mon Apr 2 13:36:38 2012 From: arne at drlinux.no (Arne K. Haaje) Date: Mon, 02 Apr 2012 12:36:38 +0200 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> Message-ID: <5628413.EKgo53ll8l@trillian> Mandag 2. april 2012 11.19.47 skrev Spyros Tsiolis: [snip] > Is the above what I need if I migrate mailboxes FROM dovecot TO dovecot ? > E.g. in my example, I only copy mailboxes from an old "user at domainA.gr" to > "user at domainB.gr". > > Is the above all I need ? That worked for me when creating a backup copy of a mailbox, but with a different login. I copied mailbox A to mailbox B, then created an entry in the auth table that pointed to mailbox B. This was with maildir. I don't know if it works the same with all other types. Regards, Arne -- Arne K. Haaje http://www.drlinux.no/ LinkedIn: http://no.linkedin.com/pub/arne-haaje/27/189/bb From stsiol at yahoo.co.uk Mon Apr 2 13:51:53 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 2 Apr 2012 11:51:53 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <4F797FAE.7030104@Media-Brokers.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> Message-ID: <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> > On 2012-04-02 6:19 AM, Spyros Tsiolis wrote: >> Is the above what I need if I migrate mailboxes FROM dovecot TO dovecot ? >> E.g. in my example, I only copy mailboxes from an old? > "user at domainA.gr" to >> "user at domainB.gr". >>? >> Is the above all I need ? >>? >> Is there a native dovecot tool for this or should I just use imapsync ? >? > Are these on ths same box? If so, why not just mv them?? Hi Charles, Yes, on the same box, on the same drive, on the same partition/filesystem. I have a box with dovecot/XMail (pop3/smtp) and two domains; DomainA and DomainB The company bosses decided to move about 20 mailboxes from DomainA to DomainB. As far as I can tell everything went well except for one user (so far) that he tries to move fresh mail to (mv'd) folders and it doesn't get there. Also when he tries to delete messages, they don't get deleted. I did "cp -r" actually and kept the old mailboxes nearby just in case. Thanks, s. ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From CMarcus at Media-Brokers.com Mon Apr 2 13:55:12 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 02 Apr 2012 06:55:12 -0400 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <4F798590.6060701@Media-Brokers.com> On 2012-04-02 6:51 AM, Spyros Tsiolis wrote: > As far as I can tell everything went well except for one > user (so far) that he tries to move fresh mail to (mv'd) > folders and it doesn't get there. > Also when he tries to delete messages, they don't get > deleted. > I did "cp -r" actually and kept the old mailboxes nearby > just in case. What do the logs show when he tries this? Did you compare FS permissions on a working setup and the non-working one? -- Best regards, Charles From nmilas at noa.gr Mon Apr 2 14:44:48 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Mon, 02 Apr 2012 14:44:48 +0300 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <4F799130.2020806@noa.gr> On 2/4/2012 1:51 ??, Spyros Tsiolis wrote: > I did "cp -r" actually and kept the old mailboxes nearby > just in case. If on CentOS/RHEL (don't know for other systems), you could try using explicitly: /bin/cp -pr for better results, because cp is aliased. You might already be aware of this, but just in case... Nick From stsiol at yahoo.co.uk Mon Apr 2 15:05:23 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 2 Apr 2012 13:05:23 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <4F799130.2020806@noa.gr> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> <4F799130.2020806@noa.gr> Message-ID: <1333368323.2437.YahooMailNeo@web132204.mail.ird.yahoo.com> > On 2/4/2012 1:51 ??, Spyros Tsiolis wrote: >? >> I did "cp -r" actually and kept the old mailboxes nearby >> just in case. >? > If on CentOS/RHEL (don't know for other systems), you could try using? > explicitly: >? > ? /bin/cp -pr >? > for better results, because cp is aliased. >? > You might already be aware of this, but just in case... >? > Nick > Hi Nick, No I didn't know this. I mean I had it stored in the back of my head but? to tell you the truth I would never use it. Yes, thank you kindly for pointing that. I'll use that switch from now on. And , yes, I am using CentOS. Can't live without it, no matter what the people say :-) Thank you again, s. ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From stsiol at yahoo.co.uk Mon Apr 2 15:49:28 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 2 Apr 2012 13:49:28 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <4F798590.6060701@Media-Brokers.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> <4F798590.6060701@Media-Brokers.com> Message-ID: <1333370968.69235.YahooMailNeo@web132201.mail.ird.yahoo.com> ----- Original Message ----- > From: Charles Marcus > To: dovecot at dovecot.org > Cc: > Sent: Monday, 2 April 2012, 12:55 > Subject: Re: [Dovecot] Mail migration troubles > > On 2012-04-02 6:51 AM, Spyros Tsiolis wrote: >> As far as I can tell everything went well except for one >> user (so far) that he tries to move fresh mail to (mv'd) >> folders and it doesn't get there. >> Also when he tries to delete messages, they don't get >> deleted. >> I did "cp -r" actually and kept the old mailboxes nearby >> just in case. > > What do the logs show when he tries this? > > Did you compare FS permissions on a working setup and the non-working one? > > -- > Best regards, > > Charles > Hi Charles, Well, I checked the dovecot log files? (dovecot-info.log / dovecot.log)? It doesn't show something suspicious. Just the usual login logout and password chatter. At this point, I suspect it's thunderbird again. I asked the rest of the people how they are doing and they said they were? doing fine. So I suspect there's something wrong with this specific dovecot installation. Thank you though, s. ? ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From jdonovan at beth.k12.pa.us Mon Apr 2 16:01:04 2012 From: jdonovan at beth.k12.pa.us (jeff donovan) Date: Mon, 2 Apr 2012 09:01:04 -0400 Subject: [Dovecot] setting up ldap for sasl Message-ID: <98F8F008-100B-4CAF-BE0E-F2BBAC50B2D7@beth.k12.pa.us> Greetings im new to the list dovecot --version 2.0.13 ubuntu 11.10 i want to use dovecot to authenticate my ldap users. My local users can authenticate and send email. Using postfix sasl/tls dovecot pam. I have read many docs/tutorials, which all have a plethora of options and information. my file structure is different than most of the documentation. I have ../etc/dovecot.conf which points to. !include_try /usr/share/dovecot/protocols.d/*.protocol !include conf.d/*.conf !include_try local.conf which leads me to /etc/dovecot/conf.d, which is full of config files. All were traditionally located in dovecot.conf in previous versions. im using PAM to authenticate my local system users. I have read that could adjust PAM, to use LDAP. How do I configure dovecot to keep using pam for local users and ldap for remote ? Do i just add the passdb lines dovecot.conf: passdb ldap { args = /etc/dovecot-ldap.conf } userdb prefetch { } # The userdb below is used only by deliver. userdb ldap { args = /etc/dovecot-ldap.conf } dovecot-ldap.conf: pass_attrs = uid=user,userPassword=password, homeDirectory=userdb_home,uidNumber=userdb_uid,gidNumber=userdb_gid # For deliver: user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid any assistance would be helpful. -j -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2497 bytes Desc: not available URL: From CMarcus at Media-Brokers.com Mon Apr 2 16:20:49 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 02 Apr 2012 09:20:49 -0400 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333370968.69235.YahooMailNeo@web132201.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> <4F798590.6060701@Media-Brokers.com> <1333370968.69235.YahooMailNeo@web132201.mail.ird.yahoo.com> Message-ID: <4F79A7B1.4030908@Media-Brokers.com> On 2012-04-02 8:49 AM, Spyros Tsiolis wrote: > On2 April 2012, 12:55, Charles Marcus wrote: >> On 2012-04-02 6:51 AM, Spyros Tsiolis wrote: >>> As far as I can tell everything went well except for one >>> user (so far) that he tries to move fresh mail to (mv'd) >>> folders and it doesn't get there. >>> Also when he tries to delete messages, they don't get >>> deleted. >>> I did "cp -r" actually and kept the old mailboxes nearby >>> just in case. >> What do the logs show when he tries this? >> >> Did you compare FS permissions on a working setup and the >> non-working one? > Well, I checked the dovecot log files > (dovecot-info.log / dovecot.log) > It doesn't show something suspicious. > Just the usual login logout and > password chatter. Well, maybe one of the reasons people don't always try to help you is you often just don't respond with answers to questions... WHAT ARE THE FS PERMISSIONS ON THAT USERS MAIL FOLDERS? And don't just say 'they are ok' - SHOW them... And how about SHOWING SOME LOG EXCERPTS (and increase verbosity if needed), instead of just saying you don't see anything relevant. -- Best regards, Charles From jg at softjury.de Mon Apr 2 17:06:28 2012 From: jg at softjury.de (Jan Phillip Greimann) Date: Mon, 02 Apr 2012 16:06:28 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <4F79B264.9030007@softjury.de> STOP TALKING AND TAKE MY MONEY!!!!1111elf Am 31.03.2012 23:28, schrieb Timo Sirainen: > For the last few days I've been thinking about my company and what it really should do, and as much as the current plan seems reasonable, I think in good conscience I really can't help but to bring up an alternative plan: > > The support for Dovecot is quite good already, but wouldn't it be better if it was GUARANTEED that your email server was ALWAYS working with zero downtime or other problems? After some thinking, planning and consulting with my spiritual guides I realized I really should just go ahead and do it: > > From now on you can buy blessed Dovecot stones (http://www.dovecot.org/tmp/stone1.jpg), which are guaranteed to keep your email server running. I don't think there's really any further reason for me to provide commercial support. Probably the whole Dovecot mailing list will be pretty much useless after this as well, so I might as well stop posting there. If you have any problems, you'll just buy a Dovecot stone and the problem will go away. If for any reason that doesn't work, it means that your problem is more difficult than most and you need to buy a few more stones. Please remember that each shipment may take up to 3 weeks to reach you, so get your stones early and in sufficient quantities. > > A small email server will be happy enough with a single small Dovecot stone, but the more users you have the more stones you'll likely need. This is because most people are rather unhappy with their lives, and they project their negative energies towards their surroundings, including their email servers. To overcome this negative energy you need to have a counterbalancing positive energy, which the Dovecot stones will provide you. > > The Dovecot stones are hand painted from the finest materials available to the sister of the author of Dovecot that she could find in the local shops. Each stone contains such love and dedication towards the well being of Dovecot that you will shed tears when you lay your eyes upon their physical being. The images provided here unfortunately don't do justice to them. > > http://www.dovecot.org/tmp/stone2.jpg - This is how i keep my home desktop running all day every day. I have NEVER broken a single hard drive in my life. I don't even bother to keep backups, because I KNOW it is useless for me. > > A small Dovecot stone can be bought for 10 euros, a medium sized stone for 100 euros and a large stone for 500 euros. Each large stone can usually handle around 1000-2000 users before you need to buy more. If you have millions of users we'll need to find you a larger rock from my parents' backyard. There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago), so their prices need to be negotiated in advance. From wgrcunha at gmail.com Mon Apr 2 17:14:26 2012 From: wgrcunha at gmail.com (Francisco Wagner C. Freire) Date: Mon, 2 Apr 2012 11:14:26 -0300 Subject: [Dovecot] Backtrace on mdbox index Message-ID: Hello, I current using dovecot with mdbox and on one account i get this error: doveadm(user at domain): Panic: file mail-index-sync-ext.c: line 209 (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) doveadm(user at domain): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x3fd8a) [0x7fae684fdd8a] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7fae684fde72] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7fae684d719f] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9bdc3) [0x7fae68c1edc3] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_ext_intro+0x1e6) [0x7fae68c1fa76] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_record+0x49f) [0x7fae68c21a3f] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_map+0x234) [0x7fae68c225a4] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_map+0x86) [0x7fae68c13cd6] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x8b609) [0x7fae68c0e609] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_open+0x210) [0x7fae68c0eaa0] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x591c6) [0x7fae68bdc1c6] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_map_get_zero_ref_files+0x2d) [0x7fae68bdcd6d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_purge+0xbc) [0x7fae68bde7ac] -> doveadm() [0x40a135] -> doveadm() [0x40a813] -> doveadm(doveadm_mail_single_user+0x61) [0x40a8d1] -> doveadm() [0x40aafd] -> doveadm(doveadm_mail_try_run+0x141) [0x40af11] -> doveadm(main+0x381) [0x4109c1] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7fae67d6ec8d] -> doveadm() [0x409ff9] Aborted All doveadm commands fail on thie error, index / force-resync / search / fetch etc. Any help? From stsiol at yahoo.co.uk Mon Apr 2 17:29:39 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 2 Apr 2012 15:29:39 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <4F79A7B1.4030908@Media-Brokers.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> <4F798590.6060701@Media-Brokers.com> <1333370968.69235.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F79A7B1.4030908@Media-Brokers.com> Message-ID: <1333376979.98299.YahooMailNeo@web132205.mail.ird.yahoo.com> Hi again Charles, >>> Did you compare FS permissions on a working setup and the >>> non-working one? I don't have a non-working setup of dovecot >> Well, I checked the dovecot log files >> (dovecot-info.log / dovecot.log) >> It doesn't show something suspicious. >> Just the usual login logout and >> password chatter. >? > Well, maybe one of the reasons people don't always try to help you is you? > often just don't respond with answers to questions... Oh, I see. It's like what I ask questions about issues and? get answers on other issues... ? > WHAT ARE THE FS PERMISSIONS ON THAT USERS MAIL FOLDERS? And don't just say? > 'they are ok' - SHOW them... All users run with the virtual mail account (vmail) accounts (if I recall correctly) are stored in %maildir format. Do you want me to display user perms ? OK ! Fine. Here's the contents of the folder with the problem : [root at mailserver mailaccount]# ls -l total 8 drwx------ 6 vmail vmail 4096 Apr ?2 17:08 Maildir -rw-rw-rw- 1 vmail vmail ?141 Mar 30 17:21 user.tab Is this what you need ?? I didn't answer this not because I don't want to but because I don't know if this is what you want. There's no need to shout. ? > And how about SHOWING SOME LOG EXCERPTS (and increase verbosity if needed),? > instead of just saying you don't see anything relevant. Here goes : Apr 02 14:26:02 auth(default): Info: client out: OK1user=account at domain.gr Apr 02 14:26:02 auth(default): Info: master in: REQUEST1785294431 Apr 02 14:26:02 auth(default): Info: master out: USER1785account at domain.gruid=501gid=502home=/ho me/vmail/account at domain.gr Apr 02 14:26:02 imap-login: Info: Login: user=, method=PLAIN, rip=192.168.0.20, lip=192.168.1. 4, TLS Apr 02 14:26:02 auth(default): Info: new auth connection: pid=29450 Apr 02 14:27:02 auth(default): Info: client in: AUTH1PLAINservice=imapsecuredlip=192.168.1.4rip=192. 168.0.20lport=993rport=4494 Apr 02 14:27:02 auth(default): Info: client out: CONT1 Apr 02 14:27:02 auth(default): Info: client in: CONT1AHRpY2ldA29ZnzWzc9c5cgxTQA== Apr 02 14:27:02 auth(default): Info: passwd-file(account at domain.gr,192.168.0.20): lookup: user=account at domain.gr ?file=/etc/dovecot/passwd Apr 02 14:27:02 auth(default): Info: client out: OK1user=account at domain.gr Apr 02 14:27:02 auth(default): Info: master in: REQUEST1786294501 Apr 02 14:27:02 auth(default): Info: master out: USER1786account at domain.gruid=501gid=502home=/ho me/vmail/account at domain.gr Apr 02 14:27:02 imap-login: Info: Login: user=, method=PLAIN, rip=192.168.0.20, lip=192.168.1. 4, TLS Apr 02 14:27:02 auth(default): Info: new auth connection: pid=29453 s. ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From giom87 at libero.it Mon Apr 2 17:39:08 2012 From: giom87 at libero.it (Giovanni Mancuso) Date: Mon, 02 Apr 2012 16:39:08 +0200 Subject: [Dovecot] Connection queue full on dovecot 2.0.13 Message-ID: <4F79BA0C.1070104@libero.it> Hi, i have a problem with dovecot 2.0.13. I have one dovecot in Front-end that has only: passdb { driver = static args = proxy=y host=10.0.0.6 nopassword=y } In Back-end i have one dovecot that does auth and exports imap/pop3 ports. In dovecot's log of Front-end i see: /Apr 02 14:33:34 imap-login: Info: proxy(//user at example.com/ /): started proxying to 10.0.0.6:143: user=/>, method=PLAIN, rip=//xx.xx.xx.xx//, lip=xx.xx.xx.xx, TLS Apr 02 14:34:36 imap-login: Info: Disconnected: *Connection queue full *(auth failed, 1 attempts): user=/>, method=PLAIN, rip=//xx.xx.xx.xx//, lip=//xx.xx.xx.xx//, TLS/ I see this wiki page: http://wiki2.dovecot.org/LoginProcess but i read: It works by using a number of long running login processes, each handling a number of connections. This loses much of the security benefits of the login process design, because in case of a security hole (in Dovecot or SSL library) the attacker is now able to see other users logging in and steal their passwords, read their mails, etc. Is there another way? Thanks From zybi at talex.pl Mon Apr 2 18:45:25 2012 From: zybi at talex.pl (=?ISO-8859-2?Q?Artur_Zaprza=B3a?=) Date: Mon, 02 Apr 2012 17:45:25 +0200 Subject: [Dovecot] lmtp sometimes fails to deliver a message to all recipients Message-ID: <4F79C995.9060607@talex.pl> Dovecot version: 2.1.3 lmtp config: protocol lmtp { mail_plugins = zlib acl sieve } Running strace on lmtp reveals that when lmtp delivers a message to multiple recipients it saves the message to a file in the first mailbox and hard links that file to other mailboxes. But when in the meantime of delivery that file is deleted, lmtp fails to deliver the message to remaining mailboxes. For example a message for 46 recipients is delivered by lmtp to the first 12 only (fortunately postfix retries delivery for remaining recipients) and for the rest lmtp logs the following error messages: lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Message was expunged (guid) lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: script /vmail/domain/foo/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /vmail/domain/foo/.dovecot.sieve.log may reveal additional details) I'm seeing this error for about 10 messages per hour. Best regards, Artur Zaprza?a From adorca at uda.ad Mon Apr 2 19:11:06 2012 From: adorca at uda.ad (Aleix Dorca) Date: Mon, 2 Apr 2012 18:11:06 +0200 Subject: [Dovecot] imapsync, namespaces and Dovecot 2.x Message-ID: Hi people, I recently tried to migrate from Exchange using imapsync. It didn't work because it complained that dovecot had no namespace capablity. After looking through Dovecot's config files I found that namespaces could be defined, but if you did not a 'private' default one was created. In the 2.x wiki documentation (http://wiki2.dovecot.org/Migration) it says nothing about Namespaces. I don't know if I have to create one or not. If I add the parameters --sep2 . --prefix2 '' to imapsync then it works fine. I'm proxying using dovecot as frontend and backend, version 2.0.18 (The one that comes with Debian Wheezy). Any thoughts, please? Thanks, Aleix. From moseleymark at gmail.com Mon Apr 2 19:33:32 2012 From: moseleymark at gmail.com (Mark Moseley) Date: Mon, 2 Apr 2012 09:33:32 -0700 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: On Sat, Mar 31, 2012 at 2:28 PM, Timo Sirainen wrote: > For the last few days I've been thinking about my company and what it really should do, and as much as the current plan seems reasonable, I think in good conscience I really can't help but to bring up an alternative plan: > >... I'm slightly concerned that there's been no mention of what license these stones are going to be released under. GPL2? GPL3? Apache? I'm just hoping these aren't some sort of "open core" stones that will only work for basic features but that I'll end up needing to buy "Enterprise-grade stones" to cover large clusters. From kurt-dovecot at se.rit.edu Tue Apr 3 00:23:20 2012 From: kurt-dovecot at se.rit.edu (Kurt Mosiejczuk) Date: Mon, 02 Apr 2012 17:23:20 -0400 Subject: [Dovecot] imapdir still a configuration option? Message-ID: <4F7A18C8.3000904@se.rit.edu> I've been looking at moving an old IMAPdir-based solution to dovecot. I came across references that dovecot (to my pleasant surprise) added imapdir support in 1.1. However, in trying 2.0.13 (from an OpenBSD package) things start okay, but I see in the logs: "Initialization failed: Initializing mail storage from mail_location setting failed: Unknown mail storage driver imapdir" Poking around in the sources, I still see lots of references to imapdir, but see very little in the documentation. Is IMAPdir still supported in 2.0.x (or even 2.1.x)? I'd prefer not to have to migrate users via renaming, especially since they like the IMAPdir layout for their own searching. --Kurt From micah at riseup.net Tue Apr 3 01:26:43 2012 From: micah at riseup.net (Micah Anderson) Date: Mon, 02 Apr 2012 18:26:43 -0400 Subject: [Dovecot] Dsync automation References: Message-ID: <87k41xrcrg.fsf@algae.riseup.net> Jitendra Bhaskar writes: > I am using dovecot-2.1.3 on centos 5.7. Its running fine. I have two > systems for mail server one is for backup. Currently I am running rsync > daily. But I want to use dsync. I went through all the > http://wiki2.dovecot.org/Tools/Dsync and > http://blog.dovecot.org/2012/02/dovecot-clustering-with-dsync-based.htmlbut > I couldn't understood how to automatize dsync. Can anybody give me any > idea ho to do it ? As far as I can tell, doing a for loop around 'doveadm user \*' is the only way. That will do a dsync of all the users. On the backup server you will need to decide what to do with the data. I've been rotating it on a daily basis to try and get several daily backups, then some weekly, and monthly. I've been working on a backupninja[0] handler to do dsync backups, but its still rough. micah 0. https://labs.riseup.net/code/projects/backupninja -- From micah at riseup.net Tue Apr 3 01:34:07 2012 From: micah at riseup.net (Micah Anderson) Date: Mon, 02 Apr 2012 18:34:07 -0400 Subject: [Dovecot] 2.1.3: doveadm mailbox delete lost the -s option Message-ID: <874nt1rcf4.fsf@algae.riseup.net> Looks like the recently added -s (for unsubscribing) option to doveadm mailbox delete went missing: root at vireo# /usr/bin/doveadm mailbox delete -u micahtest -s restored delete: invalid option -- 's' doveadm mailbox delete [-u |-A] [-S ] [-s] [...] although the option still is shown as valid :) micah -- From zarf at klacto.net Tue Apr 3 02:02:17 2012 From: zarf at klacto.net (FZiegler) Date: Mon, 02 Apr 2012 19:02:17 -0400 Subject: [Dovecot] dovecot and unison Message-ID: <4F7A2FF9.7000204@klacto.net> I am successfully using dovecot purely as a personal local mail store on my desktop. (There is only one account, and it's only ever accessed by local mail clients on the machine. The point is to have a common store I can use with any client; plus, I prefer dovecot's Mailbox storage to Thunderbird's mboxes.) Now I'd like if possible, to replicate this setup on my laptop and keep both in sync with unison (http://www.cis.upenn.edu/~bcpierce/unison/), which I am already using to sync much of my home dir about once a day. I found at least one positive message regarding this topic (http://dovecot.org/list/dovecot/2010-April/048092.html), but I feel I could use some more advice. Namely, which of the ancillary files should I be syncing (or perhaps deleting before sync)? In addition to the mails themselves I'm seeing: -rw------- 1 fz 501 13 Apr 1 17:10 dovecot-keywords -rw------- 1 fz 501 2483 Apr 1 17:10 dovecot-uidlist -rw------- 1 fz 501 1040 Mar 29 14:50 dovecot.index -rw------- 1 fz 501 44032 Apr 1 17:10 dovecot.index.cache -rw------- 1 fz 501 9772 Apr 1 17:10 dovecot.index.log and at the top level (in .local/var/Maildir/): -rw------- 1 fz 501 8 Apr 1 17:10 dovecot-uidvalidity -r--r--r-- 1 fz 501 0 Apr 1 17:10 dovecot-uidvalidity.4efb323d -rw------- 1 fz 501 3288 Apr 1 17:10 dovecot.mailbox.log -rw------- 1 fz 501 4104 Mar 28 10:53 dovecot.mailbox.log.2 -rw------- 1 fz 501 5114 Apr 1 17:10 subscriptions In my naive trials I ended up at least once in a situation where Thunderbird would not see some subfolders (even though they were there). That's since fixed but I guess my question is, what's the safe recommended way to not have this happen? Thanks, FZiegler From stan at hardwarefreak.com Tue Apr 3 02:04:54 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 02 Apr 2012 18:04:54 -0500 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <4F7A3096.5050209@hardwarefreak.com> On 3/31/2012 4:28 PM, Timo Sirainen wrote: There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago) :) -- Stan From micah at riseup.net Tue Apr 3 02:15:28 2012 From: micah at riseup.net (Micah Anderson) Date: Mon, 02 Apr 2012 19:15:28 -0400 Subject: [Dovecot] dsync redesign References: <936E95DA-E032-4F8F-A323-DEBB7AAC0E1B@iki.fi> <87obrixcyp.fsf@algae.riseup.net> <4F721650.4030901@Media-Brokers.com> Message-ID: <87zkatpvxr.fsf@algae.riseup.net> Charles Marcus writes: > On 2012-03-27 11:47 AM, Micah Anderson wrote: >> One would be the ability to perform *intelligent* incremental / >> rotated backups. I can do this now by running a dsync backup >> operation and then doing manual hardlinking or moving of the backup >> directories (daily.1, daily.2, weekly.1, monthly.1, etc.), but it >> would be more intelligent if this were baked into the backup process. > > There are already numerous tools that do this flawlessly - I've been using > rsnapshot (which uses rsync) for this for years. Are you snapshotting your filesystem (using LVM, or SAN, or similar) before doing rsnapshot? Because if you aren't then rsync will not assuredly get everything in a consistent state. > I don't know if Timo should be spending his time reinventing the wheel. dsync backup is already here, and it is quite useful. > I'm much more interested in dsync working flawlessly to keep one or more > secondary servers in sync, and leave backups to backup software. I'm not against that idea, I just have not yet found a good way to use any backup software in such a way to handle large numbers of user's mail. > Although, one interesting piece that I am hopeful I'll be able to implement soon > (with Timo's professional help) is the ability to easily and automatically map > my rsnapshot snapshots directory to a read-only 'Backups' namespace that > automatically shows the snapshots by date and time as they are produced. This > way users could 'go back in time' anytime they wanted without having to call > me... :) Interesting idea, would be a great one to share with the community if you decide to do so. micah From jdonovan at beth.k12.pa.us Tue Apr 3 03:56:42 2012 From: jdonovan at beth.k12.pa.us (jeff donovan) Date: Mon, 2 Apr 2012 20:56:42 -0400 Subject: [Dovecot] setting up ldap for sasl In-Reply-To: <98F8F008-100B-4CAF-BE0E-F2BBAC50B2D7@beth.k12.pa.us> References: <98F8F008-100B-4CAF-BE0E-F2BBAC50B2D7@beth.k12.pa.us> Message-ID: <7EEE40B1-2E79-4814-B3E0-F4AE0FB45831@beth.k12.pa.us> On Apr 2, 2012, at 9:01 AM, jeff donovan wrote: > Greetings > im new to the list > dovecot --version > 2.0.13 > ubuntu 11.10 > i want to use dovecot to authenticate my ldap users. My local users can authenticate and send email. Using postfix sasl/tls dovecot pam. > > I have read many docs/tutorials, which all have a plethora of options and information. my file structure is different than most of the documentation. > I have ../etc/dovecot.conf which points to. > > !include_try /usr/share/dovecot/protocols.d/*.protocol > !include conf.d/*.conf > !include_try local.conf i included /etc/dovecot/conf.d/dovecot-ldap.conf and I get errors from dovecot config. doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/dovecot-ldap.conf line 17: Unknown setting: hosts doveconf: Error: managesieve-login: dump-capability process returned 89 i remove it, then it goes to the next option i have set; doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/dovecot-ldap.conf line 21: Unknown setting: uris doveconf: Error: managesieve-login: dump-capability process returned 89 any help would be great -j -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2497 bytes Desc: not available URL: From jdonovan at beth.k12.pa.us Tue Apr 3 05:28:13 2012 From: jdonovan at beth.k12.pa.us (jeff donovan) Date: Mon, 2 Apr 2012 22:28:13 -0400 Subject: [Dovecot] setting up ldap for sasl::solved:: In-Reply-To: <7EEE40B1-2E79-4814-B3E0-F4AE0FB45831@beth.k12.pa.us> References: <98F8F008-100B-4CAF-BE0E-F2BBAC50B2D7@beth.k12.pa.us> <7EEE40B1-2E79-4814-B3E0-F4AE0FB45831@beth.k12.pa.us> Message-ID: <18E8FC85-525D-4B46-8246-7B9A6FDC113F@beth.k12.pa.us> I had to compile with ldap-- duh. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2497 bytes Desc: not available URL: From tss at iki.fi Tue Apr 3 09:32:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 09:32:37 +0300 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> Message-ID: <1333434757.21461.32.camel@innu> On Sun, 2012-04-01 at 22:54 +0100, Spyros Tsiolis wrote: > However, I have this nagging problem with most of the thunderbird > MTAs. When I try to delete some "test" messages I sent to make sure > that everything works, it comes back with the following message on > the bottom of the Thunderbird window : > > "The current operation on InBox did not succeed. > The mail server for account "Account at bloomingdomain.gr" > responded [TRYCREATE] mailbox doesn't exist : Trash" That error probably happens when deleting a message. > Dovecot v1.2.15 AFAIK dovecot -n output would have helped. Mainly: do you have a namespace prefix or not? That's about the only thing I can think of related to that error message (e.g. dovecot has non-empty namespace prefix, while TB has empty namespace prefix). From tss at iki.fi Tue Apr 3 09:33:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 09:33:29 +0300 Subject: [Dovecot] Backtrace on mdbox index In-Reply-To: References: Message-ID: <1333434809.21461.33.camel@innu> On Mon, 2012-04-02 at 11:14 -0300, Francisco Wagner C. Freire wrote: > I current using dovecot with mdbox and on one account i get this error: > > doveadm(user at domain): Panic: file mail-index-sync-ext.c: line 209 > (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) Dovecot version? I think I've fixed this in recent v2.1.x. From tss at iki.fi Tue Apr 3 09:44:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 09:44:10 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F760B61.3090209@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> Message-ID: <1333435450.21461.35.camel@innu> On Fri, 2012-03-30 at 21:37 +0200, Patrick Westenberg wrote: > Nick Warr schrieb: > > > I think some of the new Dovecot (director?) software is user aware, but > > I don't know if it's quite ready for production. > > Yes, with director it should be something like that: > > MTA --(lmtp)--\ /--(lmtp)--> backend1 --\ > -- director -- -- NFS > MTA --(lmtp)--/ \--(lmtp)--> backend2 --/ > > > IMAP-User --> frontend1 --\ /--(imap)--> backend1 --\ > -- director -- -- NFS > IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/ The director above must be the same one for both IMAP and MTA. > So what happens if user1 at example.tld receives a mail? > > - The director decides to connect to backend1 which in turn stores > the mail on the NFS share and the index file locally? > > - Then, user1 at example.tld connects to one of the frontends. Does the > director know that, earlier, this user received a mail and proxies him > to backend1 too? Yes. Director process actually doesn't know anything about mail protocols, it simply does username -> backend mapping. The regular Dovecot IMAP/POP3/LMTP/doveadm proxying code handles the rest. From lukas.mueller at newmedia.ch Tue Apr 3 10:24:58 2012 From: lukas.mueller at newmedia.ch (=?iso-8859-1?Q?M=FCller_Lukas?=) Date: Tue, 3 Apr 2012 07:24:58 +0000 Subject: [Dovecot] Dovecot 1.2.9. next_uid was lowered Message-ID: Hi There A while ago I posted here about a crash of dovecot, that had to do with nfs and was quite certainly due to bad luck. But still one problem persists: We have a Mailbox, that is accessed from multiple users using IMAP. Occasionally all the Emails in the mailbox seem to be missing, when accessing the Mailbox through IMAP, although they're still there on the server. This seems to happen, if the users are accessing the Mailbox on Server1 and at the same time an email gets delivered to the mailbox on server2. Since it happens quite seldom, there must be other conditions or circumstances that have to be met for the problem to occur. Unfortunately I don't know which. Shortly after the mail gets delivered on server2, I get the following errors on server1: mail01 dovecot: IMAP(akzidenz at triner.ch): fdatasync(/data/vmail/example.com/user/dovecot-uidlist) failed: Input/output error mail01 dovecot: IMAP(akzidenz at triner.ch): /data/vmail/example.com/user/dovecot-uidlist: next_uid was lowered (2143 -> 2142, hdr=2142) The new mail is than missing in the uidlist and as long as no new mail arrives all folders of the mailbox appear to be empty. If a new mail arrives, it will get the uid that should have been given to the pervious mail (the one that caused the error). The previous mail still won't be listed in the uidlist, but everything works as expected. Best regards Lukas M?ller Systems Engineer _______________________________________________ NEWMEDIA S?dostschweiz Newmedia AG http://www.newmedia.ch _______________________________________________ TYPO3 & Drupal - Wir wissen wie. Ihre professionelle Web Agentur in Chur, Ilanz, Glarus und Z?rich. From lee at standen.id.au Tue Apr 3 11:02:05 2012 From: lee at standen.id.au (Lee Standen) Date: Tue, 3 Apr 2012 16:02:05 +0800 Subject: [Dovecot] Dovecot 1.2.9. next_uid was lowered In-Reply-To: References: Message-ID: I'll just say what I'm sure someone else will shortly :) The problem is NFS: http://wiki2.dovecot.org/NFS The solution is 2.x Director: http://wiki2.dovecot.org/Director Although it looks like you're currently on 1.x so it'd be a fairly significant upgrade for you. On Tue, Apr 3, 2012 at 3:24 PM, M?ller Lukas wrote: > Shortly after the mail gets delivered on server2, I get the following > errors on server1: > mail01 dovecot: IMAP(akzidenz at triner.ch): fdatasync(/data/vmail/ > example.com/user/dovecot-uidlist) failed: Input/output error > mail01 dovecot: IMAP(akzidenz at triner.ch): /data/vmail/ > example.com/user/dovecot-uidlist: next_uid was lowered (2143 -> 2142, > hdr=2142) > > From CMarcus at Media-Brokers.com Tue Apr 3 14:33:39 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 03 Apr 2012 07:33:39 -0400 Subject: [Dovecot] dsync redesign In-Reply-To: <87zkatpvxr.fsf@algae.riseup.net> References: <936E95DA-E032-4F8F-A323-DEBB7AAC0E1B@iki.fi> <87obrixcyp.fsf@algae.riseup.net> <4F721650.4030901@Media-Brokers.com> <87zkatpvxr.fsf@algae.riseup.net> Message-ID: <4F7AE013.6040402@Media-Brokers.com> On 2012-04-02 7:15 PM, Micah Anderson wrote: > Charles Marcus writes: >> On 2012-03-27 11:47 AM, Micah Anderson wrote: >>> One would be the ability to perform *intelligent* incremental / >>> rotated backups. I can do this now by running a dsync backup >>> operation and then doing manual hardlinking or moving of the >>> backup directories (daily.1, daily.2, weekly.1, monthly.1, etc.), >>> but it would be more intelligent if this were baked into the >>> backup process. >> There are already numerous tools that do this flawlessly - I've >> beenusing rsnapshot (which uses rsync) for this for years. > Are you snapshotting your filesystem (using LVM, or SAN, or similar) > before doing rsnapshot? Because if you aren't then rsync will not > assuredly get everything in a consistent state. No, and you are correct... but I run it in the middle of the night, and the system is only barely utilized at the time, so the very minor inconsistencies are not a problem overall. I will, however, be changing this to using FS snapshots once I get my mailserver virtualized (already being planned for when our new office location comes online), so that will allow me to perform snapshots multiple times during the day (I'm thinking 4 times per day will be enough). >> I don't know if Timo should be spending his time reinventing the >> wheel. > dsync backup is already here, and it is quite useful. I'm not saying it isn't, I'm just saying that there are already *plenty* of different backup tools, and I don't see the sense in Timo spending lots of time on creating a new one just for dovecot. His time would be better spent just making it easier for any other backup tool to work better. >> Although, one interesting piece that I am hopeful I'll be able to >> implement soon (with Timo's professional help) is the ability to >> easily and automatically map my rsnapshot snapshots directory to a >> read-only 'Backups' namespace that automatically shows the >> snapshots by date and time as they are produced. This way users >> could 'go back in time' anytime they wanted without having to call >> me... :) > Interesting idea, would be a great one to share with the community > if you decide to do so. Absolutely - that is already on my list for when I pay Timo's company to do this - document it on the wiki. Hopefully if any code changes are needed to make it work right, they will be minor. -- Best regards, Charles From xnasx at yandex.ru Tue Apr 3 15:33:11 2012 From: xnasx at yandex.ru (xnasx) Date: Tue, 03 Apr 2012 16:33:11 +0400 Subject: [Dovecot] Courier to Dovecot migration script issue Message-ID: <125901333456391@web125.yandex.ru> Hi! Forgive my bad English. I trying to migrate to Dovecot 2.0.18 from Courier 4.10. I using a script 'courier-dovecot-migrate.pl' from oficial wiki - it works without errors and creating dovecot-uidlist, etc. . I running Dovecot on a different port: 11149. When I connecting to port 11149 from Thunderbird, all my message headers are reloading. Please help me to solve this issue. Best regards, Victor From tss at iki.fi Tue Apr 3 16:00:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 16:00:47 +0300 Subject: [Dovecot] Per-user flags/indexes for shared mailboxes Message-ID: <1333458047.21461.48.camel@innu> The attached patch adds support for doing e.g.: namespace { type = public prefix = Public/ location = mdbox:/var/lib/public/mdbox:INDEXPVT=~/mdbox/public list = yes } So the private flags are stored in indexes under ~/mdbox/public/, while the shared flags are stored under /var/lib/public/mdbox/. The patch currently hardcodes the shared flags as only MAIL_SEEN. I'm not really sure what would be the best place to configure which flags are private. Perhaps a per-namespace default, but how to configure it per-mailbox? Also per-user keywords aren't currently supported. And it's a little bit annoying that each private flag update must increase the shared modseq for the message, but that can't really be helped. (Alternative would be for each user to have their own private modseqs, which could get difficult.) I'll probably commit this to v2.1 after some more testing. -------------- next part -------------- A non-text attachment was scrubbed... Name: private-index.diff Type: text/x-patch Size: 28899 bytes Desc: not available URL: From feltrin at gmail.com Tue Apr 3 17:17:59 2012 From: feltrin at gmail.com (Jean Michel) Date: Tue, 3 Apr 2012 11:17:59 -0300 Subject: [Dovecot] Backtrace on mdbox index In-Reply-To: <1333434809.21461.33.camel@innu> References: <1333434809.21461.33.camel@innu> Message-ID: I tried with dovecot 2.1.3 and got the same error, any ideia ? 2012/4/3 Timo Sirainen > On Mon, 2012-04-02 at 11:14 -0300, Francisco Wagner C. Freire wrote: > > > I current using dovecot with mdbox and on one account i get this error: > > > > doveadm(user at domain): Panic: file mail-index-sync-ext.c: line 209 > > (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) > > Dovecot version? I think I've fixed this in recent v2.1.x. > > > -- ------------------------------ Jean Michel Feltrin From tss at iki.fi Tue Apr 3 17:37:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 17:37:26 +0300 Subject: [Dovecot] Backtrace on mdbox index In-Reply-To: References: <1333434809.21461.33.camel@innu> Message-ID: <1333463846.21461.50.camel@innu> Could you privately send me your all of your dovecot.index, dovecot.index.log, dovecot.map.index and dovecot.map.index.log files under mdbox? None of those contain any sensitive data. On Tue, 2012-04-03 at 11:17 -0300, Jean Michel wrote: > I tried with dovecot 2.1.3 and got the same error, any ideia ? > > 2012/4/3 Timo Sirainen > > > On Mon, 2012-04-02 at 11:14 -0300, Francisco Wagner C. Freire wrote: > > > > > I current using dovecot with mdbox and on one account i get this error: > > > > > > doveadm(user at domain): Panic: file mail-index-sync-ext.c: line 209 > > > (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) > > > > Dovecot version? I think I've fixed this in recent v2.1.x. > > > > > > > > From tss at iki.fi Tue Apr 3 17:40:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 17:40:34 +0300 Subject: [Dovecot] imapdir still a configuration option? In-Reply-To: <4F7A18C8.3000904@se.rit.edu> References: <4F7A18C8.3000904@se.rit.edu> Message-ID: <1333464034.21461.51.camel@innu> On Mon, 2012-04-02 at 17:23 -0400, Kurt Mosiejczuk wrote: > I've been looking at moving an old IMAPdir-based solution to dovecot. I > came across references that dovecot (to my pleasant surprise) added > imapdir support in 1.1. However, in trying 2.0.13 (from an OpenBSD > package) things start okay, but I see in the logs: > > "Initialization failed: Initializing mail storage from mail_location > setting failed: Unknown mail storage driver imapdir" > > Poking around in the sources, I still see lots of references to imapdir, > but see very little in the documentation. > > Is IMAPdir still supported in 2.0.x (or even 2.1.x)? I'd prefer not to > have to migrate users via renaming, especially since they like the > IMAPdir layout for their own searching. mail_location = maildir:~/Maildir:LAYOUT=imapdir From tss at iki.fi Tue Apr 3 17:43:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 17:43:50 +0300 Subject: [Dovecot] Connection queue full on dovecot 2.0.13 In-Reply-To: <4F79BA0C.1070104@libero.it> References: <4F79BA0C.1070104@libero.it> Message-ID: <1333464230.21461.53.camel@innu> On Mon, 2012-04-02 at 16:39 +0200, Giovanni Mancuso wrote: > Apr 02 14:34:36 imap-login: Info: Disconnected: *Connection queue full > *(auth failed, 1 attempts): user= I see this wiki page: http://wiki2.dovecot.org/LoginProcess but i read: > > It works by using a number of long running login processes, each > handling a number of connections. This loses much of the security > benefits of the login process design, because in case of a security hole > (in Dovecot or SSL library) the attacker is now able to see other users > logging in and steal their passwords, read their mails, etc. > > Is there another way? Just increase the number of processes: service imap-login { process_limit = 10000 } From kurt-dovecot at se.rit.edu Tue Apr 3 17:50:59 2012 From: kurt-dovecot at se.rit.edu (Kurt Mosiejczuk) Date: Tue, 03 Apr 2012 10:50:59 -0400 Subject: [Dovecot] imapdir still a configuration option? In-Reply-To: <1333464034.21461.51.camel@innu> References: <4F7A18C8.3000904@se.rit.edu> <1333464034.21461.51.camel@innu> Message-ID: <4F7B0E53.50601@se.rit.edu> Timo Sirainen wrote: > mail_location = maildir:~/Maildir:LAYOUT=imapdir Excellent. That works *perfectly*. --Kurt From stsiol at yahoo.co.uk Tue Apr 3 17:55:54 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 3 Apr 2012 15:55:54 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333434757.21461.32.camel@innu> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333434757.21461.32.camel@innu> Message-ID: <1333464954.38480.YahooMailNeo@web132206.mail.ird.yahoo.com> Hi Timo,? >________________________________ > From: Timo Sirainen >To: Spyros Tsiolis ? >Cc: Dovecot ? >Sent: Tuesday, 3 April 2012, 8:32 >Subject: Re: [Dovecot] Mail migration troubles >? >On Sun, 2012-04-01 at 22:54 +0100, Spyros Tsiolis wrote: >> However, I have this nagging problem with most of the thunderbird >> MTAs. When I try to delete some "test" messages I sent to make sure >> that everything works, it comes back with the following message on >> the bottom of the Thunderbird window : >>? >> "The current operation on InBox did not succeed.? >> The mail server for account "Account at bloomingdomain.gr"? >> responded [TRYCREATE] mailbox doesn't exist : Trash" > >That error probably happens when deleting a message. Yes, it does Timo. Thank you. >> Dovecot v1.2.15 AFAIK Actually it's v1.2.16 >dovecot -n output would have helped. Mainly: do you have a namespace >prefix or not? That's about the only thing I can think of related to >that error message (e.g. dovecot has non-empty namespace prefix, while >TB has empty namespace prefix). Why do I keep forgetting this ? Damn the gods :-) !@#$ :-) Here's the output of "dovecot -n" : -------------------------------------------------------------------- [root at mailgate ~]# dovecot -n # 1.2.16: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 base_dir: /var/run/dovecot/ log_path: /var/log/dovecot/dovecot.log info_log_path: /var/log/dovecot/dovecot-info.log ssl_parameters_regenerate: 48 verbose_ssl: yes login_dir: /var/run/dovecot//login login_executable: /usr/local/dovecot/libexec/dovecot/imap-login login_greeting: * Dovecot ready * login_max_processes_count: 64 mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir mail_plugins: zlib auth default: ? verbose: yes ? debug: yes ? debug_passwords: yes ? passdb: ? ? driver: passwd-file ? ? args: /etc/dovecot/passwd ? passdb: ? ? driver: pam ? userdb: ? ? driver: static ? ? args: uid=vmail gid=vmail home=/home/vmail/%u ? userdb: ? ? driver: passwd [root at mailgate ~]#? -------------------------------------------------------------------- I suspect it's some freak thunderbird bug. It's the only explanation. I've migrated about 20 mailboxes and it's the only one acting up. Thanks again Timo, s. ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From stsiol at yahoo.co.uk Tue Apr 3 18:06:42 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 3 Apr 2012 16:06:42 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333464954.38480.YahooMailNeo@web132206.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333434757.21461.32.camel@innu> <1333464954.38480.YahooMailNeo@web132206.mail.ird.yahoo.com> Message-ID: <1333465602.15453.YahooMailNeo@web132203.mail.ird.yahoo.com> Also, What do you mean by namespace ? I am sorry I didn't get that one : >> dovecot -n output would have helped. Mainly: do you have a namespace >> prefix or not? That's about the only thing I can think of related to >> that error message (e.g. dovecot has non-empty namespace prefix, while >> TB has empty namespace prefix). I missed the "namespace" thing. s. ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From nmilas at noa.gr Tue Apr 3 18:33:59 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Tue, 03 Apr 2012 18:33:59 +0300 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333465602.15453.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333434757.21461.32.camel@innu> <1333464954.38480.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333465602.15453.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <4F7B1867.3030608@noa.gr> On 3/4/2012 6:06 ??, Spyros Tsiolis wrote: > What do you mean by namespace ? See: http://wiki2.dovecot.org/Namespaces Best regards, Nick From trashcan at odo.in-berlin.de Tue Apr 3 20:29:41 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Tue, 3 Apr 2012 19:29:41 +0200 Subject: [Dovecot] dovecot and unison In-Reply-To: <4F7A2FF9.7000204@klacto.net> References: <4F7A2FF9.7000204@klacto.net> Message-ID: <8EAB42AF-BC4A-4235-9CF9-A444BE553B10@odo.in-berlin.de> Hi -- [Sorry for my private response, instead of public one] On 03.04.2012, at 01:02, FZiegler wrote: > Now I'd like if possible, to replicate this setup on my laptop and keep > both in sync with unison (http://www.cis.upenn.edu/~bcpierce/unison/), > which I am already using to sync much of my home dir about once a day. > > I found at least one positive message regarding this topic > (http://dovecot.org/list/dovecot/2010-April/048092.html), but I feel I > could use some more advice. I used to sync my two server's maildirs for some years, but I moved to dsync mirroring some months ago, and now I'm into using replicator/dsync (see archive of March 2012). Unsion mirroring is working in general, but sometimes you might realize some duplicates being produced. Not bad, but it happens. I did realize the same with using dsync mirror instead. It's much better with the new replicator/dsync mirroring scheme, but it's not perfect, yet. (Again, see ML archives of the last weeks). > Namely, which of the ancillary files should I be syncing (or perhaps > deleting before sync)? In addition to the mails themselves I'm seeing: > > -rw------- 1 fz 501 13 Apr 1 17:10 dovecot-keywords > -rw------- 1 fz 501 2483 Apr 1 17:10 dovecot-uidlist > -rw------- 1 fz 501 1040 Mar 29 14:50 dovecot.index > -rw------- 1 fz 501 44032 Apr 1 17:10 dovecot.index.cache > -rw------- 1 fz 501 9772 Apr 1 17:10 dovecot.index.log > > and at the top level (in .local/var/Maildir/): > > -rw------- 1 fz 501 8 Apr 1 17:10 dovecot-uidvalidity > -r--r--r-- 1 fz 501 0 Apr 1 17:10 dovecot-uidvalidity.4efb323d > -rw------- 1 fz 501 3288 Apr 1 17:10 dovecot.mailbox.log > -rw------- 1 fz 501 4104 Mar 28 10:53 dovecot.mailbox.log.2 I used ... ignore = Name dovecot* ignore = Regex .*/tmp$ ignore = Regex .*/\.Drafts$ ? to ignore all of them. When runnig mirroring for the very first time, I started with one empty maildir at one server, IIRC. You better check that with a test maildir beforehand. > -rw------- 1 fz 501 5114 Apr 1 17:10 subscriptions That became mirrored. Regards, Michael From dave-lists-dovecot at weller-fahy.com Tue Apr 3 21:44:32 2012 From: dave-lists-dovecot at weller-fahy.com (David J. Weller-Fahy) Date: Tue, 3 Apr 2012 14:44:32 -0400 Subject: [Dovecot] Dovecot stones In-Reply-To: References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <20120403184431.GA792@weller-fahy.com> * Mark Moseley [2012-04-02 12:41 -0400]: > On Sat, Mar 31, 2012 at 2:28 PM, Timo Sirainen wrote: > > For the last few days I've been thinking about my company and what > > > >... > > I'm slightly concerned that there's been no mention of what license > these stones are going to be released under. GPL2? GPL3? Apache? I'm > just hoping these aren't some sort of "open core" stones that will > only work for basic features but that I'll end up needing to buy > "Enterprise-grade stones" to cover large clusters. From the description and images, I'd say they are probably binary blobs to be included in the firmware via hardware dongle. ;) -- dave [ please don't CC me ] -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From dm-list-email-dovecot at scs.stanford.edu Tue Apr 3 22:22:02 2012 From: dm-list-email-dovecot at scs.stanford.edu (dm-list-email-dovecot at scs.stanford.edu) Date: Tue, 03 Apr 2012 12:22:02 -0700 Subject: [Dovecot] dovecot and unison In-Reply-To: <4F7A2FF9.7000204@klacto.net> References: <4F7A2FF9.7000204@klacto.net> Message-ID: <87398kei3p.wl@ta.scs.stanford.edu> At Mon, 02 Apr 2012 19:02:17 -0400, FZiegler wrote: > > I am successfully using dovecot purely as a personal local mail store on > my desktop. (There is only one account, and it's only ever accessed by > local mail clients on the machine. The point is to have a common store I > can use with any client; plus, I prefer dovecot's Mailbox storage to > Thunderbird's mboxes.) > > Now I'd like if possible, to replicate this setup on my laptop and keep > both in sync with unison (http://www.cis.upenn.edu/~bcpierce/unison/), > which I am already using to sync much of my home dir about once a day. > > I found at least one positive message regarding this topic > (http://dovecot.org/list/dovecot/2010-April/048092.html), but I feel I > could use some more advice. I have a similar setup, but I use offlineimap instead of unison: http://offlineimap.org/ It seems to work pretty well. That's not to say that unison wouldn't work as well also. However, offlineimap has the advantage that it doesn't restrict you to a star topology. You can, for instance, since to your laptop at work and from your laptop at home. Note that offlineimap is slow if you don't use imap at both ends. Therefore, I use it on the local end. A simplified excerpt of my .offlineimaprc looks like this: ======== [general] accounts = DefaultAccount [Account DefaultAccount] localrepository = MyLocal remoterepository = MyRemote [Repository MyRemote] type = IMAP preauthtunnel = ssh -qax -oBatchMode=yes -oServerAliveInterval=60 MY-MAIL-SERVER 'exec env CONFIG_FILE=/PATH/TO/PRIVATE/dovecot.conf /usr/lib/dovecot/imap' [Repository MyLocal] type = IMAP preauthtunnel = CONFIG_FILE=$HOME/etc/dovecot.conf /usr/lib/dovecot/imap ======== Unfortunately, in dovecot 2.1, the full text search no longer seems to work in pre-auth mode, but I don't think that has anything to do with offlineimap. I think maybe dovecot is deprecating pre-auth mode or requires a more complicated setup. From tss at iki.fi Wed Apr 4 03:45:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 03:45:36 +0300 Subject: [Dovecot] Backtrace on mdbox index In-Reply-To: References: Message-ID: <7F642313-D6F2-4032-A913-63E9BDAB6848@iki.fi> On 2.4.2012, at 17.14, Francisco Wagner C. Freire wrote: > I current using dovecot with mdbox and on one account i get this error: > > doveadm(user at domain): Panic: file mail-index-sync-ext.c: line 209 > (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) Fixed: http://hg.dovecot.org/dovecot-2.1/rev/47526bf86c57 From tss at iki.fi Wed Apr 4 03:48:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 03:48:12 +0300 Subject: [Dovecot] 2.1.3: doveadm mailbox delete lost the -s option In-Reply-To: <874nt1rcf4.fsf@algae.riseup.net> References: <874nt1rcf4.fsf@algae.riseup.net> Message-ID: On 3.4.2012, at 1.34, Micah Anderson wrote: > Looks like the recently added -s (for unsubscribing) option to doveadm > mailbox delete went missing: > > root at vireo# /usr/bin/doveadm mailbox delete -u micahtest -s restored > delete: invalid option -- 's' > doveadm mailbox delete [-u |-A] [-S ] [-s] [...] > > although the option still is shown as valid :) Fixed: http://hg.dovecot.org/dovecot-2.1/rev/411344f9daf3 From tss at iki.fi Wed Apr 4 03:56:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 03:56:05 +0300 Subject: [Dovecot] dovecot 2.1 breaks FTS + pre-auth? In-Reply-To: <878vigok53.wl@ta.scs.stanford.edu> References: <878vigok53.wl@ta.scs.stanford.edu> Message-ID: <335CB623-DFF7-4676-AA0E-53D7AD42E5C0@iki.fi> On 31.3.2012, at 18.38, dm-list-email-dovecot at scs.stanford.edu wrote: > Hi. I use dovecot in the simplest possible way, as an IMAP server in > pre-auth mode over ssh or just locally over a unix-domain socket > (e.g., with offlineimap, which runs much faster using dovecot for the > local message store). Ideally I would like to avoid running any extra > daemons or setting up anything as root. Until recently, this has > worked fine by just setting the CONFIG_FILE environment variable to > something in my home directory. You can run Dovecot without root: http://wiki2.dovecot.org/HowTo/Rootless > Full text search used to work just fine with this configuration, and > still does on a machine I have running dovecot 2.0.13. However, on > the machine with 2.1, I get errors about /var/run/dovecot/index not > existing. > > $ printf "a select INBOX\nb search text xyzzy\nc logout\n" \ > | /usr/lib/dovecot/imap > * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE] Logged in as dm > imap(dm): Error: net_connect_unix(/var/run/dovecot/indexer) failed: No such file or directory I looked at the code and looks like there is no easy way to make it work the old way. The new way is the only way to make fts-lucene work reliably, and it's also better for fts-solr. And fts-squat is really on its way out to die. From tss at iki.fi Wed Apr 4 03:57:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 03:57:33 +0300 Subject: [Dovecot] Is it possible to migrating mail to dovecot using imapc? In-Reply-To: <695D3A65-CFBF-4DC3-9DAC-E0C299ED0E6D@tao.org.uk> References: <72230FBC-F639-415E-AD1D-7D49CE4C6287@tao.org.uk> <07E5D123-E233-4EB5-B5F8-99B90AF6A809@tao.org.uk> <76BAD212-0807-4EA6-9211-4C3D4F6518CC@iki.fi> <2F722E50-B9AC-4660-AAC2-39FBEEE14364@iki.fi> <25DE9E4A-478B-4D70-8B65-0A727B0DEFAF@tao.org.uk> <9F05AEAA-6D32-417E-85F2-240FF39D0426@iki.fi> <695D3A65-CFBF-4DC3-9DAC-E0C299ED0E6D@tao.org.uk> Message-ID: <1C12148A-959F-453C-AA98-15ED001F53B6@iki.fi> On 30.3.2012, at 23.42, Dr Josef Karthauser wrote: > % dsync -Dv -u joe at local.com -o imapc_host=mail.remoteimap.com -o imapc_port=143 -o imapc_username=joe@ remoteimap.com -o imapc_password='somepass' mirror imapc: > & /tmp/output > > produced the following output. > dsync(joe at email.com): Error: Mailbox INBOX changed its GUID (f38c263a4919764f1b560100c516a7a4 -> c92f64f79f0d1ed01e6d5b314f04886c) dsync + imapc requires indexes. So you need to use e.g. imapc:/tmp/dsync-imapc From tss at iki.fi Wed Apr 4 04:05:52 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 04:05:52 +0300 Subject: [Dovecot] Many messages clustered around the same date.saved value In-Reply-To: References: Message-ID: <31C97BDA-5B4E-4289-9087-9AC72F11F0F5@iki.fi> On 29.3.2012, at 5.41, Joseph Tam wrote: >> Ah, with mbox there isn't any usable fallback for date.saved. If it's >> not in dovecot.index.cache, the current time is used. > > I'm a little confused as to why it needed a fallback. In other words, > why wasn't date.saved put into the index as soon as the IMAP operation > copied it into "Trash"? > > If this data isn't set at that time, when does it get instantiated? > When I actually ask for it? Well..: - date.saved is stored only in dovecot.index.cache file - if it doesn't exist and is requested, the current time is returned and it's added to the cache - when date.saved has already fetched once (so it already exists in dovecot.index.cache file), and mail is saved via LDA/IMAP then it gets added there immediately when saving - dovecot.index.cache has caching decisions, and some old/unused fields may get dropped from it once in a while - maybe due to some bugs or whatever, the fields or the entire cache may get dropped for some other reason So it probably should have worked, but for some reason didn't. It would be possible to store date.saved in dovecot.index file, like mdbox does, so cache decisions wouldn't matter. But probably too much trouble to be worth it, very few mbox installations care about it. From tss at iki.fi Wed Apr 4 04:08:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 04:08:46 +0300 Subject: [Dovecot] Dovecot upgrade from 1.2.x to 2.0.x: roundcube/squirrelmail sent folder doesn't work any more In-Reply-To: References: <4F71F8D8.6040700@wiesinger.com> <95C664AD-0532-4F3B-A2EB-2FD25B79187B@iki.fi> Message-ID: On 29.3.2012, at 8.25, Gerhard Wiesinger wrote: >>> 2.0.x: with Prefix ~/Mail >>> A0003 LIST "" ~/Mail/sent >>> A0003 OK List completed.: >> > > mail_full_filesystem_access = yes > mail_location = mbox:~:INBOX=/var/mail/%u Yes, the problem happens only with mail_full_filesystem_access=yes. It already works in v2.1, so I don't think I'll bother looking into why it's not working in v2.0. From tss at iki.fi Wed Apr 4 04:11:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 04:11:04 +0300 Subject: [Dovecot] File/folder permission issues in 2.1.3 In-Reply-To: <8B296F70-22B8-487B-AD7A-47BEB8C84F62@roessner-network-solutions.com> References: <8B296F70-22B8-487B-AD7A-47BEB8C84F62@roessner-network-solutions.com> Message-ID: On 29.3.2012, at 13.24, Christian R??ner wrote: > I figured out that Dovecot does not honer secondary groups with auth/auth-worker (??), if doing LDAP/TLS stuff. I had to use file system acls to add the user "vmail" to /etc/ssl/private and to the corresponding key file: service auth { extra_groups = any groups you want auth process to have } Don't give secondary groups to any Dovecot related users. From tss at iki.fi Wed Apr 4 04:35:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 04:35:13 +0300 Subject: [Dovecot] Dovecot allows creation of folders outside of a user's directory In-Reply-To: <4F759B08.1060603@in.tum.de> References: <4F759B08.1060603@in.tum.de> Message-ID: On 30.3.2012, at 14.37, Christoph Bu?enius wrote: > in our dovecot 2.0 setup with shared folders, users can make dovecot create directories outside their mail directory. Which is a bit scary imho. > > The following command: > > . create inbox.shared.abc123 > > or even > > . create "inbox.shared.strange &ANY- characters" > > -- even though it will fail with a "permission denied" error -- will create a directory like "/mail/users/strange &ANY- characters". That directory will only contain a subdirectory "Maildir" and therein dovecot-acl-list. Fixed: http://hg.dovecot.org/dovecot-2.0/rev/b15889b82258 From tss at iki.fi Wed Apr 4 04:47:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 04:47:33 +0300 Subject: [Dovecot] namespace hierarchy In-Reply-To: <201204010928.41017.anyaddress@gmx.net> References: <201204010928.41017.anyaddress@gmx.net> Message-ID: On 1.4.2012, at 10.28, Tom Fernandes wrote: > I'm migrating from Courier to Dovecot and would like to keep the original > namespace and add two new ones on the new server. > > There's only one namespace in Courier which is "INBOX." . The seperator is "." > > In the new server I would like to have 3 namespaces (private, public, shared) > and use "/" as separator. Is it possible to keep "INBOX." for private, so that > the users mailboxes appear below the Inbox, but have public and shared on the > same hierarchy-level like the Inbox? > > When I set prefix to "INBOX/" for private I get the users mailboxes below the > Inbox, like I have it Courier now. But then at least the public, probably also > the shared namespace doesn't work any more. I get "unknown subscription > namespace" when trying to subscribe to some of the folders. If you have any subscriptions=no namespaces, you need a parent namespace that has subscriptions=yes. You could have e.g.: namespace { prefix = INBOX/ list = no inbox = yes } namespace { prefix = list = no hidden = yes alias_for = INBOX/ } So now the prefix="" is used for the subscriptions but nothing else really. Anyway, INBOX/ isn't the same as INBOX. so if any client configs are using INBOX. as namespace prefix then they don't work with INBOX/. From tss at iki.fi Wed Apr 4 04:49:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 04:49:48 +0300 Subject: [Dovecot] imapsync, namespaces and Dovecot 2.x In-Reply-To: References: Message-ID: <65D3A596-EC48-430C-8F53-4A11F30437C4@iki.fi> On 2.4.2012, at 19.11, Aleix Dorca wrote: > I recently tried to migrate from Exchange using imapsync. It didn't work because it complained that dovecot had no namespace capablity. After looking through Dovecot's config files I found that namespaces could be defined, but if you did not a 'private' default one was created. The problem is that Dovecot doesn't advertise NAMESPACE capability before client has logged in. The better solution would be to change imapsync to figure this out automatically. The other solution would be to set in Dovecot: imap_capability = +NAMESPACE From tss at iki.fi Wed Apr 4 05:35:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 05:35:33 +0300 Subject: [Dovecot] lmtp sometimes fails to deliver a message to all recipients In-Reply-To: <4F79C995.9060607@talex.pl> References: <4F79C995.9060607@talex.pl> Message-ID: <1B1369FF-6748-46DE-AB94-AF2F32061757@iki.fi> On 2.4.2012, at 18.45, Artur Zaprza?a wrote: > Running strace on lmtp reveals that when lmtp delivers a message to multiple recipients it saves the message to a file in the first mailbox and hard links that file to other mailboxes. But when in the meantime of delivery that file is deleted, lmtp fails to deliver the message to remaining mailboxes. For example a message for 46 recipients is delivered by lmtp to the first 12 only (fortunately postfix retries delivery for remaining recipients) and for the rest lmtp logs the following error messages: > > lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Message was expunged (guid) > lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: script /vmail/domain/foo/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /vmail/domain/foo/.dovecot.sieve.log may reveal additional details) Fixed in hg. From tss at iki.fi Wed Apr 4 05:49:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 05:49:40 +0300 Subject: [Dovecot] dovecot-auth restaring and caching In-Reply-To: <4F6C696C.5030900@um.es> References: <4F6AF72E.9030206@um.es> <7B9D052D-5864-42A9-AE9A-6FCE858F48C0@iki.fi> <4F6C696C.5030900@um.es> Message-ID: On 23.3.2012, at 14.15, Angel L. Mateo wrote: > El 22/03/12 19:57, Timo Sirainen escribi?: >> On 22.3.2012, at 11.55, Angel L. Mateo wrote: >> >>> The problem I'm having is that if I have no activity in the server, dovecot stops its auth process and when another message is received, it restarted it, but with an empty cache. >> >> service auth { >> idle_kill = 0 >> } >> > > In a test server I have, this have solved the problem. In my productions servers it is still being restarted. Could it be another parameter involve in this? Oops, I remembered wrong. idle_kill=0 uses default_idle_kill limit. You could set for example: idle_kill = 4294967295s From tss at iki.fi Wed Apr 4 05:51:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 05:51:19 +0300 Subject: [Dovecot] dovecot 2.0.19 Panic: file mail-index-sync-ext.c: line 209 (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) In-Reply-To: <4F6A1BA7.7030208@uvm.edu> References: <4F69DD6E.1090502@uvm.edu> <94A1158A-2F42-406A-9212-C58A4D4FF879@iki.fi> <4F6A1BA7.7030208@uvm.edu> Message-ID: <18B56699-7AF4-4ED8-A21F-4EBA9E7C6752@iki.fi> On 21.3.2012, at 20.19, Jim Lawson wrote: > On 3/21/12 10:02 AM, Timo Sirainen wrote: >> On 21.3.2012, at 15.53, Jim Lawson wrote: >> >>> Had a user who couldn't access his INBOX: >>> >>>> Mar 21 09:21:17 penguina dovecot: imap([USER]): Panic: file >>>> mail-index-sync-ext.c: line 209 (sync_ext_reorder): assertion fai >>>> led: (offset < (uint16_t)-1) >> I kind of remember that this was fixed by http://hg.dovecot.org/dovecot-2.1/rev/b4d8e950eb9d but I'm not entirely sure. I guess I should have included in the commit the error message it fixed. > > > This applies cleanly against 2.0.19; should I try it on that version, or > not recommended? http://hg.dovecot.org/dovecot-2.1/rev/47526bf86c57 should fix this. From tss at iki.fi Wed Apr 4 05:57:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 05:57:03 +0300 Subject: [Dovecot] Courier to Dovecot migration script issue In-Reply-To: <125901333456391@web125.yandex.ru> References: <125901333456391@web125.yandex.ru> Message-ID: <9BBB03B5-4742-403B-8720-FBEFB554266D@iki.fi> On 3.4.2012, at 15.33, xnasx wrote: > I trying to migrate to Dovecot 2.0.18 from Courier 4.10. > I using a script 'courier-dovecot-migrate.pl' from oficial wiki - it works without errors and creating dovecot-uidlist, etc. . > > I running Dovecot on a different port: 11149. > When I connecting to port 11149 from Thunderbird, all my message headers are reloading. > > Please help me to solve this issue. telnet localhost 143 a login user pass b select inbox c status inbox (uidvalidity) d fetch 1:* uid telnet localhost 11149 a login user pass b select inbox c status inbox (uidvalidity) d fetch 1:* uid Check if the "c" and "d" outputs are the same for both servers. If they are, then the migration was successful and the problem is something else, such as maybe the client always redownloading messages because you changed the port (or something else). From tss at iki.fi Wed Apr 4 06:16:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 06:16:54 +0300 Subject: [Dovecot] ldap userdb warning in v2.1.1 In-Reply-To: References: <4F69989A.3000106@noa.gr> Message-ID: On 21.3.2012, at 14.06, Timo Sirainen wrote: > On 21.3.2012, at 11.00, Nikolaos Milas wrote: > >> Mar 21 10:07:23 imapserver dovecot: master: Dovecot v2.1.1 starting up (core dumps disabled) >> Mar 21 10:08:17 imapserverdovecot: auth: Warning: ldap: Ignoring changed user_attrs in /etc/dovecot/dovecot-passdb-ldap.conf, because userdb ldap not used. (If this is intentional, set userdb_warning_disable=yes) >> >> I didn't see such warnings in 2.0.13. >> >> I guess I should/could remove the "user_attrs" line from dovecot-passdb-ldap.conf because it's not needed? > > Hmm. Yes, if dovecot-usrdb-ldap.conf is a separate file from dovecot-passdb-ldap.conf you can just remove it. But this reminds me that in several places I've suggested to make one of them a symlink to the other, and you can't really do it then. Perhaps I'll need to remove this warning, or maybe make it recognize the symlink case. Removed, at least for now: http://hg.dovecot.org/dovecot-2.1/rev/324df4134049 From tss at iki.fi Wed Apr 4 06:25:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 04 Apr 2012 06:25:51 +0300 Subject: [Dovecot] Dovecot 2.1 with custom OpenSSL fails to build In-Reply-To: References: <1331816286.10319.23.camel@innu.invalid> Message-ID: <4F7BBF3F.9060103@iki.fi> On 03/15/2012 11:43 PM, Andreas M. Kirchwitz wrote: > Compilation works. Great! The binaries find all their libraries. > > But two libraries are not quite okay. They don't find their SSL libs: > > libdovecot-lda.so > libdovecot-storage.so > > Since libdovecot-lda.so doesn't contain the words libssl or libcrypto, > I guess that ldd just complains because it uses libdovecot-storage.so. > Thus, libdovecot-storage.so is the (only) one left with an incomplete > library search path. > > Luckily, all binaries use some additional libraries which come with > a proper library path. So the whole things works, but it's more like > some kind of magic. It would be great if libdovecot-storage.so could > be fixed as well to make things finally perfect. Maybe this fixes it? http://hg.dovecot.org/dovecot-2.1/rev/8b91367bc3e1 From tss at iki.fi Wed Apr 4 06:31:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 06:31:07 +0300 Subject: [Dovecot] Dovecot and scalable database storage In-Reply-To: References: Message-ID: <7CD2C788-D258-458B-A915-1360B861DEE6@iki.fi> On 23.3.2012, at 4.36, wrote: > I saw some interesting mails from TImo back in 2009 talking about the idea > of using something like Cassandra db or similar as a storage platform for > both email and index/logs. I was wondering if this has been discussed > since then, and if there are any plans to support something like this in > the future? I have been playing with Cassandra and found that their > RackAwareStrategy gives you the ability to replicate writes to as many > nodes as you would like, but more importantly what nodes and one of those > nodes could be defined by what rack it lives in or what data center it > lives in. This means multiple sites high available storage clusters, > seemingly a system that dovecot could benefit from in terms of performance > and redundancy and simplicity. Any takers? There are still plans, but not in near future. I'm still planning on implementing it by first making Dovecot's code simpler and more asynchronous, and then the key-value database support could be implemented in just a few hours. From tss at iki.fi Wed Apr 4 07:33:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 07:33:02 +0300 Subject: [Dovecot] doveadm user -f index In-Reply-To: <87pqc3l1xk.fsf@algae.riseup.net> References: <87pqc3l1xk.fsf@algae.riseup.net> Message-ID: <4C96B184-EF85-4301-AEFD-F9201A423D63@iki.fi> http://hg.dovecot.org/dovecot-2.1/rev/601014feade4 doveadm user -m -f home user at domain I'm still not sure if I implemented this the user friendliest way.. It's kind of annoying to have these existing "doveadm auth" and "doveadm user" commands. Perhaps "doveadm user -m" should have been the default, with some added fields, and then the current "doveadm user" would have been as an added option. Hm. Maybe I should change it to v2.2.. On 23.3.2012, at 18.25, Micah Anderson wrote: > > I've configured my mail_location to have a different location for > performance reasons so they aren't in the same location as the > mail_location. > > The 'doveadm user -f home' is useful to find where a user's home > directory is for various scripting purposes, but I can't seem to find a > way to determine the location of the user's indexes. > > I can do something with the output of dovecot -a to find the > mail_location and then look for a configured INDEX, but then I don't > have a good way of translating the %d/%1n/%n type string formatters into > their values for a user. > > thanks for any suggestions! > micah > > -- > > From xnasx at yandex.ru Wed Apr 4 10:36:07 2012 From: xnasx at yandex.ru (xnasx) Date: Wed, 04 Apr 2012 11:36:07 +0400 Subject: [Dovecot] Courier to Dovecot migration script issue In-Reply-To: <9BBB03B5-4742-403B-8720-FBEFB554266D@iki.fi> References: <125901333456391@web125.yandex.ru> <9BBB03B5-4742-403B-8720-FBEFB554266D@iki.fi> Message-ID: <331561333524967@web30.yandex.ru> Thank you! I really don't know what I made, but now all works fine with Thunderbird and Outlook 2007. But something strange: 'uidvalidity' are the same for both servers, but UIDs of mesages are not: Courier: d fetch 1:* uid * 1 FETCH (UID 64295) * 2 FETCH (UID 64296) * 3 FETCH (UID 64297) Dovecot: * 1 FETCH (UID 64296) * 2 FETCH (UID 64297) * 3 FETCH (UID 64298) There can be it because of that that the server is used? 04.04.2012, 06:57, "Timo Sirainen" : > On 3.4.2012, at 15.33, xnasx wrote: > >> ?I trying to migrate to Dovecot 2.0.18 from Courier 4.10. >> ?I using a script 'courier-dovecot-migrate.pl' from oficial wiki - it works without errors and creating dovecot-uidlist, etc. . >> >> ?I running Dovecot on a different port: 11149. >> ?When I connecting to port 11149 from Thunderbird, all my message headers are reloading. >> >> ?Please help me to solve this issue. > > telnet localhost 143 > a login user pass > b select inbox > c status inbox (uidvalidity) > d fetch 1:* uid > > telnet localhost 11149 > a login user pass > b select inbox > c status inbox (uidvalidity) > d fetch 1:* uid > > Check if the "c" and "d" outputs are the same for both servers. If they are, then the migration was successful and the problem is something else, such as maybe the client always redownloading messages because you changed the port (or something else). From adorca at uda.ad Wed Apr 4 13:28:36 2012 From: adorca at uda.ad (Aleix Dorca) Date: Wed, 4 Apr 2012 12:28:36 +0200 Subject: [Dovecot] imapsync, namespaces and Dovecot 2.x In-Reply-To: <65D3A596-EC48-430C-8F53-4A11F30437C4@iki.fi> References: <65D3A596-EC48-430C-8F53-4A11F30437C4@iki.fi> Message-ID: El 04/04/2012, a les 3:49, Timo Sirainen va escriure: > On 2.4.2012, at 19.11, Aleix Dorca wrote: > >> I recently tried to migrate from Exchange using imapsync. It didn't work because it complained that dovecot had no namespace capablity. After looking through Dovecot's config files I found that namespaces could be defined, but if you did not a 'private' default one was created. > > The problem is that Dovecot doesn't advertise NAMESPACE capability before client has logged in. The better solution would be to change imapsync to figure this out automatically. The other solution would be to set in Dovecot: > > imap_capability = +NAMESPACE > Dead on! Thanks a lot. Aleix. From feltrin at gmail.com Wed Apr 4 14:08:21 2012 From: feltrin at gmail.com (Jean Michel) Date: Wed, 4 Apr 2012 08:08:21 -0300 Subject: [Dovecot] Backtrace on mdbox index In-Reply-To: <7F642313-D6F2-4032-A913-63E9BDAB6848@iki.fi> References: <7F642313-D6F2-4032-A913-63E9BDAB6848@iki.fi> Message-ID: Almost 100%, while running force-resync got this message this time: doveadm(acc at domain): Error: Log synchronization error at seq=49,offset=3876 for /storage/3/bd/05/server00001/users/domain/cache/storage/dovecot.map.index: Broken extension introduction: Record alignment is too large doveadm(acc at domain): Warning: fscking index file /storage/3/bd/05/server00001/users/domain/cache/storage/dovecot.map.index doveadm(acc at domain): Warning: mdbox /storage/3/bd/05/server00001/users/domain/mdbox/storage: rebuilding indexes .... doveadm(acc at domain): Panic: file mail-storage.c: line 787 (mailbox_check_mismatching_separators): assertion failed: (strncmp(vname, ns->prefix, ns->prefix_len-1) == 0) doveadm(acc at domain): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x443da) [0x7fa9c7bbd3da] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7fa9c7bbd4c2] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7fa9c7b9451f] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_open_stream+0) [0x7fa9c7e813c0] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_rebuild_in_context+0xb2b) [0x7fa9c7e43aeb] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync_begin+0x7ec) [0x7fa9c7e41abc] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync+0x46) [0x7fa9c7e41b26] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_sync_init+0x87) [0x7fa9c7e41c07] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x31) [0x7fa9c7e7ff91] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x27) [0x7fa9c7e80aa7] -> doveadm() [0x4103d7] -> doveadm() [0x40f898] -> doveadm() [0x40fbf4] -> doveadm(doveadm_mail_try_run+0x141) [0x410071] -> doveadm(main+0x3a1) [0x416c31] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7fa9c7835c8d] -> doveadm() [0x40f1c9] I'm sending again a link with the indexes and more details on your email. 2012/4/3 Timo Sirainen > On 2.4.2012, at 17.14, Francisco Wagner C. Freire wrote: > > > I current using dovecot with mdbox and on one account i get this error: > > > > doveadm(user at domain): Panic: file mail-index-sync-ext.c: line 209 > > (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/47526bf86c57 > > > -- ------------------------------ Jean Michel Feltrin From mlists at edicom.eu Wed Apr 4 14:18:00 2012 From: mlists at edicom.eu (Miguel Tormo) Date: Wed, 4 Apr 2012 13:18:00 +0200 Subject: [Dovecot] Proxy and SSO (single sign-on) Message-ID: <201204041318.00760.mlists@edicom.eu> Hello, I have a running setup with a dovecot imap4/pop3 proxy to a few dovecot backend servers which actually store the mailboxes. This is running smoothly and allows me to transparently distribute mailboxes. I'm using some "extrafield" configured in the LDAP passdb. However, now I would like to use GSSAPI (preferred) and NTLM for single sign-on. Both are pretty straightforward to configure in a single instance environment, but I don't know if they would work with proxy. For example, with GSSAPI there are two cases: 1) Just use gssapi mechanism, without PAM. Then, it a user presents a ticket the passdb ldap is not used, so the extrafields are never read. 2) Use gssapi and PAM (thus allowing using a kerberos password). But the extrafields feature isn't available with PAM passdb driver, so again the proxy won't work. The case for NTLM would fall into the first case, I think. Am I right regarding this scenario? Is there a way I could make SSO and proxying work? I'm currently using dovecot 2.0.16 (had to patch it to increment LOGIN_MAX_INBUF_SIZE to 4096 for GSSAPI to work, as I read somewhere in this list), but I could upgrade to a newer version if that allows all this to work. Thanks! From tss at iki.fi Wed Apr 4 14:18:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 14:18:53 +0300 Subject: [Dovecot] Backtrace on mdbox index In-Reply-To: References: <7F642313-D6F2-4032-A913-63E9BDAB6848@iki.fi> Message-ID: <277DFE38-DBF3-4806-B73A-CA797763321F@iki.fi> On 4.4.2012, at 14.08, Jean Michel wrote: > doveadm(acc at domain): Panic: file mail-storage.c: line 787 > (mailbox_check_mismatching_separators): assertion failed: (strncmp(vname, > ns->prefix, ns->prefix_len-1) == 0) That's a bit strange. Can you get a core of it? http://dovecot.org/bugreport.html If you can access the crash with gdb, try commands like: fr 6 (or maybe 5, or maybe 4, or 7, or whatever makes the following commands work) p *box o *box.list.ns From tss at iki.fi Wed Apr 4 14:21:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 14:21:33 +0300 Subject: [Dovecot] Proxy and SSO (single sign-on) In-Reply-To: <201204041318.00760.mlists@edicom.eu> References: <201204041318.00760.mlists@edicom.eu> Message-ID: <48AD358F-ECC4-48F5-8CAB-9B91C8BDEEE3@iki.fi> On 4.4.2012, at 14.18, Miguel Tormo wrote: > I have a running setup with a dovecot imap4/pop3 proxy to a few dovecot backend servers which actually store the mailboxes. This is running smoothly and allows me to transparently distribute mailboxes. > I'm using some "extrafield" configured in the LDAP passdb. > > However, now I would like to use GSSAPI (preferred) and NTLM for single sign-on. Both are pretty straightforward to configure in a single instance environment, but I don't know if they would work with proxy. For example, with GSSAPI there are two cases: > 1) Just use gssapi mechanism, without PAM. Then, it a user presents a ticket the passdb ldap is not used, so the extrafields are never read. The patch in http://dovecot.org/list/dovecot/2012-March/064331.html makes this work I think. I still haven't managed to look into it much though. From pw at wk-serv.de Wed Apr 4 14:38:55 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Wed, 04 Apr 2012 13:38:55 +0200 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <1333435450.21461.35.camel@innu> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> Message-ID: <4F7C32CF.7010908@wk-serv.de> Timo Sirainen schrieb: > On Fri, 2012-03-30 at 21:37 +0200, Patrick Westenberg wrote: >> MTA --(lmtp)--\ /--(lmtp)--> backend1 --\ >> -- director -- -- NFS >> MTA --(lmtp)--/ \--(lmtp)--> backend2 --/ >> >> >> IMAP-User --> frontend1 --\ /--(imap)--> backend1 --\ >> -- director -- -- NFS >> IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/ > > The director above must be the same one for both IMAP and MTA. I know and it is meant to be the same in this drawing. However, in this scenario the director is a single point of failure. What would be the best way to get loadbalancing and high-availability? Patrick From tss at iki.fi Wed Apr 4 14:46:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 14:46:11 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F7C32CF.7010908@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> Message-ID: <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> On 4.4.2012, at 14.38, Patrick Westenberg wrote: > Timo Sirainen schrieb: >> On Fri, 2012-03-30 at 21:37 +0200, Patrick Westenberg wrote: >>> MTA --(lmtp)--\ /--(lmtp)--> backend1 --\ >>> -- director -- -- NFS >>> MTA --(lmtp)--/ \--(lmtp)--> backend2 --/ >>> >>> >>> IMAP-User --> frontend1 --\ /--(imap)--> backend1 --\ >>> -- director -- -- NFS >>> IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/ >> >> The director above must be the same one for both IMAP and MTA. > > I know and it is meant to be the same in this drawing. > > However, in this scenario the director is a single point of failure. > What would be the best way to get loadbalancing and high-availability? Another director. They're meant to connect to each others and do LB/HA. From mlists at edicom.eu Wed Apr 4 14:47:47 2012 From: mlists at edicom.eu (Miguel Tormo) Date: Wed, 4 Apr 2012 13:47:47 +0200 Subject: [Dovecot] Proxy and SSO (single sign-on) In-Reply-To: <48AD358F-ECC4-48F5-8CAB-9B91C8BDEEE3@iki.fi> References: <201204041318.00760.mlists@edicom.eu> <48AD358F-ECC4-48F5-8CAB-9B91C8BDEEE3@iki.fi> Message-ID: <201204041347.47817.mlists@edicom.eu> El Mi?rcoles, 4 de Abril de 2012 13:21:33 Timo Sirainen escribi?: > On 4.4.2012, at 14.18, Miguel Tormo wrote: > > > I have a running setup with a dovecot imap4/pop3 proxy to a few dovecot backend servers which actually store the mailboxes. This is running smoothly and allows me to transparently distribute mailboxes. > > I'm using some "extrafield" configured in the LDAP passdb. > > > > However, now I would like to use GSSAPI (preferred) and NTLM for single sign-on. Both are pretty straightforward to configure in a single instance environment, but I don't know if they would work with proxy. For example, with GSSAPI there are two cases: > > 1) Just use gssapi mechanism, without PAM. Then, it a user presents a ticket the passdb ldap is not used, so the extrafields are never read. > > The patch in http://dovecot.org/list/dovecot/2012-March/064331.html makes this work I think. I still haven't managed to look into it much though. > > It definitely is worth a look. I wonder if it would make ldap extrafields lookups work with gssapi auth, I will try it and post the results. Thank you! From pw at wk-serv.de Wed Apr 4 15:02:02 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Wed, 04 Apr 2012 14:02:02 +0200 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> Message-ID: <4F7C383A.2060401@wk-serv.de> Timo Sirainen schrieb: > Another director. They're meant to connect to each others and do LB/HA. But what about my MTAs? How can I tell my two postfix servers that there are two directors and it should/can use the other one if the first is down? Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one director. From tss at iki.fi Wed Apr 4 15:06:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 15:06:29 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F7C383A.2060401@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> Message-ID: <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> On 4.4.2012, at 15.02, Patrick Westenberg wrote: > Timo Sirainen schrieb: > >> Another director. They're meant to connect to each others and do LB/HA. > > But what about my MTAs? How can I tell my two postfix servers that there are two directors and it should/can use the other one if the > first is down? > > Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. > Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one director. I don't know if Postfix supports that. Typically people use a load balancer (cluster). From jerry at seibercom.net Wed Apr 4 15:29:21 2012 From: jerry at seibercom.net (Jerry) Date: Wed, 4 Apr 2012 08:29:21 -0400 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> Message-ID: <20120404082921.7d404024@scorpio> On Wed, 4 Apr 2012 15:06:29 +0300 Timo Sirainen articulated: > On 4.4.2012, at 15.02, Patrick Westenberg wrote: > > > Timo Sirainen schrieb: > > > >> Another director. They're meant to connect to each others and do > >> LB/HA. > > > > But what about my MTAs? How can I tell my two postfix servers that > > there are two directors and it should/can use the other one if the > > first is down? > > > > Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. > > Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one > > director. > > I don't know if Postfix supports that. Typically people use a load > balancer (cluster). Perhaps posting on the Postfix forum and asking Wietse Venema would be the logical course of action. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From tss at iki.fi Wed Apr 4 15:39:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 15:39:37 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <20120404082921.7d404024@scorpio> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> <20120404082921.7d404024@scorpio> Message-ID: <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> On 4.4.2012, at 15.29, Jerry wrote: >>> Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. >>> Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one >>> director. >> >> I don't know if Postfix supports that. Typically people use a load >> balancer (cluster). > > Perhaps posting on the Postfix forum and asking Wietse Venema would be > the logical course of action. I have a guess though, or alternatively a suggestion if it already doesn't work like that: lmtp:tcp:lmtp.example.com:24 where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places. From patrickdk at patrickdk.com Wed Apr 4 15:54:28 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Wed, 04 Apr 2012 08:54:28 -0400 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> <20120404082921.7d404024@scorpio> <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> Message-ID: <20120404085428.Horde.9RyhTJLnE6FPfESE0TaTcQA@kishi.patrickdk.com> Quoting Timo Sirainen : > On 4.4.2012, at 15.29, Jerry wrote: > >>>> Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. >>>> Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one >>>> director. >>> >>> I don't know if Postfix supports that. Typically people use a load >>> balancer (cluster). >> >> Perhaps posting on the Postfix forum and asking Wietse Venema would be >> the logical course of action. > > I have a guess though, or alternatively a suggestion if it already > doesn't work like that: > > lmtp:tcp:lmtp.example.com:24 > > where lmtp.example.com expands to your two IP addresses. DNS server > handles load balancing by returning IPs in round robin and Postfix > handles fallbacking to the second IP if the first one doesn't work. > Dovecot has similar behavior in several places. Personally I use MX dns entries, lowest mx entry is localhost, then others. From pw at wk-serv.de Wed Apr 4 16:04:11 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Wed, 04 Apr 2012 15:04:11 +0200 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> <20120404082921.7d404024@scorpio> <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> Message-ID: <4F7C46CB.3050709@wk-serv.de> Timo Sirainen schrieb: > where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places. As far as I understand, lmtp.example.com with out records would result in a MX-lookup. If postfix behaves for lmtp like it does for smtp, it will try all listed MX (my directors) in turn. From tss at iki.fi Wed Apr 4 16:09:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 16:09:46 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F7C46CB.3050709@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> <20120404082921.7d404024@scorpio> <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> <4F7C46CB.3050709@wk-serv.de> Message-ID: On 4.4.2012, at 16.04, Patrick Westenberg wrote: > Timo Sirainen schrieb: > >> where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places. > > As far as I understand, lmtp.example.com with out records would result in a MX-lookup. If postfix behaves for lmtp like it does for smtp, it will try all listed MX (my directors) in turn. MX lookups are intended for SMTP, I very much doubt LMTP uses them? From p at state-of-mind.de Wed Apr 4 16:10:38 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Wed, 4 Apr 2012 15:10:38 +0200 Subject: [Dovecot] Modifying LDAP search results for user_attrs Message-ID: <20120404131037.GB23371@state-of-mind.de> I need to authenticate users via LDAP either by mailaddress or login name. In both cases the mailbox location is /src/mail/%d/%n. I can easily deduct that path from the mailaddress, but not so from the login name. Can I get the mail address as part of fetching user_attrs and modify it on the fly? Something along the lines of this: user_attrs = mail:/srv/mail/%d/%n=home,uidNumber=uid,gidNumber=gid Or would I be able to modify this with a post-login script? p at rick -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From tss at iki.fi Wed Apr 4 16:16:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 16:16:22 +0300 Subject: [Dovecot] Modifying LDAP search results for user_attrs In-Reply-To: <20120404131037.GB23371@state-of-mind.de> References: <20120404131037.GB23371@state-of-mind.de> Message-ID: On 4.4.2012, at 16.10, Patrick Ben Koetter wrote: > I need to authenticate users via LDAP either by mailaddress or login name. > In both cases the mailbox location is /src/mail/%d/%n. I can easily deduct > that path from the mailaddress, but not so from the login name. If you have something like mailAddress=user in your pass_attrs and user_attrs, so that the username gets translated to the mailaddress then you can use %n/%d in dovecot.conf. > Can I get the mail address as part of fetching user_attrs and modify it on the > fly? Something along the lines of this: > > user_attrs = mail:/srv/mail/%d/%n=home,uidNumber=uid,gidNumber=gid > > Or would I be able to modify this with a post-login script? The correct syntax is: user_attrs = =home=mail:/srv/mail/%d/%n, .. but this assumes that the login is user at domain. Without changing the username to mail address you can't use %d/%n directly, and post-login script would be the only way to go. From patrickdk at patrickdk.com Wed Apr 4 16:57:01 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Wed, 04 Apr 2012 09:57:01 -0400 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> <20120404082921.7d404024@scorpio> <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> <4F7C46CB.3050709@wk-serv.de> Message-ID: <20120404095701.Horde.oqR4cpLnE6FPfFMtl5U1tdA@kishi.patrickdk.com> Quoting Timo Sirainen : > On 4.4.2012, at 16.04, Patrick Westenberg wrote: > >> Timo Sirainen schrieb: >> >>> where lmtp.example.com expands to your two IP addresses. DNS >>> server handles load balancing by returning IPs in round robin and >>> Postfix handles fallbacking to the second IP if the first one >>> doesn't work. Dovecot has similar behavior in several places. >> >> As far as I understand, lmtp.example.com with out records would >> result in a MX-lookup. If postfix behaves for lmtp like it does for >> smtp, it will try all listed MX (my directors) in turn. > > MX lookups are intended for SMTP, I very much doubt LMTP uses them? Yes, but postfix lmtp is the same binary as smtp, at least currently. From tss at iki.fi Wed Apr 4 17:10:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 17:10:31 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <20120404095701.Horde.oqR4cpLnE6FPfFMtl5U1tdA@kishi.patrickdk.com> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> <20120404082921.7d404024@scorpio> <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> <4F7C46CB.3050709@wk-serv.de> <20120404095701.Horde.oqR4cpLnE6FPfFMtl5U1tdA@kishi.patrickdk.com> Message-ID: On 4.4.2012, at 16.57, Patrick Domack wrote: > Quoting Timo Sirainen : > >> On 4.4.2012, at 16.04, Patrick Westenberg wrote: >> >>> Timo Sirainen schrieb: >>> >>>> where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places. >>> >>> As far as I understand, lmtp.example.com with out records would result in a MX-lookup. If postfix behaves for lmtp like it does for smtp, it will try all listed MX (my directors) in turn. >> >> MX lookups are intended for SMTP, I very much doubt LMTP uses them? > > Yes, but postfix lmtp is the same binary as smtp, at least currently. That's about how to handle incoming lmtp vs. smtp connections. The outgoing smtp vs. lmtp connections are always handled by the same binary anyway. From zybi at talex.pl Wed Apr 4 19:09:50 2012 From: zybi at talex.pl (=?ISO-8859-2?Q?Artur_Zaprza=B3a?=) Date: Wed, 04 Apr 2012 18:09:50 +0200 Subject: [Dovecot] lmtp sometimes fails to deliver a message to all recipients In-Reply-To: <1B1369FF-6748-46DE-AB94-AF2F32061757@iki.fi> References: <4F79C995.9060607@talex.pl> <1B1369FF-6748-46DE-AB94-AF2F32061757@iki.fi> Message-ID: <4F7C724E.8040905@talex.pl> Timo Sirainen wrote: > On 2.4.2012, at 18.45, Artur Zaprza?a wrote: > >> Running strace on lmtp reveals that when lmtp delivers a message to multiple recipients it saves the message to a file in the first mailbox and hard links that file to other mailboxes. But when in the meantime of delivery that file is deleted, lmtp fails to deliver the message to remaining mailboxes. For example a message for 46 recipients is delivered by lmtp to the first 12 only (fortunately postfix retries delivery for remaining recipients) and for the rest lmtp logs the following error messages: >> >> lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Message was expunged (guid) >> lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: script /vmail/domain/foo/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /vmail/domain/foo/.dovecot.sieve.log may reveal additional details) > Fixed in hg. > Tested with Maildir. Works great. Thanks. The above problem was appearing when some recipients (including first one) had a sieve filter with discard action for current message. In this case, depending on the pattern of recipients having a sieve discard action, lmtp can create more than one instance of the message for a few dozen recipients. It would be nice if lmtp could create a single hardlinked instance of the message even in this case. Best regards, Artur Zaprza?a From property.of.mike.jones at gmail.com Wed Apr 4 20:52:27 2012 From: property.of.mike.jones at gmail.com (Mike Jones!) Date: Wed, 4 Apr 2012 13:52:27 -0400 Subject: [Dovecot] Listener not binding to port Message-ID: Hello. Dovecot is not binding or listening on port 993 when I start it. $ sudo netstat -tlnp | grep -c 993 0 Dovecot starts fine and runs without complaints. $ sudo service dovecot restart Restarting IMAP/POP3 mail server: dovecot. $ sudo tail /var/log/mail.log Apr 4 13:42:39 mwjones dovecot: master: Warning: Killed with signal 15 (by pid=16238 uid=0 code=kill) Apr 4 13:42:39 mwjones dovecot: master: Dovecot v2.0.18 starting up (core dumps disabled) $ sudo ps -ef | grep -i [d]ovecot root 16243 1 0 13:42 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf dovecot 16245 16243 0 13:42 ? 00:00:00 dovecot/anvil root 16246 16243 0 13:42 ? 00:00:00 dovecot/log root 16248 16243 0 13:42 ? 00:00:00 dovecot/config Still no listener :( $ sudo netstat -tlnp | grep -c 993 0 Other infos for your scrutiny. $ sudo dovecot --version 2.0.18 $ doveconf -n # 2.0.18: /etc/dovecot/dovecot.conf # OS: Linux 3.2.5-grsec x86_64 Debian wheezy/sid ext4 auth_debug = yes auth_mechanisms = plain login auth_verbose = yes first_valid_uid = 1000 log_timestamp = "%Y-%m-%d %H:%M:%S " mail_debug = yes mail_location = maildir:/home/vmail/%d/%n/Maildir passdb { args = /home/vmail/%d/etc/passwd driver = passwd-file } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } service imap-login { service_count = 1 } service ssl-params { type = startup } ssl_cert = References: <20120404131037.GB23371@state-of-mind.de> Message-ID: <6709704ac05c4ad9e121b73a76e5ae23@in.tum.de> On 2012-04-04 15:10, Patrick Ben Koetter wrote: > Or would I be able to modify this with a post-login script? Just one note: as far as I know, if your mail directory depends on a post-login script, it might be tricky to get LMTP or doveadm commands to work properly. Cheers, Christoph From eliezer at ngtech.co.il Thu Apr 5 06:44:18 2012 From: eliezer at ngtech.co.il (Eliezer Croitoru) Date: Thu, 05 Apr 2012 06:44:18 +0300 Subject: [Dovecot] Listener not binding to port In-Reply-To: References: Message-ID: <4F7D1512.5080102@ngtech.co.il> you should configure imap\pop3 services as far as i remeber... Regards, Eliezer On 04/04/2012 20:52, Mike Jones! wrote: > Hello. > > Dovecot is not binding or listening on port 993 when I start it. > > $ sudo netstat -tlnp | grep -c 993 > 0 > > Dovecot starts fine and runs without complaints. > > $ sudo service dovecot restart > Restarting IMAP/POP3 mail server: dovecot. > > $ sudo tail /var/log/mail.log > Apr 4 13:42:39 mwjones dovecot: master: Warning: Killed with signal > 15 (by pid=16238 uid=0 code=kill) > Apr 4 13:42:39 mwjones dovecot: master: Dovecot v2.0.18 starting up > (core dumps disabled) > > $ sudo ps -ef | grep -i [d]ovecot > root 16243 1 0 13:42 ? 00:00:00 /usr/sbin/dovecot -c > /etc/dovecot/dovecot.conf > dovecot 16245 16243 0 13:42 ? 00:00:00 dovecot/anvil > root 16246 16243 0 13:42 ? 00:00:00 dovecot/log > root 16248 16243 0 13:42 ? 00:00:00 dovecot/config > > Still no listener :( > > $ sudo netstat -tlnp | grep -c 993 > 0 > > Other infos for your scrutiny. > > $ sudo dovecot --version > 2.0.18 > > $ doveconf -n > # 2.0.18: /etc/dovecot/dovecot.conf > # OS: Linux 3.2.5-grsec x86_64 Debian wheezy/sid ext4 > auth_debug = yes > auth_mechanisms = plain login > auth_verbose = yes > first_valid_uid = 1000 > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_debug = yes > mail_location = maildir:/home/vmail/%d/%n/Maildir > passdb { > args = /home/vmail/%d/etc/passwd > driver = passwd-file > } > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > user = root > } > service imap-login { > service_count = 1 > } > service ssl-params { > type = startup > } > ssl_cert = ssl_key = userdb { > args = /home/vmail/%d/etc/passwd > driver = passwd-file > } > verbose_ssl = yes > > > What am I missing and how do I fix this? > > Thanks, > mwjones -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer ngtech.co.il From tss at iki.fi Thu Apr 5 09:08:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 5 Apr 2012 09:08:42 +0300 Subject: [Dovecot] Listener not binding to port In-Reply-To: References: Message-ID: <0EBD739C-2E2E-4CFB-B149-852AFD7279F4@iki.fi> On 4.4.2012, at 20.52, Mike Jones! wrote: > Dovecot is not binding or listening on port 993 when I start it. Try adding protocols=imap setting. From lists at wiesinger.com Thu Apr 5 09:16:33 2012 From: lists at wiesinger.com (Gerhard Wiesinger) Date: Thu, 5 Apr 2012 08:16:33 +0200 (CEST) Subject: [Dovecot] Dovecot upgrade from 1.2.x to 2.0.x: roundcube/squirrelmail sent folder doesn't work any more In-Reply-To: References: <4F71F8D8.6040700@wiesinger.com> <95C664AD-0532-4F3B-A2EB-2FD25B79187B@iki.fi> Message-ID: On Wed, 4 Apr 2012, Timo Sirainen wrote: > On 29.3.2012, at 8.25, Gerhard Wiesinger wrote: > >>>> 2.0.x: with Prefix ~/Mail >>>> A0003 LIST "" ~/Mail/sent >>>> A0003 OK List completed.: >>> >> >> mail_full_filesystem_access = yes >> mail_location = mbox:~:INBOX=/var/mail/%u > > Yes, the problem happens only with mail_full_filesystem_access=yes. It already works in v2.1, so I don't think I'll bother looking into why it's not working in v2.0. Hello Timo, As discussed per private mail you finally fixed it: http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e Thnx for your support. Ciao, Gerhard -- http://www.wiesinger.com/ From dvorak at cosign.jp Thu Apr 5 10:48:45 2012 From: dvorak at cosign.jp (Akihiko Sato) Date: Thu, 5 Apr 2012 16:48:45 +0900 Subject: [Dovecot] problem to configuration dovecot and postfix Message-ID: Hi, I'm having a problem with dovecot and postfix. I'm using a dovecot version 2.0.13. Postfix (Version: 2.8.5) config is changed according to the http://wiki2.dovecot.org/LDA/Postfix: I have thought to build a virtual user and virtual mailbox, and use the functionality of the Sieve. However, when the test set, the mail not received the mailbox. Record that there was a "temporary failure" in the log of Postfix. It delivered and to change the "virtual" to "dovecot", I think that's a matter of dovecot. To ensure that mail is delivered as intended, will How do I? Can anyone give me a hint what to do? # doveconf -n: # 2.0.13: /etc/dovecot/dovecot.conf # OS: Linux 3.0.0-12-server x86_64 Ubuntu 11.10 ext4 auth_debug = yes auth_debug_passwords = yes auth_verbose = yes disable_plaintext_auth = no info_log_path = /var/log/dovecot-info.log lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:/var/mail/vhosts/%d/%n managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables bo dy enotify environment mailbox date namespace { inbox = yes list = yes location = prefix = INBOX/ separator = / subscriptions = yes type = private } namespace { hidden = no list = yes location = maildir:/var/mail/shared/:INDEX=/var/mail/vhosts/%d/%n/shared/:CONTROL=/var/mail/vhosts/%d/%n/shared/ prefix = &UXFnCQ-/ separator = / subscriptions = yes type = public } passdb { args = /etc/auth/passwd.main driver = passwd-file } passdb { args = username_format=%n /etc/auth/%d/passwd driver = passwd-file } passdb { driver = pam } plugin { sieve = /var/mail/vhosts/%d/%n/_dovecot.sieve sieve_dir = /var/mail/vhosts/%d/%n/sieve/ } postmaster_address = postmaster at dom3.gs3 protocols = imap sieve service auth { user = root } ssl_cert = Apr 5 16:38:36 qbu postfix/cleanup[25500]: BDC3221212: message-id=<20120405073836.BDC3221212 at qbu> Apr 5 16:38:36 qbu postfix/qmgr[25471]: BDC3221212: from=, size=230, nrcpt=1 (queue active) Apr 5 16:38:36 qbu postfix/pipe[25502]: BDC3221212: to=, relay=dovecot, delay=0.04, delays=0.02/0/0/0.02, dsn=4.3.0, status=deferred (temporary failure) Regards, Akihiko From cor at xs4all.nl Thu Apr 5 11:15:09 2012 From: cor at xs4all.nl (Cor Bosman) Date: Thu, 5 Apr 2012 10:15:09 +0200 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F7C383A.2060401@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> Message-ID: <20120405081509.GA1598@xs4all.nl> On Wed, Apr 04, 2012 at 02:02:02PM +0200, Patrick Westenberg wrote: > Timo Sirainen schrieb: > >> Another director. They're meant to connect to each others and do LB/HA. > > But what about my MTAs? How can I tell my two postfix servers that there > are two directors and it should/can use the other one if the > first is down? We use hardware loadbalancers in front of the directors. If a director goes down, the hardware loadbalancers sees that. We also use the poolmon script to tell the directors if a certain imap server is missing. Cor From tlx at leuxner.net Thu Apr 5 11:18:14 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 5 Apr 2012 10:18:14 +0200 Subject: [Dovecot] problem to configuration dovecot and postfix In-Reply-To: References: Message-ID: <20120405081814.GA7578@nihlus.leuxner.net> On Thu, Apr 05, 2012 at 04:48:45PM +0900, Akihiko Sato wrote: > # postconf -n: > mydestination = qbu.example.com, qbu, localhost.localdomain, localhost http://www.postfix.org/VIRTUAL_README.html [...] NEVER list a virtual MAILBOX domain name as a mydestination domain! > /var/log/mail.log: > Apr 5 16:38:36 qbu postfix/pickup[25470]: BDC3221212: uid=0 from= > Apr 5 16:38:36 qbu postfix/cleanup[25500]: BDC3221212: > message-id=<20120405073836.BDC3221212 at qbu> > Apr 5 16:38:36 qbu postfix/qmgr[25471]: BDC3221212: > from=, size=230, nrcpt=1 (queue active) > Apr 5 16:38:36 qbu postfix/pipe[25502]: BDC3221212: > to=, relay=dovecot, delay=0.04, delays=0.02/0/0/0.02, > dsn=4.3.0, status=deferred (temporary failure) Also with 2.x you may want to use LMTP rather than the LDA Piping. http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From p at state-of-mind.de Thu Apr 5 11:44:08 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Thu, 5 Apr 2012 10:44:08 +0200 Subject: [Dovecot] problem to configuration dovecot and postfix In-Reply-To: <20120405081814.GA7578@nihlus.leuxner.net> References: <20120405081814.GA7578@nihlus.leuxner.net> Message-ID: <20120405084408.GB10931@state-of-mind.de> * Thomas Leuxner : > On Thu, Apr 05, 2012 at 04:48:45PM +0900, Akihiko Sato wrote: > > > # postconf -n: > > mydestination = qbu.example.com, qbu, localhost.localdomain, localhost > > http://www.postfix.org/VIRTUAL_README.html > > [...] > NEVER list a virtual MAILBOX domain name as a mydestination domain! ... unless you know what you do and wish to benefit from the side-effects. p at rick -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 316 bytes Desc: Digital signature URL: From dvorak at cosign.jp Thu Apr 5 11:54:13 2012 From: dvorak at cosign.jp (Akihiko Sato) Date: Thu, 5 Apr 2012 17:54:13 +0900 Subject: [Dovecot] problem to configuration dovecot and postfix In-Reply-To: <20120405081814.GA7578@nihlus.leuxner.net> References: <20120405081814.GA7578@nihlus.leuxner.net> Message-ID: 2012/4/5 Thomas Leuxner : > On Thu, Apr 05, 2012 at 04:48:45PM +0900, Akihiko Sato wrote: > >> # postconf -n: >> mydestination = qbu.example.com, qbu, localhost.localdomain, localhost > > http://www.postfix.org/VIRTUAL_README.html > > [...] > NEVER list a virtual MAILBOX domain name as a mydestination domain! Oops, this is my careless mistake...mydestination is empty. But status does not change. Regards, Akihiko > > Also with 2.x you may want to use LMTP rather than the LDA Piping. > > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP > > Regards > Thomas > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > > iEYEARECAAYFAk99VUYACgkQ+meF/S97aXi/4wCgkLGKNByWuxYb0E7YIdZceHHU > UwQAnArESXtRggXYRbnk64WaVfItfAY9 > =N8Aq > -----END PGP SIGNATURE----- > From tlx at leuxner.net Thu Apr 5 12:31:20 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 5 Apr 2012 11:31:20 +0200 Subject: [Dovecot] problem to configuration dovecot and postfix In-Reply-To: References: <20120405081814.GA7578@nihlus.leuxner.net> Message-ID: <20120405093120.GB7578@nihlus.leuxner.net> On Thu, Apr 05, 2012 at 05:54:13PM +0900, Akihiko Sato wrote: > Oops, this is my careless mistake...mydestination is empty. > But status does not change. > > Regards, > Akihiko Hmm. Haven't used LDA since LMTP was introduced. Are there corresponding log entries on the Dovecot side? [...] protocol lda { postmaster_address = postmaster at domain.tld log_path = /var/log/dovecot-deliver.log info_log_path = /var/log/dovecot-deliver.log } Still would go the LMTP route rather than using Pipes: > > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP > > Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From dvorak at cosign.jp Thu Apr 5 13:10:07 2012 From: dvorak at cosign.jp (Akihiko Sato) Date: Thu, 5 Apr 2012 19:10:07 +0900 Subject: [Dovecot] problem to configuration dovecot and postfix In-Reply-To: <20120405093120.GB7578@nihlus.leuxner.net> References: <20120405081814.GA7578@nihlus.leuxner.net> <20120405093120.GB7578@nihlus.leuxner.net> Message-ID: 2012/4/5 Thomas Leuxner : > On Thu, Apr 05, 2012 at 05:54:13PM +0900, Akihiko Sato wrote: >> Oops, this is my careless mistake...mydestination is empty. >> But status does not change. >> >> Regards, >> Akihiko > > Hmm. Haven't used LDA since LMTP was introduced. Are there corresponding > log entries on the Dovecot side? > > [...] > protocol lda { > ? ? ? ?postmaster_address = postmaster at domain.tld > ? ? ? ?log_path = /var/log/dovecot-deliver.log > ? ? ? ?info_log_path = /var/log/dovecot-deliver.log > } I tried to restart the dovecot put these settings, the log was not generated. Existing log (dovecot.log, dovecot-info.log) that an error is also recorded in the was not. ;-( In this case, is it better to use LMTP? > > Still would go the LMTP route rather than using Pipes: > >> > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP >> > > > Thomas > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > > iEYEARECAAYFAk99ZmgACgkQ+meF/S97aXh1/ACgo/VaG3SP587MvPDwkXGRn/lG > o08An1nKrVecLm6yS9HuG60PULJgw0I3 > =hUSv > -----END PGP SIGNATURE----- > Regards, Akihiko From busseniu at in.tum.de Thu Apr 5 13:36:03 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Thu, 05 Apr 2012 12:36:03 +0200 Subject: [Dovecot] Dovecot allows creation of folders outside of a user's directory In-Reply-To: References: <4F759B08.1060603@in.tum.de> Message-ID: <4F7D7593.2090005@in.tum.de> On 04.04.2012 03:35, Timo Sirainen wrote: > On 30.3.2012, at 14.37, Christoph Bu?enius wrote: >> -- even though it will fail with a "permission denied" error -- will create a directory like "/mail/users/strange&ANY- characters". That directory will only contain a subdirectory "Maildir" and therein dovecot-acl-list. > > Fixed: http://hg.dovecot.org/dovecot-2.0/rev/b15889b82258 > Thanks, this fixed it. By the way, your tireless work on Dovecot is amazing :) -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From CMarcus at Media-Brokers.com Thu Apr 5 15:59:22 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 05 Apr 2012 08:59:22 -0400 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header Message-ID: <4F7D972A.8050406@Media-Brokers.com> On 2012-04-05 4:18 AM, Thomas Leuxner wrote: > Also with 2.x you may want to use LMTP rather than the LDA Piping. > > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP I am preparing to convert my main client's postfix_courier-imap setup to dovecot 2.1, which currently just uses the postfix virtual delivery agent... Does anyone know if the use of LMTP (or even the dovecot LDA) still loses the x-original-to header that the postfix vda adds and that I rely heavily on (since I use a lot of aliases), and if it does, is there any solution to get the original recipient added back in before final delivery? Everything I'm reading says that LMTP is better, but I really do need this header (or one like it) to be there so I know who the original recipient was (for filtering and other purposes). Thanks, -- Best regards, Charles From l.chelchowski at eurocar.pl Thu Apr 5 16:29:13 2012 From: l.chelchowski at eurocar.pl (l.chelchowski at eurocar.pl) Date: Thu, 05 Apr 2012 15:29:13 +0200 Subject: [Dovecot] Problem with polish characters and sieve fileinto Message-ID: <4F7D9E29.3050802@eurocar.pl> Hi, I have problems with sieve fileinto and polish characters in folder names: # rule:[??????????????????] elsif anyof (header :contains "Subject" "??????????????????") { fileinto "INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-"; } dovecot log: Apr 5 15:17:42 mailer dovecot: lmtp(54203, test at test1.com): rmHVJHabfU+70wAA9hAjDg: sieve: msgid=<4F7D9B68.4030206 at test2.com>: stored mail into mailbox 'INBOX' Apr 5 15:17:42 mailer dovecot: lmtp(54203, test at test1.com): Error: rmHVJHabfU+70wAA9hAjDg: sieve: execution of script /home/vmail/test1.com/test//.dovecot.sieve failed, but implicit keep was successful (user logfile /home/vmail/test1.com/test//.dovecot.sieve.log may reveal additional details) Apr 5 15:17:42 mailer postfix/lmtp[54202]: 941B1E2681F: to=, relay=m.test1.com[private/dovecot-lmtp], delay=0.04, delays=0/0.01/0/0.03, dsn=2.0.0, status=sent (250 2.0.0 rmHVJHabfU+70wAA9hAjDg Saved) dovecot.sieve.log: sieve: info: started log at Apr 05 15:17:42. error: msgid=<4F7D9B68.4030206 at test2.com>: failed to store into mailbox 'INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-' (INBOX/&-AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-): Mailbox doesn't exist: INBOX.&-AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-. Thanks for help. -- Pozdrawiam ?ukasz From zybi at talex.pl Thu Apr 5 16:45:24 2012 From: zybi at talex.pl (=?UTF-8?B?QXJ0dXIgWmFwcnphxYJh?=) Date: Thu, 05 Apr 2012 15:45:24 +0200 Subject: [Dovecot] Problem with polish characters and sieve fileinto In-Reply-To: <4F7D9E29.3050802@eurocar.pl> References: <4F7D9E29.3050802@eurocar.pl> Message-ID: <4F7DA1F4.40207@talex.pl> l.chelchowski at eurocar.pl wrote: > Hi, > > I have problems with sieve fileinto and polish characters in folder names: > # rule:[??????????????????] > elsif anyof (header :contains "Subject" "??????????????????") > { > fileinto "INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-"; > } > Sieve script expects UTF-8 folder names. This should work: fileinto "INBOX/??????????????????"; You can also use UTF-8 mailbox names on disk if you add UTF-8 option to mail_location: mail_location = maildir:~/Maildir:UTF-8 but you have to convert existing names to UTF-8 somehow. > > dovecot log: > Apr 5 15:17:42 mailer dovecot: lmtp(54203, test at test1.com): > rmHVJHabfU+70wAA9hAjDg: sieve: msgid=<4F7D9B68.4030206 at test2.com>: stored mail > into mailbox 'INBOX' > Apr 5 15:17:42 mailer dovecot: lmtp(54203, test at test1.com): Error: > rmHVJHabfU+70wAA9hAjDg: sieve: execution of script > /home/vmail/test1.com/test//.dovecot.sieve failed, but implicit keep was > successful (user logfile /home/vmail/test1.com/test//.dovecot.sieve.log may > reveal additional details) > Apr 5 15:17:42 mailer postfix/lmtp[54202]: 941B1E2681F: to=, > relay=m.test1.com[private/dovecot-lmtp], delay=0.04, delays=0/0.01/0/0.03, > dsn=2.0.0, status=sent (250 2.0.0 rmHVJHabfU+70wAA9hAjDg Saved) > > > dovecot.sieve.log: > sieve: info: started log at Apr 05 15:17:42. > error: msgid=<4F7D9B68.4030206 at test2.com>: failed to store into mailbox > 'INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-' > (INBOX/&-AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-): Mailbox doesn't > exist: INBOX.&-AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-. > > Thanks for help. -- Talex Sp??ka Akcyjna z siedzib? w Poznaniu adres: ul. Karpia 27d, 61-619 Pozna? NIP 782-00-21-045 zarejestrowana w S?dzie Rejonowym Pozna? ? Nowe Miasto i Wilda w Poznaniu VIII Wydzia? Gospodarczy - KRS pod nr 000048779 kapita? zak?adowy: 3.000.092,00 PLN (w ca?o?ci wp?acony) Uwaga: Niniejsza wiadomo??, w szczeg?lno?ci jej tre?? oraz za??czniki, mo?e by? poufna. W przypadku, gdy nie jest Pan/Pani zamierzonym jej adresatem, informujemy, ?e wszelkie rozpowszechnianie, dystrybucja lub powielanie powy?szej wiadomo?ci jest zabronione. Jednocze?nie prosimy o powiadomienie nadawcy oraz niezw?oczne usuni?cie powy?szej wiadomo?ci wraz z za??cznikami. Dzi?kujemy, Talex S.A. w Poznaniu. Confidentiality Notice: This email, particularly its content and any attached files, may be confidential. If you are not an intended recipient, any disclosure, distribution and reproduction of this message is prohibited. In this case please notify the sender immediately and then delete this message and any attachments. Thank you, Talex S.A., Poznan. From stephan at rename-it.nl Thu Apr 5 16:47:16 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 05 Apr 2012 15:47:16 +0200 Subject: [Dovecot] Problem with polish characters and sieve fileinto In-Reply-To: <4F7D9E29.3050802@eurocar.pl> References: <4F7D9E29.3050802@eurocar.pl> Message-ID: <4F7DA264.8020004@rename-it.nl> Op 4/5/2012 3:29 PM, l.chelchowski at eurocar.pl schreef: > Hi, > > I have problems with sieve fileinto and polish characters in folder > names: > # rule:[??????????????????] > elsif anyof (header :contains "Subject" "??????????????????") > { > fileinto "INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-"; > } Looks like your Sieve GUI uses the wrong Unicode format. The above is mUTF7 as used by IMAP. However, Sieve expects UTF-8. So it should use this instead: fileinto "INBOX/??????????????????"; Regards, Stephan. From xnasx at yandex.ru Thu Apr 5 16:51:32 2012 From: xnasx at yandex.ru (Victor) Date: Thu, 05 Apr 2012 17:51:32 +0400 Subject: [Dovecot] Username bug in '%%' variables selects Message-ID: <10301333633892@web94.yandex.ru> Hi! Forgive my bad English. I trying to set up shared folders, but I can't. I shared INBOX of 'victor at domain.com' for 'victor at testmail.domain.com' There is my debugging results: doveadm acl debug -u victor at testmail.domain.com INBOX.shared.victor at domain.com. doveadm(victor at testmail.domain.com): Error: Couldn't create namespace 'INBOX.shared.' for user victor at domain: userdb didn't return a home directory, but mail_location used it (%h): maildir:%h That's I seen in logs: auth-worker: Debug: sql(victor at domain): SELECT CONCAT('/var/spool/mail/', SUBSTRING_INDEX(email, '@', -1), '/', SUBSTRING_INDEX(email, '@', 1)) as home from `users` WHERE email='victor at domain' What happened with my username? 'victor at domain' is not correct value. My config: namespace { inbox = yes location = prefix = INBOX. separator = . type = private } namespace { hidden = no list = yes location = maildir:%%h prefix = INBOX.shared.%%u. separator = . subscriptions = no type = shared } password_query = \ SELECT email as username, password \ FROM users WHERE email = '%u' user_query = SELECT CONCAT('/var/spool/mail/', SUBSTRING_INDEX(email, '@', -1), '/', SUBSTRING_INDEX(email, '@', 1)) as home from `users` WHERE email='%u' Please, help me! -- Best regards, Victor From property.of.mike.jones at gmail.com Thu Apr 5 18:08:59 2012 From: property.of.mike.jones at gmail.com (Mike Jones!) Date: Thu, 5 Apr 2012 11:08:59 -0400 Subject: [Dovecot] Listener not binding to port In-Reply-To: <0EBD739C-2E2E-4CFB-B149-852AFD7279F4@iki.fi> References: <0EBD739C-2E2E-4CFB-B149-852AFD7279F4@iki.fi> Message-ID: Thanks so much all. I was missing a few items: protocols = imap # thanks to Timo Sirainen # thanks to Noel Butler service imap-login { inet_listener imap { port = 0 } inet_listener imaps { address = * port = 993 } service_count = 1 } Here is my final doveconf -n with listener successfully running on 993: # 2.0.18: /etc/dovecot/dovecot.conf # OS: Linux 3.2.5-grsec x86_64 Debian wheezy/sid ext4 auth_debug = yes auth_mechanisms = plain login auth_verbose = yes first_valid_gid = 1030 first_valid_uid = 1030 info_log_path = /var/log/dovecot-info.log last_valid_gid = 1030 last_valid_uid = 1030 log_path = /var/log/dovecot.log log_timestamp = "%Y-%m-%d %H:%M:%S " mail_debug = yes mail_location = maildir:/home/vmail/%d/%n/Maildir passdb { args = /home/vmail/%d/etc/passwd driver = passwd-file } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { address = * port = 993 } service_count = 1 } service ssl-params { type = startup } ssl_cert = Hi all, I'm planning on implementing this in my new upcoming dovecot instance, and would like to hear thoughts on how best to accomplish this. We will be paying Timo's support company to do the work, but obviously, the less work in the form of coding he has to do to get this working (I'm hoping it won't be a lot), the more money it will save us... ;) First - I currently use rsnapshot to backup emails, so that is the use-case I'm most interested in getting working. It is rsync based, and like other rsync based backup programs it uses hardlinks to save storage space - so you can have a *lot* of backups (going back months, or even years), where each snapshot only adds a little more to the total disk space being used. The snapshots are stored with the following filesystem layout: /path/to/snapshotsdir/hourly.0 ... /path/to/snapshotsdir/hourly.4 /path/to/snapshotsdir/daily.0 ... /path/to/snapshotsdir/daily.7 /path/to/snapshotsdir/weekly.0 ... /path/to/snapshotsdir/weekly.4 /path/to/snapshotsdir/monthly.0 ... /path/to/snapshotsdir/monthly.12 /path/to/snapshotsdir/yearly.0 ... /path/to/snapshotsdir/yearly.5 The 'names' (hourly, daily, weekly, monthly, yearly) are arbitrary (this is a bit confusing to people new to rsnapshot), and would *not* be used for displaying the mail folders to the users - it is the Date/Time stamps of each of the snapshot dirs above that would be used to display the folder names under the 'Time Machine' namespace. This is, I imagine, the part that will need some actual coding by Timo to get working - maybe just some new config variables added to the namespace code for mapping the date/time stamps of the directories to user friendly folder names in the namespace. That said, I'd like to design this and have it coded such that it will work with almost any type of backup storage that stores snapshots as date/time stamped directories like this (there must be others, right?). Also, it goes without saying that this code will be (if Timo is ok with it) part of the core dovecot code going forward, so anyone else will be able to benefit from it. What I'm envisioning is something like this... 1. Define a namespace - for this example we'll call it 'Time Machine' 2. Under this namespace, each user will see their, and *only* their snapshots So, each user would see something like this: My Mail Account Inbox Drafts Templates Sent Time Machine (sorted above user created folders if possible) -4/3/12, 8:00am (first subfolder) Inbox Drafts etc... (all other folders and sub-folders shown here) +4/3/12, 12:00pm (first subfolder) etc... Other User Folders ... Or even better, I'm thinking some magical code that can group them by Date, like: -4/3/12 (first subfolder) -8:00am (next sub-folder) Inbox Drafts Etc... (all folders and sub-folders shown here) +12:00pm +4:00pm +8:00pm +4/4/12 etc... Comments? Suggestions? Flames? -- Best regards, Charles From tom at whyscream.net Thu Apr 5 19:37:30 2012 From: tom at whyscream.net (Tom Hendrikx) Date: Thu, 05 Apr 2012 18:37:30 +0200 Subject: [Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups In-Reply-To: <4F7DBA3B.5090803@Media-Brokers.com> References: <4F7DBA3B.5090803@Media-Brokers.com> Message-ID: <4F7DCA4A.70009@whyscream.net> On 05-04-12 17:28, Charles Marcus wrote: > Hi all, > > I'm planning on implementing this in my new upcoming dovecot instance, > and would like to hear thoughts on how best to accomplish this. We will > be paying Timo's support company to do the work, but obviously, the less > work in the form of coding he has to do to get this working (I'm hoping > it won't be a lot), the more money it will save us... ;) > > First - I currently use rsnapshot to backup emails, so that is the > use-case I'm most interested in getting working. It is rsync based, and > like other rsync based backup programs it uses hardlinks to save storage > space - so you can have a *lot* of backups (going back months, or even > years), where each snapshot only adds a little more to the total disk > space being used. > > What I'm envisioning is something like this... > > 1. Define a namespace - for this example we'll call it 'Time Machine' > > 2. Under this namespace, each user will see their, and *only* their > snapshots > > So, each user would see something like this: > > My Mail Account > Inbox > Drafts > Templates > Sent > Time Machine (sorted above user created folders if possible) > -4/3/12, 8:00am (first subfolder) > Inbox > Drafts > etc... (all other folders and sub-folders shown here) > +4/3/12, 12:00pm (first subfolder) > etc... > Other User Folders > ... > > Or even better, I'm thinking some magical code that can group them by > Date, like: > > -4/3/12 (first subfolder) > -8:00am (next sub-folder) > Inbox > Drafts > Etc... (all folders and sub-folders shown here) > +12:00pm > +4:00pm > +8:00pm > +4/4/12 > etc... > > Comments? Suggestions? Flames? > The first interesting point I'd see with this, is that you supply the mail client with a near endless supply of folders, which would take a lot of caching space on the clients end, either (depending on the client and its configuration) from the moment that you enable this fort hem, or after someone starts searching in their 'time machine' for some old mail. I see my mail client on a new install working quite hard to download mail headers for 2 years of postfix/dovecot/etc mailing lists, so what happens if you provide a 'time machine' namespace going 1 month back, 4 with snapshots a day (i.e. 31x4 =~ 120 times more headers to download/index). -- Tom From CMarcus at Media-Brokers.com Thu Apr 5 20:02:51 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 05 Apr 2012 13:02:51 -0400 Subject: [Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups In-Reply-To: <4F7DCA4A.70009@whyscream.net> References: <4F7DBA3B.5090803@Media-Brokers.com> <4F7DCA4A.70009@whyscream.net> Message-ID: <4F7DD03B.30909@Media-Brokers.com> On 2012-04-05 12:37 PM, Tom Hendrikx wrote: > The first interesting point I'd see with this, is that you supply the > mail client with a near endless supply of folders, which would take a > lot of caching space on the clients end, either (depending on the client > and its configuration) from the moment that you enable this fort hem, or > after someone starts searching in their 'time machine' for some old mail. > > I see my mail client on a new install working quite hard to download > mail headers for 2 years of postfix/dovecot/etc mailing lists, so what > happens if you provide a 'time machine' namespace going 1 month back, 4 > with snapshots a day (i.e. 31x4 =~ 120 times more headers to > download/index). Interesting and valid point... hmmmm..... First, these folders would be read-only - a user could copy something from there back to one of his other folders, but couldn't write anything in them - so nothing would be changing under this namespace, except new snapshots magically appearing, which means that once they are indexed, the indexes would never need to be rebuilt (unless they got corrupted somehow). But, yeah, I can imagine some problems especially if someone has a ton of email. And while these would probably only be accessed rarely, in those cases where someone would want to access them, they would very likely want to be able to search, so disabling indexes wouldn't be a good idea... Since we use Thunderbird, I can of course disable offline mode for everyone, so the only time headers would be downloaded would be when the user selects (or performs a search on) one (or more) of the folders. Maybe Timo can think of something creative to minimize this problem... -- Best regards, Charles From zarf at klacto.net Thu Apr 5 20:37:37 2012 From: zarf at klacto.net (FZiegler) Date: Thu, 05 Apr 2012 13:37:37 -0400 Subject: [Dovecot] dovecot and unison In-Reply-To: <8EAB42AF-BC4A-4235-9CF9-A444BE553B10@odo.in-berlin.de> References: <4F7A2FF9.7000204@klacto.net> <8EAB42AF-BC4A-4235-9CF9-A444BE553B10@odo.in-berlin.de> Message-ID: <4F7DD861.7060303@klacto.net> Michael Grimm wrote: >> Namely, which of the ancillary files should I be syncing (or perhaps >> deleting before sync)? > I used ... > > ignore = Name dovecot* > ignore = Regex .*/tmp$ > ignore = Regex .*/\.Drafts$ > > ? to ignore all of them. Thanks Michael for this (and thanks also mazieres at stanford for your reply). I am now merrily ignoring these files too, and synced 400 MB of mail without seeing the disappearing subfolder issue again. Francois Z. From hsn at filez.com Thu Apr 5 21:46:50 2012 From: hsn at filez.com (Radim Kolar) Date: Thu, 05 Apr 2012 20:46:50 +0200 Subject: [Dovecot] create mailboxes in /var/mail via doveadm Message-ID: <4F7DE89A.4020409@filez.com> what usedid is used by doveadm for creating mailboxes? can it be setup to run as dovecot:mail? cant create mailbox from command line ponto# ls -ld /var/mail drwxrwxr-x 8 root mail 24 Apr 5 20:35 /var/mail ponto# doveadm mailbox create -u jboss INBOX doveadm(jboss): Error: Can't create mailbox INBOX: Permission denied mail_location = maildir:%h/Maildir:INBOX=/var/mail/%u From slusarz at curecanti.org Thu Apr 5 21:59:20 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Thu, 05 Apr 2012 12:59:20 -0600 Subject: [Dovecot] 2.1.3: Overly lax FETCH parsing Message-ID: <20120405125920.Horde.wazxEIF5lbhPfeuIm_gQ4uA@bigworm.curecanti.org> These commands both work in Dovecot (as of 2.1.3): 1 UID FETCH 1:* () (CHANGEDSINCE 1) * 1 FETCH (UID 30097 MODSEQ (19554)) [...] 1 OK Fetch completed. 2 FETCH 1:* () (CHANGEDSINCE 1) * 2 FETCH (MODSEQ (19554)) [...] 2 OK Fetch completed. However, RFC 3501 indicates that an empty FETCH attribute list is incorrect: fetch = "FETCH" SP sequence-set SP ("ALL" / "FULL" / "FAST" / fetch-att / "(" fetch-att *(SP fetch-att) ")") fetch-att = "ENVELOPE" / "FLAGS" / "INTERNALDATE" / "RFC822" [".HEADER" / ".SIZE" / ".TEXT"] / "BODY" ["STRUCTURE"] / "UID" / "BODY" section ["<" number "." nz-number ">"] / "BODY.PEEK" section ["<" number "." nz-number ">"] In other words, fetch-att is not allowed to be empty. (RFC 4466 does not alter this requirement when using FETCH modifiers). While useful that Dovecot is more liberal about what it receives, 3501 seems pretty clear that incorrect FETCH parameters must return a BAD. I can verify that the above commands fail on Cyrus. I don't disagree that the way Dovecot handles this is correct - or at least is the way that I wish the RFCs were written. Since this seems a bit redundant, but is necessary under the ABNF: 1 UID FETCH 1:* UID (CHANGEDSINCE 1) -or- 1 UID FETCH 1:* (UID) (CHANGEDSINCE 1) michael From centos.admin at gmail.com Thu Apr 5 23:02:17 2012 From: centos.admin at gmail.com (Emmanuel Noobadmin) Date: Fri, 6 Apr 2012 04:02:17 +0800 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? Message-ID: I'm trying to improve the setup of our Dovecot/Exim mail servers to handle the increasingly huge accounts (everybody thinks it's like infinitely growing storage like gmail and stores everything forever in their email accounts) by changing from Maildir to mdbox, and to take advantage of offloading older emails to alternative networked storage nodes. The question now is whether having a single large server or will a number of 1U servers with the same total capacity be better? Will be using RAID 1 pairs, likely XFS based on reading Hoeppner's recommendation on this and the mdadm list. Currently, I'm leaning towards multiple small servers because I think it should be better in terms of performance. At the very least even if one node gets jammed up, the rest should still be able serve up the emails for other accounts that is unless Dovecot will get locked up by that jammed transaction. Also, I could possibly arrange them in a sort of network raid 1 to gain redundancy over single machine failure. Would I be correct in these or do actual experiences say otherwise? From bruce.hammond at contractor.verio.net Fri Apr 6 01:05:53 2012 From: bruce.hammond at contractor.verio.net (Bruce Hammond) Date: Thu, 5 Apr 2012 18:05:53 -0400 Subject: [Dovecot] dovecot: auth: Error: ldap(...) failed: No such object Message-ID: <4803B289543E1244868D9FF82F670AA002E999AD@IAD-WPRD-XCHB01.corp.verio.net> I just installed the 2.1.3 version of dovecot. When I telnet to my dovecot linux server on the imap port (143) and try to login I get the error "No [UNAVAILABLE] Temporary authentication failure" on the telnet display and dovecot: auth: Error: ldap(bhammond_test at 131.103.212.38,128.121.121.193): ldap_search(base=uid=bhammond_test at 131.103.212.38,ou=users,dc=aspect filter=(&(objectClass=posixAccount)(productFeature=mail))) failed: No such object in the /var/log/maillog. I am not familiar with ldap and I haven't been able to find anything using google. Anyone have any ideas with the problem? Thanks. Here is my configuration: # 2.1.3: /usr/local/packages/dovecot2/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-274.3.1.el5.secure i686 Red Hat Enterprise Linux Server release 5.7 (Tikanga) auth_mechanisms = plain login auth_verbose = yes default_login_user = dovecot disable_plaintext_auth = no listen = * mail_location = maildir:~/mail passdb { args = /etc/devel/passdb.conf driver = ldap } pop3_lock_session = yes protocols = imap pop3 service auth { user = dovecot } service imap-login { process_limit = 128 process_min_avail = 30 service_count = 0 user = dovecot vsz_limit = 64 M } service pop3-login { process_limit = 128 process_min_avail = 30 service_count = 0 user = dovecot vsz_limit = 64 M } shutdown_clients = no ssl_cert = References: Message-ID: A timely topic as I was just mulling over ways to provide this to my users. Charles Marcus wrote: > The snapshots are stored with the following filesystem layout: > > /path/to/snapshotsdir/hourly.0 > ... This is familiar to NetApp users. > The 'names' (hourly, daily, weekly, monthly, yearly) are arbitrary (this > is a bit confusing to people new to rsnapshot), and would *not* be used > for displaying the mail folders to the users - it is the Date/Time > stamps of each of the snapshot dirs above that would be used to display > the folder names under the 'Time Machine' namespace. This is, I imagine, > the part that will need some actual coding by Timo to get working - > maybe just some new config variables added to the namespace code for > mapping the date/time stamps of the directories to user friendly folder > names in the namespace. > > That said, I'd like to design this and have it coded such that it will > work with almost any type of backup storage that stores snapshots as > date/time stamped directories like this (there must be others, right?). One idea is to take this complexity entirely out of dovecot and create a synthetic filesystem using hard or soft links (as rsnapshot has done) and create your own, with whatever weird and wonderful naming scheme you want. /path/to/TimeMachine// -> /path/to/snapshotsdir// namespace { prefix = TimeMachine location = maildir:/path/to/TimeMachine:INDEX=MEMORY ... } This might not be very scalable depending on how big your userbase is. I would probably define memory indices for this namespace, and take the performance hit on the assumption that access will be a once-in-a- while thing. On-disk indices will probably get out of date with each snapshot rollover, if if you have a lot of snapshots/mailboxes, it could consume a non-trivial amount of space without a lot of benefits. Or you could run a cron script to rename or remove old indices, but that seems more trouble than it's worth. One other consideration (at least for me) is if the INBOX and personal mail folders are stored in two separate FS's. It would be nice to fuse the two sets of backups under the same namespace, but I don't know how the namespace prefix matching works and whether you can define hierarchical namespaces like namespace { prefix = backup/inbox location = mbox:/path/to/inbox-snapdir/%u ... } namespace { prefix = backup/mail location = mbox:/path/to/mail-snapdir/%u ... } The above can also be accomplished with a synthetic filesystem. Joseph Tam From fxmulder at gmail.com Fri Apr 6 04:16:10 2012 From: fxmulder at gmail.com (James Devine) Date: Thu, 5 Apr 2012 19:16:10 -0600 Subject: [Dovecot] distributed mdbox In-Reply-To: <1ff190f691892c01a980927422f38753@mail.airstreamcomm.net> References: <1ff190f691892c01a980927422f38753@mail.airstreamcomm.net> Message-ID: On Fri, Mar 23, 2012 at 7:39 AM, wrote: > On Wed, 21 Mar 2012 09:56:12 -0600, James Devine > wrote: > > Anyone know how to setup dovecot with mdbox so that it can be used > through > > shared storage from multiple hosts? I've setup a gluster volume and am > > sharing it between 2 test clients. I'm using postfix/dovecot LDA for > > delivery and I'm using postal to send mail between 40 users. In doing > > this, I'm seeing these errors in the logs > > > > Mar 21 09:36:29 test-gluster-client2 dovecot: lda(testuser34): Error: > Fixed > > index file /mnt/testuser34/mdbox/storage/dovecot.map.index: > messages_count > > 272 -> 271 > > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: > Log > > synchronization error at seq=4,offset=3768 for > > /mnt/testuser28/mdbox/storage/dovecot.map.index: Append with UID 516, > but > > next_uid = 517 > > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: > Log > > synchronization error at seq=4,offset=4220 for > > /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update > > for invalid uid=517 > > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: > Log > > synchronization error at seq=4,offset=5088 for > > /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update > > for invalid uid=517 > > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Warning: > > fscking index file /mnt/testuser28/mdbox/storage/dovecot.map.index > > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser34): Warning: > > fscking index file /mnt/testuser34/mdbox/storage/dovecot.map.index > > > > > > This is my dovecot config currently: > > > > jdevine at test-gluster-client2:~> dovecot -n > > # 2.0.13: /etc/dovecot/dovecot.conf > > # OS: Linux 3.0.0-13-server x86_64 Ubuntu 11.10 > > lock_method = dotlock > > mail_fsync = always > > mail_location = mdbox:~/mdbox > > mail_nfs_index = yes > > mail_nfs_storage = yes > > mmap_disable = yes > > passdb { > > driver = pam > > } > > protocols = " imap" > > ssl_cert = > ssl_key = > userdb { > > driver = passwd > > } > > I was able to get dovecot working across a gluster cluster a few weeks ago > and it worked just fine. I would recommend using the native gluster mount > option (need to install gluster software on clients), and using distributed > replicated as your replication mechanism. If you're running two gluster > servers you should have a replica count of two with distributed replicated. > You should test first to make sure you can create a file in both mounts > and see it from every mount point in the cluster, as well as interact with > it. It's also very important to make sure your servers are running with > synchronized clocks from an NTP server. Very bad things happen to a > (dovecot or gluster) cluster out of sync with NTP. > > What storage method are you using? I'm able to produce errors within seconds of starting postal with more than one thread From thomasvoneyben at gmail.com Fri Apr 6 12:52:34 2012 From: thomasvoneyben at gmail.com (Thomas von Eyben) Date: Fri, 6 Apr 2012 11:52:34 +0200 Subject: [Dovecot] Outlook (2010) -> Dovecot (IMAP) >10x slower with high network load and many folders Message-ID: Hi, I am seeing a >10x as slow performance when trying to complete a "send/receive" from an Outlook 2010 client to Dovecot via IMAP, but only when the LAN is fully loaded with other traffic, EG file copying. It seems the problem is when outlook is trying to identify folders that have changed since last "send/receive" thus traversing the hierachy. Observations: - Apple's Mail.app does not have problems when exposed to the same environment. - Eliminating Outlook is not an option. - The tests have been performed during the easter vacation, thus almost no other clients are using the network/mailserver. Under normal load there are 2-300 users using the network. Description of the environment: Server sw: Mac OS X Server 10.6.8 running dovecot: 1.1.20apple0.5 (OS = fully updated 10.6.x) Server hw: Xserve Quadcore intel Xeon 2.26 GHz 12 GB RAM - Never having a cpu load > 20%, mail is stored on a Promise vtrack RAID connected via Fibre Channel Client: Windows 7, Outlook 2010 The client have maybe 50 folders (and each folder about a handfull of subfolders) the size of the mailbox is around 3-5 GB. Test results: CLIENT-1 is having the problems when CLIENT-2 is using all the (100Mbps) bandwidth eg. copying files to MAIL-SRV. If I move CLIENT-1 to CLIENT-3 then almost all the delay is gone. NB.: I have not (yet) tested if the problem also exists when CLIENT-2 generates traffic to MAIL-SRV as opposed to OTHER-SRV (but I am expecting the same problems). When dumping the traffic on CLIENT-1 (with Wireshark) one thing is catching my eyes for the 'bad' case: There is a very long delay after each "REQUEST IDLE" until the next "REQUEST DONE" ca. 0,3 seconds. Comparing to the 'good' setup the pause at the same place in the communication is less than 1 ms ! So why this delay? Where/how shall I continue my debugging? - Run dtruss/dtrace scripts on the server? - Get dovecot to output more debug info (I guess it's doing it's best so no problems will be seen there?)? Are there any dovecot configurations that can be altered to workaround [what I think is a limitation in Outlook's IMAP implementation]? The current dovecot configuration is Apple's defaults (+POP3 disabled)) I am seing one warning from dovecotd -n though: - But I see the same warning on a clean installed not-yet-configured OS X Server so I guess it's not 'that' bad: Warning: fd limit 256 is lower than what Dovecot can use under full load (more than 456). Either grow the limit or change login_max_processes_count and max_mail_processes settings Physical setup: +---------------+ ?+------------------------------+ ? ?+-----------------+ | CLIENT-1 ? ? ?| ?| CLIENT-2 ? ? ? ? ? ? ? ? ? ? | ? ?| CLIENT-3 ? ? ? ?| | Outlook "bad" | ?| Traffic generator, eg. Samba | ? ?| Outlook ?"good" | +---------------+ ?+------------------------------+ ? ?+-----------------+ ? ? ? ? | ? ? ? ? ? ? ? ? ? ? ? | ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| +-------------------------------------------------+ ? ? ? ? ? ? | | ? ? ? ? ? 100Mbps switch (clients) ? ? ? ? ? ? ?| ? ? ? ? ? ? | +-------------------------------------------------+ ? ? ? ? ? ? | ? ? ? ? ? ? ? ? ? ? ?| ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? | +------------------------------------------------------------------------+ | ? ? ? ? ? ? ? ? ? ? 1Gbps switch (for servers) ? ? ? ? ? ? ? ? ? ? ? ? | +------------------------------------------------------------------------+ ? ? ? ? ? ? ? | ? ? ? ? ? ? ? ? ? ? ? ? | +-----------------------------+ ? ?+-----------+ | MAIL-SRV: Dovecot, AFP, SMB | ? ?| OTHER-SRV | +-----------------------------+ ? ?+-----------+ Wireshark dump - IMAP communication: |Time ? ? | 10.211.55.3 ? ? ? ? ? ? ? ? ? ? ? ? ? | | ? ? ? ? | ? ? ? ? ? ? ? ? ? | 10.0.0.10 ? ? ? ? | |9.851 ? ?| ? ? ? ? Response: gatq OK L ? ? ? ? ? |IMAP: Response: gatq OK Logged in. | ? ? ? ? |(49433) ?<------------------ ?(143) ? ?| |9.852 ? ?| ? ? ? ? Request: o47u SELEC ? ? ? ? ? |IMAP: Request: o47u SELECT "1_GROUPS" | ? ? ? ? |(49433) ?------------------> ?(143) ? ?| |9.853 ? ?| ? ? ? ? Response: * FLAGS ( ? ? ? ? ? |IMAP: Response: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) | ? ? ? ? |(49433) ?<------------------ ?(143) ? ?| |9.854 ? ?| ? ? ? ? Request: 3y4b NOOP ? ? ? ? ? ?|IMAP: Request: 3y4b NOOP | ? ? ? ? |(49433) ?------------------> ?(143) ? ?| |9.854 ? ?| ? ? ? ? Response: 3y4b OK N ? ? ? ? ? |IMAP: Response: 3y4b OK NOOP completed. | ? ? ? ? |(49433) ?<------------------ ?(143) ? ?| |9.855 ? ?| ? ? ? ? Request: 4vlj IDLE ? ? ? ? ? ?|IMAP: Request: 4vlj IDLE | ? ? ? ? |(49433) ?------------------> ?(143) ? ?| |9.856 ? ?| ? ? ? ? Response: + idling ? ? ? ? ? ?|IMAP: Response: + idling | ? ? ? ? |(49433) ?<------------------ ?(143) ? ?| |10.108 ? | ? ? ? ? Request: DONE ? ? ? ? ? ? ? ? |IMAP: Request: DONE | ? ? ? ? |(49433) ?------------------> ?(143) ? ?| |10.108 ? | ? ? ? ? Response: 4vlj OK I ? ? ? ? ? |IMAP: Response: 4vlj OK Idle completed. | ? ? ? ? |(49433) ?<------------------ ?(143) ? ?| |10.108 ? | ? ? ? ? Request: wh89 SELEC ? ? ? ? ? |IMAP: Request: wh89 SELECT "1_GROUPS.Adm" BR Thomas von Eyben From adrian.minta at gmail.com Fri Apr 6 13:23:32 2012 From: adrian.minta at gmail.com (Adrian Minta) Date: Fri, 06 Apr 2012 13:23:32 +0300 Subject: [Dovecot] Outlook (2010) -> Dovecot (IMAP) >10x slower with high network load and many folders In-Reply-To: References: Message-ID: <4F7EC424.3010504@gmail.com> It seems that you have packets lost in the network. MAC and Windows have different network stacks so this may count for the different behavior. -- Best regards, Adrian Minta From xnasx at yandex.ru Fri Apr 6 14:21:53 2012 From: xnasx at yandex.ru (Victor) Date: Fri, 06 Apr 2012 15:21:53 +0400 Subject: [Dovecot] Username bug in '%%' variables selects In-Reply-To: <10301333633892@web94.yandex.ru> References: <10301333633892@web94.yandex.ru> Message-ID: <915031333711313@web74.yandex.ru> I solved this issue by changing a namespace to native type. namespace { separator = / prefix = inbox = yes } namespace { separator = . prefix = INBOX. inbox = no hidden = yes list = no alias_for = } namespace { type = shared separator = / prefix = shared/%%u/ list = yes subscriptions = no hidden = no location = maildir:%%h:INDEX=%h/shared/%%u } It did not turn out to make the transparent migration. :( Outlooks 2007 reloaded all scheme and frozen. It's seems a bug in Dovecot. From busseniu at in.tum.de Fri Apr 6 14:51:14 2012 From: busseniu at in.tum.de (=?UTF-8?B?Q2hyaXN0b3BoIEJ1w59lbml1cw==?=) Date: Fri, 06 Apr 2012 13:51:14 +0200 Subject: [Dovecot] Courier to Dovecot migration script issue In-Reply-To: <331561333524967@web30.yandex.ru> References: <125901333456391@web125.yandex.ru> <9BBB03B5-4742-403B-8720-FBEFB554266D@iki.fi> <331561333524967@web30.yandex.ru> Message-ID: <4F7ED8B2.6030809@in.tum.de> Hi, on 04/04/2012 09:36 AM, xnasx wrote: > 'uidvalidity' are the same for both servers, but UIDs of mesages are not: > > Courier: > d fetch 1:* uid > * 1 FETCH (UID 64295) > * 2 FETCH (UID 64296) > * 3 FETCH (UID 64297) > > Dovecot: > > * 1 FETCH (UID 64296) > * 2 FETCH (UID 64297) > * 3 FETCH (UID 64298) Does your Courier directory include a courierpop3sizelist file? The migration script places higher importance on the POP3 order than on IMAP UIDs. Also, in order to see if the old 64295 corresponds to the new 64298, you can use "fetch 1:* (uid rfc822.size body.peek[header.fields (message-id)])". Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From pw at wk-serv.de Fri Apr 6 16:58:18 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Fri, 06 Apr 2012 15:58:18 +0200 Subject: [Dovecot] Director (was: Hints for a NFS-Setup) In-Reply-To: <4F75A929.9090902@mobilia.it> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> Message-ID: <4F7EF67A.8060001@wk-serv.de> Hi again, I tried to setup a test invironemnt like this: MTA --(lmtp)--\ /--(lmtp)--> backend1 --\ -- director -- -- NFS MTA --(lmtp)--/ \--(lmtp)--> backend2 --/ IMAP-User --> frontend1 --\ /--(imap)--> backend1 --\ -- director -- -- NFS IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/ but now I'm very confused. Is it actually possible to setup a host (or two) as a director only or will I have to enable the director service on each frontend and MTA? Is anyone using a similar setup and could post his configuration for the different hosts? Happy Easter! Patrick From fxmulder at gmail.com Fri Apr 6 18:39:43 2012 From: fxmulder at gmail.com (James Devine) Date: Fri, 6 Apr 2012 09:39:43 -0600 Subject: [Dovecot] distributed mdbox In-Reply-To: References: <1ff190f691892c01a980927422f38753@mail.airstreamcomm.net> Message-ID: As it turns out I can duplicate this problem with a single dovecot server and a single gluster server using mdbox, so maybe not caching? This being the case I don't think director would help On Thu, Apr 5, 2012 at 7:16 PM, James Devine wrote: > > > On Fri, Mar 23, 2012 at 7:39 AM, wrote: > >> On Wed, 21 Mar 2012 09:56:12 -0600, James Devine >> wrote: >> > Anyone know how to setup dovecot with mdbox so that it can be used >> through >> > shared storage from multiple hosts? I've setup a gluster volume and am >> > sharing it between 2 test clients. I'm using postfix/dovecot LDA for >> > delivery and I'm using postal to send mail between 40 users. In doing >> > this, I'm seeing these errors in the logs >> > >> > Mar 21 09:36:29 test-gluster-client2 dovecot: lda(testuser34): Error: >> Fixed >> > index file /mnt/testuser34/mdbox/storage/dovecot.map.index: >> messages_count >> > 272 -> 271 >> > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: >> Log >> > synchronization error at seq=4,offset=3768 for >> > /mnt/testuser28/mdbox/storage/dovecot.map.index: Append with UID 516, >> but >> > next_uid = 517 >> > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: >> Log >> > synchronization error at seq=4,offset=4220 for >> > /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update >> > for invalid uid=517 >> > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: >> Log >> > synchronization error at seq=4,offset=5088 for >> > /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update >> > for invalid uid=517 >> > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Warning: >> > fscking index file /mnt/testuser28/mdbox/storage/dovecot.map.index >> > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser34): Warning: >> > fscking index file /mnt/testuser34/mdbox/storage/dovecot.map.index >> > >> > >> > This is my dovecot config currently: >> > >> > jdevine at test-gluster-client2:~> dovecot -n >> > # 2.0.13: /etc/dovecot/dovecot.conf >> > # OS: Linux 3.0.0-13-server x86_64 Ubuntu 11.10 >> > lock_method = dotlock >> > mail_fsync = always >> > mail_location = mdbox:~/mdbox >> > mail_nfs_index = yes >> > mail_nfs_storage = yes >> > mmap_disable = yes >> > passdb { >> > driver = pam >> > } >> > protocols = " imap" >> > ssl_cert = > > ssl_key = > > userdb { >> > driver = passwd >> > } >> >> I was able to get dovecot working across a gluster cluster a few weeks ago >> and it worked just fine. I would recommend using the native gluster mount >> option (need to install gluster software on clients), and using >> distributed >> replicated as your replication mechanism. If you're running two gluster >> servers you should have a replica count of two with distributed >> replicated. >> You should test first to make sure you can create a file in both mounts >> and see it from every mount point in the cluster, as well as interact with >> it. It's also very important to make sure your servers are running with >> synchronized clocks from an NTP server. Very bad things happen to a >> (dovecot or gluster) cluster out of sync with NTP. >> >> What storage method are you using? I'm able to produce errors within > seconds of starting postal with more than one thread From stephan at rename-it.nl Fri Apr 6 20:08:17 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 06 Apr 2012 19:08:17 +0200 Subject: [Dovecot] Problem with polish characters and sieve fileinto In-Reply-To: <4F7DA264.8020004@rename-it.nl> References: <4F7D9E29.3050802@eurocar.pl> <4F7DA264.8020004@rename-it.nl> Message-ID: <4F7F2301.4020702@rename-it.nl> On 4/5/2012 3:47 PM, Stephan Bosch wrote: > Op 4/5/2012 3:29 PM, l.chelchowski at eurocar.pl schreef: >> Hi, >> >> I have problems with sieve fileinto and polish characters in folder >> names: >> # rule:[??????????????????] >> elsif anyof (header :contains "Subject" "??????????????????") >> { >> fileinto "INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-"; >> } > > Looks like your Sieve GUI uses the wrong Unicode format. The above is > mUTF7 as used by IMAP. However, Sieve expects UTF-8. So it should use > this instead: > > fileinto "INBOX/??????????????????"; Added item to the troubleshooting wiki page: http://master.wiki2.dovecot.org/Pigeonhole/Sieve/Troubleshooting#Mailbox_Names_with_non-ASCII_Characters_Cause_Problems By the looks of it, you are also using the RoundCube SieveRules plugin. The solution for RoundCube mentioned in the wiki should solve it for you. Regards, Stephan. From dmiller at amfes.com Fri Apr 6 21:53:31 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Fri, 06 Apr 2012 11:53:31 -0700 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: <4F7D972A.8050406@Media-Brokers.com> References: <4F7D972A.8050406@Media-Brokers.com> Message-ID: On 4/5/2012 5:59 AM, Charles Marcus wrote: > On 2012-04-05 4:18 AM, Thomas Leuxner wrote: > > Also with 2.x you may want to use LMTP rather than the LDA Piping. > > > > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP > > I am preparing to convert my main client's postfix_courier-imap setup > to dovecot 2.1, which currently just uses the postfix virtual delivery > agent... > > Does anyone know if the use of LMTP (or even the dovecot LDA) still > loses the x-original-to header that the postfix vda adds and that I > rely heavily on (since I use a lot of aliases), and if it does, is > there any solution to get the original recipient added back in before > final delivery? > > Everything I'm reading says that LMTP is better, but I really do need > this header (or one like it) to be there so I know who the original > recipient was (for filtering and other purposes). > I'm currently using Postfix 2.7, Dovecot 2.1, and the Dovecot LDA. I have a pure virtual user environment stored in LDAP. My messages include X-Original-To and Delivered-To headers. I had difficulty getting the LMTP transport to work previously - I may revisit that. -- Daniel From CMarcus at Media-Brokers.com Fri Apr 6 23:00:19 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 06 Apr 2012 16:00:19 -0400 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: References: <4F7D972A.8050406@Media-Brokers.com> Message-ID: <4F7F4B53.2090700@Media-Brokers.com> On 2012-04-06 2:53 PM, Daniel L. Miller wrote: > I'm currently using Postfix 2.7, Dovecot 2.1, and the Dovecot LDA. I > have a pure virtual user environment stored in LDAP. My messages include > X-Original-To and Delivered-To headers. Well that is great news... at least I'll be able to use the LDA, if not LMTP... Thanks! :) > I had difficulty getting the LMTP transport to work previously - I may > revisit that. If you do, by all means reply back on whether or not the headers are still there... Thanks again, -- Best regards, Charles From kgc at corp.sonic.net Sat Apr 7 02:40:01 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Fri, 6 Apr 2012 16:40:01 -0700 Subject: [Dovecot] POP3 dele to Trash? Message-ID: <20120406234001.GG89906@corp.sonic.net> To simplify recovery of accidentally deleted messages we'd like to send all messages that are deleted by pop clients to their Trash folder. (Which is auto expired already.) This allows a POP client who deletes an important email to restore it themselves by logging into our webmail client and move the lost message from Trash to the Inbox where it would be fetched by their client again. Has anyone already done this? Should this be possible via a plugin? I see the deleted-to-trash imap plugin. We are using Maildir if it makes a difference. -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From kgc at corp.sonic.net Sat Apr 7 03:10:26 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Fri, 06 Apr 2012 17:10:26 -0700 Subject: [Dovecot] POP3 dele to Trash? In-Reply-To: <20120406234001.GG89906@corp.sonic.net> References: <20120406234001.GG89906@corp.sonic.net> Message-ID: <4F7F85F2.3000909@corp.sonic.net> On 04/06/12 16:40, Kelsey Cummings wrote: > Has anyone already done this? Should this be possible via a plugin? > I see the deleted-to-trash imap plugin. We are using Maildir if it > makes a difference. Of course, this is exactly what the Lazy Expunge plugin does, isn't it? -K From wgillespie+dovecot at es2eng.com Sat Apr 7 04:16:14 2012 From: wgillespie+dovecot at es2eng.com (Willie Gillespie) Date: Fri, 06 Apr 2012 19:16:14 -0600 Subject: [Dovecot] Outlook (2010) -> Dovecot (IMAP) >10x slower with high network load and many folders In-Reply-To: References: Message-ID: <4F7F955E.2050508@es2eng.com> On 4/6/2012 3:52 AM, Thomas von Eyben wrote: > Test results: > CLIENT-1 is having the problems when CLIENT-2 is using all the > (100Mbps) bandwidth eg. copying files to MAIL-SRV. > If I move CLIENT-1 to CLIENT-3 then almost all the delay is gone. > NB.: I have not (yet) tested if the problem also exists when CLIENT-2 > generates traffic to MAIL-SRV as opposed to OTHER-SRV (but I am > expecting the same problems). So the link between your 100 Mbps switch and the 1 Gbps switch is saturated by CLIENT-2, so CLIENT-1 is just getting the leftovers? Since CLIENT-3 doesn't go through that 100 Mbps switch, it obviously doesn't see that issue. From plmb21 at gmail.com Sat Apr 7 06:48:43 2012 From: plmb21 at gmail.com (PL MB) Date: Fri, 6 Apr 2012 20:48:43 -0700 Subject: [Dovecot] Setting ACL for master user after login Message-ID: Hello, I'd like to log in to normal user accounts as a master user but retain the normal users' ACLs. The Master Users page on the Dovecot 1.x wiki (1) says that I can set the master user's ACLs in a postlogin script. The documentation for master users on the 2.x wiki (2) no longer has any statements about master user ACLs. Has something important in this regard changed? Can I no longer override the ACLs in a postlogin script? Thanks in advance. (1) http://wiki.dovecot.org/Authentication/MasterUsers (2) http://wiki2.dovecot.org/Authentication/MasterUsers From andy at xecu.net Sat Apr 7 10:13:42 2012 From: andy at xecu.net (Andy Dills) Date: Sat, 7 Apr 2012 03:13:42 -0400 (EDT) Subject: [Dovecot] Director pop-login and imap-login processes exiting on signal 11 Message-ID: <20120407030811.I46246@shell.xecu.net> We recently upgraded our cluster to 2.1.3, to enable director proxying. Everything appears to be working fine for the most part; the only odd thing is that I'm seeing a lot of entries in the logs like this: Apr 7 02:18:05 mail-out06 dovecot: pop3-login: Fatal: master: service(pop3-login): child 75029 killed with signal 11 (core not dumped - set service pop3-login { drop_priv_before_exec=yes }) This is on the proxy side, not that backend side. When I try to get a dump out of it, and add drop_prive_before_exec and chroot= to the pop3-login statement on the proxy, I keep running into permissions errors with the various service sockets. Any suggestions? Thanks, Andy --- Andy Dills Xecunet, Inc. www.xecu.net 301-682-9972 --- From stan at hardwarefreak.com Sat Apr 7 13:19:46 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 07 Apr 2012 05:19:46 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: References: Message-ID: <4F8014C2.3010606@hardwarefreak.com> On 4/5/2012 3:02 PM, Emmanuel Noobadmin wrote: Hi Emmanuel, > I'm trying to improve the setup of our Dovecot/Exim mail servers to > handle the increasingly huge accounts (everybody thinks it's like > infinitely growing storage like gmail and stores everything forever in > their email accounts) by changing from Maildir to mdbox, and to take > advantage of offloading older emails to alternative networked storage > nodes. I'll assume "networked storage nodes" means NFS, not FC/iSCSI SAN, in which case you'd have said "SAN". > The question now is whether having a single large server or will a > number of 1U servers with the same total capacity be better? Less complexity and cost is always better. CPU throughput isn't a factor in mail workloads--it's all about IO latency. A 1U NFS server with 12 drive JBOD is faster, cheaper, easier to setup and manage, sucks less juice and dissipates less heat than 4 1U servers each w/ 4 drives. I don't recall seeing your user load or IOPS requirements so I'm making some educated guesses WRT your required performance and total storage. I came up with the following system that should be close to suitable, for ~$10k USD. The 4 node system runs ~$12k USD. At $2k this isn't substantially higher. But when we double the storage of each architecture we're at ~$19k, vs ~$26k for an 8 node cluster, a difference of ~$7k. That's $1k shy of another 12 disk JBOD. Since CPU is nearly irrelevant for a mail workload, you can see it's much cheaper to scale capacity and IOPS with a single node w/fat storage than with skinny nodes w/thin storage. Ok, so here's the baseline config I threw together: http://h10010.www1.hp.com/wwpc/us/en/sm/WF06b/15351-15351-3328412-241644-3328421-4091396-4158470-4158440.html?dnr=1 8 cores is plenty, 2 boot drives mirrored on B110i, 16GB (4x4GB) http://www.lsi.com/products/storagecomponents/Pages/LSISAS9205-8e.aspx http://h10010.www1.hp.com/wwpc/us/en/sm/WF06b/12169-304616-3930445-3930445-3930445-3954787-4021626-4021628.html?dnr=1 w/ 12 2TB 7.2K SATA drives, configured as md concat+RAID1 pairs with 12 allocation groups, 12TB usable. Format the md device with the defaults: $ mkfs.xfs /dev/md0 Mount with inode64. No XFS stripe alignment to monkey with. No md chunk size or anything else to worry about. XFS' allocation group design is pure elegance here. If 12 TB isn't sufficient, or if you need more space later, you can daisy chain up to 3 additional D2600 JBODs for ~$8500 USD each, just add cables. This quadruples IOPS, throughput, and capacity--96TB total, 48TB net. Simply create 6 more mdraid1 devices and grow the linear array with them. Then do an xfs_growfs to bring the extra 12TB of free space into the filesystem. If you're budget conscious and/or simply prefer quality inexpensive whitebox/DIY type gear, as I do, you can get 24 x 2TB drives in one JBOD chassis for $7400 USD. That twice the drives, capacity, IOPS, for ~$2500 less than the HP JBOD. And unlike the HP 'enterprise SATA' drives, the 2TB WD Black series have a 5 year warranty, and work great with mdraid. Chassis and drives at Newegg: http://www.newegg.com/Product/Product.aspx?Item=N82E16816133047 http://www.newegg.com/Product/Product.aspx?Item=N82E16822136792 You can daisy chain 3 of these off one HBA SFF8088 port, 6 total on our LSI 9205-8e above, for a total of 144 2TB drives, 72 effective spindles in our concat+RAID1 setup, 144TB net space. > Will be > using RAID 1 pairs, likely XFS based on reading Hoeppner's > recommendation on this and the mdadm list. To be clear, the XFS configuration I recommend/promote for mailbox storage is very specific and layered. The layers must all be used together to get the performance. These layers consist of using multiple hardware or software RAID1 pairs and concatenating them with an md linear array. You then format that md device with the XFS defaults, or a specific agcount if you know how to precisely tune AG layout based on disk size and your anticipated concurrency level of writers. Putting XFS on a singe RAID1 pair, as you seem to be describing above for the multiple "thin" node case, and hitting one node with parallel writes to multiple user mail dirs, you'll get less performance than EXT3/4 on that mirror pair--possibly less than half, depending on the size of the disks and thus the number of AGs created. The 'secret' to XFS performance with this workload is concatenation of spindles. Without it you can't spread the AGs--thus directories, thus parallel file writes--horizontally across the spindles--and this is the key. By spreading AGs 'horizontally' across the disks in a concat, instead of 'vertically' down a striped array, you accomplish two important things: 1. You dramatically reduce disk head seeking by using the concat array. With XFS on a RAID10 array of 24 2TB disks you end up with 24 AGs evenly spaced vertically down each disk in the array, following the stripe pattern. Each user mailbox is stored in a different directory. Each directory was created in a different AG. So if you have 96 users writing their dovecot index concurrently, you have at worst case a minimum 192 head movements occurring back and forth across the entire platter of each disk, and likely not well optimized by TCQ/NCQ. Why 192 instead of 96? The modification time in the directory metadata must be updated for each index file, among other things. 2. Because we decrease seeks dramatically we also decrease response latency significantly. With the RAID1+concat+XFS we have 12 disks each with only 2 AGs spaced evenly down each platter. We have the same 4 user mail dirs in each AG, but in this case only 8 user mail dirs are contained on each disk instead of portions all 96. With the same 96 concurrent writes to indexes, in this case end up with only 16 seeks per drive--again, one to update each index file and one to update the metadata. Assuming these drives have a max seek rate of 150 which is the average for 7.2k drives, it will take 192/150 = 1.28 seconds for these operations on the RAID10 array. With the concat array it will only take 16/150 = 0.11 seconds. Extrapolating from that demonstrates that the concat array can handle 1.28/0.11 = 11.6*96 = 1,111 concurrent user index updates in the same time as the RAID10 array, just over 10 times more users. Granted, these are rough theoretical numbers--an index plus metadata update isn't always going to cause a seek on every chunk in a stripe, etc. But this does paint a very accurate picture of the differences in mailbox workload disk seek patterns between XFS on concat and RAID10 with the same hardware. In production one should be able to handle at minimum 2x more users, probably many more, with the RAID1+concat+XFS vs RAID10+XFS setup on the same hardware. > Currently, I'm leaning towards multiple small servers because I think > it should be better in terms of performance. This usually isn't the case with mail. It's impossible to split up the user files across the storage nodes in a way that balances block usage on each node and user access to those blocks. Hotspots are inevitable in both categories. You may achieve the same total performance of a single server, maybe slightly surpass it depending on user load, but you end up spending extra money on building resources that are idle most of the time, in the case of CPU and NICs, or under/over utilized, in the case of disk capacity in each node. Switch ports aren't horribly expensive today, but you're still wasting some with the farm setup. > At the very least even if > one node gets jammed up, the rest should still be able serve up the > emails for other accounts that is unless Dovecot will get locked up by > that jammed transaction. Some host failure redundancy is about all you'd gain from the farm setup. Dovecot shouldn't barf due to one NFS node being down, only hiccup. I.e. only imap process accessing files on the downed node would have trouble. > Also, I could possibly arrange them in a sort > of network raid 1 to gain redundancy over single machine failure. Now you're sounding like Charles Marcus, but worse. ;) Stay where you are, and brush your hair away from your forehead. I'm coming over with my branding iron that says "K.I.S.S" > Would I be correct in these or do actual experiences say otherwise? Oracles on Mount Interweb profess that 2^5 nodes wide scale out is the holy grail. IBM's mainframe evangelists tell us to put 5 million mail users on a SystemZ with hundreds of Linux VMs. I think bliss for most of us is found somewhere in the middle. -- Stan From centos.admin at gmail.com Sat Apr 7 17:43:09 2012 From: centos.admin at gmail.com (Emmanuel Noobadmin) Date: Sat, 7 Apr 2012 22:43:09 +0800 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F8014C2.3010606@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> Message-ID: On 4/7/12, Stan Hoeppner wrote: Firstly, thanks for the comprehensive reply. :) > I'll assume "networked storage nodes" means NFS, not FC/iSCSI SAN, in > which case you'd have said "SAN". I haven't decided on that but it would either be NFS or iSCSI over Gigabit. I don't exactly get a big budget for this. iSCSI because I planned to do md/mpath over two separate switches so that if one switch explodes, the email service would still work. > Less complexity and cost is always better. CPU throughput isn't a > factor in mail workloads--it's all about IO latency. A 1U NFS server > with 12 drive JBOD is faster, cheaper, easier to setup and manage, sucks > less juice and dissipates less heat than 4 1U servers each w/ 4 drives. My worry is that if that one server dies, everything is dead. With at least a pair of servers, I could keep it running, or if necessary, restore the accounts on the dead servers from backup, make some config changes and have everything back running while waiting for replacement hardware. > I don't recall seeing your user load or IOPS requirements so I'm making > some educated guesses WRT your required performance and total storage. I'm embarrassed to admit I don't have hard numbers on the user load except the rapidly dwindling disk space count and the fact when the web-based mail application try to list and check disk quota, it can bring the servers to a crawl. My lame excuse is that I'm just the web dev who got caught holding the server admin potato. > is nearly irrelevant for a mail workload, you can see it's much cheaper > to scale capacity and IOPS with a single node w/fat storage than with > skinny nodes w/thin storage. Ok, so here's the baseline config I threw > together: One of my concern is that heavy IO on the same server slow the overall performance even though the theoretical IOPS of the total drives are the same on 1 and on X servers. Right now, the servers are usually screeching to a halt, to the point of even locking out SSH access due to IOWait sending the load in top to triple digits. > Some host failure redundancy is about all you'd gain from the farm > setup. Dovecot shouldn't barf due to one NFS node being down, only > hiccup. I.e. only imap process accessing files on the downed node would > have trouble. But if I only have one big storage node and that went down, Dovecot would barf wouldn't it? Or would the mdbox format mean Dovecot would still use the local storage, just that users can't access the offloaded messages? >> Also, I could possibly arrange them in a sort >> of network raid 1 to gain redundancy over single machine failure. > > Now you're sounding like Charles Marcus, but worse. ;) Stay where you > are, and brush your hair away from your forehead. I'm coming over with > my branding iron that says "K.I.S.S" Lol, I have no idea who Charles is, but I always feel safer if there was some kind of backup. Especially since I don't have the time to dedicate myself to server administration, by the time I notice something is bad, it might be too late for anything but the backup. Of course management and clients don't agree with me since backup/redundancy costs money. :) From dmiller at amfes.com Sat Apr 7 21:06:48 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Sat, 07 Apr 2012 11:06:48 -0700 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: <4F7F4B53.2090700@Media-Brokers.com> References: <4F7D972A.8050406@Media-Brokers.com> <4F7F4B53.2090700@Media-Brokers.com> Message-ID: On 4/6/2012 1:00 PM, Charles Marcus wrote: > On 2012-04-06 2:53 PM, Daniel L. Miller wrote: >> I'm currently using Postfix 2.7, Dovecot 2.1, and the Dovecot LDA. I >> have a pure virtual user environment stored in LDAP. My messages include >> X-Original-To and Delivered-To headers. > > Well that is great news... at least I'll be able to use the LDA, if > not LMTP... > > Thanks! :) > >> I had difficulty getting the LMTP transport to work previously - I may >> revisit that. > > If you do, by all means reply back on whether or not the headers are > still there... > > Thanks again, > From the documentation... http://www.postfix.org/virtual.8.html The*virtual*(8) delivery agent prepends a "*From* /sender/ /time/*_*/stamp/" envelope header to each message, prepends a *Delivered-To:* message header with the envelope recipient address, prepends an*X-Original-To:* header with the recip- ient address as given to Postfix, prepends a*Return-Path:* message header with the envelope sender address, prepends a> character to lines beginning with "*From* ", and appends an empty line. Using the Postfix pipe agent, which is what is used with the Dovecot LDA, http://www.postfix.org/pipe.8.html *flags=BDFORXhqu.*> (optional) Optional message processing flags. By default, a message is copied unchanged. *B* Append a blank line at the end of each mes- sage. This is required by some mail user agents that recognize "*From* " lines only when preceded by a blank line. *D* Prepend a "*Delivered-To:* /recipient/" message header with the envelope recipient address. Note: for this to work, the/transport/*_desti-* *nation_recipient_limit* must be 1 (see SIN- GLE-RECIPIENT DELIVERY above for details). The*D* flag also enforces loop detection (Postfix 2.5 and later): if a message already contains a*Delivered-To:* header with the same recipient address, then the message is returned as undeliverable. The address comparison is case insensitive. This feature is available as of Postfix 2.0. *F* Prepend a "*From* /sender time/*_*/stamp/" envelope header to the message content. This is expected by, for example,*UUCP* software. *O* Prepend an "*X-Original-To:* /recipient/" mes- sage header with the recipient address as given to Postfix. Note: for this to work, the*/transport/_destination_recipient_limit * must be 1 (see SINGLE-RECIPIENT DELIVERY above for details). Unfortunately, the docs for the ltmp agent http://www.postfix.org/lmtp.8.html don't say anything about adding these headers. I tried asking on the Postfix list - didn't get much of an answer. -- Daniel From jerry at seibercom.net Sat Apr 7 21:30:38 2012 From: jerry at seibercom.net (Jerry) Date: Sat, 7 Apr 2012 14:30:38 -0400 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: References: <4F7D972A.8050406@Media-Brokers.com> <4F7F4B53.2090700@Media-Brokers.com> Message-ID: <20120407143038.5252acf9@scorpio> On Sat, 07 Apr 2012 11:06:48 -0700 Daniel L. Miller articulated: > Unfortunately, the docs for the ltmp agent > http://www.postfix.org/lmtp.8.html don't say anything about adding > these headers. I tried asking on the Postfix list - didn't get much > of an answer. I may be wrong; however, from what I have been able to understand in regards to the Postfix documentation, if it does not explicitly claim to have a feature, then that feature is not available. In other words, if it doesn't state it can do it, it can't. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From thomasvoneyben at gmail.com Sat Apr 7 23:13:15 2012 From: thomasvoneyben at gmail.com (Thomas von Eyben) Date: Sat, 7 Apr 2012 22:13:15 +0200 Subject: [Dovecot] Outlook (2010) -> Dovecot (IMAP) >10x slower with high network load and many folders In-Reply-To: <4F7F955E.2050508@es2eng.com> References: <4F7F955E.2050508@es2eng.com> Message-ID: On Sat, Apr 7, 2012 at 3:16 AM, Willie Gillespie wrote: > On 4/6/2012 3:52 AM, Thomas von Eyben wrote: >> >> Test results: >> CLIENT-1 is having the problems when CLIENT-2 is using all the >> (100Mbps) bandwidth eg. copying files to MAIL-SRV. >> If I move CLIENT-1 to CLIENT-3 then almost all the delay is gone. >> NB.: I have not (yet) tested if the problem also exists when CLIENT-2 >> generates traffic to MAIL-SRV as opposed to OTHER-SRV (but I am >> expecting the same problems). > > > So the link between your 100 Mbps switch and the 1 Gbps switch is saturated > by CLIENT-2, so CLIENT-1 is just getting the leftovers? > > Since CLIENT-3 doesn't go through that 100 Mbps switch, it obviously doesn't > see that issue. Yes - that's my current "workaround" (perhaps also solution), I'm wondering if the performance is really expected to be _so_ bad when other users are utilizing the LAN. (You seem to indicate that what I am observing is expected and is "just" caused by [un-intended] semi-bad behavior from other users?) BR TvE From dovecot at r.paypc.com Sat Apr 7 23:45:08 2012 From: dovecot at r.paypc.com (Robin) Date: Sat, 07 Apr 2012 13:45:08 -0700 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F8014C2.3010606@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> Message-ID: <4F80A754.4000703@r.paypc.com> > Putting XFS on a singe RAID1 pair, as you seem to be describing above > for the multiple "thin" node case, and hitting one node with parallel > writes to multiple user mail dirs, you'll get less performance than > EXT3/4 on that mirror pair--possibly less than half, depending on the > size of the disks and thus the number of AGs created. The 'secret' to > XFS performance with this workload is concatenation of spindles. > Without it you can't spread the AGs--thus directories, thus parallel > file writes--horizontally across the spindles--and this is the key. By > spreading AGs 'horizontally' across the disks in a concat, instead of > 'vertically' down a striped array, you accomplish two important things: > > 1. You dramatically reduce disk head seeking by using the concat array. > With XFS on a RAID10 array of 24 2TB disks you end up with 24 AGs > evenly spaced vertically down each disk in the array, following the > stripe pattern. Each user mailbox is stored in a different directory. > Each directory was created in a different AG. So if you have 96 users > writing their dovecot index concurrently, you have at worst case a > minimum 192 head movements occurring back and forth across the entire > platter of each disk, and likely not well optimized by TCQ/NCQ. Why 192 > instead of 96? The modification time in the directory metadata must be > updated for each index file, among other things. Does the XFS allocator automatically distribute AGs in this way even when disk usage is extremely light, i.e, a freshly formatted system with user directories initially created, and then the actual mailbox contents copied into them? If this is indeed the case, then what you describe is a wondrous revelation, since you're scaling out the number of simultaneous metadata reads+writes/second as you add RAID1 pairs, if my understanding of this is correct. I'm assuming of course, but should look at the code, that metadata locks imposed by the filesystem "distribute" as the number of pairs increase - if it's all just one Big Lock, then that wouldn't be the case. Forgive my laziness, as I could just experiment and take a look at the on-disk structures myself, but I don't have four empty drives handy to experiment. The bandwidth improvements due to striping (RAID0/5/6 style) are no help for metadata-intensive IO loads, and probably of little value for even mdbox loads too, I suspect, unless the mdbox max size is set to something pretty large, no? Have you tried other filesystems and seen if they distribute metadata in a similarly efficient and scalable manner across concatenated drive sets? Is there ANY point to using striping at all, a la "RAID10" in this? I'd have thought just making as many RAID1 pairs out of your drives as possible would be the ideal strategy - is this not the case? =R= From stan at hardwarefreak.com Sun Apr 8 03:46:20 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 07 Apr 2012 19:46:20 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F80A754.4000703@r.paypc.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F80A754.4000703@r.paypc.com> Message-ID: <4F80DFDC.2030703@hardwarefreak.com> On 4/7/2012 3:45 PM, Robin wrote: > >> Putting XFS on a singe RAID1 pair, as you seem to be describing above >> for the multiple "thin" node case, and hitting one node with parallel >> writes to multiple user mail dirs, you'll get less performance than >> EXT3/4 on that mirror pair--possibly less than half, depending on the >> size of the disks and thus the number of AGs created. The 'secret' to >> XFS performance with this workload is concatenation of spindles. >> Without it you can't spread the AGs--thus directories, thus parallel >> file writes--horizontally across the spindles--and this is the key. By >> spreading AGs 'horizontally' across the disks in a concat, instead of >> 'vertically' down a striped array, you accomplish two important things: >> >> 1. You dramatically reduce disk head seeking by using the concat array. >> With XFS on a RAID10 array of 24 2TB disks you end up with 24 AGs >> evenly spaced vertically down each disk in the array, following the >> stripe pattern. Each user mailbox is stored in a different directory. >> Each directory was created in a different AG. So if you have 96 users >> writing their dovecot index concurrently, you have at worst case a >> minimum 192 head movements occurring back and forth across the entire >> platter of each disk, and likely not well optimized by TCQ/NCQ. Why 192 >> instead of 96? The modification time in the directory metadata must be >> updated for each index file, among other things. > > Does the XFS allocator automatically distribute AGs in this way even > when disk usage is extremely light, i.e, a freshly formatted system with > user directories initially created, and then the actual mailbox contents > copied into them? It doesn't distribute AGs. There are a static number created during mkfs.xfs. The inode64 allocator round robins new directory creation across the AGs, and does the same with files created in those directories. Having the directory metadata and file extents in the same AG decreases head movement and thus seek latency for mixed metadata/extent high IOPS workloads. > If this is indeed the case, then what you describe is a wondrous > revelation, since you're scaling out the number of simultaneous metadata > reads+writes/second as you add RAID1 pairs, if my understanding of this > is correct. Correct. And adding more space and IOPS is uncomplicated. No chunk calculations, no restriping of the array. You simply grow the md linear array adding the new disk device. Then grow XFS to add the new free space to the filesystem. AFAIK this can be done infinitely, theoretically. I'm guessing md has a device count limit somewhere. If not your bash line buffer might. ;) > I'm assuming of course, but should look at the code, that > metadata locks imposed by the filesystem "distribute" as the number of > pairs increase - if it's all just one Big Lock, then that wouldn't be > the case. XFS locking is done as minimally as possibly and is insanely fast. I've not come across any reported performance issues relating to it. And yes, any single metadata lock will occur in a single AG on one mirror pair using the concat setup. > Forgive my laziness, as I could just experiment and take a look at the > on-disk structures myself, but I don't have four empty drives handy to > experiment. Don't sweat it. All of this stuff is covered in the XFS Filesystem Structure Guide, exciting reading if you enjoy a root canal while watching snales race: http://xfs.org/docs/xfsdocs-xml-dev/XFS_Filesystem_Structure//tmp/en-US/html/index.html > The bandwidth improvements due to striping (RAID0/5/6 style) are no help > for metadata-intensive IO loads, and probably of little value for even > mdbox loads too, I suspect, unless the mdbox max size is set to > something pretty large, no? The problem with striped parity RAID is not allocation, which takes place in free space and is pretty fast. The problem is the extra read seeks and bandwidth of the RMW cycle when you modify an existing stripe. Updating a single flag in a Dovecot index causes md or the hardware RAID controller to read the entire stripe into buffer space or RAID cache, modify the flag byte, recalculate parity, then write the whole stripe and parity block back out across all the disks. With a linear concat of RAID1 pairs we're simply rewriting a single 4KB filesystem block, maybe only a single 512B sector. I'm at the edge of my knowledge here. I don't know exactly how Timo does the index updates. Regardless of the method, the index update is light years faster with the concat setup as there is no RMW and full stripe writeback as with the RAID5/6 case. > Have you tried other filesystems and seen if they distribute metadata in > a similarly efficient and scalable manner across concatenated drive sets? EXT, any version, does not. ReiserFS does not. Both require disk striping to achieve any parallelism. With concat they both simply start writing at the beginning sectors of the first RAID1 pair and 4 years later maybe reach the last pair as they fill up the volume. ;) JFS has a more advanced allocation strategy that EXT or ReiserFS, not as advanced as XFS. I've never read of a concat example with JFS and I've never tested it. It's all but a dead filesystem at this point anyway, less than 2 dozen commits in 8 years last I checked, and these were simple bug fixes and changes to keep it building on new kernels. If it's not suffering bit rot now I'm sure it will be in the near future. > Is there ANY point to using striping at all, a la "RAID10" in this? I'd > have thought just making as many RAID1 pairs out of your drives as > possible would be the ideal strategy - is this not the case? If you're using XFS, and your workload is overwhelmingly mail, RAID1+concat is the only way to fly, and it flies. If the workload is not mail, say large file streaming writes, then you're limited to 100-200MB/s, a single drive of throughput, as each file is written to a single directory on a single AG on a single disk. For streaming write performance you'll need striping. If you have many concurrent large streaming writes, you'll want to concat multiple striped arrays. -- Stan From stan at hardwarefreak.com Sun Apr 8 21:21:47 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sun, 08 Apr 2012 13:21:47 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: References: <4F8014C2.3010606@hardwarefreak.com> Message-ID: <4F81D73B.9030901@hardwarefreak.com> On 4/7/2012 9:43 AM, Emmanuel Noobadmin wrote: > On 4/7/12, Stan Hoeppner wrote: > > Firstly, thanks for the comprehensive reply. :) > >> I'll assume "networked storage nodes" means NFS, not FC/iSCSI SAN, in >> which case you'd have said "SAN". > > I haven't decided on that but it would either be NFS or iSCSI over > Gigabit. I don't exactly get a big budget for this. iSCSI because I > planned to do md/mpath over two separate switches so that if one > switch explodes, the email service would still work. So it seems you have two courses of action: 1. Identify individual current choke points and add individual systems and storage to eliminate those choke points. 2. Analyze your entire workflow and all systems, identifying all choke points, then design a completely new well integrated storage architecture that solves all current problems and addresses future needs Adding an NFS server and moving infrequently (old) accessed files to alternate storage will alleviate your space problems. But it will probably not fix some of the other problems you mention, such as servers bogging down and becoming unresponsive, as that's not a space issue. The cause of that would likely be an IOPS issue, meaning you don't have enough storage spindles to service requests in a timely manner. >> Less complexity and cost is always better. CPU throughput isn't a >> factor in mail workloads--it's all about IO latency. A 1U NFS server >> with 12 drive JBOD is faster, cheaper, easier to setup and manage, sucks >> less juice and dissipates less heat than 4 1U servers each w/ 4 drives. > > My worry is that if that one server dies, everything is dead. With at > least a pair of servers, I could keep it running, or if necessary, > restore the accounts on the dead servers from backup, make some config > changes and have everything back running while waiting for replacement > hardware. You are a perfect candidate for VMware ESX. The HA feature will do exactly what you want. If one physical node in the cluster dies, HA automatically restarts the dead VMs on other nodes, transparently. Clients will will have to reestablish connections, but everything else will pretty much be intact. Worse case scenario will possibly be a few corrupted mailboxes that were being written when the hardware crashed. A SAN is required for such a setup. I had extensive experience with ESX and HA about 5 years ago and it works as advertised. After 5 years it can only have improved. It's not "cheap" but usually pays for itself due to being able to consolidate the workload of dozens of physical servers into just 2 or 3 boxes. >> I don't recall seeing your user load or IOPS requirements so I'm making >> some educated guesses WRT your required performance and total storage. > > I'm embarrassed to admit I don't have hard numbers on the user load > except the rapidly dwindling disk space count and the fact when the > web-based mail application try to list and check disk quota, it can > bring the servers to a crawl. Maybe just starting with a description of your current hardware setup and number of total users/mailboxes would be a good starting point. How many servers do you have, what storage is connected to each, percent of MUA POP/IMAP connections from user PCs versus those from webmail applications, etc, etc. Probably the single most important piece of information would be the hardware specs of your current Dovecot server, CPUs/RAM/disk array, etc, and what version of Dovecot you're running. The focus of your email is building a storage server strictly to offload old mail and free up space on the Dovecot server. From the sound of things, this may not be sufficient to solve all your problems. > My lame excuse is that I'm just the web > dev who got caught holding the server admin potato. Baptism by fire. Ouch. What doesn't kill you makes you stronger. ;) >> is nearly irrelevant for a mail workload, you can see it's much cheaper >> to scale capacity and IOPS with a single node w/fat storage than with >> skinny nodes w/thin storage. Ok, so here's the baseline config I threw >> together: > > One of my concern is that heavy IO on the same server slow the overall > performance even though the theoretical IOPS of the total drives are > the same on 1 and on X servers. Right now, the servers are usually > screeching to a halt, to the point of even locking out SSH access due > to IOWait sending the load in top to triple digits. If multiple servers are screeching to a halt due to iowait, either all of your servers individual disks are overloaded, or you already have shared storage. We really need more info on your current architecture. Right now we don't knw if we're talking about 4 servers or 40., 100 users or 10,000. >> Some host failure redundancy is about all you'd gain from the farm >> setup. Dovecot shouldn't barf due to one NFS node being down, only >> hiccup. I.e. only imap process accessing files on the downed node would >> have trouble. > > But if I only have one big storage node and that went down, Dovecot > would barf wouldn't it? > Or would the mdbox format mean Dovecot would still use the local > storage, just that users can't access the offloaded messages? If the big storage node is strictly alt storage, and it dies, Dovecot will still access its main mdbox storage just fine. It simply wouldn't be able to access the alt storage and would log errors for those requests. If you design a whole new architecture from scratch, going with ESX and an iSCSI SAN this whole line of thinking is moot as there is no SPOF. This can be done with as little as two physical servers and one iSCSI SAN array which has dual redundant controllers in the base config. Depending on your actual IOPS needs, you could possibly consolidate everything you have now into two physical servers and one iSCSI SAN array, for between $30-40K USD in hardware and $8-10K in ESX licenses. That just a guess on ESX as I don't know the current pricing. Even if it's that "high" it's far more than worth the price due to the capability. Such a setup allows you to run all of your Exim, webmail, Dovecot, etc servers on two machines, and you usually get much better performance than with individual boxes, especially if you manually place the VMs on the nodes for lowest network latency. For instance, if you place your webmail server VM on the same host as the Dovecot VM, TCP packet latency drops from the high micro/low milliscond range into the mid nanosecond range--a 1000x decrease in latency. Why? The packet transfer is now a memory-to-memory copy through the hypervisor. The packets never reach a physical network interface. You can do some of these things with free Linux hypervisors, but AFAIK the poor management interfaces for them make the price of ESX seem like a bargain. >>> Also, I could possibly arrange them in a sort >>> of network raid 1 to gain redundancy over single machine failure. >> >> Now you're sounding like Charles Marcus, but worse. ;) Stay where you >> are, and brush your hair away from your forehead. I'm coming over with >> my branding iron that says "K.I.S.S" > Lol, I have no idea who Charles is, but I always feel safer if there > was some kind of backup. Especially since I don't have the time to > dedicate myself to server administration, by the time I notice > something is bad, it might be too late for anything but the backup. Search the list archives for Charles' thread about bringing up a 2nd office site. His desire was/is to duplicate machines at the 2nd site for redundancy, when the proper thing to do is duplicate them at the primary site, and simply duplicate the network links between sites. My point to you and Charles is that you never add complexity for the sake of adding complexity. > Of course management and clients don't agree with me since > backup/redundancy costs money. :) So does gasoline, but even as the price has more than doubled in 3 years in the States, people haven't stopped buying it. Why? They have to have it. The case is the same for certain levels of redundancy. You simply have to have it. You job is properly explaining that need. Ask the CEO/CFO how much money the company will lose in productivity if nobody has email for 1 workday, as that is how long it will take to rebuild it from scratch and restore all the data when it fails. Then ask what the cost is if all the email is completely lost because they were to cheap to pay for a backup solution? To executives, money in the bank is like the family jewels in their trousers. Kicking the family jewels and generating that level of pain seriously gets their attention. Likewise, if a failed server plus rebuild/restore costs $50K in lost productivity, spending $20K on a solution to prevent that from happening is a good investment. Explain it in terms execs understand. Have industry data to back your position. There plenty of it available. -- Stan From dmiller at amfes.com Mon Apr 9 04:20:19 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Sun, 08 Apr 2012 18:20:19 -0700 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: <20120407143038.5252acf9@scorpio> References: <4F7D972A.8050406@Media-Brokers.com> <4F7F4B53.2090700@Media-Brokers.com> <20120407143038.5252acf9@scorpio> Message-ID: On Sat, 7 Apr 2012 14:30:38 -0400, Jerry wrote: > On Sat, 07 Apr 2012 11:06:48 -0700 > Daniel L. Miller articulated: > >> Unfortunately, the docs for the ltmp agent http://www.postfix.org/lmtp.8.html [1] don't say anything about adding these headers. I tried asking on the Postfix list - didn't get much of an answer. > > I may be wrong; however, from what I have been able to understand in > regards to the Postfix documentation, if it does not explicitly claim to > have a feature, then that feature is not available. In other words, if > it doesn't state it can do it, it can't. As I just stated on that list - even though a given feature may be documented, the possible uses of that feature may not be immediately apparent. And while the Postfix lda & virtual transports have the "flag" parameters, and the lmtp transport does not - the lmtp transport DOES have a whole slew of other parameters not available in the lda. So I was simply inquiring if there was a way to achieve my goal - given that my understanding of smtp handling in general, and Postfix in particular, are extremely limited. For some reason, I seem to irritate people with my polite questions - while others who are (in my opinion) downright rude and aggressive get assistance and acceptance. Maybe I need to start being more of a jerk on purpose... -- Daniel Links: ------ [1] http://www.postfix.org/lmtp.8.html From tss at iki.fi Mon Apr 9 09:47:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 09:47:11 +0300 Subject: [Dovecot] lmtp sometimes fails to deliver a message to all recipients In-Reply-To: <4F7C724E.8040905@talex.pl> References: <4F79C995.9060607@talex.pl> <1B1369FF-6748-46DE-AB94-AF2F32061757@iki.fi> <4F7C724E.8040905@talex.pl> Message-ID: <981BEDB0-63E3-43C1-906A-390E4BD053CA@iki.fi> On 4.4.2012, at 19.09, Artur Zaprza?a wrote: >>> lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Message was expunged (guid) >>> lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: script /vmail/domain/foo/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /vmail/domain/foo/.dovecot.sieve.log may reveal additional details) >> Fixed in hg. >> > Tested with Maildir. Works great. Thanks. > > The above problem was appearing when some recipients (including first one) had a sieve filter with discard action for current message. In this case, depending on the pattern of recipients having a sieve discard action, lmtp can create more than one instance of the message for a few dozen recipients. It would be nice if lmtp could create a single hardlinked instance of the message even in this case. The problem here isn't the discard action, but that Sieve is used at all. The hard linking happens currently only for users who don't have Sieve scripts. I've a plan to fix this, but it's not a simple fix and it's pretty low priority currently. From tss at iki.fi Mon Apr 9 09:54:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 09:54:11 +0300 Subject: [Dovecot] 2.1.3: Overly lax FETCH parsing In-Reply-To: <20120405125920.Horde.wazxEIF5lbhPfeuIm_gQ4uA@bigworm.curecanti.org> References: <20120405125920.Horde.wazxEIF5lbhPfeuIm_gQ4uA@bigworm.curecanti.org> Message-ID: On 5.4.2012, at 21.59, Michael M Slusarz wrote: > While useful that Dovecot is more liberal about what it receives, 3501 seems pretty clear that incorrect FETCH parameters must return a BAD. I can verify that the above commands fail on Cyrus. It's a SHOULD, not a MUST: Servers SHOULD enforce the syntax outlined in this specification strictly. Any client command with a protocol syntax error, including (but not limited to) missing or extraneous spaces or arguments, SHOULD be rejected, and the client given a BAD server completion response. But since it's not much trouble to fix it: http://hg.dovecot.org/dovecot-2.1/rev/19e09ab09383 From tss at iki.fi Mon Apr 9 09:59:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 09:59:55 +0300 Subject: [Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups In-Reply-To: References: Message-ID: <558E2E13-E08D-46F3-B68A-0AA63DF43D38@iki.fi> On 6.4.2012, at 1.46, Joseph Tam wrote: > One other consideration (at least for me) is if the INBOX and > personal mail folders are stored in two separate FS's. It would be nice > to fuse the two sets of backups under the same namespace, but I don't > know how the namespace prefix matching works and whether you can define > hierarchical namespaces like > > namespace { > prefix = backup/inbox > location = mbox:/path/to/inbox-snapdir/%u > ... > } > > namespace { > prefix = backup/mail > location = mbox:/path/to/mail-snapdir/%u > ... > } You can define hierarchical namespaces, although they've probably not been used outside my few tests. Well, except shared// autocreated namespaces are already children to shared/ namespace, so I guess they should work. From tss at iki.fi Mon Apr 9 10:03:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:03:39 +0300 Subject: [Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups In-Reply-To: <4F7DD03B.30909@Media-Brokers.com> References: <4F7DBA3B.5090803@Media-Brokers.com> <4F7DCA4A.70009@whyscream.net> <4F7DD03B.30909@Media-Brokers.com> Message-ID: On 5.4.2012, at 20.02, Charles Marcus wrote: > On 2012-04-05 12:37 PM, Tom Hendrikx wrote: >> The first interesting point I'd see with this, is that you supply the >> mail client with a near endless supply of folders, which would take a >> lot of caching space on the clients end, either (depending on the client >> and its configuration) from the moment that you enable this fort hem, or >> after someone starts searching in their 'time machine' for some old mail. > > Since we use Thunderbird, I can of course disable offline mode for everyone, so the only time headers would be downloaded would be when the user selects (or performs a search on) one (or more) of the folders. Do they need to be accessible via Thunderbird, or maybe only via a webmail? Or perhaps a secondary (normally disabled?) TB account where you've specified a "backup/" namespace prefix (which is normally hidden)? From tss at iki.fi Mon Apr 9 10:07:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:07:40 +0300 Subject: [Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups In-Reply-To: <4F7DBA3B.5090803@Media-Brokers.com> References: <4F7DBA3B.5090803@Media-Brokers.com> Message-ID: On 5.4.2012, at 18.28, Charles Marcus wrote: > The snapshots are stored with the following filesystem layout: > > /path/to/snapshotsdir/hourly.0 > ... > /path/to/snapshotsdir/hourly.4 > /path/to/snapshotsdir/daily.0 .. > The 'names' (hourly, daily, weekly, monthly, yearly) are arbitrary (this is a bit confusing to people new to rsnapshot), and would *not* be used for displaying the mail folders to the users - it is the Date/Time stamps of each of the snapshot dirs above that would be used to display the folder names under the 'Time Machine' namespace. This is, I imagine, the part that will need some actual coding by Timo to get working - maybe just some new config variables added to the namespace code for mapping the date/time stamps of the directories to user friendly folder names in the namespace. I guess there could be kind of a "filter fs layout" that modifies the filesystem layout a bit and lets the underlying layout handle the rest: namespace { location = maildir:/path/to/snapshotsdir:LAYOUT=timestamp } Although it's annoying that it's not possible to have per-layout settings currently.. But I guess if this was implemented as plugin it would be enough to have: plugin { timestamp_layout = maildir++ } From tss at iki.fi Mon Apr 9 10:13:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:13:39 +0300 Subject: [Dovecot] Director (was: Hints for a NFS-Setup) In-Reply-To: <4F7EF67A.8060001@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F7EF67A.8060001@wk-serv.de> Message-ID: <613701AE-34D1-4366-887E-72F576A01A1D@iki.fi> On 6.4.2012, at 16.58, Patrick Westenberg wrote: > Hi again, > > I tried to setup a test invironemnt like this: > > MTA --(lmtp)--\ /--(lmtp)--> backend1 --\ > -- director -- -- NFS > MTA --(lmtp)--/ \--(lmtp)--> backend2 --/ > > > IMAP-User --> frontend1 --\ /--(imap)--> backend1 --\ > -- director -- -- NFS > IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/ > > but now I'm very confused. Is it actually possible to setup a host (or two) as a director only or will I have to enable the director service on each frontend and MTA? The cleanest way to run director is to have 2 or more servers running only director itself. If you want to have less servers, it's also possible to place a Dovecot director configuration to any other servers as well, but that's conceptually more complex. For MTA you'd simply tell its LMTP client to connect to director servers, which could be one of: a) Load balancer's IP address b) Host name that expands to all directors' IP addresses. If the first one is down, the LMTP client (hopefully! verify!) connects to the second one. From tss at iki.fi Mon Apr 9 10:15:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:15:37 +0300 Subject: [Dovecot] distributed mdbox In-Reply-To: References: <1ff190f691892c01a980927422f38753@mail.airstreamcomm.net> Message-ID: <7864D800-17D2-48E3-832D-D44FA912FC5C@iki.fi> Yeah, not caching then. I know Glusterfs people implemented some fixes/workarounds to make Dovecot work better. I don't know if all of those fixes are in the public glusterfs. On 6.4.2012, at 18.39, James Devine wrote: > As it turns out I can duplicate this problem with a single dovecot server > and a single gluster server using mdbox, so maybe not caching? This being > the case I don't think director would help > > On Thu, Apr 5, 2012 at 7:16 PM, James Devine wrote: > >> >> >> On Fri, Mar 23, 2012 at 7:39 AM, wrote: >> >>> On Wed, 21 Mar 2012 09:56:12 -0600, James Devine >>> wrote: >>>> Anyone know how to setup dovecot with mdbox so that it can be used >>> through >>>> shared storage from multiple hosts? I've setup a gluster volume and am >>>> sharing it between 2 test clients. I'm using postfix/dovecot LDA for >>>> delivery and I'm using postal to send mail between 40 users. In doing >>>> this, I'm seeing these errors in the logs >>>> >>>> Mar 21 09:36:29 test-gluster-client2 dovecot: lda(testuser34): Error: >>> Fixed >>>> index file /mnt/testuser34/mdbox/storage/dovecot.map.index: >>> messages_count >>>> 272 -> 271 >>>> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: >>> Log >>>> synchronization error at seq=4,offset=3768 for >>>> /mnt/testuser28/mdbox/storage/dovecot.map.index: Append with UID 516, >>> but >>>> next_uid = 517 >>>> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: >>> Log >>>> synchronization error at seq=4,offset=4220 for >>>> /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update >>>> for invalid uid=517 >>>> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: >>> Log >>>> synchronization error at seq=4,offset=5088 for >>>> /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update >>>> for invalid uid=517 >>>> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Warning: >>>> fscking index file /mnt/testuser28/mdbox/storage/dovecot.map.index >>>> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser34): Warning: >>>> fscking index file /mnt/testuser34/mdbox/storage/dovecot.map.index >>>> >>>> >>>> This is my dovecot config currently: >>>> >>>> jdevine at test-gluster-client2:~> dovecot -n >>>> # 2.0.13: /etc/dovecot/dovecot.conf >>>> # OS: Linux 3.0.0-13-server x86_64 Ubuntu 11.10 >>>> lock_method = dotlock >>>> mail_fsync = always >>>> mail_location = mdbox:~/mdbox >>>> mail_nfs_index = yes >>>> mail_nfs_storage = yes >>>> mmap_disable = yes >>>> passdb { >>>> driver = pam >>>> } >>>> protocols = " imap" >>>> ssl_cert = >>> ssl_key = >>> userdb { >>>> driver = passwd >>>> } >>> >>> I was able to get dovecot working across a gluster cluster a few weeks ago >>> and it worked just fine. I would recommend using the native gluster mount >>> option (need to install gluster software on clients), and using >>> distributed >>> replicated as your replication mechanism. If you're running two gluster >>> servers you should have a replica count of two with distributed >>> replicated. >>> You should test first to make sure you can create a file in both mounts >>> and see it from every mount point in the cluster, as well as interact with >>> it. It's also very important to make sure your servers are running with >>> synchronized clocks from an NTP server. Very bad things happen to a >>> (dovecot or gluster) cluster out of sync with NTP. >>> >>> What storage method are you using? I'm able to produce errors within >> seconds of starting postal with more than one thread From tss at iki.fi Mon Apr 9 10:17:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:17:42 +0300 Subject: [Dovecot] POP3 dele to Trash? In-Reply-To: <4F7F85F2.3000909@corp.sonic.net> References: <20120406234001.GG89906@corp.sonic.net> <4F7F85F2.3000909@corp.sonic.net> Message-ID: <540057BA-FA72-4A34-B9C1-EBEA8458A210@iki.fi> On 7.4.2012, at 3.10, Kelsey Cummings wrote: > On 04/06/12 16:40, Kelsey Cummings wrote: >> Has anyone already done this? Should this be possible via a plugin? >> I see the deleted-to-trash imap plugin. We are using Maildir if it >> makes a difference. > > Of course, this is exactly what the Lazy Expunge plugin does, isn't it? Not exactly, the messages would go to lazyexpunge-namespace-prefix/INBOX. But maybe close enough? Otherwise would require writing a new plugin. From tss at iki.fi Mon Apr 9 10:27:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:27:04 +0300 Subject: [Dovecot] Setting ACL for master user after login In-Reply-To: References: Message-ID: <1F913824-E90D-4807-ABC4-78E119303CF0@iki.fi> On 7.4.2012, at 6.48, PL MB wrote: > I'd like to log in to normal user accounts as a master user but retain > the normal users' ACLs. > > The Master Users page on the Dovecot 1.x wiki (1) says that I can set > the master user's ACLs in a postlogin script. The documentation for > master users on the 2.x wiki (2) no longer has any statements about > master user ACLs. > > Has something important in this regard changed? Can I no longer > override the ACLs in a postlogin script? No, it's just that the ACL text was added there after wiki2 was forked. I updated now http://master.wiki2.dovecot.org/Authentication/MasterUsers#ACLs I'm pretty sure the userdb way works in v2.1, possibly also in v2.0 and probably not in v1.x. From tss at iki.fi Mon Apr 9 10:29:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:29:20 +0300 Subject: [Dovecot] Director pop-login and imap-login processes exiting on signal 11 In-Reply-To: <20120407030811.I46246@shell.xecu.net> References: <20120407030811.I46246@shell.xecu.net> Message-ID: <70AB2E58-42C9-49B6-BE13-C553893E19EF@iki.fi> On 7.4.2012, at 10.13, Andy Dills wrote: > Apr 7 02:18:05 mail-out06 dovecot: pop3-login: Fatal: master: > service(pop3-login): child 75029 killed with signal 11 (core not dumped - > set service pop3-login { drop_priv_before_exec=yes }) v2.1.3 proxying was buggy with SSL connections. Probably crashes because of that. I was supposed to release v2.1.4 already but.. From tss at iki.fi Mon Apr 9 10:33:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:33:18 +0300 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: <4F7D972A.8050406@Media-Brokers.com> References: <4F7D972A.8050406@Media-Brokers.com> Message-ID: On 5.4.2012, at 15.59, Charles Marcus wrote: > Does anyone know if the use of LMTP (or even the dovecot LDA) still loses the x-original-to header that the postfix vda adds and that I rely heavily on (since I use a lot of aliases), and if it does, is there any solution to get the original recipient added back in before final delivery? LMTP adds a new Delivered-To: header when there is a single RCPT TO. You can force a single RCPT TO from Postfix side by setting lmtp_destination_recipient_limit=1. LMTP doesn't add/remove/change X-Original-To: header. From stu at spacehopper.org Mon Apr 9 10:49:35 2012 From: stu at spacehopper.org (Stuart Henderson) Date: Mon, 9 Apr 2012 07:49:35 +0000 (UTC) Subject: [Dovecot] Outlook (2010) -> Dovecot (IMAP) >10x slower with high network load and many folders References: Message-ID: On 2012-04-06, Thomas von Eyben wrote: > I am seeing a >10x as slow performance when trying to complete a > "send/receive" from an Outlook 2010 client to Dovecot via IMAP, but > only when the LAN is fully loaded with other traffic, EG file copying. > It seems the problem is when outlook is trying to identify folders > that have changed since last "send/receive" thus traversing the > hierachy. Not sure why it would only affect Outlook clients, but if your switches are managed, you might like to check if flow control is enabled and, if so, try disabling it. From tss at iki.fi Mon Apr 9 11:19:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 11:19:03 +0300 Subject: [Dovecot] Director simplification? Message-ID: <71D6ACD2-8FF6-4E74-8C27-BF54DDACA3E5@iki.fi> An idea I just had: Director basically works by assigning the backend IP address by: ip = vhosts[ md5(username) mod vhosts_count ].ip The rest of director is about what happens when vhosts[] or vhosts_count changes. What about instead doing this on IP address level? ip = ip_pool[ md5(username) mod ip_pool_size ] When a backend dies, you'll reassign the backend's IPs to other backends. Each backend should have many IPs. The main restriction here is that the IP pool cannot change without stopping the entire Dovecot. But if you initially allocate enough IPs, that shouldn't be a problem. And the advantage of this over the current director? To guarantee that one director can't break others, because they don't need to communicate with each others. The disadvantage of course is that it's a little less flexible and requires more planning ahead. The IP address reassignment would also need some distro-specific scripts. This could be implemented as an alternative director-lite or something. The doveadm director status-related commands could still work with it. From tss at iki.fi Mon Apr 9 13:19:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 13:19:50 +0300 Subject: [Dovecot] v2.1.4 released Message-ID: http://dovecot.org/releases/2.1/dovecot-2.1.4.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.4.tar.gz.sig + Added mail_temp_scan_interval setting and changed its default value from 8 hours to 1 week. + Added pop3-migration plugin for easily doing a transparent IMAP+POP3 migration to Dovecot: http://wiki2.dovecot.org/Migration/Dsync + doveadm user: Added -m parameter to show some of the mail settings. - Proxying SSL connections crashed in v2.1.[23] - fts-solr: Indexing mail bodies was broken. - director: Several changes to significantly improve error handling - doveadm import didn't import messages' flags - mail_full_filesystem_access=yes was broken - Make sure IMAP clients can't create directories when accessing nonexistent users' mailboxes via shared namespace. - Dovecot auth clients authenticating via TCP socket could have failed with bogus "PID already in use" errors. From tss at iki.fi Mon Apr 9 13:19:52 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 13:19:52 +0300 Subject: [Dovecot] v2.0.20 released Message-ID: <3972C017-7380-4936-8231-AF660DEC9106@iki.fi> http://dovecot.org/releases/2.0/dovecot-2.0.20.tar.gz http://dovecot.org/releases/2.0/dovecot-2.0.20.tar.gz.sig + doveadm user: Added -m parameter to show some of the mail settings. - doveadm import didn't import messages' flags - Make sure IMAP clients can't create directories when accessing nonexistent users' mailboxes via shared namespace. - Dovecot auth clients authenticating via TCP socket could have failed with bogus "PID already in use" errors. From andy at xecu.net Mon Apr 9 13:58:33 2012 From: andy at xecu.net (Andy Dills) Date: Mon, 9 Apr 2012 06:58:33 -0400 (EDT) Subject: [Dovecot] Director pop-login and imap-login processes exiting on signal 11 In-Reply-To: <70AB2E58-42C9-49B6-BE13-C553893E19EF@iki.fi> References: <20120407030811.I46246@shell.xecu.net> <70AB2E58-42C9-49B6-BE13-C553893E19EF@iki.fi> Message-ID: <20120409065728.K14159@shell.xecu.net> On Mon, 9 Apr 2012, Timo Sirainen wrote: > On 7.4.2012, at 10.13, Andy Dills wrote: > > > Apr 7 02:18:05 mail-out06 dovecot: pop3-login: Fatal: master: > > service(pop3-login): child 75029 killed with signal 11 (core not dumped - > > set service pop3-login { drop_priv_before_exec=yes }) > > v2.1.3 proxying was buggy with SSL connections. Probably crashes because > of that. I was supposed to release v2.1.4 already but.. Thanks Timo. I can confirm this is fixed in 2.1.4. Andy --- Andy Dills Xecunet, Inc. www.xecu.net 301-682-9972 --- From CMarcus at Media-Brokers.com Mon Apr 9 15:50:42 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 09 Apr 2012 08:50:42 -0400 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: References: <4F7D972A.8050406@Media-Brokers.com> Message-ID: <4F82DB22.2050201@Media-Brokers.com> On 2012-04-09 3:33 AM, Timo Sirainen wrote: > On 5.4.2012, at 15.59, Charles Marcus wrote: > >> Does anyone know if the use of LMTP (or even the dovecot LDA) still >> loses the x-original-to header that the postfix vda adds and that I >> rely heavily on (since I use a lot of aliases), and if it does, is >> there any solution to get the original recipient added back in >> before final delivery? > LMTP adds a new Delivered-To: header when there is > a single RCPT TO. You can force a single RCPT TO from Postfix side by > setting lmtp_destination_recipient_limit=1. LMTP doesn't > add/remove/change X-Original-To: header. Ok, thanks Timo... but... Are you saying that this 'Delivered-To:' header can somehow be leveraged to provide the same info as the x-original-to header? If not, since it was the postfix virtual delivery agent that added the x-original-to, and since using lmtp means I would not be using the postfix vda, is the appropriate place to add this header in dovecot's lmtp implementation (and if so, how hard would it be)? Or would this need to be done somehow on the postfix side (if so, I'll go ask on the postfix list)? Sorry for my ignorance - but as I said, I rely on this header (I use a ton of aliases, and without it I can't see the original (alias) recipient), so I need to determine if I'm going to be able to use lmtp or not (obviously, I would much prefer to do so)... Thanks again Timo... -- Best regards, Charles From tss at iki.fi Mon Apr 9 15:53:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 15:53:56 +0300 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: <4F82DB22.2050201@Media-Brokers.com> References: <4F7D972A.8050406@Media-Brokers.com> <4F82DB22.2050201@Media-Brokers.com> Message-ID: On 9.4.2012, at 15.50, Charles Marcus wrote: >> LMTP adds a new Delivered-To: header when there is >> a single RCPT TO. You can force a single RCPT TO from Postfix side by >> setting lmtp_destination_recipient_limit=1. LMTP doesn't >> add/remove/change X-Original-To: header. > > Ok, thanks Timo... but... > > Are you saying that this 'Delivered-To:' header can somehow be leveraged to provide the same info as the x-original-to header? I guess X-Original-To is the same address as what Postfix sees as the original RCPT TO address before alias expansion and such? In that case, see my today's mail in Postfix list.. From pw at wk-serv.de Mon Apr 9 16:25:41 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Mon, 09 Apr 2012 15:25:41 +0200 Subject: [Dovecot] Director In-Reply-To: <613701AE-34D1-4366-887E-72F576A01A1D@iki.fi> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F7EF67A.8060001@wk-serv.de> <613701AE-34D1-4366-887E-72F576A01A1D@iki.fi> Message-ID: <4F82E355.8000901@wk-serv.de> Timo Sirainen schrieb: > b) Host name that expands to all directors' IP addresses. If the first one is down, the LMTP client (hopefully! verify!) connects to the second one. Yes, Postfix' LMTP works fine with a hostname expanding to two IPs. From pw at wk-serv.de Mon Apr 9 16:35:01 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Mon, 09 Apr 2012 15:35:01 +0200 Subject: [Dovecot] Problem with quota warning Message-ID: <4F82E585.7030201@wk-serv.de> Hi again, my test environment is working now and finally I tried to implement my quota warning but it is not working. LDA says the mailbox is full even it is not (only 81% and that is why the quota warning is executed). Further I used :noenforcing in my script. Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: Namespace INBOX.: Using permissions from /var/mail/k-team.info/test/mdbox: mode=0700 gid=-1 Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: Quota root: name=User quota backend=dict args=:file://mdbox/dovecot-quota:noenforcing Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: Quota warning: bytes=0 (90%) messages=0 reverse=no command=quota-warning 90 raw mail user Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: Quota warning: bytes=0 (75%) messages=0 reverse=no command=quota-warning 75 raw mail user Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: dict quota: user=raw mail user, uri=file://mdbox/dovecot-quota:noenforcing, noenforcing=0 Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: none: root=, index=, control=, inbox=, alt= Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: Destination address: test at k-team.info (source: user at hostname) Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): msgid=unspecified: save failed to INBOX: Quota exceeded (mailbox for user is full) Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): msgid=unspecified: rejected: Quota exceeded (mailbox for user is full) Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): msgid=: Return-Path missing, rejection reason: Quota exceeded (mailbox for user is full) # 2.1.3: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.4 auth_debug = yes auth_debug_passwords = yes auth_verbose = yes listen = 172.17.99.96 mail_debug = yes mail_gid = vmail mail_home = /var/mail/%d/%n mail_location = mdbox:~/mdbox:INDEX=/var/indexes/%d/%n mail_plugins = quota mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { autocreate = INBOX.Sent autocreate2 = INBOX.Trash autocreate3 = INBOX.Drafts autocreate4 = INBOX.Spamverdacht autosubscribe = INBOX.Sent autosubscribe2 = INBOX.Trash autosubscribe3 = INBOX.Drafts autosubscribe4 = INBOX.Spamverdacht quota = dict:User quota::file:%h/mdbox/dovecot-quota quota_rule2 = INBOX.Trash:storage=+100M quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=75%% quota-warning 75 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_global_path = /usr/local/etc/dovecot/sieve/default.sieve } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { user = vmail } } service imap-login { service_count = 1 } service lmtp { inet_listener lmtp { address = 172.17.99.96 port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 } service quota-warning { executable = script /usr/local/etc/dovecot/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = required ssl_cert = References: <4F7D972A.8050406@Media-Brokers.com> <4F82DB22.2050201@Media-Brokers.com> Message-ID: <4F82E7CB.6010206@Media-Brokers.com> On 2012-04-09 8:53 AM, Timo Sirainen wrote: > I guess X-Original-To is the same address as what Postfix sees as the > original RCPT TO address before alias expansion and such? In that > case, see my today's mail in Postfix list. Yep... and hoping that you and Wietse can work out some way to support it... Thanks for participating in the discussion over there... :) -- Best regards, Charles From lists at luigirosa.com Mon Apr 9 16:44:46 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Mon, 09 Apr 2012 15:44:46 +0200 Subject: [Dovecot] mount Message-ID: <4F82E7CE.3070502@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I have a Dovecot installation on CentOS 5 where I sometimes mount external fs in /mnt Every Dovecot data is in local / file system, nothing is mounted elswhere After upgrading to 1.2.4 I rebooted the system for other reasons and at startup I got this on Dovecot log: master: Warning: /mnt is no longer mounted. If this is intentional, remove it with doveadm mount No /mnt entry in /etc/fstab and nothing pmounted under /mnt I THINK that the last time I used /mnt to mount something was few weeks ago to update VMware tools. Is there a way toi tell Dovecot to ignore /mnt ? Ciao, luigi - -- / +--[Luigi Rosa]-- \ $100 invested at 7% interest for 100 years will become $100,000, at which time it will be worth absolutely nothing. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk+C58gACgkQ3kWu7Tfl6ZRc0wCgl0Z4OtblYbfYwzvOp1/vUifV PqYAoIvfltvmq3cijvDbOEKV2Tai2rpu =hyrI -----END PGP SIGNATURE----- From =?UTF-8?Q?=D0=91=D1=80=D0=B0=D0=BD=D0=BA=D0=BE_=D0=9C=D0=B0=D1=98?= Mon Apr 9 16:51:49 2012 From: =?UTF-8?Q?=D0=91=D1=80=D0=B0=D0=BD=D0=BA=D0=BE_=D0=9C=D0=B0=D1=98?= (=?UTF-8?Q?=D0=91=D1=80=D0=B0=D0=BD=D0=BA=D0=BE_=D0=9C=D0=B0=D1=98?=) Date: Mon, 09 Apr 2012 15:51:49 +0200 Subject: [Dovecot] Username from rfc822Name subject alternative name Message-ID: Hello, I'm looking into adding support for extracting the username from client certificate's rfc822Name (from the subjectAltName extension). The question I have is what would be the best approach to do this? Current implementation has a kind of clean code since it just goes through the subject name, extracting the values with X509_NAME_get_text_by_NID (while NID is obtained with OBJ_txt2nid). If I were to add this, it's bound to make the code a little bit more complicated since SAN's can't be retrieved in the same way. So far in terms of options I have, I can see the following: 1. Create a distinct configuration option for the ssl_cert_username_field (i.e. specify something like "sanrfc822Name" to have Dovecot extract the username from the designated alternative name). 2. Make the current code fail-over to rfc822Name SAN if emailAddress is provided for ssl_cert_username (less invasion in code, but less flexibility as well). Any input/recommendation/directioning is welcome. I've wanted to actually first write a patch, and then submit it, but I think it might be better to check what would be preferable by Dovecot maintainers/devs. Best regards -- Branko Majic Jabber: branko at majic.rs Please use only Free formats when sending attachments to me. ?????? ????? ?????: branko at majic.rs ????? ??? ?? ??????? ?????? ????????? ? ????????? ?????????. From tss at iki.fi Mon Apr 9 16:57:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 16:57:48 +0300 Subject: [Dovecot] mount In-Reply-To: <4F82E7CE.3070502@luigirosa.com> References: <4F82E7CE.3070502@luigirosa.com> Message-ID: On 9.4.2012, at 16.44, Luigi Rosa wrote: > I have a Dovecot installation on CentOS 5 where I sometimes mount external fs > in /mnt > > Every Dovecot data is in local / file system, nothing is mounted elswhere .. > Is there a way toi tell Dovecot to ignore /mnt ? doveadm mount add /mnt ignore From lists at luigirosa.com Mon Apr 9 16:59:05 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Mon, 09 Apr 2012 15:59:05 +0200 Subject: [Dovecot] mount In-Reply-To: References: <4F82E7CE.3070502@luigirosa.com> Message-ID: <4F82EB29.8030709@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Timo Sirainen said the following on 09/04/12 15:57: >> Is there a way toi tell Dovecot to ignore /mnt ? > doveadm mount add /mnt ignore Thanks, next time I will RTFM first. Ciao, luigi - -- / +--[Luigi Rosa]-- \ [UNIX] n. A random quote, item of trivia, joke or maxim printed to the user's tty at login time or (less commonly) at logout time. Items from this jargon file have often been used as fortune cookies. --Jargon File -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk+C6ykACgkQ3kWu7Tfl6ZQI6QCgt4E3Imx1OeaB6SqjIjWDhjS0 xqUAoKizTRivIkvKkQE5SS7zwCtPlL9B =RCsz -----END PGP SIGNATURE----- From achekalin at lazurit.com Mon Apr 9 17:58:23 2012 From: achekalin at lazurit.com (Alexander Chekalin) Date: Mon, 09 Apr 2012 17:58:23 +0300 Subject: [Dovecot] Multiply mailboxes vs one huge Message-ID: <4F82F90F.8020902@lazurit.com> Hello, as I need to store a lot of messages on my IMAP server (order of 900K-1000K; this is an archive for some time, maybe a year or so), I see some "slowness" in dealing with such a huge amount. I mainly need to do searches like "get all messages from user1 at domain1.com to user2 at domain2.tld recieved between date1 and date2". So I really interested will it be wise to a) split all messages into several smaller mailboxes (per-month, or per-day, or create 2-level-structure like "month/day/") b) use dbox (vs currently used mbox) storage scheme (I'm afraid of mdbox as I still not sure I'll be able to parse it by scripts later "just in case") Dovecot is the latest one (2.1.3). No compression Dovecot-side, but it mails are in zfs volume with compression on. I ask this mainly due to my not fully understand how Dovecot indexes are working. I also test another approach: to use my own index somewhere outside Dovecot which will store reference between emails and UIDs, and dates and UIDs, so I'll simple query my index for things I need. But then, that's exactly what IMAP index can do, so I simple slow my search down, isn't it? The only reason I think about my own index is I won't use 'all header' as search scope, I need to deal only with From:, To:, Cc:, Bcc: (if any), Recieved (if nowehere else I see the from/to info), and date field(s) - I doubt IMAP will care for that for me. Yours, Alexander From tss at iki.fi Mon Apr 9 18:03:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 18:03:38 +0300 Subject: [Dovecot] Multiply mailboxes vs one huge In-Reply-To: <4F82F90F.8020902@lazurit.com> References: <4F82F90F.8020902@lazurit.com> Message-ID: On 9.4.2012, at 17.58, Alexander Chekalin wrote: > as I need to store a lot of messages on my IMAP server (order of 900K-1000K; this is an archive for some time, maybe a year or so), I see some "slowness" in dealing with such a huge amount. I mainly need to do searches like "get all messages from user1 at domain1.com to user2 at domain2.tld recieved between date1 and date2". So by "received between date" you mean the IMAP INTERNALDATE as opposed to Date: header? These kind of searches are looked up from the index/cache files, and the performance should be exactly the same with all of the mailbox formats. It would be useful to figure out what exactly is causing the slowness. Is the SEARCH command slow? Something else? Is the slowness about user CPU, system CPU or disk IO? From andre.rodier at gmail.com Mon Apr 9 19:26:50 2012 From: andre.rodier at gmail.com (Andre Rodier) Date: Mon, 9 Apr 2012 17:26:50 +0100 Subject: [Dovecot] per user sieve after filters Message-ID: <20120409162650.GA29690@london.sagso.home> Hello, Thanks for dovecot, as it's still the best mail server. I'd like to use per users sieve_after scripts. Can I put in my dovecot config file, something like that: sieve_after = %h/Mails/Sieve/After/ It would be very useful for me, as I'd like to add vacation script to be executed from this place. Kind regards, Andr?. From stephan at rename-it.nl Mon Apr 9 21:01:42 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 09 Apr 2012 20:01:42 +0200 Subject: [Dovecot] per user sieve after filters In-Reply-To: <20120409162650.GA29690@london.sagso.home> References: <20120409162650.GA29690@london.sagso.home> Message-ID: <4F832406.8060108@rename-it.nl> On 4/9/2012 6:26 PM, Andre Rodier wrote: > Hello, > > Thanks for dovecot, as it's still the best mail server. > > I'd like to use per users sieve_after scripts. > > Can I put in my dovecot config file, something like that: > > sieve_after = %h/Mails/Sieve/After/ > > It would be very useful for me, as I'd like to add vacation script to be > executed from this place. I must say I've never tested something like that, but it should work. Regards, Stephan. From centos.admin at gmail.com Mon Apr 9 22:15:02 2012 From: centos.admin at gmail.com (Emmanuel Noobadmin) Date: Tue, 10 Apr 2012 03:15:02 +0800 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F81D73B.9030901@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> Message-ID: On 4/9/12, Stan Hoeppner wrote: > So it seems you have two courses of action: > 1. Identify individual current choke points and add individual systems > and storage to eliminate those choke points. > > 2. Analyze your entire workflow and all systems, identifying all choke > points, then design a completely new well integrated storage > architecture that solves all current problems and addresses future needs I started to do this and realize I have a serious mess on hand that makes delving in other people's uncommented source code seem like a joy :D Management added to this by deciding if we're going to offload the email storage to a network storage, we might as well consolidate everything into that shared storage system so we don't have TBs of un-utilized space. So I might not even be able to use your tested XFS + concat solution since it may not be optimal for VM images and databases. As the requirements' changed, I'll stop asking here as it's no longer really relevant just for Dovecot purposes. > You are a perfect candidate for VMware ESX. The HA feature will do > exactly what you want. If one physical node in the cluster dies, HA > automatically restarts the dead VMs on other nodes, transparently. > Clients will will have to reestablish connections, but everything else > will pretty much be intact. Worse case scenario will possibly be a few > corrupted mailboxes that were being written when the hardware crashed. > A SAN is required for such a setup. Thanks for the suggestion, I will need to find some time to look into this although we've mostly been using KVM for virtualization so far. Although the "SAN" part will probably prevent this from being accepted due to cost. >> My lame excuse is that I'm just the web >> dev who got caught holding the server admin potato. > > Baptism by fire. Ouch. What doesn't kill you makes you stronger. ;) True, but I'd hate to be the customer who get to pick up the pieces when things explode due to unintended negligence by a dev trying to level up by multi-classing as an admin. > physical network interface. You can do some of these things with free > Linux hypervisors, but AFAIK the poor management interfaces for them > make the price of ESX seem like a bargain. Unfortunately, the usual kind of customers we have here, spending that kind of budget isn't justifiable. The only reason we're providing email services is because customers wanted freebies and they felt there was no reason why we can't give them emails on our servers, they are all "servers" after all. So I have to make do with OTS commodity parts and free software for the most parts. From achekalin at lazurit.com Mon Apr 9 22:39:40 2012 From: achekalin at lazurit.com (Alexander Chekalin) Date: Mon, 09 Apr 2012 22:39:40 +0300 Subject: [Dovecot] Multiply mailboxes vs one huge In-Reply-To: References: <4F82F90F.8020902@lazurit.com> Message-ID: <4F833AFC.60301@lazurit.com> Hello, Timo, I feel a bit unsure about "which 'date' I mean", since I always consider the only date from Date: header. But which value is used as INTERNALDATE then? As soon as I use (for now) maildir storage type, all the metadata are stored in messages. So I expect Dovecot somehow parse and use Date: field itself, or I'm wrong with it? And also what's about messages without Date header at all? But the Date isn't the worst thing. Look, to have my archive work I setup server-side filter which redirect all messages it processed also to my archive mailbox. This way, each message (after such a redirect) targeted to 'archive at mydomain', instead of its original destination email. The only place I can find out the original recipient is to parse 'Recieved' field(-s). As I think I understand that none of these headers (Date or Received) are to be used for SEARCH anyway, and this was the idea behind creating my own index. But wait, is there any way I can make Dovecot also index additional fields (yes, I talk about 'Received') - then it'll be the best solution! Thank you, Timo, for your work, yours, Alexander 09.04.2012 18:03, Timo Sirainen ???????: > On 9.4.2012, at 17.58, Alexander Chekalin wrote: > >> as I need to store a lot of messages on my IMAP server (order of 900K-1000K; this is an archive for some time, maybe a year or so), I see some "slowness" in dealing with such a huge amount. I mainly need to do searches like "get all messages from user1 at domain1.com to user2 at domain2.tld recieved between date1 and date2". > So by "received between date" you mean the IMAP INTERNALDATE as opposed to Date: header? These kind of searches are looked up from the index/cache files, and the performance should be exactly the same with all of the mailbox formats. It would be useful to figure out what exactly is causing the slowness. Is the SEARCH command slow? Something else? Is the slowness about user CPU, system CPU or disk IO? > From marc at perkel.com Tue Apr 10 04:28:32 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 09 Apr 2012 18:28:32 -0700 Subject: [Dovecot] v2.1.4 released - broken In-Reply-To: References: Message-ID: <4F838CC0.2000900@perkel.com> I'm seeing this immediately after upgrading from 2.1.3 Apr 09 18:22:43 imap(chris at powerpage.org): Error: user chris at powerpage.org: Initialization failed: Initializing mail storage from mail_location setting failed: Home directory not set for user. Can't expand ~/ for mail root dir in: /vhome/powerpage.org/home/chris:INDEX=/email/imap-cache/powerpage.org-chris mail_location = maildir:/vhome/%d/home/%n:INDEX=/email/imap-cache/%d-%n From a.kostyrev at serverc.ru Tue Apr 10 05:37:27 2012 From: a.kostyrev at serverc.ru (=?koi8-r?B?68/T1NnSxdcg4czFy9PBzsTSIOHMxcvTxcXXyd4=?=) Date: Tue, 10 Apr 2012 13:37:27 +1100 Subject: [Dovecot] Authentication mechanism and Password scheme Message-ID: <213B51F00051AE48A9F0E112880177178F794E@Delta.sc.local> Good day! I'm just trying to figure out that my understanding of subject is correct. So, if I want to store passwords in my database encrypted with SSHA512 scheme, my only choice for Authentication mechanism is plaintext? Thanks in advance! -- ? ?????????, ???????? ????????? ????????? ????????????? ??? "??????-?????" ???.: (423) 262-02-62 (???. 2037) ????: (423) 262-02-10 From stan at hardwarefreak.com Tue Apr 10 08:00:19 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 10 Apr 2012 00:00:19 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> Message-ID: <4F83BE63.9050603@hardwarefreak.com> On 4/9/2012 2:15 PM, Emmanuel Noobadmin wrote: > Unfortunately, the usual kind of customers we have here, spending that > kind of budget isn't justifiable. The only reason we're providing > email services is because customers wanted freebies and they felt > there was no reason why we can't give them emails on our servers, they > are all "servers" after all. > > So I have to make do with OTS commodity parts and free software for > the most parts. OTS meaning you build your own systems from components? Too few in the business realm do so today. :( It sounds like budget overrides redundancy then. You can do an NFS cluster with SAN and GFS2, or two servers with their own storage and DRBD mirroring. Here's how to do the latter: http://www.howtoforge.com/high_availability_nfs_drbd_heartbeat The total cost is about the same for each solution as an iSCSI SAN array of drive count X is about the same cost as two JBOD disk arrays of count X*2. Redundancy in this case is expensive no matter the method. Given how infrequent host failures are, and the fact your storage is redundant, it may make more sense to simply keep spare components on hand and swap what fails--PSU, mobo, etc. Interestingly, I designed a COTS server back in January to handle at least 5k concurrent IMAP users, using best of breed components. If you or someone there has the necessary hardware skills, you could assemble this system and simply use it for NFS instead of Dovecot. The parts list: secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=17069985 In case the link doesn't work, the core components are: SuperMicro H8SGL G34 mobo w/dual Intel GbE, 2GHz 8-core Opteron 32GB Kingston REG ECC DDR3, LSI 9280-4i4e, Intel 24 port SAS expander 20 x 1TB WD RE4 Enterprise 7.2K SATA2 drives NORCO RPC-4220 4U 20 Hot-Swap Bays, SuperMicro 865W PSU All other required parts are in the Wish List. I've not written assembly instructions. I figure anyone who would build this knows what s/he is doing. Price today: $5,376.62 Configuring all 20 drives as a RAID10 LUN in the MegaRAID HBA would give you a 10TB net Linux device and 10 stripe spindles of IOPS and bandwidth. Using RAID6 would yield 18TB net and 18 spindles of read throughput, however parallel write throughput will be at least 3-6x slower than RAID10, which is why nobody uses RAID6 for transactional workloads. If you need more transactional throughput you could use 20 WD6000HLHX 600GB 10K RPM WD Raptor drives. You'll get 40% more throughput and 6TB net space with RAID10. They'll cost you $1200 more, or $6,576.62 total. Well worth the $1200 for 40% more throughput, if 6TB is enough. Both of the drives I've mentioned here are enterprise class drives, feature TLER, and are on the LSI MegaRAID SAS hardware compatibility list. The price of the 600GB Raptor has come down considerably since I designed this system, or I'd have used them instead. Anyway, lots of option out there. But $6,500 is pretty damn cheap for a quality box with 32GB RAM, enterprise RAID card, and 20x10K RPM 600GB drives. The MegaRAID 9280-4i4e has an external SFF8088 port For an additional $6,410 you could add an external Norco SAS expander JBOD chassis and 24 more 600GB 10K RPM Raptors, for 13.2TB of total net RAID10 space, and 22 10k spindles of IOPS performance from 44 total drives. That's $13K for a 5K random IOPS, 13TB, 44 drive NFS RAID COTS server solution, $1000/TB, $2.60/IOPS. Significantly cheaper than an HP, Dell, IBM solution of similar specs, each of which will set you back at least 20 large. Note the chassis I've spec'd have single PSUs, not the dual or triple redundant supplies you'll see on branded hardware. With a relatively stable climate controlled environment and a good UPS with filtering, quality single supplies are fine. In fact, in the 4U form factor single supplies are usually more reliable due to superior IC packaging and airflow through the heatsinks, not to mention much quieter. -- Stan From lists at wiesinger.com Tue Apr 10 08:46:31 2012 From: lists at wiesinger.com (Gerhard Wiesinger) Date: Tue, 10 Apr 2012 07:46:31 +0200 Subject: [Dovecot] v2.1.4 released - broken In-Reply-To: <4F838CC0.2000900@perkel.com> References: <4F838CC0.2000900@perkel.com> Message-ID: <4F83C937.1080707@wiesinger.com> On 10.04.2012 03:28, Marc Perkel wrote: > > I'm seeing this immediately after upgrading from 2.1.3 > > Apr 09 18:22:43 imap(chris at powerpage.org): Error: user > chris at powerpage.org: Initialization failed: Initializing mail storage > from mail_location setting failed: Home directory not set for user. > Can't expand ~/ for mail root dir in: > /vhome/powerpage.org/home/chris:INDEX=/email/imap-cache/powerpage.org-chris > > mail_location = maildir:/vhome/%d/home/%n:INDEX=/email/imap-cache/%d-%n > I'm guessing this occurs from the following bugfix: http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e Reverting this patch helps? What is your config (doveconf -n)? Ciao, Gerhard From centos.admin at gmail.com Tue Apr 10 09:09:18 2012 From: centos.admin at gmail.com (Emmanuel Noobadmin) Date: Tue, 10 Apr 2012 14:09:18 +0800 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F83BE63.9050603@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> Message-ID: On 4/10/12, Stan Hoeppner wrote: >> So I have to make do with OTS commodity parts and free software for >> the most parts. > > OTS meaning you build your own systems from components? Too few in the > business realm do so today. :( For the inhouse stuff and budget customers yes, in fact both the email servers are on seconded hardware that started life as something else. I spec HP servers for our app servers to customers who are willing to pay for their own colocated or onsite servers but still there are customers who balk at the cost and so go OTS or virtualized. > SuperMicro H8SGL G34 mobo w/dual Intel GbE, 2GHz 8-core Opteron > 32GB Kingston REG ECC DDR3, LSI 9280-4i4e, Intel 24 port SAS expander > 20 x 1TB WD RE4 Enterprise 7.2K SATA2 drives > NORCO RPC-4220 4U 20 Hot-Swap Bays, SuperMicro 865W PSU > All other required parts are in the Wish List. I've not written > assembly instructions. I figure anyone who would build this knows what > s/he is doing. > > Price today: $5,376.62 This price looks like something I might be able to push through although I'll probably have to go SATA instead of SAS due to cost of keeping spares. > Configuring all 20 drives as a RAID10 LUN in the MegaRAID HBA would give > you a 10TB net Linux device and 10 stripe spindles of IOPS and > bandwidth. Using RAID6 would yield 18TB net and 18 spindles of read > throughput, however parallel write throughput will be at least 3-6x > slower than RAID10, which is why nobody uses RAID6 for transactional > workloads. Not likely to go with RAID 5 or 6 due to concerns about the uncorrectable read errors risks on rebuild with large arrays. Is the MegaRAID being used as the actual RAID controller or just as a HBA? I have been avoiding hardware RAID because of a really bad experience with RAID 5 on an obsolete controller that eventually died without replacement and couldn't be recovered. Since then, it's always been RAID 1 and, after I discovered mdraid, using them as purely HBA with mdraid for the flexibility of being able to just pull the drives into a new system if necessary without having to worry about the controller. > Both of the drives I've mentioned here are enterprise class drives, > feature TLER, and are on the LSI MegaRAID SAS hardware compatibility > list. The price of the 600GB Raptor has come down considerably since I > designed this system, or I'd have used them instead. > > Anyway, lots of option out there. But $6,500 is pretty damn cheap for a > quality box with 32GB RAM, enterprise RAID card, and 20x10K RPM 600GB > drives. > > The MegaRAID 9280-4i4e has an external SFF8088 port For an additional > $6,410 you could add an external Norco SAS expander JBOD chassis and 24 > more 600GB 10K RPM Raptors, for 13.2TB of total net RAID10 space, and 22 > 10k spindles of IOPS performance from 44 total drives. That's $13K for > a 5K random IOPS, 13TB, 44 drive NFS RAID COTS server solution, > $1000/TB, $2.60/IOPS. Significantly cheaper than an HP, Dell, IBM > solution of similar specs, each of which will set you back at least 20 > large. Would this setup work well too for serving up VM images? I've been trying to find a solution for the virtualized app servers images as well but the distributed FSes currently are all bad with random reads/writes it seems. XFS seem to be good with large files like db and vm images with random internal write/read so given my time constraints, it would be nice to have a single configuration that works generally well for all the needs I have to oversee. > Note the chassis I've spec'd have single PSUs, not the dual or triple > redundant supplies you'll see on branded hardware. With a relatively > stable climate controlled environment and a good UPS with filtering, > quality single supplies are fine. In fact, in the 4U form factor single > supplies are usually more reliable due to superior IC packaging and > airflow through the heatsinks, not to mention much quieter. Same reason I do my best to avoid 1U servers, the space/heat issues worries me. Yes, I'm guilty of worrying too much but that had saved me on several occasions. From tss at iki.fi Tue Apr 10 10:10:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 10:10:28 +0300 Subject: [Dovecot] v2.1.4 released - broken In-Reply-To: <4F838CC0.2000900@perkel.com> References: <4F838CC0.2000900@perkel.com> Message-ID: On 10.4.2012, at 4.28, Marc Perkel wrote: > I'm seeing this immediately after upgrading from 2.1.3 > > Apr 09 18:22:43 imap(chris at powerpage.org): Error: user chris at powerpage.org: Initialization failed: Initializing mail storage from mail_location setting failed: Home directory not set for user. Can't expand ~/ for mail root dir in: /vhome/powerpage.org/home/chris:INDEX=/email/imap-cache/powerpage.org-chris > > mail_location = maildir:/vhome/%d/home/%n:INDEX=/email/imap-cache/%d-%n Now would be a good time to add home directories for users. :) Or apply: http://hg.dovecot.org/dovecot-2.1/rev/20e1aa322b1e From tss at iki.fi Tue Apr 10 10:11:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 10:11:05 +0300 Subject: [Dovecot] Authentication mechanism and Password scheme In-Reply-To: <213B51F00051AE48A9F0E112880177178F794E@Delta.sc.local> References: <213B51F00051AE48A9F0E112880177178F794E@Delta.sc.local> Message-ID: On 10.4.2012, at 5.37, ???????? ????????? ?????????? wrote: > Good day! > I'm just trying to figure out that my understanding of subject is correct. > > So, if I want to store passwords in my database encrypted with SSHA512 scheme, > my only choice for Authentication mechanism is plaintext? Yeah, that's correct. From tss at iki.fi Tue Apr 10 10:16:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 10:16:47 +0300 Subject: [Dovecot] Multiply mailboxes vs one huge In-Reply-To: <4F833AFC.60301@lazurit.com> References: <4F82F90F.8020902@lazurit.com> <4F833AFC.60301@lazurit.com> Message-ID: On 9.4.2012, at 22.39, Alexander Chekalin wrote: > Hello, Timo, > > I feel a bit unsure about "which 'date' I mean", since I always consider the only date from Date: header. But which value is used as INTERNALDATE then? As soon as I use (for now) maildir storage type, all the metadata are stored in messages. So I expect Dovecot somehow parse and use Date: field itself, or I'm wrong with it? The INTERNALDATE means the same as "received date", while the Date: header is the "sent date". With mbox format the received date is stored in the separating From-lines. IMAP supports searching and sorting messages by either INTERNALDATE or Date: header > And also what's about messages without Date header at all? The searching just doesn't match those messages then. Sorting falls back to using received date. > But the Date isn't the worst thing. Look, to have my archive work I setup server-side filter which redirect all messages it processed also to my archive mailbox. This way, each message (after such a redirect) targeted to 'archive at mydomain', instead of its original destination email. The only place I can find out the original recipient is to parse 'Recieved' field(-s). > > As I think I understand that none of these headers (Date or Received) are to be used for SEARCH anyway, and this was the idea behind creating my own index. But wait, is there any way I can make Dovecot also index additional fields (yes, I talk about 'Received') - then it'll be the best solution! If you do a SEARCH HEADER Received, then Dovecot adds the Received headers to dovecot.index.cache file and the subsequent searches should be quite fast, although the Received headers increase the cache file's size quite a lot. Also alternatively you can enable full text search indexes (Lucene or Solr) and the search is then done from them. From adrian.minta at gmail.com Tue Apr 10 13:22:18 2012 From: adrian.minta at gmail.com (Adrian Minta) Date: Tue, 10 Apr 2012 13:22:18 +0300 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F83BE63.9050603@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> Message-ID: <4F8409DA.1090202@gmail.com> On 04/10/12 08:00, Stan Hoeppner wrote: > Interestingly, I designed a COTS server back in January to handle at > least 5k concurrent IMAP users, using best of breed components. If you > or someone there has the necessary hardware skills, you could assemble > this system and simply use it for NFS instead of Dovecot. The parts > list: > secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=17069985 Don't forget the Battery Backup Unit for RAID card !!! From pw at wk-serv.de Tue Apr 10 13:44:08 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 10 Apr 2012 12:44:08 +0200 Subject: [Dovecot] Problem with quota warning In-Reply-To: <4F82E585.7030201@wk-serv.de> References: <4F82E585.7030201@wk-serv.de> Message-ID: <4F840EF8.7080109@wk-serv.de> Hi, I switched from dovecot-lda to lmtp for my quota warnings but there is something wrong with the parameters: /usr/local/libexec/dovecot/lmtp -d $USER -o "plugin/quota=dict:User quota::file:%h/mdbox/dovecot-quota:noenforcing" /usr/local/libexec/dovecot/lmtp: invalid option -- 'd' /usr/local/libexec/dovecot/lmtp -r $USER -o "plugin/quota=dict:User quota::file:%h/mdbox/dovecot-quota:noenforcing" /usr/local/libexec/dovecot/lmtp: invalid option -- 'r' Both parameters are listed on http://wiki2.dovecot.org/LMTP. So is this my fault or wrong information on the page? Patrick From tss at iki.fi Tue Apr 10 15:31:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 15:31:13 +0300 Subject: [Dovecot] Problem with quota warning In-Reply-To: <4F840EF8.7080109@wk-serv.de> References: <4F82E585.7030201@wk-serv.de> <4F840EF8.7080109@wk-serv.de> Message-ID: <3F52251E-6411-448E-B397-207EFD176478@iki.fi> On 10.4.2012, at 13.44, Patrick Westenberg wrote: > I switched from dovecot-lda to lmtp for my quota warnings but there is something wrong with the parameters: > > /usr/local/libexec/dovecot/lmtp -d $USER -o "plugin/quota=dict:User quota::file:%h/mdbox/dovecot-quota:noenforcing" This is not how to use LMTP. You need to tell your MTA to connect to it via UNIX socket or via TCP. You don't execute the LMTP. > Both parameters are listed on http://wiki2.dovecot.org/LMTP. > So is this my fault or wrong information on the page? It says "Compared to dovecot-lda parameters", so it's describing where dovecot-lda parameters are taken with LMTP. I guess it could be said in some clearer way, but I don't really know how. From pw at wk-serv.de Tue Apr 10 15:54:45 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 10 Apr 2012 14:54:45 +0200 Subject: [Dovecot] Problem with quota warning In-Reply-To: <3F52251E-6411-448E-B397-207EFD176478@iki.fi> References: <4F82E585.7030201@wk-serv.de> <4F840EF8.7080109@wk-serv.de> <3F52251E-6411-448E-B397-207EFD176478@iki.fi> Message-ID: <4F842D95.3070005@wk-serv.de> Timo Sirainen schrieb: > On 10.4.2012, at 13.44, Patrick Westenberg wrote: > >> I switched from dovecot-lda to lmtp for my quota warnings but there is something wrong with the parameters: >> >> /usr/local/libexec/dovecot/lmtp -d $USER -o "plugin/quota=dict:User quota::file:%h/mdbox/dovecot-quota:noenforcing" > > This is not how to use LMTP. You need to tell your MTA to connect to it via UNIX socket or via TCP. You don't execute the LMTP. There is no MTA installed on this system. This is the backend server. dovecot-lda doesn't work too, look at my first post with this subject. From tss at iki.fi Tue Apr 10 15:58:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 15:58:16 +0300 Subject: [Dovecot] Problem with quota warning In-Reply-To: <4F82E585.7030201@wk-serv.de> References: <4F82E585.7030201@wk-serv.de> Message-ID: <28D02445-BBDC-41F5-A7AA-2AD9C5E5093F@iki.fi> On 9.4.2012, at 16.35, Patrick Westenberg wrote: > cat << EOF | /usr/local/libexec/dovecot/dovecot-lda -d $USER -o "plugin/quota=dict:User quota::file:%h/mdbox/dovecot-quota:noenforcing" The :noenforcing isn't in the right location. dict:User quota::noenforcing:file:%h/... From pw at wk-serv.de Tue Apr 10 16:26:11 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 10 Apr 2012 15:26:11 +0200 Subject: [Dovecot] Problem with quota warning In-Reply-To: <28D02445-BBDC-41F5-A7AA-2AD9C5E5093F@iki.fi> References: <4F82E585.7030201@wk-serv.de> <28D02445-BBDC-41F5-A7AA-2AD9C5E5093F@iki.fi> Message-ID: <4F8434F3.3080708@wk-serv.de> Timo Sirainen schrieb: > The :noenforcing isn't in the right location. > > dict:User quota::noenforcing:file:%h/... Aww.. thank you! From luc at bigdata.nb.ca Tue Apr 10 16:45:13 2012 From: luc at bigdata.nb.ca (Luc Belliveau) Date: Tue, 10 Apr 2012 10:45:13 -0300 Subject: [Dovecot] old messages being delivered to blackberry Message-ID: <4F843969.6040806@bigdata.nb.ca> Several users are reporting old messages (sometime years old) are being delivered as new on their blackberry devices... I've confirmed that this happens to all blackberry users... I believe the problem is related to this error: Maildir: Expunged message reappeared, giving a new UID In fact, this is the only mail related error message on the server... the mail is stored on a glusterfs share, and I'm using director to prevent simultaneous access... although I wasn't able to get LMTP to work with Director... to simplify things, I've temporarily only enabled 1 dovecot server anyway, to get everything else working first... I'll fiddle with lmtp later... so right now only 1 server is accessing the mailboxes. Delivery is also done from that one server, from postfix. Possibly caused by GlusterFS, the indexes were still getting corrupted, even with only 1 server... so I've moved the indexes to a local directory on each server. I'm no longer getting index related errors... but the expunged error is still occuring... and the error seems to happen a few moments before the BB gets a bunch of messages... so it's very likely related. I would like to continue using GlusterFS if possible; I haven't found any indication that it is outright unsupported, as long as I'm using Director... I am using dovecot 2.0.16. Any help would be appreciated! Here is my configuration: # 2.0.16: /etc/dovecot/dovecot.conf # OS: Linux 3.1.9-1.4-desktop i686 openSUSE 12.1 (i586) auth_mechanisms = plain login disable_plaintext_auth = no log_timestamp = "%Y-%m-%d %H:%M:%S " mail_fsync = always mail_location = maildir:~/Maildir:INDEX=/var/indexes/%u mail_nfs_storage = yes mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve = /var/vmail/%d/%n/.sieve } postmaster_address = postmaster at bigdata.nb.ca service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } user = root } service lmtp { inet_listener lmtp { port = 24 } unix_listener /var/spool/postfix/private/dovecot-lmtp { mode = 0660 user = postfix } } ssl_ca = References: <4F843969.6040806@bigdata.nb.ca> Message-ID: <5CDED2FA-3654-4378-A2D3-8A0BFAF7B5CC@iki.fi> On 10.4.2012, at 16.45, Luc Belliveau wrote: > Several users are reporting old messages (sometime years old) are being delivered as new on their blackberry devices... I've confirmed that this happens to all blackberry users... I believe the problem is related to this error: > > Maildir: Expunged message reappeared, giving a new UID This means: 1. readdir() returns maildir file X 2. Later readdir() doesn't return file X, so Dovecot thinks the mail is expunged and also tells the client the mail is expunged 3. Later readdir() once again returns file X, so Dovecot can't really do anything except treat it as a new mail. > Possibly caused by GlusterFS, the indexes were still getting corrupted, even with only 1 server... so I've moved the indexes to a local directory on each server. I'm no longer getting index related errors... but the expunged error is still occuring... and the error seems to happen a few moments before the BB gets a bunch of messages... so it's very likely related. > > I would like to continue using GlusterFS if possible; I haven't found any indication that it is outright unsupported, as long as I'm using Director... If glusterfs can't do readdir() reliably, there's not a whole lot you can do.. Except maybe switch to another mailbox format. But if you're getting index corruption also with 1 Dovecot server, it doesn't look very promising. From apm at one.com Tue Apr 10 18:28:29 2012 From: apm at one.com (Peter Mogensen) Date: Tue, 10 Apr 2012 17:28:29 +0200 Subject: [Dovecot] 2.0/2.1 - different behavior for LIST-EXTENDED Message-ID: <4F84519D.9070104@one.com> Hi Timo, We are sitting here wondering if this difference in behaviour between dovecot 2.0.17 and 2.1.3 is intended. When you create a folder, subscribe to it and rename it (without changing the subscription) these are the behaviours: For 2.0.17: . list (SUBSCRIBED) "" "*" RETURN (STATUS (MESSAGES)) * LIST (\Subscribed \NonExistent) "." "INBOX.test" For 2.1.3: . list (SUBSCRIBED) "" "*" RETURN (STATUS (MESSAGES)) * LIST (\Subscribed) "." "INBOX.test" * NO Mailbox doesn't exist: test If you don't use rfc5819 the folder will just get silently ignored by dovecot 2.1.x, but if you actually try to get the number of messages you'll get the error. It seems to me from reading rfc5258 that the 2.0.x behaviour is the correct ?? /Peter From s.locatelli at hexanet.fr Tue Apr 10 18:29:45 2012 From: s.locatelli at hexanet.fr (=?ISO-8859-1?Q?St=E9phane_Locatelli?=) Date: Tue, 10 Apr 2012 17:29:45 +0200 Subject: [Dovecot] Per IP virtual domain Message-ID: Hello, My dovecot server (2.0.18 on FreeBSD 9) has multiple interface. I would like to configure the authentication (pop and imap) to append different domain according to the local IP address. I've tested this: local 10.0.0.1 { auth_default_realm = domain1.com } local 10.0.0.2 { auth_default_realm = domain2.com } But it doesn't work. I get this error message: doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf line 58: Auth settings not supported inside local/remote blocks: auth_default_realm -- Stephane Locatelli From tss at iki.fi Tue Apr 10 19:04:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 19:04:59 +0300 Subject: [Dovecot] 2.0/2.1 - different behavior for LIST-EXTENDED In-Reply-To: <4F84519D.9070104@one.com> References: <4F84519D.9070104@one.com> Message-ID: <73DC1DBA-930F-41F3-9220-68C2D53339C9@iki.fi> On 10.4.2012, at 18.28, Peter Mogensen wrote: > For 2.0.17: > . list (SUBSCRIBED) "" "*" RETURN (STATUS (MESSAGES)) > * LIST (\Subscribed \NonExistent) "." "INBOX.test" > > For 2.1.3: > . list (SUBSCRIBED) "" "*" RETURN (STATUS (MESSAGES)) > * LIST (\Subscribed) "." "INBOX.test" > * NO Mailbox doesn't exist: test This was fixed in v2.1.4. From tss at iki.fi Tue Apr 10 19:06:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 19:06:38 +0300 Subject: [Dovecot] Per IP virtual domain In-Reply-To: References: Message-ID: <39EBDB8E-C0E2-43D5-9548-6CDF274D51E3@iki.fi> On 10.4.2012, at 18.29, St?phane Locatelli wrote: > My dovecot server (2.0.18 on FreeBSD 9) has multiple interface. I would > like to configure the authentication (pop and imap) to append different > domain according to the local IP address. > > I've tested this: > > local 10.0.0.1 { > auth_default_realm = domain1.com > } > > local 10.0.0.2 { > auth_default_realm = domain2.com > } > > But it doesn't work. I get this error message: > > doveconf: Fatal: Error in configuration file > /usr/local/etc/dovecot/dovecot.conf line 58: Auth settings not supported > inside local/remote blocks: auth_default_realm As it says, currently they aren't supported. It would need some extra code. Depending on your passdb you may be able to do this in your passdb configuration. From tss at iki.fi Tue Apr 10 19:10:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 19:10:04 +0300 Subject: [Dovecot] Username from rfc822Name subject alternative name In-Reply-To: References: Message-ID: <578B8B92-3EC4-4137-B4D0-3B68A767CAB5@iki.fi> On 9.4.2012, at 16.51, ?????? ????? wrote: > I'm looking into adding support for extracting the username from client certificate's rfc822Name (from the subjectAltName extension). > > The question I have is what would be the best approach to do this? Current implementation has a kind of clean code since it just goes through the subject name, extracting the values with X509_NAME_get_text_by_NID (while NID is obtained with OBJ_txt2nid). If I were to add this, it's bound to make the code a little bit more complicated since SAN's can't be retrieved in the same way. > > So far in terms of options I have, I can see the following: > > 1. Create a distinct configuration option for the ssl_cert_username_field (i.e. specify something like "sanrfc822Name" to have Dovecot extract the username from the designated alternative name). I'm not sure if this is a good idea, but what about: ssl_cert_username_field = subjectAltName:rfc822Name > 2. Make the current code fail-over to rfc822Name SAN if emailAddress is provided for ssl_cert_username (less invasion in code, but less flexibility as well). Automatic failover seems dangerous. From s.locatelli at hexanet.fr Tue Apr 10 19:12:41 2012 From: s.locatelli at hexanet.fr (=?ISO-8859-1?Q?St=E9phane_Locatelli?=) Date: Tue, 10 Apr 2012 18:12:41 +0200 Subject: [Dovecot] Per IP virtual domain In-Reply-To: <39EBDB8E-C0E2-43D5-9548-6CDF274D51E3@iki.fi> References: <39EBDB8E-C0E2-43D5-9548-6CDF274D51E3@iki.fi> Message-ID: 2012/4/10 Timo Sirainen > > As it says, currently they aren't supported. It would need some extra > code. Depending on your passdb you may be able to do this in your passdb > configuration. Currently I'm using dovecot 1.x on my production server with mysql passdb. I have ugly SQL queries to do this and my idea was to upgrade to dovecot 2.x to avoid this. password_query = SELECT mbox AS user , passwd as password FROM mbox WHERE mbox=IF('%s' = 'smtp', '%u', IF('%u' like '%%@%%', '%u', concat('%u', case '%l' WHEN '10.0.0.1' THEN '@domain1.com' WHEN '10.0.0.2' THEN '@domain2.com' WHEN '10.0.0.3' THEN '@domain3.com' END ) ) ) AND passwd IS NOT NULL AND (status='active' OR status ='inactive'); -- Stephane Locatelli From lists at wildgooses.com Tue Apr 10 19:35:09 2012 From: lists at wildgooses.com (Ed W) Date: Tue, 10 Apr 2012 17:35:09 +0100 Subject: [Dovecot] Authentication mechanism and Password scheme In-Reply-To: References: <213B51F00051AE48A9F0E112880177178F794E@Delta.sc.local> Message-ID: <4F84613D.8040508@wildgooses.com> On 10/04/2012 08:11, Timo Sirainen wrote: > On 10.4.2012, at 5.37, ???????? ????????? ?????????? wrote: > >> Good day! >> I'm just trying to figure out that my understanding of subject is correct. >> >> So, if I want to store passwords in my database encrypted with SSHA512 scheme, >> my only choice for Authentication mechanism is plaintext? > Yeah, that's correct. > Does dovecot 2.0 also support SCRAM-SHA? I only mention because it's come up on my radar recently and as I understand it, it solves the issue of either having - plain text db of passwords, encrypted login - encrypted db of passwords, plaintext login With SCRAM you have both sides "encrypted". (Clearly it's also desirable that the hash algorithm is well chosen to be resistant to bruteforce, so some might argue that bcrypt/scrypt is even more desirable since there is not yet a GPU implementation - However, at least SHA is a decent stab at things) Can you confirm my understanding is correct? Next question is whether any current mail client supports SCRAM..? Regards Ed W From gwurster at rim.com Tue Apr 10 20:17:23 2012 From: gwurster at rim.com (Glenn Wurster) Date: Tue, 10 Apr 2012 17:17:23 +0000 Subject: [Dovecot] Bug in HIGHESTMODSEQ when MODSEQ tracking is not enabled for version 2.0.18. Message-ID: <48A61B0976102F488C64EF8A8F1E929D074BBA@XMB118CNC.rim.net> Greetings, I'm new to the list. I have encountered what I believe to be a bug in the return of HIGHESTMODSEQ when talking to a IMAP client. HIGHESTMODSEQ does not appear to be incrementing when a new message arrives. The exact steps I took are: 1) Create a new mail account 2) Send a mail to the account, check that you can receive the email. Send a second mail to the account. 3) perform a SELECT on the mailbox from a new session. PMLK4 SELECT INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 2 EXISTS * 1 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1334074920] UIDs valid * OK [UIDNEXT 3] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest PMLK4 OK [READ-WRITE] Select completed. 4) Retrieve the mail. Then perform another SELECT on the mailbox from yet another session. LLEP4 SELECT INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 2 EXISTS * 0 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1334074920] UIDs valid * OK [UIDNEXT 3] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest LLEP4 OK [READ-WRITE] Select completed. 5) Send another mail to the account, and check SELECT again. FKDA4 SELECT INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 3 EXISTS * 1 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1334074920] UIDs valid * OK [UIDNEXT 4] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest FKDA4 OK [READ-WRITE] Select completed. It appears as if HIGHESTMODSEQ is not being updated. I can get HIGHESTMODSEQ to start updating correctly if I send a "UID FETCH 1 MODSEQ" or similar command, which appears to enable MODSEQ tracking at the server (according to the comment around line 173 in file src/lib-index/mail-index-modseq.c), but until that command is sent, MODSEQ tracking is not enabled and hence HIGHESTMODSEQ is always going to return 1. According to RFC4551, the combination of HIGHESTMODSEQ and UIDVALIDITY should be sufficient to determine if the metadata associated with the mailbox has changed, but in this case looking at only those two parameters does not yield sufficient information about changes in the mailbox. The mail client I'm using relies on the combination of HIGHESTMODSEQ and UIDVALIDITY to determine if there are changes in the mailbox, and hence does not see new mail come in. It seems that Dovecot should not be returning HIGHESTMODSEQ in response to a command if MODSEQ tracking is not enabled, but I could be wrong. I've attached my configuration (it's Dovecot 2.0.18 running on Debian Stable). Let me know if you need more information. Glenn. --------------------------------------------------------------------- This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful. -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot.conf Type: application/octet-stream Size: 452 bytes Desc: dovecot.conf URL: From tss at iki.fi Tue Apr 10 21:21:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 21:21:25 +0300 Subject: [Dovecot] Authentication mechanism and Password scheme In-Reply-To: <4F84613D.8040508@wildgooses.com> References: <213B51F00051AE48A9F0E112880177178F794E@Delta.sc.local> <4F84613D.8040508@wildgooses.com> Message-ID: <8AB6491F-9277-45BF-8FF0-2C90D7941E8A@iki.fi> On 10.4.2012, at 19.35, Ed W wrote: > Does dovecot 2.0 also support SCRAM-SHA? v2.1 does. > I only mention because it's come up on my radar recently and as I understand it, it solves the issue of either having > - plain text db of passwords, encrypted login > - encrypted db of passwords, plaintext login > > With SCRAM you have both sides "encrypted". The same way as with DIGEST-MD5 and several others. Each mechanism requires that the server-side password is saved using a hash specific to that auth mechanism, none of them support generic MD5/SHA/etc hashes or other mechanisms' hashes. Looks like Dovecot's current SCRAM-SHA1 supports only plaintext passwords, but it would be possible to add SCRAM-SHA1 password scheme similar to others. From tss at iki.fi Tue Apr 10 21:42:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 21:42:28 +0300 Subject: [Dovecot] Bug in HIGHESTMODSEQ when MODSEQ tracking is not enabled for version 2.0.18. In-Reply-To: <48A61B0976102F488C64EF8A8F1E929D074BBA@XMB118CNC.rim.net> References: <48A61B0976102F488C64EF8A8F1E929D074BBA@XMB118CNC.rim.net> Message-ID: <11BD08AA-2E29-48DF-973D-4D5B4CCE2FF7@iki.fi> On 10.4.2012, at 20.17, Glenn Wurster wrote: > It appears as if HIGHESTMODSEQ is not being updated. I can get HIGHESTMODSEQ to start updating correctly if I send a "UID FETCH 1 MODSEQ" or similar command, which appears to enable MODSEQ tracking at the server (according to the comment around line 173 in file src/lib-index/mail-index-modseq.c), but until that command is sent, MODSEQ tracking is not enabled and hence HIGHESTMODSEQ is always going to return 1. Yes, modseqs aren't tracked in a mailbox until client expresses an interest for them. It would be a waste of disk space to save them since 99% of users don't need them. > According to RFC4551, the combination of HIGHESTMODSEQ and UIDVALIDITY should be sufficient to determine if the metadata associated with the mailbox has changed, but in this case looking at only those two parameters does not yield sufficient information about changes in the mailbox. The mail client I'm using relies on the combination of HIGHESTMODSEQ and UIDVALIDITY to determine if there are changes in the mailbox, and hence does not see new mail come in. Yeah, it does seem that the RFC says that.. > It seems that Dovecot should not be returning HIGHESTMODSEQ in response to a command if MODSEQ tracking is not enabled, but I could be wrong. I've attached my configuration (it's Dovecot 2.0.18 running on Debian Stable). RFC 4551 says that HIGHESTMODSEQ or NOMODSEQ MUST be returned. Hmm. Perhaps: 1) If the session is known to have modseqs enabled, immediately enable modseqs for newly created mailboxes 2) If a mailbox doesn't have modseqs enabled, return NOMODSEQ. This isn't ideal, but seems like the only possibility. From jeetuindian at gmail.com Tue Apr 10 22:07:48 2012 From: jeetuindian at gmail.com (Jitendra Bhaskar) Date: Wed, 11 Apr 2012 00:37:48 +0530 Subject: [Dovecot] Dsync test on 2.1.3 Message-ID: Hi, I am using dovecot version 2.1.3 on centos. I tested dsync from my live server to one of backup server. But I got that its working for most of the user and there is problem for few user. Some users got duplicated mail to their mailbox and I saw there is some extra folders i.e Sent Items_* , Draft_* and Trash_* in few users mail box. and I synced only from main server to backup server. I didn't enable mail_plugin in backup server. My configuration for testing of Dsync on master server is as: mail_plugins = $mail_plugins notify replication service aggregator { fifo_listener replication-notify-fifo { user = vmail mode = 0666 } unix_listener replication-notify { user = vmail mode = 0666 } } service replicator { # start replication at startup process_min_avail = 1 } plugin { #host1 replicates to host2 mail_replica = remote:root at backup_server } #dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -l%{lock_timeout} -n%{namespace} dsync_remote_cmd = ssh -l root %{host} doveadm dsync-server -u%{user} -l%{10} -n%{namespace} service doveadm { user = root } And in backup server : service doveadm { user = root } 1. Problem is I am not getting why some user getting lots of duplicate mails. 2. Why duplicate folders like Sent Items_* and Trash_* created for few users 3. I used only one way syncing then why both replicated. Note: i am using root only for testing purpose * Thanks & Regards * *Jitendra Kumar Bhaskar* Cell:- +91 7306311531 +91 8102997821 From fxmulder at gmail.com Tue Apr 10 23:19:26 2012 From: fxmulder at gmail.com (James Devine) Date: Tue, 10 Apr 2012 14:19:26 -0600 Subject: [Dovecot] lazy_expunge questions Message-ID: I'm messing around with lazy expunge and I just had a couple of questions. If I delete messages or folders with messages, I see them being moved to the expunge namespace, but if I delete an empty folder, all traces of it seem to disappear. Is there a way to move empty folders to the expunge namespace also? My second question is in regards to restoring folders. If I try to execute the imap command 'RENAME EXPUNGED/TEST INBOX/TEST' I get 'NO [CANNOT] Can't rename mailboxes to/from expunge namespace'. Just wondering if this is possible or do I need to recreate the folder and then move messages into it. From gwurster at rim.com Tue Apr 10 23:50:07 2012 From: gwurster at rim.com (Glenn Wurster) Date: Tue, 10 Apr 2012 20:50:07 +0000 Subject: [Dovecot] Bug in HIGHESTMODSEQ when MODSEQ tracking is not enabled for version 2.0.18. In-Reply-To: <11BD08AA-2E29-48DF-973D-4D5B4CCE2FF7@iki.fi> References: <48A61B0976102F488C64EF8A8F1E929D074BBA@XMB118CNC.rim.net> <11BD08AA-2E29-48DF-973D-4D5B4CCE2FF7@iki.fi> Message-ID: <48A61B0976102F488C64EF8A8F1E929D074E9F@XMB118CNC.rim.net> > Yes, modseqs aren't tracked in a mailbox until client expresses an > interest for them. It would be a waste of disk space to save them > since 99% of users don't need them. Makes sense, our mail client gets caught in the middle though, because it uses HIGHESTMODSEQ to track mailbox updates without using MODSEQ options on SELECT/FETCH to track message updates. > RFC 4551 says that HIGHESTMODSEQ or NOMODSEQ MUST be returned. Hmm. > Perhaps: > > 1) If the session is known to have modseqs enabled, immediately enable > modseqs for newly created mailboxes MODSEQ seems to be enabled on the mailbox, not on the session. For disconnected IMAP, there will frequently be new sessions established, and so we may not have a good indicator of whether the client is using MODSEQ (based on my reading). The current approach of looking for a MODSEQ related command seems to work in most cases, and so I'd be tempted to keep the current functionality (minus the incorrect reporting of HIGHESTMODSEQ if we can find a solution). > 2) If a mailbox doesn't have modseqs enabled, return NOMODSEQ. This > isn't ideal, but seems like the only possibility. The RFC also states that if we return NOMODSEQ we'd have to return a tagged BAD response to "UID FETCH 1 MODSEQ", which appears to one of the commands that enables MODSEQ for Dovecot ("SELECT INBOX (CONDSTORE)" also enables it...). What about returning a BAD response and at the same time start tracking MODSEQ so that future SELECT commands would return HIGHESTMODSEQ? Do we know what email clients are using CONDSTORE options and how they'd react to a mailbox suddenly having MODSEQ capabilities after we just told them it didn't? Glenn. --------------------------------------------------------------------- This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful. From branko at majic.rs Wed Apr 11 00:07:54 2012 From: branko at majic.rs (=?UTF-8?B?0JHRgNCw0L3QutC+INCc0LDRmNC40Zs=?=) Date: Tue, 10 Apr 2012 23:07:54 +0200 Subject: [Dovecot] Username from rfc822Name subject alternative name In-Reply-To: <578B8B92-3EC4-4137-B4D0-3B68A767CAB5@iki.fi> References: <578B8B92-3EC4-4137-B4D0-3B68A767CAB5@iki.fi> Message-ID: <20120410230754.1ab2831b@majic.rs> Another solution could be having another property that specifies whether the information should be pulled in from subject or subjectAltName. But I think what you've proposed would be ok as well, and it'd allow for future expansion too (if anyone decides to use some other SAN). I can get working on adding the support for this (might take a couple of days until I sort it out - getting quite busy here :). Would a patch both against the 1.x and 2.x be welcome? ???? Tue, 10 Apr 2012 19:10:04 +0300 Timo Sirainen ??????: > On 9.4.2012, at 16.51, ?????? ????? wrote: > > > I'm looking into adding support for extracting the username from > > client certificate's rfc822Name (from the subjectAltName extension). > > > > The question I have is what would be the best approach to do this? > > Current implementation has a kind of clean code since it just goes > > through the subject name, extracting the values with > > X509_NAME_get_text_by_NID (while NID is obtained with OBJ_txt2nid). > > If I were to add this, it's bound to make the code a little bit > > more complicated since SAN's can't be retrieved in the same way. > > > > So far in terms of options I have, I can see the following: > > > > 1. Create a distinct configuration option for the > > ssl_cert_username_field (i.e. specify something like > > "sanrfc822Name" to have Dovecot extract the username from the > > designated alternative name). > > I'm not sure if this is a good idea, but what about: > > ssl_cert_username_field = subjectAltName:rfc822Name > > > 2. Make the current code fail-over to rfc822Name SAN if > > emailAddress is provided for ssl_cert_username (less invasion in > > code, but less flexibility as well). > > Automatic failover seems dangerous. -- Branko Majic Jabber: branko at majic.rs Please use only Free formats when sending attachments to me. ?????? ????? ?????: branko at majic.rs ????? ??? ?? ??????? ?????? ????????? ? ????????? ?????????. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From robert at schetterer.org Wed Apr 11 00:08:56 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 10 Apr 2012 23:08:56 +0200 Subject: [Dovecot] sieve replace dot string for foldername Message-ID: <4F84A168.5090700@schetterer.org> Hi, i am playing about mail archive this is what allready works for me require ["variables","date","fileinto","mailbox","envelope","subaddress","regex"]; # Extract date info if currentdate :matches "year" "*" { set "year" "${1}"; } if currentdate :matches "month" "*" { set "month" "${1}"; } if currentdate :matches "day" "*" { set "day" "${1}"; } if envelope :detail :matches "to" "*" { set "to" "${1}"; } if true { fileinto :create "INBOX.${year}.${month}.${day}.${to}"; stop; } but ${to} may have a dot included, foldernames with dot are not allowed in my dove setup yet ( and i dont want to change it ) i am looking for a way ( guess regex ) for replace a dot in string replaced by a space char on the fly -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Wed Apr 11 00:44:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 00:44:04 +0300 Subject: [Dovecot] Username from rfc822Name subject alternative name In-Reply-To: <20120410230754.1ab2831b@majic.rs> References: <578B8B92-3EC4-4137-B4D0-3B68A767CAB5@iki.fi> <20120410230754.1ab2831b@majic.rs> Message-ID: <83E9DEB2-6986-4097-95EF-1532D1FB873F@iki.fi> I can add the patch to v2.1, but probably not to earlier versions. On 11.4.2012, at 0.07, ?????? ????? wrote: > Another solution could be having another property that specifies > whether the information should be pulled in from subject or > subjectAltName. > > But I think what you've proposed would be ok as well, and it'd allow > for future expansion too (if anyone decides to use some other SAN). I > can get working on adding the support for this (might take a couple > of days until I sort it out - getting quite busy here :). > > Would a patch both against the 1.x and 2.x be welcome? > > ???? Tue, 10 Apr 2012 19:10:04 +0300 > Timo Sirainen ??????: > >> On 9.4.2012, at 16.51, ?????? ????? wrote: >> >>> I'm looking into adding support for extracting the username from >>> client certificate's rfc822Name (from the subjectAltName extension). >>> >>> The question I have is what would be the best approach to do this? >>> Current implementation has a kind of clean code since it just goes >>> through the subject name, extracting the values with >>> X509_NAME_get_text_by_NID (while NID is obtained with OBJ_txt2nid). >>> If I were to add this, it's bound to make the code a little bit >>> more complicated since SAN's can't be retrieved in the same way. >>> >>> So far in terms of options I have, I can see the following: >>> >>> 1. Create a distinct configuration option for the >>> ssl_cert_username_field (i.e. specify something like >>> "sanrfc822Name" to have Dovecot extract the username from the >>> designated alternative name). >> >> I'm not sure if this is a good idea, but what about: >> >> ssl_cert_username_field = subjectAltName:rfc822Name >> >>> 2. Make the current code fail-over to rfc822Name SAN if >>> emailAddress is provided for ssl_cert_username (less invasion in >>> code, but less flexibility as well). >> >> Automatic failover seems dangerous. > > -- > Branko Majic > Jabber: branko at majic.rs > Please use only Free formats when sending attachments to me. > > ?????? ????? > ?????: branko at majic.rs > ????? ??? ?? ??????? ?????? ????????? ? ????????? ?????????. From stephan at rename-it.nl Wed Apr 11 01:06:48 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 11 Apr 2012 00:06:48 +0200 Subject: [Dovecot] sieve replace dot string for foldername In-Reply-To: <4F84A168.5090700@schetterer.org> References: <4F84A168.5090700@schetterer.org> Message-ID: <4F84AEF8.8000504@rename-it.nl> On 4/10/2012 11:08 PM, Robert Schetterer wrote: > Hi, i am playing about mail archive > > this is what allready works for me > > require > ["variables","date","fileinto","mailbox","envelope","subaddress","regex"]; > > # Extract date info > if currentdate :matches "year" "*" { set "year" "${1}"; } > if currentdate :matches "month" "*" { set "month" "${1}"; } > if currentdate :matches "day" "*" { set "day" "${1}"; } > if envelope :detail :matches "to" "*" { set "to" "${1}"; } > > if true { > fileinto :create "INBOX.${year}.${month}.${day}.${to}"; > stop; > } > > but ${to} may have a dot included, foldernames > with dot are not allowed in my dove setup yet ( and i dont want to > change it ) > > i am looking for a way ( guess regex ) > for replace a dot in string replaced by a space char > on the fly Hmm, no, I see no possibilities for that really. Sieve doesn't have such advanced string manipulation capabilities unfortunately. You could use the vnd.dovecot.execute extension (extprograms plugin) for that, but I wouldn't call that a nice solution by any means. Regards, Stephan. From robert at schetterer.org Wed Apr 11 01:44:33 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 11 Apr 2012 00:44:33 +0200 Subject: [Dovecot] sieve replace dot string for foldername In-Reply-To: <4F84AEF8.8000504@rename-it.nl> References: <4F84A168.5090700@schetterer.org> <4F84AEF8.8000504@rename-it.nl> Message-ID: <4F84B7D1.1050507@schetterer.org> Am 11.04.2012 00:06, schrieb Stephan Bosch: > On 4/10/2012 11:08 PM, Robert Schetterer wrote: >> Hi, i am playing about mail archive >> >> this is what allready works for me >> >> require >> ["variables","date","fileinto","mailbox","envelope","subaddress","regex"]; >> >> >> # Extract date info >> if currentdate :matches "year" "*" { set "year" "${1}"; } >> if currentdate :matches "month" "*" { set "month" "${1}"; } >> if currentdate :matches "day" "*" { set "day" "${1}"; } >> if envelope :detail :matches "to" "*" { set "to" "${1}"; } >> >> if true { >> fileinto :create "INBOX.${year}.${month}.${day}.${to}"; >> stop; >> } >> >> but ${to} may have a dot included, foldernames >> with dot are not allowed in my dove setup yet ( and i dont want to >> change it ) >> >> i am looking for a way ( guess regex ) >> for replace a dot in string replaced by a space char >> on the fly > > Hmm, no, I see no possibilities for that really. Sieve doesn't have such > advanced string manipulation capabilities unfortunately. > > You could use the vnd.dovecot.execute extension (extprograms plugin) for > that, but I wouldn't call that a nice solution by any means. > > Regards, > > Stephan. > > > Thx Stephan, so i dont have to search anymore perhaps i ll use listescape plugin for dot foldernames no idea if it breaks existing setup stuff yet -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From lee at standen.id.au Wed Apr 11 06:54:35 2012 From: lee at standen.id.au (Lee Standen) Date: Wed, 11 Apr 2012 11:54:35 +0800 Subject: [Dovecot] F5 SLB iRule - POP3 TLS Offload & Add Realm Message-ID: Hey Guys, I figured someone might have use of this at some point. I've created an iRule for our F5 Load Balancer which performs the following tasks: - Appends STLS to the POP3 Capability list (it does this blindly, so expects the backend POP3 server to not return this) - Watches for the STLS command and initiates SSL negotiation with the client (leaves the connection to the backend POP3 server unencrypted) - Watches for the USER command (login attempt) and appends a realm if the user hasn't supplied one - Stops watching once a successful login occurs It's designed to be attached to port 110 and port 995 at the same time, and will disable SSL if the user is connecting to port 110 until the STLS command is issued. This is a first pass, so there's probably room for improvement, but it does seem to work. This will require LTM v10 (at least), as the SSL::collect command didn't exist prior to that version. Here's the rule: when CLIENT_ACCEPTED { if { [TCP::local_port clientside] == "110" } { # Disable SSL if we're on port 110 (as we're doing TLS Offload) set secure 0 SSL::disable } else { set secure 1 } set realm "mydomain.com" set debug 0 set loggedin 0 } when SERVER_CONNECTED { TCP::collect } when CLIENTSSL_HANDSHAKE { SSL::collect set secure 1 } when CLIENTSSL_DATA { if { $debug } { log local0. "CLIENT DATA: [SSL::payload]" } set lcpayload [string tolower [SSL::payload]] if { $lcpayload starts_with "user" } { scan [SSL::payload] {%s%s} command user set pos [string first $user [SSL::payload] 0] SSL::payload replace $pos [string length $user] "$user@$realm" if { $debug } { log local0. [SSL::payload] } } SSL::release if { $loggedin == 0 } { SSL::collect } } when CLIENT_DATA { if { $debug } { log local0. "CLIENT DATA: [TCP::payload]" } set lcpayload [string tolower [TCP::payload]] if { $lcpayload starts_with "stls" } { TCP::respond "+OK Begin TLS negotiation\r\n" TCP::payload replace 0 [TCP::payload length] "" TCP::release SSL::enable return } elseif { $lcpayload starts_with "user" } { scan [TCP::payload] {%s%s} command user set pos [string first $user [TCP::payload] 0] TCP::payload replace $pos [string length $user] "$user@$realm" if { $debug } { log local0. [TCP::payload] } } TCP::release if { $loggedin == 0 } { TCP::collect } } when SERVER_DATA { if { $debug } { log local0. "SERVER DATA: [TCP::payload]" } set lcpayload [string tolower [TCP::payload]] if { $lcpayload starts_with "+ok logged in" } { TCP::release set loggedin 1 event disable if { $debug } { log local0. "POP3 Login Successful" } return } elseif { $lcpayload contains "capa" } { TCP::payload replace [expr [TCP::payload length] - 3] 0 "STLS\r\n" } if { $secure == 1 && $loggedin == 0 } { clientside { SSL::collect } } elseif { $secure == 0 && $loggedin == 0 } { clientside { TCP::collect } } TCP::release if { $loggedin == 0 } { TCP::collect } } From dmiller at amfes.com Wed Apr 11 08:06:34 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Tue, 10 Apr 2012 22:06:34 -0700 Subject: [Dovecot] Folder Aliases - Revisited Yet Again Four Years Later In-Reply-To: <1281704852.3046.120.camel@kurkku.sapo.corppt.com> References: <4C648BF7.4020209@amfes.com> <1281704852.3046.120.camel@kurkku.sapo.corppt.com> Message-ID: On 8/13/2010 6:07 AM, Timo Sirainen wrote: > On Thu, 2010-08-12 at 17:04 -0700, Daniel L. Miller wrote: >> If we can't get clients to agree on a "standard" set of folders >> (probably reasonably safe assumption) > [...] > > > I'm kind of annoyed with listescape though. Hopefully for v2.1 I can > figure out some API changes to make it much easier to modify mailbox > names, and I guess also make such aliasing plugin much easier to > implement. > Did you happen to look into this in your spare time? -- Daniel From tss at iki.fi Wed Apr 11 08:12:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 08:12:36 +0300 Subject: [Dovecot] Folder Aliases - Revisited Yet Again Four Years Later In-Reply-To: References: <4C648BF7.4020209@amfes.com> <1281704852.3046.120.camel@kurkku.sapo.corppt.com> Message-ID: On 11.4.2012, at 8.06, Daniel L. Miller wrote: > On 8/13/2010 6:07 AM, Timo Sirainen wrote: >> On Thu, 2010-08-12 at 17:04 -0700, Daniel L. Miller wrote: >>> If we can't get clients to agree on a "standard" set of folders >>> (probably reasonably safe assumption) >> [...] >> >> >> I'm kind of annoyed with listescape though. Hopefully for v2.1 I can >> figure out some API changes to make it much easier to modify mailbox >> names, and I guess also make such aliasing plugin much easier to >> implement. > > Did you happen to look into this in your spare time? Yes, this is much easier with v2.1 I think. The new listescape plugin is just a tiny wrapper that will probably just be replaced by a setting some day in future. I haven't actually tried to write such alias plugin though. From tss at iki.fi Wed Apr 11 08:55:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 08:55:04 +0300 Subject: [Dovecot] Folder Aliases - Revisited Yet Again Four Years Later In-Reply-To: References: <4C648BF7.4020209@amfes.com> <1281704852.3046.120.camel@kurkku.sapo.corppt.com> Message-ID: <91485582-FA24-41C8-B478-62B4EF174C15@iki.fi> On 11.4.2012, at 8.12, Timo Sirainen wrote: > Yes, this is much easier with v2.1 I think. The new listescape plugin is just a tiny wrapper that will probably just be replaced by a setting some day in future. I haven't actually tried to write such alias plugin though. Here: http://dovecot.org/patches/2.1/mailbox-alias-plugin.c But as it is, it doesn't really work in a useful way, since e.g.: x create real x OK Create completed. x list "" real x OK List completed. x list "" alias * LIST (\HasNoChildren) "/" "alias" x OK List completed. x create real x NO [ALREADYEXISTS] Mailbox already exists x status real (messages) * STATUS "real" (MESSAGES 0) x OK Status completed. Perhaps a more useful feature would be if LIST showed both alias and real? That would need a bit more code. From tss at iki.fi Wed Apr 11 09:01:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 09:01:16 +0300 Subject: [Dovecot] lazy_expunge questions In-Reply-To: References: Message-ID: <4791F1D6-6BF2-4FB8-82CE-7C99DB50B4EA@iki.fi> On 10.4.2012, at 23.19, James Devine wrote: > I'm messing around with lazy expunge and I just had a couple of questions. > > If I delete messages or folders with messages, I see them being moved to > the expunge namespace, but if I delete an empty folder, all traces of it > seem to disappear. Is there a way to move empty folders to the expunge > namespace also? If you give 3 namespaces to lazyexpunge, it uses another way of moving stuff, and that includes also moving empty folders. But with 1 namespace it doesn't currently move it. Perhaps it should. > My second question is in regards to restoring folders. If I try to execute > the imap command 'RENAME EXPUNGED/TEST INBOX/TEST' I get 'NO [CANNOT] Can't > rename mailboxes to/from expunge namespace'. Just wondering if this is > possible or do I need to recreate the folder and then move messages into it. You need to recreate + move. Renaming across namespaces is a little tricky. From tss at iki.fi Wed Apr 11 09:26:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 09:26:20 +0300 Subject: [Dovecot] Bug tracker Message-ID: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> I started thinking that perhaps I should move my TODO list to a bug tracker. But because of reasons I've explained a few times before, I don't want a full blown public bug tracking system. The requirements for it are: * I am the only person who can add new bugs. Everyone else reports bugs/requests to this mailing list as before. (Well, I guess Stephan could use this as well if he wants to.) * Everyone can comment existing bugs. * Dovecot mailing list integration: Commenting a bug sends a mail to the mailing list. Replies to those comments go back to bug tracker (probably based on some [#1234] tag in subject). I would have the option of adding a comment that doesn't go to the mailing list (= adding some internal comment that nobody else cares about). Notifications about new bugs won't go to the mailing list (most likely it was created due to a recent mailing list post). So the main difference to how things work now is that people would be able to easily browse existing bugs and add comments to them. I would add bugs there only when I'm not planning on fixing them within a few days. I wouldn't add each and every feature request there, only the things that I'm actually interested in developing. So the idea would be to actually get the bug tracker emptied at some point, not to be a graveyard of unimportant feature requests that about 1-2 people in the world would want. So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. From martin.rabl at rablnet.de Wed Apr 11 09:43:46 2012 From: martin.rabl at rablnet.de (Martin Rabl) Date: Wed, 11 Apr 2012 08:43:46 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: <8095D302-D3CE-4A4E-AB12-64DB2EF7BD96@rablnet.de> What about Mantis? --- Martin Rabl Am 11.04.2012 um 08:26 schrieb Timo Sirainen : > I started thinking that perhaps I should move my TODO list to a bug tracker. But because of reasons I've explained a few times before, I don't want a full blown public bug tracking system. The requirements for it are: > > * I am the only person who can add new bugs. Everyone else reports bugs/requests to this mailing list as before. (Well, I guess Stephan could use this as well if he wants to.) > > * Everyone can comment existing bugs. > > * Dovecot mailing list integration: Commenting a bug sends a mail to the mailing list. Replies to those comments go back to bug tracker (probably based on some [#1234] tag in subject). I would have the option of adding a comment that doesn't go to the mailing list (= adding some internal comment that nobody else cares about). Notifications about new bugs won't go to the mailing list (most likely it was created due to a recent mailing list post). > > So the main difference to how things work now is that people would be able to easily browse existing bugs and add comments to them. I would add bugs there only when I'm not planning on fixing them within a few days. I wouldn't add each and every feature request there, only the things that I'm actually interested in developing. So the idea would be to actually get the bug tracker emptied at some point, not to be a graveyard of unimportant feature requests that about 1-2 people in the world would want. > > So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. > From janfrode at tanso.net Wed Apr 11 09:44:19 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 11 Apr 2012 08:44:19 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: <20120411064419.GA25675@dibs.tanso.net> On Wed, Apr 11, 2012 at 09:26:20AM +0300, Timo Sirainen wrote: > > So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. > I didn't see open source as a requirement, so then I would give a plug for Jira, which is the nicest/prettiest thing :-) And they provide free hosted solution: http://www.atlassian.com/software/jira/pricing Apache/ASF is a heavy jira user, in case you're not familiar with it: http://wiki.apache.org/general/ApacheJira https://issues.apache.org/jira/ -jf From tss at iki.fi Wed Apr 11 09:46:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 09:46:47 +0300 Subject: [Dovecot] Bug tracker In-Reply-To: <8095D302-D3CE-4A4E-AB12-64DB2EF7BD96@rablnet.de> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> <8095D302-D3CE-4A4E-AB12-64DB2EF7BD96@rablnet.de> Message-ID: <65B699AA-10F7-4CF8-81F7-AA59895D54E9@iki.fi> Probably the ugliest/user-unfriendliest bug tracker UI that I've ever seen :) On 11.4.2012, at 9.43, Martin Rabl wrote: > What about Mantis? > > > --- > Martin Rabl > > Am 11.04.2012 um 08:26 schrieb Timo Sirainen : > >> I started thinking that perhaps I should move my TODO list to a bug tracker. But because of reasons I've explained a few times before, I don't want a full blown public bug tracking system. The requirements for it are: >> >> * I am the only person who can add new bugs. Everyone else reports bugs/requests to this mailing list as before. (Well, I guess Stephan could use this as well if he wants to.) >> >> * Everyone can comment existing bugs. >> >> * Dovecot mailing list integration: Commenting a bug sends a mail to the mailing list. Replies to those comments go back to bug tracker (probably based on some [#1234] tag in subject). I would have the option of adding a comment that doesn't go to the mailing list (= adding some internal comment that nobody else cares about). Notifications about new bugs won't go to the mailing list (most likely it was created due to a recent mailing list post). >> >> So the main difference to how things work now is that people would be able to easily browse existing bugs and add comments to them. I would add bugs there only when I'm not planning on fixing them within a few days. I wouldn't add each and every feature request there, only the things that I'm actually interested in developing. So the idea would be to actually get the bug tracker emptied at some point, not to be a graveyard of unimportant feature requests that about 1-2 people in the world would want. >> >> So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. >> > From tss at iki.fi Wed Apr 11 09:49:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 09:49:18 +0300 Subject: [Dovecot] Bug tracker In-Reply-To: <20120411064419.GA25675@dibs.tanso.net> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> <20120411064419.GA25675@dibs.tanso.net> Message-ID: On 11.4.2012, at 9.44, Jan-Frode Myklebust wrote: > On Wed, Apr 11, 2012 at 09:26:20AM +0300, Timo Sirainen wrote: >> >> So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. >> > > I didn't see open source as a requirement, so then I would give a plug > for Jira, which is the nicest/prettiest thing :-) I don't think it supports one of my requirements: > I would have the option of adding a comment that doesn't go to the mailing list Unless that's been added in a newer version. From ngu.antoine at gmail.com Wed Apr 11 09:49:45 2012 From: ngu.antoine at gmail.com (Antoine Nguyen) Date: Wed, 11 Apr 2012 08:49:45 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: Le 11 avril 2012 08:26, Timo Sirainen a ?crit : > I started thinking that perhaps I should move my TODO list to a bug > tracker. But because of reasons I've explained a few times before, I don't > want a full blown public bug tracking system. The requirements for it are: > > * I am the only person who can add new bugs. Everyone else reports > bugs/requests to this mailing list as before. (Well, I guess Stephan could > use this as well if he wants to.) > > * Everyone can comment existing bugs. > > * Dovecot mailing list integration: Commenting a bug sends a mail to the > mailing list. Replies to those comments go back to bug tracker (probably > based on some [#1234] tag in subject). I would have the option of adding a > comment that doesn't go to the mailing list (= adding some internal comment > that nobody else cares about). Notifications about new bugs won't go to the > mailing list (most likely it was created due to a recent mailing list post). > > So the main difference to how things work now is that people would be able > to easily browse existing bugs and add comments to them. I would add bugs > there only when I'm not planning on fixing them within a few days. I > wouldn't add each and every feature request there, only the things that I'm > actually interested in developing. So the idea would be to actually get the > bug tracker emptied at some point, not to be a graveyard of unimportant > feature requests that about 1-2 people in the world would want. > > So, any suggestions for what software could do these things? I think > Request Tracker has those features, but it's not really the > nicest/prettiest thing. > > Maybe Redmine ? (http://www.redmine.org/ It's more than just a bug tracker but I think it answers your needs. Antoine Modoboa developer (http://modoboa.org/) From martin.rabl at rablnet.de Wed Apr 11 09:53:24 2012 From: martin.rabl at rablnet.de (Martin Rabl) Date: Wed, 11 Apr 2012 08:53:24 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: <65B699AA-10F7-4CF8-81F7-AA59895D54E9@iki.fi> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> <8095D302-D3CE-4A4E-AB12-64DB2EF7BD96@rablnet.de> <65B699AA-10F7-4CF8-81F7-AA59895D54E9@iki.fi> Message-ID: ;-) Ok, next one: TRAC ? --- Martin Rabl Am 11.04.2012 um 08:46 schrieb Timo Sirainen : > Probably the ugliest/user-unfriendliest bug tracker UI that I've ever seen :) > > On 11.4.2012, at 9.43, Martin Rabl wrote: > >> What about Mantis? >> >> >> --- >> Martin Rabl >> >> Am 11.04.2012 um 08:26 schrieb Timo Sirainen : >> >>> I started thinking that perhaps I should move my TODO list to a bug tracker. But because of reasons I've explained a few times before, I don't want a full blown public bug tracking system. The requirements for it are: >>> >>> * I am the only person who can add new bugs. Everyone else reports bugs/requests to this mailing list as before. (Well, I guess Stephan could use this as well if he wants to.) >>> >>> * Everyone can comment existing bugs. >>> >>> * Dovecot mailing list integration: Commenting a bug sends a mail to the mailing list. Replies to those comments go back to bug tracker (probably based on some [#1234] tag in subject). I would have the option of adding a comment that doesn't go to the mailing list (= adding some internal comment that nobody else cares about). Notifications about new bugs won't go to the mailing list (most likely it was created due to a recent mailing list post). >>> >>> So the main difference to how things work now is that people would be able to easily browse existing bugs and add comments to them. I would add bugs there only when I'm not planning on fixing them within a few days. I wouldn't add each and every feature request there, only the things that I'm actually interested in developing. So the idea would be to actually get the bug tracker emptied at some point, not to be a graveyard of unimportant feature requests that about 1-2 people in the world would want. >>> >>> So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. >>> >> > From tlx at leuxner.net Wed Apr 11 09:54:28 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Wed, 11 Apr 2012 08:54:28 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: <20120411065428.GA3099@nihlus.leuxner.net> On Wed, Apr 11, 2012 at 09:26:20AM +0300, Timo Sirainen wrote: > So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. > May be worth to take a look at http://www.thebuggenie.com/. Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From p at state-of-mind.de Wed Apr 11 10:10:07 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Wed, 11 Apr 2012 09:10:07 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: <20120411071006.GC28281@state-of-mind.de> * Antoine Nguyen : > > I started thinking that perhaps I should move my TODO list to a bug > > tracker. But because of reasons I've explained a few times before, I don't > > want a full blown public bug tracking system. The requirements for it are: > > > > * I am the only person who can add new bugs. Everyone else reports > > bugs/requests to this mailing list as before. (Well, I guess Stephan could > > use this as well if he wants to.) > > > > * Everyone can comment existing bugs. > > > > * Dovecot mailing list integration: Commenting a bug sends a mail to the > > mailing list. Replies to those comments go back to bug tracker (probably > > based on some [#1234] tag in subject). I would have the option of adding a > > comment that doesn't go to the mailing list (= adding some internal comment > > that nobody else cares about). Notifications about new bugs won't go to the > > mailing list (most likely it was created due to a recent mailing list post). > > > > So the main difference to how things work now is that people would be able > > to easily browse existing bugs and add comments to them. I would add bugs > > there only when I'm not planning on fixing them within a few days. I > > wouldn't add each and every feature request there, only the things that I'm > > actually interested in developing. So the idea would be to actually get the > > bug tracker emptied at some point, not to be a graveyard of unimportant > > feature requests that about 1-2 people in the world would want. > > > > So, any suggestions for what software could do these things? I think > > Request Tracker has those features, but it's not really the > > nicest/prettiest thing. > > > > > Maybe Redmine ? (http://www.redmine.org/ > > It's more than just a bug tracker but I think it answers your needs. +1 We moved from trac to redmine about a year ago and still are very happy about it. Our non-technical customers can deal with it well. Redmine allows to hide projects and it knows the concept of sub-projects. It's versatile in terms of repositories (mercurial supported) and ticket handling is fine too. Sometimes I miss a more comfortable wiki editor, but that's a minor tradeoff. p at rick -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From warren at decoy.co.za Wed Apr 11 10:18:28 2012 From: warren at decoy.co.za (Warren Baker) Date: Wed, 11 Apr 2012 09:18:28 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: <20120411071006.GC28281@state-of-mind.de> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> <20120411071006.GC28281@state-of-mind.de> Message-ID: On Wed, Apr 11, 2012 at 9:10 AM, Patrick Ben Koetter

wrote: > * Antoine Nguyen : >> >> It's more than just a bug tracker but I think it answers your needs. > > +1 > > We moved from trac to redmine about a year ago and still are very happy about > it. Our non-technical customers can deal with it well. Redmine allows to hide > projects and it knows the concept of sub-projects. It's versatile in terms of > repositories (mercurial supported) and ticket handling is fine too. > > Sometimes I miss a more comfortable wiki editor, but that's a minor tradeoff. +2 - we have been using it for another open source project for a number of years and it works well. We have it hooked up into github and it works without a glitch. thx -- .warren From stan at hardwarefreak.com Wed Apr 11 10:18:49 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 11 Apr 2012 02:18:49 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> Message-ID: <4F853059.1000706@hardwarefreak.com> On 4/10/2012 1:09 AM, Emmanuel Noobadmin wrote: > On 4/10/12, Stan Hoeppner wrote: >> SuperMicro H8SGL G34 mobo w/dual Intel GbE, 2GHz 8-core Opteron >> 32GB Kingston REG ECC DDR3, LSI 9280-4i4e, Intel 24 port SAS expander >> 20 x 1TB WD RE4 Enterprise 7.2K SATA2 drives >> NORCO RPC-4220 4U 20 Hot-Swap Bays, SuperMicro 865W PSU >> All other required parts are in the Wish List. I've not written >> assembly instructions. I figure anyone who would build this knows what >> s/he is doing. >> >> Price today: $5,376.62 > > This price looks like something I might be able to push through It's pretty phenomenally low considering what all you get, especially 20 enterprise class drives. > although I'll probably have to go SATA instead of SAS due to cost of > keeping spares. The 10K drives I mentioned are SATA not SAS. WD's 7.2k RE and 10k Raptor series drives are both SATA but have RAID specific firmware, better reliability, longer warranties, etc. The RAID specific firmware is why both are tested and certified by LSI with their RAID cards. >> Configuring all 20 drives as a RAID10 LUN in the MegaRAID HBA would give >> you a 10TB net Linux device and 10 stripe spindles of IOPS and >> bandwidth. Using RAID6 would yield 18TB net and 18 spindles of read >> throughput, however parallel write throughput will be at least 3-6x >> slower than RAID10, which is why nobody uses RAID6 for transactional >> workloads. > > Not likely to go with RAID 5 or 6 due to concerns about the > uncorrectable read errors risks on rebuild with large arrays. Is the Not to mention rebuild times for large width RAID5/6. > MegaRAID being used as the actual RAID controller or just as a HBA? It's a top shelf RAID controller, 512MB cache, up to 240 drives, SSD support, the works. It's an LSI "Feature Line" card: http://www.lsi.com/products/storagecomponents/Pages/6GBSATA_SASRAIDCards.aspx The specs: http://www.lsi.com/products/storagecomponents/Pages/MegaRAIDSAS9280-4i4e.aspx You'll need the cache battery module for safe write caching, which I forgot in the wish list (now added), $160: http://www.newegg.com/Product/Product.aspx?Item=N82E16816118163&Tpk=LSIiBBU08 With your workload and RAID10 you should run with all 512MB configured as write cache. Linux caches all reads so using any controller cache for reads is a waste. Using all 512MB for write cache will increase random write IOPS. Note the 9280 allows up to 64 LUNs, so you can do tiered storage within this 20 bay chassis. For spares management you'd probably not want to bother with two different sized drives. I didn't mention the 300GB 10K Raptors previously due to their limited capacity. Note they're only $15 more apiece than the 1TB RE4 drives in the original parts list. For a total of $300 more you get the same 40% increase in IOPs of the 600GB model, but you'll only have 3TB net space after RAID10. If 3TB is sufficient space for your needs, that extra 40% IOPS makes this config a no brainer. The decreased latency of the 10K drives will give a nice boost to VM read performance, especially when using NFS. Write performance probably won't be much different due to the generous 512MB write cache on the controller. I also forgot to mention that with BBWC enabled you can turn off XFS barriers, which will dramatically speed up Exim queues and Dovecot writes, all writes actually. Again, you probably don't want the spares management overhead of two different disk types on the shelf, but you could stick these 10K 300s in the first 16 slots, and put the 2TB RE4 drive in the last 4 slots, RAID10 on the 10K drives, RAID5 on the 2TB drives. This yields an 8 spindle high IOPS RAID10 of 2.4TB and a lower performance RAID5 of 6TB for near line storage such as your Dovecot alt storage, VM templates, etc, 8.4TB net, 1.6TB less than the original 10TB setup. Total additional cost is $920 for this setup. You'd have two XFS filesystems (with quite different mkfs parameters). > I have been avoiding hardware RAID because of a really bad experience > with RAID 5 on an obsolete controller that eventually died without > replacement and couldn't be recovered. Since then, it's always been > RAID 1 and, after I discovered mdraid, using them as purely HBA with > mdraid for the flexibility of being able to just pull the drives into > a new system if necessary without having to worry about the > controller. Assuming you have the right connector configuration for your drive/enclosure on the replacement card, you can usually swap out one LSI RAID card with any other LSI RAID card in the same, or newer, generation. It'll read the configuration metadata from the disks and be up an running in minutes. This feature has been around all the way back to the AMI/Mylex cards of the late 1990s. LSI acquired both companies, who were #1 and #2 in RAID, which is why LSI is so successful today. Back in those days LSI simply supplied the ASICs to AMI and Mylex. I have an AMI MegaRAID 428, top of the line in 1998, lying around somewhere. Still working when I retired it many years ago. FYI, LSI is the OEM provider of RAID and SAS/SATA HBA ASIC silicon for the tier 1 HBA and mobo down markets. Dell, HP, IBM, Intel, Oracle (Sun), Siemens/Fujitsu, all use LSI silicon and firmware. Some simply rebadge OEM LSI cards with their own model and part numbers. IBM and Dell specifically have been doing this rebadging for well over a decade, long before LSI acquired Mylex and AMI. The Dell PERC/2 is a rebadged AMI MegaRAID 428. Software and hardware RAID each have their pros and cons. I prefer hardware RAID for write cache performance and many administrative reasons, including SAF-TE enclosure management (fault LEDs, alarms, etc) so you know at a glance which drive has failed and needs replacing, email and SNMP notification of events, automatic rebuild, configurable rebuild priority, etc, etc, and good performance with striping and mirroring. Parity RAID performance often lags behind md with heavy workloads but not with light/medium. FWIW I rarely use parity RAID, due to the myriad performance downsides. For ultra high random IOPS workloads, or when I need a single filesystem space larger than the drive limit or practical limit for one RAID HBA, I'll stitch hardware RAID1 or small stripe width RAID 10 arrays (4-8 drives, 2-4 spindles) together with md RAID 0 or 1. >> Both of the drives I've mentioned here are enterprise class drives, >> feature TLER, and are on the LSI MegaRAID SAS hardware compatibility >> list. The price of the 600GB Raptor has come down considerably since I >> designed this system, or I'd have used them instead. >> >> Anyway, lots of option out there. But $6,500 is pretty damn cheap for a >> quality box with 32GB RAM, enterprise RAID card, and 20x10K RPM 600GB >> drives. >> >> The MegaRAID 9280-4i4e has an external SFF8088 port For an additional >> $6,410 you could add an external Norco SAS expander JBOD chassis and 24 >> more 600GB 10K RPM Raptors, for 13.2TB of total net RAID10 space, and 22 >> 10k spindles of IOPS performance from 44 total drives. That's $13K for >> a 5K random IOPS, 13TB, 44 drive NFS RAID COTS server solution, >> $1000/TB, $2.60/IOPS. Significantly cheaper than an HP, Dell, IBM >> solution of similar specs, each of which will set you back at least 20 >> large. > > Would this setup work well too for serving up VM images? I've been > trying to find a solution for the virtualized app servers images as > well but the distributed FSes currently are all bad with random > reads/writes it seems. XFS seem to be good with large files like db > and vm images with random internal write/read so given my time > constraints, it would be nice to have a single configuration that > works generally well for all the needs I have to oversee. Absolutely. If you setup these 20 drives as a single RAID10, soft/hard or hybrid, with the LSI cache set to 100% write-back, with a single XFS filesystem with 10 allocation groups and proper stripe alignment, you'll get maximum performance for pretty much any conceivable workload. Your only limitations will be possible NFS or TCP tuning issues, and maybe having only two GbE ports. For small random IOPS such as Exim queues, Dovecot store, VM image IO, etc, the two GbE ports are plenty. But if you add any large NFS file copies into the mix, such as copying new VM templates or ISO images over, etc, or do backups over NFS instead of directly on the host machine at the XFS level, then two bonded GbE ports might prove a bottleneck. The mobo has 2 PCIe x8 slots and one x4 slot. One of the x8 slots is an x16 physical connector. You'll put the LSI card in the x16 slot. If you mount the Intel SAS expander to the chassis as I do instead of in a slot, you have one free x8 and one free x4 slot. Given the $250 price, I'd simply ad an Intel quad port GbE NIC to the order. Link aggregate all 4 ports on day one and use one IP address for the NFS traffic. Use the two on board ports for management etc. This should give you a theoretical 400MB/s of peak NFS throughput, which should be plenty no matter what workload you throw at it. >> Note the chassis I've spec'd have single PSUs, not the dual or triple >> redundant supplies you'll see on branded hardware. With a relatively >> stable climate controlled environment and a good UPS with filtering, >> quality single supplies are fine. In fact, in the 4U form factor single >> supplies are usually more reliable due to superior IC packaging and >> airflow through the heatsinks, not to mention much quieter. > > Same reason I do my best to avoid 1U servers, the space/heat issues > worries me. Yes, I'm guilty of worrying too much but that had saved me > on several occasions. Just about every 1U server I've seen that's been racked for 3 or more years has warped under its own weight. I even saw an HPQ 2U that was warped this way, badly warped. In this instance the slide rail bolts had never been tightened down to the rack--could spin them by hand. Since the chassis side panels weren't secured, and there was lateral play, the weight of the 6 drives caused the side walls of the case to fold into a mild trapezoid, which allowed the bottom and top panels to bow. Let this be a lesson boys and girls: always tighten your rack bolts. :) -- Stan From tss at iki.fi Wed Apr 11 10:23:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 10:23:56 +0300 Subject: [Dovecot] Bug tracker In-Reply-To: References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: On 11.4.2012, at 9.49, Antoine Nguyen wrote: > Maybe Redmine ? (http://www.redmine.org/ I don't think it either supports: > I would have the option of adding a comment that doesn't go to the mailing list But I'm beginning to think that no BTS supports that in the way I want. And it's not a huge problem for me really, just means that this mailing list may get a bit more pointless notifications than necessary. Although I'm also not seeing much documentation about email notifications in general. Can it be configured to send comment updates, but no other updates? I guess this could also be kludged by dropping unwanted emails in MTA. From janfrode at tanso.net Wed Apr 11 10:25:22 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 11 Apr 2012 09:25:22 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> <20120411064419.GA25675@dibs.tanso.net> Message-ID: <20120411072522.GA26184@dibs.tanso.net> On Wed, Apr 11, 2012 at 09:49:18AM +0300, Timo Sirainen wrote: > > > > I didn't see open source as a requirement, so then I would give a plug > > for Jira, which is the nicest/prettiest thing :-) > > I don't think it supports one of my requirements: > > > I would have the option of adding a comment that doesn't go to the mailing list > > Unless that's been added in a newer version. > There is an option for restricting who can view your comment, plus "Email notifications will only be sent to people who have permission to view the relevant issue" http://confluence.atlassian.com/display/JIRA/Creating+a+Notification+Scheme so I would expect it to be possible to define that the mailinglist is not member of a group-b, while everyone else is, and restrict the comment to that group. But best would probably be to discuss it with atlassion support... -jf From lee at standen.id.au Wed Apr 11 10:59:28 2012 From: lee at standen.id.au (Lee Standen) Date: Wed, 11 Apr 2012 15:59:28 +0800 Subject: [Dovecot] Bug tracker In-Reply-To: <20120411072522.GA26184@dibs.tanso.net> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> <20120411064419.GA25675@dibs.tanso.net> <20120411072522.GA26184@dibs.tanso.net> Message-ID: We use JIRA internally and it's probably the best bug tracker I've ever seen. The permissions system is very flexible, and to the best of my knowledge it should support everything you've described in your original post. On Wed, Apr 11, 2012 at 3:25 PM, Jan-Frode Myklebust wrote: > On Wed, Apr 11, 2012 at 09:49:18AM +0300, Timo Sirainen wrote: > > > > > > I didn't see open source as a requirement, so then I would give a plug > > > for Jira, which is the nicest/prettiest thing :-) > > > > I don't think it supports one of my requirements: > > > > > I would have the option of adding a comment that doesn't go to the > mailing list > > > > Unless that's been added in a newer version. > > > > There is an option for restricting who can view your comment, plus > "Email notifications will only be sent to people who have permission to > view the relevant issue" > > > http://confluence.atlassian.com/display/JIRA/Creating+a+Notification+Scheme > > so I would expect it to be possible to define that the mailinglist is > not member of a group-b, while everyone else is, and restrict the comment > to that group. > > But best would probably be to discuss it with atlassion support... > > > -jf > From ngu.antoine at gmail.com Wed Apr 11 11:05:05 2012 From: ngu.antoine at gmail.com (Antoine Nguyen) Date: Wed, 11 Apr 2012 10:05:05 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: Le 11 avril 2012 09:23, Timo Sirainen a ?crit : > On 11.4.2012, at 9.49, Antoine Nguyen wrote: > > > Maybe Redmine ? (http://www.redmine.org/ > > I don't think it either supports: > > > I would have the option of adding a comment that doesn't go to the > mailing list > > But I'm beginning to think that no BTS supports that in the way I want. > And it's not a huge problem for me really, just means that this mailing > list may get a bit more pointless notifications than necessary. > > Although I'm also not seeing much documentation about email notifications > in general. Can it be configured to send comment updates, but no other > updates? I guess this could also be kludged by dropping unwanted emails in > MTA. > > AFAIK, yes. You can choose on which kind of event Redmine sends a notification. The documentation is not the best I've seen but the administration panel is pretty simple to understand. Antoine From warren at decoy.co.za Wed Apr 11 11:24:55 2012 From: warren at decoy.co.za (Warren Baker) Date: Wed, 11 Apr 2012 10:24:55 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: On Wed, Apr 11, 2012 at 10:05 AM, Antoine Nguyen wrote: > Le 11 avril 2012 09:23, Timo Sirainen a ?crit : >> > I would have the option of adding a comment that doesn't go to the >> mailing list >> >> But I'm beginning to think that no BTS supports that in the way I want. >> And it's not a huge problem for me really, just means that this mailing >> list may get a bit more pointless notifications than necessary. >> >> Although I'm also not seeing much documentation about email notifications >> in general. Can it be configured to send comment updates, but no other >> updates? I guess this could also be kludged by dropping unwanted emails in >> MTA. >> >> > AFAIK, yes. You can choose on which kind of event Redmine sends a > notification. The documentation is not the best I've seen but the > administration panel is pretty simple to understand. > There is a default notification option which is set to 'Only for things I watch or I'm involved in' - which in your case Timo you would receive everything since you would be the project owner. There are also global settings that you can configure which provide the ability to send notifications for certain events only: - Issue added - Issue updated - Note added - Status updated - Priority updated - News added - Comment added to a news - Document added - File added - Message added - Wiki page added - Wiki page updated I think there is also a plugin to disable email notifications entirely for a user. -- .warren From mlists at edicom.eu Wed Apr 11 12:35:25 2012 From: mlists at edicom.eu (Miguel Tormo) Date: Wed, 11 Apr 2012 11:35:25 +0200 Subject: [Dovecot] Proxy and SSO (single sign-on) In-Reply-To: <201204041347.47817.mlists@edicom.eu> References: <201204041318.00760.mlists@edicom.eu> <48AD358F-ECC4-48F5-8CAB-9B91C8BDEEE3@iki.fi> <201204041347.47817.mlists@edicom.eu> Message-ID: <201204111135.25742.mlists@edicom.eu> El Mi?rcoles, 4 de Abril de 2012 13:47:47 Miguel Tormo escribi?: > El Mi?rcoles, 4 de Abril de 2012 13:21:33 Timo Sirainen escribi?: > > On 4.4.2012, at 14.18, Miguel Tormo wrote: > > > > > I have a running setup with a dovecot imap4/pop3 proxy to a few dovecot backend servers which actually store the mailboxes. This is running smoothly and allows me to transparently distribute mailboxes. > > > I'm using some "extrafield" configured in the LDAP passdb. > > > > > > However, now I would like to use GSSAPI (preferred) and NTLM for single sign-on. Both are pretty straightforward to configure in a single instance environment, but I don't know if they would work with proxy. For example, with GSSAPI there are two cases: > > > 1) Just use gssapi mechanism, without PAM. Then, it a user presents a ticket the passdb ldap is not used, so the extrafields are never read. > > > > The patch in http://dovecot.org/list/dovecot/2012-March/064331.html makes this work I think. I still haven't managed to look into it much though. > > > > > > It definitely is worth a look. I wonder if it would make ldap extrafields lookups work with gssapi auth, I will try it and post the results. Well just in case someone is interested: I applied Sam Morris' patch and got this working. Very nice indeed. To keep in mind: - It's necessary to set a master password for the backend server so the proxy instance can impersonate any user when connecting to the backend instance. I needed to do this to make GSSAPI with proxy work (thus the proxy instance is who actually does the authentication), however NTLM worked without a masster password, it seems this authenticatiom mechanism can be forwarded as is. - In the LDAP configuration, "auth bind" must be set to "no". Obviously, when using sso you don't provide a password, so there is no way it can bind to the LDAP server with your credentials. There is still a problem that might be more related to the MUA, but still if anyone has a suggestion I would appreciate it. The problem is the following: if you are a roaming user (ie: with a laptop) when you are outside the LAN you cannot get a kerberos ticket thus the GSSAPI auth fails. Then you have to change your account configuration and select a password-based authentication mechanism. This is not very convenient. However, this is different if compared with kerberos authentication with a web browser: if you have a valid ticket, access is granted; if not, the browser prompts for user/password and then you are granted access if the supplied credentials are valid. Do you have any idea if something like this is possible to accomplish with IMAP? Thank you and regards, From CMarcus at Media-Brokers.com Wed Apr 11 13:20:47 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 11 Apr 2012 06:20:47 -0400 Subject: [Dovecot] Bug tracker In-Reply-To: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: <4F855AFF.7050704@Media-Brokers.com> On 2012-04-11 2:26 AM, Timo Sirainen wrote: > Notifications about new bugs won't go to the mailing list (most > likely it was created due to a recent mailing list post). I actually would like to see these, and I imagine I'm not alone... If you'd prefer they don't go to the users list, maybe create a new 'notifications' (or maybe even a -dev) list that anyone could subscribe to to see these? > So, any suggestions for what software could do these things? I think > Request Tracker has those features, but it's not really the > nicest/prettiest thing. +1 to Redmine. I only used it for a short time, but I liked it, and the fact that you can integrate it with git or mercurial is a big plus. Yes, it is a full blown bug tracker, but if, in the future, you add more developers (maybe you get some angel funding), you already have a full blown bug tracker in place, including the extras it comes with. -- Best regards, Charles From interfasys at gmail.com Wed Apr 11 12:05:59 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Wed, 11 Apr 2012 10:05:59 +0100 Subject: [Dovecot] Bug tracker Message-ID: <4F854977.9000804@gmail.com> I second that. New version 3.2 supports sub-projects, editions and components and is user-friendly. Cheers, Olivier > On Wed, Apr 11, 2012 at 09:26:20AM +0300, Timo Sirainen wrote: > > So, any suggestions for what software could do these things? I think > Request > Tracker has those features, but it's not really the nicest/prettiest > thing. > > May be worth to take a look at http://www.thebuggenie.com/. > > Thomas > From helga.mayer at uni-hohenheim.de Wed Apr 11 17:13:21 2012 From: helga.mayer at uni-hohenheim.de (Helga Mayer) Date: Wed, 11 Apr 2012 16:13:21 +0200 Subject: [Dovecot] Problems with Apple Mail and attachments Message-ID: <20120411161321.165760epurnu7i4h@webmail.uni-hohenheim.de> Hello, has anyone experiences with the Apple mail client, to be precise : Mail 5.2? When I copy a message with an attachment from the inbox to a different folder using 'move' or 'copy' , Apple Mail does not properly display the attachment in the new folder. When I copy the same message using 'archive' the attachment shows up properly. There is no difference on the server side. We are using maildirs. The raw messages in the various folders are identical. Deleting plist on the Mac didn't help. After removing and recreation of the account (client side) the attachments are displayed correctly again. Is this an apple problem or has it got anything to do with dovecot.index.cache ? Kind Regards Helga Mayer From yzhou at medplus.com Wed Apr 11 17:49:05 2012 From: yzhou at medplus.com (Zhou, Yan) Date: Wed, 11 Apr 2012 14:49:05 +0000 Subject: [Dovecot] ldap idle connection timeout in DoveCot 1.0.13? Message-ID: Hi there, We are using DoveCot 1.0.13, it connects to LDAP server for authentication. It seems that DoveCot keeps the idle LDAP connection open. Our firewall is terminating these connections after some time of idle activity (2 hours), then, we run into authentication problem. If we restart either LDAP or DoveCot, then it is fine. Can we set some kind of LDAP idle connection timeout in DoveCot? /etc/dovecot-ldap.conf. I do not see any configuration available for 1.0.13. Another alternative is to set idle connection timeout in LDAP, but we prefer doing that in DoveCot. Thanks, Yan Confidentiality Notice: The information contained in this electronic transmission is confidential and may be legally privileged. It is intended only for the addressee(s) named above. If you are not an intended recipient, be aware that any disclosure, copying, distribution or use of the information contained in this transmission is prohibited and may be unlawful. If you have received this transmission in error, please notify us by telephone (513) 229-5500 or by email (postmaster at MedPlus.com). After replying, please erase it from your computer system. From fxmulder at gmail.com Wed Apr 11 18:34:54 2012 From: fxmulder at gmail.com (James Devine) Date: Wed, 11 Apr 2012 09:34:54 -0600 Subject: [Dovecot] lazy_expunge questions In-Reply-To: <4791F1D6-6BF2-4FB8-82CE-7C99DB50B4EA@iki.fi> References: <4791F1D6-6BF2-4FB8-82CE-7C99DB50B4EA@iki.fi> Message-ID: On Wed, Apr 11, 2012 at 12:01 AM, Timo Sirainen wrote: > On 10.4.2012, at 23.19, James Devine wrote: > > > I'm messing around with lazy expunge and I just had a couple of > questions. > > > > If I delete messages or folders with messages, I see them being moved to > > the expunge namespace, but if I delete an empty folder, all traces of it > > seem to disappear. Is there a way to move empty folders to the expunge > > namespace also? > > If you give 3 namespaces to lazyexpunge, it uses another way of moving > stuff, and that includes also moving empty folders. But with 1 namespace it > doesn't currently move it. Perhaps it should. > > > My second question is in regards to restoring folders. If I try to > execute > > the imap command 'RENAME EXPUNGED/TEST INBOX/TEST' I get 'NO [CANNOT] > Can't > > rename mailboxes to/from expunge namespace'. Just wondering if this is > > possible or do I need to recreate the folder and then move messages into > it. > > You need to recreate + move. Renaming across namespaces is a little tricky. > > It looks like 3 namespace support in the lazy expunge plugin was removed in 2.1+ ? From tss at iki.fi Wed Apr 11 19:10:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 19:10:36 +0300 Subject: [Dovecot] lazy_expunge questions In-Reply-To: References: <4791F1D6-6BF2-4FB8-82CE-7C99DB50B4EA@iki.fi> Message-ID: On 11.4.2012, at 18.34, James Devine wrote: > It looks like 3 namespace support in the lazy expunge plugin was removed in > 2.1+ ? Yeah, so it seems. Updated the wiki2. From tss at iki.fi Wed Apr 11 19:36:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 19:36:30 +0300 Subject: [Dovecot] ldap idle connection timeout in DoveCot 1.0.13? In-Reply-To: References: Message-ID: On 11.4.2012, at 17.49, Zhou, Yan wrote: > We are using DoveCot 1.0.13, it connects to LDAP server for authentication. It seems that DoveCot keeps the idle LDAP connection open. Yes. > Our firewall is terminating these connections after some time of idle activity (2 hours), then, we run into authentication problem. If we restart either LDAP or DoveCot, then it is fine. > > Can we set some kind of LDAP idle connection timeout in DoveCot? /etc/dovecot-ldap.conf. I do not see any configuration available for 1.0.13. No. But if you upgrade to a newer Dovecot (v2.x probably) this is solved by automatic transparent reconnection. From tss at iki.fi Wed Apr 11 19:37:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 19:37:59 +0300 Subject: [Dovecot] Problems with Apple Mail and attachments In-Reply-To: <20120411161321.165760epurnu7i4h@webmail.uni-hohenheim.de> References: <20120411161321.165760epurnu7i4h@webmail.uni-hohenheim.de> Message-ID: On 11.4.2012, at 17.13, Helga Mayer wrote: > has anyone experiences with the Apple mail client, to be precise : Mail 5.2? > When I copy a message with an attachment from the inbox to a different folder using 'move' or 'copy' , Apple Mail does not properly display the attachment in the new folder. .. > Is this an apple problem or has it got anything to do with dovecot.index.cache ? Highly unlikely to be a Dovecot problem. Of course, you can verify if it's a caching problem by disabling indexes and seeing if you can reproduce the problem: mail_location = maildir:~/Maildir:INDEX=MEMORY From thierry at odry.net Wed Apr 11 19:44:59 2012 From: thierry at odry.net (Thierry de Montaudry) Date: Wed, 11 Apr 2012 18:44:59 +0200 Subject: [Dovecot] Problems with Apple Mail and attachments In-Reply-To: <20120411161321.165760epurnu7i4h@webmail.uni-hohenheim.de> References: <20120411161321.165760epurnu7i4h@webmail.uni-hohenheim.de> Message-ID: On 11 Apr 2012, at 16:13, Helga Mayer wrote: > Hello, > > has anyone experiences with the Apple mail client, to be precise : Mail 5.2? > When I copy a message with an attachment from the inbox to a different folder using 'move' or 'copy' , Apple Mail does not properly display the attachment in the new folder. > > When I copy the same message using 'archive' the attachment shows up properly. > There is no difference on the server side. We are using maildirs. The raw > messages in the various folders are identical. > Deleting plist on the Mac didn't help. > After removing and recreation of the account (client side) the attachments are displayed correctly again. > > Is this an apple problem or has it got anything to do with dovecot.index.cache ? > > Kind Regards > Helga Mayer > > Hi Helga, I'm using Apple Mail 5.2 with a dovecot server (2.0.13, with maildir), and recently had no trouble like that, even though working with a mailbox that has about 10000 emails, a lot of folders, and moving mails around all the time. I've seen a similar problem a while ago (1 year maybe more), but used the mailbox Rebuild option on the client, which fixed it without having to delete and recreate the account. Your problem might just be a local index corruption, which can happen when loosing your Internet connection. Regards, Thierry From lists at wildgooses.com Wed Apr 11 19:50:09 2012 From: lists at wildgooses.com (Ed W) Date: Wed, 11 Apr 2012 17:50:09 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F853059.1000706@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> Message-ID: <4F85B641.5060103@wildgooses.com> Re XFS. Have you been watching BTRFS recently? I will concede that despite the authors considering it production ready I won't be using it for my servers just yet. However, it's benchmarking on single disk benchmarks fairly similarly to XFS and in certain cases (multi-threaded performance) can be somewhat better. I haven't yet seen any benchmarks on larger disk arrays yet, eg 6+ disks, so no idea how it scales up. Basically what I have seen seems "competitive" I don't have such hardware spare to benchmark, but I would be interested to hear from someone who benchmarks your RAID1+linear+XFS suggestion, especially if they have compared a cutting edge btrfs kernel on the same array? One of the snags of md RAID1 vs RAID6 is the lack of checksumming in the event of bad blocks. (I'm not sure what actually happens when md scrubbing finds a bad sector with raid1..?). For low performance requirements I have become paranoid and been using RAID6 vs RAID10, filesystems with sector checksums seem attractive... Regards Ed W From adrian.minta at gmail.com Wed Apr 11 23:48:00 2012 From: adrian.minta at gmail.com (Adrian Minta) Date: Wed, 11 Apr 2012 23:48:00 +0300 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F85B641.5060103@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> Message-ID: <4F85EE00.9030208@gmail.com> On 04/11/12 19:50, Ed W wrote: ... > One of the snags of md RAID1 vs RAID6 is the lack of checksumming in > the event of bad blocks. (I'm not sure what actually happens when md > scrubbing finds a bad sector with raid1..?). For low performance > requirements I have become paranoid and been using RAID6 vs RAID10, > filesystems with sector checksums seem attractive... > RAID6 is very slow for write operations. That's why is the worst choice for maildir. From CMarcus at Media-Brokers.com Wed Apr 11 21:50:11 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 11 Apr 2012 14:50:11 -0400 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F85EE00.9030208@gmail.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F85EE00.9030208@gmail.com> Message-ID: <4F85D263.3070201@Media-Brokers.com> On 2012-04-11 4:48 PM, Adrian Minta wrote: > On 04/11/12 19:50, Ed W wrote: >> One of the snags of md RAID1 vs RAID6 is the lack of checksumming in >> the event of bad blocks. (I'm not sure what actually happens when md >> scrubbing finds a bad sector with raid1..?). For low performance >> requirements I have become paranoid and been using RAID6 vs RAID10, >> filesystems with sector checksums seem attractive... > RAID6 is very slow for write operations. That's why is the worst choice > for maildir. He did say '"For *low* *performance* requirements..." ... ;) -- Best regards, Charles From stan at hardwarefreak.com Thu Apr 12 01:46:44 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 11 Apr 2012 17:46:44 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F8409DA.1090202@gmail.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F8409DA.1090202@gmail.com> Message-ID: <4F8609D4.2040205@hardwarefreak.com> On 4/10/2012 5:22 AM, Adrian Minta wrote: > On 04/10/12 08:00, Stan Hoeppner wrote: >> Interestingly, I designed a COTS server back in January to handle at >> least 5k concurrent IMAP users, using best of breed components. If you >> or someone there has the necessary hardware skills, you could assemble >> this system and simply use it for NFS instead of Dovecot. The parts >> list: >> secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=17069985 > > Don't forget the Battery Backup Unit for RAID card !!! Heh, thanks for the reminder Adrian. :) I got to your email a little late--already corrected the omission. Yes, battery or flash backup for the RAID cache is always a necessity when doing write-back caching. -- Stan From alietsantiesteban at gmail.com Thu Apr 12 02:03:04 2012 From: alietsantiesteban at gmail.com (Aliet Santiesteban Sifontes) Date: Wed, 11 Apr 2012 18:03:04 -0500 Subject: [Dovecot] ldap idle connection timeout in DoveCot 1.0.13? In-Reply-To: References: Message-ID: I had this problem running Dovecot 2.x where LDAP servers are located on another firewall zone, we use Juniper SSG550. The problem was that the firewall was dropping the ldap idle connections so client authentication was failing in dovecot for a while and after a time it reconnects, Dovecot/Openldap-Server never knows that the firewall has dropped the connection because this is the default, the firewall doesn't send TCP -Reset to the client and the server, in Juniper/Netscreen you can do a workaround to speed up the process by configuring the zone to send reset back to the client and the server. Check you have on the firewall: set flow tcp-mss unset flow no-tcp-seq-check set flow tcp-syn-check unset flow tcp-syn-bit-check set flow reverse-route clear-text prefer set flow reverse-route tunnel always Edit your zone and enable "If TCP non SYN, send RESET back" checkbox: This fixed the delay for us, it would be a nice feature at dovecot side... best regards El 11 de abril de 2012 11:36, Timo Sirainen escribi?: > On 11.4.2012, at 17.49, Zhou, Yan wrote: > > > We are using DoveCot 1.0.13, it connects to LDAP server for > authentication. It seems that DoveCot keeps the idle LDAP connection open. > > Yes. > > > Our firewall is terminating these connections after some time of idle > activity (2 hours), then, we run into authentication problem. If we restart > either LDAP or DoveCot, then it is fine. > > > > Can we set some kind of LDAP idle connection timeout in DoveCot? > /etc/dovecot-ldap.conf. I do not see any configuration available for > 1.0.13. > > No. But if you upgrade to a newer Dovecot (v2.x probably) this is solved > by automatic transparent reconnection. > > From stan at hardwarefreak.com Thu Apr 12 04:18:08 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 11 Apr 2012 20:18:08 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F85B641.5060103@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> Message-ID: <4F862D50.8000206@hardwarefreak.com> On 4/11/2012 11:50 AM, Ed W wrote: > Re XFS. Have you been watching BTRFS recently? > > I will concede that despite the authors considering it production ready > I won't be using it for my servers just yet. However, it's benchmarking > on single disk benchmarks fairly similarly to XFS and in certain cases > (multi-threaded performance) can be somewhat better. I haven't yet seen > any benchmarks on larger disk arrays yet, eg 6+ disks, so no idea how it > scales up. Basically what I have seen seems "competitive" Links? > I don't have such hardware spare to benchmark, but I would be interested > to hear from someone who benchmarks your RAID1+linear+XFS suggestion, > especially if they have compared a cutting edge btrfs kernel on the same > array? http://btrfs.boxacle.net/repository/raid/history/History_Mail_server_simulation._num_threads=128.html This is with an 8 wide LVM stripe over 8 17 drive hardware RAID0 arrays. If the disks had been setup as a concat of 68 RAID1 pairs, XFS would have turned in numbers significantly higher, anywhere from a 100% increase to 500%. It's hard to say because the Boxacle folks didn't show the XFG AG config they used. The concat+RAID1 setup can decrease disk seeks by many orders of magnitude vs striping. Everyone knows as seeks go down IOPS go up. Even with this very suboptimal disk setup, XFS still trounces everything but JFS which is a close 2nd. BTRFS is way down in the pack. It would be nice to see these folks update these results with a 3.2.6 kernel, as both BTRFS and XFS have improved significantly since 2.6.35. EXT4 and JFS have seen little performance work since. In fact JFS has seen no commits but bug fixes and changes to allow compiling with recent kernels. > One of the snags of md RAID1 vs RAID6 is the lack of checksumming in the > event of bad blocks. (I'm not sure what actually happens when md > scrubbing finds a bad sector with raid1..?). For low performance > requirements I have become paranoid and been using RAID6 vs RAID10, > filesystems with sector checksums seem attractive... Except we're using hardware RAID1 here and mdraid linear. Thus the controller takes care of sector integrity. RAID6 yields nothing over RAID10, except lower performance, and more usable space if more than 4 drives are used. -- Stan From centos.admin at gmail.com Thu Apr 12 05:23:19 2012 From: centos.admin at gmail.com (Emmanuel Noobadmin) Date: Thu, 12 Apr 2012 10:23:19 +0800 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F862D50.8000206@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> Message-ID: On 4/12/12, Stan Hoeppner wrote: > On 4/11/2012 11:50 AM, Ed W wrote: >> One of the snags of md RAID1 vs RAID6 is the lack of checksumming in the >> event of bad blocks. (I'm not sure what actually happens when md >> scrubbing finds a bad sector with raid1..?). For low performance >> requirements I have become paranoid and been using RAID6 vs RAID10, >> filesystems with sector checksums seem attractive... > > Except we're using hardware RAID1 here and mdraid linear. Thus the > controller takes care of sector integrity. RAID6 yields nothing over > RAID10, except lower performance, and more usable space if more than 4 > drives are used. How would the control ensure sector integrity unless it is writing additional checksum information to disk? I thought only a few filesystems like ZFS does the sector checksum to detect if any data corruption occurred. I suppose the controller could throw an error if the two drives returned data that didn't agree with each other but it wouldn't know which is the accurate copy but that wouldn't protect the integrity of the data, at least not directly without additional human intervention I would think. From tlx at leuxner.net Thu Apr 12 10:11:32 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 12 Apr 2012 09:11:32 +0200 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes Message-ID: <20120412071132.GA2583@nihlus.leuxner.net> Some change between bf5ae73e9475 and 584bd77c38fd seems to have broken something in the SSL Handshake. A previously valid server certificate is deemed invalid by various mail clients. http://hg.dovecot.org/dovecot-2.1/rev/bf5ae73e9475 works fine while http://hg.dovecot.org/dovecot-2.1/rev/584bd77c38fd does not. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From tss at iki.fi Thu Apr 12 10:43:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Apr 2012 10:43:22 +0300 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: <20120412071132.GA2583@nihlus.leuxner.net> References: <20120412071132.GA2583@nihlus.leuxner.net> Message-ID: On 12.4.2012, at 10.11, Thomas Leuxner wrote: > Some change between bf5ae73e9475 and 584bd77c38fd seems to have broken > something in the SSL Handshake. A previously valid server certificate is > deemed invalid by various mail clients. > > http://hg.dovecot.org/dovecot-2.1/rev/bf5ae73e9475 works fine while > http://hg.dovecot.org/dovecot-2.1/rev/584bd77c38fd does not. What kind of a certificate do you have? You have an intermediary cert that exists only in ssl_ca file? I couldn't reproduce this with a test. But anyway, reverted for now: http://hg.dovecot.org/dovecot-2.1/rev/f80f18d0ffa3 Now how do I fix the memory leak then?... From tss at iki.fi Thu Apr 12 10:49:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Apr 2012 10:49:12 +0300 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: References: <20120412071132.GA2583@nihlus.leuxner.net> Message-ID: On 12.4.2012, at 10.43, Timo Sirainen wrote: > On 12.4.2012, at 10.11, Thomas Leuxner wrote: > >> Some change between bf5ae73e9475 and 584bd77c38fd seems to have broken >> something in the SSL Handshake. A previously valid server certificate is >> deemed invalid by various mail clients. >> >> http://hg.dovecot.org/dovecot-2.1/rev/bf5ae73e9475 works fine while >> http://hg.dovecot.org/dovecot-2.1/rev/584bd77c38fd does not. > > What kind of a certificate do you have? You have an intermediary cert that exists only in ssl_ca file? I couldn't reproduce this with a test. But anyway, reverted for now: http://hg.dovecot.org/dovecot-2.1/rev/f80f18d0ffa3 > > Now how do I fix the memory leak then?... http://hg.dovecot.org/dovecot-2.1/rev/85ad4baedd43 ? From tlx at leuxner.net Thu Apr 12 11:16:45 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 12 Apr 2012 10:16:45 +0200 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: References: <20120412071132.GA2583@nihlus.leuxner.net> Message-ID: <20120412081645.GB2583@nihlus.leuxner.net> On Thu, Apr 12, 2012 at 10:43:22AM +0300, Timo Sirainen wrote: > What kind of a certificate do you have? You have an intermediary cert that exists only in ssl_ca file? I couldn't reproduce this with a test. But anyway, reverted for now: http://hg.dovecot.org/dovecot-2.1/rev/f80f18d0ffa3 > Thawte. They only do intermediates for some time now. $ openssl x509 -in /etc/ssl/certs/spectre_leuxner_net_2011.crt -noout -subject -issuer -dates subject= /O=spectre.leuxner.net/OU=Go to https://www.thawte.com/repository/index.html/OU=Thawte SSL123 certificate/OU=Domain Validated/CN=spectre.leuxner.net issuer= /C=US/O=Thawte, Inc./OU=Domain Validated SSL/CN=Thawte DV SSL CA notBefore=May 16 00:00:00 2011 GMT notAfter=Jun 14 23:59:59 2012 GMT [...] ssl_ca = From tss at iki.fi Thu Apr 12 11:17:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Apr 2012 11:17:50 +0300 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: <20120412081645.GB2583@nihlus.leuxner.net> References: <20120412071132.GA2583@nihlus.leuxner.net> <20120412081645.GB2583@nihlus.leuxner.net> Message-ID: <11B240E5-087D-43A2-A687-633AB3C4A367@iki.fi> On 12.4.2012, at 11.16, Thomas Leuxner wrote: > On Thu, Apr 12, 2012 at 10:43:22AM +0300, Timo Sirainen wrote: >> What kind of a certificate do you have? You have an intermediary cert that exists only in ssl_ca file? I couldn't reproduce this with a test. But anyway, reverted for now: http://hg.dovecot.org/dovecot-2.1/rev/f80f18d0ffa3 >> > > Thawte. They only do intermediates for some time now. But do you keep your intermediate cert in ssl_ca file or ssl_cert file? From tlx at leuxner.net Thu Apr 12 11:33:58 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 12 Apr 2012 10:33:58 +0200 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: <11B240E5-087D-43A2-A687-633AB3C4A367@iki.fi> References: <20120412071132.GA2583@nihlus.leuxner.net> <20120412081645.GB2583@nihlus.leuxner.net> <11B240E5-087D-43A2-A687-633AB3C4A367@iki.fi> Message-ID: <20120412083358.GA14349@nihlus.leuxner.net> On Thu, Apr 12, 2012 at 11:17:50AM +0300, Timo Sirainen wrote: > But do you keep your intermediate cert in ssl_ca file or ssl_cert file? Separate. Root and intermediate are in ssl_ca: $ cat /etc/ssl/certs/SSL123_CA_Bundle.pem -----BEGIN CERTIFICATE----- MIIEjzCCA3egAwIBAgIQdhASihe2grs6H50amjXAkjANBgkqhkiG9w0BAQUFADCB qTELMAkGA1UEBhMCVVMxFTATBgNVBAoTDHRoYXd0ZSwgSW5jLjEoMCYGA1UECxMf Q2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjE4MDYGA1UECxMvKGMpIDIw MDYgdGhhd3RlLCBJbmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxHzAdBgNV BAMTFnRoYXd0ZSBQcmltYXJ5IFJvb3QgQ0EwHhcNMTAwMjE4MDAwMDAwWhcNMjAw MjE3MjM1OTU5WjBeMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMVGhhd3RlLCBJbmMu MR0wGwYDVQQLExREb21haW4gVmFsaWRhdGVkIFNTTDEZMBcGA1UEAxMQVGhhd3Rl IERWIFNTTCBDQTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMuYyTY/ 0pzYFgfUSWP5g7DoAi3MXFp0l6YT7xMT3gV8p+bKACPaOfnvE89Sxa+a48q+84LZ iz2q4cyuiFBmoy3sYRR1SasOJPGsRFsLKKIzIHYeBmBqZwVxi7pmYhZ6s20Nx9CU QMaMPR6SDGI0DUSJ1feJ/intGI/2mysI92qr2EiXWvSf7Qx1UiL31V6EAJ/ASg0x d0xk0BLmDzrwocDVXB3nXy3C99Y2GNmVbkROyVgUTbaOu83eYh76W7W9GCuYrKyT P1Ba9RQLos+2855PWs1awzYj2hqvsE3WSiIDj0MCGb3qrN3EejUyFPFyLghVQAz0 B0FBrzg3hClCslUCAwEAAaOB/DCB+TAyBggrBgEFBQcBAQQmMCQwIgYIKwYBBQUH MAGGFmh0dHA6Ly9vY3NwLnRoYXd0ZS5jb20wEgYDVR0TAQH/BAgwBgEB/wIBADA0 BgNVHR8ELTArMCmgJ6AlhiNodHRwOi8vY3JsLnRoYXd0ZS5jb20vVGhhd3RlUENB LmNybDAOBgNVHQ8BAf8EBAMCAQYwKQYDVR0RBCIwIKQeMBwxGjAYBgNVBAMTEVZl cmlTaWduTVBLSS0yLTExMB0GA1UdDgQWBBSrRORd7IPH2cCFn/fhxpeQsIw/mDAf BgNVHSMEGDAWgBR7W0XPr87Lev0xkhpqtvNG61dIUDANBgkqhkiG9w0BAQUFAAOC AQEABLr7rLv8S1QRoy2Iszy9AG2KGraNxMGD+MdTKsEybjqBoVR92ho/OkVPNudC sApChZegrPvlh6eDT+ixt5tYZW4mgAuSTUdVuWEWUWXpK/Fo2Vi4A4HRt2Yc07zF pntfPsU4RnbndbSgDEvOosKpwcw2c3v7uSQkoF6n9vq7DChDnh3wTvA/2CSwIdxt Le6/Wjv6iJx0bK8h3ZLswxXvlHUmRtamP79mSKod790n5rdRiTh9E4QMQPzQtfHg 2/lPL0ActI5HImG4TJbe8F8Rfk8R2exQRyIOxR3iZEnnaGNFOorZcfRe8W63FE0+ bxQe3FL+vN8MvSk/dvsRX2hoFQ== -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- MIIERTCCA66gAwIBAgIQM2VQCHmtc+IwueAdDX+skTANBgkqhkiG9w0BAQUFADCB zjELMAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJ Q2FwZSBUb3duMR0wGwYDVQQKExRUaGF3dGUgQ29uc3VsdGluZyBjYzEoMCYGA1UE CxMfQ2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjEhMB8GA1UEAxMYVGhh d3RlIFByZW1pdW0gU2VydmVyIENBMSgwJgYJKoZIhvcNAQkBFhlwcmVtaXVtLXNl cnZlckB0aGF3dGUuY29tMB4XDTA2MTExNzAwMDAwMFoXDTIwMTIzMDIzNTk1OVow gakxCzAJBgNVBAYTAlVTMRUwEwYDVQQKEwx0aGF3dGUsIEluYy4xKDAmBgNVBAsT H0NlcnRpZmljYXRpb24gU2VydmljZXMgRGl2aXNpb24xODA2BgNVBAsTLyhjKSAy MDA2IHRoYXd0ZSwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MR8wHQYD VQQDExZ0aGF3dGUgUHJpbWFyeSBSb290IENBMIIBIjANBgkqhkiG9w0BAQEFAAOC AQ8AMIIBCgKCAQEArKDw+4BZ1JzHpM+doVlzCRBFDA0sbmjxbFtIaElZN/wLMxnC d3/MEC2VNBzm600JpxzSuMmXNgK3idQkXwbAzESUlI0CYm/rWt0RjSiaXISQEHoN vXRmL2o4oOLVVETrHQefB7pv7un9Tgsp9T6EoAHxnKv4HH6JpOih2HFlDaNRe+68 0iJgDblbnd+6/FFbC6+Ysuku6QToYofeK8jXTsFMZB7dz4dYukpPymgHHRydSsbV L5HMfHFyHMXAZ+sy/cmSXJTahcCbv1N9Kwn0jJ2RH5dqUsveCTakd9h7h1BE1T5u KWn7OUkmHgmlgHtALevoJ4XJ/mH9fuZ8lx3VnQIDAQABo4HCMIG/MA8GA1UdEwEB /wQFMAMBAf8wOwYDVR0gBDQwMjAwBgRVHSAAMCgwJgYIKwYBBQUHAgEWGmh0dHBz Oi8vd3d3LnRoYXd0ZS5jb20vY3BzMA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQU e1tFz6/Oy3r9MZIaarbzRutXSFAwQAYDVR0fBDkwNzA1oDOgMYYvaHR0cDovL2Ny bC50aGF3dGUuY29tL1RoYXd0ZVByZW1pdW1TZXJ2ZXJDQS5jcmwwDQYJKoZIhvcN AQEFBQADgYEAhKhMyT4qvJrizI8LsiV3xGGJiWNa1KMVQNT7Xj+0Q+pjFytrmXSe Cajd1FYVLnp5MV9jllMbNNkV6k9tcMq+9oKp7dqFd8x2HGqBCiHYQZl/Xi6Cweiq 95OBBaqStB+3msAHF/XLxrRMDtdW3HEgdDjWdMbWj2uvi42gbCkLYeA= -----END CERTIFICATE----- $ dovecot --version 2.1.4 (584bd77c38fd) Seems to have fixed it. Thanks. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From tss at iki.fi Thu Apr 12 11:35:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Apr 2012 11:35:48 +0300 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: <20120412083358.GA14349@nihlus.leuxner.net> References: <20120412071132.GA2583@nihlus.leuxner.net> <20120412081645.GB2583@nihlus.leuxner.net> <11B240E5-087D-43A2-A687-633AB3C4A367@iki.fi> <20120412083358.GA14349@nihlus.leuxner.net> Message-ID: On 12.4.2012, at 11.33, Thomas Leuxner wrote: > On Thu, Apr 12, 2012 at 11:17:50AM +0300, Timo Sirainen wrote: >> But do you keep your intermediate cert in ssl_ca file or ssl_cert file? > > Separate. Root and intermediate are in ssl_ca: The documentation tells to put the intermediary to ssl_cert though. I didn't even know it worked in ssl_ca. But I guess I won't intentionally break it.. From tlx at leuxner.net Thu Apr 12 11:41:52 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 12 Apr 2012 10:41:52 +0200 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: References: <20120412071132.GA2583@nihlus.leuxner.net> <20120412081645.GB2583@nihlus.leuxner.net> <11B240E5-087D-43A2-A687-633AB3C4A367@iki.fi> <20120412083358.GA14349@nihlus.leuxner.net> Message-ID: <20120412084152.GB14349@nihlus.leuxner.net> On Thu, Apr 12, 2012 at 11:35:48AM +0300, Timo Sirainen wrote: > On 12.4.2012, at 11.33, Thomas Leuxner wrote: > > > On Thu, Apr 12, 2012 at 11:17:50AM +0300, Timo Sirainen wrote: > >> But do you keep your intermediate cert in ssl_ca file or ssl_cert file? > > > > Separate. Root and intermediate are in ssl_ca: > > The documentation tells to put the intermediary to ssl_cert though. I didn't even know it worked in ssl_ca. But I guess I won't intentionally break it.. Hmmm. I did emulate Thawte instructions though: https://search.thawte.com/support/ssl-digital-certificates/index?page=content&id=SO15464&actp=LIST&viewlocale=en_US https://search.thawte.com/library/VERISIGN/ALL_OTHER/thawte%20ca/SSL123_CA_Bundle.pem [...] SSLCertificateFile /usr/local/ssl/crt/domainname.crt SSLCertificateKeyFile /usr/local/ssl/private/server.key SSLCACertificateFile /usr/local/ssl/crt/cabundle.crt -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From stan at hardwarefreak.com Thu Apr 12 13:20:31 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 12 Apr 2012 05:20:31 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> Message-ID: <4F86AC6F.4070304@hardwarefreak.com> On 4/11/2012 9:23 PM, Emmanuel Noobadmin wrote: > On 4/12/12, Stan Hoeppner wrote: >> On 4/11/2012 11:50 AM, Ed W wrote: >>> One of the snags of md RAID1 vs RAID6 is the lack of checksumming in the >>> event of bad blocks. (I'm not sure what actually happens when md >>> scrubbing finds a bad sector with raid1..?). For low performance >>> requirements I have become paranoid and been using RAID6 vs RAID10, >>> filesystems with sector checksums seem attractive... >> >> Except we're using hardware RAID1 here and mdraid linear. Thus the >> controller takes care of sector integrity. RAID6 yields nothing over >> RAID10, except lower performance, and more usable space if more than 4 >> drives are used. > > How would the control ensure sector integrity unless it is writing > additional checksum information to disk? I thought only a few > filesystems like ZFS does the sector checksum to detect if any data > corruption occurred. I suppose the controller could throw an error if > the two drives returned data that didn't agree with each other but it > wouldn't know which is the accurate copy but that wouldn't protect the > integrity of the data, at least not directly without additional human > intervention I would think. When a drive starts throwing uncorrectable read errors, the controller faults the drive and tells you to replace it. Good hardware RAID controllers are notorious for their penchant to kick drives that would continue to work just fine in mdraid or as a single drive for many more years. The mindset here is that anyone would rather spent $150-$2500 dollars on a replacement drive than take a chance with his/her valuable data. Yes I typed $2500. EMC charges over $2000 for a single Seagate disk drive with an EMC label and serial# on it. The serial number is what prevents one from taking the same off the shelf Seagate drive at $300 and mounting it in a $250,000 EMC array chassis. The controller firmware reads the S/N from each connected drive and will not allow foreign drives to be used. HP, IBM, Oracle/Sun, etc do this as well. Which is why they make lots of profit, and is why I prefer open storage systems. -- Stan From lists at wildgooses.com Thu Apr 12 13:58:52 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 12 Apr 2012 11:58:52 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F86AC6F.4070304@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> Message-ID: <4F86B56C.6010201@wildgooses.com> On 12/04/2012 11:20, Stan Hoeppner wrote: > On 4/11/2012 9:23 PM, Emmanuel Noobadmin wrote: >> On 4/12/12, Stan Hoeppner wrote: >>> On 4/11/2012 11:50 AM, Ed W wrote: >>>> One of the snags of md RAID1 vs RAID6 is the lack of checksumming in the >>>> event of bad blocks. (I'm not sure what actually happens when md >>>> scrubbing finds a bad sector with raid1..?). For low performance >>>> requirements I have become paranoid and been using RAID6 vs RAID10, >>>> filesystems with sector checksums seem attractive... >>> Except we're using hardware RAID1 here and mdraid linear. Thus the >>> controller takes care of sector integrity. RAID6 yields nothing over >>> RAID10, except lower performance, and more usable space if more than 4 >>> drives are used. >> How would the control ensure sector integrity unless it is writing >> additional checksum information to disk? I thought only a few >> filesystems like ZFS does the sector checksum to detect if any data >> corruption occurred. I suppose the controller could throw an error if >> the two drives returned data that didn't agree with each other but it >> wouldn't know which is the accurate copy but that wouldn't protect the >> integrity of the data, at least not directly without additional human >> intervention I would think. > When a drive starts throwing uncorrectable read errors, the controller > faults the drive and tells you to replace it. Good hardware RAID > controllers are notorious for their penchant to kick drives that would > continue to work just fine in mdraid or as a single drive for many more > years. The mindset here is that anyone would rather spent $150-$2500 > dollars on a replacement drive than take a chance with his/her valuable > data. > I'm asking a subtlely different question. The claim by ZFS/BTRFS authors and others is that data silently "bit rots" on it's own. The claim is therefore that you can have a raid1 pair where neither drive reports a hardware failure, but each gives you different data? I can't personally claim to have observed this, so it remains someone else's theory... (for background my experience is simply: RAID10 for high performance arrays and RAID6 for all my personal data - I intend to investigate your linear raid idea in the future though) I do agree that if one drive reports a read error, then it's quite easy to guess which pair of the array is wrong... Just as an aside, I don't have a lot of failure experience. However, the few I have had (perhaps 6-8 events now) is that there is a massive correlation in failure time with RAID1, eg one pair I had lasted perhaps 2 years and then both failed within 6 hours of each other. I also had a bad experience with RAID 5 that wasn't being scrubbed regularly and when one drive started reporting errors (ie lack of monitoring meant it had been bad for a while), the rest of the array turned out to be a patchwork of read errors - linux raid then turns out to be quite fragile in the presence of a small number of read failures and it's extremely difficult to salvage the 99% of the array which is ok due to the disks getting kicked out... (of course regular scrubs would have prevented getting so deep into that situation - it was a small cheap nas box without such features) Ed W From tss at iki.fi Thu Apr 12 14:09:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Apr 2012 14:09:31 +0300 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F86B56C.6010201@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> Message-ID: <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> On 12.4.2012, at 13.58, Ed W wrote: > The claim by ZFS/BTRFS authors and others is that data silently "bit rots" on it's own. The claim is therefore that you can have a raid1 pair where neither drive reports a hardware failure, but each gives you different data? That's one reason why I planned on adding a checksum to each message in dbox. But I forgot to actually do that. I guess I could add it for new messages in some upcoming version. Then Dovecot could optionally verify the checksum before returning the message to client, and if it detects corruption perhaps automatically read it from some alternative location (e.g. if dsync replication is enabled ask from another replica). And Dovecot index files really should have had some small (8/16/32bit) checksums of stuff as well.. From lists at wildgooses.com Thu Apr 12 14:45:51 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 12 Apr 2012 12:45:51 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F862D50.8000206@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> Message-ID: <4F86C06F.3060109@wildgooses.com> On 12/04/2012 02:18, Stan Hoeppner wrote: > On 4/11/2012 11:50 AM, Ed W wrote: >> Re XFS. Have you been watching BTRFS recently? >> >> I will concede that despite the authors considering it production ready >> I won't be using it for my servers just yet. However, it's benchmarking >> on single disk benchmarks fairly similarly to XFS and in certain cases >> (multi-threaded performance) can be somewhat better. I haven't yet seen >> any benchmarks on larger disk arrays yet, eg 6+ disks, so no idea how it >> scales up. Basically what I have seen seems "competitive" > Links? http://btrfs.ipv5.de/index.php?title=Main_Page#Benchmarking See the regular Phoronix benchmarks in particular. However, I believe these are all single disk? >> I don't have such hardware spare to benchmark, but I would be interested >> to hear from someone who benchmarks your RAID1+linear+XFS suggestion, >> especially if they have compared a cutting edge btrfs kernel on the same >> array? > http://btrfs.boxacle.net/repository/raid/history/History_Mail_server_simulation._num_threads=128.html > > This is with an 8 wide LVM stripe over 8 17 drive hardware RAID0 arrays. > If the disks had been setup as a concat of 68 RAID1 pairs, XFS would > have turned in numbers significantly higher, anywhere from a 100% > increase to 500%. My instinct is that this is an irrelevant benchmark for BTRFS because its performance characteristics for these workloads have changed so significantly? I would be far more interested in a 3.2 and then a 3.6/3.7 benchmark in a years time In particular recent benchmarks on Phoronix show btrfs exceeding XFS performance on heavily threaded benchmarks - however, I doubt this is representative of performance on a multi-disk benchmark? > It would be nice to see these folks update these > results with a 3.2.6 kernel, as both BTRFS and XFS have improved > significantly since 2.6.35. EXT4 and JFS have seen little performance > work since. My understanding is that there was a significant multi-thread performance boost for EXT4 in the last year kind of timeframe? I don't have a link to hand, but someone did some work to reduce lock contention (??) which I seem to recall made a very large difference on multi-user or multi-cpu workloads? I seem to recall that the summary was that it allowed Ext4 to scale up to a good fraction of XFS performance on "medium sized" systems? (I believe that XFS still continues to scale far better than anything else on large systems) Point is that I think it's a bit unfair to say that little has changed on Ext4? It still seems to be developing faster than "maintenance only" However, well OT... The original question was: anyone tried very recent BTRFS on a multi-disk system. Seems like the answer is no. My proposal is that it may be worth watching in the future Cheers Ed W P.S. I have always been intrigued by the idea that a COW based filesystem could potentially implement much faster "RAID" parity, because it can avoid reading the whole stripe. The idea is that you treat unallocated space as "zero", which means you can compute the incremental parity with only a read/write of the checksum value (and with a COW filesystem you only ever update by rewriting to new "zero'd" space). I had in mind something like a fixed parity disk (RAID4?) and allowing the parity disk to be "write behind" cached in ram (ie exposed to risk of: power fails AND data disk fails at the same time). My code may not be following along for a while though... From zybi at talex.pl Thu Apr 12 14:47:49 2012 From: zybi at talex.pl (=?ISO-8859-2?Q?Artur_Zaprza=B3a?=) Date: Thu, 12 Apr 2012 13:47:49 +0200 Subject: [Dovecot] "doveadm mailbox status" destroys compressed messages without W= attribute Message-ID: <4F86C0E5.6090709@talex.pl> I still have some old messages from previous mail server. This messages have no W= attribute either in file name or dovecot-uidlist and are compressed with gzip. Running "doveadm mailbox status -A vsize \*" will result in the following messages: doveadm(foo at domain): Error: Cached message size larger than expected (2580 > 1451) doveadm(foo at domain): Error: Maildir filename has wrong S value, renamed the file from /vmail/domain/foo/Maildir/.Sent/cur/1206550323.M125837P13306V0000000000000812I0000809E.oldname,S=2580:2,SZ to /vmail/domain/foo/Maildir/.Sent/cur/1206550323.M125837P13306V0000000000000812I0000809E.oldname,S=1451:2,SZ doveadm(foo at domain): Error: Corrupted index cache file /vmail/domain/foo/Maildir/.Sent/dovecot.index.cache: Broken physical size for mail UID 2 doveadm(foo at domain): Error: Cached message size larger than expected (2580 > 1451) doveadm(foo at domain): Error: Corrupted index cache file /vmail/domain/foo/Maildir/.Sent/dovecot.index.cache: Broken physical size for mail UID 2 doveadm(foo at domain): Error: read(/vmail/domain/foo/Maildir/.Sent/cur/1206550323.M125837P13306V0000000000000812I0000809E.oldname,S=2580:2,SZ) failed: Input/output error (uid=2) (Size of uncompressed message is 2580 and compressed size is 1451) I have enabled zlib plugin for imap, pop3, lda and lmtp. But how to enable it for doveadm? -- Best regards, Artur Zaprza?a From tss at iki.fi Thu Apr 12 14:49:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Apr 2012 14:49:44 +0300 Subject: [Dovecot] "doveadm mailbox status" destroys compressed messages without W= attribute In-Reply-To: <4F86C0E5.6090709@talex.pl> References: <4F86C0E5.6090709@talex.pl> Message-ID: <5CA9296A-87CB-4F47-AB1B-67533F866248@iki.fi> On 12.4.2012, at 14.47, Artur Zaprza?a wrote: > I have enabled zlib plugin for imap, pop3, lda and lmtp. But how to enable it for doveadm? Just set it globally: mail_plugins = zlib From lists at wildgooses.com Thu Apr 12 15:10:20 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 12 Apr 2012 13:10:20 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> Message-ID: <4F86C62C.8030305@wildgooses.com> On 12/04/2012 12:09, Timo Sirainen wrote: > On 12.4.2012, at 13.58, Ed W wrote: > >> The claim by ZFS/BTRFS authors and others is that data silently "bit rots" on it's own. The claim is therefore that you can have a raid1 pair where neither drive reports a hardware failure, but each gives you different data? > That's one reason why I planned on adding a checksum to each message in dbox. But I forgot to actually do that. I guess I could add it for new messages in some upcoming version. Then Dovecot could optionally verify the checksum before returning the message to client, and if it detects corruption perhaps automatically read it from some alternative location (e.g. if dsync replication is enabled ask from another replica). And Dovecot index files really should have had some small (8/16/32bit) checksums of stuff as well.. > I have to say - I haven't actually seen this happen... Do any of your big mailstore contacts observe this, eg rackspace, etc? I think it's worth thinking about the failure cases before implementing something to be honest? Just sticking in a checksum possibly doesn't help anyone unless it's on the right stuff and in the right place? Off the top of my head: - Someone butchers the file on disk (disk error or someone edits it with vi) - Restore of some files goes subtly wrong, eg tool tries to be clever and fails, snapshot taken mid-write, etc? - Filesystem crash (sudden power loss), how to deal with partial writes? Things I might like to do *if* there were some suitable "checksums" available: - Use the checksum as some kind of guid either for the whole message, the message minus the headers, or individual mime sections - Use the checksums to assist with replication speed/efficiency (dsync or custom imap commands) - File RFCs for new imap features along the "lemonde" lines which allow clients to have faster recovery from corrupted offline states... - Single instance storage (presumably already done, and of course this has some subtleties in the face of deliberate attack) - Possibly duplicate email suppression (but really this is an LDA problem...) - Storage backends where emails are redundantly stored and might not ALL be on a single server (find me the closest copy of email X) - derivations of this might be interesting for compliance archiving of messages? - Fancy key-value storage backends might use checksums as part of the key value (either for the whole or parts of the message) The mail server has always looked like a kind of key-value store to my eye. However, traditional key-value isn't usually optimised for "streaming reads", hence dovecot seems like a "key value store, optimised for sequential high speed streaming access to the key values"... Whilst it seems increasingly unlikely that a traditional key-value store will work well to replace say mdbox, I wonder if it's not worth looking at the replication strategies of key-value stores to see if those ideas couldn't lead to new features for mdbox? Cheers Ed W From dlie76 at yahoo.com.au Thu Apr 12 16:27:38 2012 From: dlie76 at yahoo.com.au (Daminto Lie) Date: Thu, 12 Apr 2012 06:27:38 -0700 (PDT) Subject: [Dovecot] vacation plugins for squirrelmail Message-ID: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> Hi, I am afraid I have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. Any help would be very much appreciated. Thank you From zybi at talex.pl Thu Apr 12 16:47:41 2012 From: zybi at talex.pl (=?UTF-8?B?QXJ0dXIgWmFwcnphxYJh?=) Date: Thu, 12 Apr 2012 15:47:41 +0200 Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> Message-ID: <4F86DCFD.5020007@talex.pl> Daminto Lie wrote: > Hi, > > I am afraid I have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. > > Any help would be very much appreciated. > > Thank you > I'm using Avelsieve 1.9.9 with a set of my own bugfixes: http://email.uoa.gr/avelsieve/ From dirk.jahnke-zumbusch at desy.de Thu Apr 12 17:08:31 2012 From: dirk.jahnke-zumbusch at desy.de (Dirk Jahnke-Zumbusch) Date: Thu, 12 Apr 2012 16:08:31 +0200 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F86C62C.8030305@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> <4F86C62C.8030305@wildgooses.com> Message-ID: <4F86E1DF.30800@desy.de> Hi there, > I have to say - I haven't actually seen this happen... Do any of your > big mailstore contacts observe this, eg rackspace, etc? Just to throw in to the discussion that with (silent) data corruption not only "the disk" is involved but many other parts of your systems. So perhaps you would like to have a look at https://indico.desy.de/getFile.py/access?contribId=65&sessionId=42&resId=0&materialId=slides&confId=257 http://indico.cern.ch/getFile.py/access?contribId=3&sessionId=0&resId=1&materialId=paper&confId=13797 The documents are from 2007 but the principals are still the same. Kind regards Dirk From helga.mayer at uni-hohenheim.de Thu Apr 12 18:31:35 2012 From: helga.mayer at uni-hohenheim.de (Helga Mayer) Date: Thu, 12 Apr 2012 17:31:35 +0200 Subject: [Dovecot] Problems with Apple Mail and attachments In-Reply-To: References: <20120411161321.165760epurnu7i4h@webmail.uni-hohenheim.de> Message-ID: <20120412173135.10654u9eo7r2pumv@webmail.uni-hohenheim.de> Quoting Thierry de Montaudry : > I've seen a similar problem a while ago (1 year maybe more), but > used the mailbox Rebuild option on the client, which fixed it > without having to delete and recreate the account. Your problem > might just be a local index corruption, which can happen when > loosing your Internet connection. Thank you, we will will try it. Regards Helga Helga Mayer Universit?t Hohenheim Kommunikations-, Informations- und Medienzentrum (630) IT-Dienste | Mail Schloss-Westhof-S?d | 70599 Stuttgart Tel.: +49 711 459-22838 | Fax: +49 711 459-23449 https://kim.uni-hohenheim.de From am at am.cx Thu Apr 12 18:35:15 2012 From: am at am.cx (Andrea Mistrali) Date: Thu, 12 Apr 2012 17:35:15 +0200 Subject: [Dovecot] Problems with master user Message-ID: Hi to all! I?m trying to setup master users, but I have some problems. Namely I can authenticate, but after it I cannot access INBOX or other mailboxes of the user. My configuration is: passdb { driver = ldap args = /etc/dovecot/ldap-passdb.conf } passdb { driver = sql args = /etc/dovecot/sql.conf } passdb { driver = passwd-file args = /etc/dovecot/passwd.masterusers master = yes pass = yes } userdb { driver = sql args = /etc/dovecot/sql.conf } (I look up for auth in LDAP server first, if it fails I look up in DB, else I check for master user) and relevant files are /etc/dovecot/sql.conf ?????????? password_query = SELECT fullusername as user, \ password, \ uid AS userdb_uid, \ gid AS userdb_gid, \ home AS userdb_home, \ mail AS userdb_mail, \ groups as userdb_acl_groups, \ quota_rule as userdb_quota_rule \ FROM pd_users_full WHERE \ username = '%n' AND \ domain = '%d' AND \ external_auth IS FALSE AND \ master_user IS FALSE AND \ %Ls_ok IS TRUE user_query = SELECT fullusername as user, \ uid, \ gid, \ home, \ mail, \ groups as acl_groups, \ quota_rule \ FROM pd_users_full WHERE \ username = '%n' AND \ domain = '%d' AND \ master_user IS FALSE iterate_query = SELECT fullusername as username, fullusername as user \ FROM pd_users_full where master_user IS FALSE ORDER BY domain,username /etc/dovecot/ldap-passdb.conf ?????????????? uris = ldap://dioniso.cube.lan base = cn=users,dc=cube,dc=lan auth_bind = yes auth_bind_userdn = uid=%n,cn=users,dc=cube,dc=lan pass_attrs = uid=username, \ userPassword=password, \ # uidNumber=userdb_uid, \ # =userdb_home=/var/mail/cubeholding.com/%Lu, \ # =userdb_domain=cubeholding.com, \ # =userdb_mail=maildir:~/maildir/:INBOX=~/maildir/INBOX:LAYOUT=fs:INDEX=~/indexes/ pass_filter = (&(objectClass=posixAccount)(uid=%n)(mail=*@%d)) # Attributes and filter to get a list of all users # iterate_attrs = uid=username iterate_attrs = uid=user iterate_filter = (&(objectClass=posixAccount)(mail=*@%d)) If I test with doveadm auth and doveadm user I receive this: # doveadm auth -x service=imap andre at am.cx\*master at am.cx XXXXXX passdb: andre at am.cx*master at am.cx auth succeeded extra fields: user=andre at am.cx # doveadm user andre at am.cx userdb: andre at am.cx uid : 10010 gid : 8 home : /var/mail/am.cx/andre mail : maildir:~/maildir:INBOX=~/maildir/INBOX:LAYOUT=fs:INDEX=~/indexes/ acl_groups: quota_rule: *:storage=10G and in log files I see: 20120412 17:31:26 auth: Info: passdb(master at am.cx,master): Master user logging in as andre at am.cx 20120412 17:31:26 auth: Info: ldap(andre at am.cx): invalid credentials (given password: XXXXXX) but if I try the real thing: # telnet localhost 143 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE ACL QUOTA NAMESPACE STARTTLS AUTH=PLAIN AUTH=LOGIN AUTH=GSSAPI] Dovecot ready. 0 login andre at am.cx*master at am.cx XXXXXX 0 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE ACL QUOTA NAMESPACE COMPRESS=DEFLATE QUOTA ACL RIGHTS=texk] Logged in 0 select INBOX 0 NO [SERVERBUG] Internal error occurred. Refer to server log for more information. [2012-04-12 17:33:15] and in log file I have: 20120412 17:34:25 auth: Info: passdb(master at am.cx,127.0.0.1,master): Master user logging in as andre at am.cx 20120412 17:34:25 auth: Info: ldap(andre at am.cx,127.0.0.1): invalid credentials (given password: silmaril) 20120412 17:34:25 imap-login: Info: Login: pid=1673, , 127.0.0.1/127.0.0.1, PLAIN, secured 20120412 17:34:47 imap(andre at am.cx): Error: Opening INBOX failed: Mailbox doesn't exist: INBOX Can someone tell me what is wrong in my setup? TIA A. From am at am.cx Thu Apr 12 19:11:36 2012 From: am at am.cx (Andrea Mistrali) Date: Thu, 12 Apr 2012 18:11:36 +0200 Subject: [Dovecot] Problems with master user In-Reply-To: <533C3A4E-2E3A-4B95-84DB-98A4903BBD2E@am.cx> References: <533C3A4E-2E3A-4B95-84DB-98A4903BBD2E@am.cx> Message-ID: Il giorno 12/apr/2012, alle ore 17.35, Andrea Mistrali ha scritto: > Hi to all! > I?m trying to setup master users, but I have some problems. Namely I can authenticate, but after it I cannot access INBOX or other mailboxes of the user. > snip > Can someone tell me what is wrong in my setup? Solved! It is a problem of ACL as stated at http://master.wiki2.dovecot.org/Authentication/MasterUsers#ACLs Sorry A. From root.kev at gmail.com Thu Apr 12 23:48:12 2012 From: root.kev at gmail.com (Root Kev) Date: Thu, 12 Apr 2012 16:48:12 -0400 Subject: [Dovecot] POP3 Dovecot Auth CPU usage 75%+ In-Reply-To: References: Message-ID: Hello all, I hope someone can help me, I have been testing out Dovecot to switch from popa3d which I use at the moment. When I get several users connecting and disconnection multiple times, the Dovecot process with command Auth uses 50-90% of the CPU for the period which they are connecting. I am wondering if there is something that I may have misconfigured, or if there is something that I can change so that this spike doesn't occur. If anyone could shed some light on the issue, I would appreciate it, Kevin /var/mail# dovecot -n # 2.1.4: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-33-generic-pae i686 Ubuntu 10.04.4 LTS ext4 auth_cache_size = 10 M auth_verbose = yes disable_plaintext_auth = no instance_name = Mail Popper 1 listen = 172.20.20.222 login_greeting = Mail Popper 1 Ready mail_location = mbox:/var/empty:INBOX=/var/mail/%u:INDEX=MEMORY mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = shadow } protocols = pop3 service pop3-login { service_count = 0 } ssl = no userdb { driver = passwd } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } From root.kev at gmail.com Thu Apr 12 23:46:46 2012 From: root.kev at gmail.com (Root Kev) Date: Thu, 12 Apr 2012 16:46:46 -0400 Subject: [Dovecot] POP3 Dovecot Auth CPU usage 75%+ Message-ID: Hello all, I hope someone can help me, I have been testing out Dovecot to switch from popa3d which I use at the moment. When I get several users connecting and disconnection multiple times, the Dovecot process with command Auth uses 50-90% of the CPU for the period which they are connecting. I am wondering if there is something that I may have misconfigured, or if there is something that I can change so that this spike doesn't occur. If anyone could shed some light on the issue, I would appreciate it, Kevin /var/mail# dovecot -n # 2.1.4: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-33-generic-pae i686 Ubuntu 10.04.4 LTS ext4 auth_cache_size = 10 M auth_verbose = yes disable_plaintext_auth = no instance_name = Mail Popper 1 listen = 172.20.20.222 login_greeting = Mail Popper 1 Ready mail_location = mbox:/var/empty:INBOX=/var/mail/%u:INDEX=MEMORY mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = shadow } protocols = pop3 service pop3-login { service_count = 0 } ssl = no userdb { driver = passwd } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } From michael at orlitzky.com Fri Apr 13 02:33:22 2012 From: michael at orlitzky.com (Michael Orlitzky) Date: Thu, 12 Apr 2012 19:33:22 -0400 Subject: [Dovecot] [OT] Outlook identities Message-ID: <4F876642.1040701@orlitzky.com> Nothing to do with Dovecot, but I figured this is the best place to ask. Do any of the newer versions of Outlook have proper identities support like Thunderbird, mutt, Roundcube, i.e. every other mail client on Earth? We have customers who set up ten different mailboxes for one person because otherwise Outlook won't Do the Right Thing. Is there some way to make it behave like the others? * When sending new mail, you can choose which address to use. * When replying to mail, it sends from the address that the message was sent to by default. * All mail winds up in one inbox. Outlook (2003, 2007) does do this if you set up different mail accounts, but we shouldn't have to do that. From stan at hardwarefreak.com Fri Apr 13 08:29:52 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 13 Apr 2012 00:29:52 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F86B56C.6010201@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> Message-ID: <4F87B9D0.5010300@hardwarefreak.com> On 4/12/2012 5:58 AM, Ed W wrote: > The claim by ZFS/BTRFS authors and others is that data silently "bit > rots" on it's own. The claim is therefore that you can have a raid1 pair > where neither drive reports a hardware failure, but each gives you > different data? You need to read those articles again very carefully. If you don't understand what they mean by "1 in 10^15 bits non-recoverable read error rate" and combined probability, let me know. And this has zero bearing on RAID1. And RAID1 reads don't work the way you describe above. I explained this in some detail recently. > I do agree that if one drive reports a read error, then it's quite easy > to guess which pair of the array is wrong... Been working that way for more than 2 decades Ed. :) Note that "RAID1" has that "1" for a reason. It was the first RAID level. It was in production for many many years before parity RAID hit the market. It is the most well understood of all RAID levels, and the simplest. -- Stan From centos.admin at gmail.com Fri Apr 13 09:12:48 2012 From: centos.admin at gmail.com (Emmanuel Noobadmin) Date: Fri, 13 Apr 2012 14:12:48 +0800 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F86AC6F.4070304@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> Message-ID: On 4/12/12, Stan Hoeppner wrote: > On 4/11/2012 9:23 PM, Emmanuel Noobadmin wrote: > I suppose the controller could throw an error if >> the two drives returned data that didn't agree with each other but it >> wouldn't know which is the accurate copy but that wouldn't protect the >> integrity of the data, at least not directly without additional human >> intervention I would think. > > When a drive starts throwing uncorrectable read errors, the controller > faults the drive and tells you to replace it. Good hardware RAID > controllers are notorious for their penchant to kick drives that would > continue to work just fine in mdraid or as a single drive for many more > years. What I meant wasn't the drive throwing uncorrectable read errors but the drives are returning different data that each think is correct or both may have sent the correct data but one of the set got corrupted on the fly. After reading the articles posted, maybe the correct term would be the controller receiving silently corrupted data, say due to bad cable on one. If the controller simply returns the fastest result, it could be the bad sector and that doesn't protect the integrity of the data right? if the controller gets 1st half from one drive and 2nd half from the other drive to speed up performance, we could still get the corrupted half and the controller itself still can't tell if the sector it got was corrupted isn't it? If the controller compares the two sectors from the drives, it may be able to tell us something is wrong but there isn't anyway for it to know which one of the sector was a good read and which isn't, or is there? From blevi.linux at gmail.com Fri Apr 13 09:22:34 2012 From: blevi.linux at gmail.com (Birta Levente) Date: Fri, 13 Apr 2012 09:22:34 +0300 Subject: [Dovecot] [OT] Outlook identities In-Reply-To: <4F876642.1040701@orlitzky.com> References: <4F876642.1040701@orlitzky.com> Message-ID: <4F87C62A.8060105@gmail.com> On 13/04/2012 02:33, Michael Orlitzky wrote: > Nothing to do with Dovecot, but I figured this is the best place to ask. > > Do any of the newer versions of Outlook have proper identities support > like Thunderbird, mutt, Roundcube, i.e. every other mail client on Earth? > > We have customers who set up ten different mailboxes for one person > because otherwise Outlook won't Do the Right Thing. Is there some way to > make it behave like the others? > > * When sending new mail, you can choose which address to use. > * When replying to mail, it sends from the address that the message > was sent to by default. > * All mail winds up in one inbox. > > Outlook (2003, 2007) does do this if you set up different mail accounts, > but we shouldn't have to do that. Control Panel / E-mail (Mail in win7) / Profiles ... I hope this help you .... Levi From robert at schetterer.org Fri Apr 13 09:35:53 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 13 Apr 2012 08:35:53 +0200 Subject: [Dovecot] [OT] Outlook identities In-Reply-To: <4F876642.1040701@orlitzky.com> References: <4F876642.1040701@orlitzky.com> Message-ID: <4F87C949.8010405@schetterer.org> Am 13.04.2012 01:33, schrieb Michael Orlitzky: > Nothing to do with Dovecot, but I figured this is the best place to ask. > > Do any of the newer versions of Outlook have proper identities support > like Thunderbird, mutt, Roundcube, i.e. every other mail client on Earth? > > We have customers who set up ten different mailboxes for one person > because otherwise Outlook won't Do the Right Thing. Is there some way to > make it behave like the others? > > * When sending new mail, you can choose which address to use. > * When replying to mail, it sends from the address that the message > was sent to by default. > * All mail winds up in one inbox. > > Outlook (2003, 2007) does do this if you set up different mail accounts, > but we shouldn't have to do that. you can do it with exchange, no wonder outlook isnt a internet mail client in first line its the client of exchange, so people should use real internet mail clients ( TB etc ), if m$ would more be compatible, the need of exchange may more less , but this should not be , as cashmakers should live forever *g http://www.msexchange.org/tutorials/Sending-As.html there are a few tricks to goal with profiles etc, but i saw nothing that was equal to thunderbird identities but there a many lists an payed support for outlook, you may asked there i also looked in my newest outlook version and didnt found something related i found this add on http://www.emailaddressmanager.com/outlook/email_accounts.html -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From am at am.cx Fri Apr 13 10:02:42 2012 From: am at am.cx (Andrea Mistrali) Date: Fri, 13 Apr 2012 09:02:42 +0200 Subject: [Dovecot] [OT] Outlook identities In-Reply-To: <4F876642.1040701@orlitzky.com> References: <4F876642.1040701@orlitzky.com> Message-ID: Il giorno 13/apr/2012, alle ore 01.33, Michael Orlitzky ha scritto: > Nothing to do with Dovecot, but I figured this is the best place to ask. > > Do any of the newer versions of Outlook have proper identities support > like Thunderbird, mutt, Roundcube, i.e. every other mail client on Earth? > > We have customers who set up ten different mailboxes for one person > because otherwise Outlook won't Do the Right Thing. Is there some way to > make it behave like the others? > > * When sending new mail, you can choose which address to use. > * When replying to mail, it sends from the address that the message > was sent to by default. > * All mail winds up in one inbox. > > Outlook (2003, 2007) does do this if you set up different mail accounts, > but we shouldn't have to do that. If it can comfort you even Apple Mail.app ha no support for full identities like Thunderbird and every other good mail client (I think of Eudora, back to 1999, that had full identities support) have, except for some little trick that does not do ?The Right Thing". We ended writing a little PyObjc application that adds support for full identities to Mail.app, but it is just a hack and dunno if it will work forever :/ A. From tss at iki.fi Fri Apr 13 14:51:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 13 Apr 2012 14:51:06 +0300 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F86C62C.8030305@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> <4F86C62C.8030305@wildgooses.com> Message-ID: <918556F5-6EE9-4651-A481-46720480FE46@iki.fi> On 12.4.2012, at 15.10, Ed W wrote: > On 12/04/2012 12:09, Timo Sirainen wrote: >> On 12.4.2012, at 13.58, Ed W wrote: >> >>> The claim by ZFS/BTRFS authors and others is that data silently "bit rots" on it's own. The claim is therefore that you can have a raid1 pair where neither drive reports a hardware failure, but each gives you different data? >> That's one reason why I planned on adding a checksum to each message in dbox. But I forgot to actually do that. I guess I could add it for new messages in some upcoming version. Then Dovecot could optionally verify the checksum before returning the message to client, and if it detects corruption perhaps automatically read it from some alternative location (e.g. if dsync replication is enabled ask from another replica). And Dovecot index files really should have had some small (8/16/32bit) checksums of stuff as well.. >> > > I have to say - I haven't actually seen this happen... Do any of your big mailstore contacts observe this, eg rackspace, etc? I haven't heard. But then again people don't necessarily notice if it has. > Things I might like to do *if* there were some suitable "checksums" available: > - Use the checksum as some kind of guid either for the whole message, the message minus the headers, or individual mime sections Messages already have a GUID. And the rest of that is kind of done with the single instance storage stuff.. I was thinking of using SHA1 of the entire message with headers as the checksum, and save it into dbox metadata field. I also thought about checksumming the metadata fields as well, but that would need another checksum as the first one can have other uses as well besides verifying the message integrity. > - Use the checksums to assist with replication speed/efficiency (dsync or custom imap commands) It would be of some use with dbox index rebuilding. I don't think it would help with dsync. > - File RFCs for new imap features along the "lemonde" lines which allow clients to have faster recovery from corrupted offline states... Too much trouble, no one would implement it :) > - Storage backends where emails are redundantly stored and might not ALL be on a single server (find me the closest copy of email X) - derivations of this might be interesting for compliance archiving of messages? > - Fancy key-value storage backends might use checksums as part of the key value (either for the whole or parts of the message) GUID would work for these as well, without the possibility of a hash collision. From tss at iki.fi Fri Apr 13 14:55:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 13 Apr 2012 14:55:07 +0300 Subject: [Dovecot] POP3 Dovecot Auth CPU usage 75%+ In-Reply-To: References: Message-ID: <0B6D0FE2-2831-460E-8B2B-C255C49DEBAE@iki.fi> On 12.4.2012, at 23.46, Root Kev wrote: > I hope someone can help me, I have been testing out Dovecot to switch from > popa3d which I use at the moment. When I get several users connecting and > disconnection multiple times, the Dovecot process with command Auth uses > 50-90% of the CPU for the period which they are connecting. I am wondering > if there is something that I may have misconfigured, or if there is > something that I can change so that this spike doesn't occur. So is it the "auth" process or "auth worker" process? What if you add: > passdb { > driver = shadow > } > userdb { > driver = passwd args = blocking=yes > } does that move the CPU usage from "auth" to "auth worker" process? Is it using /etc/shadow and /etc/passwd files? Are they large? Do you have enabled other weird stuff in /etc/nsswitch.conf (and were there some other files related to them as well?) From lists at wildgooses.com Fri Apr 13 15:17:19 2012 From: lists at wildgooses.com (Ed W) Date: Fri, 13 Apr 2012 13:17:19 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <918556F5-6EE9-4651-A481-46720480FE46@iki.fi> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> <4F86C62C.8030305@wildgooses.com> <918556F5-6EE9-4651-A481-46720480FE46@iki.fi> Message-ID: <4F88194F.4030807@wildgooses.com> On 13/04/2012 12:51, Timo Sirainen wrote: >> - Use the checksums to assist with replication speed/efficiency (dsync or custom imap commands) > It would be of some use with dbox index rebuilding. I don't think it would help with dsync. .. >> - File RFCs for new imap features along the "lemonde" lines which allow clients to have faster recovery from corrupted offline states... > Too much trouble, no one would implement it :) I presume you have seen that cyrus is working on various distributed options? Standardising this through imap might work if they also buy into it? >> - Storage backends where emails are redundantly stored and might not ALL be on a single server (find me the closest copy of email X) - derivations of this might be interesting for compliance archiving of messages? >> - Fancy key-value storage backends might use checksums as part of the key value (either for the whole or parts of the message) > GUID would work for these as well, without the possibility of a hash collision. I was thinking that the win for key-value store as a backend is if you can reduce the storage requirements or do better placement of the data (mail text replicated widely, attachments stored on higher latency storage?). Hence whilst I don't see this being a win with current options, if it were done then it would almost certainly be "per mime part", eg storing all large attachments in one place and the rest of the message somewhere else, perhaps with different redundancy levels per type OK, this is all completely pie in the sky. Please don't build it! All I meant was that these are the kind of things that someone might one day desire to do and hence they would have competing requirements for what to checksum... Cheers Ed W From tss at iki.fi Fri Apr 13 15:21:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 13 Apr 2012 15:21:49 +0300 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F88194F.4030807@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> <4F86C62C.8030305@wildgooses.com> <918556F5-6EE9-4651-A481-46720480FE46@iki.fi> <4F88194F.4030807@wildgooses.com> Message-ID: <6A4E7A1D-3B55-48DB-A4B0-EF5C01FA829A@iki.fi> On 13.4.2012, at 15.17, Ed W wrote: > On 13/04/2012 12:51, Timo Sirainen wrote: >>> - Use the checksums to assist with replication speed/efficiency (dsync or custom imap commands) >> It would be of some use with dbox index rebuilding. I don't think it would help with dsync. > .. >>> - File RFCs for new imap features along the "lemonde" lines which allow clients to have faster recovery from corrupted offline states... >> Too much trouble, no one would implement it :) > > I presume you have seen that cyrus is working on various distributed options? Standardising this through imap might work if they also buy into it? Probably more trouble than worth. I doubt anyone would want to run a cross-Dovecot/Cyrus cluster. >>> - Storage backends where emails are redundantly stored and might not ALL be on a single server (find me the closest copy of email X) - derivations of this might be interesting for compliance archiving of messages? >>> - Fancy key-value storage backends might use checksums as part of the key value (either for the whole or parts of the message) >> GUID would work for these as well, without the possibility of a hash collision. > > I was thinking that the win for key-value store as a backend is if you can reduce the storage requirements or do better placement of the data (mail text replicated widely, attachments stored on higher latency storage?). Hence whilst I don't see this being a win with current options, if it were done then it would almost certainly be "per mime part", eg storing all large attachments in one place and the rest of the message somewhere else, perhaps with different redundancy levels per type > > OK, this is all completely pie in the sky. Please don't build it! All I meant was that these are the kind of things that someone might one day desire to do and hence they would have competing requirements for what to checksum... That can almost be done already .. the attachments are saved and accessed via a lib-fs API. It wouldn't be difficult to write a backend for some key-value databases. So with about one day's coding you could already have Dovecot save all message attachments to a key-value db, and you can configure redundancy in the db's configs. From stan at hardwarefreak.com Fri Apr 13 15:33:19 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 13 Apr 2012 07:33:19 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> Message-ID: <4F881D0F.2090802@hardwarefreak.com> On 4/13/2012 1:12 AM, Emmanuel Noobadmin wrote: > On 4/12/12, Stan Hoeppner wrote: >> On 4/11/2012 9:23 PM, Emmanuel Noobadmin wrote: >> I suppose the controller could throw an error if >>> the two drives returned data that didn't agree with each other but it >>> wouldn't know which is the accurate copy but that wouldn't protect the >>> integrity of the data, at least not directly without additional human >>> intervention I would think. >> >> When a drive starts throwing uncorrectable read errors, the controller >> faults the drive and tells you to replace it. Good hardware RAID >> controllers are notorious for their penchant to kick drives that would >> continue to work just fine in mdraid or as a single drive for many more >> years. > > What I meant wasn't the drive throwing uncorrectable read errors but > the drives are returning different data that each think is correct or > both may have sent the correct data but one of the set got corrupted > on the fly. After reading the articles posted, maybe the correct term > would be the controller receiving silently corrupted data, say due to > bad cable on one. This simply can't happen. What articles are you referring to? If the author is stating what you say above, he simply doesn't know what he's talking about. > If the controller simply returns the fastest result, it could be the > bad sector and that doesn't protect the integrity of the data right? I already answered this in a previous post. > if the controller gets 1st half from one drive and 2nd half from the > other drive to speed up performance, we could still get the corrupted > half and the controller itself still can't tell if the sector it got > was corrupted isn't it? No, this is not correct. > If the controller compares the two sectors from the drives, it may be > able to tell us something is wrong but there isn't anyway for it to > know which one of the sector was a good read and which isn't, or is > there? Yes it can, and it does. Emmanuel, Ed, we're at a point where I simply don't have the time nor inclination to continue answering these basic questions about the base level functions of storage hardware. You both have serious misconceptions about how many things work. To answer the questions you're asking will require me to teach you the basics of hardware signaling protocols, SCSI, SATA, Fiber Channel, and Ethernet transmission error detection protocols, disk drive firmware error recovery routines, etc, etc, etc. I don't mind, and actually enjoy, passing knowledge. But the amount that seems to be required here to bring you up to speed is about 2^15 times above and beyond the scope of mailing list conversation. In closing, I'll simply say this: If hardware, whether a mobo-down SATA chip, or a $100K SGI SAN RAID controller, allowed silent data corruption or transmission to occur, there would be no storage industry, and we'll all still be using pen and paper. The questions you're asking were solved by hardware and software engineers decades ago. You're fretting and asking about things that were solved decades ago. -- Stan From jtl+dovecot at uvm.edu Fri Apr 13 16:12:02 2012 From: jtl+dovecot at uvm.edu (Jim Lawson) Date: Fri, 13 Apr 2012 09:12:02 -0400 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F881D0F.2090802@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> Message-ID: <4F882622.8040701@uvm.edu> On 04/13/2012 08:33 AM, Stan Hoeppner wrote: >> What I meant wasn't the drive throwing uncorrectable read errors but >> the drives are returning different data that each think is correct or >> both may have sent the correct data but one of the set got corrupted >> on the fly. After reading the articles posted, maybe the correct term >> would be the controller receiving silently corrupted data, say due to >> bad cable on one. > This simply can't happen. What articles are you referring to? If the > author is stating what you say above, he simply doesn't know what he's > talking about. ?! Stan, are you really saying that silent data corruption "simply can't happen"? People who have been studying this have been talking about it for years now. It can happen in the same way that Emmanuel describes. USENIX FAST08: http://static.usenix.org/event/fast08/tech/bairavasundaram.html CERN: http://storagemojo.com/2007/09/19/cerns-data-corruption-research/ http://fuji.web.cern.ch/fuji/talk/2007/kelemen-2007-C5-Silent_Corruptions.pdf LANL: http://institute.lanl.gov/resilience/conferences/2009/HPCResilience09_Michalak.pdf There are others if you search for it. This problem has been well-known in large (petabyte+) data storage systems for some time. Jim From lists at wildgooses.com Fri Apr 13 17:04:17 2012 From: lists at wildgooses.com (Ed W) Date: Fri, 13 Apr 2012 15:04:17 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <6A4E7A1D-3B55-48DB-A4B0-EF5C01FA829A@iki.fi> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> <4F86C62C.8030305@wildgooses.com> <918556F5-6EE9-4651-A481-46720480FE46@iki.fi> <4F88194F.4030807@wildgooses.com> <6A4E7A1D-3B55-48DB-A4B0-EF5C01FA829A@iki.fi> Message-ID: <4F883261.2040309@wildgooses.com> On 13/04/2012 13:21, Timo Sirainen wrote: > On 13.4.2012, at 15.17, Ed W wrote: > >> On 13/04/2012 12:51, Timo Sirainen wrote: >>>> - Use the checksums to assist with replication speed/efficiency (dsync or custom imap commands) >>> It would be of some use with dbox index rebuilding. I don't think it would help with dsync. >> .. >>>> - File RFCs for new imap features along the "lemonde" lines which allow clients to have faster recovery from corrupted offline states... >>> Too much trouble, no one would implement it :) >> I presume you have seen that cyrus is working on various distributed options? Standardising this through imap might work if they also buy into it? > Probably more trouble than worth. I doubt anyone would want to run a cross-Dovecot/Cyrus cluster. No definitely not. Sorry I just meant that you are both working on similar things. Standardising the basics that each use might be useful in the future > That can almost be done already .. the attachments are saved and accessed via a lib-fs API. It wouldn't be difficult to write a backend for some key-value databases. So with about one day's coding you could already have Dovecot save all message attachments to a key-value db, and you can configure redundancy in the db's configs. Hmm, super. Ed W From stan at hardwarefreak.com Fri Apr 13 17:20:29 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 13 Apr 2012 09:20:29 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F882622.8040701@uvm.edu> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F882622.8040701@uvm.edu> Message-ID: <4F88362D.9080400@hardwarefreak.com> On 4/13/2012 8:12 AM, Jim Lawson wrote: > On 04/13/2012 08:33 AM, Stan Hoeppner wrote: >>> What I meant wasn't the drive throwing uncorrectable read errors but >>> the drives are returning different data that each think is correct or >>> both may have sent the correct data but one of the set got corrupted >>> on the fly. After reading the articles posted, maybe the correct term >>> would be the controller receiving silently corrupted data, say due to >>> bad cable on one. >> This simply can't happen. What articles are you referring to? If the >> author is stating what you say above, he simply doesn't know what he's >> talking about. > > > ?! Stan, are you really saying that silent data corruption "simply > can't happen"? Yes, I did. Did you read the context in which I made that statement? > People who have been studying this have been talking > about it for years now. Yes, they have. Did you miss the paragraph where I stated exactly that? Did you also miss the part about the probably of such being dictated by total storage system size and access rate? > It can happen in the same way that Emmanuel > describes. No, it can't. Not in the way Emmanuel described. I already stated the reason, and all of this research backs my statement. You won't see this with a 2 drive mirror, or a 20 drive RAID10. Not until each drive has a capacity in the 15TB+ range, if not more, and again, depending on the total system size. This doesn't address the "RAID5", better known as "parity RAID" write hole, which is a separate issue. Which is also one of the reasons I don't use it. In lieu of an actual controller firmware bug, or mdraid or lvm bug, you'll never see this on small scale systems. > USENIX FAST08: > > http://static.usenix.org/event/fast08/tech/bairavasundaram.html > > CERN: > > http://storagemojo.com/2007/09/19/cerns-data-corruption-research/ > > http://fuji.web.cern.ch/fuji/talk/2007/kelemen-2007-C5-Silent_Corruptions.pdf > > LANL: > > http://institute.lanl.gov/resilience/conferences/2009/HPCResilience09_Michalak.pdf > > There are others if you search for it. This problem has been well-known > in large (petabyte+) data storage systems for some time. And again, this is the crux of it. One doesn't see this problem until one hits extreme scale, which I spent at least a paragraph or two explaining, referencing the same research. Please re-read my post at least twice, critically. Then tell me if I've stated anything substantively different than what any of these researches have. The statements "shouldn't" "wouldn't" and "can't" are based on probabilities. "Can't" or "won't" does not need equal probability 0. The probability of this type of silent data corruption occurring on a 2 disk or 20 disk array of today's drives is not zero over 10 years, but it is so low the effective statement is "can't" or "won't" see this corruption. As I said, when we reach 15-30TB+ disk drives, this may change for small count arrays. -- Stan From root.kev at gmail.com Fri Apr 13 17:32:34 2012 From: root.kev at gmail.com (Root Kev) Date: Fri, 13 Apr 2012 10:32:34 -0400 Subject: [Dovecot] POP3 Dovecot Auth CPU usage 75%+ In-Reply-To: <0B6D0FE2-2831-460E-8B2B-C255C49DEBAE@iki.fi> References: <0B6D0FE2-2831-460E-8B2B-C255C49DEBAE@iki.fi> Message-ID: I tried making the changes that you suggested but it didn't seem to make a noticeable difference. It should be using the shadow file directly. The shadow file has the default Ubuntu system accounts and 16 user accounts, so overall fairly small. The nsswitch.conf file is set as default: # /etc/nsswitch.conf # # Example configuration of GNU Name Service Switch functionality. # If you have the `glibc-doc-reference' and `info' packages installed, try: # `info libc "Name Service Switch"' for information about this file. passwd: compat group: compat shadow: compat hosts: files dns networks: files protocols: db files services: db files ethers: db files rpc: db files netgroup: nis An example of users connecting and the Auth process using alot of CPU (from top): Cpu(s): 87.4%us, 8.0%sy, 0.0%ni, 2.3%id, 0.0%wa, 0.7%hi, 1.7%si, 0.0%st Mem: 1026096k total, 533924k used, 492172k free, 60340k buffers Swap: 1757176k total, 0k used, 1757176k free, 414212k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 643 dovecot 20 0 3096 1616 1208 S 50.7 0.2 0:01.76 auth 644 root 20 0 3096 1524 1140 S 1.3 0.1 0:00.08 auth 642 dovenull 20 0 4276 1612 1256 S 1.0 0.2 0:00.03 pop3-login 623 root 20 0 2704 1020 772 S 0.7 0.1 0:00.02 dovecot 627 root 20 0 4344 2808 1056 S 0.7 0.3 0:00.03 config 631 syslog 20 0 33916 1924 1036 S 0.3 0.2 0:01.61 rsyslogd 696 serverma 20 0 5464 2564 2040 R 0.3 0.2 0:00.01 pop3 1 root 20 0 2652 1604 1216 S 0.0 0.2 0:01.59 init 2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd Thanks for any other ideas.... Kevin On Fri, Apr 13, 2012 at 7:55 AM, Timo Sirainen wrote: > On 12.4.2012, at 23.46, Root Kev wrote: > > So is it the "auth" process or "auth worker" process? What if you add: > > > passdb { > > driver = shadow > > } > > userdb { > > driver = passwd > args = blocking=yes > > } > > does that move the CPU usage from "auth" to "auth worker" process? Is it > using /etc/shadow and /etc/passwd files? Are they large? Do you have > enabled other weird stuff in /etc/nsswitch.conf (and were there some other > files related to them as well?) > > From tss at iki.fi Fri Apr 13 17:37:43 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 13 Apr 2012 17:37:43 +0300 Subject: [Dovecot] POP3 Dovecot Auth CPU usage 75%+ In-Reply-To: References: <0B6D0FE2-2831-460E-8B2B-C255C49DEBAE@iki.fi> Message-ID: <337BF2E0-9F19-4AB4-A81C-2DCE76484662@iki.fi> On 13.4.2012, at 17.32, Root Kev wrote: > 643 dovecot 20 0 3096 1616 1208 S 50.7 0.2 0:01.76 > auth Well, I can't think of any good reason for this. Try strace -p 643 (or whatever pid of the process eating CPU) for a few seconds and send its output? Also maybe get a couple of gdb backtraces to see where it's doing stuff: gdb -p 643 bt quit and repeat it a few times From f.bonnet at esiee.fr Fri Apr 13 18:01:59 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Fri, 13 Apr 2012 17:01:59 +0200 Subject: [Dovecot] French characters from 2.0.x to 2.1.3 Message-ID: <4F883FE7.9080401@esiee.fr> Hello Maybe it's a coincidence but it "seems" since I upgrade from 2.0.18 to 2.1.3 that some French characters ( a with accents by example ) are not well displayed anymore using thunderbird and are replaced by a "?" character Thanks for any info From lists at wildgooses.com Fri Apr 13 18:09:31 2012 From: lists at wildgooses.com (Ed W) Date: Fri, 13 Apr 2012 16:09:31 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F87B9D0.5010300@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <4F87B9D0.5010300@hardwarefreak.com> Message-ID: <4F8841AB.6010108@wildgooses.com> On 13/04/2012 06:29, Stan Hoeppner wrote: > On 4/12/2012 5:58 AM, Ed W wrote: > >> The claim by ZFS/BTRFS authors and others is that data silently "bit >> rots" on it's own. The claim is therefore that you can have a raid1 pair >> where neither drive reports a hardware failure, but each gives you >> different data? > You need to read those articles again very carefully. If you don't > understand what they mean by "1 in 10^15 bits non-recoverable read error > rate" and combined probability, let me know. OK, I'll bite. I only have an honours degree in mathematics from a well known university, so grateful if you could dumb it down appropriately? Lets start with what "those articles" are you referring to? I don't see any articles if I go literally up the chain from this email, but you might be talking about any one of the lots of other emails in this thread or even some other email thread? Wikipedia has it's faults, but it dumbs the "silent corruption" claim down to: http://en.wikipedia.org/wiki/ZFS "an undetected error for every 67TB" And a CERN study apparently claims "far higher than one in every 10^16 bits" Now, I'm NOT professing any experience of axe to grind here. I'm simply asking by what feature do you believe either software or hardware RAID1 is capable of detecting which pair is correct when both pairs of a raid one disk return different results and there is no hardware failure to clue us that one pair suffered a read error? Please don't respond with a maths pissing competition, it's an innocent question about what levels of data checking are done on each piece of the hardware chain? My (probably flawed) understanding is that popular RAID 1 implementations don't add any additional sector checksums over and above what the drives/filesystem/etc add already offer - is this the case? > And this has zero bearing on RAID1. And RAID1 reads don't work the way > you describe above. I explained this in some detail recently. Where? > Been working that way for more than 2 decades Ed. :) Note that "RAID1" > has that "1" for a reason. It was the first RAID level. What should I make of RAID0 then? Incidentally do you disagree with the history of RAID evolution on Wikipedia? http://en.wikipedia.org/wiki/RAID Regards Ed W From lists at wildgooses.com Fri Apr 13 18:31:35 2012 From: lists at wildgooses.com (Ed W) Date: Fri, 13 Apr 2012 16:31:35 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F881D0F.2090802@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> Message-ID: <4F8846D7.7020900@wildgooses.com> On 13/04/2012 13:33, Stan Hoeppner wrote: >> What I meant wasn't the drive throwing uncorrectable read errors but >> the drives are returning different data that each think is correct or >> both may have sent the correct data but one of the set got corrupted >> on the fly. After reading the articles posted, maybe the correct term >> would be the controller receiving silently corrupted data, say due to >> bad cable on one. > This simply can't happen. What articles are you referring to? If the > author is stating what you say above, he simply doesn't know what he's > talking about. It quite clearly can??! Just grab your drive, lever the connector off a little bit until it's a bit flaky and off you go? *THIS* type of problem I have heard of and you can find easy examples with a quick google search of any hobbyist storage board. Very common other examples are such problems due to failing PSUs and other interference driven examples causing explicit disk errors (and once the error rate goes up, some will make it past the checksum) Note this is NOT what I was originally asking about. My interest is more about when the hardware is working reliably and as you agree, the error levels are vastly lower. However, it would be incredibly foolish to claim that it's not trivial to construct a scenario where bad hardware causes plenty of silent corruption? >> If the controller simply returns the fastest result, it could be the >> bad sector and that doesn't protect the integrity of the data right? > I already answered this in a previous post. Not obviously?! I will also add my understanding that linux software RAID1,5&6 *DO NOT* read all disks and hence will not be aware when disks have different data. In fact with software raid you need to run a regular "scrub" job to check this consistency. I also believe that most commodity hardware raid implementations work exactly the same way and a background scrub is needed to detect inconsistent arrays. However, feel free to correct that understanding? >> if the controller gets 1st half from one drive and 2nd half from the >> other drive to speed up performance, we could still get the corrupted >> half and the controller itself still can't tell if the sector it got >> was corrupted isn't it? > No, this is not correct. I definitely think you are wrong and Emmanuel is right? If the controller gets a good read from the disk then it will trust that read and will NOT check the result with the other disk (or parity in the case of RAID5/6). If that read was incorrect for some reason then the data will be passed as good. >> If the controller compares the two sectors from the drives, it may be >> able to tell us something is wrong but there isn't anyway for it to >> know which one of the sector was a good read and which isn't, or is >> there? > Yes it can, and it does. No it definitely does not!! At least not with linux software raid and I don't believe on commodity hardware controllers either! (You would be able to tell because the disk IO would be doubled) Linux software raid 1 isn't that smart, but reads only one disk and trusts the answer if the read did not trigger an error. It does not check the other disk except during an explicit disk scrub. > Emmanuel, Ed, we're at a point where I simply don't have the time nor > inclination to continue answering these basic questions about the base > level functions of storage hardware. You mean those "answers" like: "I answered that in another thread" or "you need to read 'those' articles again" Referring to some unknown and hard to find previous emails is not the same as answering? Also you are wondering off at extreme tangents. The question is simple: - Disk 1 Read good, checksum = A - Disk 2 Read good, checksum = B Disks are a raid 1 pair. How do we know which disk is correct. Please specify raid 1 implementation and mechanism used with any answer > To answer the questions > you're asking will require me to teach you the basics of hardware > signaling protocols, SCSI, SATA, Fiber Channel, and Ethernet > transmission error detection protocols, disk drive firmware error > recovery routines, etc, etc, etc. I really think not... A simple statement of: - Each sector on disk has a certain sized checksum - Controller checks checksum on read - Sent back over SATA connection, with a certain sized checksum - After that you are on your own vs corruption ...Should cover it I think? > In closing, I'll simply say this: If hardware, whether a mobo-down SATA > chip, or a $100K SGI SAN RAID controller, allowed silent data corruption > or transmission to occur, there would be no storage industry, and we'll > all still be using pen and paper. The questions you're asking were > solved by hardware and software engineers decades ago. You're fretting > and asking about things that were solved decades ago. So why are so many people getting excited about it now? Note, there have been plenty of shoddy disk controller implementations before today - ie there exists hardware on sale with *known* defects. Despite that the industry continues without collapse. Now you claim that if corruption is silent and people only tend to notice it much later and under certain edge conditions that this can't be possible because it should cause the industry to collapse..??? ...Not buying your logic... Ed W From michael at orlitzky.com Fri Apr 13 19:08:31 2012 From: michael at orlitzky.com (Michael Orlitzky) Date: Fri, 13 Apr 2012 12:08:31 -0400 Subject: [Dovecot] [OT] Outlook identities In-Reply-To: <4F87C62A.8060105@gmail.com> References: <4F876642.1040701@orlitzky.com> <4F87C62A.8060105@gmail.com> Message-ID: <4F884F7F.5070002@orlitzky.com> On 04/13/12 02:22, Birta Levente wrote: >> >> Outlook (2003, 2007) does do this if you set up different mail accounts, >> but we shouldn't have to do that. > > Control Panel / E-mail (Mail in win7) / Profiles ... > > I hope this help you .... This is more work than setting up multiple accounts =) From michael at orlitzky.com Fri Apr 13 19:13:30 2012 From: michael at orlitzky.com (Michael Orlitzky) Date: Fri, 13 Apr 2012 12:13:30 -0400 Subject: [Dovecot] [OT] Outlook identities In-Reply-To: <4F87C949.8010405@schetterer.org> References: <4F876642.1040701@orlitzky.com> <4F87C949.8010405@schetterer.org> Message-ID: <4F8850AA.80903@orlitzky.com> On 04/13/12 02:35, Robert Schetterer wrote: > > you can do it with exchange, > no wonder outlook isnt a internet mail client in first line > its the client of exchange, so people should use real internet mail > clients ( TB etc ), if m$ would more be compatible, the need of exchange > may more less , but this should not be , as cashmakers should live > forever *g > > http://www.msexchange.org/tutorials/Sending-As.html > > there are a few tricks to goal with profiles etc, but i saw nothing > that was equal to thunderbird identities > Exchange... the cure is worse than the disease! This isn't looking good -- I guess I'll continue to do what I have been: telling people to switch off of Outlook if they want their mail client to not suck. From robert at schetterer.org Fri Apr 13 20:51:59 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 13 Apr 2012 19:51:59 +0200 Subject: [Dovecot] sieve pipe gpg archive mails Message-ID: <4F8867BF.7030207@schetterer.org> hi @ll would it be possible to use http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Pipe to gpg sign archived mail, incoming via lmtp the intention is to store all archived mail gpg signed to notice some changes the mails at possible restore case has anyone done it bevor , or something equal ? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Fri Apr 13 21:33:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 13 Apr 2012 21:33:05 +0300 Subject: [Dovecot] sieve pipe gpg archive mails In-Reply-To: <4F8867BF.7030207@schetterer.org> References: <4F8867BF.7030207@schetterer.org> Message-ID: On 13.4.2012, at 20.51, Robert Schetterer wrote: > would it be possible to use > > http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Pipe > > to gpg sign archived mail, incoming via lmtp > > the intention is to store all archived mail gpg signed > to notice some changes the mails at possible restore case > > has anyone done it bevor , or something equal ? What about mails saved via IMAP APPEND? From mcbdovecot at robuust.nl Fri Apr 13 22:10:04 2012 From: mcbdovecot at robuust.nl (Maarten Bezemer) Date: Fri, 13 Apr 2012 21:10:04 +0200 (CEST) Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F8846D7.7020900@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F8846D7.7020900@wildgooses.com> Message-ID: On Fri, 13 Apr 2012, Ed W wrote: > On 13/04/2012 13:33, Stan Hoeppner wrote: >>> What I meant wasn't the drive throwing uncorrectable read errors but >>> the drives are returning different data that each think is correct or >>> both may have sent the correct data but one of the set got corrupted >>> on the fly. After reading the articles posted, maybe the correct term >>> would be the controller receiving silently corrupted data, say due to >>> bad cable on one. >> This simply can't happen. What articles are you referring to? If the >> author is stating what you say above, he simply doesn't know what he's >> talking about. > It quite clearly can??! I totally agree with Ed here. Drives sure can and sometimes really do return different data, without reporting errors. Also, data can get corrupted on any of the busses or chips it passes through. The math about 10^15 or 10^16 and all that stuff is not only about array sizes. It's also about data transfer. I've seen silent corruption on a few systems myself. (Luckily, only 3 times in a couple years.) Those systems were only in the 2TB-5TB size category, which is substantially lower than the 67TB claimed elsewhere. Yet, statistically, it's well within normal probability levels. Linux mdraid only reads one mirror as long as the drives don't return an error. Easy to check, the read speeds are way beyond a single drive's read speed. When the kernel would have to read all (possibly more than two) mirrors, and compare them, and make a decision based on this comparison, things would be horribly slow. Hardware raid typically uses this exact same approach. This goes for Areca, 3ware, LSI, which cover most of the regular (i.e. non-SAN) professional hardware raid setups. If you don't believe it, just don't take my word for it but test it for yourself. Cleanly power down a raid1 array, take the individual drives, put them into a simple desktop machine, and write different data to both, using some raw disk writing tool like dd. Then put the drives back into the raid1 array, power it up, and re-read the information. You'll see data from both drives will be intermixed as parts of the reads come from one disk, and parts come from the other. Only when you order the raid array to do a verification pass, it'll start screaming and yelling. At least, I hope it will... But as explained elsewhere, silent corruption can occur at numerous places. If you don't have an explicit checksumming/checking mechanism, there are indeed cases that will haunt you if you don't do regular scrubbing or at least do regular verification runs. Heck, that's why Linux mdadm comes with cron jobs to do just that, and hardware raid controllers have similar scheduling capabilities. Of course, scrubbing/verification is not going to magically protect you from all problems. But you would at least get notifications if it detects problems. >>> If the controller compares the two sectors from the drives, it may be >>> able to tell us something is wrong but there isn't anyway for it to >>> know which one of the sector was a good read and which isn't, or is >>> there? >> Yes it can, and it does. > > No it definitely does not!! At least not with linux software raid and I don't > believe on commodity hardware controllers either! (You would be able to tell > because the disk IO would be doubled) Obviously there is no way to tell which versions of a story are correct if you are not biased to believe one of the storytellers and distrust the other. You would have to add a checksum layer for that. (And hope the checksum isn't the part that got corrupted!) >> To answer the questions >> you're asking will require me to teach you the basics of hardware >> signaling protocols, SCSI, SATA, Fiber Channel, and Ethernet >> transmission error detection protocols, disk drive firmware error >> recovery routines, etc, etc, etc. I'm quite familiar with the basics of these protocols. I'm also quite familiar with the flaws in several implementations of "seemingly straightforward protocols". More often than not, there's a pressing need to get new devices onto the market before the competition has something similar and you loose your advantage. More often than not, this results in suboptimal implementations of all those fine protocols and algorithms. And let's face it: flaws in error recovery routines often don't surface until someone actually needs those routines. As long as drives (or any other device) are functioning as expected, everything is all right. But as soon as something starts to get flaky, error recovery has to kick in but may just as well fail to do the right thing. Just consider the real-world analogy of politicians. They do or say something stupid every once in a while, and error recovery (a.k.a. damage control) has to kick in. But even though those well trained professionals, having decades of experience in the political arena, sometimes simply fail to do the right thing. They may have overlooked some pesky details, or they may take actions that don't have the expected outcome because... indeed, things work differently in damage control mode, and the only law you can trust is physics: you always go down when you can't stay on your feet. With hard drives, raid controllers, mainboards, data buses, it's exactly the same. If _something_ isn't working as it should, how should we know which part of it we _can_ trust? >> In closing, I'll simply say this: If hardware, whether a mobo-down SATA >> chip, or a $100K SGI SAN RAID controller, allowed silent data corruption >> or transmission to occur, there would be no storage industry, and we'll >> all still be using pen and paper. The questions you're asking were >> solved by hardware and software engineers decades ago. You're fretting >> and asking about things that were solved decades ago. Isn't it just "worked around" by adding more layers of checksuming and adding more redundancy into the mix? Don't believe this "storage industry" because they tell you it's OK. It simply is not OK. You might want to talk to people in the data and computing cluster business about their opinion on "storage industry professionals"... Timo's suggestion to add checksums to mailboxes/metadata could help to (at least) report these types of failures. Re-reading from different storage when available could also recover the data that got corrupted, but I'm not sure what would be the best way to handle these situations. If you know there is a corruption problem on one of your storage locations, you might want to switch that to read-only asap. Automagically trying to recover might not be the best thing to do. Given all kinds of different use cases, I think that should at least be configurable :-P -- Maarten From lists at necoro.eu Fri Apr 13 23:41:02 2012 From: lists at necoro.eu (=?UTF-8?B?UmVuw6kgTmV1bWFubg==?=) Date: Fri, 13 Apr 2012 22:41:02 +0200 Subject: [Dovecot] imapc / namespace question Message-ID: <4F888F5E.5030709@necoro.eu> Hi, I've got two questions regarding imapc and namespaces: 1) Is there a way of stating, that I only want part of the hierarchy on the end of an imapc connection to be available here? My "problem": On the other end the layout is: INBOX.Shared.{some,nice,folders} If I mirror this into my dovecot hierarchy with prefix "Gemeinsam" I get the following folder structure: Gemeinsam.INBOX.Shared.{some,nice,folders} But I want to ignore the two top-most parts of the hierarchy, i.e. I want it to look: Gemeinsam.{some,nice,folders} Is this possible? I tried with 'location=imapc:INBOX.Shared.' but this did not change anything. 2) "Private" namespaces only make sense, if location contains something user-specific like "%u" or "~". Is this correct? If yes: Is there some other way to make something like an imapc connection only appear for one user, if there is nothing in the location or imapc_user to denote the user it belongs to and hence I cannot use "private"? The only way I see currently is to use "shared" and set up ACLs to forbid all users except one to access the namespace (which gets slightly ugly as I have to use global ACLs, as -- I think -- there is no way to store ACLs for an imapc storage). Thanks, Ren? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From joe at tao.org.uk Sat Apr 14 00:59:36 2012 From: joe at tao.org.uk (Dr Josef Karthauser) Date: Fri, 13 Apr 2012 22:59:36 +0100 Subject: [Dovecot] How do I test if the anti spam plugin is working? In-Reply-To: <3779AD95-CA9E-484B-8B63-039F50B0426E@tao.org.uk> References: <3779AD95-CA9E-484B-8B63-039F50B0426E@tao.org.uk> Message-ID: <4692B327-5985-4227-82A4-C30B4A291C4E@tao.org.uk> On 30 Mar 2012, at 12:29, Dr Josef Karthauser wrote: > I've configured the dspam anti spam plugin, but it doesn't appear to be doing anything when I move mail between mailboxes. Can anyone help me determine what's going on? > > The plugin appears to be loading; at least if I don't define all the required configuration options I get a complaint in the log file. But beyond that I don't see any activity. > > [cut] > I don't even appear to be seeing any log entries from the plugin. > > I've moving an email from my main mailbox into a mailbox called 'SPAM', which is how I thought that it was supposed to be triggered. It turns out that I had the signature header configured incorrectly. :). Joe From stan at hardwarefreak.com Sat Apr 14 06:31:04 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 13 Apr 2012 22:31:04 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F8846D7.7020900@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F8846D7.7020900@wildgooses.com> Message-ID: <4F88EF78.7040202@hardwarefreak.com> On 4/13/2012 10:31 AM, Ed W wrote: > On 13/04/2012 13:33, Stan Hoeppner wrote: >> In closing, I'll simply say this: If hardware, whether a mobo-down SATA >> chip, or a $100K SGI SAN RAID controller, allowed silent data corruption >> or transmission to occur, there would be no storage industry, and we'll >> all still be using pen and paper. The questions you're asking were >> solved by hardware and software engineers decades ago. You're fretting >> and asking about things that were solved decades ago. > > So why are so many people getting excited about it now? "So many"? I know of one person "getting excited" about it. Data densities and overall storage sizes and complexity at the top end of the spectrum are increasing at a faster rate than the consistency/validation mechanisms. That's the entire point of the various academic studies on the issue. Note that the one study required a sample set of 1.5 million disk drives. If the phenomenon were a regular occurrence as you would have everyone here believe, they could have used a much smaller sample set. Ed, this is an academic exercise. Academia leads industry. Almost always has. Academia blows the whistle and waves hands, prompting industry to take action. There is nothing normal users need to do to address this problem. The hardware and software communities will make the necessary adjustments to address this issue before it filters down to the general user community in a half decade or more--when normal users have a 10-20 drive array of 500TB to 1PB or more. Having the prestigious degree that you do, you should already understand the relationship between academic research and industry, and the considerable lead times involved. -- Stan From stan at hardwarefreak.com Sat Apr 14 06:48:07 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 13 Apr 2012 22:48:07 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F8846D7.7020900@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F8846D7.7020900@wildgooses.com> Message-ID: <4F88F377.1070801@hardwarefreak.com> On 4/13/2012 10:31 AM, Ed W wrote: > You mean those "answers" like: > "you need to read 'those' articles again" > > Referring to some unknown and hard to find previous emails is not the > same as answering? No, referring to this: On 4/12/2012 5:58 AM, Ed W wrote: > The claim by ZFS/BTRFS authors and others is that data silently "bit > rots" on it's own. Is it not a correct assumption that you read this in articles? If you read this in books, scrolls, or chiseled tablets, my apologies for assuming it was articles. -- Stan From ngu.antoine at gmail.com Sat Apr 14 12:50:32 2012 From: ngu.antoine at gmail.com (Antoine Nguyen) Date: Sat, 14 Apr 2012 11:50:32 +0200 Subject: [Dovecot] Unseen messages question Message-ID: Hi list, this question is related to the IMAP protocol itself, not really to Dovecot. I'm trying to understand what is the more efficient way to maintain the number of unseen messages of the currently selected mailbox. RFC3501 says a client must not issue a STATUS command to the selected mailbox and that information sent by a SELECT is enough. My current idea follows these steps : * Issue a STATUS before the mailbox is selected => I know how many unseen messages it contains * SELECT the mailbox => I got the eventual first unseen message in this mailbox but I don't understand how this info can be useful * Maintain the unseen counter (on client side) according to what the user do * Send a NOOP command every X minutes and look at the RECENT response to see if there are new messages I think it works pretty well when the mailbox is opened only once. Let's imagine this mailbox is opened twice, by different clients. If one client marks a message as \Seen, how can the second client know about this change? Thanks for your help, Antoine Nguyen http://modoboa.org/ From lists at wildgooses.com Sat Apr 14 13:00:40 2012 From: lists at wildgooses.com (Ed W) Date: Sat, 14 Apr 2012 11:00:40 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F88F377.1070801@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F8846D7.7020900@wildgooses.com> <4F88F377.1070801@hardwarefreak.com> Message-ID: <4F894AC8.9060406@wildgooses.com> On 14/04/2012 04:48, Stan Hoeppner wrote: > On 4/13/2012 10:31 AM, Ed W wrote: > >> You mean those "answers" like: >> "you need to read 'those' articles again" >> >> Referring to some unknown and hard to find previous emails is not the >> same as answering? > No, referring to this: > > On 4/12/2012 5:58 AM, Ed W wrote: > >> The claim by ZFS/BTRFS authors and others is that data silently "bit >> rots" on it's own. > Is it not a correct assumption that you read this in articles? If you > read this in books, scrolls, or chiseled tablets, my apologies for > assuming it was articles. > WHAT?!! The original context was that you wanted me to learn some very specific thing that you accused me of misunderstanding, and then it turns out that the thing I'm supposed to learn comes from re-reading every email, every blog post, every video, every slashdot post, every wiki, every ... that mentions ZFS's reason for including end to end checksumming?!! Please stop wasting our time and get specific You have taken my email which contained a specific question, been asked of you multiple times now and yet you insist on only answering irrelevant details with a pointed and personal dig on each answer. The rudeness is unnecessary, and your evasiveness of answers does not fill me with confidence that you actually know the answer... For the benefit of anyone reading this via email archives or whatever, I think the conclusion we have reached is that: modern systems are now a) a complex sum of pieces, any of which can cause an error to be injected, b) the level of error correction which was originally specified as being sufficient is now starting to be reached in real systems, possibly even consumer systems. There is no "solution", however, the first step is to enhance "detection". Various solutions have been proposed, all increase cost, computation or have some disadvantage - however, one of the more promising detection mechanisms is an end to end checksum, which will then have the effect of augmenting ALL the steps in the chain, not just one specific step. As of today, only a few filesystems offer this, roll on more adopting it Regards Ed W From janfrode at tanso.net Sat Apr 14 13:04:22 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sat, 14 Apr 2012 12:04:22 +0200 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F881D0F.2090802@hardwarefreak.com> References: <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> Message-ID: <20120414100422.GA5353@dibs.tanso.net> On Fri, Apr 13, 2012 at 07:33:19AM -0500, Stan Hoeppner wrote: > > > > What I meant wasn't the drive throwing uncorrectable read errors but > > the drives are returning different data that each think is correct or > > both may have sent the correct data but one of the set got corrupted > > on the fly. After reading the articles posted, maybe the correct term > > would be the controller receiving silently corrupted data, say due to > > bad cable on one. > > This simply can't happen. What articles are you referring to? If the > author is stating what you say above, he simply doesn't know what he's > talking about. It has happened to me, with RAID5 not RAID1. It was a firmware bug in the raid controller that caused the RAID array to go silently corrupted. The HW reported everything green -- but the filesystem was reporting lots of strange errors.. This LUN was part of a larger filesystem striped over multiple LUNs, so parts of the fs was OK, while other parts was corrupt. It was this bug: http://delivery04.dhe.ibm.com/sar/CMA/SDA/02igj/7/ibm_fw1_ds4kfc_07605200_anyos_anycpu.chg - Fix 432525 - CR139339 Data corruption found on drive after reconstruct from GHSP (Global Hot Spare) > In closing, I'll simply say this: If hardware, whether a mobo-down SATA > chip, or a $100K SGI SAN RAID controller, allowed silent data corruption > or transmission to occur, there would be no storage industry, and we'll > all still be using pen and paper. The questions you're asking were > solved by hardware and software engineers decades ago. You're fretting > and asking about things that were solved decades ago. Look at the plans are for your favorite fs: http://www.youtube.com/watch?v=FegjLbCnoBw They're planning on doing metadata checksumming to be sure they don't receive corrupted metadata from the backend storage, and say that data validation is a storage subsystem *or* application problem. Hardly a solved problem.. -jf From lists at wildgooses.com Sat Apr 14 13:22:37 2012 From: lists at wildgooses.com (Ed W) Date: Sat, 14 Apr 2012 11:22:37 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F88EF78.7040202@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F8846D7.7020900@wildgooses.com> <4F88EF78.7040202@hardwarefreak.com> Message-ID: <4F894FED.70901@wildgooses.com> On 14/04/2012 04:31, Stan Hoeppner wrote: > On 4/13/2012 10:31 AM, Ed W wrote: >> On 13/04/2012 13:33, Stan Hoeppner wrote: >>> In closing, I'll simply say this: If hardware, whether a mobo-down SATA >>> chip, or a $100K SGI SAN RAID controller, allowed silent data corruption >>> or transmission to occur, there would be no storage industry, and we'll >>> all still be using pen and paper. The questions you're asking were >>> solved by hardware and software engineers decades ago. You're fretting >>> and asking about things that were solved decades ago. >> So why are so many people getting excited about it now? > "So many"? I know of one person "getting excited" about it. You love being vague don't you? Go on, I'll bite again, do you mean yourself? :-) > Data densities and overall storage sizes and complexity at the top end > of the spectrum are increasing at a faster rate than the > consistency/validation mechanisms. That's the entire point of the > various academic studies on the issue. Again, you love being vague. By your dismissive "academic studies" phrase, do you mean studies done on a major industrial player, ie NetApp in this case? Or do you mean that it's rubbish because they asked someone with some background in statistics to do the work, rather than asking someone sitting nearby in the office to do it? I don't think the researcher broke into NetApp to do this research, so we have to conclude that the industrial partner was onboard. NetApp seem to do a bunch of engineering of their own (got enough patents..) that I think we can safely assume they very much do their own research on this and it's not just "academic"... I doubt they publish all their own internal research, be thankful you got to see some of the results this way... > Note that the one study required > a sample set of 1.5 million disk drives. If the phenomenon were a > regular occurrence as you would have everyone here believe, they could > have used a much smaller sample set. Sigh... You could criticise the study if it had a small number of drives as being under-representive and now you criticise a large study for having too many observations... You cannot have "too many" observations when measuring a small and unpredictable phenomena... Where does it say that they could NOT have reproduced this study with just 10 drives? If you have 1.5 million available, why not use all the results?? > Ed, this is an academic exercise. Academia leads industry. Almost > always has. Academia blows the whistle and waves hands, prompting > industry to take action. Sigh... We are back to the start of the email thread again... Gosh you seem to love arguing and muddying the water for zero reason but to have the last word? It's *trivial* to do a google search and hit *lots* of reports of corruptions in various parts of the system, from corrupting drivers, to hardware which writes incorrectly, to operating system flaws. I just found a bunch more in the Redhat database today while looking for something else. You yourself are very vocal on avoiding certain brands of HD controller which have been rumoured to cause corrupted data... (and thankyou for revealing that kind of thing - it's very helpful) Don't veer off at a tangent now: The *original* email this has spawned is about a VERY specific point. RAID1 appears to offer less protection against a class of error conditions than does RAID6. Nothing more, nothing less. Don't veer off and talk about the minutiae of testing studies at universities, this is a straightforward claim that you have been jumping around and avoiding answering with claims of needing to educate me on SCSI protocols and other fatuous responses. Nor deviate and discuss that RAID6 is inappropriate for many situations - we all get that... > There is nothing normal users need to do to address this problem. ...except sit tight and hope they don't loose anything important! :-) > Having the prestigious degree that you do, you should already understand > the relationship between academic research and industry, and the > considerable lead times involved. I'm guessing you haven't attended higher education then? You are confusing graduate and post-graduate systems... Byee Ed W From jerry at seibercom.net Sat Apr 14 13:51:13 2012 From: jerry at seibercom.net (Jerry) Date: Sat, 14 Apr 2012 06:51:13 -0400 Subject: [Dovecot] [OT] Outlook identities In-Reply-To: <4F8850AA.80903@orlitzky.com> References: <4F876642.1040701@orlitzky.com> <4F87C949.8010405@schetterer.org> <4F8850AA.80903@orlitzky.com> Message-ID: <20120414065113.79eb4065@scorpio> On Fri, 13 Apr 2012 12:13:30 -0400 Michael Orlitzky articulated: > Exchange... the cure is worse than the disease! This isn't looking > good -- I guess I'll continue to do what I have been: telling people > to switch off of Outlook if they want their mail client to not suck. First of all, there are no existing RFC's that require any MUA to meet the requirements that you desire. So please, stop your wining and crying. It is embarrassing. Second, there are avenues available that can make Outlook behave in a fashion that should be acceptable to you. If you choose not to pursue them, then that is you business. I have had to endure hours of tedious nonsense to get a simple sound card to work under a *.nix environment when I could have simply plugged it into a machine running Microsoft Windows and had it working immediately. Your "the cure is worse than the disease" is just self-serving bull-shit. Outlook + MS Exchange offers features that no other MUA presently comes close to being able to duplicate in an office environment. If these don't fit your needs, then please find an MUA that does. No one is holding a gun to your head. However, your desire to force others to abandon something that works fine for them to simple suit your narrow view of what an MUA should or should not do stinks of fascism. I use Outlook at work and claws-mail at home. Each one fits perfectly into the environment I have chosen to use it in. By the way, after examining your original post, I cannot find a single thing that the proper use of filtering rules and plugins cannot easily accomplish. Instead of your customers using a different MUA, they should consider changing to a new service provider. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From amk at spamfence.net Sat Apr 14 14:04:17 2012 From: amk at spamfence.net (Andreas M. Kirchwitz) Date: Sat, 14 Apr 2012 11:04:17 +0000 (UTC) Subject: [Dovecot] Dovecot 2.1 with custom OpenSSL fails to build References: <1331816286.10319.23.camel@innu.invalid> <4F7BBF3F.9060103@iki.fi> Message-ID: Timo Sirainen wrote: >> But two libraries are not quite okay. They don't find their SSL libs: >> >> libdovecot-lda.so >> libdovecot-storage.so > > Maybe this fixes it? > > http://hg.dovecot.org/dovecot-2.1/rev/8b91367bc3e1 Works perfectly! Great, now all components find their libraries by themselves. Thanks a lot for fixing this issue which seemed quite complicated. Very good, thank you ... Andreas From kjonca at o2.pl Sat Apr 14 14:27:59 2012 From: kjonca at o2.pl (Kamil =?iso-8859-2?Q?Jo=F1ca?=) Date: Sat, 14 Apr 2012 13:27:59 +0200 Subject: [Dovecot] Compressed mbox - patch Message-ID: <871unq7dts.fsf@alfa.kjonca> Some time ago I complained about very slow access to compressed mboxes. Unfortunately it looks like that it is very little interest in it, so I have to investigate some things by myself. Firstly: some rationale. Why do I prefer use mbox/maildir over mdbox. Short answer "bus factor" for support mdbox (not only dovecot) Longer answer: if something goes wrong withm maildir/mbox i can use other tools (mutt, or formail or even text editor) and with mdbox ... I am not ISP, I use dovecot as a "gateway" to my (rather huge) mail archive. Most of these mails are rather valuable for me, so I prefer use something "well-known-and-tested". (I can't do like most ISP's do: write in "Terms of Service" that mail can be lost or damaged and we give no warranty :) ) So then: Below my patch. It contains 2 changes: 1. when buffer is compressed, we try to save last marked offset. 2. Increase temporary buffer for decompression. without these changes 1.5 GB of bzip compressed mbox with ~20K messages can't be open in 1.5 day After applying 1. change it can be open in ~1.5 h With both changes it was a few minutes. Maybe it is a good idea to add config parameter to specify size of decompress buffer? Patch is against v2.0.18 -------------- next part -------------- A non-text attachment was scrubbed... Name: compress.patch Type: text/x-diff Size: 3898 bytes Desc: not available URL: -------------- next part -------------- -- Gdyby kto? mia? zb?dny Toshiba G450 - to ch?tnie przejm? ;) < asuffield> a workstation is anything you can stick on somebodies desk and con them into using -- in #debian-devel From kjonca at o2.pl Sat Apr 14 15:13:58 2012 From: kjonca at o2.pl (Kamil =?iso-8859-2?Q?Jo=F1ca?=) Date: Sat, 14 Apr 2012 14:13:58 +0200 Subject: [Dovecot] Sieve pipe extension - can it retur something? Message-ID: <87ty0m5x4p.fsf@alfa.kjonca> I have a question about sieve pipe: can it return something to further processing? For example in procmail I can do: --8<---------------cut here---------------start------------->8--- :0 VIRUS=`$CLAMSCAN --mbox --disable-summary --stdout -` --8<---------------cut here---------------end--------------->8--- and then test VIRUS variable. Maybe I missing something, when read http://hg.rename-it.nl/pigeonhole-0.2-sieve-pipe/raw-file/tip/doc/rfc/spec-bosch-sieve-pipe.txt KJ -- http://sporothrix.wordpress.com/2011/01/16/usa-sie-krztusza-kto-nastepny/ Gloffing is a state of mine. From branko at majic.rs Sat Apr 14 16:30:06 2012 From: branko at majic.rs (=?UTF-8?B?0JHRgNCw0L3QutC+INCc0LDRmNC40Zs=?=) Date: Sat, 14 Apr 2012 15:30:06 +0200 Subject: [Dovecot] Dovecot 2.1.4 and client certificates Message-ID: <20120414153006.19ce7e3c@majic.rs> Version: 2.1.4 OS: Gentoo stable/amd64 OpenSSL version: 1.0.0h I'm having a slight problem with the client certificates in Dovecot 2.1.4. I've set-up the client certificate verification/authentication, and it seems that Dovecot is choking on the trustchain with CRL's that I'm providing to it (attached to this mail). When I enable the client authentication using certificates, and pick the certificate from my client (I've also tried it out with gnutls-cli as well), I get the following errors in Dovecot's log: imap-login: Info: Invalid certificate: Different CRL scope: /CN=Example Root CA/O=Example Inc./C=RS As per the wiki2 configuration page, I've set up the truststore in the following order (everything PEM-encoded): Example Person CA Certificate Example Person CA CRL Example Root CA Certificate Example Root CA CRL Person CA is the one issuing the end-entity certificates, of course. I'm also attaching the certificate I've used for testing. On additional note, the imap-login process also got stuck writing out the error message to the log file, refusing to die when receiving the SIGTERM (had to send SIGKILL). A similar set-up used to work under Dovecot in Debian Squeeze (version 1.2.15). The same file copied over to Dovecot 2.1.4's configuration won't work. I've compiled Dovecot by hand, and I'm not running it in any kind of chroot (this is a developer set-up so I could add support for rfc822Name username extraction I mentioned a week or so ago without messing around as root). Best regards -- Branko Majic Jabber: branko at majic.rs Please use only Free formats when sending attachments to me. ?????? ????? ?????: branko at majic.rs ????? ??? ?? ??????? ?????? ????????? ? ????????? ?????????. -------------- next part -------------- A non-text attachment was scrubbed... Name: trustchain.pem Type: application/x-x509-ca-cert Size: 6640 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: branko_majic.crt Type: application/x-x509-ca-cert Size: 1700 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From cor at xs4all.nl Sat Apr 14 19:24:24 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sat, 14 Apr 2012 18:24:24 +0200 Subject: [Dovecot] LMTP auth problem Message-ID: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> hey all, im getting the following error: Apr 14 14:29:44 lmtpdirector1 dovecot: auth: Error: passdb(scorpio,127.0.0.1): Auth client doesn't have permissions to do a PASS lookup: /var/run/dovecot/auth-userdb mode=0666, but not owned by UID 112(dovecot) Apr 14 14:29:44 lmtpdirector1 dovecot: lmtp(18298): Error: user scorpio: Auth PASS lookup failed My config. Director servers running both imap and lmtp with a matching set of real servers accepting imap/lmtp. Imap is working fine, and has been working fine for a while. Im trying to add lmtp to the director, but i cant seem to get that working. We're passing passdb on to the real servers. How does this work with lmtp? protocols = imap lmtp protocol lmtp { auth_socket_path = director-userdb } lmtp_proxy = yes # passdb check on real servers passdb { driver = static args = proxy=y nopassword=y } Cor From cor at xs4all.nl Sat Apr 14 19:52:40 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sat, 14 Apr 2012 18:52:40 +0200 Subject: [Dovecot] LMTP auth problem In-Reply-To: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> References: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> Message-ID: <20120414165240.GA31983@xs4all.nl> Of course the moment I post I seem to have figured it out.. service auth { unix_listener auth-userdb { mode = 0777 } } Is this safe if your servers are secure? Cor From tlx at leuxner.net Sat Apr 14 20:10:44 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sat, 14 Apr 2012 19:10:44 +0200 Subject: [Dovecot] LMTP auth problem In-Reply-To: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> References: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> Message-ID: <71BEFF67-4A0A-442A-9ED0-C4B1B23A4E7F@leuxner.net> Am 14.04.2012 um 18:24 schrieb Cor Bosman: > Apr 14 14:29:44 lmtpdirector1 dovecot: auth: Error: passdb(scorpio,127.0.0.1): Auth client doesn't have permissions to do a PASS lookup: /var/run/dovecot/auth-userdb mode=0666, but not owned by UID 112(dovecot) > Apr 14 14:29:44 lmtpdirector1 dovecot: lmtp(18298): Error: user scorpio: Auth PASS lookup failed I'd just try 'user = dovecot' rather than making it wide open because that's what the log basically says. $ doveconf -d | grep 'unix_listener auth-userdb' -A 4 unix_listener auth-userdb { group = mode = 0666 user = } Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From cor at xs4all.nl Sat Apr 14 20:21:35 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sat, 14 Apr 2012 19:21:35 +0200 Subject: [Dovecot] LMTP auth problem In-Reply-To: <71BEFF67-4A0A-442A-9ED0-C4B1B23A4E7F@leuxner.net> References: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> <71BEFF67-4A0A-442A-9ED0-C4B1B23A4E7F@leuxner.net> Message-ID: <20120414172135.GA32608@xs4all.nl> > > Apr 14 14:29:44 lmtpdirector1 dovecot: auth: Error: passdb(scorpio,127.0.0.1): Auth client doesn't have permissions to do a PASS lookup: /var/run/dovecot/auth-userdb mode=0666, but not owned by UID 112(dovecot) > > Apr 14 14:29:44 lmtpdirector1 dovecot: lmtp(18298): Error: user scorpio: Auth PASS lookup failed > > I'd just try 'user = dovecot' rather than making it wide open because that's what the log basically says. > > $ doveconf -d | grep 'unix_listener auth-userdb' -A 4 > unix_listener auth-userdb { > group = > mode = 0666 > user = > } > My config was the same as yours. That didnt work for me. But if I add user = dovecot mode = 0666 That does work. Of course, the difference between 777 and 666 is minimal. I think 666 is handled as a special case in the code? Cor From stephan at rename-it.nl Sun Apr 15 01:36:08 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 15 Apr 2012 00:36:08 +0200 Subject: [Dovecot] Sieve pipe extension - can it retur something? In-Reply-To: <87ty0m5x4p.fsf@alfa.kjonca> References: <87ty0m5x4p.fsf@alfa.kjonca> Message-ID: <4F89FBD8.9040501@rename-it.nl> Op 4/14/2012 2:13 PM, Kamil Jo?ca schreef: > > I have a question about sieve pipe: can it return something to further > processing? > For example in procmail I can do: > --8<---------------cut here---------------start------------->8--- > :0 > VIRUS=`$CLAMSCAN --mbox --disable-summary --stdout -` > --8<---------------cut here---------------end--------------->8--- > and then test VIRUS variable. > > Maybe I missing something, when read > http://hg.rename-it.nl/pigeonhole-0.2-sieve-pipe/raw-file/tip/doc/rfc/spec-bosch-sieve-pipe.txt For Pigeonhole 0.3/Dovecot 2.1 there is a new plugin called ExtPrograms. Apart from the 'pipe' extension it adds the 'execute' extension that should match just what you want: http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/raw-file/d4683490a878/doc/rfc/spec-bosch-sieve-extprograms.txt Regards, Stephan. From stan at hardwarefreak.com Sun Apr 15 01:39:55 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 14 Apr 2012 17:39:55 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <20120414100422.GA5353@dibs.tanso.net> References: <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <20120414100422.GA5353@dibs.tanso.net> Message-ID: <4F89FCBB.4070305@hardwarefreak.com> On 4/14/2012 5:04 AM, Jan-Frode Myklebust wrote: > On Fri, Apr 13, 2012 at 07:33:19AM -0500, Stan Hoeppner wrote: >>> >>> What I meant wasn't the drive throwing uncorrectable read errors but >>> the drives are returning different data that each think is correct or >>> both may have sent the correct data but one of the set got corrupted >>> on the fly. After reading the articles posted, maybe the correct term >>> would be the controller receiving silently corrupted data, say due to >>> bad cable on one. >> >> This simply can't happen. What articles are you referring to? If the >> author is stating what you say above, he simply doesn't know what he's >> talking about. > > It has happened to me, with RAID5 not RAID1. It was a firmware bug > in the raid controller that caused the RAID array to go silently > corrupted. The HW reported everything green -- but the filesystem was > reporting lots of strange errors.. This LUN was part of a larger > filesystem striped over multiple LUNs, so parts of the fs was OK, while > other parts was corrupt. > > It was this bug: > > http://delivery04.dhe.ibm.com/sar/CMA/SDA/02igj/7/ibm_fw1_ds4kfc_07605200_anyos_anycpu.chg > - Fix 432525 - CR139339 Data corruption found on drive after > reconstruct from GHSP (Global Hot Spare) Note my comments were specific to the RAID1 case, or a concatenated set of RAID1 devices. And note the discussion was framed around silent corruption in the absence of bugs and hardware failure, or should I say, where no bugs or hardware failures can be identified. > > >> In closing, I'll simply say this: If hardware, whether a mobo-down SATA >> chip, or a $100K SGI SAN RAID controller, allowed silent data corruption >> or transmission to occur, there would be no storage industry, and we'll >> all still be using pen and paper. The questions you're asking were >> solved by hardware and software engineers decades ago. You're fretting >> and asking about things that were solved decades ago. > > Look at the plans are for your favorite fs: > > http://www.youtube.com/watch?v=FegjLbCnoBw > > They're planning on doing metadata checksumming to be sure they don't > receive corrupted metadata from the backend storage, and say that data > validation is a storage subsystem *or* application problem. You can't made sure you don't receive corrupted data. You take steps to mitigate the negative effects of it if and when it happens. The XFS devs are planning this for the future. If the problem was here now, this work would have already been done. > Hardly a solved problem.. It has been up to this point. The issue going forward is that current devices don't employ sufficient consistency checking to meet future needs. And the disk drive makers apparently don't want to consume the additional bits required to properly do this in the drives. If they'd dedicate far more bits to ECC we may not have this issue. But since it appears this isn't going to change, kernel, filesystem and application developers are taking steps to mitigate it. Again, this "silent corruption" issue as described in the various academic papers is a future problem for most, not a current problem. It's only a current problem for those are the bleeding edge of large scale storage. Note that firmware bugs in individual products aren't part of this issue. Those will be with us forever in various products because humans make mistakes. No amount of filesystem or application code can mitigate those. The solution to that is standard best practices: snapshots, backups, or even mirroring all your storage across different vendor hardware. -- Stan From stan at hardwarefreak.com Sun Apr 15 03:05:19 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 14 Apr 2012 19:05:19 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F894AC8.9060406@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F8846D7.7020900@wildgooses.com> <4F88F377.1070801@hardwarefreak.com> <4F894AC8.9060406@wildgooses.com> Message-ID: <4F8A10BF.3020706@hardwarefreak.com> On 4/14/2012 5:00 AM, Ed W wrote: > On 14/04/2012 04:48, Stan Hoeppner wrote: >> On 4/13/2012 10:31 AM, Ed W wrote: >> >>> You mean those "answers" like: >>> "you need to read 'those' articles again" >>> >>> Referring to some unknown and hard to find previous emails is not the >>> same as answering? >> No, referring to this: >> >> On 4/12/2012 5:58 AM, Ed W wrote: >> >>> The claim by ZFS/BTRFS authors and others is that data silently "bit >>> rots" on it's own. >> Is it not a correct assumption that you read this in articles? If you >> read this in books, scrolls, or chiseled tablets, my apologies for >> assuming it was articles. >> > > WHAT?!! The original context was that you wanted me to learn some very > specific thing that you accused me of misunderstanding, and then it > turns out that the thing I'm supposed to learn comes from re-reading > every email, every blog post, every video, every slashdot post, every > wiki, every ... that mentions ZFS's reason for including end to end > checksumming?!! No, the original context was your town crier statement that the sky is falling due to silent data corruption. I pointed out that this is not the case, currently, that most wouldn't see this until quite a few years down the road. I provided facts to back my statement, which you didn't seem to grasp or comprehend. I pointed this out and your top popped with a cloud of steam. > Please stop wasting our time and get specific Whose time am I wasting Ed? You're the primary person one on this list who wastes everyone's time with these drawn out threads, usually unrelated to Dovecot. I have been plenty specific. The problem is you lack the knowledge and understanding of hardware communication. You're upset because I'm not pointing out the knowledge you seem to lack? Is that not a waste of everyone's time? Is that not be even "more insulting"? Causing even more excited/heated emails from you? > You have taken my email which contained a specific question, been asked > of you multiple times now and yet you insist on only answering > irrelevant details with a pointed and personal dig on each answer. The > rudeness is unnecessary, and your evasiveness of answers does not fill > me with confidence that you actually know the answer... Ed, I have not been rude. I've been attempting to prevent you dragging us into the mud, which you've done, as you often do. How specific would you like me to get? This is what you seem to be missing: Drives perform per sector CRC before transmitting data to the HBA. ATA, SATA, SCSI, SAS, fiber channel devices and HBAs all perform CRC on wire data. The PCI/PCI-X/PCIe buses/channels and Southbridge all perform CRC on wire data. HyperTransport, and Intel's proprietary links also perform CRC on wire transmissions. Server memory is protected by ECC, some by ChipKill which can tolerate double bit errors. With today's systems and storage densities, with error correcting code on all data paths within the system, and on the drives themselves, "silent data corruption" is not an issue--in absence of defective hardware or a bug, which are not relevant to the discussion. > For the benefit of anyone reading this via email archives or whatever, I > think the conclusion we have reached is that: modern systems are now a) > a complex sum of pieces, any of which can cause an error to be injected, Errors occur all the time. And they're corrected nearly all of the time, on modern complex systems. Silent errors do not occur frequently, usually not at all, on most modern systems. > b) the level of error correction which was originally specified as being > sufficient is now starting to be reached in real systems, FSVO 'real systems'. The few occurrences of "silent data corruption" I'm aware of have been documented in academic papers published by researches working at taxpayer funded institutions. In the case of CERN, the problem was a firmware bug in the Western Digital drives that caused an issue with the 3Ware controllers. This kind of thing happens when using COTS DIY hardware in the absence of proper load validation testing. So this case doesn't really fit the Henny-penny silent data corruption scenario as a firmware bug caused it. One that should have been caught and corrected during testing. In the other cases I'm aware of, all were HPC systems which generated SDC under extended high loads, and these SDCs nearly all occurred somewhere other than the storage systems--CPUs, RAM, interconnect, etc. HPC apps tend to run the CPUs, interconnects, storage, etc, at full bandwidth for hours at a time, across tens of thousands of nodes, so the probability of SDC is much higher simply due to scale. > possibly even > consumer systems. Possibly? If you're going to post pure conjecture why not say "possibly even iPhones or Androids"? There's no data to back either claim. Stick to the facts. > There is no "solution", however, the first step is to > enhance "detection". Various solutions have been proposed, all increase > cost, computation or have some disadvantage - however, one of the more > promising detection mechanisms is an end to end checksum, which will > then have the effect of augmenting ALL the steps in the chain, not just > one specific step. As of today, only a few filesystems offer this, roll > on more adopting it So after all the steam blowing, we're back to where we started. I disagree with your assertion that this is an issue that we--meaning "average" users not possessing 1PB storage systems or massive clusters--need to be worried about TODAY. I gave sound reasons as to why this is the case. You've given us 'a couple of academic papers say the sky is falling so I'm repeating the sky is falling'. Without apparently truly understanding the issue. The data available and the experience of the vast majority of IT folks backs my position--which is why that's my position. There is little to no data supporting your position. I say this isn't going to be an issue for average users, if at all, for a few years to come. You say it's here now. That's a fairly minor point of disagreement to cause such a heated (on your part) lengthy exchange. BTW, if you see anything I've stated as rude you've apparently not been on the Interwebs long. ;) -- Stan From dovecot-user at spambox.dk Sun Apr 15 12:42:16 2012 From: dovecot-user at spambox.dk (Henrik Larsson) Date: Sun, 15 Apr 2012 11:42:16 +0200 Subject: [Dovecot] 2.1.2 Corrupted squat uidlist In-Reply-To: <4F7164DC.7010706@unict.it> References: <4F7164DC.7010706@unict.it> Message-ID: <4F8A97F8.70808@spambox.dk> On 27-03-2012 08:57, Luca Palazzo wrote: > Hi Timo and All, > after upgrading to 2.1.2 i'm getting a lot of these messages: > Error: Corrupted squat uidlist file XXXXXX wrong indexid After an upgrade to 2.1.3 i see this as well. Any thoughts? Apr 15 03:43:43 imap(xxxx): Error: Corrupted squat uidlist file /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid Apr 15 06:30:27 imap(xxxx): Error: Corrupted squat uidlist file /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid Apr 15 07:43:55 pop3(xxxx): Error: Corrupted squat uidlist file /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid Apr 15 09:00:01 imap(xxxx): Error: Corrupted squat uidlist file /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid # doveconf -n # 2.1.3: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 8.2-STABLE amd64 auth_mechanisms = plain login digest-md5 cram-md5 first_valid_uid = 125 listen = * log_path = /var/log/dovecot mail_plugins = fts fts_squat zlib mail_privileged_group = postfix mail_temp_dir = /var/db/dovecot managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = prefix = separator = . type = private } namespace { hidden = yes inbox = no list = no location = prefix = INBOX. separator = . type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { fts = squat fts_squat = partial=4 full=10 sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve service auth-worker { user = $default_internal_user } service auth { unix_listener /home/mail/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } service lmtp { executable = lmtp -L unix_listener /home/mail/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } ssl_cert = Hi Timo the virtual setup got broke after update to 2.1 ( its still working under 2.0 ) on other loadbalanced server namespace default { separator = / prefix = "" list = yes subscriptions = yes hidden = no } namespace virtual { prefix = "virtual/" separator = / location = virtual:/etc/dovecot/virtual:LAYOUT=maildir++:INDEX=~/virtual hidden = yes list = no subscriptions= no } namespace real { prefix = "RealMails/" separator = / list = no hidden = yes } after upgrade i have Error: Couldn't open INBOX: Virtual mailbox open failed because of mailbox /RealMails/: Invalid mailbox name --------------------- also i wanted to include namespace inbox { #mailbox name { # auto=create will automatically create this mailbox. # auto=subscribe will both create and subscribe to the mailbox. #auto = no # Space separated list of IMAP SPECIAL-USE attributes as specified by # RFC 6154: \All \Archive \Drafts \Flagged \Junk \Sent \Trash #special_use = #} # These mailboxes are widely used and could perhaps be created automatically: mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Trash { special_use = \Trash } # For \Sent mailboxes there are two widely used names. We'll mark both of # them as \Sent. User typically deletes one of them if duplicates are created. mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } # If you have a virtual "All messages" mailbox: mailbox virtual/All { special_use = \All } # If you have a virtual "Flagged" mailbox: mailbox virtual/Flagged { special_use = \Flagged } } but i found no working namespace combination with virtual -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Sun Apr 15 16:14:25 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 15 Apr 2012 15:14:25 +0200 Subject: [Dovecot] http://xi.rename-it.nl upgrade 2.0 - 2.1 dovecot-common (<< 2:2.1.alpha1-0~auto+47) Message-ID: <4F8AC9B1.2030307@schetterer.org> Hi, i got depency problem like dovecot-common (<< 2:2.1.alpha1-0~auto+47) dpkg -i dovecot-common_2.1.4-0~auto+3_all.de after all dove seems to run without install dovecot-common basicly -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From mhlavink at redhat.com Mon Apr 16 12:55:28 2012 From: mhlavink at redhat.com (Michal Hlavinka) Date: Mon, 16 Apr 2012 11:55:28 +0200 Subject: [Dovecot] dovecot and systemd In-Reply-To: <1331820329.10319.32.camel@innu> References: <4F61EFE8.1000901@redhat.com> <1331820329.10319.32.camel@innu> Message-ID: <4F8BEC90.8060504@redhat.com> On 03/15/2012 03:05 PM, Timo Sirainen wrote: > On Thu, 2012-03-15 at 14:34 +0100, Michal Hlavinka wrote: >> What exactly should happen when >> dovecot.conf does not match dovecot.socket configuration? > > Dovecot's systemd code was written by one of you Redhat guys. I had some > similar thoughts when I applied the patch, but didn't really know what > to do about it, so I didn't do anything. So: I don't know. Maybe some > other project has solved this somehow already? Seems other projects did not solve this yet. Most projects provide just one service. In that case, they serve any connection they get, because they know what to do. I've discussed this with systemd upstream and we've decided that the best solution is to log error message and close that socket. > Dovecot anyway needs its own internal UNIX listeners. Should all > internal inet listeners be disabled? Could Dovecot somehow talk to > systemd and ask what listeners it's using for Dovecot and log warnings > if they don't match? I don't understand this question completely. What it does already is that during start up, when dovecot creates sockets, it checks what sockets already exist and creates only the missing ones. Systemd provides following functions: sd_is_fifo (3) - Check the type of a file descriptor sd_is_mq (3) - Check the type of a file descriptor sd_is_socket (3) - Check the type of a file descriptor sd_is_socket_inet (3) - Check the type of a file descriptor sd_is_socket_unix (3) - Check the type of a file descriptor sd_listen_fds (3) - Check for file descriptors passed by the init system. http://0pointer.de/public/systemd-man/sd_listen_fds.html I wrote simple patch that close the extra sockets. It's tested and works fine. You'll maybe want to move that function to different place and/or change wording of error messages. Michal -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-2.0.19-systemdfix.patch Type: text/x-patch Size: 2669 bytes Desc: not available URL: From root.kev at gmail.com Mon Apr 16 17:55:54 2012 From: root.kev at gmail.com (Root Kev) Date: Mon, 16 Apr 2012 10:55:54 -0400 Subject: [Dovecot] POP3 Dovecot Auth CPU usage 75%+ In-Reply-To: <337BF2E0-9F19-4AB4-A81C-2DCE76484662@iki.fi> References: <0B6D0FE2-2831-460E-8B2B-C255C49DEBAE@iki.fi> <337BF2E0-9F19-4AB4-A81C-2DCE76484662@iki.fi> Message-ID: I think my last email may have been bounced due to attachment size, I have put a snippet of the captures below. The CPU is still going to high percent of usage when my test mailboxes are used. An ideas on how to bring down the Auth CPU usage would be greatly appreciated! Thanks, Kevin Stace on the Auth process: epoll_wait(13, {{EPOLLIN, {u32=150109008, u64=150109008}}}, 29, 149958) = 1 gettimeofday({1334328634, 21072}, NULL) = 0 read(29, "VERSION\t1\t1\nREQUEST\t1011351553\t3"..., 1024) = 72 time(NULL) = 1334328634 writev(29, [{"USER\t1011351553\tservermailbox1\ts"..., 108}, {"\n", 1}], 2) = 109 gettimeofday({1334328634, 27993}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=149927248, u64=149927248}}}, 29, 149992) = 1 gettimeofday({1334328634, 32215}, NULL) = 0 accept(11, {sa_family=AF_FILE, NULL}, [2]) = 30 fcntl64(30, F_GETFL) = 0x2 (flags O_RDWR) fcntl64(30, F_SETFL, O_RDWR|O_NONBLOCK) = 0 gettimeofday({1334328634, 32342}, NULL) = 0 fstat64(30, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0 _llseek(30, 0, 0xbffd24c0, SEEK_CUR) = -1 ESPIPE (Illegal seek) getsockname(30, {sa_family=AF_FILE, path="/usr/local/var/run/dovecot"}, [41]) = 0 epoll_ctl(13, EPOLL_CTL_ADD, 30, {EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP, {u32=150123496, u64=150123496}}) = 0 write(30, "VERSION\t1\t1\nSPID\t2093\n", 22) = 22 gettimeofday({1334328634, 32625}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=150123496, u64=150123496}}}, 29, 1000) = 1 gettimeofday({1334328634, 32721}, NULL) = 0 read(30, "VERSION\t1\t1\n", 1024) = 12 gettimeofday({1334328634, 32792}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=150123496, u64=150123496}}}, 29, 1000) = 1 gettimeofday({1334328634, 32883}, NULL) = 0 read(30, "REQUEST\t3624009729\t3062\t16\tbe004"..., 1012) = 60 time(NULL) = 1334328634 writev(30, [{"USER\t3624009729\tservermailbox\tsy"..., 105}, {"\n", 1}], 2) = 106 gettimeofday({1334328634, 33062}, NULL) = 0 epoll_wait(13, {{EPOLLIN|EPOLLHUP, {u32=150094520, u64=150094520}}}, 29, 999) = 1 gettimeofday({1334328634, 33766}, NULL) = 0 read(28, "", 6243) = 0 epoll_ctl(13, EPOLL_CTL_DEL, 28, {0, {u32=150094520, u64=150094520}}) = 0 close(28) = 0 epoll_wait(13, {{EPOLLIN|EPOLLHUP, {u32=150109008, u64=150109008}}}, 29, -1) = 1 gettimeofday({1334328634, 40036}, NULL) = 0 read(29, "", 952) = 0 epoll_ctl(13, EPOLL_CTL_DEL, 29, {0, {u32=150109008, u64=150109008}}) = 0 close(29) = 0 gettimeofday({1334328634, 40163}, NULL) = 0 gettimeofday({1334328634, 40197}, NULL) = 0 epoll_wait(13, {{EPOLLIN|EPOLLHUP, {u32=150123496, u64=150123496}}}, 29, 1000) = 1 gettimeofday({1334328634, 44007}, NULL) = 0 read(30, "", 952) = 0 epoll_ctl(13, EPOLL_CTL_DEL, 30, {0, {u32=150123496, u64=150123496}}) = 0 close(30) = 0 gettimeofday({1334328634, 44148}, NULL) = 0 gettimeofday({1334328634, 44184}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=150065544, u64=150065544}}}, 29, 1000) = 1 gettimeofday({1334328634, 52466}, NULL) = 0 read(26, "AUTH\t1\tPLAIN\tservice=pop3\tlip=17"..., 8170) = 122 gettimeofday({1334328634, 52582}, NULL) = 0 writev(12, [{"PENALTY-GET\t172.20.20.110", 25}, {"\n", 1}], 2) = 26 gettimeofday({1334328634, 52698}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=149924840, u64=149924840}}}, 29, 992) = 1 gettimeofday({1334328634, 52760}, NULL) = 0 read(12, "0 0\n", 424) = 4 time(NULL) = 1334328634 gettimeofday({1334328634, 93200}, NULL) = 0 writev(26, [{"OK\t1\tuser=servermailbox1", 24}, {"\n", 1}], 2) = 25 read(12, 0x8f36c14, 420) = -1 EAGAIN (Resource temporarily unavailable) gettimeofday({1334328634, 93651}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=150065544, u64=150065544}}}, 29, 951) = 1 gettimeofday({1334328634, 93715}, NULL) = 0 read(26, "AUTH\t2\tPLAIN\tservice=pop3\tlip=17"..., 8048) = 118 gettimeofday({1334328634, 93808}, NULL) = 0 writev(12, [{"PENALTY-GET\t172.20.20.110", 25}, {"\n", 1}], 2) = 26 gettimeofday({1334328634, 93919}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=149924840, u64=149924840}}}, 29, 951) = 1 gettimeofday({1334328634, 93980}, NULL) = 0 read(12, "0 0\n", 420) = 4 time(NULL) = 1334328634 gettimeofday({1334328634, 133578}, NULL) = 0 writev(26, [{"OK\t2\tuser=servermailbox", 23}, {"\n", 1}], 2) = 24 read(12, 0x8f36c18, 416) = -1 EAGAIN (Resource temporarily unavailable) gettimeofday({1334328634, 133998}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=149927248, u64=149927248}}}, 29, 911) = 1 gettimeofday({1334328634, 134064}, NULL) = 0 accept(11, {sa_family=AF_FILE, NULL}, [2]) = 28 fcntl64(28, F_GETFL) = 0x2 (flags O_RDWR) fcntl64(28, F_SETFL, O_RDWR|O_NONBLOCK) = 0 gettimeofday({1334328634, 134200}, NULL) = 0 fstat64(28, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0 _llseek(28, 0, 0xbffd24c0, SEEK_CUR) = -1 ESPIPE (Illegal seek) getsockname(28, {sa_family=AF_FILE, path="/usr/local/var/run/dovecot"}, [41]) = 0 epoll_ctl(13, EPOLL_CTL_ADD, 28, {EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP, {u32=150094520, u64=150094520}}) = 0 Debug on the auth process: Apr 13 11:01:58 devsmtp dovecot: auth: Debug: cache(qamailbox,172.20.20.222): miss Apr 13 11:01:58 devsmtp dovecot: auth-worker(3432): Debug: Loading modules from directory: /usr/local/lib/dovecot/auth Apr 13 11:01:58 devsmtp dovecot: auth-worker(3432): Debug: shadow(qamailbox,172.20.20.222): lookup Apr 13 11:01:58 devsmtp dovecot: auth: Debug: client out: OK#0111#011user=qamailbox Apr 13 11:01:58 devsmtp dovecot: auth: Debug: master in: REQUEST#0111220673537#0113397#0111#0115609887f745a84903ce3699d23e7b886 Apr 13 11:01:58 devsmtp dovecot: auth: Debug: userdb-cache(qamailbox,172.20.20.222): miss Apr 13 11:01:58 devsmtp dovecot: auth-worker(3432): Debug: passwd(qamailbox,172.20.20.222): lookup Apr 13 11:01:58 devsmtp dovecot: auth: Debug: master out: USER#0111220673537#011qamailbox#011system_groups_user=qamailbox#011uid=1002#011gid=1002#011home=/home/qamailbox Apr 13 11:01:58 devsmtp dovecot: pop3-login: Login: user=, method=PLAIN, rip=172.20.20.222, lip=172.20.20.222, mpid=3433, secured Apr 13 11:02:05 devsmtp dovecot: pop3(qamailbox): Disconnected: Logged out top=0/0, retr=1/15637, del=0/50, size=779917 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: auth client connected (pid=3397) Apr 13 11:05:31 devsmtp dovecot: auth: Debug: client in: AUTH#0112#011PLAIN#011service=pop3#011lip=172.20.20.222#011rip=172.20.20.110#011lport=110#011rport=53254#011resp=AHNlcnZlcm1haWxib3gxADEyMzQ1Ng== Apr 13 11:05:31 devsmtp dovecot: auth: Debug: cache(servermailbox1,172.20.20.110): miss Apr 13 11:05:31 devsmtp dovecot: auth-worker(3459): Debug: Loading modules from directory: /usr/local/lib/dovecot/auth Apr 13 11:05:31 devsmtp dovecot: auth-worker(3459): Debug: shadow(servermailbox1,172.20.20.110): lookup Apr 13 11:05:31 devsmtp dovecot: auth: Debug: client out: OK#0112#011user=servermailbox1 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: master in: REQUEST#011358219777#0113397#0112#01132ecf6b93729a90ece98bbb643446ea6 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: userdb-cache(servermailbox1,172.20.20.110): miss Apr 13 11:05:31 devsmtp dovecot: auth-worker(3459): Debug: passwd(servermailbox1,172.20.20.110): lookup Apr 13 11:05:31 devsmtp dovecot: auth: Debug: master out: USER#011358219777#011servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:31 devsmtp dovecot: pop3-login: Login: user=, method=PLAIN, rip=172.20.20.110, lip=172.20.20.222, mpid=3460 Apr 13 11:05:31 devsmtp dovecot: pop3(servermailbox1): Disconnected: Logged out top=5/6281, retr=5/77906, del=0/50, size=778637 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: client in: AUTH#0113#011PLAIN#011service=pop3#011lip=172.20.20.222#011rip=172.20.20.110#011lport=110#011rport=53255#011resp=AHNlcnZlcm1haWxib3gxADEyMzQ1Ng== Apr 13 11:05:31 devsmtp dovecot: auth: Debug: cache(servermailbox1,172.20.20.110): hit: {CRYPT}$6$bpFXtlOP$zczdubFhGyRjJA0PD9lr/QbWAdweg3jThwkxkfCEGgxPxFpualm1ea.8rECmM1UZ0YuTNKWpiGPHwLhys1luy0#011user=servermailbox1#011user=servermailbox1 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: client out: OK#0113#011user=servermailbox1 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: master in: REQUEST#0112057699329#0113397#0113#01132ecf6b93729a90ece98bbb643446ea6 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: userdb-cache(servermailbox1,172.20.20.110): hit: servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: master out: USER#0112057699329#011servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:31 devsmtp dovecot: pop3-login: Login: user=, method=PLAIN, rip=172.20.20.110, lip=172.20.20.222, mpid=3461 Apr 13 11:05:31 devsmtp dovecot: pop3(servermailbox1): Disconnected: Logged out top=0/0, retr=0/0, del=0/50, size=778637 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=pop3#011lip=172.20.20.222#011rip=172.20.20.110#011lport=110#011rport=53261#011resp=AHNlcnZlcm1haWxib3gxADEyMzQ1Ng== Apr 13 11:05:33 devsmtp dovecot: auth: Debug: cache(servermailbox1,172.20.20.110): hit: {CRYPT}$6$bpFXtlOP$zczdubFhGyRjJA0PD9lr/QbWAdweg3jThwkxkfCEGgxPxFpualm1ea.8rECmM1UZ0YuTNKWpiGPHwLhys1luy0#011user=servermailbox1#011user=servermailbox1 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: client out: OK#0111#011user=servermailbox1 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: master in: REQUEST#0113081109505#0113386#0111#01146c7d95e8b31022008fef693a1ef018c Apr 13 11:05:33 devsmtp dovecot: auth: Debug: userdb-cache(servermailbox1,172.20.20.110): hit: servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: master out: USER#0113081109505#011servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:33 devsmtp dovecot: pop3-login: Login: user=, method=PLAIN, rip=172.20.20.110, lip=172.20.20.222, mpid=3462 Apr 13 11:05:33 devsmtp dovecot: pop3(servermailbox1): Disconnected: Logged out top=0/0, retr=0/0, del=1/50, size=778637 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: client in: AUTH#0114#011PLAIN#011service=pop3#011lip=172.20.20.222#011rip=172.20.20.110#011lport=110#011rport=53262#011resp=AHNlcnZlcm1haWxib3gxADEyMzQ1Ng== Apr 13 11:05:33 devsmtp dovecot: auth: Debug: cache(servermailbox1,172.20.20.110): hit: {CRYPT}$6$bpFXtlOP$zczdubFhGyRjJA0PD9lr/QbWAdweg3jThwkxkfCEGgxPxFpualm1ea.8rECmM1UZ0YuTNKWpiGPHwLhys1luy0#011user=servermailbox1#011user=servermailbox1 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: client out: OK#0114#011user=servermailbox1 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: master in: REQUEST#0112632187905#0113397#0114#01132ecf6b93729a90ece98bbb643446ea6 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: userdb-cache(servermailbox1,172.20.20.110): hit: servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: master out: USER#0112632187905#011servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:33 devsmtp dovecot: pop3-login: Login: user=, method=PLAIN, rip=172.20.20.110, lip=172.20.20.222, mpid=3463 Apr 13 11:05:33 devsmtp dovecot: pop3(servermailbox1): Disconnected: Logged out top=0/0, retr=0/0, del=0/49, size=763044 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: client in: AUTH#0115#011PLAIN#011service=pop3#011lip=172.20.20.222#011rip=172.20.20.110#011lport=110#011rport=53263#011resp=AHNlcnZlcm1haWxib3gxADEyMzQ1Ng== Apr 13 11:05:33 devsmtp dovecot: auth: Debug: cache(servermailbox1,172.20.20.110): hit: {CRYPT}$6$bpFXtlOP$zczdubFhGyRjJA0PD9lr/QbWAdweg3jThwkxkfCEGgxPxFpualm1ea.8rECmM1UZ0YuTNKWpiGPHwLhys1luy0#011user=servermailbox1#011user=servermailbox1 From robert at schetterer.org Mon Apr 16 18:48:13 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 16 Apr 2012 17:48:13 +0200 Subject: [Dovecot] problems upgrade 2.0 - 2.1 virtual inbox broken / special use namespace In-Reply-To: <4F8AC473.7030307@schetterer.org> References: <4F8AC473.7030307@schetterer.org> Message-ID: <4F8C3F3D.3030103@schetterer.org> Am 15.04.2012 14:52, schrieb Robert Schetterer: > Hi Timo > > the virtual setup got broke after update to 2.1 > ( its still working under 2.0 ) on other loadbalanced server > > namespace default { > separator = / > prefix = "" > list = yes > subscriptions = yes > hidden = no > } > > namespace virtual { > prefix = "virtual/" > separator = / > location = virtual:/etc/dovecot/virtual:LAYOUT=maildir++:INDEX=~/virtual > hidden = yes > list = no > subscriptions= no > } > > namespace real { > prefix = "RealMails/" > separator = / > list = no > hidden = yes > } > > > after upgrade i have > > Error: Couldn't open INBOX: Virtual mailbox open failed because of > mailbox /RealMails/: Invalid mailbox name > > --------------------- ok i identified the problem but didnt find a working other solution +RealMails/ << not longer work in 2.1 +RealMails/* -RealMails/Trash -RealMails/Trash/* -RealMails/Sent -RealMails/Sent/* -RealMails/Templates -RealMails/Templates/* -RealMails/Drafts -RealMails/Drafts/* -RealMails/Archives -RealMails/Archives/* all > > also > > i wanted > to include > > namespace inbox { > > #mailbox name { > # auto=create will automatically create this mailbox. > # auto=subscribe will both create and subscribe to the mailbox. > #auto = no > > # Space separated list of IMAP SPECIAL-USE attributes as specified by > # RFC 6154: \All \Archive \Drafts \Flagged \Junk \Sent \Trash > #special_use = > #} > > # These mailboxes are widely used and could perhaps be created > automatically: > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Trash { > special_use = \Trash > } > > # For \Sent mailboxes there are two widely used names. We'll mark both of > # them as \Sent. User typically deletes one of them if duplicates are > created. > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > > # If you have a virtual "All messages" mailbox: > mailbox virtual/All { > special_use = \All > } > > # If you have a virtual "Flagged" mailbox: > mailbox virtual/Flagged { > special_use = \Flagged > } > } > > but i found no working namespace combination with virtual -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Mon Apr 16 22:01:26 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 16 Apr 2012 21:01:26 +0200 Subject: [Dovecot] problems upgrade 2.0 - 2.1 virtual inbox broken / special use namespace /solved In-Reply-To: <4F8C3F3D.3030103@schetterer.org> References: <4F8AC473.7030307@schetterer.org> <4F8C3F3D.3030103@schetterer.org> Message-ID: <4F8C6C86.7000707@schetterer.org> Am 16.04.2012 17:48, schrieb Robert Schetterer: > Am 15.04.2012 14:52, schrieb Robert Schetterer: >> Hi Timo >> >> the virtual setup got broke after update to 2.1 >> ( its still working under 2.0 ) on other loadbalanced server >> >> namespace default { >> separator = / >> prefix = "" >> list = yes >> subscriptions = yes >> hidden = no >> } >> >> namespace virtual { >> prefix = "virtual/" >> separator = / >> location = virtual:/etc/dovecot/virtual:LAYOUT=maildir++:INDEX=~/virtual >> hidden = yes >> list = no >> subscriptions= no >> } >> >> namespace real { >> prefix = "RealMails/" >> separator = / >> list = no >> hidden = yes >> } >> >> >> after upgrade i have >> >> Error: Couldn't open INBOX: Virtual mailbox open failed because of >> mailbox /RealMails/: Invalid mailbox name >> >> --------------------- > > ok i identified the problem > but didnt find a working other solution > > > > +RealMails/ << not longer work in 2.1 > +RealMails/* > -RealMails/Trash > -RealMails/Trash/* > -RealMails/Sent > -RealMails/Sent/* > -RealMails/Templates > -RealMails/Templates/* > -RealMails/Drafts > -RealMails/Drafts/* > -RealMails/Archives > -RealMails/Archives/* > all > > > >> >> also >> >> i wanted >> to include >> >> namespace inbox { >> >> #mailbox name { >> # auto=create will automatically create this mailbox. >> # auto=subscribe will both create and subscribe to the mailbox. >> #auto = no >> >> # Space separated list of IMAP SPECIAL-USE attributes as specified by >> # RFC 6154: \All \Archive \Drafts \Flagged \Junk \Sent \Trash >> #special_use = >> #} >> >> # These mailboxes are widely used and could perhaps be created >> automatically: >> mailbox Drafts { >> special_use = \Drafts >> } >> mailbox Junk { >> special_use = \Junk >> } >> mailbox Trash { >> special_use = \Trash >> } >> >> # For \Sent mailboxes there are two widely used names. We'll mark both of >> # them as \Sent. User typically deletes one of them if duplicates are >> created. >> mailbox Sent { >> special_use = \Sent >> } >> mailbox "Sent Messages" { >> special_use = \Sent >> } >> >> # If you have a virtual "All messages" mailbox: >> mailbox virtual/All { >> special_use = \All >> } >> >> # If you have a virtual "Flagged" mailbox: >> mailbox virtual/Flagged { >> special_use = \Flagged >> } >> } >> >> but i found no working namespace combination with virtual > > for small test this setting should work with 2.1 and special use namespace notice default namespace virtual example in the wiki must change to namespace inbox so the i.e sql lookup must change CASE '%s' WHEN 'pop3' THEN NULL ELSE 'yes' END AS 'namespace/inbox /inbox', \ CASE '%s' WHEN 'pop3' THEN 'yes' ELSE NULL END AS 'namespace/virtual/inbox', \ 10-mail.conf .. namespace inbox { prefix = separator = / list = yes } 15-mailboxes.conf as you like --------------------------------- the virtual fix at my setup is RealMails +RealMails/* -RealMails/Trash -RealMails/Trash/* -RealMails/Sent -RealMails/Sent/* -RealMails/Templates -RealMails/Templates/* -RealMails/Drafts -RealMails/Drafts/* -RealMails/Archives -RealMails/Archives/* all -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From dlie76 at yahoo.com.au Tue Apr 17 05:31:56 2012 From: dlie76 at yahoo.com.au (Daminto Lie) Date: Mon, 16 Apr 2012 19:31:56 -0700 (PDT) Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> <4F86DCA7.1060806@talex.pl> <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> Message-ID: <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> Thank you folks for your replies. I realised that the dovecot 1.2.9 that I installed via ubuntu synaptic package manager has already come with sieve plugins. After making some modifications on the dovecot.conf file, I went to download and install avelsieve 1.0.1 (stable version) onto the squirrelmail. Then, I run in squirrelmail conf.pl and added the plugin. I logged in and could see "Filter" option right on the top header row. I went to click it and nothing came up. I checked in the mail.log but did not see any errors there. Here is my snapshot of dovecot configuration # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS ext4 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap listen: *:143 ssl: no disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login verbose_proctitle: yes first_valid_uid: 1001 last_valid_uid: 600 mail_privileged_group: mail mail_location: maildir:/var/vmail/%u/Maildir mbox_write_locks: fcntl dotlock imap_client_workarounds: outlook-idle delay-newmail netscape-eoh tb-extra-mailbox-sep lda: ? postmaster_address: postmaster at ourcompany.com ? mail_plugins: sieve ? quota_full_tempfail: yes ? deliver_log_format: msgid=%m: %$ ? sendmail_path: /usr/lib/sendmail ? rejection_reason: Your message to <%t> was automatically rejected:%n%r ? auth_socket_path: /var/run/dovecot-auth-master auth default: ? mechanisms: plain login ? username_format: %Lu ? verbose: yes ? debug: yes ? debug_passwords: yes ? passdb: ??? driver: pam ? passdb: ??? driver: ldap ??? args: /etc/dovecot/dovecot-ldap.conf ? userdb: ??? driver: passwd ? userdb: ??? driver: static ??? args: uid=1001 gid=1001 home=/var/vmail/%u allow_all_users=yes ? socket: ??? type: listen ??? client: ????? path: /var/spool/postfix/private/auth ????? mode: 432 ????? user: postfix ????? group: mail ??? master: ????? path: /var/run/dovecot-auth-master ????? mode: 432 ????? user: vmail ????? group: vmail plugin: ? sieve: /var/vmail/%u/.sieve Also, I have restarted dovecot after making changes to the dovecot.conf. Any help would be very much appreciated. Thank you ? ________________________________ From: Artur Zaprza?a To: Dovecot Mailing List Cc: Daminto Lie Sent: Thursday, 12 April 2012 11:46 PM Subject: Re: [Dovecot] vacation plugins for squirrelmail Daminto Lie wrote: > Hi, > > I am afraid I have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. > > Any help would be very much appreciated. > > Thank you > I'm using Avelsieve 1.9.9 with a set of my own bugfixes: http://email.uoa.gr/avelsieve/ -- Talex Sp??ka Akcyjna z siedzib? w Poznaniu adres: ul. Karpia 27d, 61-619 Pozna? NIP 782-00-21-045 zarejestrowana w S?dzie Rejonowym Pozna? ? Nowe Miasto i Wilda w Poznaniu VIII Wydzia? Gospodarczy - KRS pod nr 000048779 kapita? zak?adowy: 3.000.092,00 PLN (w ca?o?ci wp?acony) Uwaga: Niniejsza wiadomo??, w szczeg?lno?ci jej tre?? oraz za??czniki, mo?e by? poufna. W przypadku, gdy nie jest Pan/Pani zamierzonym jej adresatem, informujemy, ?e wszelkie rozpowszechnianie, dystrybucja lub powielanie powy?szej wiadomo?ci jest zabronione. Jednocze?nie prosimy o powiadomienie nadawcy oraz niezw?oczne usuni?cie powy?szej wiadomo?ci wraz z za??cznikami. Dzi?kujemy, Talex S.A. w Poznaniu. Confidentiality Notice: This email, particularly its content and any attached files, may be confidential. If you are not an intended recipient, any disclosure, distribution and reproduction of this message is prohibited. In this case please notify the sender immediately and then delete this message and any attachments. Thank you, Talex S.A., Poznan. From dovecot-list at mohtex.net Tue Apr 17 06:25:44 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Tue, 17 Apr 2012 10:25:44 +0700 Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> <4F86DCA7.1060806@talex.pl> <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> Message-ID: <4F8CE2B8.3080801@mohtex.net> Show us your avelsieve-config in your-squirrelmail-dir/plugins/avelsieve/config/config.php Daminto Lie wrote the following on 17.04.2012 09:31: > Thank you folks for your replies. > > > I realised that the dovecot 1.2.9 that I installed via ubuntu synaptic package manager has already come with sieve plugins. After making some modifications on the dovecot.conf file, I went to download and install avelsieve 1.0.1 (stable version) onto the squirrelmail. Then, I run in squirrelmail conf.pl and added the plugin. I logged in and could see "Filter" option right on the top header row. I went to click it and nothing came up. I checked in the mail.log but did not see any errors there. > > Here is my snapshot of dovecot configuration > # 1.2.9: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS ext4 > log_timestamp: %Y-%m-%d %H:%M:%S > protocols: imap > listen: *:143 > ssl: > no > disable_plaintext_auth: no > login_dir: /var/run/dovecot/login > login_executable: /usr/lib/dovecot/imap-login > verbose_proctitle: yes > first_valid_uid: 1001 > last_valid_uid: 600 > mail_privileged_group: mail > mail_location: maildir:/var/vmail/%u/Maildir > mbox_write_locks: fcntl dotlock > imap_client_workarounds: outlook-idle delay-newmail netscape-eoh tb-extra-mailbox-sep > lda: > postmaster_address: postmaster at ourcompany.com > mail_plugins: sieve > quota_full_tempfail: yes > deliver_log_format: msgid=%m: %$ > sendmail_path: /usr/lib/sendmail > rejection_reason: Your message to<%t> was automatically rejected:%n%r > auth_socket_path: /var/run/dovecot-auth-master > auth default: > mechanisms: plain login > username_format: %Lu > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: > pam > passdb: > driver: ldap > args: /etc/dovecot/dovecot-ldap.conf > userdb: > driver: passwd > userdb: > driver: static > args: uid=1001 gid=1001 home=/var/vmail/%u allow_all_users=yes > socket: > type: listen > client: > path: /var/spool/postfix/private/auth > mode: 432 > user: postfix > group: mail > master: > path: /var/run/dovecot-auth-master > mode: 432 > user: vmail > group: vmail > plugin: > sieve: /var/vmail/%u/.sieve > > Also, I have restarted dovecot after making changes to the dovecot.conf. > > Any help would be very much appreciated. > > Thank you > > > > > ________________________________ > From: Artur Zaprza?a > To: Dovecot Mailing List > Cc: Daminto Lie > Sent: Thursday, 12 April 2012 11:46 PM > Subject: Re: [Dovecot] vacation plugins for squirrelmail > > Daminto Lie wrote: >> Hi, >> >> I am afraid I > have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. >> Any help would be very much appreciated. >> >> Thank you >> > I'm using Avelsieve 1.9.9 with a set of my own bugfixes: > http://email.uoa.gr/avelsieve/ > > From dlie76 at yahoo.com.au Tue Apr 17 09:14:57 2012 From: dlie76 at yahoo.com.au (Daminto Lie) Date: Mon, 16 Apr 2012 23:14:57 -0700 (PDT) Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <4F8CE2B8.3080801@mohtex.net> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> <4F86DCA7.1060806@talex.pl> <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> <4F8CE2B8.3080801@mohtex.net> Message-ID: <1334643297.87733.YahooMailNeo@web113417.mail.gq1.yahoo.com> Thank you for your reply. Here it is as follows if(!defined('AVELSIEVE_DEBUG')) { ??? define('AVELSIEVE_DEBUG', 0); } global $avelsieve_backend; $avelsieve_backend = 'File'; /*global $sieveport; $sieveport = 2000;*/ global $sieve_preferred_sasl_mech; $sieve_preferred_sasl_mech = 'PLAIN'; global $avelsieve_disabletls; $avelsieve_disabletls = false; global $avelsieve_file_backend_options, $data_dir, $username; $avelsieve_file_backend_options = array( ??? 'avelsieve_default_file' => "$data_dir/$username.sievesource" ); /*global $avelsieve_oldcyrus; $avelsieve_oldcyrus = true;*/ global $avelsieve_enable_envelope_auth; $avelsieve_enable_envelope_auth = true; global $avelsieve_custom_sieve_implementation; $avelsieve_custom_sieve_implementation = ''; global $avelsieve_striproot; $avelsieve_striproot = ''; global $avelsieve_hardcoded_capabilities; $avelsieve_hardcoded_capabilities = array( ??? 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' ); /*global $avelsieve_imapproxymode, $avelsieve_imapproxyserv; $avelsieve_imapproxymode = false; $avelsieve_imapproxyserv = array( ??? 'localhost' => 'imap.example.org' );*/ global $avelsieve_ldapuserdatamode; $avelsieve_ldapuserdatamode = false; /*global $avelsieve_cyrusadmins_map; $avelsieve_cyrusadmins_map = array( ??? 'cyrusimap' => 'cyrussieve' );*/ $conservative = false; $useimages = true; global $translate_return_msgs; $translate_return_msgs = false; $imagetheme = 'famfamfam'; //$imagetheme = 'bluecurve_24x24'; //$imagetheme = 'bluecurve_16x16'; global $startitems; $startitems = 3; global $maxitems; $maxitems = 10; global $headers; $headers = array( ?'From', 'To', 'Cc', 'Bcc', 'Subject', 'Reply-To', 'Sender', 'List-Id', ?'MailingList', 'Mailing-List', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', ?'Resent-From',? 'Resent-To', 'X-Mailer', 'X-Mailing-List', ?'X-Spam-Flag', 'X-Spam-Status', ?'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', ?'Return-Path', 'Received', 'Auto-Submitted', ?'X-Spam-Flag', 'X-Spam-Status','X-Spam-Tests' ?); global $notifymethods; $notifymethods = array( 'mailto', 'sms' ); //$notifymethods = false; // $disable_avelsieve_capabilities = array("notify"); global $disable_avelsieve_capabilities; $disable_avelsieve_capabilities = array(); global $avelsieveheaderlink; $avelsieveheaderlink = true; global $avelsieve_default_mode; $avelsieve_default_mode = 'terse'; global $avelsieve_enable_rules; $avelsieve_enable_rules = array(); global $avelsieve_rules_settings; $avelsieve_rules_settings = array(); foreach($avelsieve_enable_rules as $r) { ??? if(file_exists(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php')) { ??????? require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php'); ??? } else { ??????? require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.default.php'); ??? } } $avelsieve_spam_highlight_enable = false; Hope this help you to help me. Thank you ________________________________ From: Tamsy To: "dovecot at dovecot.org" Sent: Tuesday, 17 April 2012 1:25 PM Subject: Re: [Dovecot] vacation plugins for squirrelmail Show us your avelsieve-config in your-squirrelmail-dir/plugins/avelsieve/config/config.php Daminto Lie wrote the following on 17.04.2012 09:31: > Thank you folks for your replies. > > > I realised that the dovecot 1.2.9 that I installed via ubuntu synaptic package manager has already come with sieve plugins. After making some modifications on the dovecot.conf file, I went to download and install avelsieve 1.0.1 (stable version) onto the squirrelmail. Then, I run in squirrelmail conf.pl and added the plugin. I logged in and could see "Filter" option right on the top header row. I went to click it and nothing came up. I checked in the mail.log but did not see any errors there. > > Here is my snapshot of dovecot configuration > # 1.2.9: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS ext4 > log_timestamp: %Y-%m-%d %H:%M:%S > protocols: imap > listen: *:143 > ssl: >? no > disable_plaintext_auth: no > login_dir: /var/run/dovecot/login > login_executable: /usr/lib/dovecot/imap-login > verbose_proctitle: yes > first_valid_uid: 1001 > last_valid_uid: 600 > mail_privileged_group: mail > mail_location: maildir:/var/vmail/%u/Maildir > mbox_write_locks: fcntl dotlock > imap_client_workarounds: outlook-idle delay-newmail netscape-eoh tb-extra-mailbox-sep > lda: >? ? postmaster_address: postmaster at ourcompany.com >? ? mail_plugins: sieve >? ? quota_full_tempfail: yes >? ? deliver_log_format: msgid=%m: %$ >? ? sendmail_path: /usr/lib/sendmail >? ? rejection_reason: Your message to<%t>? was automatically rejected:%n%r >? ? auth_socket_path: /var/run/dovecot-auth-master > auth default: >? ? mechanisms: plain login >? ? username_format: %Lu >? ? verbose: yes >? ? debug: yes >? ? debug_passwords: yes >? ? passdb: >? ? ? driver: >? pam >? ? passdb: >? ? ? driver: ldap >? ? ? args: /etc/dovecot/dovecot-ldap.conf >? ? userdb: >? ? ? driver: passwd >? ? userdb: >? ? ? driver: static >? ? ? args: uid=1001 gid=1001 home=/var/vmail/%u allow_all_users=yes >? ? socket: >? ? ? type: listen >? ? ? client: >? ? ? ? path: /var/spool/postfix/private/auth >? ? ? ? mode: 432 >? ? ? ? user: postfix >? ? ? ? group: mail >? ? ? master: >? ? ? ? path: /var/run/dovecot-auth-master >? ? ? ? mode: 432 >? ? ? ? user: vmail >? ? ? ? group: vmail > plugin: >? ? sieve: /var/vmail/%u/.sieve > > Also, I have restarted dovecot after making changes to the dovecot.conf. > > Any help would be very much appreciated. > > Thank you > >? > > > ________________________________ >? From: Artur Zaprza?a > To: Dovecot Mailing List > Cc: Daminto Lie > Sent: Thursday, 12 April 2012 11:46 PM > Subject: Re: [Dovecot] vacation plugins for squirrelmail > > Daminto Lie wrote: >> Hi, >> >> I am afraid I >? have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. >> Any help would be very much appreciated. >> >> Thank you >> > I'm using Avelsieve 1.9.9 with a set of my own bugfixes: > http://email.uoa.gr/avelsieve/ > > From dovecot-list at mohtex.net Tue Apr 17 09:51:11 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Tue, 17 Apr 2012 13:51:11 +0700 Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <1334643297.87733.YahooMailNeo@web113417.mail.gq1.yahoo.com> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> <4F86DCA7.1060806@talex.pl> <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> <4F8CE2B8.3080801@mohtex.net> <1334643297.87733.YahooMailNeo@web113417.mail.gq1.yahoo.com> Message-ID: <4F8D12DF.9020508@mohtex.net> Hi Daminto, That looks pretty much unconfigured. No wonder nothing is coming up. I have attached the avelsieve config file we are using on one of our domains. Don't copy it blindly. At least you should configure $sieveport to the port you are using on your server for Sieve. And put your domain name where it reads YOURDOMAINNAMEHERE.com. For the arrays at $headers you should tune the filters you are using with your headers. Hope this helps (working fine with our domain and Squirrel-Mail). Rgds Tamsy Daminto Lie wrote the following on 17.04.2012 13:14: > Thank you for your reply. > > Here it is as follows > > > if(!defined('AVELSIEVE_DEBUG')) { > define('AVELSIEVE_DEBUG', 0); > } > > global $avelsieve_backend; > $avelsieve_backend = 'File'; > > /*global $sieveport; > $sieveport = 2000;*/ > > global $sieve_preferred_sasl_mech; > $sieve_preferred_sasl_mech = 'PLAIN'; > > global $avelsieve_disabletls; > $avelsieve_disabletls = false; > > global $avelsieve_file_backend_options, $data_dir, $username; > $avelsieve_file_backend_options = array( > 'avelsieve_default_file' => "$data_dir/$username.sievesource" > ); > > /*global $avelsieve_oldcyrus; > $avelsieve_oldcyrus = true;*/ > > global $avelsieve_enable_envelope_auth; > $avelsieve_enable_envelope_auth = true; > > global $avelsieve_custom_sieve_implementation; > $avelsieve_custom_sieve_implementation = ''; > > global $avelsieve_striproot; > $avelsieve_striproot = ''; > > global $avelsieve_hardcoded_capabilities; > $avelsieve_hardcoded_capabilities = array( > 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' > ); > > /*global $avelsieve_imapproxymode, $avelsieve_imapproxyserv; > $avelsieve_imapproxymode = false; > $avelsieve_imapproxyserv = array( > 'localhost' => 'imap.example.org' > );*/ > > global $avelsieve_ldapuserdatamode; > $avelsieve_ldapuserdatamode = false; > > /*global $avelsieve_cyrusadmins_map; > $avelsieve_cyrusadmins_map = array( > 'cyrusimap' => 'cyrussieve' > );*/ > > $conservative = false; > > $useimages = true; > > global $translate_return_msgs; > $translate_return_msgs = false; > > $imagetheme = 'famfamfam'; > //$imagetheme = 'bluecurve_24x24'; > //$imagetheme = 'bluecurve_16x16'; > > global $startitems; > $startitems = 3; > > global $maxitems; > $maxitems = 10; > > global $headers; > $headers = array( > 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Reply-To', 'Sender', 'List-Id', > 'MailingList', 'Mailing-List', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', > 'Resent-From', 'Resent-To', 'X-Mailer', 'X-Mailing-List', > 'X-Spam-Flag', 'X-Spam-Status', > 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', > 'Return-Path', 'Received', 'Auto-Submitted', > 'X-Spam-Flag', 'X-Spam-Status','X-Spam-Tests' > ); > > global $notifymethods; > $notifymethods = array( > 'mailto', 'sms' > ); > > //$notifymethods = false; > > // $disable_avelsieve_capabilities = array("notify"); > global $disable_avelsieve_capabilities; > $disable_avelsieve_capabilities = array(); > > global $avelsieveheaderlink; > $avelsieveheaderlink = true; > > global $avelsieve_default_mode; > $avelsieve_default_mode = 'terse'; > > global $avelsieve_enable_rules; > $avelsieve_enable_rules = array(); > > global $avelsieve_rules_settings; > $avelsieve_rules_settings = array(); > foreach($avelsieve_enable_rules as $r) { > if(file_exists(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php')) { > require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php'); > } else { > require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.default.php'); > } > } > > $avelsieve_spam_highlight_enable = false; > > > Hope this help you to help me. > > Thank you > > > > > ________________________________ > From: Tamsy > To: "dovecot at dovecot.org" > Sent: Tuesday, 17 April 2012 1:25 PM > Subject: Re: [Dovecot] vacation plugins for squirrelmail > > Show us your avelsieve-config in > your-squirrelmail-dir/plugins/avelsieve/config/config.php > > > Daminto Lie wrote the following on 17.04.2012 09:31: >> Thank you folks for your replies. >> >> >> I realised that the dovecot 1.2.9 that I installed via ubuntu synaptic package manager has already come with sieve plugins. After making some modifications on the dovecot.conf file, I went to download and install avelsieve 1.0.1 (stable version) onto the squirrelmail. Then, I run in squirrelmail conf.pl and added the plugin. I logged in and could see "Filter" option right on the top header row. I went to click it and nothing came up. I checked in the mail.log but did not see any errors there. >> >> Here is my snapshot of dovecot configuration >> # 1.2.9: /etc/dovecot/dovecot.conf >> # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS ext4 >> log_timestamp: %Y-%m-%d %H:%M:%S >> protocols: imap >> listen: *:143 >> ssl: >> no >> disable_plaintext_auth: no >> login_dir: /var/run/dovecot/login >> login_executable: /usr/lib/dovecot/imap-login >> verbose_proctitle: yes >> first_valid_uid: 1001 >> last_valid_uid: 600 >> mail_privileged_group: mail >> mail_location: maildir:/var/vmail/%u/Maildir >> mbox_write_locks: fcntl dotlock >> imap_client_workarounds: outlook-idle delay-newmail netscape-eoh tb-extra-mailbox-sep >> lda: >> postmaster_address: postmaster at ourcompany.com >> mail_plugins: sieve >> quota_full_tempfail: yes >> deliver_log_format: msgid=%m: %$ >> sendmail_path: /usr/lib/sendmail >> rejection_reason: Your message to<%t> was automatically rejected:%n%r >> auth_socket_path: /var/run/dovecot-auth-master >> auth default: >> mechanisms: plain login >> username_format: %Lu >> verbose: yes >> debug: yes >> debug_passwords: yes >> passdb: >> driver: >> pam >> passdb: >> driver: ldap >> args: /etc/dovecot/dovecot-ldap.conf >> userdb: >> driver: passwd >> userdb: >> driver: static >> args: uid=1001 gid=1001 home=/var/vmail/%u allow_all_users=yes >> socket: >> type: listen >> client: >> path: /var/spool/postfix/private/auth >> mode: 432 >> user: postfix >> group: mail >> master: >> path: /var/run/dovecot-auth-master >> mode: 432 >> user: vmail >> group: vmail >> plugin: >> sieve: /var/vmail/%u/.sieve >> >> Also, I have restarted dovecot after making changes to the dovecot.conf. >> >> Any help would be very much appreciated. >> >> Thank you >> >> >> >> >> ________________________________ >> From: Artur Zaprza?a >> To: Dovecot Mailing List >> Cc: Daminto Lie >> Sent: Thursday, 12 April 2012 11:46 PM >> Subject: Re: [Dovecot] vacation plugins for squirrelmail >> >> Daminto Lie wrote: >>> Hi, >>> >>> I am afraid I >> have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. >>> Any help would be very much appreciated. >>> >>> Thank you >>> >> I'm using Avelsieve 1.9.9 with a set of my own bugfixes: >> http://email.uoa.gr/avelsieve/ >> >> From dovecot-list at mohtex.net Tue Apr 17 10:00:23 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Tue, 17 Apr 2012 14:00:23 +0700 Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <1334643297.87733.YahooMailNeo@web113417.mail.gq1.yahoo.com> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> <4F86DCA7.1060806@talex.pl> <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> <4F8CE2B8.3080801@mohtex.net> <1334643297.87733.YahooMailNeo@web113417.mail.gq1.yahoo.com> Message-ID: <4F8D1507.7060609@mohtex.net> Seems not to be attached to my former mail. Here it comes again, inline: avelsieve config.php ############################### * @copyright 2002-2004 Alexandros Vellis * @package plugins * @subpackage avelsieve */ /** * Debug Mode. Enable this (change to 1) if you need to send a bug report, * or to 2 or 3 if you are a developer! * * Valid values: * 0 = No debugging output * 1 = Normal debugging output * 2 = Firebug-enhanced debugging output * 3 = Enhanced debugging output and no Sieve capabilities checks - enables * all UI features! */ if(!defined('AVELSIEVE_DEBUG')) { define('AVELSIEVE_DEBUG', 1); } /* ======================================================================== */ /* =================== IMAP Server / SIEVE Setup ========================= */ /* ======================================================================== */ /** @var string Backend to use. Available backends are: * 'ManageSieve': Uses the ManageSieve protocol. e.g. Cyrus * 'File': Writes files straight to disk. e.g. Exim4, Dovecot LDA. */ global $avelsieve_backend; $avelsieve_backend = 'ManageSieve'; /* ======================================================================== */ /* =================== ManageSieve Backend Options ======================== */ /* ======================================================================== */ /* Port where ManageSieve servers (e.g. Cyrus IMAP's timsieved) listens to. * Default is 2000. Note that another possible port that your server might * listen to, is 4190, which is now the official (IANA-assigned) port for * Sieve. */ global $sieveport; $sieveport = 4190; /** * @var string Space separated list of preferred SASL mechanisms for the * authentication to ManageSieve server. e.g. "PLAIN DIGEST-MD5";*/ global $sieve_preferred_sasl_mech; $sieve_preferred_sasl_mech = 'PLAIN'; /** * @var boolean Disable STARTTLS for ManageSieve. You can set this to true, * if you do not wish to use encryption via TLS mechanisms (i.e. the server * is not configured properly, or this is a local connection and TLS is not * needed. * Note that STARTTLS is supported only in PHP5+. In PHP4 this option will * have no effect and STARTTLS will be disabled anyway. */ global $avelsieve_disabletls; $avelsieve_disabletls = false; /* ======================================================================== */ /* ======================= File Backend Options =========================== */ /* ======================================================================== */ global $avelsieve_file_backend_options, $data_dir, $username; $avelsieve_file_backend_options = array( 'avelsieve_default_file' => "$data_dir/$username.sievesource" ); /* ======================================================================== */ /* ====== Implementation- and Server-Specific Options ==================== */ /* ======================================================================== */ /* In Cyrus 2.3+, the notification action is a bit more complex than the * others. The oldcyrus variable is for supporting the partially implemented * notify extension implementation of Cyrus < 2.3. If you have Cyrus < 2.3, * just set this to true. * * This only changes the informational / help text displayed in avelsieve. * * Cyrus < 2.3 : $from$, $env-from$, $subject$ * Cyrus 2.3+ : $from$, $env-from$, $subject$, $text$, $text[n]$ */ global $avelsieve_oldcyrus; $avelsieve_oldcyrus = true; /* If you have Cyrus with an lmtpd that can understand the "auth" argument to * the :envelope test as the SMTP/LMTP auth, or any other Sieve implementation, * then you can enable this to provide this functionality to the user. * * This was not clarified in the base spec of RFC 3028. It will be done * correctly in a new version of Cyrus, based on a new draft / spec. */ global $avelsieve_enable_envelope_auth; $avelsieve_enable_envelope_auth = true; /* Some Implementations of Sieve need certain things in order to operate * correctly. If you use any of the following server implementations, you MUST * set this variable to the corresponding value for the filtering to work * correctly. * Valid values are: * - Any RFC3028-mostly-compatible implementation: '' (empty). * - Exim MTA: 'exim' * - MFL (as supported by mvmf): 'mfl' */ global $avelsieve_custom_sieve_implementation; $avelsieve_custom_sieve_implementation = ''; /* For delivery agents that don't know how to handle some mailbox prefixes, you * can enable this option. Example for Dovecot LDA: 'INBOX.'; */ global $avelsieve_striproot; $avelsieve_striproot = 'INBOX.'; /* If the backend does not support capabilities reporting, such as the File * Backend, then you should define which capabilities are used by the server * implementation. * * The following are the capabilities supported by Exim4 as of Exim version * 4.60, according to README.SIEVE. You can change them if a new version of * Exim provides more functionality: * 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' * * The following are the capabilities that are suported by MFM, according to: * http://www.mvmf.org/mfl/language.shtml#sieve * 'envelope', 'fileinto', 'reject', 'relational', 'subaddress', 'regex', * 'editheader', 'copy', 'vacation', 'comparator-i;ascii-casemap', * 'comparator-i;octet' * * The following capabilities have been reported to work with Dovecot LDA: * 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric', * 'imapflags', 'subaddress','relational','regex' * */ /* global $avelsieve_hardcoded_capabilities; * $avelsieve_hardcoded_capabilities = array( * 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' * ); */ global $avelsieve_hardcoded_capabilities; $avelsieve_hardcoded_capabilities = array( 'fileinto', 'reject', 'envelope', 'encoded-character', 'vacation', 'subaddress', 'comparator-i;ascii-numeric', 'relational', 'regex', 'imap4flags', 'copy', 'include', 'variables', 'body', 'enotify', 'environment', 'mailbox', 'date' ); /** @var boolean Enable ImapProxy mode. * If you use imapproxy, because imapproxy cannot understand and proxy the * SIEVE protocol, you must connect to the SIEVE daemon (usually on the IMAP * server) itself. So you need to set $imapproxymode to true, and define a * mapping, from the imapproxy host (usually localhost) to your real IMAP * server (usually the same that is defined on Imapproxy's configuration). * * This will not work if you use a perdition-style proxy, where different users * go to different IMAP servers; it applies mostly to people running imapproxy * for speed and want a quick hack. */ global $avelsieve_imapproxymode, $avelsieve_imapproxyserv; $avelsieve_imapproxymode = false; $avelsieve_imapproxyserv = array( 'localhost' => 'YOURDOMAINNAMEHERE.com' ); /** @var boolean Ldapuserdata mode: Gets user's email addresses (including * mailAlternate & mailAuthorized) from LDAP Prefs Backend plugin's cache */ global $avelsieve_ldapuserdatamode; $avelsieve_ldapuserdatamode = false; /** @var array Map of cyrus administrator users, for proxy authentication */ global $avelsieve_cyrusadmins_map; $avelsieve_cyrusadmins_map = array( 'cyrusimap' => 'cyrussieve' ); /* ======================================================================== */ /* =============== Avelsieve Interface / Behavior Setup ================== */ /* ======================================================================== */ /* Be conservative to our updates on the SIEVE server? If true, a button * entitled "Save Changes" will appear, which will give the user the * functionality to register her changes. 'false' is recommended. */ $conservative = false; /* Use images for the move up / down, delete rule buttons and STOP? */ $useimages = true; /* Translate the messages returned by the "Reject" and "Vacation" actions? The * default behaviour since 0.9 is not to translate them. Change to true if in * an intranet environment or in a same-language environment. */ global $translate_return_msgs; $translate_return_msgs = false; /* Theme to use for the images. A directory with the same name must exist under * plugins/avelsieve/$imagetheme, that contains the files: up.png, down.png, * del.png, dup.png, edit.png, top.png, bottom.png. */ $imagetheme = 'famfamfam'; //$imagetheme = 'bluecurve_24x24'; //$imagetheme = 'bluecurve_16x16'; /* Number of items to display _initially_, when displaying the header match * rule */ global $startitems; $startitems = 3; /* Maximum number of items to allow in one header match rule. */ global $maxitems; $maxitems = 10; /* Headers to display in listbox widget, when adding a new header rule. */ /* global $headers; * $headers = array( * 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Reply-To', 'Sender', 'List-Id', * 'MailingList', 'Mailing-List', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', * 'Resent-From', 'Resent-To', 'X-Mailer', 'X-Mailing-List', * 'X-Spam-Flag', 'X-Spam-Status', * 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', * 'Return-Path', 'Received', 'Auto-Submitted', * 'X-Spam-Flag', 'X-Spam-Status','X-Spam-Tests' * ); */ global $headers; $headers = array( 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Sender', 'Reply-To', 'List-Id', 'MailingList', 'Mailing-List', 'X-DKIM-Authentication-Results', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', 'Resent-From', 'Resent-To', 'X-Mailer', 'X-Mailing-List', 'X-SpamTest-Status-Extended', 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', 'Return-Path', 'Received', 'Auto-Submitted', ); /* Available :method's for the :notify extension (if applicable) */ global $notifymethods; $notifymethods = array( 'mailto', 'sms' ); /* use the value "false" if you want to provide a simple input box so that * users can edit the method themselves : */ //$notifymethods = false; /* Capabilities to disable. If you would like to force avelsieve not to display * certain features, even though there _is_ a capability for them in Sieve * server, you should specify these here. For instance, if you would * like to disable the notify extension, even though ManageSieve advertises it, * you should add 'notify' in this array: $force_disable_avelsieve_capability = * array("notify");. This will still leave the defined feature on, and if the * user can upload her own scripts then she can use that feature; this option * just disables the GUI of it. Leave as-is (empty array) if you do not need * that. * * Look in $implemented_capabilities array in include/constants.inc.php for * valid values */ // $disable_avelsieve_capabilities = array("notify"); global $disable_avelsieve_capabilities; $disable_avelsieve_capabilities = array(); /* Display Filters link in the top Squirrelmail header? */ global $avelsieveheaderlink; $avelsieveheaderlink = true; /* Default rules table display mode, one of 'verbose' or 'terse' */ global $avelsieve_default_mode; $avelsieve_default_mode = 'terse'; /* ======================================================================== */ /* ========================= Custom rules Configuration =================== */ /* ======================================================================== */ /** * @var array * List of additional rules to enable. This must be the numeric id of the * rule. * * 10: Spam Rule (as existed in Avelsieve) * 11: Junk-Mail Rule (new-style SPAM Rule, only one exists per Sieve script) * 12: Global Whitelist (only one exists per Sieve script) * 13: Custom Sieve Code (allows users to enter whatever Sieve code they like - * Alpha/Experimental) * * Example: * array(11, 12); */ global $avelsieve_enable_rules; $avelsieve_enable_rules = array(); global $avelsieve_rules_settings; $avelsieve_rules_settings = array(); foreach($avelsieve_enable_rules as $r) { if(file_exists(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php')) { require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php'); } else { require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.default.php'); } } /* Please keep the following setting false; it is alpha + needs Squirrelmail * to be patched in three or four places. */ $avelsieve_spam_highlight_enable = false; ############################### Hi Daminto, That looks pretty much unconfigured. No wonder nothing is coming up. I have attached the avelsieve config file we are using on one of our domains. Don't copy it blindly. At least you should configure $sieveport to the port you are using on your server for Sieve. And put your domain name where it reads YOURDOMAINNAMEHERE.com. For the arrays at $headers you should tune the filters you are using with your headers. Hope this helps (working fine with our domain and Squirrel-Mail). Rgds Tamsy Daminto Lie wrote the following on 17.04.2012 13:14: > Thank you for your reply. > > Here it is as follows > > > if(!defined('AVELSIEVE_DEBUG')) { > define('AVELSIEVE_DEBUG', 0); > } > > global $avelsieve_backend; > $avelsieve_backend = 'File'; > > /*global $sieveport; > $sieveport = 2000;*/ > > global $sieve_preferred_sasl_mech; > $sieve_preferred_sasl_mech = 'PLAIN'; > > global $avelsieve_disabletls; > $avelsieve_disabletls = false; > > global $avelsieve_file_backend_options, $data_dir, $username; > $avelsieve_file_backend_options = array( > 'avelsieve_default_file' => "$data_dir/$username.sievesource" > ); > > /*global $avelsieve_oldcyrus; > $avelsieve_oldcyrus = true;*/ > > global $avelsieve_enable_envelope_auth; > $avelsieve_enable_envelope_auth = true; > > global $avelsieve_custom_sieve_implementation; > $avelsieve_custom_sieve_implementation = ''; > > global $avelsieve_striproot; > $avelsieve_striproot = ''; > > global $avelsieve_hardcoded_capabilities; > $avelsieve_hardcoded_capabilities = array( > 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' > ); > > /*global $avelsieve_imapproxymode, $avelsieve_imapproxyserv; > $avelsieve_imapproxymode = false; > $avelsieve_imapproxyserv = array( > 'localhost' => 'imap.example.org' > );*/ > > global $avelsieve_ldapuserdatamode; > $avelsieve_ldapuserdatamode = false; > > /*global $avelsieve_cyrusadmins_map; > $avelsieve_cyrusadmins_map = array( > 'cyrusimap' => 'cyrussieve' > );*/ > > $conservative = false; > > $useimages = true; > > global $translate_return_msgs; > $translate_return_msgs = false; > > $imagetheme = 'famfamfam'; > //$imagetheme = 'bluecurve_24x24'; > //$imagetheme = 'bluecurve_16x16'; > > global $startitems; > $startitems = 3; > > global $maxitems; > $maxitems = 10; > > global $headers; > $headers = array( > 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Reply-To', 'Sender', 'List-Id', > 'MailingList', 'Mailing-List', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', > 'Resent-From', 'Resent-To', 'X-Mailer', 'X-Mailing-List', > 'X-Spam-Flag', 'X-Spam-Status', > 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', > 'Return-Path', 'Received', 'Auto-Submitted', > 'X-Spam-Flag', 'X-Spam-Status','X-Spam-Tests' > ); > > global $notifymethods; > $notifymethods = array( > 'mailto', 'sms' > ); > > //$notifymethods = false; > > // $disable_avelsieve_capabilities = array("notify"); > global $disable_avelsieve_capabilities; > $disable_avelsieve_capabilities = array(); > > global $avelsieveheaderlink; > $avelsieveheaderlink = true; > > global $avelsieve_default_mode; > $avelsieve_default_mode = 'terse'; > > global $avelsieve_enable_rules; > $avelsieve_enable_rules = array(); > > global $avelsieve_rules_settings; > $avelsieve_rules_settings = array(); > foreach($avelsieve_enable_rules as $r) { > if(file_exists(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php')) { > require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php'); > } else { > require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.default.php'); > } > } > > $avelsieve_spam_highlight_enable = false; > > > Hope this help you to help me. > > Thank you > > > > > ________________________________ > From: Tamsy > To: "dovecot at dovecot.org" > Sent: Tuesday, 17 April 2012 1:25 PM > Subject: Re: [Dovecot] vacation plugins for squirrelmail > > Show us your avelsieve-config in > your-squirrelmail-dir/plugins/avelsieve/config/config.php > > > Daminto Lie wrote the following on 17.04.2012 09:31: >> Thank you folks for your replies. >> >> >> I realised that the dovecot 1.2.9 that I installed via ubuntu synaptic package manager has already come with sieve plugins. After making some modifications on the dovecot.conf file, I went to download and install avelsieve 1.0.1 (stable version) onto the squirrelmail. Then, I run in squirrelmail conf.pl and added the plugin. I logged in and could see "Filter" option right on the top header row. I went to click it and nothing came up. I checked in the mail.log but did not see any errors there. >> >> Here is my snapshot of dovecot configuration >> # 1.2.9: /etc/dovecot/dovecot.conf >> # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS ext4 >> log_timestamp: %Y-%m-%d %H:%M:%S >> protocols: imap >> listen: *:143 >> ssl: >> no >> disable_plaintext_auth: no >> login_dir: /var/run/dovecot/login >> login_executable: /usr/lib/dovecot/imap-login >> verbose_proctitle: yes >> first_valid_uid: 1001 >> last_valid_uid: 600 >> mail_privileged_group: mail >> mail_location: maildir:/var/vmail/%u/Maildir >> mbox_write_locks: fcntl dotlock >> imap_client_workarounds: outlook-idle delay-newmail netscape-eoh tb-extra-mailbox-sep >> lda: >> postmaster_address: postmaster at ourcompany.com >> mail_plugins: sieve >> quota_full_tempfail: yes >> deliver_log_format: msgid=%m: %$ >> sendmail_path: /usr/lib/sendmail >> rejection_reason: Your message to<%t> was automatically rejected:%n%r >> auth_socket_path: /var/run/dovecot-auth-master >> auth default: >> mechanisms: plain login >> username_format: %Lu >> verbose: yes >> debug: yes >> debug_passwords: yes >> passdb: >> driver: >> pam >> passdb: >> driver: ldap >> args: /etc/dovecot/dovecot-ldap.conf >> userdb: >> driver: passwd >> userdb: >> driver: static >> args: uid=1001 gid=1001 home=/var/vmail/%u allow_all_users=yes >> socket: >> type: listen >> client: >> path: /var/spool/postfix/private/auth >> mode: 432 >> user: postfix >> group: mail >> master: >> path: /var/run/dovecot-auth-master >> mode: 432 >> user: vmail >> group: vmail >> plugin: >> sieve: /var/vmail/%u/.sieve >> >> Also, I have restarted dovecot after making changes to the dovecot.conf. >> >> Any help would be very much appreciated. >> >> Thank you >> >> >> >> >> ________________________________ >> From: Artur Zaprza?a >> To: Dovecot Mailing List >> Cc: Daminto Lie >> Sent: Thursday, 12 April 2012 11:46 PM >> Subject: Re: [Dovecot] vacation plugins for squirrelmail >> >> Daminto Lie wrote: >>> Hi, >>> >>> I am afraid I >> have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. >>> Any help would be very much appreciated. >>> >>> Thank you >>> >> I'm using Avelsieve 1.9.9 with a set of my own bugfixes: >> http://email.uoa.gr/avelsieve/ >> >> From f.bonnet at esiee.fr Tue Apr 17 10:54:35 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Tue, 17 Apr 2012 09:54:35 +0200 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) Message-ID: <4F8D21BB.2030407@esiee.fr> Hello I need some feedbacks advices of experienced admins I will have to setup in few monthes an email system for approx 50K "intensives" users. The only mandatory thing will be I must use HP proliant servers The operating system will be FreeBSD or Linux Thank you for any advices From dchenusa at yahoo.com Tue Apr 17 12:16:22 2012 From: dchenusa at yahoo.com (D Chen) Date: Tue, 17 Apr 2012 02:16:22 -0700 (PDT) Subject: [Dovecot] Dovecot and Outlook Express "plaintext authentication not allowed..." Message-ID: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> Server: Ubuntu 11.10 server with postfix (MTA), dovecot (MDA) Mail client: Outlook Express in Windows XP with IMAP on port 143, SMTP on port 25 When tried to set up a new Outlook Express mail acct, the following error encountered: ? ? ?Your IMAP server wishes to alert you to the following: ? ? ?plaintext authentication not allowed without SSL/TLS, but your client did it anyway. ?If ? ? ?anyone was listening, the password was exposed. When asked "would you like to download folders from the mail server you added?" then got the following error: The server has rejected your login. ?Please verify that your user name and password are correct. I re-entered several time with the correct username and password, but still responded the same error. Any insights would be greatly appreciated. Here is the message from the /var/log/mail.log: Apr 17 01:54:17 server dovecot: imap-login: Disconnected (tried to use disabled plaintext auth): rip= 192.168.20.51, lip=192.168.20.100Apr 17 01:54:28 server dovecot: imap-login: Disconnected (tried to use disabled plaintext auth): rip= 192.168.20.51, lip=192.168.20.100 What might be wrong in my dovecot.conf file. ?Below is my dovecot.conf file: # 2.0.13: /etc/dovecot/dovecot.conf # OS: Linux 3.0.0-16-server x86_64 Ubuntu 11.10? log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:~/Maildir mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date passdb { ? driver = pam } passdb { ? driver = pam } plugin { ? sieve = ~/.dovecot.sieve ? sieve_dir = ~/sieve } protocols = imap pop3 sieve service auth { ? unix_listener /var/spool/postfix/private/auth-client { ? ? group = postfix ? ? mode = 0660 ? ? user = postfix ? } ? unix_listener /var/spool/postfix/private/dovecot-auth { ? ? group = postfix ? ? mode = 0660 ? ? user = postfix ? } ? user = root } ssl_cert = was automatically rejected:%n%r } From CMarcus at Media-Brokers.com Tue Apr 17 13:26:37 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 17 Apr 2012 06:26:37 -0400 Subject: [Dovecot] Dovecot and Outlook Express "plaintext authentication not allowed..." In-Reply-To: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> References: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> Message-ID: <4F8D455D.9060805@Media-Brokers.com> On 2012-04-17 5:16 AM, D Chen wrote: > When tried to set up a new Outlook Express mail acct, the following error encountered: > > Your IMAP server wishes to alert you to the following: > plaintext authentication not allowed without SSL/TLS, but your client did it anyway. Ummm... hello? McFly? ;) You need to configure SSL on Outlook Express (why on gods green earth you are using that is beyond me), as the error plainly says. Until you do this it doesn't matter what you enter for a username/password. Also, you should be using SSL on port 993, because I'm fairly certain that Outlook Express doesn't support STARTTLS on port 143. And off topic but in the same vein, for postfix/SMTP relay you should be using the submission port (STARTTLS on port 587) for most clients, but I think Outlook Express again doesn't work with it, so you will need to use the deprecated SSL on port 465 (these two need to be enabled in postfix/master.cf - just uncomment the examples). -- Best regards, Charles From blevi.linux at gmail.com Tue Apr 17 13:47:26 2012 From: blevi.linux at gmail.com (Birta Levente) Date: Tue, 17 Apr 2012 13:47:26 +0300 Subject: [Dovecot] Dovecot and Outlook Express "plaintext authentication not allowed..." In-Reply-To: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> References: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> Message-ID: <4F8D4A3E.8000401@gmail.com> On 17/04/2012 12:16, D Chen wrote: > Server: Ubuntu 11.10 server with postfix (MTA), dovecot (MDA) > Mail client: Outlook Express in Windows XP with IMAP on port 143, SMTP on port 25 > > When tried to set up a new Outlook Express mail acct, the following error encountered: > > Your IMAP server wishes to alert you to the following: > plaintext authentication not allowed without SSL/TLS, but your client did it anyway. If > anyone was listening, the password was exposed. > > When asked "would you like to download folders from the mail server you added?" then got the > following error: > > The server has rejected your login. Please verify that your user name and password are correct. > > I re-entered several time with the correct username and password, but still responded the same error. > > Any insights would be greatly appreciated. > > > Here is the message from the /var/log/mail.log: > > > Apr 17 01:54:17 server dovecot: imap-login: Disconnected (tried to use disabled plaintext auth): rip= > 192.168.20.51, lip=192.168.20.100Apr 17 01:54:28 server dovecot: imap-login: Disconnected (tried to use disabled plaintext auth): rip= > 192.168.20.51, lip=192.168.20.100 > > What might be wrong in my dovecot.conf file. Below is my dovecot.conf file: > > # 2.0.13: /etc/dovecot/dovecot.conf > # OS: Linux 3.0.0-16-server x86_64 Ubuntu 11.10 > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_location = maildir:~/Maildir > mail_privileged_group = mail > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date > passdb { > driver = pam > } > passdb { > driver = pam > } > plugin { > sieve = ~/.dovecot.sieve > sieve_dir = ~/sieve > } > protocols = imap pop3 sieve > service auth { > unix_listener /var/spool/postfix/private/auth-client { > group = postfix > mode = 0660 > user = postfix > } > unix_listener /var/spool/postfix/private/dovecot-auth { > group = postfix > mode = 0660 > user = postfix > } > user = root > } > ssl_cert = ssl_cipher_list = ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM > ssl_key = userdb { > driver = passwd > } > userdb { > driver = passwd > } > protocol imap { > imap_client_workarounds = delay-newmail > mail_max_userip_connections = 10 > } > protocol pop3 { > mail_max_userip_connections = 10 > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > pop3_uidl_format = %08Xu%08Xv > } > protocol lda { > deliver_log_format = msgid=%m: %$ > mail_plugins = sieve > postmaster_address = postmaster > quota_full_tempfail = yes > rejection_reason = Your message to<%t> was automatically rejected:%n%r > } > Maybe auth_mechanisms = plain login cram-md5 From robert at schetterer.org Tue Apr 17 14:47:57 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 17 Apr 2012 13:47:57 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken Message-ID: <4F8D586D.3010706@schetterer.org> Hi Tim, after upgrade to 2.1 acl with shared namespace seems to be broken it looks like depend to No lookup right to mailbox: shared/ it runs perfect under 2.0.20 -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From mlopez at gtdinternet.com Tue Apr 17 14:54:15 2012 From: mlopez at gtdinternet.com (=?ISO-8859-1?Q?Mauricio_L=F3pez_Riffo?=) Date: Tue, 17 Apr 2012 08:54:15 -0300 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8D21BB.2030407@esiee.fr> References: <4F8D21BB.2030407@esiee.fr> Message-ID: <4F8D59E7.6010403@gtdinternet.com> Frank, Here we have approx. 200K users with 4000 concurrent connections (90% POP3 users) All servers in virtual environment Vmware, supermicro servers and Netapp Metrocluster storage solutions (nfs storage with 10G ethernet network) POP3 sessions take betwen 40 and 300 milisecons at connect, auth and list. All accounts lives in LDAP, CentOS 5 and exim like a mta relay. Regards El 17-04-2012 4:54, Frank Bonnet escribi?: > Hello > > I need some feedbacks advices of experienced admins > I will have to setup in few monthes an email system > for approx 50K "intensives" users. > > The only mandatory thing will be I must use HP proliant servers > > The operating system will be FreeBSD or Linux > > Thank you for any advices > > -- Mauricio L?pez Riffo Red Hat Certified Engineer 804006455319519 Administrador de Servicios Internet ?rea Ingenier?a Gtd Internet S.A. http://www.grupogtd.com/ Moneda 920, Oficina 602 - Fono : +562 4139742 From janfrode at tanso.net Tue Apr 17 15:42:41 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 17 Apr 2012 14:42:41 +0200 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8D59E7.6010403@gtdinternet.com> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> Message-ID: <20120417124241.GA23120@dibs.tanso.net> On Tue, Apr 17, 2012 at 08:54:15AM -0300, Mauricio L?pez Riffo wrote: > > Here we have approx. 200K users with 4000 concurrent connections > (90% POP3 users) How do you measure "concurrent" POP3 users? > All servers in virtual environment Vmware, > supermicro servers and Netapp Metrocluster storage solutions (nfs > storage with 10G ethernet network) POP3 sessions take betwen 40 and > 300 milisecons at connect, auth and list. All accounts lives in > LDAP, CentOS 5 and exim like a mta relay. Very interesting config. We're close to 1M accounts, GPFS cluster fs, LDAP, RHEL5/6 and postfix + dovecot director for pop/imap/lmtp, and moving from maildir to mdbox. What mailbox-format are you using? Do you have a director, or accounts sticky to a server some other way? How's the NFS performance? I've always bean weary that NFS works terribly with many small files (i.e. maildir).. What does the metrocluster give you? Is it for disaster recovery on second location, or do you have two active locations working against the same filesystem? -jf From f.bonnet at esiee.fr Tue Apr 17 16:01:31 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Tue, 17 Apr 2012 15:01:31 +0200 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8D59E7.6010403@gtdinternet.com> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> Message-ID: <4F8D69AB.5090706@esiee.fr> hello Thanks for your answer , MY problem will be IMAPS connections I dunno how much I will have but it would be possible that we'll have 4000/6000 imaps concurent connections during working hours . POP3 users will be very few Le 17/04/2012 13:54, Mauricio L?pez Riffo a ?crit : > Frank, > > Here we have approx. 200K users with 4000 concurrent connections > (90% POP3 users) All servers in virtual environment Vmware, > supermicro servers and Netapp Metrocluster storage solutions (nfs > storage with 10G ethernet network) POP3 sessions take betwen 40 and > 300 milisecons at connect, auth and list. All accounts lives in LDAP, > CentOS 5 and exim like a mta relay. > > Regards > > > > El 17-04-2012 4:54, Frank Bonnet escribi?: >> Hello >> >> I need some feedbacks advices of experienced admins >> I will have to setup in few monthes an email system >> for approx 50K "intensives" users. >> >> The only mandatory thing will be I must use HP proliant servers >> >> The operating system will be FreeBSD or Linux >> >> Thank you for any advices >> >> > From mlopez at gtdinternet.com Tue Apr 17 16:10:02 2012 From: mlopez at gtdinternet.com (=?ISO-8859-1?Q?Mauricio_L=F3pez_Riffo?=) Date: Tue, 17 Apr 2012 10:10:02 -0300 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <20120417124241.GA23120@dibs.tanso.net> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <20120417124241.GA23120@dibs.tanso.net> Message-ID: <4F8D6BAA.7060501@gtdinternet.com> Jan, >> How do you measure "concurrent" POP3 users? We use cacti for metrics like concurrent connections or pop3 delay and Zabbix for alarms. >> What mailbox-format are you using? Do you have a director, or accounts >> sticky to a server some other way? Maildir like a mailformat and currently without a director, but we testing a new environment with director to reduce number of servers (7 virtual servers with 4 Vcpu, 6gb ram) At the meantime a lvs piranha do the work of steaky connections but its not enought, for this reason will use a director. >> How's the NFS performance? I've always bean weary that NFS works terribly with many small files (i.e. maildir).. At peak hours our storage traffic about 10 thousend total ops (bewten read and write, 80/20) and cpu mark 70% of use. >> What does the metrocluster give you? Is it for disaster recovery on >> second location, or do you have two active locations working against the >>same filesystem? Our storage have two missions, first hight availability because we have two datacenter (environment N +1) and posibility of use two storages in cluster for the same filesystems en this two datacenters (Vmware and Mail storage resides in NFS filesystem) >> Very interesting config. We're close to 1M accounts, GPFS cluster fs, LDAP, >> RHEL5/6 and postfix + dovecot director for pop/imap/lmtp, and moving from >> maildir to mdbox. 1M = 1 milion ? How many servers you have? hardware? Any help o contribution, you welcome :) Regards El 17-04-2012 9:42, Jan-Frode Myklebust escribi?: > On Tue, Apr 17, 2012 at 08:54:15AM -0300, Mauricio L?pez Riffo wrote: >> Here we have approx. 200K users with 4000 concurrent connections >> (90% POP3 users) > How do you measure "concurrent" POP3 users? > >> All servers in virtual environment Vmware, >> supermicro servers and Netapp Metrocluster storage solutions (nfs >> storage with 10G ethernet network) POP3 sessions take betwen 40 and >> 300 milisecons at connect, auth and list. All accounts lives in >> LDAP, CentOS 5 and exim like a mta relay. > Very interesting config. We're close to 1M accounts, GPFS cluster fs, LDAP, > RHEL5/6 and postfix + dovecot director for pop/imap/lmtp, and moving from > maildir to mdbox. > > What mailbox-format are you using? Do you have a director, or accounts > sticky to a server some other way? > > How's the NFS performance? I've always bean weary that NFS works > terribly with many small files (i.e. maildir).. > > What does the metrocluster give you? Is it for disaster recovery on > second location, or do you have two active locations working against the > same filesystem? > > > -jf -- Mauricio L?pez Riffo Red Hat Certified Engineer 804006455319519 Administrador de Servicios Internet ?rea Ingenier?a Gtd Internet S.A. http://www.grupogtd.com/ Moneda 920, Oficina 602 - Fono : +562 4139742 From goetz.reinicke at filmakademie.de Tue Apr 17 17:21:39 2012 From: goetz.reinicke at filmakademie.de (=?ISO-8859-15?Q?G=F6tz_Reinicke?=) Date: Tue, 17 Apr 2012 16:21:39 +0200 Subject: [Dovecot] 487 imap-login processes - is this ok Message-ID: <4F8D7C73.9090700@filmakademie.de> Hi, recently I checked our mailserver for some more tuning and optimizing and noticed, that currently I do have 487 imap-login processes and I'm asking myself if that's o.k. We do have about 1.000 user in total and not everybody is currently logged in. May be this is o.k. but if not, what might I check? We run the dovecot package provided by redhat EL 5.8 dovecot-1.0.7-7.el5_7.1 Thanks for any comment or suggestion. Regards . G?tz -- G?tz Reinicke IT-Koordinator Tel. +49 7141 969 420 Fax +49 7141 969 55 420 E-Mail goetz.reinicke at filmakademie.de Filmakademie Baden-W?rttemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: J?rgen Walter MdL Staatssekret?r im Ministerium f?r Wissenschaft, Forschung und Kunst Baden-W?rttemberg Gesch?ftsf?hrer: Prof. Thomas Schadt -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5161 bytes Desc: S/MIME Kryptografische Unterschrift URL: From robert at schetterer.org Tue Apr 17 17:40:19 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 17 Apr 2012 16:40:19 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken In-Reply-To: <4F8D586D.3010706@schetterer.org> References: <4F8D586D.3010706@schetterer.org> Message-ID: <4F8D80D3.2050505@schetterer.org> Am 17.04.2012 13:47, schrieb Robert Schetterer: > Hi Tim, i should get some "o" > after upgrade to 2.1 > acl with shared namespace seems to be broken > it looks like depend to > > No lookup right to mailbox: shared/ > > it runs perfect under 2.0.20 looks its depended to bug handling doveadm acl debug shows on both versions Fatal: ACL shared dict iteration failed but dove 2.1 does handle this in another way means, results in no rights , and no list for that -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From jd.beaubien at gmail.com Tue Apr 17 19:19:49 2012 From: jd.beaubien at gmail.com (Jean-Daniel Beaubien) Date: Tue, 17 Apr 2012 12:19:49 -0400 Subject: [Dovecot] Howto reindex with solr? Message-ID: Hi, I am currently testing a 2.1 dovecot setup with fts-solr. Search speed is simply amazing. I decided to try to delete and rebuild the solr indexes, but i can't seem to rebuild them thru dovecot. At first, when telnetting to the imap server, the search initially built the index, and then subsequent searches are blazingly fast. 1- I deleted all the solr indexes with the following query: *:* 2- Flag the mailbox to be re-indexed: doveadm force-resync INBOX At this point, it doesn't work. When I telnet into the imap server, the searches always come back empty 3- I also tried a few other commands: doveadm fts rescan, doveadm index INBOX. Still no search results. Nothing seems to be appearing in the logs when I issue all the doveadm commands. What did I miss? Thanks, -JD From cor at xs4all.nl Tue Apr 17 20:57:01 2012 From: cor at xs4all.nl (Cor Bosman) Date: Tue, 17 Apr 2012 19:57:01 +0200 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8D59E7.6010403@gtdinternet.com> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> Message-ID: <20120417175701.GA28791@xs4all.nl> > Here we have approx. 200K users with 4000 concurrent connections > (90% POP3 users) All servers in virtual environment Vmware, supermicro > servers and Netapp Metrocluster storage solutions (nfs storage with 10G > ethernet network) POP3 sessions take betwen 40 and 300 milisecons at > connect, auth and list. All accounts lives in LDAP, CentOS 5 and exim > like a mta relay. Similar setup here. Maybe 15-20K concurrent connections, imap only (pop is not handled by dovecot yet) about 800K mailboxes. We have all bare metal servers, linux. Currently 35 servers handling the load easily. We could run on 1/3rd of the load probably. In front of the 35 servers are 3 directors, handling imap only, although im in the process of adding LMTP/sieve to the mix. Backend storage is NetApp Metrocluster over 2 datacenters. Cor From joe.beaubien at gmail.com Tue Apr 17 21:36:33 2012 From: joe.beaubien at gmail.com (Joe Beaubien) Date: Tue, 17 Apr 2012 14:36:33 -0400 Subject: [Dovecot] Re-indexing of fts-solr not working Message-ID: Hi, I am currently testing a 2.1 dovecot setup with fts-solr. Search speed is simply amazing. I decided to try to delete and rebuild the solr indexes, but i can't seem to rebuild them thru dovecot. At first, when telnetting to the imap server, the search initially built the index, and then subsequent searches are blazingly fast. 1- I deleted all the solr indexes with the following query: *:* 2- Flag the mailbox to be re-indexed: doveadm force-resync INBOX At this point, it doesn't work. When I telnet into the imap server, the searches always come back empty 3- I also tried a few other commands: doveadm fts rescan, doveadm index INBOX. Still no search results. Nothing seems to be appearing in the logs when I issue all the doveadm commands. What did I miss? Thanks, Joe From janfrode at tanso.net Tue Apr 17 23:08:20 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 17 Apr 2012 22:08:20 +0200 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8D6BAA.7060501@gtdinternet.com> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <20120417124241.GA23120@dibs.tanso.net> <4F8D6BAA.7060501@gtdinternet.com> Message-ID: <20120417200820.GA30743@dibs.tanso.net> On Tue, Apr 17, 2012 at 10:10:02AM -0300, Mauricio L?pez Riffo wrote: > > 1M = 1 milion ? 976508 to be exact :-) but it's very much a useless number. Lots and lots of these are inactive. A better number is probably that we're seeing about 80 logins/second for the last hour.. (just checked now, not sure if this is the most busy hour or not). > How many servers you have? hardware? 7 backend dovecot servers (two IBM x336, three x346 and two x3550, with a 8 GB for the x336/x346 and 16GB memory memory for the x3550's). 2 frontend dovecot directors (IBM x3550). None of these are really very busy, so we could probably reduce the number of backends a bit if we wanted. Our struggle is the number of iops we're able to get from the backend storage (IBM DS4800), mostly a problem when we have storms of incoming marketing messages in addition to the pop/imap traffic. -jf From dchenusa at yahoo.com Tue Apr 17 23:41:20 2012 From: dchenusa at yahoo.com (D Chen) Date: Tue, 17 Apr 2012 13:41:20 -0700 (PDT) Subject: [Dovecot] Dovecot and Outlook Express "plaintext authentication not allowed..." In-Reply-To: <4F8D455D.9060805@Media-Brokers.com> References: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> <4F8D455D.9060805@Media-Brokers.com> Message-ID: <1334695280.2248.YahooMailNeo@web161606.mail.bf1.yahoo.com> Hi Charles, Thanks for the response. Per your suggestion, in Outlook Express I changed to use SSL on port 993 for IMAP, and port 465 for SMTP,?the creation of the new account succeeded without error! ?However, when tried to send to my yahoo mail or gmail acct, ?got the following error: (apparently failed to send!) "The message could not be sent because one of the recipients was rejected by the server. ?The rejected email address was 'dchenusa at yahoo.com.' ?Subject: 'test out to yahoo mail' Account:'dchen192.168.20.100' Server: 192.168.20.100 Protocol: SMTP, server response: 554 5.7.1 : client host rejected access denied, port: 465, Secured [SSL]: Yes, server Error: 554, Error number: 0x800CCC79" PS. in the same client PC, using Thunderbird client, IMAP and SMTP configured with either with SSL or without SSL, both outgoing mails were sent successfully! Any thought? Thx again, Dennis ________________________________ From: Charles Marcus To: dovecot at dovecot.org Sent: Tuesday, April 17, 2012 3:26 AM Subject: Re: [Dovecot] Dovecot and Outlook Express "plaintext authentication not allowed..." On 2012-04-17 5:16 AM, D Chen wrote: > When tried to set up a new Outlook Express mail acct, the following error encountered: > >? ? ? Your IMAP server wishes to alert you to the following: >? ? ? plaintext authentication not allowed without SSL/TLS, but your client did it anyway. Ummm... hello? McFly? ;) You need to configure SSL on Outlook Express (why on gods green earth you are using that is beyond me), as the error plainly says. Until you do this it doesn't matter what you enter for a username/password. Also, you should be using SSL on port 993, because I'm fairly certain that Outlook Express doesn't support STARTTLS on port 143. And off topic but in the same vein, for postfix/SMTP relay you should be using the submission port (STARTTLS on port 587) for most clients, but I think Outlook Express again doesn't work with it, so you will need to use the deprecated SSL on port 465 (these two need to be enabled in postfix/master.cf - just uncomment the examples). -- Best regards, Charles From jerry at seibercom.net Wed Apr 18 00:27:18 2012 From: jerry at seibercom.net (Jerry) Date: Tue, 17 Apr 2012 17:27:18 -0400 Subject: [Dovecot] Dovecot and Outlook Express "plaintext authentication not allowed..." In-Reply-To: <1334695280.2248.YahooMailNeo@web161606.mail.bf1.yahoo.com> References: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> <4F8D455D.9060805@Media-Brokers.com> <1334695280.2248.YahooMailNeo@web161606.mail.bf1.yahoo.com> Message-ID: <20120417172718.3af8127d@scorpio> On Tue, 17 Apr 2012 13:41:20 -0700 (PDT) D Chen articulated: > Per your suggestion, in Outlook Express I changed to use SSL on port > 993 for IMAP, and port 465 for SMTP,?the creation of the new account > succeeded without error! ?However, when tried to send to my yahoo > mail or gmail acct, ?got the following error: (apparently failed to > send!) > > "The message could not be sent because one of the recipients was > rejected by the server. ?The rejected email address was > 'dchenusa at yahoo.com.' ?Subject: 'test out to yahoo mail' > Account:'dchen192.168.20.100' Server: 192.168.20.100 > Protocol: SMTP, server response: 554 5.7.1 > : client host rejected access denied, port: > 465, Secured [SSL]: Yes, server Error: 554, Error number: 0x800CCC79" > > PS. in the same client PC, using Thunderbird client, IMAP and SMTP > configured with either with SSL or without SSL, both outgoing mails > were sent successfully! > > Any thought? Well, my first thought is that you should lost the tendency to "Top Post". Second, why are you screwing around with the SMTP settings? Dovecot could not care less about them. Third, Outlook Express has been abandoned years ago. Why are you still using it? If you really need it, try Windows Live Essentials . I know users who have it working with Dovecot just fine. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From kgc at corp.sonic.net Wed Apr 18 01:29:17 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Tue, 17 Apr 2012 15:29:17 -0700 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> Message-ID: <4F8DEEBD.9080206@corp.sonic.net> On 03/30/12 07:35, Timo Sirainen wrote: > IP stickiness isn't enough if user uses more than one IMAP client, which is pretty common nowadays. And doesn't help at all with LDA. Timo, having the LDA flow through the director with maildir and NFS is only an issue if you want to use the dovecot delivery agent since it manipulates the indexes on delivery, right? This shouldn't be a problem if you are running a completely distinct delivery cluster running a LDA like procmail. -K From noel.butler at ausics.net Wed Apr 18 05:24:29 2012 From: noel.butler at ausics.net (Noel Butler) Date: Wed, 18 Apr 2012 12:24:29 +1000 Subject: [Dovecot] 487 imap-login processes - is this ok In-Reply-To: <4F8D7C73.9090700@filmakademie.de> References: <4F8D7C73.9090700@filmakademie.de> Message-ID: <1334715869.4078.3.camel@tardis> Older versions of dovecot (all of v1) can under some circumstances screwup and leave hung login processes, this more often than not occurs if you have network issues, requiring a restart of dovecot to clear them. Timo has apparently, IIRC, worked around this for v2.x and it should not happen. On Tue, 2012-04-17 at 16:21 +0200, G?tz Reinicke wrote: > Hi, > > recently I checked our mailserver for some more tuning and optimizing > and noticed, that currently I do have 487 imap-login processes and I'm > asking myself if that's o.k. > > We do have about 1.000 user in total and not everybody is currently > logged in. > > May be this is o.k. but if not, what might I check? > > We run the dovecot package provided by redhat EL 5.8 dovecot-1.0.7-7.el5_7.1 > > Thanks for any comment or suggestion. > > Regards . G?tz -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From dlie76 at yahoo.com.au Wed Apr 18 04:12:49 2012 From: dlie76 at yahoo.com.au (Daminto Lie) Date: Tue, 17 Apr 2012 18:12:49 -0700 (PDT) Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <4F8D1507.7060609@mohtex.net> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> <4F86DCA7.1060806@talex.pl> <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> <4F8CE2B8.3080801@mohtex.net> <1334643297.87733.YahooMailNeo@web113417.mail.gq1.yahoo.com> <4F8D1507.7060609@mohtex.net> Message-ID: <1334711569.21467.YahooMailNeo@web113409.mail.gq1.yahoo.com> Thank you Tamsy for sharing with me your config file. Initially, I thought I tried not to use managesieve and that's why I skipped that part there. But after seeing your config, I'd try to use managesieve. I would need to modify my dovecot config file first to incorporate the managesieve. And I also learnt that I need to put in my domain name. In this case, I would try this again next week as this server has not been made as a production server just yet. I thought if I could get all the things I need to work before the day of putting it online to the Internet. That would save me a lot of work. Thank you once again ________________________________ From: Tamsy To: Dovecot Mailing List Sent: Tuesday, 17 April 2012 5:00 PM Subject: Re: [Dovecot] vacation plugins for squirrelmail Seems not to be attached to my former mail. Here it comes again, inline: avelsieve config.php ############################### * @copyright 2002-2004 Alexandros Vellis * @package plugins * @subpackage avelsieve */ /** * Debug Mode. Enable this (change to 1) if you need to send a bug report, * or to 2 or 3 if you are a developer! * * Valid values: * 0 = No debugging output * 1 = Normal debugging output * 2 = Firebug-enhanced debugging output * 3 = Enhanced debugging output and no Sieve capabilities checks - enables *? ? all UI features! */ if(!defined('AVELSIEVE_DEBUG')) { ? ? define('AVELSIEVE_DEBUG', 1); } /* ======================================================================== */ /* =================== IMAP Server / SIEVE Setup? ========================= */ /* ======================================================================== */ /** @var string Backend to use. Available backends are: * 'ManageSieve': Uses the ManageSieve protocol. e.g. Cyrus * 'File': Writes files straight to disk. e.g. Exim4, Dovecot LDA. */ global $avelsieve_backend; $avelsieve_backend = 'ManageSieve'; /* ======================================================================== */ /* =================== ManageSieve Backend Options ======================== */ /* ======================================================================== */ /* Port where ManageSieve servers (e.g. Cyrus IMAP's timsieved) listens to. * Default is 2000. Note that another possible port that your server might * listen to, is 4190, which is now the official (IANA-assigned) port for * Sieve. */ global $sieveport; $sieveport = 4190; /** * @var string Space separated list of preferred SASL mechanisms for the * authentication to ManageSieve server. e.g. "PLAIN DIGEST-MD5";*/ global $sieve_preferred_sasl_mech; $sieve_preferred_sasl_mech = 'PLAIN'; /** * @var boolean Disable STARTTLS for ManageSieve. You can set this to true, * if you do not wish to use encryption via TLS mechanisms (i.e. the server * is not configured properly, or this is a local connection and TLS is not * needed. * Note that STARTTLS is supported only in PHP5+. In PHP4 this option will * have no effect and STARTTLS will be disabled anyway. */ global $avelsieve_disabletls; $avelsieve_disabletls = false; /* ======================================================================== */ /* ======================= File Backend Options =========================== */ /* ======================================================================== */ global $avelsieve_file_backend_options, $data_dir, $username; $avelsieve_file_backend_options = array( ? ? 'avelsieve_default_file' => "$data_dir/$username.sievesource" ); /* ======================================================================== */ /* ====== Implementation- and Server-Specific? Options ==================== */ /* ======================================================================== */ /* In Cyrus 2.3+, the notification action is a bit more complex than the * others. The oldcyrus variable is for supporting the partially implemented * notify extension implementation of Cyrus < 2.3. If you have Cyrus < 2.3, * just set this to true. * * This only changes the informational / help text displayed in avelsieve. * * Cyrus < 2.3 : $from$, $env-from$, $subject$ * Cyrus 2.3+? : $from$, $env-from$, $subject$, $text$, $text[n]$ */ global $avelsieve_oldcyrus; $avelsieve_oldcyrus = true; /* If you have Cyrus with an lmtpd that can understand the "auth" argument to * the :envelope test as the SMTP/LMTP auth, or any other Sieve implementation, * then you can enable this to provide this functionality to the user. * * This was not clarified in the base spec of RFC 3028. It will be done * correctly in a new version of Cyrus, based on a new draft / spec. */ global $avelsieve_enable_envelope_auth; $avelsieve_enable_envelope_auth = true; /* Some Implementations of Sieve need certain things in order to operate * correctly. If you use any of the following server implementations, you MUST * set this variable to the corresponding value for the filtering to work * correctly. * Valid values are: * - Any RFC3028-mostly-compatible implementation: '' (empty). * - Exim MTA: 'exim' * - MFL (as supported by mvmf): 'mfl' */ global $avelsieve_custom_sieve_implementation; $avelsieve_custom_sieve_implementation = ''; /* For delivery agents that don't know how to handle some mailbox prefixes, you * can enable this option. Example for Dovecot LDA: 'INBOX.'; */ global $avelsieve_striproot; $avelsieve_striproot = 'INBOX.'; /* If the backend does not support capabilities reporting, such as the File * Backend, then you should define which capabilities are used by the server * implementation. * * The following are the capabilities supported by Exim4 as of Exim version * 4.60, according to README.SIEVE. You can change them if a new version of * Exim provides more functionality: *? 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' * * The following are the capabilities that are suported by MFM, according to: * http://www.mvmf.org/mfl/language.shtml#sieve *? 'envelope', 'fileinto', 'reject', 'relational', 'subaddress', 'regex', *? 'editheader', 'copy', 'vacation', 'comparator-i;ascii-casemap', *? 'comparator-i;octet' * * The following capabilities have been reported to work with Dovecot LDA: *? 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric', *? 'imapflags', 'subaddress','relational','regex' * */ /* global $avelsieve_hardcoded_capabilities; * $avelsieve_hardcoded_capabilities = array( *? ? 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' * ); */ global $avelsieve_hardcoded_capabilities; $avelsieve_hardcoded_capabilities = array( ? ? 'fileinto', 'reject', 'envelope', 'encoded-character', 'vacation', ? ? 'subaddress', 'comparator-i;ascii-numeric', 'relational', 'regex', ? ? 'imap4flags', 'copy', 'include', 'variables', 'body', 'enotify', ? ? 'environment', 'mailbox', 'date' ); /** @var boolean Enable ImapProxy mode. * If you use imapproxy, because imapproxy cannot understand and proxy the * SIEVE protocol, you must connect to the SIEVE daemon (usually on the IMAP * server) itself. So you need to set $imapproxymode to true, and define a * mapping, from the imapproxy host (usually localhost) to your real IMAP * server (usually the same that is defined on Imapproxy's configuration). * * This will not work if you use a perdition-style proxy, where different users * go to different IMAP servers; it applies mostly to people running imapproxy * for speed and want a quick hack. */ global $avelsieve_imapproxymode, $avelsieve_imapproxyserv; $avelsieve_imapproxymode = false; $avelsieve_imapproxyserv = array( ? ? 'localhost' => 'YOURDOMAINNAMEHERE.com' ); /** @var boolean Ldapuserdata mode: Gets user's email addresses (including * mailAlternate & mailAuthorized) from LDAP Prefs Backend plugin's cache */ global $avelsieve_ldapuserdatamode; $avelsieve_ldapuserdatamode = false; /** @var array Map of cyrus administrator users, for proxy authentication */ global $avelsieve_cyrusadmins_map; $avelsieve_cyrusadmins_map = array( ? ? 'cyrusimap' => 'cyrussieve' ); /* ======================================================================== */ /* =============== Avelsieve Interface / Behavior Setup? ================== */ /* ======================================================================== */ /* Be conservative to our updates on the SIEVE server? If true, a button * entitled "Save Changes" will appear, which will give the user the * functionality to register her changes. 'false' is recommended. */ $conservative = false; /* Use images for the move up / down, delete rule buttons and STOP? */ $useimages = true; /* Translate the messages returned by the "Reject" and "Vacation" actions? The * default behaviour since 0.9 is not to translate them. Change to true if in * an intranet environment or in a same-language environment. */ global $translate_return_msgs; $translate_return_msgs = false; /* Theme to use for the images. A directory with the same name must exist under * plugins/avelsieve/$imagetheme, that contains the files: up.png, down.png, * del.png, dup.png, edit.png, top.png, bottom.png. */ $imagetheme = 'famfamfam'; //$imagetheme = 'bluecurve_24x24'; //$imagetheme = 'bluecurve_16x16'; /* Number of items to display _initially_, when displaying the header match * rule */ global $startitems; $startitems = 3; /* Maximum number of items to allow in one header match rule. */ global $maxitems; $maxitems = 10; /* Headers to display in listbox widget, when adding a new header rule. */ /* global $headers; * $headers = array( *? 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Reply-To', 'Sender', 'List-Id', *? 'MailingList', 'Mailing-List', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', *? 'Resent-From',? 'Resent-To', 'X-Mailer', 'X-Mailing-List', *? 'X-Spam-Flag', 'X-Spam-Status', *? 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', *? 'Return-Path', 'Received', 'Auto-Submitted', *? 'X-Spam-Flag', 'X-Spam-Status','X-Spam-Tests' *? ); */ global $headers; $headers = array( 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Sender', 'Reply-To', 'List-Id', 'MailingList', 'Mailing-List', 'X-DKIM-Authentication-Results', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', 'Resent-From',? 'Resent-To', 'X-Mailer', 'X-Mailing-List', 'X-SpamTest-Status-Extended', 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', 'Return-Path', 'Received', 'Auto-Submitted', ); /* Available :method's for the :notify extension (if applicable) */ global $notifymethods; $notifymethods = array( 'mailto', 'sms' ); /* use the value "false" if you want to provide a simple input box so that * users can edit the method themselves : */ //$notifymethods = false; /* Capabilities to disable. If you would like to force avelsieve not to display * certain features, even though there _is_ a capability for them in Sieve * server, you should specify these here. For instance, if you would * like to disable the notify extension, even though ManageSieve advertises it, * you should add 'notify' in this array: $force_disable_avelsieve_capability = * array("notify");. This will still leave the defined feature on, and if the * user can upload her own scripts then she can use that feature; this option * just disables the GUI of it. Leave as-is (empty array) if you do not need * that. * * Look in $implemented_capabilities array in include/constants.inc.php for * valid values */ // $disable_avelsieve_capabilities = array("notify"); global $disable_avelsieve_capabilities; $disable_avelsieve_capabilities = array(); /* Display Filters link in the top Squirrelmail header? */ global $avelsieveheaderlink; $avelsieveheaderlink = true; /* Default rules table display mode, one of 'verbose' or 'terse' */ global $avelsieve_default_mode; $avelsieve_default_mode = 'terse'; /* ======================================================================== */ /* ========================= Custom rules Configuration =================== */ /* ======================================================================== */ /** * @var array * List of additional rules to enable. This must be the numeric id of the * rule. * * 10: Spam Rule (as existed in Avelsieve) * 11: Junk-Mail Rule (new-style SPAM Rule, only one exists per Sieve script) * 12: Global Whitelist (only one exists per Sieve script) * 13: Custom Sieve Code (allows users to enter whatever Sieve code they like - *? ? Alpha/Experimental) * * Example: * array(11, 12); */ global $avelsieve_enable_rules; $avelsieve_enable_rules = array(); global $avelsieve_rules_settings; $avelsieve_rules_settings = array(); foreach($avelsieve_enable_rules as $r) { ? ? if(file_exists(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php')) { ? ? ? ? require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php'); ? ? } else { ? ? ? ? require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.default.php'); ? ? } } /* Please keep the following setting false; it is alpha + needs Squirrelmail * to be patched in three or four places. */ $avelsieve_spam_highlight_enable = false; ############################### Hi Daminto, That looks pretty much unconfigured. No wonder nothing is coming up. I have attached the avelsieve config file we are using on one of our domains. Don't copy it blindly. At least you should configure $sieveport to the port you are using on your server for Sieve. And put your domain name where it reads YOURDOMAINNAMEHERE.com. For the arrays at $headers you should tune the filters you are using with your headers. Hope this helps (working fine with our domain and Squirrel-Mail). Rgds Tamsy Daminto Lie wrote the following on 17.04.2012 13:14: > Thank you for your reply. > > Here it is as follows > > > if(!defined('AVELSIEVE_DEBUG')) { >? ? ? define('AVELSIEVE_DEBUG', 0); > } > > global $avelsieve_backend; > $avelsieve_backend = 'File'; > > /*global $sieveport; > $sieveport = 2000;*/ > > global $sieve_preferred_sasl_mech; > $sieve_preferred_sasl_mech = 'PLAIN'; > > global $avelsieve_disabletls; > $avelsieve_disabletls = false; > > global $avelsieve_file_backend_options, $data_dir, $username; > $avelsieve_file_backend_options = array( >? ? ? 'avelsieve_default_file' =>? "$data_dir/$username.sievesource" > ); > > /*global $avelsieve_oldcyrus; > $avelsieve_oldcyrus = true;*/ > > global $avelsieve_enable_envelope_auth; > $avelsieve_enable_envelope_auth = true; > > global $avelsieve_custom_sieve_implementation; > $avelsieve_custom_sieve_implementation = ''; > > global $avelsieve_striproot; > $avelsieve_striproot = ''; > > global $avelsieve_hardcoded_capabilities; > $avelsieve_hardcoded_capabilities = array( >? ? ? 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' > ); > > /*global $avelsieve_imapproxymode, $avelsieve_imapproxyserv; > $avelsieve_imapproxymode = false; > $avelsieve_imapproxyserv = array( >? ? ? 'localhost' =>? 'imap.example.org' > );*/ > > global $avelsieve_ldapuserdatamode; > $avelsieve_ldapuserdatamode = false; > > /*global $avelsieve_cyrusadmins_map; > $avelsieve_cyrusadmins_map = array( >? ? ? 'cyrusimap' =>? 'cyrussieve' > );*/ > > $conservative = false; > > $useimages = true; > > global $translate_return_msgs; > $translate_return_msgs = false; > > $imagetheme = 'famfamfam'; > //$imagetheme = 'bluecurve_24x24'; > //$imagetheme = 'bluecurve_16x16'; > > global $startitems; > $startitems = 3; > > global $maxitems; > $maxitems = 10; > > global $headers; > $headers = array( >? 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Reply-To', 'Sender', 'List-Id', >? 'MailingList', 'Mailing-List', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', >? 'Resent-From',? 'Resent-To', 'X-Mailer', 'X-Mailing-List', >? 'X-Spam-Flag', 'X-Spam-Status', >? 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', >? 'Return-Path', 'Received', 'Auto-Submitted', >? 'X-Spam-Flag', 'X-Spam-Status','X-Spam-Tests' >? ); > > global $notifymethods; > $notifymethods = array( > 'mailto', 'sms' > ); > > //$notifymethods = false; > > // $disable_avelsieve_capabilities = array("notify"); > global $disable_avelsieve_capabilities; > $disable_avelsieve_capabilities = array(); > > global $avelsieveheaderlink; > $avelsieveheaderlink = true; > > global $avelsieve_default_mode; > $avelsieve_default_mode = 'terse'; > > global $avelsieve_enable_rules; > $avelsieve_enable_rules = array(); > > global $avelsieve_rules_settings; > $avelsieve_rules_settings = array(); > foreach($avelsieve_enable_rules as $r) { >? ? ? if(file_exists(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php')) { >? ? ? ? ? require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php'); >? ? ? } else { >? ? ? ? ? require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.default.php'); >? ? ? } > } > > $avelsieve_spam_highlight_enable = false; > > > Hope this help you to help me. > > Thank you > > > > > ________________________________ >? From: Tamsy > To: "dovecot at dovecot.org" > Sent: Tuesday, 17 April 2012 1:25 PM > Subject: Re: [Dovecot] vacation plugins for squirrelmail > > Show us your avelsieve-config in > your-squirrelmail-dir/plugins/avelsieve/config/config.php > > > Daminto Lie wrote the following on 17.04.2012 09:31: >> Thank you folks for your replies. >> >> >> I realised that the dovecot 1.2.9 that I installed via ubuntu synaptic package manager has already come with sieve plugins. After making some modifications on the dovecot.conf file, I went to download and install avelsieve 1.0.1 (stable version) onto the squirrelmail. Then, I run in squirrelmail conf.pl and added the plugin. I logged in and could see "Filter" option right on the top header row. I went to click it and nothing came up. I checked in the mail.log but did not see any errors there. >> >> Here is my snapshot of dovecot configuration >> # 1.2.9: /etc/dovecot/dovecot.conf >> # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS ext4 >> log_timestamp: %Y-%m-%d %H:%M:%S >> protocols: imap >> listen: *:143 >> ssl: >>? ? no >> disable_plaintext_auth: no >> login_dir: /var/run/dovecot/login >> login_executable: /usr/lib/dovecot/imap-login >> verbose_proctitle: yes >> first_valid_uid: 1001 >> last_valid_uid: 600 >> mail_privileged_group: mail >> mail_location: maildir:/var/vmail/%u/Maildir >> mbox_write_locks: fcntl dotlock >> imap_client_workarounds: outlook-idle delay-newmail netscape-eoh tb-extra-mailbox-sep >> lda: >>? ? ? postmaster_address: postmaster at ourcompany.com >>? ? ? mail_plugins: sieve >>? ? ? quota_full_tempfail: yes >>? ? ? deliver_log_format: msgid=%m: %$ >>? ? ? sendmail_path: /usr/lib/sendmail >>? ? ? rejection_reason: Your message to<%t>? was automatically rejected:%n%r >>? ? ? auth_socket_path: /var/run/dovecot-auth-master >> auth default: >>? ? ? mechanisms: plain login >>? ? ? username_format: %Lu >>? ? ? verbose: yes >>? ? ? debug: yes >>? ? ? debug_passwords: yes >>? ? ? passdb: >>? ? ? ? driver: >>? ? pam >>? ? ? passdb: >>? ? ? ? driver: ldap >>? ? ? ? args: /etc/dovecot/dovecot-ldap.conf >>? ? ? userdb: >>? ? ? ? driver: passwd >>? ? ? userdb: >>? ? ? ? driver: static >>? ? ? ? args: uid=1001 gid=1001 home=/var/vmail/%u allow_all_users=yes >>? ? ? socket: >>? ? ? ? type: listen >>? ? ? ? client: >>? ? ? ? ? path: /var/spool/postfix/private/auth >>? ? ? ? ? mode: 432 >>? ? ? ? ? user: postfix >>? ? ? ? ? group: mail >>? ? ? ? master: >>? ? ? ? ? path: /var/run/dovecot-auth-master >>? ? ? ? ? mode: 432 >>? ? ? ? ? user: vmail >>? ? ? ? ? group: vmail >> plugin: >>? ? ? sieve: /var/vmail/%u/.sieve >> >> Also, I have restarted dovecot after making changes to the dovecot.conf. >> >> Any help would be very much appreciated. >> >> Thank you >> >>? >> >> ________________________________ >>? ? From: Artur Zaprza?a >> To: Dovecot Mailing List >> Cc: Daminto Lie >> Sent: Thursday, 12 April 2012 11:46 PM >> Subject: Re: [Dovecot] vacation plugins for squirrelmail >> >> Daminto Lie wrote: >>> Hi, >>> >>> I am afraid I >>? ? have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. >>> Any help would be very much appreciated. >>> >>> Thank you >>> >> I'm using Avelsieve 1.9.9 with a set of my own bugfixes: >> http://email.uoa.gr/avelsieve/ >> >> From cor at xs4all.nl Wed Apr 18 11:22:32 2012 From: cor at xs4all.nl (Cor Bosman) Date: Wed, 18 Apr 2012 10:22:32 +0200 Subject: [Dovecot] trash plugin Message-ID: <6332F986-C1C4-421E-9EFC-362D429047F4@xs4all.nl> The trash plugin docs say: "Normally if a message can't be saved/copied because it would bring user over quota, the save/copy fails with "Quota exceeded" error. The trash plugin can be used to avoid such situations by making Dovecot automatically expunge oldest messages from configured mailboxes until the message can be saved" This leads me to believe that if a message cant be stored, the plugin would avoid returning a quota error, but instead delete/expunge messages until the original message can fit, and then complete the command. But I see something different, at least using the plugin in imap (havent tried it on lmtp). When I drag a message to a folder thats almost full, and the message doesnt fit, I get an actual quota error: 23 uid copy 454343 "Spam" 23 NO [OVERQUOTA] Quota exceeded (mailbox for user is full) After my client sees this error, I see the trash plugin creating space. But then it's too late right? The client already has a visible error. Am I just misunderstanding how the trash plugin is supposed to work? Dovecot 2.1.4, config is on a set of test servers: namespace { type = private separator = / inbox = yes } namespace { type = private separator = / inbox = no prefix = Spam/ location = maildir:/var/spool/mail/%1u/%1.1u/%u/.Spam:INDEX=/var/spool/mail/dovecot-control/indexes/%1u/%2u/%u/.Spam:CONTROL=/var/spool/mail/dovecot-control/%1u/%2u/%u/.Spam subscriptions = no list = yes hidden = yes } plugin { quota = maildir:Spam quota:ns=Spam/ quota_rule = *:storage=5000K } My goal is to create a FIFO spam folder that never returns a quota error, but instead just deletes the oldest emails as new spam comes in. Regards, Cor From cor at xs4all.nl Wed Apr 18 11:53:34 2012 From: cor at xs4all.nl (Cor Bosman) Date: Wed, 18 Apr 2012 10:53:34 +0200 Subject: [Dovecot] returning a namespace location from userdb? Message-ID: <567E4A7D-930E-4C57-B79D-8C94554D7C72@xs4all.nl> hey all, is it possible to return the location of namespace from the userdb lookup? The code is a bit unclear about it. There seems to be a part of the docs saying: If you want to override settings inside sections, you can separate the section name and key with '/'. For example: namespace default { inbox = yes } The inbox setting can be overridden by returning namespace/default/inbox=no extra field I realize you can set 'location=' in the namespace, but we have mailbox locations that cant be expressed by the normal variable substitution vars in dovecot (dont ask). So we have a userdb that returns the correct mail location. This works fine. But im currently trying to implement a second namespace, and Im having trouble returning a mailbox location for this specific namespace from our userdb. Is this possible? If so, what would the userdb have to return exactly? namespace spam { type = private separator = / inbox = no prefix = Spam/ subscriptions = no list = yes hidden = yes } Regards, Cor From pw at wk-serv.de Wed Apr 18 12:49:14 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Wed, 18 Apr 2012 11:49:14 +0200 Subject: [Dovecot] Missing 20-managesieve.conf in 2.1.4? Message-ID: <4F8E8E1A.3080408@wk-serv.de> Hello, I installed Dovecot 2.1.4 and Pigeonhole 0.3.0 and wondered that there is no 20-managesieve.conf file anymore. It is no problem to use one of my old files. I just want to know whether this is a bug or not. Regards Patrick From CMarcus at Media-Brokers.com Wed Apr 18 13:22:27 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 18 Apr 2012 06:22:27 -0400 Subject: [Dovecot] OT Re: Dovecot and Outlook Express "plaintext authentication not allowed..." In-Reply-To: <1334695280.2248.YahooMailNeo@web161606.mail.bf1.yahoo.com> References: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> <4F8D455D.9060805@Media-Brokers.com> <1334695280.2248.YahooMailNeo@web161606.mail.bf1.yahoo.com> Message-ID: <4F8E95E3.6020500@Media-Brokers.com> On 2012-04-17 4:41 PM, D Chen wrote: > Per your suggestion, in Outlook Express I changed to use SSL on port > 993 for IMAP, and port 465 for SMTP, the creation of the new account > succeeded without error! However, when tried to send to my yahoo mail or > gmail acct, got the following error: (apparently failed to send!) Changed subject to OT because this is not relevant to dovecot... > "The message could not be sent because one of the recipients was rejected by the server. The > rejected email address was 'dchenusa at yahoo.com.' Subject: 'test out to yahoo mail' > Account:'dchen192.168.20.100' > Server: 192.168.20.100 > Protocol: SMTP, server response: 554 5.7.1 That is a 'Relay access denied' error (assuming yahoo uses correct error codes, which is probably a bad assumption)... Either Yahoo doesn't support port 465 (try the submission port for your yahoo smtp server - 587)... or you have a free account which doesn't allow relaying using their servers (so you'd have to use something else for sending - what does the Thunderbird client use for sending for that account?)... But as suggested... don't use Outlook Express, use a modern mail client (Thunderbird is my preference, but someone else suggested Windows Live Mail which does suck much less than Outlook Express)... -- Best regards, Charles From stephan at rename-it.nl Wed Apr 18 13:34:15 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 18 Apr 2012 12:34:15 +0200 Subject: [Dovecot] Missing 20-managesieve.conf in 2.1.4? In-Reply-To: <4F8E8E1A.3080408@wk-serv.de> References: <4F8E8E1A.3080408@wk-serv.de> Message-ID: <4F8E98A7.2040305@rename-it.nl> On 4/18/2012 11:49 AM, Patrick Westenberg wrote: > Hello, > > I installed Dovecot 2.1.4 and Pigeonhole 0.3.0 and wondered that there > is no 20-managesieve.conf file anymore. > > It is no problem to use one of my old files. I just want to know whether > this is a bug or not. I wouldn't know why it is missing in your installation. It is still contained in the tarball. How did you install it exactly? Regards, Stephan. From pw at wk-serv.de Wed Apr 18 14:06:34 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Wed, 18 Apr 2012 13:06:34 +0200 Subject: [Dovecot] Missing 20-managesieve.conf in 2.1.4? In-Reply-To: <4F8E98A7.2040305@rename-it.nl> References: <4F8E8E1A.3080408@wk-serv.de> <4F8E98A7.2040305@rename-it.nl> Message-ID: <4F8EA03A.7000601@wk-serv.de> Stephan Bosch schrieb: > I wouldn't know why it is missing in your installation. It is still > contained in the tarball. How did you install it exactly? I installed it the same way as usual: ./configure make make install No errors reported. I checked the configure options and also tried --with-managesieve but the file is still missing. However, this is my first installation on Scientific Linux. Patrick From john.robinson at anonymous.org.uk Wed Apr 18 14:36:51 2012 From: john.robinson at anonymous.org.uk (John Robinson) Date: Wed, 18 Apr 2012 12:36:51 +0100 Subject: [Dovecot] mbox folders on-disc layout Message-ID: <4F8EA753.4070605@anonymous.org.uk> I'd like to change the on-disk layout for my mboxes from being ~/mail/foo/bar to ~/mail/foo.bar so that I can have folders containing both messages and subfolders, without having subfolders beginning with . and then having to do all the other fiddling with locations of index files etc. that goes along with LAYOUT=maildir++, and also leaving the filesystem layout looking sensible so my users see their mailboxes instead of the index and control files. Is this possible or do I need to patch something? (CentOS 6, dovecot 2.0.9) Cheers, John. From stephan at rename-it.nl Wed Apr 18 15:42:14 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 18 Apr 2012 14:42:14 +0200 Subject: [Dovecot] http://xi.rename-it.nl upgrade 2.0 - 2.1 dovecot-common (<< 2:2.1.alpha1-0~auto+47) In-Reply-To: <4F8AC9B1.2030307@schetterer.org> References: <4F8AC9B1.2030307@schetterer.org> Message-ID: <4F8EB6A6.6010206@rename-it.nl> Op 4/15/2012 3:14 PM, Robert Schetterer schreef: > Hi, > i got > depency problem like dovecot-common (<< 2:2.1.alpha1-0~auto+47) > > dpkg -i dovecot-common_2.1.4-0~auto+3_all.de > > after all dove seems to run without install dovecot-common basicly You should be able to drop dovecot-common entirely. It is nothing but a dummy package now. Regards, Stephan. From tss at iki.fi Wed Apr 18 16:45:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 16:45:45 +0300 Subject: [Dovecot] Statistics scripts Message-ID: <42DEDB45-6B9A-47BB-BFB1-B82ECC086839@iki.fi> I wrote two scripts, which you can use with the stats plugin enabled to get a better "doveadm stats top" than it currently is: http://dovecot.org/tools/stats-top.pl http://dovecot.org/tools/stats.pl From tss at iki.fi Wed Apr 18 19:29:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 19:29:58 +0300 Subject: [Dovecot] trash plugin In-Reply-To: <6332F986-C1C4-421E-9EFC-362D429047F4@xs4all.nl> References: <6332F986-C1C4-421E-9EFC-362D429047F4@xs4all.nl> Message-ID: <14B830E2-9EFD-45CA-B0F7-BEDDF0FCDFF1@iki.fi> On 18.4.2012, at 11.22, Cor Bosman wrote: > The trash plugin docs say: > > "Normally if a message can't be saved/copied because it would bring user over quota, the save/copy fails with "Quota exceeded" error. The trash plugin can be used to avoid such situations by making Dovecot automatically expunge oldest messages from configured mailboxes until the message can be saved" > > This leads me to believe that if a message cant be stored, the plugin would avoid returning a quota error, but instead delete/expunge messages until the original message can fit, and then complete the command. Yes. > But I see something different, at least using the plugin in imap (havent tried it on lmtp). When I drag a message to a folder thats almost full, and the message doesnt fit, I get an actual quota error: > > 23 uid copy 454343 "Spam" > 23 NO [OVERQUOTA] Quota exceeded (mailbox for user is full) > > After my client sees this error, I see the trash plugin creating space. But then it's too late right? The client already has a visible error. Am I just misunderstanding how the trash plugin is supposed to work? Fixed: http://hg.dovecot.org/dovecot-2.1/rev/51b40cffb628 From tss at iki.fi Wed Apr 18 19:42:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 19:42:21 +0300 Subject: [Dovecot] returning a namespace location from userdb? In-Reply-To: <567E4A7D-930E-4C57-B79D-8C94554D7C72@xs4all.nl> References: <567E4A7D-930E-4C57-B79D-8C94554D7C72@xs4all.nl> Message-ID: On 18.4.2012, at 11.53, Cor Bosman wrote: > hey all, is it possible to return the location of namespace from the userdb lookup? Yes. > But im currently trying to implement a second namespace, and Im having trouble returning a mailbox location for this specific namespace from our userdb. Is this possible? If so, what would the userdb have to return exactly? > > namespace spam { namespace/spam/location=maildir:/something/different From tss at iki.fi Wed Apr 18 19:45:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 19:45:13 +0300 Subject: [Dovecot] mbox folders on-disc layout In-Reply-To: <4F8EA753.4070605@anonymous.org.uk> References: <4F8EA753.4070605@anonymous.org.uk> Message-ID: <35C2B470-438E-4CEC-A42A-0BC2C39A0159@iki.fi> On 18.4.2012, at 14.36, John Robinson wrote: > I'd like to change the on-disk layout for my mboxes from being > ~/mail/foo/bar > to > ~/mail/foo.bar > so that I can have folders containing both messages and subfolders, without having subfolders beginning with . and then having to do all the other fiddling with locations of index files etc. that goes along with LAYOUT=maildir++, and also leaving the filesystem layout looking sensible so my users see their mailboxes instead of the index and control files. > > Is this possible or do I need to patch something? Not possible without patching. There's also the other possibility of doing something like: # Trick mbox configuration which allows a mail folder which contains both # messages and sub-folders mail_location = mbox:~/mail/mailboxes:DIRNAME=mBoX-MeSsAgEs:INDEX=~/mail/index:CONTROL=~/mail/control from http://wiki2.dovecot.org/MboxChildFolders From tss at iki.fi Wed Apr 18 19:46:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 19:46:42 +0300 Subject: [Dovecot] 487 imap-login processes - is this ok In-Reply-To: <4F8D7C73.9090700@filmakademie.de> References: <4F8D7C73.9090700@filmakademie.de> Message-ID: <74640698-B3E5-46BB-88FC-98B088CF0330@iki.fi> On 17.4.2012, at 17.21, G?tz Reinicke wrote: > recently I checked our mailserver for some more tuning and optimizing > and noticed, that currently I do have 487 imap-login processes and I'm > asking myself if that's o.k. > > We do have about 1.000 user in total and not everybody is currently > logged in. > > May be this is o.k. but if not, what might I check? Normal in "high-security mode": http://wiki.dovecot.org/LoginProcess Most of those 487 processes are probably handling SSL traffic. From tss at iki.fi Wed Apr 18 19:47:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 19:47:11 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F8DEEBD.9080206@corp.sonic.net> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F8DEEBD.9080206@corp.sonic.net> Message-ID: <8B60CEC2-4945-4513-9873-CCEEA0344C7A@iki.fi> On 18.4.2012, at 1.29, Kelsey Cummings wrote: > On 03/30/12 07:35, Timo Sirainen wrote: >> IP stickiness isn't enough if user uses more than one IMAP client, which is pretty common nowadays. And doesn't help at all with LDA. > > Timo, having the LDA flow through the director with maildir and NFS is only an issue if you want to use the dovecot delivery agent since it manipulates the indexes on delivery, right? This shouldn't be a problem if you are running a completely distinct delivery cluster running a LDA like procmail. Right. If indexes or dovecot-uidlist isn't updated then it doesn't matter. From tss at iki.fi Wed Apr 18 19:57:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 19:57:26 +0300 Subject: [Dovecot] Re-indexing of fts-solr not working In-Reply-To: References: Message-ID: <1766B7F6-4A32-41DA-9655-13E0AE43A8C7@iki.fi> On 17.4.2012, at 21.36, Joe Beaubien wrote: > I am currently testing a 2.1 dovecot setup with fts-solr. Search speed is > simply amazing. I decided to try to delete and rebuild the solr indexes, > but i can't seem to rebuild them thru dovecot. With Maildir you could just delete dovecot.index* files. But with dbox that's not a good idea. Anyway, implemented now the minimum needed to get doveadm fts rescan to work: http://hg.dovecot.org/dovecot-2.1/rev/d145ea6f7061 From tss at iki.fi Wed Apr 18 20:01:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 20:01:14 +0300 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken In-Reply-To: <4F8D80D3.2050505@schetterer.org> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> Message-ID: On 17.4.2012, at 17.40, Robert Schetterer wrote: > doveadm acl debug > > shows on both versions > > Fatal: ACL shared dict iteration failed Doesn't it show any other error message before this? Either to terminal or to error log? What's your doveconf -n? From tss at iki.fi Wed Apr 18 20:06:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 20:06:45 +0300 Subject: [Dovecot] LMTP auth problem In-Reply-To: <20120414172135.GA32608@xs4all.nl> References: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> <71BEFF67-4A0A-442A-9ED0-C4B1B23A4E7F@leuxner.net> <20120414172135.GA32608@xs4all.nl> Message-ID: On 14.4.2012, at 20.21, Cor Bosman wrote: > My config was the same as yours. That didnt work for me. But if I add > > user = dovecot > mode = 0666 Hmm. Perhaps I should make this the default. > That does work. Of course, the difference between 777 and 666 is > minimal. I think 666 is handled as a special case in the code? Yes, it's handled specially. 0666 is now a safe default for multi-user systems, since it returns data for a user only if the calling process's UID matches the returned "uid". 0777 makes it truly world-accessible. From news0104.6.weezy at spamgourmet.com Wed Apr 18 20:00:48 2012 From: news0104.6.weezy at spamgourmet.com (Andreas Helmcke) Date: Wed, 18 Apr 2012 19:00:48 +0200 Subject: [Dovecot] namespace hierarchy In-Reply-To: References: <201204010928.41017.anyaddress@gmx.net> Message-ID: <4F8EF340.4010407@spamgourmet.com> Am 04.04.2012 03:47, schrieb Timo Sirainen: > On 1.4.2012, at 10.28, Tom Fernandes wrote: > >> In the new server I would like to have 3 namespaces (private, public, shared) >> and use "/" as separator. Is it possible to keep "INBOX." for private, so that >> the users mailboxes appear below the Inbox, but have public and shared on the >> same hierarchy-level like the Inbox? >> > > If you have any subscriptions=no namespaces, you need a parent namespace that has subscriptions=yes. You could have e.g.: > > namespace { > prefix = INBOX/ > list = no > inbox = yes > } > namespace { > prefix = > list = no > hidden = yes > alias_for = INBOX/ > } > > So now the prefix="" is used for the subscriptions but nothing else really. I tried this, but sadly with dovecot V2.1.1 it is not working. Steps to reproduce: + create empty mail directory + create one folder (Allgemeines/Tabel) in public namespace with doveadm + start mail-client (in my case thunderbird) + try to subscribe to the new public folder Error message in log: Apr 18 18:19:34 elablnmail02 dovecot: imap(ahelmcke): Warning: Subscriptions file /home/mail/user/ahelmcke/subscriptions: Removing invalid entry: Allgemeines/Tabel doveconf -n: # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) ext4 auth_master_user_separator = * first_valid_uid = 400 mail_gid = vmail mail_location = maildir:/home/mail/user/%u mail_uid = vmail mbox_write_locks = fcntl namespace Allgemeines { location = maildir:/home/mail/Allgemeines:INDEX=/home/mail/user/%u/SEEN.Allgemeines prefix = Allgemeines/ separator = / subscriptions = no type = public } namespace dummy { alias_for = INBOX/ hidden = yes list = no location = prefix = separator = / type = private } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile sieve = /var/lib/dovecot/sieve/%u.sieve sieve_global_dir = /var/lib/dovecot/sieve/global/ sieve_global_path = /var/lib/dovecot/sieve/default.sieve } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } service imap-login { inet_listener imap { address = * } inet_listener imaps { address = * } } ssl = required ssl_cert = References: <201204010928.41017.anyaddress@gmx.net> <4F8EF340.4010407@spamgourmet.com> Message-ID: <305E07F6-1D75-4B6D-862A-D7697442A770@iki.fi> On 18.4.2012, at 20.00, Andreas Helmcke wrote: >> If you have any subscriptions=no namespaces, you need a parent namespace that has subscriptions=yes. You could have e.g.: >> >> namespace { >> prefix = INBOX/ >> list = no >> inbox = yes >> } >> namespace { >> prefix = >> list = no >> hidden = yes >> alias_for = INBOX/ >> } >> >> So now the prefix="" is used for the subscriptions but nothing else really. > > I tried this, but sadly with dovecot V2.1.1 it is not working. Oh, I hadn't thought of one problem: It's now using the same subscriptions file for both prefix="" and prefix=INBOX/ namespaces. This confuses it. So you need to place the prefix="" namespace somewhere else, for example: > namespace dummy { > alias_for = INBOX/ location = maildir:/home/mail/user/%u/public > hidden = yes > list = no > location = > prefix = > separator = / > type = private > } From robert at schetterer.org Wed Apr 18 20:44:23 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 18 Apr 2012 19:44:23 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken In-Reply-To: References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> Message-ID: <4F8EFD77.90103@schetterer.org> Am 18.04.2012 19:01, schrieb Timo Sirainen: > On 17.4.2012, at 17.40, Robert Schetterer wrote: > >> doveadm acl debug >> >> shows on both versions >> >> Fatal: ACL shared dict iteration failed > > Doesn't it show any other error message before this? Either to terminal or to error log? What's your doveconf -n? Hi Timo, i didnt found more usefull errors before all are like Debug: acl: No lookup right to mailbox: but that istn true, i checked it in the filesystem and tried new acls etc ( and they working under 2.0.20 ) to me it looks like some acl check/list problem perhaps depend on stuff like in i.e http://hg.dovecot.org/dovecot-2.1/rev/65a75939ac2c as i said the old 2.0.20 works still fine on the other server the only change i did was with virtual and namespace inbox to get special_use working > this is from the dove version # 2.1.4 (85ad4baedd43): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-37-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 50 M auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes auth_verbose_passwords = plain auth_worker_max_count = 300 dict { acl = mysql:/etc/dovecot/dovecot-dict-acl-sql.conf.ext expire = mysql:/etc/dovecot/dovecot-dict-expire-sql.conf.ext quotadict = mysql:/etc/dovecot/dovecot-dict-quota-sql.conf.ext } disable_plaintext_auth = no first_valid_gid = 1001 first_valid_uid = 1001 hostname = mail01.example.com last_valid_gid = 1001 last_valid_uid = 1001 listen = * login_greeting = imap, pop ready mail_access_groups = vmail mail_debug = yes mail_fsync = always mail_gid = 1001 mail_location = maildir:~/ mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = " stats zlib expire virtual fts fts_squat mail_log autocreate notify acl listescape" mail_privileged_group = vmail mail_uid = 1001 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags notify mmap_disable = yes namespace { list = yes location = maildir:/usr/local/virtual/%%d/%%u:INDEX=~/shared/%%d/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { hidden = no list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } mailbox name { special_use = \Drafts \Junk \Sent \Trash } prefix = separator = / subscriptions = yes } namespace real { hidden = yes list = no location = prefix = RealMails/ separator = / } namespace virtual { hidden = yes list = no location = virtual:/etc/dovecot/virtual2:LAYOUT=maildir++:INDEX=~/virtual prefix = virtual/ separator = / } passdb { args = /etc/dovecot/dovecot-sql-master.conf.ext driver = sql master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl autocreate = Trash autocreate2 = Junk autocreate3 = Sent autocreate4 = Drafts autocreate5 = Templates autocreate6 = Hostmaster-Backup autocreate7 = archiv-backup-mailspooler autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Drafts autosubscribe5 = Templates expire = Trash expire2 = Trash/* expire3 = Junk expire4 = Junk/* expire5 = Hostmaster-Backup expire6 = Hostmaster-Backup/* expire7 = archiv-backup-mailspooler/* expire_dict = proxy::expire fts = squat home = /usr/local/virtual/%d/%u mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size mail_log_group_events = yes quota = dict:::proxy::quotadict quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /usr/local/virtual/%d/%u/dovecot.sieve sieve_before = /etc/dovecot/sieve/global.sieve sieve_dir = /usr/local/virtual/%d/%u/sieve sieve_execute_bin_dir = /usr/lib/dovecot/sieve-execute sieve_execute_socket_dir = sieve-execute sieve_extensions = +notify +imapflags sieve_filter_bin_dir = /usr/lib/dovecot/sieve-filter sieve_filter_socket_dir = sieve-filter sieve_global_dir = /etc/dovecot/sieve/ sieve_global_extensions = +vnd.dovecot.execute +vnd.dovecot.pipe +vnd.dovecot.filter sieve_global_path = /etc/dovecot/sieve/global.sieve sieve_pipe_bin_dir = /usr/lib/dovecot/sieve-pipe sieve_pipe_socket_dir = sieve-pipe sieve_plugins = sieve_extprograms stats_refresh = 30 secs stats_track_cmds = yes zlib_save = gz zlib_save_level = 6 } postmaster_address = hostmaster at mail01.example.com protocols = imap pop3 lmtp sieve sendmail_path = /usr/lib/sendmail service anvil { client_limit = 1000 } service auth-worker { user = root } service auth { client_limit = 1000 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service dict { extra_groups = vmail unix_listener dict { group = vmail mode = 0660 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 8 service_count = 1 } service imap-postlogin { executable = script-login /usr/local/bin/postlogin_imap.sh } service imap { executable = imap imap-postlogin process_limit = 1024 vsz_limit = 256 M } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } vsz_limit = 256 M } service managesieve-login { inet_listener { address = 127.0.0.1 212.52.224.210 212.52.224.205 port = 2000 } vsz_limit = 256 M } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service pop3-postlogin { executable = script-login /usr/local/bin/postlogin_pop3.sh } service pop3 { executable = pop3 pop3-postlogin process_limit = 1024 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { mode = 0666 user = vmail } user = vmail } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl_ca = References: <1766B7F6-4A32-41DA-9655-13E0AE43A8C7@iki.fi> Message-ID: Awesome, thanks. Will this appear in the 2.1.5 release? On Wed, Apr 18, 2012 at 12:57 PM, Timo Sirainen wrote: > On 17.4.2012, at 21.36, Joe Beaubien wrote: > > > I am currently testing a 2.1 dovecot setup with fts-solr. Search speed is > > simply amazing. I decided to try to delete and rebuild the solr indexes, > > but i can't seem to rebuild them thru dovecot. > > With Maildir you could just delete dovecot.index* files. But with dbox > that's not a good idea. Anyway, implemented now the minimum needed to get > doveadm fts rescan to work: > > http://hg.dovecot.org/dovecot-2.1/rev/d145ea6f7061 > > From tss at iki.fi Wed Apr 18 22:43:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 22:43:46 +0300 Subject: [Dovecot] Re-indexing of fts-solr not working In-Reply-To: References: <1766B7F6-4A32-41DA-9655-13E0AE43A8C7@iki.fi> Message-ID: Sure. Everything in hg goes to the next release. On 18.4.2012, at 22.41, Joe Beaubien wrote: > Awesome, thanks. > > Will this appear in the 2.1.5 release? > > > On Wed, Apr 18, 2012 at 12:57 PM, Timo Sirainen wrote: > >> On 17.4.2012, at 21.36, Joe Beaubien wrote: >> >>> I am currently testing a 2.1 dovecot setup with fts-solr. Search speed is >>> simply amazing. I decided to try to delete and rebuild the solr indexes, >>> but i can't seem to rebuild them thru dovecot. >> >> With Maildir you could just delete dovecot.index* files. But with dbox >> that's not a good idea. Anyway, implemented now the minimum needed to get >> doveadm fts rescan to work: >> >> http://hg.dovecot.org/dovecot-2.1/rev/d145ea6f7061 >> >> From jtam.home at gmail.com Thu Apr 19 02:11:17 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 18 Apr 2012 16:11:17 -0700 (PDT) Subject: [Dovecot] alias_for Message-ID: I noticed the addition of a new namespace configuration "alias_for". What pracical difference is adding this over and above making namespace aliases hidden and disabling list, as in older versions of dovecot configurations? Joseph Tam From janfrode at tanso.net Thu Apr 19 11:22:45 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 19 Apr 2012 10:22:45 +0200 Subject: [Dovecot] migrate 15K users to new domain name Message-ID: <20120419082245.GA4050@dibs.tanso.net> I need to migrate 15K users to a new domain name, and plan to use "dsync mirror" in the transition phase. Could someone confirm that this should work: Before giving users access to new-domain do a first sync to get all the stale data over: for user in $old-domain; do dsync mirror $user at old-domain $user at new-domain done Configure sieve vacation filter to forward all messages from $user at old-domain to $user at new-domain, and notify sender of changed address. Give users access to both new-domain and old-domain, and do a final sync. for user in $old-domain; do dsync mirror $user at old-domain $user at new-domain dsync mirror $user at old-domain $user at new-domain # twice in case the first was slow drop all messages for $user at old-domain Leave notice message for $user at old-domain saying he should use new-domain done Wait a few weeks/months, and then drop all users at old-domain. Does this look sensible? -jf From dchenusa at yahoo.com Thu Apr 19 11:44:35 2012 From: dchenusa at yahoo.com (D Chen) Date: Thu, 19 Apr 2012 01:44:35 -0700 (PDT) Subject: [Dovecot] IMAP SSL incoming test need help ! Message-ID: <1334825075.56841.YahooMailNeo@web161604.mail.bf1.yahoo.com> Ubuntu 11.10 server with postfix/dovecot/squirrelmail configured. Outgoing SMTP server seemed worked fine, however, the incoming IMAP with SSL on port 993 doesn't seem to receive any mail at all !? ?Here is the mail.log during the test session from an external yahoo mail (dchenusa at yahoo.com) to the internal server testmail.biokeyinc.com (dchen at testmail.biokeyinc.com) The log did NOT show any error ! ?The sender showed sent mail! but no mail reach to the destination! Any help would be appreciated. Apr 19 01:20:02 testmail postfix/smtpd[4103]: connect from nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:03 testmail postfix/smtpd[4103]: 091ED200973: client=nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:03 testmail postfix/cleanup[4108]: 091ED200973: message-id=<32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com> Apr 19 01:20:03 testmail postfix/qmgr[2007]: 091ED200973: from=, size=2840, nrcpt=1 (queue active) Apr 19 01:20:03 testmail postfix/smtpd[4103]: disconnect from nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:12 testmail postfix/smtpd[4113]: connect from localhost[127.0.0.1] Apr 19 01:20:12 testmail postfix/smtpd[4113]: 33AE9200AE9: client=localhost[127.0.0.1] Apr 19 01:20:12 testmail postfix/cleanup[4108]: 33AE9200AE9: message-id=<32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com> Apr 19 01:20:12 testmail postfix/qmgr[2007]: 33AE9200AE9: from=, size=3536, nrcpt=1 (queue active) Apr 19 01:20:12 testmail postfix/smtpd[4113]: disconnect from localhost[127.0.0.1] Apr 19 01:20:12 testmail amavis[2042]: (02042-06) Passed CLEAN, [98.138.91.180] [98.138.87.4] -> , Message-ID: <32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com>, mail_id: rqyVZiQDwSsG, Hits: -0.109, size: 2837, queued_as: 33AE9200AE9, dkim_id=dchenusa at yahoo.com, at yahoo.com, 8647 ms Apr 19 01:20:12 testmail postfix/smtp[4109]: 091ED200973: to=, relay=127.0.0.1[127.0.0.1]:10024, delay=9.5, delays=0.87/0.01/0/8.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 33AE9200AE9) Apr 19 01:20:12 testmail postfix/qmgr[2007]: 091ED200973: removed Apr 19 01:20:12 testmail postfix/local[4114]: 33AE9200AE9: to=, relay=local, delay=0.18, delays=0.1/0/0/0.07, dsn=2.0.0, status=sent (delivered to command: procmail -a "$EXTENSION") Apr 19 01:20:12 testmail postfix/qmgr[2007]: 33AE9200AE9: removed Apr 19 01:21:30 testmail dovecot: imap-login: Login: user=, method=PLAIN, rip=63.195.90.22, lip=192.168.20.100, mpid=4122, TLS Apr 19 01:23:13 testmail dovecot: imap(dchen): Disconnected: Disconnected in IDLE bytes=99/708 Apr 19 01:23:23 testmail postfix/anvil[4105]: statistics: max connection rate 1/60s for (smtp:98.138.91.180) at Apr 19 01:20:02 Apr 19 01:23:23 testmail postfix/anvil[4105]: statistics: max connection count 1 for (smtp:98.138.91.180) at Apr 19 01:20:02 Apr 19 01:23:23 testmail postfix/anvil[4105]: statistics: max cache size 1 at Apr 19 01:20:02 From pw at wk-serv.de Thu Apr 19 12:34:50 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 19 Apr 2012 11:34:50 +0200 Subject: [Dovecot] =?utf-8?q?Missing_20-managesieve=2Econf_in_2=2E1=2E4=3F?= In-Reply-To: <4F8E98A7.2040305@rename-it.nl> References: <4F8E8E1A.3080408@wk-serv.de> <4F8E98A7.2040305@rename-it.nl> Message-ID: Hi Stephan, after re-installing the whole system the files have been installed properly. Patrick From stan at hardwarefreak.com Thu Apr 19 12:40:43 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 19 Apr 2012 04:40:43 -0500 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8D69AB.5090706@esiee.fr> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <4F8D69AB.5090706@esiee.fr> Message-ID: <4F8FDD9B.6060007@hardwarefreak.com> On 4/17/2012 8:01 AM, Frank Bonnet wrote: > have 4000/6000 imaps concurent connections during working hours . > > POP3 users will be very few How much disk space do you plan to offer per user mail directory? Will you be using quotas? >>> I need some feedbacks advices of experienced admins >>> I will have to setup in few monthes an email system >>> for approx 50K "intensives" users. >>> >>> The only mandatory thing will be I must use HP proliant servers >>> >>> The operating system will be FreeBSD or Linux Quite a coincidence Frank. It's a shame it has to be an HP solution. I just finished designing a high quality high performance 4U 72 drive server yesterday that will easily handle 15K concurrent IMAP users, for only ~$24K USD, $0.48/user @50K users. So it may not be of interest to you, but maybe to others. It is capable of ~7K random 4KB r/w IOPS sustained, has 10TB net space for an average ~200MB/user mail directory assuming 50K users. The parts for this machine run ~$24K USD at Newegg. I just made the wishlist public so it should be available tomorrow or Friday. I'll provide the link when it's available. All components used are top quality, best available in the channel. The reliability of the properly assembled server will rival that of any HP/Dell/IBM machine. For those not familiar with SuperMicro, they manufacture many of Intel's retail boards and have for a decade+. The majority of the COTS systems used in large academic HPC clusters are built with SuperMicro chassis and motherboards, as well as some 1000+ node US DOE clusters. Here are the basics: 72x 2.5" bay 4U chassis, 3x SAS backplanes each w/redundant expanders: http://www.newegg.com/Product/Product.aspx?Item=N82E16811152212 78x Seagate 10K SAS 300GB drives--includes 6 spares Triple LSI 9261-8i dual port 512MB BBWC RAID controllers each with 2 redundant load balanced connections to a backplane 24 drives per controller for lowest latency, maximum throughput, 1.5GB total write cache, a rebuild affects only one controller, etc SuperMicro mainboard, 2x 6-core 3.3GHz AMD Interlagos Opteron CPUs 64GB Reg ECC DDR3-1066, 8x8GB DIMMs, 34GB/s aggregate bandwidth Dual Intel Quad port GbE NICs, 10 total Intel GbE ports Use the 2 mobo ports for redundant management links Aggregate 4 ports, 2 on each quad NIC, for mail traffic Aggregate the remaining 4 for remote backup, future connection to an iSCSI SAN array, etc Or however works best--having 8 GbEs gives flexibility and these two cards are only $500 of the total 2x Intel 20GB SSD internal fixed drives, hardware mirrored by the onboard LSI SAS chip, for boot/OS The key to performance, and yielding a single file tree, is once again using XFS to take advantage of this large spindle count across 3 RAID controllers. Unlike previous configurations where I recommended using a straight md concatenation of hardware RAID1 pairs, in this case we're going to use a concatenation of 6 hardware RAID10 arrays. There are a couple of reasons for doing so in this case: 1. Using 36 device names in a single md command line is less than intuitive and possibly error prone. Using 6 is more manageable. 2. We have 3 BBWC RAID controllers w/24 drives each. This is a high performance server and will see a high IO load in production. In many cases one would use an external filesystem journal, which we could easily do and get great performance with our mirrored SSDs. However, the SSDs are not backed by BBWC, so a UPS failure or system crash could hose the log journal. So we'll go with the default internal journal which will be backed by the BBWC. Going internal with the log in this mail scenario can cause a serious amount of extra IOPS on the filesystem data section, this being Allocation Group 0. If we did the "normal" RAID1 concat, all the log IO would hit the first RAID1 pair. On this system, the load may hit that spindle pretty hard, making access to mailboxes in AG0 slower than others. With 6 RAID10 arrays in a concat, the internal log writes will be striped across 6 spindles in the first array. With 512MB BBWC backing that array and optimizing writeout, and with delaylog, this will yield optimal log write performance without slowing down mailbox file access in AG0. To create such a setup we'd do something like this, assuming the mobo LSI controller yields sd[ab], and the 6 array devices on the PCIe LSI cards yield sd[cdefgh] 1. Create two RAID10 arrays, each of 12 drives, in the WebBIOS GUI of each LSI card, using a strip size of 32KB which should yield good random r/w performance for any mailbox format. Use the following policies for each array: RW, Normal, Wback, Direct, Disable, No, and use the full size. Create the concatenated md device: $ mdadm -C /dev/md0 -l linear -n 6 /dev/sd[cdefgh] Then we format it with XFS, optimizing the AG layout for our mailbox workload, and allocation write stripe alignment to each hardware array: $ mkfs.xfs -d agcount=24 su=32k sw=6 /dev/md0 This yields 4 AGs per RAID10 array which will minimize the traditional inode64 head seeking overhead on striped arrays, while still yielding fantastic allocation parallelism with 24 AGs. Optimal fstab for MTA queue/mailbox workload, assuming kernel 2.6.39+: /dev/md0 /mail xfs defaults,inode64,nobarrier 0 0 We disable write barriers as we have BBWC. And that 1.5GB of BBWC will yield extremely low Dovecot write latency and throughput. Given the throughput available, if you're running Postfix on this box, you will want to create a directory on this filesystem for the Postfix spool. Postfix puts the spool files in many dozens, hundreds of subdirectories, so you'll get 100% parallelism across all AGs, thus all disks. It's very likely none of you will decide to build this system. My hope is that some of the design concepts and components used, along with the low cost but high performance of this machine, may be educational or simply give people new ideas, steer them in directions they may not have previously considered. -- Stan From odhiambo at gmail.com Thu Apr 19 14:01:44 2012 From: odhiambo at gmail.com (Odhiambo Washington) Date: Thu, 19 Apr 2012 14:01:44 +0300 Subject: [Dovecot] migrate 15K users to new domain name In-Reply-To: <20120419082245.GA4050@dibs.tanso.net> References: <20120419082245.GA4050@dibs.tanso.net> Message-ID: On Thu, Apr 19, 2012 at 11:22, Jan-Frode Myklebust wrote: > I need to migrate 15K users to a new domain name, and plan to use "dsync > mirror" in the transition phase. Could someone confirm that this should > work: > > Before giving users access to new-domain do a first sync to get all the > stale data over: > > for user in $old-domain; do > dsync mirror $user at old-domain $user at new-domain > done > > Configure sieve vacation filter to forward all messages from > $user at old-domain to $user at new-domain, and notify sender of changed > address. > > Give users access to both new-domain and old-domain, and do a final > sync. > > for user in $old-domain; do > dsync mirror $user at old-domain $user at new-domain > dsync mirror $user at old-domain $user at new-domain # twice in > case the first was slow > drop all messages for $user at old-domain > Leave notice message for $user at old-domain saying he should > use new-domain > done > > Wait a few weeks/months, and then drop all users at old-domain. > > > Does this look sensible? > > > -jf > What do you mean by a "new domain" in this context? Is the server changing? Is the storage changing? In my thinking, a domain change is as simple as using a rewrite rule in your MTA. I am sure I didn't understand your requirement and as such my response is equally vague. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I can't hear you -- I'm using the scrambler. Please consider the environment before printing this email. -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 652 bytes Desc: not available URL: From robert at schetterer.org Thu Apr 19 15:09:10 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 14:09:10 +0200 Subject: [Dovecot] doveadm delete folders ? Message-ID: <4F900066.9030808@schetterer.org> Hi , is there a feature like doveadm expunge savedbefore ... for imap folder delete ? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From luc at bigdata.nb.ca Thu Apr 19 15:14:16 2012 From: luc at bigdata.nb.ca (Luc Belliveau) Date: Thu, 19 Apr 2012 09:14:16 -0300 Subject: [Dovecot] old messages being delivered to blackberry In-Reply-To: <5CDED2FA-3654-4378-A2D3-8A0BFAF7B5CC@iki.fi> References: <4F843969.6040806@bigdata.nb.ca> <5CDED2FA-3654-4378-A2D3-8A0BFAF7B5CC@iki.fi> Message-ID: <4F900198.1080703@bigdata.nb.ca> I've switched to NFS instead of GlusterFS, and the problems have disappeared. It's too bad... GlusterFS had some nice features ... On 04/10/2012 12:11 PM, Timo Sirainen wrote: > On 10.4.2012, at 16.45, Luc Belliveau wrote: > >> Several users are reporting old messages (sometime years old) are being delivered as new on their blackberry devices... I've confirmed that this happens to all blackberry users... I believe the problem is related to this error: >> >> Maildir: Expunged message reappeared, giving a new UID > This means: > > 1. readdir() returns maildir file X > 2. Later readdir() doesn't return file X, so Dovecot thinks the mail is expunged and also tells the client the mail is expunged > 3. Later readdir() once again returns file X, so Dovecot can't really do anything except treat it as a new mail. > >> Possibly caused by GlusterFS, the indexes were still getting corrupted, even with only 1 server... so I've moved the indexes to a local directory on each server. I'm no longer getting index related errors... but the expunged error is still occuring... and the error seems to happen a few moments before the BB gets a bunch of messages... so it's very likely related. >> >> I would like to continue using GlusterFS if possible; I haven't found any indication that it is outright unsupported, as long as I'm using Director... > If glusterfs can't do readdir() reliably, there's not a whole lot you can do.. Except maybe switch to another mailbox format. But if you're getting index corruption also with 1 Dovecot server, it doesn't look very promising. > From gedalya at gedalya.net Thu Apr 19 16:19:54 2012 From: gedalya at gedalya.net (Gedalya) Date: Thu, 19 Apr 2012 15:19:54 +0200 Subject: [Dovecot] doveadm delete folders ? In-Reply-To: <4F900066.9030808@schetterer.org> References: <4F900066.9030808@schetterer.org> Message-ID: <4F9010FA.8000704@gedalya.net> On 4/19/2012 2:09 PM, Robert Schetterer wrote: > Hi , > is there a feature like > doveadm expunge savedbefore ... > for imap folder delete ? I was wondering the same thing - when a user moves a whole folder into Trash, doveadm expunge mailbox "Trash.*" savedbefore would eventually remove all the messages, but the folder itself remains a subfolder of Trash. It would make sense to be able to remove subfolders which have been there for more than x days and are now empty. From janfrode at tanso.net Thu Apr 19 16:28:03 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 19 Apr 2012 15:28:03 +0200 Subject: [Dovecot] migrate 15K users to new domain name In-Reply-To: References: <20120419082245.GA4050@dibs.tanso.net> Message-ID: <20120419132803.GA9396@dibs.tanso.net> On Thu, Apr 19, 2012 at 02:01:44PM +0300, Odhiambo Washington wrote: > > What do you mean by a "new domain" in this context? The user's email addresses are changing from username at old.domain to username at new-domain. > Is the server changing? No. > Is the storage changing? The user's home directory is based on the user's email address, so this is changing. > In my thinking, a domain change is as simple as using a rewrite rule in > your MTA. Also the user's login-names needs to change from old to new domain, and all their data needs to move from old to new domain. -jf From cor at xs4all.nl Thu Apr 19 16:37:25 2012 From: cor at xs4all.nl (Cor Bosman) Date: Thu, 19 Apr 2012 15:37:25 +0200 Subject: [Dovecot] multiple Return-Path headers added using lmtp director Message-ID: <72F28E8A-EE19-48B9-AD36-0147DACD73B2@xs4all.nl> Emails arrive with 2 Return-Paths when using lmtp director. Is this something configurable in the director or is this a bug? Return-Path: Delivered-To: Received: from lmtpdirector1.xs4all.net ([194.109.26.176]) by lmtp2.xs4all.net (Dovecot) with LMTP id gOe2E3YSkE/aEAAAYH/+MA for ; Thu, 19 Apr 2012 15:26:14 +0200 Return-Path: Received: from mxdrop-builder.xs4all.nl ([194.109.23.200]) by lmtpdirector1.xs4all.net (Dovecot) with LMTP id CnOvKTkzjU9sJQAAbddgHA ; Thu, 19 Apr 2012 15:26:14 +0200 Regards From robert at schetterer.org Thu Apr 19 16:57:21 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 15:57:21 +0200 Subject: [Dovecot] doveadm delete folders ? In-Reply-To: <4F9010FA.8000704@gedalya.net> References: <4F900066.9030808@schetterer.org> <4F9010FA.8000704@gedalya.net> Message-ID: <4F9019C1.4070106@schetterer.org> Am 19.04.2012 15:19, schrieb Gedalya: > On 4/19/2012 2:09 PM, Robert Schetterer wrote: >> Hi , >> is there a feature like >> doveadm expunge savedbefore ... >> for imap folder delete ? > I was wondering the same thing - when a user moves a whole folder into > Trash, doveadm expunge mailbox "Trash.*" savedbefore would eventually > remove all the messages, but the folder itself remains a subfolder of > Trash. It would make sense to be able to remove subfolders which have > been there for more than x days and are now empty. > jep i.e for all empty and not longer needed archive date folders -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From odhiambo at gmail.com Thu Apr 19 17:03:01 2012 From: odhiambo at gmail.com (Odhiambo Washington) Date: Thu, 19 Apr 2012 17:03:01 +0300 Subject: [Dovecot] migrate 15K users to new domain name In-Reply-To: <20120419132803.GA9396@dibs.tanso.net> References: <20120419082245.GA4050@dibs.tanso.net> <20120419132803.GA9396@dibs.tanso.net> Message-ID: On Thu, Apr 19, 2012 at 16:28, Jan-Frode Myklebust wrote: > On Thu, Apr 19, 2012 at 02:01:44PM +0300, Odhiambo Washington wrote: > > > > What do you mean by a "new domain" in this context? > > The user's email addresses are changing from username at old.domain to > username at new-domain. > That can be handled by address rewrites within the MTA. > > > Is the server changing? > > No. > > > Is the storage changing? > > The user's home directory is based on the user's email address, so this is > changing. > In my setup, I have virtual users. So the home directory is in the /var/spool/virtual/$domain/$user/mdbox How is yours setup? If the domain name changed, from domain1 to domain2, I believe it would be easy to change as follows: cd /var/spool/virtual/ mv $domain1 $domain2 Well, it the size of $domain1 is in TBs, I'd probably do it another way as doing mv is a bit dangerous with very large datasets: cd /var/spool/virtual mkdir domain2 cd domain1 for d in `ls -1`; do mv $d domain2/$d; done [ or something closer] > > In my thinking, a domain change is as simple as using a rewrite rule in > > your MTA. > > Also the user's login-names needs to change from old to new domain, and > all their data needs to move from old to new domain. > > And the login names are stored in a flatfile or db?? Either way, you can do a rename. Dump the database and just use vi to rename old-domain to new-domain, then drop the db and import the dump. mysqldump dbname dbname.sql vi dbname.sql :g/old-domain/s//new-domain/g mysqladmin drop dbname mysqladmin create dbname mysql dbname < dbname.sql You can also edit the flatfile to s/old-domain/new-domain/g cat flatfile | xargs sed -i.BAK 's/old-domain/new-domain/g' Maybe I still don't understand you:-) -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I can't hear you -- I'm using the scrambler. Please consider the environment before printing this email. -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 652 bytes Desc: not available URL: From janfrode at tanso.net Thu Apr 19 17:19:30 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 19 Apr 2012 16:19:30 +0200 Subject: [Dovecot] migrate 15K users to new domain name In-Reply-To: References: <20120419082245.GA4050@dibs.tanso.net> <20120419132803.GA9396@dibs.tanso.net> Message-ID: <20120419141930.GB10069@dibs.tanso.net> On Thu, Apr 19, 2012 at 05:03:01PM +0300, Odhiambo Washington wrote: > > > > In my setup, I have virtual users. So the home directory is in the > /var/spool/virtual/$domain/$user/mdbox > > How is yours setup? mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln > If the domain name changed, from domain1 to domain2, I > believe it would be easy to change as follows: > > cd /var/spool/virtual/ > mv $domain1 $domain2 If I could figure out what the %256LRHu hash is, mv would probably be a very good solution.. > > And the login names are stored in a flatfile or db?? LDAP > Either way, you can do a rename. No, we need to keep the old username/password working, so that all users will get notified of the changed -- even if they take off on a 6 month vacation the day before the change. > > Maybe I still don't understand you:-) You seem to be understanding perfectly well. I've been looking myself blind on dsync mirror, when a simple "mv" probably will work just as well :-) -jf From robert at schetterer.org Thu Apr 19 17:45:34 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 16:45:34 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works In-Reply-To: <4F8EFD77.90103@schetterer.org> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> Message-ID: <4F90250E.3000200@schetterer.org> Am 18.04.2012 19:44, schrieb Robert Schetterer: > Am 18.04.2012 19:01, schrieb Timo Sirainen: >> On 17.4.2012, at 17.40, Robert Schetterer wrote: >> >>> doveadm acl debug >>> >>> shows on both versions >>> >>> Fatal: ACL shared dict iteration failed >> >> Doesn't it show any other error message before this? Either to terminal or to error log? What's your doveconf -n? > > > Hi Timo, > i didnt found more usefull errors before > all are like Debug: acl: No lookup right to mailbox: > but that istn true, i checked it in the filesystem > and tried new acls etc > ( and they working under 2.0.20 ) > > to me it looks like some acl check/list problem > perhaps depend on stuff like in i.e > http://hg.dovecot.org/dovecot-2.1/rev/65a75939ac2c > > as i said the old 2.0.20 works still fine > on the other server > the only change i did was with virtual and namespace inbox > to get special_use working > >> > this is from the dove version > > # 2.1.4 (85ad4baedd43): /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-37-server x86_64 Ubuntu 10.04.4 LTS > auth_cache_negative_ttl = 0 > auth_cache_size = 50 M > auth_debug = yes > auth_debug_passwords = yes > auth_master_user_separator = * > auth_mechanisms = plain login > auth_socket_path = /var/run/dovecot/auth-userdb > auth_verbose = yes > auth_verbose_passwords = plain > auth_worker_max_count = 300 > dict { > acl = mysql:/etc/dovecot/dovecot-dict-acl-sql.conf.ext > expire = mysql:/etc/dovecot/dovecot-dict-expire-sql.conf.ext > quotadict = mysql:/etc/dovecot/dovecot-dict-quota-sql.conf.ext > } > disable_plaintext_auth = no > first_valid_gid = 1001 > first_valid_uid = 1001 > hostname = mail01.example.com > last_valid_gid = 1001 > last_valid_uid = 1001 > listen = * > login_greeting = imap, pop ready > mail_access_groups = vmail > mail_debug = yes > mail_fsync = always > mail_gid = 1001 > mail_location = maildir:~/ > mail_nfs_index = yes > mail_nfs_storage = yes > mail_plugins = " stats zlib expire virtual fts fts_squat mail_log > autocreate notify acl listescape" > mail_privileged_group = vmail > mail_uid = 1001 > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date ihave imapflags notify > mmap_disable = yes > namespace { > list = yes > location = maildir:/usr/local/virtual/%%d/%%u:INDEX=~/shared/%%d/%%u > prefix = shared/%%u/ > separator = / > subscriptions = no > type = shared > } > namespace inbox { > hidden = no > list = yes > location = > mailbox Drafts { > auto = subscribe > special_use = \Drafts > } > mailbox Junk { > auto = subscribe > special_use = \Junk > } > mailbox Sent { > auto = subscribe > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > auto = subscribe > special_use = \Trash > } > mailbox name { > special_use = \Drafts \Junk \Sent \Trash > } > prefix = > separator = / > subscriptions = yes > } > namespace real { > hidden = yes > list = no > location = > prefix = RealMails/ > separator = / > } > namespace virtual { > hidden = yes > list = no > location = virtual:/etc/dovecot/virtual2:LAYOUT=maildir++:INDEX=~/virtual > prefix = virtual/ > separator = / > } > passdb { > args = /etc/dovecot/dovecot-sql-master.conf.ext > driver = sql > master = yes > pass = yes > } > passdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > plugin { > acl = vfile:/etc/dovecot/global-acls:cache_secs=300 > acl_shared_dict = proxy::acl > autocreate = Trash > autocreate2 = Junk > autocreate3 = Sent > autocreate4 = Drafts > autocreate5 = Templates > autocreate6 = Hostmaster-Backup > autocreate7 = archiv-backup-mailspooler > autosubscribe = Trash > autosubscribe2 = Junk > autosubscribe3 = Sent > autosubscribe4 = Drafts > autosubscribe5 = Templates > expire = Trash > expire2 = Trash/* > expire3 = Junk > expire4 = Junk/* > expire5 = Hostmaster-Backup > expire6 = Hostmaster-Backup/* > expire7 = archiv-backup-mailspooler/* > expire_dict = proxy::expire > fts = squat > home = /usr/local/virtual/%d/%u > mail_log_events = delete undelete expunge copy mailbox_delete > mailbox_rename > mail_log_fields = uid box msgid size > mail_log_group_events = yes > quota = dict:::proxy::quotadict > quota_warning = storage=95%% quota-warning 95 %u > quota_warning2 = storage=80%% quota-warning 80 %u > sieve = /usr/local/virtual/%d/%u/dovecot.sieve > sieve_before = /etc/dovecot/sieve/global.sieve > sieve_dir = /usr/local/virtual/%d/%u/sieve > sieve_execute_bin_dir = /usr/lib/dovecot/sieve-execute > sieve_execute_socket_dir = sieve-execute > sieve_extensions = +notify +imapflags > sieve_filter_bin_dir = /usr/lib/dovecot/sieve-filter > sieve_filter_socket_dir = sieve-filter > sieve_global_dir = /etc/dovecot/sieve/ > sieve_global_extensions = +vnd.dovecot.execute +vnd.dovecot.pipe > +vnd.dovecot.filter > sieve_global_path = /etc/dovecot/sieve/global.sieve > sieve_pipe_bin_dir = /usr/lib/dovecot/sieve-pipe > sieve_pipe_socket_dir = sieve-pipe > sieve_plugins = sieve_extprograms > stats_refresh = 30 secs > stats_track_cmds = yes > zlib_save = gz > zlib_save_level = 6 > } > postmaster_address = hostmaster at mail01.example.com > protocols = imap pop3 lmtp sieve > sendmail_path = /usr/lib/sendmail > service anvil { > client_limit = 1000 > } > service auth-worker { > user = root > } > service auth { > client_limit = 1000 > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0666 > user = postfix > } > unix_listener auth-userdb { > group = vmail > mode = 0600 > user = vmail > } > } > service dict { > extra_groups = vmail > unix_listener dict { > group = vmail > mode = 0660 > } > } > service imap-login { > inet_listener imap { > port = 143 > } > inet_listener imaps { > port = 993 > ssl = yes > } > process_min_avail = 8 > service_count = 1 > } > service imap-postlogin { > executable = script-login /usr/local/bin/postlogin_imap.sh > } > service imap { > executable = imap imap-postlogin > process_limit = 1024 > vsz_limit = 256 M > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0660 > user = postfix > } > vsz_limit = 256 M > } > service managesieve-login { > inet_listener { > address = 127.0.0.1 212.52.224.210 212.52.224.205 > port = 2000 > } > vsz_limit = 256 M > } > service pop3-login { > inet_listener pop3 { > port = 110 > } > inet_listener pop3s { > port = 995 > ssl = yes > } > } > service pop3-postlogin { > executable = script-login /usr/local/bin/postlogin_pop3.sh > } > service pop3 { > executable = pop3 pop3-postlogin > process_limit = 1024 > } > service quota-warning { > executable = script /usr/local/bin/quota-warning.sh > unix_listener quota-warning { > mode = 0666 > user = vmail > } > user = vmail > } > service stats { > fifo_listener stats-mail { > mode = 0600 > user = vmail > } > } > ssl_ca = ssl_cert = ssl_key = userdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > userdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > verbose_proctitle = yes > protocol lda { > mail_plugins = quota sieve virtual acl mail_log notify expire listescape > } > protocol imap { > imap_client_workarounds = delay-newmail > imap_id_log = * > imap_id_send = * > imap_idle_notify_interval = 2 mins > imap_logout_format = bytes=%i/%o > imap_max_line_length = 128 k > mail_max_userip_connections = 100 > mail_plugins = imap_acl acl fts fts_squat mail_log autocreate notify > expire virtual quota imap_quota zlib imap_zlib listescape stats imap_stats > } > protocol lmtp { > mail_plugins = quota sieve virtual acl fts fts_squat mail_log notify > expire listescape > } > protocol sieve { > mail_max_userip_connections = 100 > } > protocol pop3 { > mail_max_userip_connections = 100 > mail_plugins = expire virtual quota mail_log fts fts_squat notify > listescape > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > pop3_enable_last = yes > pop3_fast_size_lookups = yes > pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s > pop3_no_flag_updates = no > pop3_reuse_xuidl = no > pop3_uidl_format = %g > } > > > Hi Timo, i did some more tests in shared namespace prefix shared/%%u/ is not working shared/%%n/ is working i cant use that cause of mutiple domains naming convention ,users are user at domain.de etc -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Thu Apr 19 17:59:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 17:59:39 +0300 Subject: [Dovecot] migrate 15K users to new domain name In-Reply-To: <20120419141930.GB10069@dibs.tanso.net> References: <20120419082245.GA4050@dibs.tanso.net> <20120419132803.GA9396@dibs.tanso.net> <20120419141930.GB10069@dibs.tanso.net> Message-ID: On 19.4.2012, at 17.19, Jan-Frode Myklebust wrote: > mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln > > If I could figure out what the %256LRHu hash is, mv would probably be a > very good solution.. With v2.1.4 you could do something like: doveadm -c dummy.conf user -m user at domain where dummy.conf contains the minimum configuration needed: mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln ssl = no It still does a userdb lookup though, so either the user at domain needs to exist in userdb or you need to create a separate dummy Dovecot installation using userdb static that the dummy.conf uses. From tss at iki.fi Thu Apr 19 18:16:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 18:16:37 +0300 Subject: [Dovecot] doveadm delete folders ? In-Reply-To: <4F900066.9030808@schetterer.org> References: <4F900066.9030808@schetterer.org> Message-ID: <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> On 19.4.2012, at 15.09, Robert Schetterer wrote: > Hi , > is there a feature like > doveadm expunge savedbefore ... > for imap folder delete ? Here: http://hg.dovecot.org/dovecot-2.1/rev/2d8bafd11569 http://hg.dovecot.org/dovecot-2.1/rev/782570f644f7 From tss at iki.fi Thu Apr 19 18:23:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 18:23:22 +0300 Subject: [Dovecot] multiple Return-Path headers added using lmtp director In-Reply-To: <72F28E8A-EE19-48B9-AD36-0147DACD73B2@xs4all.nl> References: <72F28E8A-EE19-48B9-AD36-0147DACD73B2@xs4all.nl> Message-ID: On 19.4.2012, at 16.37, Cor Bosman wrote: > Emails arrive with 2 Return-Paths when using lmtp director. Is this something configurable in the director or is this a bug? Fixed: http://hg.dovecot.org/dovecot-2.1/rev/85b3975c9008 From zhbmaillistonly at gmail.com Thu Apr 19 18:30:17 2012 From: zhbmaillistonly at gmail.com (Zhang Huangbin) Date: Thu, 19 Apr 2012 23:30:17 +0800 Subject: [Dovecot] Custom LDAP filter for doveadm-mailbox? Message-ID: <1E42E3B7EAAF4EC4A4D8D70AC3AFB045@gmail.com> Dear Timo, With Dovecot-2.0.16, 'doveadm mailbox list -A' will perform LDAP search with filter "(objectClass=posixAccount)", and get value of attribute 'uid' as mail user. Is it possible to custom the ldap filter and returned attribute name? For example: - LDAP filter: (&(objectClass=mailUser)(accountStatus=active)) - Returned attribute name: mail Did i miss something? Thanks. ---- Zhang Huangbin iRedMail: Open Source Mail Server Solution for Red Hat Enterprise Linux, CentOS, Scientific Linux, Debian, Ubuntu, Mint, Gentoo, openSUSE, FreeBSD: http://www.iredmail.org/ From tss at iki.fi Thu Apr 19 18:41:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 18:41:28 +0300 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works In-Reply-To: <4F90250E.3000200@schetterer.org> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> <4F90250E.3000200@schetterer.org> Message-ID: <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> On 19.4.2012, at 17.45, Robert Schetterer wrote: > Hi Timo, > i did some more tests > in shared namespace > prefix > > shared/%%u/ is not working > > > shared/%%n/ is working > > i cant use that cause of mutiple domains > naming convention ,users are user at domain.de etc I can't really think of why that would make a difference. I did a few tests and couldn't reproduce the problem. Try if changing the location in the shared namespace makes a difference: namespace { list = yes location = maildir:%%h:INDEX=~/shared/%%d/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } BTW. You could remove autocreate plugin by moving the rest of the autocreates to mailbox { auto=create } settings. From tss at iki.fi Thu Apr 19 18:44:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 18:44:07 +0300 Subject: [Dovecot] alias_for In-Reply-To: References: Message-ID: <49931079-28AE-4A18-AE32-1F38735D7255@iki.fi> On 19.4.2012, at 2.11, Joseph Tam wrote: > I noticed the addition of a new namespace configuration "alias_for". > What pracical difference is adding this over and above making namespace > aliases hidden and disabling list, as in older versions of dovecot > configurations? It's used by doveadm and dsync to skip duplicate mailboxes. I've also been thinking that it could be made automatic by comparing which namespaces have identical "location" settings. From tss at iki.fi Thu Apr 19 18:44:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 18:44:56 +0300 Subject: [Dovecot] Custom LDAP filter for doveadm-mailbox? In-Reply-To: <1E42E3B7EAAF4EC4A4D8D70AC3AFB045@gmail.com> References: <1E42E3B7EAAF4EC4A4D8D70AC3AFB045@gmail.com> Message-ID: <41EDF5DE-49DF-4900-86CA-52F8A6B05717@iki.fi> On 19.4.2012, at 18.30, Zhang Huangbin wrote: > With Dovecot-2.0.16, 'doveadm mailbox list -A' will perform LDAP search with filter "(objectClass=posixAccount)", and get value of attribute 'uid' as mail user. Is it possible to custom the ldap filter and returned attribute name? For example: > > - LDAP filter: (&(objectClass=mailUser)(accountStatus=active)) > - Returned attribute name: mail > > Did i miss something? See iterate_* settings in example dovecot-ldap.conf.ext. From news1204.10.weezy at spamgourmet.com Wed Apr 18 20:09:58 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Wed, 18 Apr 2012 19:09:58 +0200 Subject: [Dovecot] namespace hierarchy In-Reply-To: References: <201204010928.41017.anyaddress@gmx.net> Message-ID: Am 04.04.2012 03:47, schrieb Timo Sirainen: > On 1.4.2012, at 10.28, Tom Fernandes wrote: > >> In the new server I would like to have 3 namespaces (private, public, shared) >> and use "/" as separator. Is it possible to keep "INBOX." for private, so that >> the users mailboxes appear below the Inbox, but have public and shared on the >> same hierarchy-level like the Inbox? >> > > If you have any subscriptions=no namespaces, you need a parent namespace that has subscriptions=yes. You could have e.g.: > > namespace { > prefix = INBOX/ > list = no > inbox = yes > } > namespace { > prefix = > list = no > hidden = yes > alias_for = INBOX/ > } > > So now the prefix="" is used for the subscriptions but nothing else really. I tried this, but sadly with dovecot V2.1.1 it is not working. Steps to reproduce: + create empty mail directory + create one folder (Allgemeines/Tabel) in public namespace with doveadm + start mail-client (in my case thunderbird) + try to subscribe to the new public folder Error message in log: Apr 18 18:19:34 elablnmail02 dovecot: imap(ahelmcke): Warning: Subscriptions file /home/mail/user/ahelmcke/subscriptions: Removing invalid entry: Allgemeines/Tabel doveconf -n: # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) ext4 auth_master_user_separator = * first_valid_uid = 400 mail_gid = vmail mail_location = maildir:/home/mail/user/%u mail_uid = vmail mbox_write_locks = fcntl namespace Allgemeines { location = maildir:/home/mail/Allgemeines:INDEX=/home/mail/user/%u/SEEN.Allgemeines prefix = Allgemeines/ separator = / subscriptions = no type = public } namespace dummy { alias_for = INBOX/ hidden = yes list = no location = prefix = separator = / type = private } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile sieve = /var/lib/dovecot/sieve/%u.sieve sieve_global_dir = /var/lib/dovecot/sieve/global/ sieve_global_path = /var/lib/dovecot/sieve/default.sieve } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } service imap-login { inet_listener imap { address = * } inet_listener imaps { address = * } } ssl = required ssl_cert = Hello, after finally having managed to setup up dovecot with virtual users and public folders I tried to get it connected to postfix. The problem is, that I can't get postix to deliver mail via dovecot-lda. After some testing it seems to come down to dovecot-lda behaving different when called as root then when called as user vmail. Basic setting: - dovecot 2.1.1 - all mailboxes owned by user vmail (using mail_uid and mail_gid) Test 1: as user root: /usr/libexec/dovecot/dovecot-lda -d ahelmcke < some_mail working as expected; some_mail gets stored in users INBOX Test 2: same command executed as user vmail. dovecot-lda terminates immediately. syslog-output with mail_debug = yes: Test 1: Apr 19 16:38:04 elablnmail02 dovecot: lda: Debug: auth input: ahelmcke Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: Effective uid=494, gid=491, home=/home/mail/user/ahelmcke Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=/home/mail/user/ahelmcke, alt= Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke/public, index=, control=, inbox=, alt= Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace Allgemeines: type=public, prefix=Allgemeines/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no location=maildir:/home/mail/Allgemeines:INDEX=/home/mail/user/ahelmcke/SEEN.Allgemeines Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/Allgemeines, index=/home/mail/user/ahelmcke/SEEN.Allgemeines, control=, inbox=, alt= Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: none: root=, index=, control=, inbox=, alt= Apr 19 16:51:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Destination address: ahelmcke at elablnmail02.ela-bln.local (source: user at hostname) Apr 19 16:51:37 elablnmail02 dovecot: lda(ahelmcke): msgid=unspecified: saved mail to INBOX Test 2: Apr 19 16:37:37 elablnmail02 dovecot: lda: Debug: auth input: ahelmcke Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Effective uid=494, gid=491, home=/home/mail/user/ahelmcke Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=/home/mail/user/ahelmcke, alt= Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke/public, index=, control=, inbox=, alt= Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, subscriptions=yes location= Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mdbox: access(/home/mail/user/ahelmcke/mdbox, rwx): failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mdbox: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: sdbox: access(/home/mail/user/ahelmcke/sdbox, rwx): failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: sdbox: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir: access(/home/mail/user/ahelmcke/Maildir, rwx): failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has .imap/: stat(/home/mail/user/ahelmcke/mail/.imap) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has inbox: stat(/home/mail/user/ahelmcke/mail/inbox) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has mbox: stat(/home/mail/user/ahelmcke/mail/mbox) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has .imap/: stat(/home/mail/user/ahelmcke/Mail/.imap) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has inbox: stat(/home/mail/user/ahelmcke/Mail/inbox) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has mbox: stat(/home/mail/user/ahelmcke/Mail/mbox) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: sdbox: access(/home/mail/user/ahelmcke/sdbox, rwx): failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: sdbox: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Error: user ahelmcke: Initialization failed: Namespace '': Mail storage autodetection failed with home=/home/mail/user/ahelmcke Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Fatal: Invalid user settings. Refer to server log for more information. doveconf -n # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) auth_master_user_separator = * auth_verbose = yes first_valid_uid = 400 mail_debug = yes mail_gid = vmail mail_home = /home/mail/user/%u mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl namespace Allgemeines { location = maildir:/home/mail/Allgemeines:INDEX=/home/mail/user/%u/SEEN.Allgemeines prefix = Allgemeines/ separator = / subscriptions = no type = public } namespace dummy { alias_for = INBOX/ hidden = yes list = no location = maildir:/home/mail/user/%u/public/ prefix = separator = / type = private } namespace inbox { inbox = yes location = maildir:/home/mail/user/%u prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile sieve = /home/mail/user/%u/sieve.active sieve_dir = /home/mail/user/%u/sieve.scripts sieve_global_dir = /var/lib/dovecot/sieve/global/ sieve_global_path = /var/lib/dovecot/sieve/default.sieve } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = * } inet_listener imaps { address = * } } service managesieve-login { inet_listener sieve { address = * port = 4190 } } ssl = required ssl_cert = References: <1E42E3B7EAAF4EC4A4D8D70AC3AFB045@gmail.com> <41EDF5DE-49DF-4900-86CA-52F8A6B05717@iki.fi> Message-ID: <16B8C19B02364F77908C1521F76E4AA8@gmail.com> On Thursday, April 19, 2012 at 11:44 PM, Timo Sirainen wrote: > See iterate_* settings in example dovecot-ldap.conf.ext. It works as expected. Thanks. :) ---- Zhang Huangbin iRedMail: Open Source Mail Server Solution for Red Hat Enterprise Linux, CentOS, Scientific Linux, Debian, Ubuntu, Mint, Gentoo, openSUSE, FreeBSD: http://www.iredmail.org/ From tss at iki.fi Thu Apr 19 18:53:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 18:53:32 +0300 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: References: Message-ID: On 19.4.2012, at 17.56, Andreas Helmcke wrote: > namespace inbox { > inbox = yes > location = maildir:/home/mail/user/%u > prefix = INBOX/ > separator = / > type = private > } Move the location from the above namespace to global: mail_location = maildir:/home/mail/user/%u Although it probably still shouldn't have failed.. From news1204.10.weezy at spamgourmet.com Thu Apr 19 19:09:52 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Thu, 19 Apr 2012 18:09:52 +0200 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: References: Message-ID: <2vp569xe2o.ln2@mail.ela-soft.com> Am 19.04.2012 17:53, schrieb Timo Sirainen: > On 19.4.2012, at 17.56, Andreas Helmcke wrote: > >> namespace inbox { >> location = maildir:/home/mail/user/%u >> .... > > Move the location from the above namespace to global: > > mail_location = maildir:/home/mail/user/%u Actually it /had/ been there and I moved it to inbox while trying to find the problem. So, change reverted (excerpt from doveconf -n): mail_location = maildir:/home/mail/user/%u namespace inbox { inbox = yes location = prefix = INBOX/ separator = / type = private } Now dovecot-lda is complaining about duplicate namespaces (still only when run as vmail; run it as root and it is working): Apr 19 17:58:07 elablnmail02 dovecot: lda: Debug: auth input: ahelmcke Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Effective uid=494, gid=491, home=/home/mail/user/ahelmcke Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=/home/mail/user/ahelmcke, alt= Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke/public, index=, control=, inbox=, alt= Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=, alt= Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Error: user ahelmcke: Initialization failed: namespace configuration error: Duplicate namespace prefix: "" Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Fatal: Invalid user settings. Refer to server log for more information. Greetings Andreas From robert at schetterer.org Thu Apr 19 19:30:57 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 18:30:57 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works In-Reply-To: <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> <4F90250E.3000200@schetterer.org> <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> Message-ID: <4F903DC1.5070501@schetterer.org> Am 19.04.2012 17:41, schrieb Timo Sirainen: > On 19.4.2012, at 17.45, Robert Schetterer wrote: > >> Hi Timo, >> i did some more tests >> in shared namespace >> prefix >> >> shared/%%u/ is not working >> >> >> shared/%%n/ is working >> >> i cant use that cause of mutiple domains >> naming convention ,users are user at domain.de etc > > I can't really think of why that would make a difference. I did a few tests and couldn't reproduce the problem. Try if changing the location in the shared namespace makes a difference: > > namespace { > list = yes > location = maildir:%%h:INDEX=~/shared/%%d/%%u hi Timo, > prefix = shared/%%u/ is not working !!! under 2.1 / it works under 2.0.20 prefix = shared/%%n/ is working but then i see only users of my domain acl shared with their user part of i.e user at domain.de --shared | user ( of my own domain ) i cant use this because i have to share cross domain and users may have same names in different domains what i need is --shared | user at domainwhatever.de > separator = / > subscriptions = no > type = shared > } > > BTW. You could remove autocreate plugin by moving the rest of the autocreates to mailbox { auto=create } settings. i know this nevertheless i try location change -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From almarzuki2001 at hotmail.com Thu Apr 19 21:10:33 2012 From: almarzuki2001 at hotmail.com (Hadi Salem) Date: Thu, 19 Apr 2012 21:10:33 +0300 Subject: [Dovecot] dovecot LDA with sendmail Message-ID: Hi, Im configuring sendmail with dovecot for virtual users using password file and file for user name. on centos. dovecot-1.0.7-7.el5 sendmail-8.13.8-2.el5 dovecot ?n # 1.0.7: /etc/dovecot.conf base_dir: /var/run/dovecot/ log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot.log ssl_disable: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login default_mail_env: maildir:/home/vmail/%d/%n mail_location: maildir:/home/vmail/%d/%n mail_debug: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib64/dovecot/imap mail_plugin_dir(imap): /usr/lib64/dovecot/imap mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3 auth default: verbose: yes debug: yes debug_passwords: yes passdb: driver: passwd-file args: /etc/dovecot/passwd userdb: driver: passwd-file args: /etc/dovecot/users socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: root group: root I have also added dovecot LDA for sendmail /usr/share/sendmail-cf/mailer/dovecot.m4 ######################*****############## ### DOVECOT Mailer specification ### ##################*****################## Mdovecot, P=/usr/libexec/dovecot/deliver, F=DFMPhnu9,S=EnvFromSMTP/HdrFromSMTP, R=EnvToSMTP/HdrFromSMTP,T=DNS/RFC822/X-Unix,A=/usr/libexec/dovecot/deliver -d $u And also [root at host1 mail]# cat /etc/mail/mailertable example.com dovecot:dovecot however when im sending email to the virtual user im getting this error, Apr 19 20:52:13 host1 sendmail[22311]: q3JHqDAR022309: to=, ctladdr= (0/0), delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120292, relay=dovecot, dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/libexec/dovecot/deliver) exited with EX_TEMPFAIL Dovecot LDA can?t deliver the mail to the user!!! From robert at schetterer.org Thu Apr 19 21:36:28 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 20:36:28 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works In-Reply-To: <4F903DC1.5070501@schetterer.org> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> <4F90250E.3000200@schetterer.org> <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> <4F903DC1.5070501@schetterer.org> Message-ID: <4F905B2C.8040204@schetterer.org> Am 19.04.2012 18:30, schrieb Robert Schetterer: > Am 19.04.2012 17:41, schrieb Timo Sirainen: >> On 19.4.2012, at 17.45, Robert Schetterer wrote: >> >>> Hi Timo, >>> i did some more tests >>> in shared namespace >>> prefix >>> >>> shared/%%u/ is not working >>> >>> >>> shared/%%n/ is working >>> >>> i cant use that cause of mutiple domains >>> naming convention ,users are user at domain.de etc >> >> I can't really think of why that would make a difference. I did a few tests and couldn't reproduce the problem. Try if changing the location in the shared namespace makes a difference: >> >> namespace { >> list = yes >> location = maildir:%%h:INDEX=~/shared/%%d/%%u > > hi Timo, > >> prefix = shared/%%u/ > is not working !!! under 2.1 / it works under 2.0.20 > > prefix = shared/%%n/ > is working > > > > but then i see only users of my domain acl shared > with their user part of i.e user at domain.de > > --shared > | > user ( of my own domain ) > > i cant use this because i have to share cross domain > and users may have same names in different domains > > > what i need is > > --shared > | > user at domainwhatever.de > >> separator = / >> subscriptions = no >> type = shared >> } >> > >> BTW. You could remove autocreate plugin by moving the rest of the autocreates to mailbox { auto=create } settings. > i know this > > nevertheless i try location change > just for info location change to location = maildir:%%h:INDEX=~/shared/%%d/%%u does not lead to get it work prefix = shared/%%u/ simply dont works i have reverted back to exact namespace config of the 2.0.20 server, same result, so something broken in 2.1 -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Thu Apr 19 23:00:40 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 22:00:40 +0200 Subject: [Dovecot] doveadm delete folders ? In-Reply-To: <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> References: <4F900066.9030808@schetterer.org> <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> Message-ID: <4F906EE8.1030900@schetterer.org> Am 19.04.2012 17:16, schrieb Timo Sirainen: > On 19.4.2012, at 15.09, Robert Schetterer wrote: > >> Hi , >> is there a feature like >> doveadm expunge savedbefore ... >> for imap folder delete ? > > Here: > > http://hg.dovecot.org/dovecot-2.1/rev/2d8bafd11569 > http://hg.dovecot.org/dovecot-2.1/rev/782570f644f7 > thx Timo ! -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Thu Apr 19 23:27:21 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 22:27:21 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works / solved by remove listescape In-Reply-To: <4F905B2C.8040204@schetterer.org> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> <4F90250E.3000200@schetterer.org> <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> <4F903DC1.5070501@schetterer.org> <4F905B2C.8040204@schetterer.org> Message-ID: <4F907529.2020403@schetterer.org> > > prefix = shared/%%u/ simply dont works > > i have reverted back to exact namespace config > of the 2.0.20 server, same result, so something broken in 2.1 > > Hi Timo problem solved by removing listescape plugin the acl paths did / instead of dot in domainname any chance to get listescape fixed ? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From stan at hardwarefreak.com Fri Apr 20 03:31:13 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 19 Apr 2012 19:31:13 -0500 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <20120417200820.GA30743@dibs.tanso.net> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <20120417124241.GA23120@dibs.tanso.net> <4F8D6BAA.7060501@gtdinternet.com> <20120417200820.GA30743@dibs.tanso.net> Message-ID: <4F90AE51.8080407@hardwarefreak.com> On 4/17/2012 3:08 PM, Jan-Frode Myklebust wrote: > Our struggle is the number of > iops we're able to get from the backend storage (IBM DS4800), mostly > a problem when we have storms of incoming marketing messages in addition > to the pop/imap traffic. This issue has come up twice on the Postfix list in less than a month. You can fix this specific problem very easily. Only marketing servers and busy/misconfigured list servers make many parallel connections to your MX hosts. Allowing them to blast all those messages over parallel connections is what bogs down your spool storage. The fix is simple: limit all SMTP clients to a small number of parallel connections. This will slow down marketing and list server blasts without affecting normal sending MTAs. To do so, add this to /etc/postfix/main.cf: smtpd_client_connection_count_limit = 4* The default Postfix process limit is 100. The concurrent connection limit is 1/2 the process limit, so 50 parallel connections per client IP are allowed by default. If remote hosts also do connection caching, they can force feed your MTA many hundreds of messages/sec. Limiting concurrent connections will decrease their mail rate to a small fraction of what you're seeing now, reducing IOPS load on your spool storage significantly. * This is a good starting point. You may need to tweak it up a little bit. Some list servers (such as XFS) will unsub members if their multiple connections keep getting refused, so tweak this value until you find your sweet spot. -- Stan From dchenusa at yahoo.com Fri Apr 20 03:38:01 2012 From: dchenusa at yahoo.com (Dennis Chen) Date: Thu, 19 Apr 2012 17:38:01 -0700 Subject: [Dovecot] Fwd: IMAP SSL incoming test need help ! References: <1334876809.51362.YahooMailNeo@web161602.mail.bf1.yahoo.com> Message-ID: <82BE5266-7A1D-4E1F-9862-F503668D6EAF@yahoo.com> Sent from my iPhone Begin forwarded message: > From: D Chen > Date: April 19, 2012 4:06:49 PM PDT > To: lists-dovecot > Subject: Re: [Dovecot] IMAP SSL incoming test need help ! > Reply-To: D Chen > > Thanks for nice pointing out in my mail.log! > > I could not find where is the procmail log located under /var/log ! > > As procmail is "suspicious", I recalled that I selected either use procmail or not use for local delivery while I re-configure the postfix. I ran "sudo dpkg-reconfigure postfix" again, and selected NOT use "Procmail" for local delivery, restart postfix and ran the tested again, it worked now ! examined the mail.log, it said "delivered to maildir" instead of procmail. > > why works NOT using procmail ? thx. > > From: lists-dovecot > To: D Chen > Sent: Thursday, April 19, 2012 5:22 AM > Subject: Re: [Dovecot] IMAP SSL incoming test need help ! > > > > ------------ Original Message ------------ > > Date: Thursday, April 19, 2012 01:44:35 AM -0700 > > From: D Chen > > To: Dovecot Mailing List > > Subject: [Dovecot] IMAP SSL incoming test need help ! > > > > Ubuntu 11.10 server with postfix/dovecot/squirrelmail configured. > > > > Outgoing SMTP server seemed worked fine, however, the incoming > > IMAP with SSL on port 993 doesn't seem to receive any mail at all > > !? Here is the mail.log during the test session from an external > > yahoo mail (dchenusa at yahoo.com) to the internal server > > testmail.biokeyinc.com (dchen at testmail.biokeyinc.com) The log did > > NOT show any error ! The sender showed sent mail! but no mail > > reach to the destination! Any help would be appreciated. > > > > Apr 19 01:20:02 testmail postfix/smtpd[4103]: connect from > > nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:03 > > testmail postfix/smtpd[4103]: 091ED200973: > > client=nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 > > 01:20:03 testmail postfix/cleanup[4108]: 091ED200973: > > message-id=<32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com> Apr 19 > > 01:20:03 testmail postfix/qmgr[2007]: 091ED200973: > > from=, size=2840, nrcpt=1 (queue active) Apr > > 19 01:20:03 testmail postfix/smtpd[4103]: disconnect from > > nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:12 > > testmail postfix/smtpd[4113]: connect from localhost[127.0.0.1] > > Apr 19 01:20:12 testmail postfix/smtpd[4113]: 33AE9200AE9: > > client=localhost[127.0.0.1] Apr 19 01:20:12 testmail > > postfix/cleanup[4108]: 33AE9200AE9: > > message-id=<32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com> Apr 19 > > 01:20:12 testmail postfix/qmgr[2007]: 33AE9200AE9: > > from=, size=3536, nrcpt=1 (queue active) Apr > > 19 01:20:12 testmail postfix/smtpd[4113]: disconnect from > > localhost[127.0.0.1] Apr 19 01:20:12 testmail amavis[2042]: > > (02042-06) Passed CLEAN, [98.138.91.180] [98.138.87.4] > > -> , > > Message-ID: <32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com>, > > mail_id: rqyVZiQDwSsG, Hits: -0.109, size: 2837, queued_as: > > 33AE9200AE9, dkim_id=dchenusa at yahoo.com, at yahoo.com, 8647 ms Apr 19 > > 01:20:12 testmail postfix/smtp[4109]: 091ED200973: > > to=, > > relay=127.0.0.1[127.0.0.1]:10024, delay=9.5, > > delays=0.87/0.01/0/8.6, dsn=2.0.0, status=sent (250 2.0.0 from > > MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 33AE9200AE9) Apr > > 19 01:20:12 testmail postfix/qmgr[2007]: 091ED200973: removed Apr > > 19 01:20:12 testmail postfix/local[4114]: 33AE9200AE9: > > to=, relay=local, delay=0.18, > > delays=0.1/0/0/0.07, dsn=2.0.0, status=sent (delivered to command: > > procmail -a "$EXTENSION") Apr 19 01:20:12 testmail > > postfix/qmgr[2007]: 33AE9200AE9: removed Apr 19 01:21:30 testmail > > dovecot: imap-login: Login: user=, method=PLAIN, > > rip=63.195.90.22, lip=192.168.20.100, mpid=4122, TLS Apr 19 > > 01:23:13 testmail dovecot: imap(dchen): Disconnected: Disconnected > > in IDLE bytes=99/708 Apr 19 01:23:23 testmail postfix/anvil[4105]: > > statistics: max connection rate 1/60s for (smtp:98.138.91.180) at > > Apr 19 01:20:02 Apr 19 01:23:23 testmail postfix/anvil[4105]: > > statistics: max connection count 1 for (smtp:98.138.91.180) at Apr > > 19 01:20:02 Apr 19 01:23:23 testmail postfix/anvil[4105]: > > statistics: max cache size 1 at Apr 19 01:20:02 > > ------------ End Original Message ------------ > > > I'm going to bet that procmail is eating the inbound message: > > Apr 19 01:20:12 testmail postfix/local[4114]: 33AE9200AE9: > to=, relay=local, delay=0.18, > delays=0.1/0/0/0.07, dsn=2.0.0, status=sent (delivered to command: > > procmail -a "$EXTENSION") > > and it's not getting to the user's inbox. If a message doesn't get > to the user's inbox then it's not dovecot's issue that it can't be > retrieved. > > fyi -- imap/dovecot have nothing to do your inbound delivery. > > > however, the incoming IMAP with SSL on port 993 doesn't > > seem to receive any mail > > that's handled by your MTA, which in your case is postfix, with > procmail in the mix. > > I would try looking at the procmail log to see what's happening > there. > > - Richard > > > > > > > > > > > From jtam.home at gmail.com Fri Apr 20 04:45:38 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Thu, 19 Apr 2012 18:45:38 -0700 (PDT) Subject: [Dovecot] dovecot LDA with sendmail In-Reply-To: References: Message-ID: Hadi Salem writes: > Im configuring sendmail with dovecot for virtual users using > password file and file for user name. on centos. > > dovecot-1.0.7-7.el5 Maybe I can beat Charles Marcus to the punch and recommend you upgrade to the latest version, otherwise you'll run into bugs that have already been fixed. > Mdovecot, > P=/usr/libexec/dovecot/deliver, F=DFMPhnu9,S=EnvFromSMTP/HdrFromSMTP, > R=EnvToSMTP/HdrFromSMTP,T=DNS/RFC822/X-Unix,A=/usr/libexec/dovecot/deliver -d > $u > ... > Apr 19 20:52:13 host1 sendmail[22311]: q3JHqDAR022309: > to=, ctladdr= (0/0), > delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120292, relay=dovecot, > dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/libexec/dovecot/deliver) exited > with EX_TEMPFAIL Is there any logs from deliver? If not, look at this page to configure logging http://wiki.dovecot.org/LDA That's your best chance at finding out the problem. I don't have virtual users, so I can't venture to say whether this is part of the problem. I know I had to fiddle with the mailer flags (you have "DFMPhnu9") to get my setup working, but I don't remember what I did or why. Try adding the "S" flag and see if that helps. Joseph Tam From dovecot-user at spambox.dk Fri Apr 20 10:40:02 2012 From: dovecot-user at spambox.dk (Henrik Larsson) Date: Fri, 20 Apr 2012 08:40:02 +0100 Subject: [Dovecot] 2.1.3 Corrupted squat uidlist In-Reply-To: <4F8A97F8.70808@spambox.dk> References: <4F7164DC.7010706@unict.it> <4F8A97F8.70808@spambox.dk> Message-ID: <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> This problem is still showing up every day in the logs. I tried to delete the dovecot.index.search and dovecot.index.search.uids files for all users, but the errors are still there after a re-build of the search index with "/usr/local/bin/doveadm search -A TEXT xyzabczzz". If you need any further details, please let me know. Best regards Henrik Larsson On 15-04-2012 10:42, Henrik Larsson wrote: > On 27-03-2012 08:57, Luca Palazzo wrote: >> Hi Timo and All, >> after upgrading to 2.1.2 i'm getting a lot of these messages: >> Error: Corrupted squat uidlist file XXXXXX wrong indexid > After an upgrade to 2.1.3 i see this as well. Any thoughts? > > Apr 15 03:43:43 imap(xxxx): Error: Corrupted squat uidlist file > /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong > indexid > Apr 15 06:30:27 imap(xxxx): Error: Corrupted squat uidlist file > /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong > indexid > Apr 15 07:43:55 pop3(xxxx): Error: Corrupted squat uidlist file > /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong > indexid > Apr 15 09:00:01 imap(xxxx): Error: Corrupted squat uidlist file > /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong > indexid > > # doveconf -n > # 2.1.3: /usr/local/etc/dovecot/dovecot.conf > # OS: FreeBSD 8.2-STABLE amd64 > auth_mechanisms = plain login digest-md5 cram-md5 > first_valid_uid = 125 > listen = * > log_path = /var/log/dovecot > mail_plugins = fts fts_squat zlib > mail_privileged_group = postfix > mail_temp_dir = /var/db/dovecot > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date ihave > namespace { > inbox = yes > location = > prefix = > separator = . > type = private > } > namespace { > hidden = yes > inbox = no > list = no > location = > prefix = INBOX. > separator = . > type = private > } > passdb { > args = /usr/local/etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > plugin { > fts = squat > fts_squat = partial=4 full=10 > sieve = ~/.dovecot.sieve > sieve_dir = ~/sieve > } > protocols = imap pop3 lmtp sieve > service auth-worker { > user = $default_internal_user > } > service auth { > unix_listener /home/mail/postfix/private/dovecot-auth { > group = postfix > mode = 0660 > user = postfix > } > } > service lmtp { > executable = lmtp -L > unix_listener /home/mail/postfix/private/dovecot-lmtp { > group = postfix > mode = 0660 > user = postfix > } > } > ssl_cert = ssl_key = userdb { > args = /usr/local/etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > protocol lmtp { > log_path = /var/log/dovecot-deliver > mail_plugins = fts fts_squat zlib sieve > } > protocol lda { > mail_plugins = fts fts_squat zlib sieve > } > protocol imap { > mail_plugins = fts fts_squat zlib imap_zlib > } > > > Best regards > Henrik Larsson From dchenusa at yahoo.com Fri Apr 20 11:29:31 2012 From: dchenusa at yahoo.com (Dennis Chen) Date: Fri, 20 Apr 2012 01:29:31 -0700 Subject: [Dovecot] Fwd: IMAP SSL incoming test need help ! References: <82BE5266-7A1D-4E1F-9862-F503668D6EAF@yahoo.com> Message-ID: <485DD6C9-EFCB-4644-9BFA-D0C0CCA224EA@yahoo.com> Sent from my iPhone Begin forwarded message: > From: Dennis Chen > Date: April 19, 2012 5:38:01 PM PDT > To: dovecot at dovecot.org > Subject: Fwd: [Dovecot] IMAP SSL incoming test need help ! > > > > Sent from my iPhone > > Begin forwarded message: > >> From: D Chen >> Date: April 19, 2012 4:06:49 PM PDT >> To: lists-dovecot >> Subject: Re: [Dovecot] IMAP SSL incoming test need help ! >> Reply-To: D Chen >> > >> Thanks for nice pointing out in my mail.log! >> >> I could not find where is the procmail log located under /var/log ! >> >> As procmail is "suspicious", I recalled that I selected either use procmail or not use for local delivery while I re-configure the postfix. I ran "sudo dpkg-reconfigure postfix" again, and selected NOT use "Procmail" for local delivery, restart postfix and ran the tested again, it worked now ! examined the mail.log, it said "delivered to maildir" instead of procmail. >> >> why works NOT using procmail ? thx. >> >> From: lists-dovecot >> To: D Chen >> Sent: Thursday, April 19, 2012 5:22 AM >> Subject: Re: [Dovecot] IMAP SSL incoming test need help ! >> >> >> >> ------------ Original Message ------------ >> > Date: Thursday, April 19, 2012 01:44:35 AM -0700 >> > From: D Chen >> > To: Dovecot Mailing List >> > Subject: [Dovecot] IMAP SSL incoming test need help ! >> > >> > Ubuntu 11.10 server with postfix/dovecot/squirrelmail configured. >> > >> > Outgoing SMTP server seemed worked fine, however, the incoming >> > IMAP with SSL on port 993 doesn't seem to receive any mail at all >> > !? Here is the mail.log during the test session from an external >> > yahoo mail (dchenusa at yahoo.com) to the internal server >> > testmail.biokeyinc.com (dchen at testmail.biokeyinc.com) The log did >> > NOT show any error ! The sender showed sent mail! but no mail >> > reach to the destination! Any help would be appreciated. >> > >> > Apr 19 01:20:02 testmail postfix/smtpd[4103]: connect from >> > nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:03 >> > testmail postfix/smtpd[4103]: 091ED200973: >> > client=nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 >> > 01:20:03 testmail postfix/cleanup[4108]: 091ED200973: >> > message-id=<32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com> Apr 19 >> > 01:20:03 testmail postfix/qmgr[2007]: 091ED200973: >> > from=, size=2840, nrcpt=1 (queue active) Apr >> > 19 01:20:03 testmail postfix/smtpd[4103]: disconnect from >> > nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:12 >> > testmail postfix/smtpd[4113]: connect from localhost[127.0.0.1] >> > Apr 19 01:20:12 testmail postfix/smtpd[4113]: 33AE9200AE9: >> > client=localhost[127.0.0.1] Apr 19 01:20:12 testmail >> > postfix/cleanup[4108]: 33AE9200AE9: >> > message-id=<32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com> Apr 19 >> > 01:20:12 testmail postfix/qmgr[2007]: 33AE9200AE9: >> > from=, size=3536, nrcpt=1 (queue active) Apr >> > 19 01:20:12 testmail postfix/smtpd[4113]: disconnect from >> > localhost[127.0.0.1] Apr 19 01:20:12 testmail amavis[2042]: >> > (02042-06) Passed CLEAN, [98.138.91.180] [98.138.87.4] >> > -> , >> > Message-ID: <32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com>, >> > mail_id: rqyVZiQDwSsG, Hits: -0.109, size: 2837, queued_as: >> > 33AE9200AE9, dkim_id=dchenusa at yahoo.com, at yahoo.com, 8647 ms Apr 19 >> > 01:20:12 testmail postfix/smtp[4109]: 091ED200973: >> > to=, >> > relay=127.0.0.1[127.0.0.1]:10024, delay=9.5, >> > delays=0.87/0.01/0/8.6, dsn=2.0.0, status=sent (250 2.0.0 from >> > MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 33AE9200AE9) Apr >> > 19 01:20:12 testmail postfix/qmgr[2007]: 091ED200973: removed Apr >> > 19 01:20:12 testmail postfix/local[4114]: 33AE9200AE9: >> > to=, relay=local, delay=0.18, >> > delays=0.1/0/0/0.07, dsn=2.0.0, status=sent (delivered to command: >> > procmail -a "$EXTENSION") Apr 19 01:20:12 testmail >> > postfix/qmgr[2007]: 33AE9200AE9: removed Apr 19 01:21:30 testmail >> > dovecot: imap-login: Login: user=, method=PLAIN, >> > rip=63.195.90.22, lip=192.168.20.100, mpid=4122, TLS Apr 19 >> > 01:23:13 testmail dovecot: imap(dchen): Disconnected: Disconnected >> > in IDLE bytes=99/708 Apr 19 01:23:23 testmail postfix/anvil[4105]: >> > statistics: max connection rate 1/60s for (smtp:98.138.91.180) at >> > Apr 19 01:20:02 Apr 19 01:23:23 testmail postfix/anvil[4105]: >> > statistics: max connection count 1 for (smtp:98.138.91.180) at Apr >> > 19 01:20:02 Apr 19 01:23:23 testmail postfix/anvil[4105]: >> > statistics: max cache size 1 at Apr 19 01:20:02 >> >> ------------ End Original Message ------------ >> >> >> I'm going to bet that procmail is eating the inbound message: >> >> Apr 19 01:20:12 testmail postfix/local[4114]: 33AE9200AE9: >> to=, relay=local, delay=0.18, >> delays=0.1/0/0/0.07, dsn=2.0.0, status=sent (delivered to command: >> >> procmail -a "$EXTENSION") >> >> and it's not getting to the user's inbox. If a message doesn't get >> to the user's inbox then it's not dovecot's issue that it can't be >> retrieved. >> >> fyi -- imap/dovecot have nothing to do your inbound delivery. >> >> > however, the incoming IMAP with SSL on port 993 doesn't >> > seem to receive any mail >> >> that's handled by your MTA, which in your case is postfix, with >> procmail in the mix. >> >> I would try looking at the procmail log to see what's happening >> there. >> >> - Richard >> >> >> >> >> >> >> >> >> >> >> From john.robinson at anonymous.org.uk Fri Apr 20 12:01:45 2012 From: john.robinson at anonymous.org.uk (John Robinson) Date: Fri, 20 Apr 2012 10:01:45 +0100 Subject: [Dovecot] mbox folders on-disc layout In-Reply-To: <35C2B470-438E-4CEC-A42A-0BC2C39A0159@iki.fi> References: <4F8EA753.4070605@anonymous.org.uk> <35C2B470-438E-4CEC-A42A-0BC2C39A0159@iki.fi> Message-ID: <4F9125F9.6060601@anonymous.org.uk> On 18/04/2012 17:45, Timo Sirainen wrote: > On 18.4.2012, at 14.36, John Robinson wrote: > >> I'd like to change the on-disk layout for my mboxes from being >> ~/mail/foo/bar >> to >> ~/mail/foo.bar >> so that I can have folders containing both messages and subfolders, without having subfolders beginning with . and then having to do all the other fiddling with locations of index files etc. that goes along with LAYOUT=maildir++, and also leaving the filesystem layout looking sensible so my users see their mailboxes instead of the index and control files. >> >> Is this possible or do I need to patch something? > > Not possible without patching. There's also the other possibility of doing something like: > > # Trick mbox configuration which allows a mail folder which contains both > # messages and sub-folders > mail_location = mbox:~/mail/mailboxes:DIRNAME=mBoX-MeSsAgEs:INDEX=~/mail/index:CONTROL=~/mail/control > > from http://wiki2.dovecot.org/MboxChildFolders Yes, I read all that, and thought all the options were pretty ugly - and the comment that it's a "trick" configuration would seem to suggest that the original author thought so to. And I think Maildir++ is pretty ugly too. If I wanted to try writing a new LAYOUT, where should I begin? Cheers, John. From CMarcus at Media-Brokers.com Fri Apr 20 14:07:57 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 20 Apr 2012 07:07:57 -0400 Subject: [Dovecot] dovecot LDA with sendmail In-Reply-To: References: Message-ID: <4F91438D.3040200@Media-Brokers.com> On 2012-04-19 9:45 PM, Joseph Tam wrote: > Hadi Salem writes: >> Im configuring sendmail with dovecot for virtual users using >> password file and file for user name. on centos. >> dovecot-1.0.7-7.el5 > Maybe I can beat Charles Marcus to the punch and recommend you upgrade > to the latest version, otherwise you'll run into bugs that have already > been fixed. Lol... I stopped reading at '...sendmail with dovecot...', otherwise I'd have beat you to it... ;) -- Best regards, Charles From news1204.10.weezy at spamgourmet.com Fri Apr 20 14:11:09 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Fri, 20 Apr 2012 13:11:09 +0200 Subject: [Dovecot] default mail quota when using per user quota Message-ID: Hello, using dovecot 2.1 and per user mail quota via ldap is there a way to have a default quota which gets used, if the "quota" field in ldap is not set? I tried with: plugin { quota = maildir:User quota quota_rule = *:storage=3G } userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } and in dovecot-ldap.conf: user_attrs = dcMailQuota=quota_rule=*:storage=%$ but with this configuration the mail quota /allways/ gets loaded from ldap resulting in users without dcMailQuota attribute having no quota at all. Greetings Andreas From nmilas at noa.gr Fri Apr 20 15:26:51 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Fri, 20 Apr 2012 15:26:51 +0300 Subject: [Dovecot] default mail quota when using per user quota In-Reply-To: References: Message-ID: <4F91560B.5040009@noa.gr> On 20/4/2012 2:11 ??, Andreas Helmcke wrote: > using dovecot 2.1 and per user mail quota via ldap is there a way to have a default quota which gets used, if the > "quota" field in ldap is not set? You may see: http://www.mail-archive.com/dovecot at dovecot.org/msg44010.html Regards, Nick From campbell at cnpapers.com Fri Apr 20 15:30:27 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Fri, 20 Apr 2012 08:30:27 -0400 Subject: [Dovecot] Error in logs indicating broken files Message-ID: <4F9156E3.4050801@cnpapers.com> Still new to dovecot, I'm seeing a couple of errors in my log file indicating corruption of index files. Error: Corrupted index cache file/home/xxxxxx/mail/.imap/sent-mail/dovecot.index.cache: Broken virtual size for mail UID 67: 1 Time(s) Error: FETCH [] for mailbox ~/mail/sent-mail UID 67 got too little data: 6203 vs 7478: 1 Time(s) This account is used by multiple users, but it is not a shared account (haven't set any of those up yet - still new to me), so I would expect corruption possibilities when multiple users modify the account folders in some manner. Until I get some experience under my belt and make this a truly "shared" account, is there any way to resolve the corruption of the files mentioned above? Maybe just delete them? thanks steve campbell From ml at smtp.fakessh.eu Fri Apr 20 15:25:01 2012 From: ml at smtp.fakessh.eu (ml) Date: Fri, 20 Apr 2012 14:25:01 +0200 Subject: [Dovecot] 2.1.3 Corrupted squat uidlist In-Reply-To: <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> References: <4F7164DC.7010706@unict.it> <4F8A97F8.70808@spambox.dk> <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> Message-ID: Le 2012-04-20 09:40, Henrik Larsson a ?crit?: > This problem is still showing up every day in the logs. > > I tried to delete the dovecot.index.search and > dovecot.index.search.uids files for all users, but the errors are > still there after a re-build of the search index with > "/usr/local/bin/doveadm search -A TEXT xyzabczzz". > > If you need any further details, please let me know. > > > Best regards > Henrik Larsson > > > On 15-04-2012 10:42, Henrik Larsson wrote: >> On 27-03-2012 08:57, Luca Palazzo wrote: >>> Hi Timo and All, >>> after upgrading to 2.1.2 i'm getting a lot of these messages: >>> Error: Corrupted squat uidlist file XXXXXX wrong indexid >> After an upgrade to 2.1.3 i see this as well. Any thoughts? >> >> Apr 15 03:43:43 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> Apr 15 06:30:27 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> Apr 15 07:43:55 pop3(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> Apr 15 09:00:01 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> >> # doveconf -n >> # 2.1.3: /usr/local/etc/dovecot/dovecot.conf >> # OS: FreeBSD 8.2-STABLE amd64 >> auth_mechanisms = plain login digest-md5 cram-md5 >> first_valid_uid = 125 >> listen = * >> log_path = /var/log/dovecot >> mail_plugins = fts fts_squat zlib >> mail_privileged_group = postfix >> mail_temp_dir = /var/db/dovecot >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date ihave >> namespace { >> inbox = yes >> location = >> prefix = >> separator = . >> type = private >> } >> namespace { >> hidden = yes >> inbox = no >> list = no >> location = >> prefix = INBOX. >> separator = . >> type = private >> } >> passdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> plugin { >> fts = squat >> fts_squat = partial=4 full=10 >> sieve = ~/.dovecot.sieve >> sieve_dir = ~/sieve >> } >> protocols = imap pop3 lmtp sieve >> service auth-worker { >> user = $default_internal_user >> } >> service auth { >> unix_listener /home/mail/postfix/private/dovecot-auth { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> service lmtp { >> executable = lmtp -L >> unix_listener /home/mail/postfix/private/dovecot-lmtp { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> ssl_cert = > ssl_key = > userdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> protocol lmtp { >> log_path = /var/log/dovecot-deliver >> mail_plugins = fts fts_squat zlib sieve >> } >> protocol lda { >> mail_plugins = fts fts_squat zlib sieve >> } >> protocol imap { >> mail_plugins = fts fts_squat zlib imap_zlib >> } >> >> >> Best regards >> Henrik Larsson comparable error exists for corrupted Maildir with the wrong filename i don't no the exact name of error i do not use zlib for this reason i thinks a patch from timo fix this feature later -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation From news1204.10.weezy at spamgourmet.com Fri Apr 20 17:31:33 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Fri, 20 Apr 2012 16:31:33 +0200 Subject: [Dovecot] default mail quota when using per user quota In-Reply-To: <4F91560B.5040009@noa.gr> References: <4F91560B.5040009@noa.gr> Message-ID: Am 20.04.2012 14:26, schrieb Nikolaos Milas: > > You may see: http://www.mail-archive.com/dovecot at dovecot.org/msg44010.html I do not see any relevant differences to my setup. Do you /really/ have a default quota of 4G if roomNumber is empty or rather have unlimited quota then? Greetings Andreas From cor at xs4all.nl Fri Apr 20 19:05:20 2012 From: cor at xs4all.nl (Cor Bosman) Date: Fri, 20 Apr 2012 18:05:20 +0200 Subject: [Dovecot] quota not being calculated Message-ID: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> It looks like my quota isnt being calculated properly after I started setting quota to a specific folder. The quota in that folder is always starting out at 0, and only new email is being added to the quota. If I remove the maildirsize file, and recalculate, it still starts at 0. Once email arrives in the folder, it gets added to maildirsize. This is with maildirquota, in perhaps a bit of an exotic config. The idea is to set a specific quota to Spam folder, using the trash plugin to keep it at a certain size. . getquotaroot "Spam" * QUOTAROOT "Spam" "User quota" "Spam quota" * QUOTA "User quota" (STORAGE 40848 1228800) * QUOTA "Spam quota" (STORAGE 0 20000) . OK Getquotaroot completed. Once 1 email gets added: .Spam# cat maildirsize 20480000S 0 0 2017 1 namespace { hidden = no ignore_on_failure = no inbox = yes list = yes location = maildir:%h:INDEX=/var/spool/mail/dovecot-control/indexes/%1u/%2u/%u:CONTROL=/var/spool/mail/dovecot-control/%1u/%2u/%u/INBOX prefix = separator = / subscriptions = yes type = private } namespace spam { hidden = yes ignore_on_failure = no inbox = no list = yes location = maildir:%h/.Spam:INDEX=/var/spool/mail/dovecot-control/indexes/%1u/%2u/%u/.Spam:CONTROL=/var/spool/mail/dovecot-control/%1u/%2u/%u/.Spam prefix = Spam/ separator = / subscriptions = no type = private } plugin { quota = fs:User quota quota2 = maildir:Spam quota:ns=Spam/ quota2_rule = *:storage=20000K sieve = /var/spool/mail/dovecot-control/sieve/%1u/%2u/%u/dovecot.sieve sieve_before = /etc/sieve/before sieve_dir = /var/spool/mail/dovecot-control/sieve/%1u/%2u/%u/scripts trash = /etc/dovecot/conf.d/dovecot-trash.conf.ext } (full config: http://pastebin.com/Mui4X7Zh) From tim.ruehsen at gmx.de Fri Apr 20 17:27:54 2012 From: tim.ruehsen at gmx.de (Tim Ruehsen) Date: Fri, 20 Apr 2012 16:27:54 +0200 Subject: [Dovecot] array code issue ? Message-ID: <201204201627.54213.tim.ruehsen@gmx.de> Hi, I just took a look into the dovecot 2.1 sources and just saw a possible issue in array.h. This code snippet as an example: #static inline void * #array_get_modifiable_i(struct array *array, unsigned int *count_r) #{ # *count_r = array->buffer->used / array->element_size; # return buffer_get_modifiable_data(array->buffer, NULL); #} array->buffer->used and array->element_size are of type 'size_t' which is 64bit on amd64 and others while 'count_r' is a 32bit value. At least, I see ugly warnings with -Wconversion (which I personally like to use). I know, it is unlikely that 'array->buffer->used / array->element_size' exceeds 32bit range. But then, dovecot's source is so well written, that the above code seems to disturb dovecot's code aesthetics. And who knows... in a few years (when we have THz and TBytes on our desktops) emails (and array sizes) might exceed everything that we think of today. Tim From mafonso at hangas.net Fri Apr 20 19:19:48 2012 From: mafonso at hangas.net (Miguel Afonso) Date: Fri, 20 Apr 2012 17:19:48 +0100 Subject: [Dovecot] dbox vs. mdbox In-Reply-To: References: <4D69FCB5.3090100@wildgooses.com> <58BED0A6-9EA0-4B4D-9065-69B22033D627@iki.fi> Message-ID: Hi, I finally deployed a migration test plant and done some conversions last night, so I took the chance to collect some data of these tests. I've set up a virtual machine with a dovecot 2.0.13 installation in order test the mailbox conversion. This machine has two adicional disks, one with the source data and other to store the conversion results. The source data is a restore from the latest production Maildir backup and I did a conversion to sdbox, and then to mdbox. I attach DiskIO and CPU plots for both conversions orange Line is read from source disk, blue line is write to destination. sdbox on the left and mdbox on the right. Ok, this is a conversion process and not actual mailbox usage. And lacks info on how does dbox behaves in read operations.. but this is what I got for now. [image: Inline image 1] [image: Inline image 2] Miguel On Thu, Mar 29, 2012 at 12:16 PM, Miguel Afonso wrote: > > > On Thu, Mar 29, 2012 at 12:30 AM, Timo Sirainen wrote: > >> >> The main problem is that it's difficult to do any "real world" tests with >> IMAP, especially when users are using many different kinds of IMAP clients. >> So I'm very interested in hearing some numbers (and disk IO graphs for a >> few weeks would be great) before your migration and after your migration, >> but the numbers for your tests might not mean all that much. > > > I was considering using the imaptest tool to simulate IMAP activity. I > would keep the same machine configuration, only varying the mailbox format > while running imaptest against each setup for a few hours/days. > > I'm now converting the original Maildir format to both dbox formats and > I'll give it a try. I'll share some graphs afterwards. > > -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 24202 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 27261 bytes Desc: not available URL: From tss at iki.fi Fri Apr 20 20:13:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 20 Apr 2012 20:13:29 +0300 Subject: [Dovecot] array code issue ? In-Reply-To: <201204201627.54213.tim.ruehsen@gmx.de> References: <201204201627.54213.tim.ruehsen@gmx.de> Message-ID: On 20.4.2012, at 17.27, Tim Ruehsen wrote: > I just took a look into the dovecot 2.1 sources and just saw a possible issue > in array.h. > > This code snippet as an example: > #static inline void * > #array_get_modifiable_i(struct array *array, unsigned int *count_r) > #{ > # *count_r = array->buffer->used / array->element_size; > # return buffer_get_modifiable_data(array->buffer, NULL); > #} > > array->buffer->used and array->element_size are of type 'size_t' which is > 64bit on amd64 and others while 'count_r' is a 32bit value. At least, I see > ugly warnings with -Wconversion (which I personally like to use). I've been planning on trying out some of clang's warning flags. Last time I used -Wconversion with gcc it was giving way too many warnings to be usable, but clang's -Wconversion looked better when I quickly looked at it. > I know, it is unlikely that 'array->buffer->used / array->element_size' > exceeds 32bit range. But then, dovecot's source is so well written, that the > above code seems to disturb dovecot's code aesthetics. :) Yeah, I intentionally decided to use unsigned int here. It's a bit of wasteful and ugly to use size_t everywhere.. I guess the code could be made something like: size_t count = array->buffer->used / array->element_size; I_assert(count < UINT_MAX); *count_r = (unsigned int)count; Or something like that. Although these array functions are sometimes in performance critical paths, so adding extra code isn't very good either. Perhaps a simple cast to make the warning go away.. Probably the element_size could also be changed to be unsigned int. > And who knows... in a few years (when we have THz and TBytes on our desktops) > emails (and array sizes) might exceed everything that we think of today. The email sizes yes, but probably not the number of emails in a mailbox. From nmilas at noa.gr Fri Apr 20 21:01:39 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Fri, 20 Apr 2012 21:01:39 +0300 Subject: [Dovecot] default mail quota when using per user quota In-Reply-To: References: <4F91560B.5040009@noa.gr> Message-ID: <4F91A483.8000406@noa.gr> On 20/4/2012 5:31 ??, Andreas Helmcke wrote: > Do you/really/ have a default quota of 4G if roomNumber is empty or rather have unlimited quota then? Yes, default quota is 4G for all mailboxes; if roomNumber is defined, then that value is used instead. In practice we are using non-default values for very few maiboxes. We have a relatively small number of users/mailboxes (<300), so we have the luxury of enjoying relatively large mailboxes (with some "overbooking" - because most users rarely use more than 1G). Nick From CMarcus at Media-Brokers.com Fri Apr 20 21:05:42 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 20 Apr 2012 14:05:42 -0400 Subject: [Dovecot] Error in logs indicating broken files In-Reply-To: <4F9156E3.4050801@cnpapers.com> References: <4F9156E3.4050801@cnpapers.com> Message-ID: <4F91A576.1090709@Media-Brokers.com> On 2012-04-20 8:30 AM, Steve Campbell wrote: > Still new to dovecot, > > I'm seeing a couple of errors in my log file indicating corruption of > index files. It is customary to provide at least some bare minimal system details, like, dovecot -n output (which includes the version), platform/OS, etc... -- Best regards, Charles From jeff at j-simmons.net Fri Apr 20 22:26:41 2012 From: jeff at j-simmons.net (Jeff Simmons) Date: Fri, 20 Apr 2012 12:26:41 -0700 Subject: [Dovecot] dsync: convert only mbox in /var/mail to maildir? Message-ID: <201204201226.41710.jeff@j-simmons.net> I'm migrating an old UW pop server to a new Dovecot server. The old server did not allow mail to be stored on the server, so only mbox files exist in /var/mail, with nothing in /home/$user. I'd like to copy the /var/mail directory over to the new server and then use dsync to convert the mbox files and put them in /home/$user/Maildir, but every permutation of the command line I've tried dies with a variation on can't find/write either /var/mail/mail or /home/$user/mail, seems it won't deal with mbox files without an associated mail directory. Can dsync be used for this kind of conversion, or should I look elsewhere for conversion utilities? Thanks for helping a noob who can't traverse a wiki without leaving a trail of breadcrumbs. -- Jeff Simmons jeff at j-simmons.net Simmons Consulting - Network Engineering, Administration, Security From victormanuelo at gmail.com Fri Apr 20 23:40:18 2012 From: victormanuelo at gmail.com (=?UTF-8?Q?Victor_O=C3=B1ate?=) Date: Fri, 20 Apr 2012 16:10:18 -0430 Subject: [Dovecot] sieve against ldap in Dovecot 1.2 vacation messeage Message-ID: Hi. I need configure "sieve" against ldap this is my file configuration dovecot.conf plugin { quota = dirsize:user sieve_dir = ~/sieve } protocol lda { mail_plugins = sieve } auth default { mechanisms = plain passdb pam { } userdb ldap { args = /etc/dovecot/dovecot-ldap.conf } user = nobody } I dont know what I have to put in dovecot-ldap.conf, ofcourse I have a atribut in LDAP by user with a vacation messeage "I'm out of office, please contact Mr. Black" this is dovecot-ldap.conf hosts = server auth_bind = yes ldap_version = 3 base = dc=dominio,dc=uk deref = never scope = subtree user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid,mail=Maildir,mailQuotaSize=quota_rule=*:storage=%$ user_filter = (&(objectClass=posixAccount)(uid=%u)) The principal idea is keep the control the message by user with LDAP and Dovecot(IMAP). without touching the postfix configuration file is that possible, or am I dreaming. Thanks.. Victor O?ate. From tss at iki.fi Sat Apr 21 00:18:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Apr 2012 00:18:24 +0300 Subject: [Dovecot] default mail quota when using per user quota In-Reply-To: References: Message-ID: <553C5242-5931-47B1-9FA9-0B64A7216197@iki.fi> On 20.4.2012, at 14.11, Andreas Helmcke wrote: > using dovecot 2.1 and per user mail quota via ldap is there a way to have a default quota which gets used, if the > "quota" field in ldap is not set? By "not set" you mean it doesn't exist at all, or it exists but is empty? > I tried with: > > plugin { > quota = maildir:User quota > quota_rule = *:storage=3G > } > > userdb { > args = /etc/dovecot/dovecot-ldap.conf > driver = ldap > } > > and in dovecot-ldap.conf: > user_attrs = dcMailQuota=quota_rule=*:storage=%$ If dcMailQuota isn't returned, then the global quota_rule is used. If it is returned as empty, I guess it gets treated as unlimited quota. There's currently no easy way avoid this (a difficult way could be e.g. post-login scripting, but that works only for imap/pop3). From tss at iki.fi Sat Apr 21 00:23:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Apr 2012 00:23:02 +0300 Subject: [Dovecot] mbox folders on-disc layout In-Reply-To: <4F9125F9.6060601@anonymous.org.uk> References: <4F8EA753.4070605@anonymous.org.uk> <35C2B470-438E-4CEC-A42A-0BC2C39A0159@iki.fi> <4F9125F9.6060601@anonymous.org.uk> Message-ID: <73A197D0-6D8A-4C7D-B6DE-F6B89EC40A0B@iki.fi> On 20.4.2012, at 12.01, John Robinson wrote: > Yes, I read all that, and thought all the options were pretty ugly - and the comment that it's a "trick" configuration would seem to suggest that the original author thought so to. And I think Maildir++ is pretty ugly too. If I wanted to try writing a new LAYOUT, where should I begin? src/lib-storage/mailbox-list-private.h has the API you need to implement src/lib-storage/list/mailbox-list-fs* is the "fs" implementation src/lib-storage/list/mailbox-list-maildir* is the "maildir++" implementation So decide which one looks more like the one you want, and either copy&paste it or start extending it. For example "imapdir" is nearly the same implementation as "maildir++" but almost all of the code is shared. From tss at iki.fi Sat Apr 21 00:27:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Apr 2012 00:27:59 +0300 Subject: [Dovecot] dsync: convert only mbox in /var/mail to maildir? In-Reply-To: <201204201226.41710.jeff@j-simmons.net> References: <201204201226.41710.jeff@j-simmons.net> Message-ID: <04D01B3D-CD58-4E0A-8C1A-36B01E4792E7@iki.fi> On 20.4.2012, at 22.26, Jeff Simmons wrote: > I'm migrating an old UW pop server to a new Dovecot server. The old server did > not allow mail to be stored on the server, so only mbox files exist in > /var/mail, with nothing in /home/$user. I'd like to copy the /var/mail > directory over to the new server and then use dsync to convert the mbox files > and put them in /home/$user/Maildir, but every permutation of the command line > I've tried dies with a variation on can't find/write either /var/mail/mail or > /home/$user/mail, seems it won't deal with mbox files without an associated > mail directory. Can dsync be used for this kind of conversion, or should I > look elsewhere for conversion utilities? Assuming in destination server the mbox is in /var/mail/$user and you want the destination to ~/Maildir: in dovecot.conf set: mail_location = maildir.~/Maildir rm -rf /tmp/emptydir dsync mirror mbox:/tmp/emptydir:/var/mail/$user be sure to delete the emptydir between users. From tss at iki.fi Sat Apr 21 00:29:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Apr 2012 00:29:40 +0300 Subject: [Dovecot] quota not being calculated In-Reply-To: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> References: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> Message-ID: <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> On 20.4.2012, at 19.05, Cor Bosman wrote: > It looks like my quota isnt being calculated properly after I started setting quota to a specific folder. The quota in that folder is always starting out at 0, and only new email is being added to the quota. If I remove the maildirsize file, and recalculate, it still starts at 0. Once email arrives in the folder, it gets added to maildirsize. This is with maildirquota, in perhaps a bit of an exotic config. The idea is to set a specific quota to Spam folder, using the trash plugin to keep it at a certain size. I'll try later, but try if it would work with dict quota file? http://wiki2.dovecot.org/Quota/Dict From news1204.10.weezy at spamgourmet.com Sat Apr 21 00:47:44 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Fri, 20 Apr 2012 23:47:44 +0200 Subject: [Dovecot] default mail quota when using per user quota (news1204: message 6 of 20) In-Reply-To: <553C5242-5931-47B1-9FA9-0B64A7216197@iki.fi> References: <553C5242-5931-47B1-9FA9-0B64A7216197@iki.fi> Message-ID: <4F91D980.8070307@spamgourmet.com> Am 20.04.2012 23:18, schrieb Timo Sirainen: > By "not set" you mean it doesn't exist at all, or it exists but is empty? I tried both (at least I do think I tried both). > If dcMailQuota isn't returned, then the global quota_rule is used. If it is returned as empty, I guess it gets treated as unlimited quota. There's currently no easy way avoid this (a difficult way could be e.g. post-login scripting, but that works only for imap/pop3). > Maybe openldap does not distingish between this cases. I will try again on monday, when I am back at work. From tss at iki.fi Sat Apr 21 00:54:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Apr 2012 00:54:01 +0300 Subject: [Dovecot] default mail quota when using per user quota (news1204: message 6 of 20) In-Reply-To: <4F91D980.8070307@spamgourmet.com> References: <553C5242-5931-47B1-9FA9-0B64A7216197@iki.fi> <4F91D980.8070307@spamgourmet.com> Message-ID: On 21.4.2012, at 0.47, Andreas Helmcke wrote: >> If dcMailQuota isn't returned, then the global quota_rule is used. If it is returned as empty, I guess it gets treated as unlimited quota. There's currently no easy way avoid this (a difficult way could be e.g. post-login scripting, but that works only for imap/pop3). >> > > Maybe openldap does not distingish between this cases. I will try again on > monday, when I am back at work. Hm. Or maybe I broke it in v2.1 when I changed this code. I'll also check later. :) From stan at hardwarefreak.com Sat Apr 21 03:22:05 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 20 Apr 2012 19:22:05 -0500 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8FDD9B.6060007@hardwarefreak.com> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <4F8D69AB.5090706@esiee.fr> <4F8FDD9B.6060007@hardwarefreak.com> Message-ID: <4F91FDAD.7050102@hardwarefreak.com> On 4/19/2012 4:40 AM, Stan Hoeppner wrote: > On 4/17/2012 8:01 AM, Frank Bonnet wrote: > >> have 4000/6000 imaps concurent connections during working hours . >>>> for approx 50K "intensives" users. >>>> >>>> The only mandatory thing will be I must use HP proliant servers >>>> >>>> The operating system will be FreeBSD or Linux > I just made the wishlist public so it should be available tomorrow or > Friday. I'll provide the link when it's available. And here it is: http://secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=16797311 Since your requirement is for an HP solution, following is an HP server and storage system solution of roughly identical performance and redundancy to the SuperMicro based system I detailed. The HP system solution is $44,263, almost double the cost at $20,000 more. Due to the stupidity of Newegg requiring all wish lists to be reviewed before going live, I'll simply provide the links to all the products. Yes boys and girls, Newegg isn't just consumer products. They carry nearly the entire line of HP Proliant servers and storage, including the 4-way 48-core Opteron DL585 G7 w/64GB, the P2000 fiber channel array, and much more. In this case they sell every product needed to assemble this complete mail server solution: 1x http://www.newegg.com/Product/Product.aspx?Item=N82E16859105807 8x http://www.newegg.com/Product/Product.aspx?Item=N82E16820326150 3x http://www.newegg.com/Product/Product.aspx?Item=N82E16816401143 80x http://www.newegg.com/Product/Product.aspx?Item=N82E16822332061 3x http://www.newegg.com/Product/Product.aspx?Item=N82E16816118109 3x http://www.newegg.com/Product/Product.aspx?Item=N82E16816118163 2x http://www.newegg.com/Product/Product.aspx?Item=N82E16816133048 2x http://www.newegg.com/Product/Product.aspx?Item=N82E16833106050 The 9280-8e RAID controllers are identical to 9261-8i boards but have 2 external vs internal x4 6Gb SAS ports. I spec them instead of the Smart Array boards as they're far cheaper, easier to work with, and offer equal or superior performance. Thus everything written below is valid for this system as well, with the exception that you would configure 1 global hot spare in each chassis since these units have 25 drive bays instead of 24. The D2700 units come with 20" 8088 cables. I an additional spec'd two 3ft cables to make sure we reach all 3 disk chassis from the server, thinking the sever would be on top with the 3 disk chassis below. I hope this and my previous post are helpful in one aspect or another to Frank and anyone else. I spent more than a few minutes on these designs. ;) Days in fact on the SuperMicro design, only a couple of hours on the HP. It wouldn't have taken quite so long if all PCIe slots were created equal (x8), which they're not, or if modern servers didn't require 4 different types of DIMMs depending on how many slots you want to fill and how much expansion capacity you need without having to throw out all the previous memory, which many folks end up doing out of ignorance. Memory configuration is simply too darn complicated with high cap servers containing 8 channels and 24 slots. > The key to performance, and yielding a single file tree, is once again > using XFS to take advantage of this large spindle count across 3 RAID > controllers. Unlike previous configurations where I recommended using a > straight md concatenation of hardware RAID1 pairs, in this case we're > going to use a concatenation of 6 hardware RAID10 arrays. There are a > couple of reasons for doing so in this case: > > 1. Using 36 device names in a single md command line is less than > intuitive and possibly error prone. Using 6 is more manageable. > > 2. We have 3 BBWC RAID controllers w/24 drives each. This is a high > performance server and will see a high IO load in production. In many > cases one would use an external filesystem journal, which we could > easily do and get great performance with our mirrored SSDs. However, > the SSDs are not backed by BBWC, so a UPS failure or system crash could > hose the log journal. So we'll go with the default internal journal > which will be backed by the BBWC. > > Going internal with the log in this mail scenario can cause a serious > amount of extra IOPS on the filesystem data section, this being > Allocation Group 0. If we did the "normal" RAID1 concat, all the log IO > would hit the first RAID1 pair. On this system, the load may hit that > spindle pretty hard, making access to mailboxes in AG0 slower than > others. With 6 RAID10 arrays in a concat, the internal log writes will > be striped across 6 spindles in the first array. With 512MB BBWC > backing that array and optimizing writeout, and with delaylog, this will > yield optimal log write performance without slowing down mailbox file > access in AG0. To create such a setup we'd do something like this, > assuming the mobo LSI controller yields sd[ab], and the 6 array devices > on the PCIe LSI cards yield sd[cdefgh] > > 1. Create two RAID10 arrays, each of 12 drives, in the WebBIOS GUI of > each LSI card, using a strip size of 32KB which should yield good random > r/w performance for any mailbox format. Use the following policies for > each array: RW, Normal, Wback, Direct, Disable, No, and use the full > size. > > Create the concatenated md device: > $ mdadm -C /dev/md0 -l linear -n 6 /dev/sd[cdefgh] > > Then we format it with XFS, optimizing the AG layout for our mailbox > workload, and allocation write stripe alignment to each hardware array: > $ mkfs.xfs -d agcount=24 su=32k sw=6 /dev/md0 > > This yields 4 AGs per RAID10 array which will minimize the traditional > inode64 head seeking overhead on striped arrays, while still yielding > fantastic allocation parallelism with 24 AGs. > > Optimal fstab for MTA queue/mailbox workload, assuming kernel 2.6.39+: > /dev/md0 /mail xfs defaults,inode64,nobarrier 0 0 > > We disable write barriers as we have BBWC. And that 1.5GB of BBWC will > yield extremely low Dovecot write latency and throughput. > > Given the throughput available, if you're running Postfix on this box, > you will want to create a directory on this filesystem for the Postfix > spool. Postfix puts the spool files in many dozens, hundreds of > subdirectories, so you'll get 100% parallelism across all AGs, thus all > disks. > > It's very likely none of you will decide to build this system. My hope > is that some of the design concepts and components used, along with the > low cost but high performance of this machine, may be educational or > simply give people new ideas, steer them in directions they may not have > previously considered. -- Stan From jtam.home at gmail.com Sat Apr 21 06:32:30 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Fri, 20 Apr 2012 20:32:30 -0700 (PDT) Subject: [Dovecot] Error in logs indicating broken files In-Reply-To: References: Message-ID: Steve Campbell writes: > I'm seeing a couple of errors in my log file indicating corruption of > index files. > > Error: Corrupted index cache file/home/xxxxxx/mail/.imap/sent-mail/dovecot.index.cache: Broken virtual size for mail UID 67: 1 Time(s) > Error: FETCH [] for mailbox ~/mail/sent-mail UID 67 got too little data: 6203 vs 7478: 1 Time(s) > > This account is used by multiple users, but it is not a shared account > (haven't set any of those up yet - still new to me), so I would expect > corruption possibilities when multiple users modify the account folders > in some manner. This is just one dovecot IMAP client process moaning after another user client process pulled the rug out from under it by modifying the mailbox. I get a few dozen log entries of this sort per week with ~200 IMAP users without shared access. Unless your users actually complain about a problem, I think it's safe to ignore them. Joseph Tam From lgruen at gmx.ch Sat Apr 21 09:20:32 2012 From: lgruen at gmx.ch (=?ISO-8859-15?Q?=22L=2E_Gr=FCn=22?=) Date: Sat, 21 Apr 2012 08:20:32 +0200 Subject: [Dovecot] Correction of home and mail directory config Message-ID: <4F9251B0.5030501@gmx.ch> Hello, after running dovecot for a while, I stumbled accross the warning, not to have mail and home directory settings point to one and same directory. Here is the output of dovecot -n: > dovecot -n # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686 i686 Debian 6.0.4 ext3 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imaps imap disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login mail_max_userip_connections: 100 mail_privileged_group: mail mail_uid: 5000 mail_gid: 5000 mail_location: maildir:/var/vmail/%d/%n/Maildir mail_full_filesystem_access: yes mbox_write_locks: fcntl dotlock namespace: type: private separator: . inbox: yes list: yes subscriptions: yes lda: auth_socket_path: /var/run/dovecot/auth-master postmaster_address: xxxx at xxxx.org mail_plugins: sieve log_path: auth default: mechanisms: plain login passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: static args: uid=5000 gid=5000 home=/var/vmail/%d/%n/Maildir allow_all_users=yes socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail In "args: uid=5000 gid=5000 home=/var/vmail/%d/%n/Maildir allow_all_users=yes" I should delete "Maildir". My question is: What files do I have to copy from /Maildir to the home directory /var/vmail/%d/%n/ ? Are there other side effects, I have to consider? Thank you very much for your help! Stefan From cor at xs4all.nl Sat Apr 21 11:01:51 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sat, 21 Apr 2012 10:01:51 +0200 Subject: [Dovecot] quota not being calculated In-Reply-To: <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> References: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> Message-ID: > >> It looks like my quota isnt being calculated properly after I started setting quota to a specific folder. The quota in that folder is always starting out at 0, and only new email is being added to the quota. If I remove the maildirsize file, and recalculate, it still starts at 0. Once email arrives in the folder, it gets added to maildirsize. This is with maildirquota, in perhaps a bit of an exotic config. The idea is to set a specific quota to Spam folder, using the trash plugin to keep it at a certain size. > > I'll try later, but try if it would work with dict quota file? http://wiki2.dovecot.org/Quota/Dict > This almost works, but maybe there's a small bug. Config (2.1.4) namespace spam { type = private separator = / inbox = no prefix = Spam/ location = maildir:%h/.Spam:INDEX=/var/spool/mail/dovecot-control/indexes/%1u/%2u/%u/.Spam:CONTROL=/var/spool/mail/dovecot-control/%1u/%2u/%u/.Spam subscriptions = no list = yes hidden = yes } plugin { quota = fs:User quota quota2 = dict:Spam Quota::ns=spam:file:%h/spam-quota quota2_rule = *:storage=20M } Now I see: . getquotaroot "Spam" * QUOTAROOT "Spam" "User quota" "Spam Quota" * QUOTA "User quota" (STORAGE 10128 1228800) * QUOTA "Spam Quota" (STORAGE 8727 20480) The problem is that the storage calculated for the spam ns is wrong. It is less than 1MB. The 8727 seems to be the whole maildir including all folders. This seems to be caused by: Apr 21 10:00:11 lmtp1 dovecot: imap(cor): Error: quota: Unknown namespace: spam Regards, Cor From janfrode at tanso.net Sat Apr 21 12:52:57 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sat, 21 Apr 2012 11:52:57 +0200 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F90AE51.8080407@hardwarefreak.com> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <20120417124241.GA23120@dibs.tanso.net> <4F8D6BAA.7060501@gtdinternet.com> <20120417200820.GA30743@dibs.tanso.net> <4F90AE51.8080407@hardwarefreak.com> Message-ID: <20120421095257.GA19877@dibs.tanso.net> On Thu, Apr 19, 2012 at 07:31:13PM -0500, Stan Hoeppner wrote: > > This issue has come up twice on the Postfix list in less than a month. Oh, thanks! I'll look into those list posts.. I had mostly given up solving this by rate limits and decided to throw hardware at the problem when I saw the log entries for sender *.anpdm.com.. Seems to be a newsletter sender, which I found as 203 different mailserver ip-addresses in our incoming mailserver logs, from 53 different B-nets and 8 different A-nets. Will give smtpd_client_connection_count_limit a try.. -jf From janfrode at tanso.net Sat Apr 21 13:22:52 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sat, 21 Apr 2012 12:22:52 +0200 Subject: [Dovecot] migrate 15K users to new domain name In-Reply-To: References: <20120419082245.GA4050@dibs.tanso.net> <20120419132803.GA9396@dibs.tanso.net> <20120419141930.GB10069@dibs.tanso.net> Message-ID: <20120421102252.GA20159@dibs.tanso.net> On Thu, Apr 19, 2012 at 05:59:39PM +0300, Timo Sirainen wrote: > > With v2.1.4 you could do something like: > > doveadm -c dummy.conf user -m user at domain > > where dummy.conf contains the minimum configuration needed: > > mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln > ssl = no > Thanks! Works perfect. -jf From tss at iki.fi Sat Apr 21 13:32:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Apr 2012 13:32:21 +0300 Subject: [Dovecot] quota not being calculated In-Reply-To: References: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> Message-ID: <2D57E521-3DB3-41D9-B8AB-F280DE546061@iki.fi> On 21.4.2012, at 11.01, Cor Bosman wrote: > prefix = Spam/ .. > quota2 = dict:Spam Quota::ns=spam:file:%h/spam-quota .. > Apr 21 10:00:11 lmtp1 dovecot: imap(cor): Error: quota: Unknown namespace: spam Oh. It would make more sense to have ns=spam, but unfortunately I hadn't thought of namespace names before v2.1. You need to use ns=Spam/ From cor at xs4all.nl Sat Apr 21 13:42:35 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sat, 21 Apr 2012 12:42:35 +0200 Subject: [Dovecot] quota not being calculated In-Reply-To: <2D57E521-3DB3-41D9-B8AB-F280DE546061@iki.fi> References: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> <2D57E521-3DB3-41D9-B8AB-F280DE546061@iki.fi> Message-ID: On Apr 21, 2012, at 12:32 PM, Timo Sirainen wrote: > On 21.4.2012, at 11.01, Cor Bosman wrote: > >> prefix = Spam/ > . >> quota2 = dict:Spam Quota::ns=spam:file:%h/spam-quota > . >> Apr 21 10:00:11 lmtp1 dovecot: imap(cor): Error: quota: Unknown namespace: spam > > Oh. It would make more sense to have ns=spam, but unfortunately I hadn't thought of namespace names before v2.1. You need to use ns=Spam/ Exact same issue with dict quota. It starts out at 0 (even though there are 116 messages in the Spam folder. Then when 1 email gets saved to the Spam folder, the counter is increased by the size of that 1 email. Cor From nmilas at noa.gr Sat Apr 21 13:53:45 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Sat, 21 Apr 2012 13:53:45 +0300 Subject: [Dovecot] default mail quota when using per user quota (news1204: message 6 of 20) In-Reply-To: References: <553C5242-5931-47B1-9FA9-0B64A7216197@iki.fi> <4F91D980.8070307@spamgourmet.com> Message-ID: <4F9291B9.4080701@noa.gr> On 21/4/2012 12:54 ??, Timo Sirainen wrote: > Hm. Or maybe I broke it in v2.1 when I changed this code. I'll also check later. I'm afraid that's possible. The same configuration in 2.0.13 produces (doveadm quota get -u tester): Quota name Type Value Limit % User quota STORAGE 237803 4194304 5 User quota MESSAGE 531 - 0 and in 2.1.1: Quota name Type Value Limit % User quota STORAGE 0 - 0 User quota MESSAGE 0 - 0 (In the latter test server value should be 0 - because there are no messages there - but limit should be as above. Things do not change after: doveadm quota recalc -u tester) Regards, Nick From almarzuki2001 at hotmail.com Sat Apr 21 17:51:55 2012 From: almarzuki2001 at hotmail.com (Hadi Salem) Date: Sat, 21 Apr 2012 17:51:55 +0300 Subject: [Dovecot] dovecot LDA with sendmail In-Reply-To: <4F91438D.3040200@Media-Brokers.com> References: , , <4F91438D.3040200@Media-Brokers.com> Message-ID: Hi, i installed new version 2.0.20. but its the same problem # 2.0.20: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-164.el5 x86_64 CentOS release 5.4 (Final) auth_debug = yes auth_debug_passwords = yes auth_verbose = yes debug_log_path = /var/log/dovecot.log default_login_user = vmail disable_plaintext_auth = no hostname = host1.bigmama.com info_log_path = /var/log/dovecot.log log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:~/Maildir mail_privileged_group = vmail passdb { args = scheme=PLAIN username_format=%u /etc/dovecot/users driver = passwd-file } postmaster_address = root at host1.bigmama.com service auth { unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } ssl = no userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } Apr 21 17:47:16 host1 sendmail[32561]: q3LElF79032559: to=, ctladdr= (0/0), delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120292, relay=dovecot, dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/local/libexec/dovecot/dovecot-lda) exited with EX_TEMPFAIL > Date: Fri, 20 Apr 2012 07:07:57 -0400 > From: CMarcus at Media-Brokers.com > To: dovecot at dovecot.org > Subject: Re: [Dovecot] dovecot LDA with sendmail > > On 2012-04-19 9:45 PM, Joseph Tam wrote: > > Hadi Salem writes: > >> Im configuring sendmail with dovecot for virtual users using > >> password file and file for user name. on centos. > >> dovecot-1.0.7-7.el5 > > > Maybe I can beat Charles Marcus to the punch and recommend you upgrade > > to the latest version, otherwise you'll run into bugs that have already > > been fixed. > > Lol... I stopped reading at '...sendmail with dovecot...', otherwise I'd > have beat you to it... ;) > > -- > > Best regards, > > Charles From stan at hardwarefreak.com Sat Apr 21 22:59:44 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 21 Apr 2012 14:59:44 -0500 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <20120421095257.GA19877@dibs.tanso.net> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <20120417124241.GA23120@dibs.tanso.net> <4F8D6BAA.7060501@gtdinternet.com> <20120417200820.GA30743@dibs.tanso.net> <4F90AE51.8080407@hardwarefreak.com> <20120421095257.GA19877@dibs.tanso.net> Message-ID: <4F9311B0.7000503@hardwarefreak.com> On 4/21/2012 4:52 AM, Jan-Frode Myklebust wrote: > On Thu, Apr 19, 2012 at 07:31:13PM -0500, Stan Hoeppner wrote: >> >> This issue has come up twice on the Postfix list in less than a month. > > Oh, thanks! I'll look into those list posts.. I had mostly given up > solving this by rate limits and decided to throw hardware at the problem > when I saw the log entries for sender *.anpdm.com.. Seems to be a newsletter > sender, which I found as 203 different mailserver ip-addresses in our > incoming mailserver logs, from 53 different B-nets and 8 different A-nets. Yeah, they're a newsletter service provider. > Will give smtpd_client_connection_count_limit a try.. Setting this to 1 or 2 should severely slow their delivery rate. You can also do rate limiting at a much more fine grained level with a Postfix policy daemon such as postfwd (Postfix firewall daemon), though the setup is a bit more complicated. -- Stan From daniel.parthey at informatik.tu-chemnitz.de Sat Apr 21 23:29:28 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 21 Apr 2012 22:29:28 +0200 Subject: [Dovecot] How to remove duplicate GUID messages from storage? Message-ID: <20120421202928.GA12998@daniel.localdomain> Hi, I tried to import messages like this: doveadm import -u username at example.org mdbox:/mail/dovecot/example.org/username/mail mbox:/root/mail.txt all This seemed to have imported the same messages in the mail storage, as they were already there. And now I have got a lot of duplicates. Then I deleted the mbox:/root/mail mail stuff from "mailboxes" folder and the storage remained, with duplicate GUIDs. The following command doveadm -v force-resync -u username at example.org INBOX outputs lots of messages like this one, with different GUID doveadm(username at example.org): Error: mdbox /mail/dovecot/example.org/username/mail/storage: Duplicate GUID 0b8a032d66a0924fb42c0000de5f8128 in m.55:45484041 and m.14:52173045 The messages at m.55:45484041 and m.14:52173045 have the same content, since they were seemingly imported from the mail store itself, they are in the storage twice now and require twice as much disk space as before. How can I manually remove these identical, duplicate messages from the storage to save space? Dovecot does not do it automatically. Kind regards, Daniel From jtam.home at gmail.com Sun Apr 22 07:36:57 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Sat, 21 Apr 2012 21:36:57 -0700 (PDT) Subject: [Dovecot] dovecot LDA with sendmail In-Reply-To: References: Message-ID: > i installed new version 2.0.20. but its the same problem > > args = username_format=%u /etc/dovecot/users ... > > Apr 21 17:47:16 host1 sendmail[32561]: q3LElF79032559: to=, ctladdr= (0/0), delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120292, relay=dovecot, dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/local/libexec/dovecot/dovecot-lda) exited with EX_TEMPFAIL Any logs for dovecot? Same sendmail configs as before or did you try the extra flag I suggested? Joseph Tam From dovecot at webrz.net Sun Apr 22 11:03:55 2012 From: dovecot at webrz.net (Jos Chrispijn) Date: Sun, 22 Apr 2012 10:03:55 +0200 Subject: [Dovecot] Dovecot upgrade Message-ID: <4F93BB6B.7000501@webrz.net> Can someone tell me how I can upgrade from Dovecot 1.x to 2.x best? thanks for your reply, Jos Chrispijn From cor at xs4all.nl Sun Apr 22 11:08:44 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sun, 22 Apr 2012 10:08:44 +0200 Subject: [Dovecot] Dovecot upgrade In-Reply-To: <4F93BB6B.7000501@webrz.net> References: <4F93BB6B.7000501@webrz.net> Message-ID: <0BC7794E-1F1E-45BF-8539-61D5D8BAEE74@xs4all.nl> On Apr 22, 2012, at 10:03 AM, Jos Chrispijn wrote: > Can someone tell me how I can upgrade from Dovecot 1.x to 2.x best? > thanks for your reply, > Jos Chrispijn Have you read this? http://wiki2.dovecot.org/Upgrading/2.0 Cor From tlx at leuxner.net Sun Apr 22 11:08:52 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sun, 22 Apr 2012 10:08:52 +0200 Subject: [Dovecot] Dovecot upgrade In-Reply-To: <4F93BB6B.7000501@webrz.net> References: <4F93BB6B.7000501@webrz.net> Message-ID: Am 22.04.2012 um 10:03 schrieb Jos Chrispijn: > Can someone tell me how I can upgrade from Dovecot 1.x to 2.x best? > thanks for your reply, > Jos Chrispijn http://wiki2.dovecot.org/Upgrading/2.0 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dg at dguhl.org Sun Apr 22 12:06:20 2012 From: dg at dguhl.org (Dennis Guhl) Date: Sun, 22 Apr 2012 11:06:20 +0200 Subject: [Dovecot] sieve against ldap in Dovecot 1.2 vacation messeage In-Reply-To: References: Message-ID: <20120422090620.GA22644@laptop-dg.leere.eu> On Fri, Apr 20, 2012 at 04:10:18PM -0430, Victor O?ate wrote: > Hi. > I need configure "sieve" against ldap You can't configure sieve against LDAP. Sieve is a filter language, which is called by the LDA and configured by files in your home dir. For more information about sieve (with Dovecot 1) look here: http://wiki.dovecot.org/LDA/Sieve and here: http://wiki.dovecot.org/LDA/Sieve/Dovecot [..] > The principal idea is keep the control the message by user with LDAP > and Dovecot(IMAP). without touching the postfix configuration file > is that possible, or am I dreaming. What you want is ManageSieve: http://wiki.dovecot.org/ManageSieve more specifically: http://wiki.dovecot.org/ManageSieve/Clients Dennis From dg at dguhl.org Sun Apr 22 12:10:18 2012 From: dg at dguhl.org (Dennis Guhl) Date: Sun, 22 Apr 2012 11:10:18 +0200 Subject: [Dovecot] Correction of home and mail directory config In-Reply-To: <4F9251B0.5030501@gmx.ch> References: <4F9251B0.5030501@gmx.ch> Message-ID: <20120422091018.GB22644@laptop-dg.leere.eu> On Sat, Apr 21, 2012 at 08:20:32AM +0200, "L. Gr?n" wrote: > Hello, > after running dovecot for a while, I stumbled accross the warning, > not to have mail and home directory settings point to one and same > directory. > Here is the output of dovecot -n: > > > dovecot -n > # 1.2.15: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-5-686 i686 Debian 6.0.4 ext3 [..] > mail_location: maildir:/var/vmail/%d/%n/Maildir [..] > auth default: > mechanisms: plain login > passdb: > driver: sql > args: /etc/dovecot/dovecot-sql.conf > userdb: > driver: static > args: uid=5000 gid=5000 home=/var/vmail/%d/%n/Maildir [..] > In "args: uid=5000 gid=5000 home=/var/vmail/%d/%n/Maildir > allow_all_users=yes" I should delete "Maildir". Yes, exactly. > My question is: > What files do I have to copy from /Maildir to the home directory > /var/vmail/%d/%n/ ? Everything concerning sieve should be in the home dir. > Are there other side effects, I have to consider? Less warnings ;) Dennis From cor at xs4all.nl Sun Apr 22 12:33:03 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sun, 22 Apr 2012 11:33:03 +0200 Subject: [Dovecot] Correction of home and mail directory config In-Reply-To: <20120422091018.GB22644@laptop-dg.leere.eu> References: <4F9251B0.5030501@gmx.ch> <20120422091018.GB22644@laptop-dg.leere.eu> Message-ID: <98FF7560-C0B6-495E-9E9C-DA9D1C742085@xs4all.nl> > Everything concerning sieve should be in the home dir. Why? It can be anywhere you want as long as it doesnt conflict with the names of your mailstore. Cor From dovecot-user at spambox.dk Mon Apr 23 11:30:06 2012 From: dovecot-user at spambox.dk (Henrik Larsson) Date: Mon, 23 Apr 2012 09:30:06 +0100 Subject: [Dovecot] 2.1.4 Corrupted squat uidlist In-Reply-To: <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> References: <4F7164DC.7010706@unict.it> <4F8A97F8.70808@spambox.dk> <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> Message-ID: <829b37b74c07acbf148f7bee6a1e175e@larsson.as> Just to let you know, 2.1.4 didn't solve this either. If you need any futher info to get this solved, please let me know. Best regards Henrik Larsson > This problem is still showing up every day in the logs. > > I tried to delete the dovecot.index.search and > dovecot.index.search.uids files for all users, but the errors are > still there after a re-build of the search index with > "/usr/local/bin/doveadm search -A TEXT xyzabczzz". > > If you need any further details, please let me know. > > > Best regards > Henrik Larsson > > > On 15-04-2012 10:42, Henrik Larsson wrote: >> On 27-03-2012 08:57, Luca Palazzo wrote: >>> Hi Timo and All, >>> after upgrading to 2.1.2 i'm getting a lot of these messages: >>> Error: Corrupted squat uidlist file XXXXXX wrong indexid >> After an upgrade to 2.1.3 i see this as well. Any thoughts? >> >> Apr 15 03:43:43 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> Apr 15 06:30:27 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> Apr 15 07:43:55 pop3(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> Apr 15 09:00:01 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> >> # doveconf -n >> # 2.1.3: /usr/local/etc/dovecot/dovecot.conf >> # OS: FreeBSD 8.2-STABLE amd64 >> auth_mechanisms = plain login digest-md5 cram-md5 >> first_valid_uid = 125 >> listen = * >> log_path = /var/log/dovecot >> mail_plugins = fts fts_squat zlib >> mail_privileged_group = postfix >> mail_temp_dir = /var/db/dovecot >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date ihave >> namespace { >> inbox = yes >> location = >> prefix = >> separator = . >> type = private >> } >> namespace { >> hidden = yes >> inbox = no >> list = no >> location = >> prefix = INBOX. >> separator = . >> type = private >> } >> passdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> plugin { >> fts = squat >> fts_squat = partial=4 full=10 >> sieve = ~/.dovecot.sieve >> sieve_dir = ~/sieve >> } >> protocols = imap pop3 lmtp sieve >> service auth-worker { >> user = $default_internal_user >> } >> service auth { >> unix_listener /home/mail/postfix/private/dovecot-auth { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> service lmtp { >> executable = lmtp -L >> unix_listener /home/mail/postfix/private/dovecot-lmtp { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> ssl_cert = > ssl_key = > userdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> protocol lmtp { >> log_path = /var/log/dovecot-deliver >> mail_plugins = fts fts_squat zlib sieve >> } >> protocol lda { >> mail_plugins = fts fts_squat zlib sieve >> } >> protocol imap { >> mail_plugins = fts fts_squat zlib imap_zlib >> } >> >> >> Best regards >> Henrik Larsson From tss at iki.fi Mon Apr 23 13:20:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 13:20:24 +0300 Subject: [Dovecot] quota not being calculated In-Reply-To: References: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> <2D57E521-3DB3-41D9-B8AB-F280DE546061@iki.fi> Message-ID: <96E3D11C-8F71-4726-816A-6C448554BC70@iki.fi> On 21.4.2012, at 13.42, Cor Bosman wrote: > On Apr 21, 2012, at 12:32 PM, Timo Sirainen wrote: > >> On 21.4.2012, at 11.01, Cor Bosman wrote: >> >>> prefix = Spam/ >> . >>> quota2 = dict:Spam Quota::ns=spam:file:%h/spam-quota >> . >>> Apr 21 10:00:11 lmtp1 dovecot: imap(cor): Error: quota: Unknown namespace: spam >> >> Oh. It would make more sense to have ns=spam, but unfortunately I hadn't thought of namespace names before v2.1. You need to use ns=Spam/ > > Exact same issue with dict quota. It starts out at 0 (even though there are 116 messages in the Spam folder. Then when 1 email gets saved to the Spam folder, the counter is increased by the size of that 1 email. http://hg.dovecot.org/dovecot-2.1/rev/4c8f79d1f9f1 should fix it with dict quota. From tss at iki.fi Mon Apr 23 13:34:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 13:34:09 +0300 Subject: [Dovecot] default mail quota when using per user quota (news1204: message 6 of 20) In-Reply-To: <4F9291B9.4080701@noa.gr> References: <553C5242-5931-47B1-9FA9-0B64A7216197@iki.fi> <4F91D980.8070307@spamgourmet.com> <4F9291B9.4080701@noa.gr> Message-ID: On 21.4.2012, at 13.53, Nikolaos Milas wrote: > On 21/4/2012 12:54 ??, Timo Sirainen wrote: > >> Hm. Or maybe I broke it in v2.1 when I changed this code. I'll also check later. Just tried, it works as expected. > I'm afraid that's possible. The same configuration in 2.0.13 produces (doveadm quota get -u tester): > > Quota name Type Value Limit % > User quota STORAGE 237803 4194304 5 > User quota MESSAGE 531 - 0 > > and in 2.1.1: > > Quota name Type Value Limit % > User quota STORAGE 0 - 0 > User quota MESSAGE 0 - 0 > > (In the latter test server value should be 0 - because there are no messages there - but limit should be as above. > Things do not change after: doveadm quota recalc -u tester) What do you get in logs with auth_debug=yes? From tss at iki.fi Mon Apr 23 13:35:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 13:35:46 +0300 Subject: [Dovecot] Error in logs indicating broken files In-Reply-To: References: Message-ID: <53563F64-C80D-4E98-B722-E7154BC0FAB3@iki.fi> On 21.4.2012, at 6.32, Joseph Tam wrote: >> Error: Corrupted index cache file/home/xxxxxx/mail/.imap/sent-mail/dovecot.index.cache: Broken virtual size for mail UID 67: 1 Time(s) >> Error: FETCH [] for mailbox ~/mail/sent-mail UID 67 got too little data: 6203 vs 7478: 1 Time(s) > > This is just one dovecot IMAP client process moaning after another user > client process pulled the rug out from under it by modifying the mailbox. > I get a few dozen log entries of this sort per week with ~200 IMAP users > without shared access. Well, it is a bug and it shouldn't happen. But since it happens only with mbox, I haven't bothered to look into it. From tss at iki.fi Mon Apr 23 13:52:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 13:52:49 +0300 Subject: [Dovecot] 2.1.3 Corrupted squat uidlist In-Reply-To: <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> References: <4F7164DC.7010706@unict.it> <4F8A97F8.70808@spambox.dk> <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> Message-ID: <96F0AD8C-9832-4CA5-B1DA-3936F9E2F0FB@iki.fi> So the doveadm search itself succeeds without errors, but if you immediately do another doveadm search it fails? Or do you need to do something else also in the middle? I couldn't reproduce this with a few tests. On 20.4.2012, at 10.40, Henrik Larsson wrote: > This problem is still showing up every day in the logs. > > I tried to delete the dovecot.index.search and dovecot.index.search.uids files for all users, but the errors are still there after a re-build of the search index with "/usr/local/bin/doveadm search -A TEXT xyzabczzz". > > If you need any further details, please let me know. > > > Best regards > Henrik Larsson > > > On 15-04-2012 10:42, Henrik Larsson wrote: >> On 27-03-2012 08:57, Luca Palazzo wrote: >>> Hi Timo and All, >>> after upgrading to 2.1.2 i'm getting a lot of these messages: >>> Error: Corrupted squat uidlist file XXXXXX wrong indexid >> After an upgrade to 2.1.3 i see this as well. Any thoughts? >> >> Apr 15 03:43:43 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid >> Apr 15 06:30:27 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid >> Apr 15 07:43:55 pop3(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid >> Apr 15 09:00:01 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid >> >> # doveconf -n >> # 2.1.3: /usr/local/etc/dovecot/dovecot.conf >> # OS: FreeBSD 8.2-STABLE amd64 >> auth_mechanisms = plain login digest-md5 cram-md5 >> first_valid_uid = 125 >> listen = * >> log_path = /var/log/dovecot >> mail_plugins = fts fts_squat zlib >> mail_privileged_group = postfix >> mail_temp_dir = /var/db/dovecot >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date ihave >> namespace { >> inbox = yes >> location = >> prefix = >> separator = . >> type = private >> } >> namespace { >> hidden = yes >> inbox = no >> list = no >> location = >> prefix = INBOX. >> separator = . >> type = private >> } >> passdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> plugin { >> fts = squat >> fts_squat = partial=4 full=10 >> sieve = ~/.dovecot.sieve >> sieve_dir = ~/sieve >> } >> protocols = imap pop3 lmtp sieve >> service auth-worker { >> user = $default_internal_user >> } >> service auth { >> unix_listener /home/mail/postfix/private/dovecot-auth { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> service lmtp { >> executable = lmtp -L >> unix_listener /home/mail/postfix/private/dovecot-lmtp { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> ssl_cert = > ssl_key = > userdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> protocol lmtp { >> log_path = /var/log/dovecot-deliver >> mail_plugins = fts fts_squat zlib sieve >> } >> protocol lda { >> mail_plugins = fts fts_squat zlib sieve >> } >> protocol imap { >> mail_plugins = fts fts_squat zlib imap_zlib >> } >> >> >> Best regards >> Henrik Larsson > From tss at iki.fi Mon Apr 23 13:59:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 13:59:44 +0300 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: <2vp569xe2o.ln2@mail.ela-soft.com> References: <2vp569xe2o.ln2@mail.ela-soft.com> Message-ID: <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> On 19.4.2012, at 19.09, Andreas Helmcke wrote: > Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace > dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, > subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ Here's the dummy namespace with prefix="". > Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace : > type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, > subscriptions=yes location=maildir:/home/mail/user/ahelmcke But what is this? You appear to have an empty namespace {} in your configs? > Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Error: user > ahelmcke: Initialization failed: namespace configuration error: Duplicate > namespace prefix: "" That's why it fails with this. From tss at iki.fi Mon Apr 23 14:12:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 14:12:25 +0300 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works / solved by remove listescape In-Reply-To: <4F907529.2020403@schetterer.org> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> <4F90250E.3000200@schetterer.org> <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> <4F903DC1.5070501@schetterer.org> <4F905B2C.8040204@schetterer.org> <4F907529.2020403@schetterer.org> Message-ID: On 19.4.2012, at 23.27, Robert Schetterer wrote: >> prefix = shared/%%u/ simply dont works >> >> i have reverted back to exact namespace config >> of the 2.0.20 server, same result, so something broken in 2.1 >> >> > Hi Timo > problem solved by removing listescape plugin > the acl paths did / instead of dot in domainname > > any chance to get listescape fixed ? Ah, that makes it easy to fix: http://hg.dovecot.org/dovecot-2.1/rev/63af3274fb6f From tss at iki.fi Mon Apr 23 14:16:41 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 14:16:41 +0300 Subject: [Dovecot] imapc / namespace question In-Reply-To: <4F888F5E.5030709@necoro.eu> References: <4F888F5E.5030709@necoro.eu> Message-ID: <23ABA092-7511-4DF5-98AA-932E7338A994@iki.fi> On 13.4.2012, at 23.41, Ren? Neumann wrote: > I've got two questions regarding imapc and namespaces: > > 1) Is there a way of stating, that I only want part of the hierarchy on > the end of an imapc connection to be available here? My "problem": hg has now imapc_list_prefix. > 2) "Private" namespaces only make sense, if location contains something > user-specific like "%u" or "~". Is this correct? > If yes: Is there some other way to make something like an imapc > connection only appear for one user, if there is nothing in the location > or imapc_user to denote the user it belongs to and hence I cannot use > "private"? > The only way I see currently is to use "shared" and set up ACLs to > forbid all users except one to access the namespace (which gets slightly > ugly as I have to use global ACLs, as -- I think -- there is no way to > store ACLs for an imapc storage). Well, it's not necessarily a problem to use a private namespace for anything you want. If you set it hidden=yes (probably should do anyway) then IMAP clients won't even know that you've set it as private. Internally the main difference between private and public namespaces is the default ACLs. From tss at iki.fi Mon Apr 23 14:18:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 14:18:20 +0300 Subject: [Dovecot] French characters from 2.0.x to 2.1.3 In-Reply-To: <4F883FE7.9080401@esiee.fr> References: <4F883FE7.9080401@esiee.fr> Message-ID: <912BFCAA-3ECE-42AA-884F-0FE0BEDB4BBC@iki.fi> On 13.4.2012, at 18.01, Frank Bonnet wrote: > Maybe it's a coincidence but it "seems" > since I upgrade from 2.0.18 to 2.1.3 > that some French characters ( a with accents by example ) > are not well displayed anymore using thunderbird and are > replaced by a "?" character Dovecot doesn't send decoded characters to IMAP clients. Of course there might be something else going on, but I can't really guess what it could be. From news1204.10.weezy at spamgourmet.com Mon Apr 23 14:40:11 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Mon, 23 Apr 2012 13:40:11 +0200 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> References: <2vp569xe2o.ln2@mail.ela-soft.com> <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> Message-ID: <4F953F9B.10307@spamgourmet.com> Am 23.04.2012 12:59, schrieb Timo Sirainen: > On 19.4.2012, at 19.09, Andreas Helmcke wrote: > >> Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace : >> type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, >> subscriptions=yes location=maildir:/home/mail/user/ahelmcke > > But what is this? You appear to have an empty namespace {} in your configs? > nope. You might check yourself; complete dovecot -n: # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) ext4 auth_master_user_separator = * auth_verbose = yes first_valid_uid = 400 mail_debug = yes mail_gid = vmail mail_home = /home/mail/user/%u mail_location = maildir:/home/mail/user/%u mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl namespace Allgemeines { location = maildir:/home/mail/Allgemeines:INDEX=/home/mail/user/%u/SEEN.Allgemeines prefix = Allgemeines/ separator = / subscriptions = no type = public } namespace dummy { alias_for = INBOX/ hidden = yes list = no location = maildir:/home/mail/user/%u/public/ prefix = separator = / type = private } namespace inbox { inbox = yes location = prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { quota = maildir:User quota quota_rule = *:storage=3G } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = * } inet_listener imaps { address = * } } ssl = required ssl_cert = References: <4F883FE7.9080401@esiee.fr> <912BFCAA-3ECE-42AA-884F-0FE0BEDB4BBC@iki.fi> Message-ID: <4F954017.50702@esiee.fr> On 04/23/2012 01:18 PM, Timo Sirainen wrote: > On 13.4.2012, at 18.01, Frank Bonnet wrote: > >> Maybe it's a coincidence but it "seems" >> since I upgrade from 2.0.18 to 2.1.3 >> that some French characters ( a with accents by example ) >> are not well displayed anymore using thunderbird and are >> replaced by a "?" character > Dovecot doesn't send decoded characters to IMAP clients. Of course there might be something else going on, but I can't really guess what it could be. > OK thanks , maybe an update of thunderbird ? From tss at iki.fi Mon Apr 23 14:51:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 14:51:50 +0300 Subject: [Dovecot] Bug in HIGHESTMODSEQ when MODSEQ tracking is not enabled for version 2.0.18. In-Reply-To: <48A61B0976102F488C64EF8A8F1E929D074E9F@XMB118CNC.rim.net> References: <48A61B0976102F488C64EF8A8F1E929D074BBA@XMB118CNC.rim.net> <11BD08AA-2E29-48DF-973D-4D5B4CCE2FF7@iki.fi> <48A61B0976102F488C64EF8A8F1E929D074E9F@XMB118CNC.rim.net> Message-ID: <8A3B9DEC-6E45-4A2B-82FB-1A39BABF2247@iki.fi> On 10.4.2012, at 23.50, Glenn Wurster wrote: >> Yes, modseqs aren't tracked in a mailbox until client expresses an >> interest for them. It would be a waste of disk space to save them >> since 99% of users don't need them. > > Makes sense, our mail client gets caught in the middle though, because it uses HIGHESTMODSEQ to track mailbox updates without using MODSEQ options on SELECT/FETCH to track message updates. It would be actually possible for Dovecot to always keep track of highestmodseq, even if individual modseqs weren't tracked. I almost implemented it, but keeping it backwards compatible with old versions would have needed to make it more complex. Maybe v2.2 could do this. >> 2) If a mailbox doesn't have modseqs enabled, return NOMODSEQ. This >> isn't ideal, but seems like the only possibility. > > The RFC also states that if we return NOMODSEQ we'd have to return a tagged BAD response to "UID FETCH 1 MODSEQ", which appears to one of the commands that enables MODSEQ for Dovecot ("SELECT INBOX (CONDSTORE)" also enables it...). What about returning a BAD response and at the same time start tracking MODSEQ so that future SELECT commands would return HIGHESTMODSEQ? Do we know what email clients are using CONDSTORE options and how they'd react to a mailbox suddenly having MODSEQ capabilities after we just told them it didn't? That's kind of an annoying part of the RFC that it says the commands MUST fail with BAD.. I don't think there was really any good reason to add that text. Also Dovecot hasn't failed those commands earlier also with mailbox formats that don't support modseqs at all. So at least for now I simply made it return NOMODSEQ when modseqs aren't enabled, and the rest of the behavior is the same. From tss at iki.fi Mon Apr 23 14:55:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 14:55:21 +0300 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: <4F953F9B.10307@spamgourmet.com> References: <2vp569xe2o.ln2@mail.ela-soft.com> <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> <4F953F9B.10307@spamgourmet.com> Message-ID: On 23.4.2012, at 14.40, Andreas Helmcke wrote: > Am 23.04.2012 12:59, schrieb Timo Sirainen: >> On 19.4.2012, at 19.09, Andreas Helmcke wrote: >> >>> Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace : >>> type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, >>> subscriptions=yes location=maildir:/home/mail/user/ahelmcke >> >> But what is this? You appear to have an empty namespace {} in your configs? >> > nope. > > You might check yourself; complete dovecot -n: See if this changes the behavior: service config { unix_listener config { mode = 0666 } } I'd guess that makes the vmail work also? Although it still doesn't explain the difference between asking config from config process vs. getting it by executing doveconf. From dovecot-user at spambox.dk Mon Apr 23 14:56:13 2012 From: dovecot-user at spambox.dk (Henrik Larsson) Date: Mon, 23 Apr 2012 12:56:13 +0100 Subject: [Dovecot] 2.1.3 Corrupted squat uidlist In-Reply-To: <96F0AD8C-9832-4CA5-B1DA-3936F9E2F0FB@iki.fi> References: <4F7164DC.7010706@unict.it> <4F8A97F8.70808@spambox.dk> <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> <96F0AD8C-9832-4CA5-B1DA-3936F9E2F0FB@iki.fi> Message-ID: <9ca41dc4b98ee3c8d0e38f4c2f5d187d@larsson.as> > So the doveadm search itself succeeds without errors, but if you > immediately do another doveadm search it fails? Or do you need to do > something else also in the middle? I couldn't reproduce this with a > few tests. Is there any way to produce debug logs that will show what goes wrong. It is consistent in such a way that I will have several errors within a 24 hour period. I will be willing to add code and rebuild dovecot to produce the logs you need. Best regards Henrik Larsson From tss at iki.fi Mon Apr 23 15:02:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 15:02:37 +0300 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: <4F953F9B.10307@spamgourmet.com> References: <2vp569xe2o.ln2@mail.ela-soft.com> <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> <4F953F9B.10307@spamgourmet.com> Message-ID: <2F3EF8E1-2851-418A-8B43-EE0D91A321C8@iki.fi> Oh, right: On 23.4.2012, at 14.40, Andreas Helmcke wrote: > namespace Allgemeines { Lowercase it: namespace allgemeines { This is a bit annoying bug, but I'm not entirely sure how to fix it properly. From tss at iki.fi Mon Apr 23 15:05:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 15:05:58 +0300 Subject: [Dovecot] 2.1.3 Corrupted squat uidlist In-Reply-To: <9ca41dc4b98ee3c8d0e38f4c2f5d187d@larsson.as> References: <4F7164DC.7010706@unict.it> <4F8A97F8.70808@spambox.dk> <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> <96F0AD8C-9832-4CA5-B1DA-3936F9E2F0FB@iki.fi> <9ca41dc4b98ee3c8d0e38f4c2f5d187d@larsson.as> Message-ID: <3B05C40A-4FC7-479D-8EF3-F21C5DBEEA90@iki.fi> On 23.4.2012, at 14.56, Henrik Larsson wrote: >> So the doveadm search itself succeeds without errors, but if you >> immediately do another doveadm search it fails? Or do you need to do >> something else also in the middle? I couldn't reproduce this with a >> few tests. > > Is there any way to produce debug logs that will show what goes wrong. It is consistent in such a way that I will have several errors within a 24 hour period. > > I will be willing to add code and rebuild dovecot to produce the logs you need. Well, Squat is currently rather low priority to me .. I even considered removing it entirely, since I don't think it's very usable. fts-lucene would work much better. From l.chelchowski at eurocar.pl Mon Apr 23 15:27:08 2012 From: l.chelchowski at eurocar.pl (l.chelchowski at eurocar.pl) Date: Mon, 23 Apr 2012 14:27:08 +0200 Subject: [Dovecot] questions about SPECIAL-USE and virtual Message-ID: <4F954A9C.6000604@eurocar.pl> Hi, I have some questions about Special-Use and virtual namespace: I want to create a virtual namespace with mailboxes: Sent, Unread, Flagged, All. 1. Is it enough to put in the config file: mailbox virtual/Sent { auto = subscribe special_use = \Sent } mailbox virtual/Unread { auto = subscribe special_use = \Unread #Is it correct? } mailbox virtual/Flagged { auto = subscribe special_use = \Flagged } mailbox virtual/All { auto = subscribe special_use = \All } and it would displays mail from every namespace I have subscribed (Shared, Public)? OR I have to make also dovecot-virtual files? 2. In dovecot-virtual file can I put for mailbox virtual/Sent something like: * \Sent 3. Is the http://wiki2.dovecot.org/Plugins/Virtual full documented, if not , where can I find it? 4. Is there a documentation for Special-Use? -- ?ukasz From tss at iki.fi Mon Apr 23 15:33:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 15:33:05 +0300 Subject: [Dovecot] questions about SPECIAL-USE and virtual In-Reply-To: <4F954A9C.6000604@eurocar.pl> References: <4F954A9C.6000604@eurocar.pl> Message-ID: On 23.4.2012, at 15.27, l.chelchowski at eurocar.pl wrote: > I have some questions about Special-Use and virtual namespace: > > I want to create a virtual namespace with mailboxes: Sent, Unread, Flagged, All. > > 1. Is it enough to put in the config file: > mailbox virtual/Sent { > auto = subscribe > special_use = \Sent > } You have to put the mailbox {}es inside the namespace {} you want. Otherwise it just gives "unknown section" error. > OR I have to make also dovecot-virtual files? You have to do this also. The mailbox {} sections don't do anything magically different. They simply add the \Sent flag. The auto=subscribe can automatically subscribe the virtual mailbox, but it can't automatically create it, because new virtual mailboxes can't be created by IMAP clients. > 2. In dovecot-virtual file can I put for mailbox virtual/Sent something like: > * > \Sent No. I'm not sure what you'd want that to do. There is no \Sent message flag. > 3. Is the http://wiki2.dovecot.org/Plugins/Virtual full documented, if not , where can I find it? The only other documentation is the code. > 4. Is there a documentation for Special-Use? http://tools.ietf.org/html/rfc6154 From CMarcus at Media-Brokers.com Mon Apr 23 15:47:04 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 23 Apr 2012 08:47:04 -0400 Subject: [Dovecot] questions about SPECIAL-USE and virtual In-Reply-To: References: <4F954A9C.6000604@eurocar.pl> Message-ID: <4F954F48.7010409@Media-Brokers.com> On 2012-04-23 8:33 AM, Timo Sirainen wrote: > The auto=subscribe can automatically subscribe the virtual mailbox, > but it can't automatically create it, because new virtual mailboxes > can't be created by IMAP clients. Hmmm... Thunderbird has the ability to create virtual mailboxes too, but obviously they are created and handled locally, not on the server... How much trouble would it be to add an API so that an IMAP client *could* create and manage them on a dovecot server? And Maybe this would be a good candidate for an RFC spec? -- Best regards, Charles From tss at iki.fi Mon Apr 23 15:50:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 15:50:36 +0300 Subject: [Dovecot] questions about SPECIAL-USE and virtual In-Reply-To: <4F954F48.7010409@Media-Brokers.com> References: <4F954A9C.6000604@eurocar.pl> <4F954F48.7010409@Media-Brokers.com> Message-ID: <8D2C7878-B981-4DC9-8091-E0058BD361BA@iki.fi> On 23.4.2012, at 15.47, Charles Marcus wrote: > On 2012-04-23 8:33 AM, Timo Sirainen wrote: >> The auto=subscribe can automatically subscribe the virtual mailbox, >> but it can't automatically create it, because new virtual mailboxes >> can't be created by IMAP clients. > > Hmmm... > > Thunderbird has the ability to create virtual mailboxes too, but obviously they are created and handled locally, not on the server... > > How much trouble would it be to add an API so that an IMAP client *could* create and manage them on a dovecot server? And Maybe this would be a good candidate for an RFC spec? There have been a few drafts about virtual mailboxes, but they died at some point. I guess it could be implemented, but I'm less certain that there's actual need for it. From news1204.10.weezy at spamgourmet.com Mon Apr 23 16:30:39 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Mon, 23 Apr 2012 15:30:39 +0200 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: References: <2vp569xe2o.ln2@mail.ela-soft.com> <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> <4F953F9B.10307@spamgourmet.com> Message-ID: <4F95597F.2080908@spamgourmet.com> Am 23.04.2012 13:55, schrieb Timo Sirainen: > > See if this changes the behavior: > > service config { > unix_listener config { > mode = 0666 > } > } > > I'd guess that makes the vmail work also? Although it still doesn't explain the difference between asking config from config process vs. getting it by executing doveconf. Yes, with this entry sending as user vmail is working. Thanks. From news1204.10.weezy at spamgourmet.com Mon Apr 23 16:39:08 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Mon, 23 Apr 2012 15:39:08 +0200 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: <2F3EF8E1-2851-418A-8B43-EE0D91A321C8@iki.fi> References: <2vp569xe2o.ln2@mail.ela-soft.com> <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> <4F953F9B.10307@spamgourmet.com> <2F3EF8E1-2851-418A-8B43-EE0D91A321C8@iki.fi> Message-ID: <4F955B7C.2070009@spamgourmet.com> Am 23.04.2012 14:02, schrieb Timo Sirainen: > On 23.4.2012, at 14.40, Andreas Helmcke wrote: > >> namespace Allgemeines { > > Lowercase it: namespace allgemeines { > Did the trick, also. Sending as user vmail without the service config entry, which you recommended in the other mail, is working. From tss at iki.fi Mon Apr 23 16:49:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 16:49:12 +0300 Subject: [Dovecot] v2.1.5 released Message-ID: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz.sig A few announcements first: I'm now going to start being more strict about not answering Dovecot questions sent to me privately (especially support requests, but also bug reports, feature requests, etc). Often those questions could be answered by other people in the mailing list as well, so they just waste my time. Even if you're sure that I'm the only person who can possibly answer it, it may be something that others are interested of as well and replying privately denies them the possibility of reading about it. If you really do not want to send something publicly, you still have an option for private communication: just buy one of the Dovecot support packages. This is also the first Dovecot release that has run through automated release testing. I've been planning on doing that for last 5 years at least, and finally it's done. The testing includes running a lot of tests to verify that the IMAP server gives proper responses, stress testing doesn't break it, memory isn't leaked and several other things. The tests are run for many different kinds of configurations. Some bugs show up only for some specific combination of settings, but unfortunately because there's such a large number of them it's impossible to test every combination. So here's another advantage that Dovecot support customers have: they can have their specific configuration included in the automated tests. This release already fixes one bug I found only after testing a customer's config. Oh, and looks like I got accepted as a speaker to LinuxTag 2012. The plan is to talk about some new things in Dovecot and dsync replication (looks like I have to hurry up with the dsync rewrite). And finally, the most important changes since v2.1.4: * IMAP: When neither the session nor the mailbox has modseq tracking enabled, return the mailbox as having NOMODSEQ in SELECT/EXAMINE reply. Old versions in this situation always simply returned HIGHESTMODSEQ as 1, which could have broken some clients. + dict file: Added optional fcntl/flock locking (default is dotlock) + fts-solr: doveadm fts rescan now resets indexes, which allows reindexing mails. (This isn't a full rescan implementation like fts-lucene has.) + doveadm expunge: Added -d parameter to delete mailbox if it's empty after expunging. - IMAP: Several fixes related to mailbox listing in some configs - director: A lot of fixes and performance improvements - v2.1.4 didn't work without a mail home directory set - mbox: Deleting a mailbox didn't delete its index files. - pop3c: TOP command was sent incorrectly - trash plugin didn't work properly - LMTP: Don't add a duplicate Return-Path: header when proxying. - listescape: Don't unescape namespace prefixes. From stsiol at yahoo.co.uk Mon Apr 23 17:09:17 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 23 Apr 2012 15:09:17 +0100 (BST) Subject: [Dovecot] v2.1.5 released In-Reply-To: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <1335190157.12813.YahooMailNeo@web132202.mail.ird.yahoo.com> >Oh, and looks like I got accepted as a speaker to LinuxTag 2012. The plan is to talk about some new things in Dovecot and dsync replication (looks like I have to hurry up with the dsync rewrite). > I am quite amazed they haven't called you earlier. From what it seems you are doing a great job and dovecot is a rock-solid piece of software. Just my thoughts, s. ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From giles at coochey.net Mon Apr 23 17:12:50 2012 From: giles at coochey.net (Giles Coochey) Date: Mon, 23 Apr 2012 15:12:50 +0100 Subject: [Dovecot] v2.1.5 released In-Reply-To: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <4F956362.7010507@coochey.net> On 23/04/2012 14:49, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz > http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz.sig > > A few announcements first: > > I'm now going to start being more strict about not answering Dovecot questions sent to me privately (especially support requests, but also bug reports, feature requests, etc). Often those questions could be answered by other people in the mailing list as well, so they just waste my time. Even if you're sure that I'm the only person who can possibly answer it, it may be something that others are interested of as well and replying privately denies them the possibility of reading about it. If you really do not want to send something publicly, you still have an option for private communication: just buy one of the Dovecot support packages. > I'd just like to say thanks for the amount of time you put in not just developing the software, but supporting your userbase. I am sure everyone here is well aware of when Timo signs in each day as a flood of replies to queries on the dovecot mailing list starts to arrive to our mailboxes. It is very much appreciated, there are many OSS mailing-lists out there, but not many as vibrant as dovecots! -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4928 bytes Desc: S/MIME Cryptographic Signature URL: From l.chelchowski at eurocar.pl Mon Apr 23 17:27:54 2012 From: l.chelchowski at eurocar.pl (l.chelchowski at eurocar.pl) Date: Mon, 23 Apr 2012 16:27:54 +0200 Subject: [Dovecot] virtual plugin and shared namespace In-Reply-To: References: <4F4235B8.6090107@eurocar.pl> Message-ID: <4F9566EA.6040109@eurocar.pl> W dniu 2012-02-21 01:44, Timo Sirainen pisze: > On 20.2.2012, at 13.59, l.chelchowski at eurocar.pl wrote: > >> I have a problem with virtual plugin and geting mails form shared namespace. >> dovecot-virtual: >> ---- >> Shared/* >> Public/* >> * > .. >> When I enter the virtual mailbox Unread I get this debug info: >> virtual mailbox Virtual/Unread: Skipping non-existing mailbox Shared/test at domain.pl/ > * matches only within namespace. Shared/* matches only the entries directly under Shared/, i.e. Shared/test at domain.pl itself. Shared/test at domain.pl/* would again be a separate namespace. > >> Is it a bug or misconfig? > Maybe a "missing feature" :) I guess Shared/* should match all autocreated child namespaces.. This should help: > http://hg.dovecot.org/dovecot-2.1/rev/10994a5b64e1 > I have installed version 2.1.3 and this patch doesn't work as expected. I can't use Shared/*. Only Shared/test at domain.pl/* works. -- Pozdrawiam ?ukasz Che?chowski Informatyk Autoryzowany Dealer Ford Euro-Car Sp. z o.o. tel. 58 667 327 mailto: l.chelchowski at eurocar.pl -------------- next part -------------- A non-text attachment was scrubbed... Name: stopka2.gif Type: image/gif Size: 17309 bytes Desc: not available URL: From andrei at lctax.ro Mon Apr 23 19:17:35 2012 From: andrei at lctax.ro (Michescu Andrei) Date: Mon, 23 Apr 2012 12:17:35 -0400 Subject: [Dovecot] v2.1.5 released In-Reply-To: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <26fed0264da5829b3376056abd7dd1fd.squirrel@web.miau.ca> > Oh, and looks like I got accepted as a speaker to LinuxTag 2012. The plan > is to talk about some new things in Dovecot and dsync replication (looks > like I have to hurry up with the dsync rewrite). > Hey Timo, Thank you very much for your hard work ;) Please let us know if there is anything that we can help you clarify / design for the dsync rewrite project. Best regards, Andrei From cor at xs4all.nl Mon Apr 23 21:34:39 2012 From: cor at xs4all.nl (Cor Bosman) Date: Mon, 23 Apr 2012 20:34:39 +0200 Subject: [Dovecot] quota not being calculated In-Reply-To: <96E3D11C-8F71-4726-816A-6C448554BC70@iki.fi> References: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> <2D57E521-3DB3-41D9-B8AB-F280DE546061@iki.fi> <96E3D11C-8F71-4726-816A-6C448554BC70@iki.fi> Message-ID: > > http://hg.dovecot.org/dovecot-2.1/rev/4c8f79d1f9f1 should fix it with dict quota. Thank you, this fixed it with dict quota. Cor From dovecot at allycomm.com Mon Apr 23 22:56:13 2012 From: dovecot at allycomm.com (Jeff Kletsky) Date: Mon, 23 Apr 2012 12:56:13 -0700 Subject: [Dovecot] dsync on large mailbox "fails" Message-ID: <4F95B3DD.1070609@allycomm.com> I've been trying to convert an existing set of mbox mail to maildir using dsync, but it seems to fail when it gets to the large boxes. $ dsync -Dv mirror maildir:~/Maildir Everything seems to work reasonably well until the first of the large boxes hits: dsync(jeff): Warning: Maildir /home/jeff/Maildir/.cron.2010-07-10: Synchronization took 1210 seconds (102289 new msgs, 0 flag change attempts, 0 expunge attempts) dsync(jeff): Warning: Transaction log file /home/jeff/Maildir/.cron.2010-07-10/dovecot.index.log was locked for 1210 seconds After this, the doveadm process goes from ~75% of the CPU core utilization to 100% and stays there, even overnight. I've also tried $ dsync -Dv backup maildir:~/Maildir with similar results. While I did see some threads on dsync speed, they seemed to be minutes for a few GB of data, not hours. There is plenty of space on the drive, and, from what I understand, there is not a file-count limit on FreeBSD's ufs (at least not one that I'm anywhere near) Any suggestions? Thanks! Jeff [jeff at mail ~]$ du -h -d 0 ~/mail 3.9G /home/jeff/mail [jeff at mail ~]$ ls -lh /var/mail/jeff -rw------- 1 jeff jeff 554M Apr 23 11:27 /var/mail/jeff [jeff at mail ~]$ df -h Filesystem Size Used Avail Capacity Mounted on /dev/ada0s2g 38G 18G 16G 53% / (Note that the hostname and the location of the ssl certificate has been manually edited for this message in the following output) From dmesg.boot: Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-STABLE #0: Sat Apr 21 17:13:07 PDT 2012 root at my.host.name:/usr/obj/usr/src/sys/GENERIC i386 CPU: Intel(R) Atom(TM) CPU 330 @ 1.60GHz (1600.03-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x106c2 Family = 6 Model = 1c Stepping = 2 Features=0xbfe9fbff Features2=0x40e31d AMD Features=0x20100000 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 2147483648 (2048 MB) avail memory = 2077757440 (1981 MB) [jeff at mail ~]$ doveconf -n # 2.1.3: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 9.0-STABLE i386 auth_mechanisms = plain login disable_plaintext_auth = no first_valid_gid = 1000 first_valid_uid = 1000 listen = * mail_location = mbox:~/mail/:INBOX=/var/mail/%u mail_privileged_group = mail namespace { inbox = yes location = prefix = separator = / } passdb { args = session=yes dovecot driver = pam } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { mode = 0600 } user = root } service imap-login { inet_listener imap { port = 0 } process_limit = 200 } ssl = required ssl_cert = Hey all, has anyone ever tried turning on the trash plugin when the directory is already over quota? I see some messages being deleted, but it seems it just deletes enough to fit the new email, not enough to go below quota. Example: . getquotaroot "Spam" * QUOTAROOT "Spam" "User quota" "Spam Quota" * QUOTA "User quota" (STORAGE 591036 5242880) * QUOTA "Spam Quota" (STORAGE 21173 20480) I now move a 5K message into this folder, and I see: Apr 23 22:26:43 lmtp1 dovecot: imap(cor): expunge: box=Spam, uid=226337, msgid=<3YVKIIE4II6R0064BA7IM55C2Q4MD01VX12U1I at shedevils.com>, size=12984 Apr 23 22:26:43 lmtp1 dovecot: imap(cor): expunge: box=Spam, uid=226338, msgid=, size=5623 Apr 23 22:26:43 lmtp1 dovecot: imap(cor): save: box=Spam, uid=227759, msgid=<20120423191509.F3B584A355B at signal.groundspeak.com>, size=2694 And now: . getquotaroot "Spam" * QUOTAROOT "Spam" "User quota" "Spam Quota" * QUOTA "User quota" (STORAGE 591016 5242880) * QUOTA "Spam Quota" (STORAGE 21158 20480) . OK Getquotaroot completed. This seems like a possible bug. For one, it shouldnt be allowed to actually save a message into that folder, but it does. The email does end up in the Spam folder even though the folder is over quota. And of course it doesnt clear up enough space to have the new email fit, it just deleted enough to match the 5K. Without the trash plugin quota is being properly enforced. Everything works fine if you start out below the quota limit btw. Regards, Cor ps: this is 2.1.4, but with the recent trash and quota patches applied. From asai at globalchangemusic.org Tue Apr 24 00:50:36 2012 From: asai at globalchangemusic.org (Asai) Date: Mon, 23 Apr 2012 14:50:36 -0700 Subject: [Dovecot] v2.1.5 released In-Reply-To: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <4F95CEAC.10103@globalchangemusic.org> Let me also thank you, Timo, for your selfless excellence in developing Dovecot. --asai On 4/23/2012 6:49 AM, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz > http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz.sig > > A few announcements first: > > I'm now going to start being more strict about not answering Dovecot questions sent to me privately (especially support requests, but also bug reports, feature requests, etc). Often those questions could be answered by other people in the mailing list as well, so they just waste my time. Even if you're sure that I'm the only person who can possibly answer it, it may be something that others are interested of as well and replying privately denies them the possibility of reading about it. If you really do not want to send something publicly, you still have an option for private communication: just buy one of the Dovecot support packages. > > This is also the first Dovecot release that has run through automated release testing. I've been planning on doing that for last 5 years at least, and finally it's done. The testing includes running a lot of tests to verify that the IMAP server gives proper responses, stress testing doesn't break it, memory isn't leaked and several other things. The tests are run for many different kinds of configurations. > > Some bugs show up only for some specific combination of settings, but unfortunately because there's such a large number of them it's impossible to test every combination. So here's another advantage that Dovecot support customers have: they can have their specific configuration included in the automated tests. This release already fixes one bug I found only after testing a customer's config. > > Oh, and looks like I got accepted as a speaker to LinuxTag 2012. The plan is to talk about some new things in Dovecot and dsync replication (looks like I have to hurry up with the dsync rewrite). > > And finally, the most important changes since v2.1.4: > > * IMAP: When neither the session nor the mailbox has modseq tracking > enabled, return the mailbox as having NOMODSEQ in SELECT/EXAMINE > reply. Old versions in this situation always simply returned > HIGHESTMODSEQ as 1, which could have broken some clients. > > + dict file: Added optional fcntl/flock locking (default is dotlock) > + fts-solr: doveadm fts rescan now resets indexes, which allows > reindexing mails. (This isn't a full rescan implementation like > fts-lucene has.) > + doveadm expunge: Added -d parameter to delete mailbox if it's > empty after expunging. > - IMAP: Several fixes related to mailbox listing in some configs > - director: A lot of fixes and performance improvements > - v2.1.4 didn't work without a mail home directory set > - mbox: Deleting a mailbox didn't delete its index files. > - pop3c: TOP command was sent incorrectly > - trash plugin didn't work properly > - LMTP: Don't add a duplicate Return-Path: header when proxying. > - listescape: Don't unescape namespace prefixes. > From dovecot at allycomm.com Tue Apr 24 06:38:16 2012 From: dovecot at allycomm.com (Jeff Kletsky) Date: Mon, 23 Apr 2012 20:38:16 -0700 Subject: [Dovecot] dsync on large mailbox "fails" In-Reply-To: <4F95B3DD.1070609@allycomm.com> References: <4F95B3DD.1070609@allycomm.com> Message-ID: <4F962028.3060501@allycomm.com> On 4/23/12 12:56 PM, Jeff Kletsky wrote: > I've been trying to convert an existing set of mbox mail to maildir > using dsync, but it seems to fail when it gets to the large boxes. > > $ dsync -Dv mirror maildir:~/Maildir > > Everything seems to work reasonably well until the first of the large > boxes hits: > > dsync(jeff): Warning: Maildir /home/jeff/Maildir/.cron.2010-07-10: > Synchronization took 1210 seconds (102289 new msgs, 0 flag change > attempts, 0 expunge attempts) > dsync(jeff): Warning: Transaction log file > /home/jeff/Maildir/.cron.2010-07-10/dovecot.index.log was locked for > 1210 seconds > > After this, the doveadm process goes from ~75% of the CPU core > utilization to 100% and stays there, even overnight. > I believe that some of problem may be that there were a handful of boxes named with "." in them. These were not "flagged" in the conversion process in any way. http://wiki2.dovecot.org/Migration/MailFormat indicates that you can't have a "." in the name of a Maildir box (I dutifully stopped reading that page at "If you can successfully use dsync, you can skip the rest of this page.") In my case, it may be a contributing factor that there were boxes that ended up "stacking" inside each other: * cron * cron.2010-07-10 * cron.2010-07-10.remaining Still not blisteringly fast, but dsync(jeff): Warning: Maildir /home/jeff/Maildir/.cron_2010-07-10: Synchronization took 686 seconds (102289 new msgs, 0 flag change attempts, 0 expunge attempts) dsync(jeff): Warning: Transaction log file /home/jeff/Maildir/.cron_2010-07-10/dovecot.index.log was locked for 686 seconds is certainly better. I'm still seeing 100% CPU utilization, with the tail of ls -Altr in ~/Maildir showing drwx------ 5 jeff jeff 512 Apr 23 18:19 .XDA drwx------ 5 jeff jeff 512 Apr 23 18:19 .Spam drwx------ 5 jeff jeff 512 Apr 23 18:42 .Health drwx------ 5 jeff jeff 512 Apr 23 18:42 .cron_2010-07-10_remaining drwx------ 5 jeff jeff 512 Apr 23 18:58 .cron_2010-07-10 It appears from looking in the .cron_2010-07-10 directory that indexing is the action underway [jeff at mail ~/Maildir]$ ls -Altr .cron_2010-07-10 total 27 drwx------ 2 jeff jeff 512 Apr 23 18:15 new -rw------- 1 jeff jeff 0 Apr 23 18:15 maildirfolder drwx------ 2 jeff jeff 5819392 Apr 23 18:42 tmp drwx------ 2 jeff jeff 5237248 Apr 23 18:42 cur -rw------- 1 jeff jeff 4122971 Apr 23 18:42 dovecot-uidlist -rw------- 1 jeff jeff 36 Apr 23 18:42 dovecot-keywords -rw------- 1 jeff jeff 1650764 Apr 23 18:42 dovecot.index.log.2 -rw------- 1 jeff jeff 1637008 Apr 23 18:42 dovecot.index -rw------- 1 jeff jeff 56124 Apr 23 19:04 dovecot.index.log -rw------- 1 jeff jeff 9442304 Apr 23 19:04 dovecot.index.cache -rw------- 1 jeff jeff 24 Apr 23 19:04 dovecot-uidlist.lock (time passes) It is now 20:35 and it has not advanced. I now no longer believe the problem is the mailbox naming, but the indexing. PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 25934 jeff 1 103 0 135M 102M CPU2 2 128:47 100.00% doveadm [jeff at mail ~/Maildir]$ ls -Altr .cron_2010-07-10 total 28 drwx------ 2 jeff jeff 512 Apr 23 18:15 new -rw------- 1 jeff jeff 0 Apr 23 18:15 maildirfolder drwx------ 2 jeff jeff 5819392 Apr 23 18:42 tmp drwx------ 2 jeff jeff 5237248 Apr 23 18:42 cur -rw------- 1 jeff jeff 4122971 Apr 23 18:42 dovecot-uidlist -rw------- 1 jeff jeff 36 Apr 23 18:42 dovecot-keywords -rw------- 1 jeff jeff 1650764 Apr 23 18:42 dovecot.index.log.2 -rw------- 1 jeff jeff 1637008 Apr 23 20:26 dovecot.index -rw------- 1 jeff jeff 289120 Apr 23 20:36 dovecot.index.log -rw------- 1 jeff jeff 9442304 Apr 23 20:36 dovecot.index.cache -rw------- 1 jeff jeff 24 Apr 23 20:36 dovecot-uidlist.lock Any suggestions as to how to proceed? Jeff From robert at schetterer.org Tue Apr 24 09:56:26 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 24 Apr 2012 08:56:26 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works / solved by remove listescape In-Reply-To: References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> <4F90250E.3000200@schetterer.org> <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> <4F903DC1.5070501@schetterer.org> <4F905B2C.8040204@schetterer.org> <4F907529.2020403@schetterer.org> Message-ID: <4F964E9A.2050401@schetterer.org> Am 23.04.2012 13:12, schrieb Timo Sirainen: > On 19.4.2012, at 23.27, Robert Schetterer wrote: > >>> prefix = shared/%%u/ simply dont works >>> >>> i have reverted back to exact namespace config >>> of the 2.0.20 server, same result, so something broken in 2.1 >>> >>> >> Hi Timo >> problem solved by removing listescape plugin >> the acl paths did / instead of dot in domainname >> >> any chance to get listescape fixed ? > > Ah, that makes it easy to fix: http://hg.dovecot.org/dovecot-2.1/rev/63af3274fb6f > Hi Timo, for small tests problem seems to be fixed, thx youre great -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Tue Apr 24 10:50:06 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 24 Apr 2012 09:50:06 +0200 Subject: [Dovecot] doveadm delete folders ? / unsubscribe ? In-Reply-To: <4F906EE8.1030900@schetterer.org> References: <4F900066.9030808@schetterer.org> <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> <4F906EE8.1030900@schetterer.org> Message-ID: <4F965B2E.2010103@schetterer.org> Am 19.04.2012 22:00, schrieb Robert Schetterer: > Am 19.04.2012 17:16, schrieb Timo Sirainen: >> On 19.4.2012, at 15.09, Robert Schetterer wrote: >> >>> Hi , >>> is there a feature like >>> doveadm expunge savedbefore ... >>> for imap folder delete ? >> >> Here: >> >> http://hg.dovecot.org/dovecot-2.1/rev/2d8bafd11569 >> http://hg.dovecot.org/dovecot-2.1/rev/782570f644f7 >> > > thx Timo ! > Hi Timo doveadm expunge -d -u *@domain.example mailbox exampelfolder\* savedbefore 10d works like charm, but the folders do not get unsubscribed any chance to have i.e another add parameter ,or do it auto cause empty and deleted folders should not stay subscribed in my eyes -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From lists at wiesinger.com Tue Apr 24 11:59:20 2012 From: lists at wiesinger.com (Gerhard Wiesinger) Date: Tue, 24 Apr 2012 10:59:20 +0200 (CEST) Subject: [Dovecot] v2.1.5 released In-Reply-To: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: Hello Timo, Dovecot 2.1.5 seems to be broken. After upgrade from 2.1.3 + patch http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e I only see Inbox and Trash folder. Reverting => OK. Same config as before and disucssed per private E-mail and on the list. Guessing this comes again from: mail_full_filesystem_access=yes Ciao, Gerhard -- http://www.wiesinger.com/ On Mon, 23 Apr 2012, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz > http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz.sig > > A few announcements first: > > I'm now going to start being more strict about not answering Dovecot questions sent to me privately (especially support requests, but also bug reports, feature requests, etc). Often those questions could be answered by other people in the mailing list as well, so they just waste my time. Even if you're sure that I'm the only person who can possibly answer it, it may be something that others are interested of as well and replying privately denies them the possibility of reading about it. If you really do not want to send something publicly, you still have an option for private communication: just buy one of the Dovecot support packages. > > This is also the first Dovecot release that has run through automated release testing. I've been planning on doing that for last 5 years at least, and finally it's done. The testing includes running a lot of tests to verify that the IMAP server gives proper responses, stress testing doesn't break it, memory isn't leaked and several other things. The tests are run for many different kinds of configurations. > > Some bugs show up only for some specific combination of settings, but unfortunately because there's such a large number of them it's impossible to test every combination. So here's another advantage that Dovecot support customers have: they can have their specific configuration included in the automated tests. This release already fixes one bug I found only after testing a customer's config. > > Oh, and looks like I got accepted as a speaker to LinuxTag 2012. The plan is to talk about some new things in Dovecot and dsync replication (looks like I have to hurry up with the dsync rewrite). > > And finally, the most important changes since v2.1.4: > > * IMAP: When neither the session nor the mailbox has modseq tracking > enabled, return the mailbox as having NOMODSEQ in SELECT/EXAMINE > reply. Old versions in this situation always simply returned > HIGHESTMODSEQ as 1, which could have broken some clients. > > + dict file: Added optional fcntl/flock locking (default is dotlock) > + fts-solr: doveadm fts rescan now resets indexes, which allows > reindexing mails. (This isn't a full rescan implementation like > fts-lucene has.) > + doveadm expunge: Added -d parameter to delete mailbox if it's > empty after expunging. > - IMAP: Several fixes related to mailbox listing in some configs > - director: A lot of fixes and performance improvements > - v2.1.4 didn't work without a mail home directory set > - mbox: Deleting a mailbox didn't delete its index files. > - pop3c: TOP command was sent incorrectly > - trash plugin didn't work properly > - LMTP: Don't add a duplicate Return-Path: header when proxying. > - listescape: Don't unescape namespace prefixes. > > > From alessio at skye.it Tue Apr 24 12:15:08 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 24 Apr 2012 11:15:08 +0200 Subject: [Dovecot] v2.1.5 released In-Reply-To: References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <4F966F1C.2080204@skye.it> Il 24/04/2012 10:59, Gerhard Wiesinger ha scritto: > Hello Timo, > > Dovecot 2.1.5 seems to be broken. After upgrade from > 2.1.3 + patch http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e > I only see Inbox and Trash folder. > > Reverting => OK. > > Same config as before and disucssed per private E-mail and on the list. > Guessing this comes again from: > mail_full_filesystem_access=yes > > Ciao, > Gerhard Hi Gerhard, for us 2.1.5 works fine. We are using Maildir as mail storage type. Ciao -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From joseba.torre at ehu.es Tue Apr 24 12:25:38 2012 From: joseba.torre at ehu.es (Joseba Torre) Date: Tue, 24 Apr 2012 11:25:38 +0200 Subject: [Dovecot] Error compiling pigeonhole 0.3.0 with dovecot-2.1.5 in openbsd 5.0 Message-ID: <4F967192.2060701@ehu.es> Hi, I've just tried to compile dovecot & pigeonhole in my openbsd box, and got this: gcc -DHAVE_CONFIG_H -I. -I../.. -I/home/seajob/build/dovecot-2.1.5 -I/home/seajob/build/dovecot-2.1.5/src/lib -I/home/seajob/build/dovecot-2.1.5/src/lib-dict -I/home/seajob/build/dovecot-2.1.5/src/lib-dns -I/home/seajob/build/dovecot-2.1.5/src/lib-mail -I/home/seajob/build/dovecot-2.1.5/src/lib-imap -I/home/seajob/build/dovecot-2.1.5/src/lib-fs -I/home/seajob/build/dovecot-2.1.5/src/lib-charset -I/home/seajob/build/dovecot-2.1.5/src/lib-index -I/home/seajob/build/dovecot-2.1.5/src/lib-storage -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index/raw -I../.. -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/include -MT managesieve-quote.o -MD -MP -MF .deps/managesieve-quote.Tpo -c -o managesieve-quote.o managesieve-quote.c mv -f .deps/managesieve-quote.Tpo .deps/managesieve-quote.Po gcc -DHAVE_CONFIG_H -I. -I../.. -I/home/seajob/build/dovecot-2.1.5 -I/home/seajob/build/dovecot-2.1.5/src/lib -I/home/seajob/build/dovecot-2.1.5/src/lib-dict -I/home/seajob/build/dovecot-2.1.5/src/lib-dns -I/home/seajob/build/dovecot-2.1.5/src/lib-mail -I/home/seajob/build/dovecot-2.1.5/src/lib-imap -I/home/seajob/build/dovecot-2.1.5/src/lib-fs -I/home/seajob/build/dovecot-2.1.5/src/lib-charset -I/home/seajob/build/dovecot-2.1.5/src/lib-index -I/home/seajob/build/dovecot-2.1.5/src/lib-storage -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index/raw -I../.. -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/include -MT managesieve-parser.o -MD -MP -MF .deps/managesieve-parser.Tpo -c -o managesieve-parser.o managesieve-parser.c managesieve-parser.c: In function 'quoted_string_istream_read': managesieve-parser.c:639: error: 'EPROTO' undeclared (first use in this function) managesieve-parser.c:639: error: (Each undeclared identifier is reported only once managesieve-parser.c:639: error: for each function it appears in.) *** Error code 1 Stop in /home/seajob/build/dovecot-2.1-pigeonhole-0.3.0/src/lib-managesieve (line 92 of /usr/share/mk/sys.mk). *** Error code 1 Stop in /home/seajob/build/dovecot-2.1-pigeonhole-0.3.0/src (line 289 of Makefile). *** Error code 1 Stop in /home/seajob/build/dovecot-2.1-pigeonhole-0.3.0 (line 474 of Makefile). *** Error code 1 Stop in /home/seajob/build/dovecot-2.1-pigeonhole-0.3.0 (line 353 of Makefile). From piotr.maier at gmail.com Tue Apr 24 12:51:43 2012 From: piotr.maier at gmail.com (Piotr Maier) Date: Tue, 24 Apr 2012 11:51:43 +0200 Subject: [Dovecot] Dovecot Virtual Users with Users Domain Mapping Message-ID: I have successfully compiled, configured and ran Dovecot with virtual users feature. Here's part of my /etc/dovecot.conf configuration file: mail_location = maildir:~/Maildir auth default { mechanisms = plain login userdb passwd-file { args = /home/%d/etc/passwd } passdb passwd-file { args = /home/%d/etc/shadow } socket listen { master { path = /var/run/dovecot/auth-worker mode = 0600 } } } I faced one issue I can't resolve myself. Is there anyway to create users' domains mapping and provide username in mail_location? Example of usersdomains file: domain.com:user1 domain2.org:user3 Examples: 1. currently I have /home/domain.com/user/Maildir 2. I'd like to have /home/SYSTEM_USER/domain.com/user/Maildir Can I achieve this somehow? From tss at iki.fi Tue Apr 24 13:26:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 24 Apr 2012 13:26:32 +0300 Subject: [Dovecot] dsync on large mailbox "fails" In-Reply-To: <4F962028.3060501@allycomm.com> References: <4F95B3DD.1070609@allycomm.com> <4F962028.3060501@allycomm.com> Message-ID: <4EE7C62E-928C-499D-BE4E-9944D6013198@iki.fi> On 24.4.2012, at 6.38, Jeff Kletsky wrote: > 25934 jeff 1 103 0 135M 102M CPU2 2 128:47 100.00% doveadm Looks like it could be in an infinite loop. To find out where: 1) strace for a few seconds: strace -tt -p 25934 2>log 2) get gdb backtrace a couple of times (2-5 should be enough, preferably slightly different ones): gdb /usr/local/bin/doveadm 25934 bt full cont bt full cont ..etc.. From markus at mpetri.org Tue Apr 24 14:00:05 2012 From: markus at mpetri.org (Markus Petri) Date: Tue, 24 Apr 2012 13:00:05 +0200 Subject: [Dovecot] Permission problems when trying to create new folders with ACL plugin Message-ID: <20120424130005.0db3d425@legolas.home.ceotex.de> Hi, I have the a problem with shared folders and the ACL plugin. The user "markus" has full access to "Shared/shared-user/Data": 3 GETACL "Shared/shared-user/Data" * ACL "Shared/shared-user/Data" "markus" akxeilprwtscd 3 OK Getacl completed. but creating a new subfolder under "Data" fails: 4 CREATE "Shared/shared-user/Data/New1" 4 NO [NOPERM] Permission denied This used to work with the same configuration and Dovecot 2.0. Cheers, Markus dovecot -n output: # 2.1.5: /opt/dovecot-2.1/etc/dovecot/dovecot.conf # OS: Linux 3.2.0-2-amd64 x86_64 Debian wheezy/sid auth_mechanisms = plain login disable_plaintext_auth = no listen = 192.168.56.11 mail_location = maildir:~/Maildir mail_plugins = acl managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = prefix = separator = / subscriptions = yes type = private } namespace { inbox = no list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/index/shared/%%u prefix = Shared/%%u/ separator = / subscriptions = no type = shared } passdb { args = /opt/dovecot-2.1/etc/dovecot/passwd driver = passwd-file } plugin { acl = vfile acl_anyone = allow acl_shared_dict = file:/var/lib/vdovecot/shared-mailboxes.db } protocols = imap service auth { unix_listener auth-userdb { mode = 0600 user = vdovecot } } ssl = no userdb { args = /opt/dovecot-2.1/etc/dovecot/passwd driver = passwd-file } verbose_proctitle = yes protocol imap { mail_plugins = acl imap_acl } From stephan at rename-it.nl Tue Apr 24 14:13:54 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 24 Apr 2012 13:13:54 +0200 Subject: [Dovecot] Error compiling pigeonhole 0.3.0 with dovecot-2.1.5 in openbsd 5.0 In-Reply-To: <4F967192.2060701@ehu.es> References: <4F967192.2060701@ehu.es> Message-ID: <4F968AF2.4090807@rename-it.nl> Op 4/24/2012 11:25 AM, Joseba Torre schreef: > Hi, > > I've just tried to compile dovecot & pigeonhole in my openbsd box, and > got this: > > gcc -DHAVE_CONFIG_H -I. -I../.. -I/home/seajob/build/dovecot-2.1.5 > -I/home/seajob/build/dovecot-2.1.5/src/lib > -I/home/seajob/build/dovecot-2.1.5/src/lib-dict > -I/home/seajob/build/dovecot-2.1.5/src/lib-dns > -I/home/seajob/build/dovecot-2.1.5/src/lib-mail > -I/home/seajob/build/dovecot-2.1.5/src/lib-imap > -I/home/seajob/build/dovecot-2.1.5/src/lib-fs > -I/home/seajob/build/dovecot-2.1.5/src/lib-charset > -I/home/seajob/build/dovecot-2.1.5/src/lib-index > -I/home/seajob/build/dovecot-2.1.5/src/lib-storage > -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index > -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index/raw -I../.. > -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes > -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 > -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 > -I/usr/include -MT managesieve-quote.o -MD -MP -MF > .deps/managesieve-quote.Tpo -c -o managesieve-quote.o managesieve-quote.c > > mv -f .deps/managesieve-quote.Tpo .deps/managesieve-quote.Po > gcc -DHAVE_CONFIG_H -I. -I../.. -I/home/seajob/build/dovecot-2.1.5 > -I/home/seajob/build/dovecot-2.1.5/src/lib > -I/home/seajob/build/dovecot-2.1.5/src/lib-dict > -I/home/seajob/build/dovecot-2.1.5/src/lib-dns > -I/home/seajob/build/dovecot-2.1.5/src/lib-mail > -I/home/seajob/build/dovecot-2.1.5/src/lib-imap > -I/home/seajob/build/dovecot-2.1.5/src/lib-fs > -I/home/seajob/build/dovecot-2.1.5/src/lib-charset > -I/home/seajob/build/dovecot-2.1.5/src/lib-index > -I/home/seajob/build/dovecot-2.1.5/src/lib-storage > -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index > -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index/raw -I../.. > -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes > -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 > -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 > -I/usr/include -MT managesieve-parser.o -MD -MP -MF > .deps/managesieve-parser.Tpo -c -o managesieve-parser.o > managesieve-parser.c > managesieve-parser.c: In function 'quoted_string_istream_read': > managesieve-parser.c:639: error: 'EPROTO' undeclared (first use in > this function) > managesieve-parser.c:639: error: (Each undeclared identifier is > reported only once > managesieve-parser.c:639: error: for each function it appears in.) Looks like BSD doesn't know EPROTO (although it is POSIX). Dovecot uses EIO elsewhere for such stream errors, so I adopted that for ManageSieve as well. This should fix it: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/ef5045159e6b Regards, Stephan. From ah-news-1204 at freenet.de Tue Apr 24 17:09:58 2012 From: ah-news-1204 at freenet.de (Andreas Helmcke) Date: Tue, 24 Apr 2012 16:09:58 +0200 Subject: [Dovecot] acls not copied when creating subfolder of private INBOX Message-ID: <4F96B436.9090402@freenet.de> Using brand new dovecot 2.1.5 I still have a problem with the inheritance of acls. Setting: - maildir with private INBOX and public folders. - users maildir directory (/home/mail/user/ahelmcke) contains dovecot-acl file - public folders root directory (/home/mail/Fax) contains dovecot-acl file When creating a subfolder of the public folder the dovecot-acl files gets copied to the subfolders directory as expected. When creating a subfolder of the users INBOX the dovecot-acl files does /not/ get copied to the subfolders directory. Debug output: doveadm -Dv mailbox create -u ahelmcke Fax/Buhhhh doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib01_acl_plugin.so doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib10_quota_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_lookup (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_zlib_plugin.so: undefined symbol: i_stream_create_deflate (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_list_backend (this is usually intentional, so just ignore this message) doveadm(ahelmcke): Debug: auth input: ahelmcke quota_rule=*:storage=2000M doveadm(ahelmcke): Debug: Added userdb setting: plugin/quota_rule=*:storage=2000M doveadm(ahelmcke): Debug: Effective uid=494, gid=491, home=/home/mail/user/ahelmcke doveadm(ahelmcke): Debug: Quota root: name=User quota backend=maildir args= doveadm(ahelmcke): Debug: Quota rule: root=User quota mailbox=* bytes=2097152000 messages=0 doveadm(ahelmcke): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke doveadm(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=/home/mail/user/ahelmcke, alt= doveadm(ahelmcke): Debug: acl: initializing backend with data: vfile doveadm(ahelmcke): Debug: acl: acl username = ahelmcke doveadm(ahelmcke): Debug: acl: owner = 1 doveadm(ahelmcke): Debug: acl vfile: Global ACL directory: (none) doveadm(ahelmcke): Debug: Namespace dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ doveadm(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke/public, index=, control=, inbox=, alt= doveadm(ahelmcke): Debug: acl: initializing backend with data: vfile doveadm(ahelmcke): Debug: acl: acl username = ahelmcke doveadm(ahelmcke): Debug: acl: owner = 1 doveadm(ahelmcke): Debug: acl vfile: Global ACL directory: (none) doveadm(ahelmcke): Debug: Namespace fax: type=public, prefix=Fax/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no location=maildir:/home/mail/Fax:INDEX=/home/mail/user/ahelmcke/SEEN.Fax doveadm(ahelmcke): Debug: maildir++: root=/home/mail/Fax, index=/home/mail/user/ahelmcke/SEEN.Fax, control=, inbox=, alt= doveadm(ahelmcke): Debug: acl: initializing backend with data: vfile doveadm(ahelmcke): Debug: acl: acl username = ahelmcke doveadm(ahelmcke): Debug: acl: owner = 0 doveadm(ahelmcke): Debug: acl vfile: Global ACL directory: (none) doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/dovecot-acl doveadm(ahelmcke): Debug: Namespace Fax/: Using permissions from /home/mail/Fax: mode=0700 gid=-1 doveadm(ahelmcke): Debug: Namespace INBOX/: Using permissions from /home/mail/user/ahelmcke: mode=0700 gid=-1 doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/user/ahelmcke/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/user/ahelmcke/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/user/ahelmcke/dovecot-acl doveadm(ahelmcke): Debug: Namespace Fax/: /home/mail/Fax/.Buhhhh doesn't exist yet, using default permissions doveadm(ahelmcke): Debug: Namespace Fax/: Using permissions from /home/mail/Fax: mode=0700 gid=-1 doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: file /home/mail/Fax/.Buhhhh/dovecot-acl not found doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/.Buhhhh/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/user/ahelmcke/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/.Buhhhh/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/.Buhhhh/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/user/ahelmcke/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/.Buhhhh/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/.Buhhhh/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/user/ahelmcke/dovecot-acl doveadm -Dv mailbox create -u ahelmcke INBOX/Buhhhh doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib01_acl_plugin.so doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib10_quota_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_lookup (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_zlib_plugin.so: undefined symbol: i_stream_create_deflate (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_list_backend (this is usually intentional, so just ignore this message) doveadm(ahelmcke): Debug: auth input: ahelmcke quota_rule=*:storage=2000M doveadm(ahelmcke): Debug: Added userdb setting: plugin/quota_rule=*:storage=2000M doveadm(ahelmcke): Debug: Effective uid=494, gid=491, home=/home/mail/user/ahelmcke doveadm(ahelmcke): Debug: Quota root: name=User quota backend=maildir args= doveadm(ahelmcke): Debug: Quota rule: root=User quota mailbox=* bytes=2097152000 messages=0 doveadm(ahelmcke): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke doveadm(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=/home/mail/user/ahelmcke, alt= doveadm(ahelmcke): Debug: acl: initializing backend with data: vfile doveadm(ahelmcke): Debug: acl: acl username = ahelmcke doveadm(ahelmcke): Debug: acl: owner = 1 doveadm(ahelmcke): Debug: acl vfile: Global ACL directory: (none) doveadm(ahelmcke): Debug: Namespace dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ doveadm(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke/public, index=, control=, inbox=, alt= doveadm(ahelmcke): Debug: acl: initializing backend with data: vfile doveadm(ahelmcke): Debug: acl: acl username = ahelmcke doveadm(ahelmcke): Debug: acl: owner = 1 doveadm(ahelmcke): Debug: acl vfile: Global ACL directory: (none) doveadm(ahelmcke): Debug: Namespace fax: type=public, prefix=Fax/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no location=maildir:/home/mail/Fax:INDEX=/home/mail/user/ahelmcke/SEEN.Fax doveadm(ahelmcke): Debug: maildir++: root=/home/mail/Fax, index=/home/mail/user/ahelmcke/SEEN.Fax, control=, inbox=, alt= doveadm(ahelmcke): Debug: acl: initializing backend with data: vfile doveadm(ahelmcke): Debug: acl: acl username = ahelmcke doveadm(ahelmcke): Debug: acl: owner = 0 doveadm(ahelmcke): Debug: acl vfile: Global ACL directory: (none) doveadm(ahelmcke): Debug: Namespace INBOX/: /home/mail/user/ahelmcke/.Buhhhh doesn't exist yet, using default permissions doveadm(ahelmcke): Debug: Namespace INBOX/: Using permissions from /home/mail/user/ahelmcke: mode=0700 gid=-1 doveconf -n # 2.1.5: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) ext4 auth_master_user_separator = * auth_verbose = yes first_valid_uid = 400 mail_debug = yes mail_gid = vmail mail_home = /home/mail/user/%u mail_location = maildir:/home/mail/user/%u mail_plugins = " quota acl" mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl namespace dummy { alias_for = INBOX/ hidden = yes list = no location = maildir:/home/mail/user/%u/public/ prefix = separator = / type = private } namespace fax { location = maildir:/home/mail/Fax:INDEX=/home/mail/user/%u/SEEN.Fax prefix = Fax/ separator = / subscriptions = no type = public } namespace inbox { inbox = yes location = prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile acl_shared_dict = file:/home/mail/shared-mailboxes quota = maildir:User quota quota_rule = *:storage=3G sieve = /home/mail/user/%u/sieve.active sieve_dir = /home/mail/user/%u/sieve.scripts } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = * } inet_listener imaps { address = * } } service managesieve-login { inet_listener sieve { address = * port = 4190 } } ssl = required ssl_cert = References: <4F96B436.9090402@freenet.de> Message-ID: <4F96BE96.9080505@schetterer.org> Am 24.04.2012 16:09, schrieb Andreas Helmcke: > Setting: > - maildir with private INBOX and public folders. > - users maildir directory (/home/mail/user/ahelmcke) contains dovecot-acl file > - public folders root directory (/home/mail/Fax) contains dovecot-acl file > > When creating a subfolder of the public folder the dovecot-acl files gets copied to the subfolders directory as expected. > When creating a subfolder of the users INBOX the dovecot-acl files does /not/ get copied to the subfolders directory. i am not sure ,but but i think that works like designed http://wiki2.dovecot.org/ACL .. ACL Inheritance Every time you create a new mailbox, it gets its ACLs from the parent mailbox. If you're creating a root-level mailbox, it uses the namespace's default ACLs. There is no actual inheritance, however: If you modify parent's ACLs, the child's ACLs stay the same. There is currently no support for ACL inheritance. Namespace's default ACLs are read from "dovecot-acl" file in the namespace's mail root directory (e.g. /var/public/Maildir). Note that currently these default ACLs are used only when creating new mailboxes, they aren't used for mailboxes without ACLs. .... -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From ah-news-1204 at freenet.de Tue Apr 24 18:17:39 2012 From: ah-news-1204 at freenet.de (Andreas Helmcke) Date: Tue, 24 Apr 2012 17:17:39 +0200 Subject: [Dovecot] acls not copied when creating subfolder of private INBOX In-Reply-To: <4F96BE96.9080505@schetterer.org> References: <4F96B436.9090402@freenet.de> <4F96BE96.9080505@schetterer.org> Message-ID: <4F96C413.7080709@freenet.de> Am 24.04.2012 16:54, schrieb Robert Schetterer: > Am 24.04.2012 16:09, schrieb Andreas Helmcke: >> When creating a subfolder of the public folder the dovecot-acl files gets copied to the subfolders directory as expected. >> When creating a subfolder of the users INBOX the dovecot-acl files does /not/ get copied to the subfolders directory. > > i am not sure ,but but i think that works like designed > > http://wiki2.dovecot.org/ACL > .. > ACL Inheritance > .... I do read this as: when creating a new mailbox; acls are copied from parent. This it what it does for public folders but /not/ for the private ones. So in my opinion at least it is not working as documented. From robert at schetterer.org Tue Apr 24 18:31:08 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 24 Apr 2012 17:31:08 +0200 Subject: [Dovecot] acls not copied when creating subfolder of private INBOX In-Reply-To: <4F96C413.7080709@freenet.de> References: <4F96B436.9090402@freenet.de> <4F96BE96.9080505@schetterer.org> <4F96C413.7080709@freenet.de> Message-ID: <4F96C73C.5010202@schetterer.org> Am 24.04.2012 17:17, schrieb Andreas Helmcke: > Am 24.04.2012 16:54, schrieb Robert Schetterer: >> Am 24.04.2012 16:09, schrieb Andreas Helmcke: >>> When creating a subfolder of the public folder the dovecot-acl files gets copied to the subfolders directory as expected. >>> When creating a subfolder of the users INBOX the dovecot-acl files does /not/ get copied to the subfolders directory. >> >> i am not sure ,but but i think that works like designed >> >> http://wiki2.dovecot.org/ACL >> .. >> ACL Inheritance >> .... > > I do read this as: when creating a new mailbox; acls are copied from parent. > This it what it does for public folders but /not/ for the private ones. So in my opinion at least it is not working as > documented. i think its missunderstandable written this is what counts i think http://wiki2.dovecot.org/ACL ... "There is currently no support for ACL inheritance" .. public folders are special cases.... it makes sense to have inheritance as default acl but this may not be a good idea as default at users folders read the list archive about this stuff, i cant remember all but Timo wrote some about it, at last ,develop of that feature may be still be not finshed -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From l.rame at griensu.com Tue Apr 24 19:07:12 2012 From: l.rame at griensu.com (Leonardo =?iso-8859-1?Q?M=2E_Ram=E9?=) Date: Tue, 24 Apr 2012 13:07:12 -0300 Subject: [Dovecot] Reading Mutt mboxes from Thunderbird Message-ID: <20120424160712.GB5048@leonardo-laptop> Hi, I'm a user of Mutt, and before moving to Thunderbird I'm trying to share mboxes between them. To do this, I installed Dovecot to create an IMAP server in my local machine, to serve mutt mboxes to be read by Thunderbird. So far, I can read my inbox from /var/mail/%u, but in mutt, I have many folders in ~/mail, like 2010-mails, 2011-mails, mailing-list-x, mailing-list-y, and so on. How can I configure Dovecot to read them?. In /etc/dovecot/conf.d/10-mail.conf I have this: mail_location = mbox:~/Mail:INBOX=/var/mail/%u I'm using Ubuntu 11.10, and Dovecot 2.0.13. Regards, -- Leonardo M. Ram? Medical IT - Griensu S.A. Av. Col?n 636 - Piso 8 Of. A X5000EPT -- C?rdoba Tel.: +54(351)4246924 +54(351)4247788 +54(351)4247979 int. 19 Cel.: +54(351)156629292 From interfasys at gmail.com Tue Apr 24 19:31:38 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Tue, 24 Apr 2012 17:31:38 +0100 Subject: [Dovecot] sievec segfaults Message-ID: <4F96D56A.60907@gmail.com> Hello, sievec segfaults when it reads a file. I've tried with both the standard and gold linker. I've tried with various sieve files and they all make it crash. I've compiled pigeonhole with -ggdb, but I'm not getting much info Program received signal SIGSEGV, Segmentation fault. 0x0000000800b8e587 in strlen () from /lib/libc.so.7 (gdb) bt full #0 0x0000000800b8e587 in strlen () from /lib/libc.so.7 No symbol table info available. #1 0x0000000800a12e8c in str_append () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #2 0x000000080077f02d in sieve_binary_save () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #3 0x0000000000605bf0 in environ () No symbol table info available. #4 0x0000000000000000 in ?? () No symbol table info available. #5 0x00007fffffffeb6e in ?? () No symbol table info available. #6 0x0000000000000001 in ?? () No symbol table info available. #7 0x0000000000000000 in ?? () No symbol table info available. #8 0x000000080060878d in ?? () from /libexec/ld-elf.so.1 No symbol table info available. #9 0x0000000000605cd9 in ?? () No symbol table info available. #10 0x0000000801000ea8 in ?? () No symbol table info available. #11 0x0000000000005008 in ?? () No symbol table info available. #12 0x0000000000000000 in ?? () No symbol table info available. #13 0x0000000801099040 in ?? () No symbol table info available. #14 0x0000000000000000 in ?? () No symbol table info available. #15 0x0000000000000180 in ?? () No symbol table info available. #16 0x0000000000605bf0 in environ () No symbol table info available. #17 0x0000000000000000 in ?? () No symbol table info available. #18 0x00007fffffffeb6e in ?? () No symbol table info available. #19 0x0000000000000001 in ?? () No symbol table info available. #20 0x0000000000000000 in ?? () No symbol table info available. #21 0x0000000000000000 in ?? () No symbol table info available. #22 0x0000000000402f93 in main () No symbol table info available. Cheers, Olivier From dovecot at allycomm.com Tue Apr 24 20:05:07 2012 From: dovecot at allycomm.com (Jeff Kletsky) Date: Tue, 24 Apr 2012 10:05:07 -0700 Subject: [Dovecot] dsync on large mailbox "fails" In-Reply-To: <4EE7C62E-928C-499D-BE4E-9944D6013198@iki.fi> References: <4F95B3DD.1070609@allycomm.com> <4F962028.3060501@allycomm.com> <4EE7C62E-928C-499D-BE4E-9944D6013198@iki.fi> Message-ID: <4F96DD43.3070700@allycomm.com> On 4/24/12 3:26 AM, Timo Sirainen wrote: > On 24.4.2012, at 6.38, Jeff Kletsky wrote: > >> 25934 jeff 1 103 0 135M 102M CPU2 2 128:47 100.00% doveadm > Looks like it could be in an infinite loop. To find out where: > > 1) strace for a few seconds: > > strace -tt -p 25934 2>log > > 2) get gdb backtrace a couple of times (2-5 should be enough, preferably slightly different ones): > > gdb /usr/local/bin/doveadm 25934 > bt full > > cont > > bt full > > cont > > ..etc.. Rather than clutter everyone's inbox, even if we know they have a great IMAP server... http://wildside.wagsky.com/dovecot/2012-04-24/54097/doveadm.gdb.log.bz2 http://wildside.wagsky.com/dovecot/2012-04-24/54097/doveadm.truss.log.bz2 Thanks for looking into this! Jeff From tss at iki.fi Tue Apr 24 20:08:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 24 Apr 2012 20:08:38 +0300 Subject: [Dovecot] dsync on large mailbox "fails" In-Reply-To: <4F96DD43.3070700@allycomm.com> References: <4F95B3DD.1070609@allycomm.com> <4F962028.3060501@allycomm.com> <4EE7C62E-928C-499D-BE4E-9944D6013198@iki.fi> <4F96DD43.3070700@allycomm.com> Message-ID: <78F42D61-1402-408B-953B-F85254479761@iki.fi> On 24.4.2012, at 20.05, Jeff Kletsky wrote: > On 4/24/12 3:26 AM, Timo Sirainen wrote: >> On 24.4.2012, at 6.38, Jeff Kletsky wrote: >> >>> 25934 jeff 1 103 0 135M 102M CPU2 2 128:47 100.00% doveadm >> Looks like it could be in an infinite loop. To find out where: > http://wildside.wagsky.com/dovecot/2012-04-24/54097/doveadm.truss.log.bz2 That's a lot of getdirentries() calls in there, repeating all the time. So I don't think it's an infinite loop, just a lot of slowness repeating the same work over and over again. See if it helps if you add: maildir_very_dirty_syncs = yes From stephan at rename-it.nl Tue Apr 24 20:10:21 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 24 Apr 2012 19:10:21 +0200 Subject: [Dovecot] sievec segfaults In-Reply-To: <4F96D56A.60907@gmail.com> References: <4F96D56A.60907@gmail.com> Message-ID: <4F96DE7D.6030807@rename-it.nl> Op 4/24/2012 6:31 PM, interfaSys s?rl schreef: > Hello, > sievec segfaults when it reads a file. > I've tried with both the standard and gold linker. > I've tried with various sieve files and they all make it crash. What version? Regards, Stephan. From interfasys at gmail.com Tue Apr 24 20:14:13 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Tue, 24 Apr 2012 18:14:13 +0100 Subject: [Dovecot] sievec segfaults In-Reply-To: <4F96DE7D.6030807@rename-it.nl> References: <4F96D56A.60907@gmail.com> <4F96DE7D.6030807@rename-it.nl> Message-ID: <4F96DF65.90909@gmail.com> I pulled the latest code from the dovecot-2.1-pigeonhole repository. Cheers, Olivier On 24/04/2012 18:10, Stephan Bosch wrote: > Op 4/24/2012 6:31 PM, interfaSys s?rl schreef: >> Hello, >> sievec segfaults when it reads a file. >> I've tried with both the standard and gold linker. >> I've tried with various sieve files and they all make it crash. > > What version? > > Regards, > > Stephan. From interfasys at gmail.com Tue Apr 24 20:27:51 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Tue, 24 Apr 2012 18:27:51 +0100 Subject: [Dovecot] sievec segfaults In-Reply-To: <4F96DE7D.6030807@rename-it.nl> References: <4F96D56A.60907@gmail.com> <4F96DE7D.6030807@rename-it.nl> Message-ID: <4F96E297.50300@gmail.com> I've recompiled Dovecot with ggdb and here is a more meaningful trace. Program received signal SIGSEGV, Segmentation fault. 0x000000080067796b in str_append () from /usr/lib/dovecot/libdovecot.so.0 (gdb) bt full #0 0x000000080067796b in str_append () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #1 0x000000080047b672 in sieve_binary_save () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #2 0x0000000800461b04 in sieve_save_as () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #3 0x00000000004032e6 in main (argc=2, argv=0x7fffffffe948) at sievec.c:147 svinst = (struct sieve_instance *) 0x800c90040 st = {st_dev = 1288178348, st_ino = 6479, st_mode = 33261, st_nlink = 1, st_uid = 12, st_gid = 6, st_rdev = 4294967295, st_atimespec = { tv_sec = 1335288301, tv_nsec = 418632262}, st_mtimespec = {tv_sec = 1285859319, tv_nsec = 0}, st_ctimespec = {tv_sec = 1297096638, tv_nsec = 658928714}, st_size = 1197, st_blocks = 4, st_blksize = 4096, st_flags = 0, st_gen = 8013395, st_lspare = 0, st_birthtimespec = { tv_sec = 1297096638, tv_nsec = 658928714}} sbin = (struct sieve_binary *) 0x800c99040 dump = false scriptfile = (const __unknown__ *) 0x7fffffffec56 "/etc/test.sieve" outfile = (const __unknown__ *) 0x0 exit_status = 0 c = -1 Cheers, Olivier On 24/04/2012 18:10, Stephan Bosch wrote: > Op 4/24/2012 6:31 PM, interfaSys s?rl schreef: >> Hello, >> sievec segfaults when it reads a file. >> I've tried with both the standard and gold linker. >> I've tried with various sieve files and they all make it crash. > > What version? > > Regards, > > Stephan. From stephan at rename-it.nl Tue Apr 24 20:34:39 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 24 Apr 2012 19:34:39 +0200 Subject: [Dovecot] sievec segfaults In-Reply-To: <4F96DF65.90909@gmail.com> References: <4F96D56A.60907@gmail.com> <4F96DE7D.6030807@rename-it.nl> <4F96DF65.90909@gmail.com> Message-ID: <4F96E42F.4090206@rename-it.nl> Op 4/24/2012 7:14 PM, interfaSys s?rl schreef: > I pulled the latest code from the dovecot-2.1-pigeonhole repository. This is caused by the recent addition of dict database support for storing Sieve scripts. This should fix it: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/93bfa75d9ab9 Regards, Stephan. From almarzuki2001 at hotmail.com Tue Apr 24 20:41:00 2012 From: almarzuki2001 at hotmail.com (Hadi Salem) Date: Tue, 24 Apr 2012 20:41:00 +0300 Subject: [Dovecot] dovecot LDA error Message-ID: Hi, My dovecot version is , # 2.0.20: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-164.el5 x86_64 CentOS release 5.4 (Final) auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_verbose_passwords = plain debug_log_path = /var/log/dovecot.log disable_plaintext_auth = no hostname = host1.bigmama.com info_log_path = /var/log/dovecot.log log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:~/Maildir passdb { args = scheme=PLAIN username_format=%u /etc/dovecot/users driver = passwd-file } postmaster_address = root at host1.bigmama.com service auth { unix_listener auth-userdb { group = mail mode = 0660 user = mail } } ssl = no userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol lda { info_log_path = /var/log/dovecot-lda.log log_path = /var/log/dovecot-lda-errors.log } Im getting error from dovecot LDA logs log_path = /var/log/dovecot-lda-errors.log Apr 24 20:25:46 lda(jamal at example.com): Fatal: setgid(5000(vmail) from userdb lo okup) failed with euid=8(mail), gid=12(mail), egid=12(mail): Operation not permi tted (This binary should probably be called with process group set to 5000(vmail ) instead of 12(mail)) My usersdb config jamal at example.com:{PLAIN}123:5000:5000::/var/vmail/example.com/jamal/ user 5000 system user as vmail Permissions for the /var/vmail directory as follow drwxr-xr-x 3 vmail vmail 4096 Apr 24 20:18 /var/vmail/ drwx------ 3 vmail vmail 4096 Apr 24 20:18 /var/vmail/example.com/ drwx------ 3 vmail vmail 4096 Apr 24 20:18 /var/vmail/example.com/jamal/ drwx------ 5 vmail vmail 4096 Apr 24 20:20 /var/vmail/example.com/jamal/Maildir/ From interfasys at gmail.com Tue Apr 24 20:42:29 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Tue, 24 Apr 2012 18:42:29 +0100 Subject: [Dovecot] sievec segfaults In-Reply-To: <4F96E42F.4090206@rename-it.nl> References: <4F96D56A.60907@gmail.com> <4F96DE7D.6030807@rename-it.nl> <4F96DF65.90909@gmail.com> <4F96E42F.4090206@rename-it.nl> Message-ID: <4F96E605.6080506@gmail.com> It worked! :) Thank you. Olivier On 24/04/2012 18:34, Stephan Bosch wrote: > Op 4/24/2012 7:14 PM, interfaSys s?rl schreef: >> I pulled the latest code from the dovecot-2.1-pigeonhole repository. > > This is caused by the recent addition of dict database support for > storing Sieve scripts. > > This should fix it: > > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/93bfa75d9ab9 > > Regards, > > Stephan. From tss at iki.fi Tue Apr 24 21:10:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 24 Apr 2012 21:10:17 +0300 Subject: [Dovecot] trash plugin bug? In-Reply-To: <932BA6EA-2884-4847-B66A-E056C8068B28@xs4all.nl> References: <932BA6EA-2884-4847-B66A-E056C8068B28@xs4all.nl> Message-ID: <69567A90-450A-457A-BC38-C0DE235156D9@iki.fi> On 23.4.2012, at 23.40, Cor Bosman wrote: > Hey all, has anyone ever tried turning on the trash plugin when the directory is already over quota? I see some messages being deleted, but it seems it just deletes enough to fit the new email, not enough to go below quota. Well, normally you shouldn't be over quota I guess.. :) Anyway, http://hg.dovecot.org/dovecot-2.1/rev/ec8564741aa8 http://hg.dovecot.org/dovecot-2.1/rev/dd3798681283 From tss at iki.fi Tue Apr 24 21:13:23 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 24 Apr 2012 21:13:23 +0300 Subject: [Dovecot] doveadm delete folders ? / unsubscribe ? In-Reply-To: <4F965B2E.2010103@schetterer.org> References: <4F900066.9030808@schetterer.org> <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> <4F906EE8.1030900@schetterer.org> <4F965B2E.2010103@schetterer.org> Message-ID: <283E4240-57E8-4BF2-8DCA-6DCC78C1D5A8@iki.fi> On 24.4.2012, at 10.50, Robert Schetterer wrote: > doveadm expunge -d -u *@domain.example mailbox exampelfolder\* > savedbefore 10d > > works like charm, but the folders do not get unsubscribed > any chance to have i.e another add parameter ,or do it auto > cause empty and deleted folders should not stay subscribed in my eyes Yeah, I guess unsubscribing is always wanted: http://hg.dovecot.org/dovecot-2.1/rev/b1653b7bd165 From cor at xs4all.nl Tue Apr 24 21:23:54 2012 From: cor at xs4all.nl (Cor Bosman) Date: Tue, 24 Apr 2012 20:23:54 +0200 Subject: [Dovecot] trash plugin bug? In-Reply-To: <69567A90-450A-457A-BC38-C0DE235156D9@iki.fi> References: <932BA6EA-2884-4847-B66A-E056C8068B28@xs4all.nl> <69567A90-450A-457A-BC38-C0DE235156D9@iki.fi> Message-ID: <6B17E4EF-17E9-4BD1-95DE-93CC0E0316AD@xs4all.nl> > >> Hey all, has anyone ever tried turning on the trash plugin when the directory is already over quota? I see some messages being deleted, but it seems it just deletes enough to fit the new email, not enough to go below quota. > > Well, normally you shouldn't be over quota I guess.. :) Anyway, :) It's hard to initialize new quota rules on a large userbase without some users falling outside the expected parameters :) Sucks when you have lots of users, you always always hit edge cases. Thanks! Cor From daniel.parthey at informatik.tu-chemnitz.de Wed Apr 25 05:13:55 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 25 Apr 2012 04:13:55 +0200 Subject: [Dovecot] How to remove duplicate GUID messages from storage? In-Reply-To: <20120421202928.GA12998@daniel.localdomain> References: <20120421202928.GA12998@daniel.localdomain> Message-ID: <20120425021355.GA10493@daniel.localdomain> Daniel Parthey schrieb: > I tried to import messages like this: > doveadm import -u username at example.org mdbox:/mail/dovecot/example.org/username/mail mbox:/root/mail.txt all > > This seemed to have imported the same messages in the mail storage, > as they were already there. And now I have got a lot of duplicates. > > Then I deleted the mbox:/root/mail mail stuff from "mailboxes" folder > and the storage remained, with duplicate GUIDs. > > The following command > doveadm -v force-resync -u username at example.org INBOX > outputs lots of messages like this one, with different GUID > doveadm(username at example.org): Error: mdbox /mail/dovecot/example.org/username/mail/storage: Duplicate GUID 0b8a032d66a0924fb42c0000de5f8128 in m.55:45484041 and m.14:52173045 > > The messages at m.55:45484041 and m.14:52173045 have the same content, > since they were seemingly imported from the mail store itself, > they are in the storage twice now and require twice as much disk > space as before. > > How can I manually remove these identical, duplicate messages > from the storage to save space? Dovecot does not do it > automatically. Should I edit the mdbox storage files directly using vim in order to remove the duplicate messages which I imported by accident, or is there any dovecot mdbox "repair toolkit" with removes duplicate messages? I attached the dovecot version and config. Regards, Daniel # doveconf -n # 2.0.20: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-40-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_verbose = yes auth_verbose_passwords = sha1 deliver_log_format = mailbox: deliver: msgid=%m from=%f: %$ dict { quota = mysql:/etc/dovecot/conf.d/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = Mailbox login_log_format = mailbox: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "mailbox: mail: %s(%u): " mail_plugins = quota mail_privileged_group = vmail mail_uid = vmail managesieve_implementation_string = Sieve managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 50 M mmap_disable = yes passdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve service dict { unix_listener dict { group = vmail mode = 0660 } } service imap-login { inet_listener imap { port = 19143 } } service lmtp { inet_listener lmtp { address = * port = 19024 } } service managesieve-login { inet_listener sieve { port = 19200 } } service pop3-login { inet_listener pop3 { port = 19110 } } service quota-warning { executable = script /usr/local/bin/quota-warning unix_listener quota-warning { user = vmail } user = dovecot } ssl = no userdb { driver = prefetch } userdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_plugins = quota imap_quota } protocol lmtp { mail_plugins = quota sieve } From robert at schetterer.org Wed Apr 25 09:13:02 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 08:13:02 +0200 Subject: [Dovecot] doveadm delete folders ? / unsubscribe ? In-Reply-To: <283E4240-57E8-4BF2-8DCA-6DCC78C1D5A8@iki.fi> References: <4F900066.9030808@schetterer.org> <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> <4F906EE8.1030900@schetterer.org> <4F965B2E.2010103@schetterer.org> <283E4240-57E8-4BF2-8DCA-6DCC78C1D5A8@iki.fi> Message-ID: <4F9795EE.4030604@schetterer.org> Am 24.04.2012 20:13, schrieb Timo Sirainen: > On 24.4.2012, at 10.50, Robert Schetterer wrote: > >> doveadm expunge -d -u *@domain.example mailbox exampelfolder\* >> savedbefore 10d >> >> works like charm, but the folders do not get unsubscribed >> any chance to have i.e another add parameter ,or do it auto >> cause empty and deleted folders should not stay subscribed in my eyes > > Yeah, I guess unsubscribing is always wanted: > > http://hg.dovecot.org/dovecot-2.1/rev/b1653b7bd165 > Hi Timo, i will try after update and report, jep unsubscribing should be the uni wanted behave thx again ! -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From weston.kris at ymail.com Wed Apr 25 09:17:18 2012 From: weston.kris at ymail.com (Kris) Date: Wed, 25 Apr 2012 07:17:18 +0100 Subject: [Dovecot] dovecot %u variable problem.... Message-ID: <4F9796EE.5070907@ymail.com> Hi guys n gals, im on an ubuntu 11 VPS on linode (dovecot version 2.013). ive spent a lot of time and googling trying to get postfix, dovecot, ssl, saslauthd and mysql to all play nice together and im quite close now but theres something going on i cant quite explain and id really appreciate some help. I have been going for ten hours on this today and it seems in direct conflict with what dovecot themselves say... the problem is a variable in dovecot (%u) which is normally used as a full username in sql query strings (i.e. user at domain) and is simply dropping the domain for some reason... From dovecot docs (http://wiki2.dovecot.org/Variables): The variables that work everywhere are: %u user full username (e.g. user at domain) so i have postfix and stuff set up ok to send at least and i know this because ive tested it by hard-coding the string in the sql query myself with a successful send... but, anyway, the problem with the '%u' variable is it is doing stuff like this: (from mail.log and mysql.log) localhost dovecot: auth: Debug: sql(admin at microhard.com,xx.xx.xx.xx): SELECT home,uid,gid FROM users WHERE id = 'admin at microhard.com' this was derived from the following query in /etc/dovecot/dovecot-sql.conf.ext user_query = SELECT home,uid,gid FROM users WHERE id = '%u' so no problem there.... but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user at domain): localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' Query SELECT id as user, crypt as password FROM users WHERE id= 'admin' which is derived from the following query also in /etc/dovecot/dovecot-sql.conf.ext password_query = SELECT id as user, crypt as password FROM users WHERE id= '%u' and to further confuse me, this 'unchangeable' variable sometimes decides to right itself without any obvious reason... localhost dovecot: auth: Debug: sql(admin at microhard.com,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin at microhard.com' im wondering if its something to do with my client dropping the domain, or whether dovecot has an error, or some other thing ive overlooked... my dovecot.conf looks a lot like this: # 2.0.13: dovecot.conf # OS: Linux 3.0.18-x86_64-linode24 x86_64 Ubuntu 11.10 ext3 first_valid_uid = 5000 last_valid_uid = 5000 login_greeting = I likes my ducks n geese I do. mail_location = maildir:/var/spool/mail/virtual/%u #enables logging all failed authentication attempts. auth_verbose=yes #enables all authentication debug logging (also enables auth_verbose). Passwords are logged as . auth_debug=yes #does everything that auth_debug=yes does, but it also removes password hiding. auth_debug_passwords=yes #enables all kinds of mail related debug logging, such as showing where Dovecot is looking for mails. mail_debug=yes #enables logging SSL errors and warnings. Even without this setting if connection is closed because of an SSL error, the error is logged as the disconnection reason (v1.1+). verbose_ssl=yes passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/etc/dovecot/acls #quota = maildir:storage=10240:messages=1000 trash = /etc/dovecot/trash.conf } protocols = " imap" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = dovecot mode = 0660 user = dovecot } user = dovecot } ssl_ca = /etc/pki/dovecot/certs/ca-bundle.crt ssl_cert = References: <4F9796EE.5070907@ymail.com> Message-ID: <4F979E5E.5000409@schetterer.org> Am 25.04.2012 08:17, schrieb Kris: > this was derived from the following query in > /etc/dovecot/dovecot-sql.conf.ext > user_query = SELECT home,uid,gid FROM users WHERE id = '%u' i use like this user_query = SELECT concat('/usr/local/virtual/', maildir) AS home, \ concat('*:bytes=', mailbox.quota) AS quota_rule, \ #when saving to Trash mailbox the user gets additional 50MB "Trash:storage=+50240" AS quota_rule2, \ #when saving to Sent mailbox the user gets additional 50MB "Sent:storage=+50240" AS quota_rule3, \ #when saving to Drafts mailbox the user gets additional 50MB "Drafts:storage=+50240" AS quota_rule4, \ #when saving to Templates mailbox the user gets additional 50MB "Templates:storage=+50240" AS quota_rule5, \ #when saving to Junk mailbox the user gets additional 50MB "Junk:storage=+50240" AS quota_rule6, \ #when saving to Archives mailbox the user gets additional 50MB "Archives:storage=+50240" AS quota_rule7, \ ########### concat('maildir:/usr/local/virtual/', maildir) AS mail, \ CASE '%s' WHEN 'pop3' THEN NULL ELSE 'yes' END AS 'namespace/inbox/inbox', \ CASE '%s' WHEN 'pop3' THEN 'yes' ELSE NULL END AS 'namespace/virtual/inbox', \ 1001 AS uid, 1001 AS gid FROM mailbox WHERE username = '%u' AND active = '1' > > so no problem there.... > > but then this happens when i attempt to send mail and this means the > user gets rejected because dovecot cant find the user id in the db > without the correct string (i.e. user at domain): > localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id > as user, crypt as password FROM users WHERE id= 'admin' > Query SELECT id as user, crypt as password FROM users WHERE id= 'admin' > which is derived from the following query also in > /etc/dovecot/dovecot-sql.conf.ext > password_query = SELECT id as user, crypt as password FROM users WHERE > id= '%u' > > and to further confuse me, this 'unchangeable' variable sometimes > decides to right itself without any obvious reason... > localhost dovecot: auth: Debug: sql(admin at microhard.com,xx.xx.xx.xx): > query: SELECT id as user, crypt as password FROM users WHERE id= > 'admin at microhard.com' > > im wondering if its something to do with my client dropping the domain, > or whether dovecot has an error, or some other thing ive overlooked... password_query = SELECT username as user, password, \ 1001 as userdb_uid, \ 1001 as userdb_gid, \ "/usr/local/virtual/%d/%u/" AS userdb_home, \ "maildir:/usr/local/virtual/%d/%u/" AS userdb_mail \ FROM mailbox WHERE username = '%u' AND active = '1' AND (imap_allowed = '1' or '%Ls' = 'pop3') this should not fit to you ( postfixadmin full domain layout with virtual plugin and quota additions ,clear text passwords etc), but perhaps gives you ideas by compare, what to fix at your side, however upgrade to dove latest 2.0.20 -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From dg at dguhl.org Wed Apr 25 11:26:15 2012 From: dg at dguhl.org (Dennis Guhl) Date: Wed, 25 Apr 2012 10:26:15 +0200 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F9796EE.5070907@ymail.com> References: <4F9796EE.5070907@ymail.com> Message-ID: <20120425082615.GA14759@laptop-dg.leere.eu> On Wed, Apr 25, 2012 at 07:17:18AM +0100, Kris wrote: [..] > the problem is a variable in dovecot (%u) which is normally used as > a full username in sql query strings (i.e. user at domain) and is > simply dropping the domain for some reason... [..] > and to further confuse me, this 'unchangeable' variable sometimes > decides to right itself without any obvious reason... > localhost dovecot: auth: Debug: > sql(admin at microhard.com,xx.xx.xx.xx): query: SELECT id as user, > crypt as password FROM users WHERE id= 'admin at microhard.com' > > im wondering if its something to do with my client dropping the > domain, or whether dovecot has an error, or some other thing ive > overlooked... Indeed, it seems some of your users try to authenticate with a username without the '@domain' part. You can either try to make your users comply or you can use auth_default_realm in your config: # Default realm/domain to use if none was specified. This is # used for both SASL realms and appending @domain to username # in plaintext logins. # #auth_default_realm = Dennis [..] From cor at xs4all.nl Wed Apr 25 11:27:18 2012 From: cor at xs4all.nl (Cor Bosman) Date: Wed, 25 Apr 2012 10:27:18 +0200 Subject: [Dovecot] trash plugin bug? In-Reply-To: <69567A90-450A-457A-BC38-C0DE235156D9@iki.fi> References: <932BA6EA-2884-4847-B66A-E056C8068B28@xs4all.nl> <69567A90-450A-457A-BC38-C0DE235156D9@iki.fi> Message-ID: <0D69989C-00E4-48CC-AF08-B58C52EA3A12@xs4all.nl> > > Well, normally you shouldn't be over quota I guess.. :) Anyway, > > http://hg.dovecot.org/dovecot-2.1/rev/ec8564741aa8 > http://hg.dovecot.org/dovecot-2.1/rev/dd3798681283 This indeed fixed the problem. Thank you, Cor From weston.kris at ymail.com Wed Apr 25 11:50:20 2012 From: weston.kris at ymail.com (Kris Weston) Date: Wed, 25 Apr 2012 09:50:20 +0100 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <20120425082615.GA14759@laptop-dg.leere.eu> References: <4F9796EE.5070907@ymail.com> <20120425082615.GA14759@laptop-dg.leere.eu> Message-ID: <4F97BACC.4080902@ymail.com> thanks for your help dennis... > Indeed, it seems some of your users try to authenticate with a > username without the '@domain' part. > > You can either try to make your users comply or you can use > auth_default_realm in your config: in this case the users are me :) same username every time i connect, but each time i see a load of db lookups that are different but using this same u variable... > # Default realm/domain to use if none was specified. This is > # used for both SASL realms and appending @domain to username > # in plaintext logins. > # > #auth_default_realm = > ahhh, ok. only i have multiple domains, so how does that werk ? just put em all in ? cheerz Kris From dg at dguhl.org Wed Apr 25 11:58:05 2012 From: dg at dguhl.org (Dennis Guhl) Date: Wed, 25 Apr 2012 10:58:05 +0200 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F97BACC.4080902@ymail.com> References: <4F9796EE.5070907@ymail.com> <20120425082615.GA14759@laptop-dg.leere.eu> <4F97BACC.4080902@ymail.com> Message-ID: <20120425085805.GB14759@laptop-dg.leere.eu> On Wed, Apr 25, 2012 at 09:50:20AM +0100, Kris Weston wrote: > > thanks for your help dennis... > >Indeed, it seems some of your users try to authenticate with a > >username without the '@domain' part. > > > >You can either try to make your users comply or you can use > >auth_default_realm in your config: > > in this case the users are me :) > same username every time i connect, but each time i see a load of db > lookups that are different but using this same u variable... If it's no bug, I doubt it randomly changes the value for %u while the input is the same. So it seems the problem lies somewhere else. And if it is no layer 8 problem it must be the client ;) > > # Default realm/domain to use if none was specified. This is > > # used for both SASL realms and appending @domain to username > > # in plaintext logins. > > # > > #auth_default_realm = > > > > ahhh, ok. only i have multiple domains, so how does that werk ? > just put em all in ? IIRC no, you can only name one default. Dennis From tss at iki.fi Wed Apr 25 13:45:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 13:45:08 +0300 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F979E5E.5000409@schetterer.org> References: <4F9796EE.5070907@ymail.com> <4F979E5E.5000409@schetterer.org> Message-ID: <629E1A55-910F-48B9-8986-C87A34852F24@iki.fi> On 25.4.2012, at 9.49, Robert Schetterer wrote: > #when saving to Trash mailbox the user gets additional 50MB > "Trash:storage=+50240" AS quota_rule2, \ > #when saving to Sent mailbox the user gets additional 50MB > "Sent:storage=+50240" AS quota_rule3, \ > #when saving to Drafts mailbox the user gets additional 50MB > "Drafts:storage=+50240" AS quota_rule4, \ > #when saving to Templates mailbox the user gets additional 50MB > "Templates:storage=+50240" AS quota_rule5, \ > #when saving to Junk mailbox the user gets additional 50MB > "Junk:storage=+50240" AS quota_rule6, \ > #when saving to Archives mailbox the user gets additional 50MB > "Archives:storage=+50240" AS quota_rule7, \ > ########### BTW. These are all static values and don't really need to be in SQL query: plugin { quota_rule2 = Trash:... quota_rule3 = ... ... } > 1001 as userdb_uid, \ > 1001 as userdb_gid, \ > "/usr/local/virtual/%d/%u/" AS userdb_home, \ > "maildir:/usr/local/virtual/%d/%u/" AS userdb_mail \ Same for these. Also slightly confusing that your passdb and userdb lookups return mail setting differently. From tss at iki.fi Wed Apr 25 13:48:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 13:48:45 +0300 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F9796EE.5070907@ymail.com> References: <4F9796EE.5070907@ymail.com> Message-ID: <5B92F036-DDBE-42AB-94A1-DBF21F11DE14@iki.fi> On 25.4.2012, at 9.17, Kris wrote: > but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user at domain): > localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' What exactly is doing this lookup? SMTP AUTH? Maybe the client's SMTP authentication is configured without the @domain part in username? From tss at iki.fi Wed Apr 25 13:52:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 13:52:26 +0300 Subject: [Dovecot] How to remove duplicate GUID messages from storage? In-Reply-To: <20120421202928.GA12998@daniel.localdomain> References: <20120421202928.GA12998@daniel.localdomain> Message-ID: On 21.4.2012, at 23.29, Daniel Parthey wrote: > The following command > doveadm -v force-resync -u username at example.org INBOX > outputs lots of messages like this one, with different GUID > doveadm(username at example.org): Error: mdbox /mail/dovecot/example.org/username/mail/storage: Duplicate GUID 0b8a032d66a0924fb42c0000de5f8128 in m.55:45484041 and m.14:52173045 I think this is a force-resync bug and it shouldn't really complain about duplicates. Although I'm not entirely sure why with you it's complaining about them at all. I have anyway in TODO to look into this.. > How can I manually remove these identical, duplicate messages > from the storage to save space? Dovecot does not do it > automatically. Perhaps force-resync + purge should do that, but currently it doesn't. Also it would be nice if doveadm import didn't add duplicates in the first place. This is also something for which I have vague plans, because it would help dsync as well. From robert at schetterer.org Wed Apr 25 14:07:56 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 13:07:56 +0200 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <629E1A55-910F-48B9-8986-C87A34852F24@iki.fi> References: <4F9796EE.5070907@ymail.com> <4F979E5E.5000409@schetterer.org> <629E1A55-910F-48B9-8986-C87A34852F24@iki.fi> Message-ID: <4F97DB0C.4040901@schetterer.org> Am 25.04.2012 12:45, schrieb Timo Sirainen: > On 25.4.2012, at 9.49, Robert Schetterer wrote: > >> #when saving to Trash mailbox the user gets additional 50MB >> "Trash:storage=+50240" AS quota_rule2, \ >> #when saving to Sent mailbox the user gets additional 50MB >> "Sent:storage=+50240" AS quota_rule3, \ >> #when saving to Drafts mailbox the user gets additional 50MB >> "Drafts:storage=+50240" AS quota_rule4, \ >> #when saving to Templates mailbox the user gets additional 50MB >> "Templates:storage=+50240" AS quota_rule5, \ >> #when saving to Junk mailbox the user gets additional 50MB >> "Junk:storage=+50240" AS quota_rule6, \ >> #when saving to Archives mailbox the user gets additional 50MB >> "Archives:storage=+50240" AS quota_rule7, \ >> ########### > > BTW. These are all static values and don't really need to be in SQL query: > > plugin { > quota_rule2 = Trash:... > quota_rule3 = ... > ... > } > >> 1001 as userdb_uid, \ >> 1001 as userdb_gid, \ >> "/usr/local/virtual/%d/%u/" AS userdb_home, \ >> "maildir:/usr/local/virtual/%d/%u/" AS userdb_mail \ > > Same for these. Also slightly confusing that your passdb and userdb lookups return mail setting differently. thx timo, i will look at this, thats all historic but ever worked currently i am still fighting with some upgrade issues to 2.1 just now some sieve experimental stuff failed , i will post about it -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Apr 25 16:06:29 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 15:06:29 +0200 Subject: [Dovecot] sieve vacation redirect problem Message-ID: <4F97F6D5.2080104@schetterer.org> Hi , since upgrade sieve vacation, redirect stopped working getting postfix error while sending end of data -- message may be sent more than once i played a lot about sieve , lda submission host conf entries etc but all failed , i am not seeing any heavy bug in the logs -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Apr 25 16:29:47 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 15:29:47 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F97F6D5.2080104@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> Message-ID: <4F97FC4B.6060803@schetterer.org> Am 25.04.2012 15:06, schrieb Robert Schetterer: > Hi , since upgrade > sieve vacation, redirect stopped working getting > > postfix error > while sending end of data -- message may be sent more than once > > i played a lot about sieve , lda submission host conf entries etc but > all failed , i am not seeing any heavy bug in the logs > that not full truth i am seeing master: service(lmtp): child 31952 killed with signal 11 (core not dumped) -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Wed Apr 25 16:46:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 16:46:31 +0300 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F97FC4B.6060803@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> Message-ID: <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> On 25.4.2012, at 16.29, Robert Schetterer wrote: > master: service(lmtp): child 31952 killed with signal 11 (core not dumped) Can you get a gdb backtrace? Might be a bit difficult to get a core dump from lmtp, but you should be able to attach gdb into it: service lmtp { process_min_avail = 1 } gdb -p `pidof lmtp` cont bt full From stephan at rename-it.nl Wed Apr 25 16:51:29 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 25 Apr 2012 15:51:29 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F97FC4B.6060803@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> Message-ID: <4F980161.9080007@rename-it.nl> Op 4/25/2012 3:29 PM, Robert Schetterer schreef: > Am 25.04.2012 15:06, schrieb Robert Schetterer: >> Hi , since upgrade >> sieve vacation, redirect stopped working getting >> >> postfix error >> while sending end of data -- message may be sent more than once >> >> i played a lot about sieve , lda submission host conf entries etc but >> all failed , i am not seeing any heavy bug in the logs >> > that not full truth i am seeing > > master: service(lmtp): child 31952 killed with signal 11 (core not dumped) Please try to obtain a gdb backtrace: http://dovecot.org/bugreport.html And provide dovecot -n output Regards, Stephan. From robert at schetterer.org Wed Apr 25 17:03:49 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 16:03:49 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> Message-ID: <4F980445.4090203@schetterer.org> Am 25.04.2012 15:46, schrieb Timo Sirainen: > bt full Program received signal SIGSEGV, Segmentation fault. smtp_client_open (set=0x252b0b0, destination=0x254e410 "ncitester2 at onmotive.de", return_path=0x24e4148 "root at mail02.mailspooler.com", file_r=0x7fff2040a748) at smtp-client.c:146 146 smtp-client.c: Datei oder Verzeichnis nicht gefunden. in smtp-client.c (gdb) Undefined command: " path = 0x180000

fd = #1 0x00007fd4e58b57a5 in act_redirect_send (action=, aenv=0x254e2c0, tr_context=, keep=) at cmd-redirect.c:339 size = msgctx = crlf_input = smtp_handle = 0x24e4148 data = ret = sender = 0x24e4148 "root at mail02.mailspooler.com" recipient = 0x24e4168 "robert at schetterer.name" input = f = hide_headers = {0x7fd4e58d2ee7 "Return-Path", 0x7fd4e58d6d3c "X-Sieve", 0x7fd4e58d7929 "X-Sieve-Redirected-From"} #2 act_redirect_commit (action=, aenv=0x254e2c0, tr_context=, keep=) at cmd-redirect.c:402 ctx = 0x254e408 mail = 0x252ba90 senv = 0x7fff2040a8c0 dupeid = #3 0x00007fd4e58a7eeb in sieve_result_execute (result=, keep=) at sieve-result.c:1156 impl_keep = true rsef = 0x0 implicit_keep = true success = true commit_ok = rac = 0x254e428 first_action = #4 0x00007fd4e58b8418 in sieve_multiscript_execute (mscript=0x254e3d8, sbin=0x25516c0, ehandler=0x254c100, flags=, final=true) at sieve.c:601 No locals. #5 sieve_multiscript_run (mscript=0x254e3d8, sbin=0x25516c0, ehandler=0x254c100, flags=, final=true) at sieve.c:630 No locals. #6 0x00007fd4e5af47e6 in lda_sieve_multiscript_execute (mdctx=, storage_r=) at lda-sieve-plugin.c:514 sbin = 0x25516c0 rtflags = SIEVE_RUNTIME_FLAG_NOGLOBAL cpflags = SIEVE_COMPILE_FLAG_NOGLOBAL mscript = 0x254e3d8 svinst = 0x2546e80 last_script = 0x254b8e0 ret = scripts = ehandler = 0x254c100 i = 1 more = true error = SIEVE_ERROR_NONE #7 lda_sieve_deliver_mail (mdctx=, storage_r=) at lda-sieve-plugin.c:794 scripts = i = _data_stack_cur_id = 3 srctx = {svinst = 0x2546e80, mdctx = 0x7fff2040aac0, scripts = 0x24e3340, script_count = 2, user_script = 0x254b8e0, main_script = 0x254b8e0, msgdata = 0x7fff2040a990, scriptenv = 0x7fff2040a8c0, user_ehandler = 0x254c100, master_ehandler = 0x254b7d0, userlog = 0x24e3658 "/usr/local/virtual/schetterer.name/robert at schetterer.name/dovecot.sieve.log"} svenv = {hostname = 0x250aba0 "mail02.mailspooler.com", base_dir = 0x2536a78 "/var/run/dovecot", username = 0x25354c8 "robert at schetterer.name", home_dir = 0x2537c40 "/usr/local/virtual/schetterer.name/robert at schetterer.name/", flags = SIEVE_FLAG_HOME_RELATIVE} svinst = 0x2546e80 msgdata = {mail = 0x252ba90, return_path = 0x24f0c68 "root at mail02.mailspooler.com", orig_envelope_to = 0x24f12c0 "robert at schetterer.name", final_envelope_to = 0x24f12c0 "robert at schetterer.name", auth_user = 0x25354c8 "robert at schetterer.name", id = 0x252c410 "<20120425140007.D3556840227 at mail02.mailspooler.com>"} scriptenv = {action_log_format = 0x250ac28 "msgid=%m: %$", user = 0x2535410, default_mailbox = 0x4077ac "INBOX", postmaster_address = 0x250ab78 "hostmaster at mail02.mailspooler.com", mailbox_autocreate = false, mailbox_autosubscribe = false, script_context = 0x7fff2040aac0, smtp_open = 0x7fd4e5af4180 , smtp_close = 0x7fd4e5af4160 , duplicate_check = 0x7fd4e5af4110 , duplicate_mark = 0x7fd4e5af4140 , reject_mail = 0x7fd4e5af4100 , exec_status = 0x7fff2040aa00, trace_stream = 0x0, trace_config = {level = SIEVE_TRLVL_NONE, flags = 0}} estatus = {message_saved = false, message_forwarded = false, tried_default_save = false, keep_original = false, last_storage = 0x0} master_ehandler = 0x254b7d0 user_location = default_location = sieve_before = sieve_after = script_sequence = {arr = {buffer = 0x24e3308, element_size = 8}, v = 0x24e3308, v_modifiable = 0x24e3308} debug = error = SIEVE_ERROR_NONE ret = #8 0x00007fd4e888a445 in mail_deliver (ctx=0x7fff2040aac0, storage_r=0x7fff2040ab50) at mail-deliver.c:389 ret = #9 0x0000000000405a33 in client_deliver (client=0x24f00d0) at commands.c:538 ns = storage = 0x0 sets = mail_error = MAIL_ERROR_NONE dctx = {pool = 0x252aa90, set = 0x250ab28, session = 0x252aab0, dup_ctx = 0x2546d90, session_id = 0x24f0c50 "ma3NNmcDmE/bDAAAAfUEdQ", src_mail = 0x252ba90, src_envelope_sender = 0x24f0c68 "root at mail02.mailspooler.com", dest_user = 0x2535410, dest_addr = 0x24f12c0 "robert at schetterer.name", final_dest_addr = 0x24f12c0 "robert at schetterer.name", dest_mailbox_name = 0x4077ac "INBOX", dest_mail = 0x0, var_expand_table = 0x0, tried_default_save = false, saved_mail = false, save_dest_mail = false, mailbox_full = false, dsn = false} input = error = #10 client_deliver_next (client=0x24f00d0) at commands.c:577 rcpts = 0x24f0cc0 #11 client_input_data_write_local (client=0x24f00d0) at commands.c:671 src_mail = 0x252ba90 first_uid = 4294967295 session = 0x252aab0 old_uid = 0 #12 client_input_data_write (client=0x24f00d0) at commands.c:779 input = 0x2513530 #13 client_input_data_handle (client=0x24f00d0) at commands.c:873 data = size = 861 ---Type to continue, or q to quit--- -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Apr 25 17:10:25 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 16:10:25 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F980161.9080007@rename-it.nl> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4F980161.9080007@rename-it.nl> Message-ID: <4F9805D1.8050101@schetterer.org> Am 25.04.2012 15:51, schrieb Stephan Bosch: > Op 4/25/2012 3:29 PM, Robert Schetterer schreef: >> Am 25.04.2012 15:06, schrieb Robert Schetterer: >>> Hi , since upgrade >>> sieve vacation, redirect stopped working getting >>> >>> postfix error >>> while sending end of data -- message may be sent more than once >>> >>> i played a lot about sieve , lda submission host conf entries etc but >>> all failed , i am not seeing any heavy bug in the logs >>> >> that not full truth i am seeing >> >> master: service(lmtp): child 31952 killed with signal 11 (core not >> dumped) > > Please try to obtain a gdb backtrace: > > http://dovecot.org/bugreport.html > > And provide dovecot -n output > > Regards, > > Stephan. did test a few sieve related stuff cause the working 2.0.20 didnt worked but now my ideas are running out # 2.1.5 (d2b01dcce79f): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-37-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 50 M auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes auth_verbose_passwords = plain auth_worker_max_count = 300 default_process_limit = 250 dict { acl = mysql:/etc/dovecot/dovecot-dict-acl-sql.conf.ext expire = mysql:/etc/dovecot/dovecot-dict-expire-sql.conf.ext quotadict = mysql:/etc/dovecot/dovecot-dict-quota-sql.conf.ext } disable_plaintext_auth = no first_valid_gid = 1001 first_valid_uid = 1001 hostname = mail02.mailspooler.com last_valid_gid = 1001 last_valid_uid = 1001 listen = * login_greeting = imap, pop ready mail_access_groups = vmail mail_debug = yes mail_fsync = always mail_gid = 1001 mail_location = maildir:~/ mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = " zlib expire virtual fts fts_squat mail_log autocreate notify acl listescape" mail_privileged_group = vmail mail_uid = 1001 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags notify mmap_disable = yes namespace { inbox = no list = children location = maildir:/usr/local/virtual/%%d/%%u:INDEX=~/shared/%%d/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { hidden = no list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } mailbox name { special_use = \Drafts \Junk \Sent \Trash } prefix = separator = / subscriptions = yes } namespace real { hidden = yes list = no location = prefix = RealMails/ separator = / } namespace virtual { hidden = yes list = no location = virtual:/etc/dovecot/virtual:LAYOUT=maildir++:INDEX=~/virtual prefix = virtual/ separator = / subscriptions = no } passdb { args = /etc/dovecot/dovecot-sql-master.conf.ext driver = sql master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl autocreate = Trash autocreate2 = Junk autocreate3 = Sent autocreate4 = Drafts autocreate5 = Templates autocreate6 = Hostmaster-Backup autocreate7 = archiv-backup-mailspooler autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Drafts autosubscribe5 = Templates expire = Trash expire2 = Trash/* expire3 = Junk expire4 = Junk/* expire5 = Hostmaster-Backup expire6 = Hostmaster-Backup/* expire7 = archiv-backup-mailspooler/* expire_dict = proxy::expire fts = squat home = /usr/local/virtual/%d/%u mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size mail_log_group_events = yes quota = dict:::proxy::quotadict quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /usr/local/virtual/%d/%u/dovecot.sieve sieve_before = /etc/dovecot/sieve/global.sieve sieve_dir = /usr/local/virtual/%d/%u/sieve sieve_extensions = +fileinto +reject +envelope +encoded-character +vacation +subaddress +relational +regex +imap4flags +copy +include +variables +body +enotify +environment +mailbox +date +ihave +imapflags +notify sieve_global_dir = /etc/dovecot/sieve/ sieve_global_path = /etc/dovecot/sieve/global.sieve zlib_save = gz zlib_save_level = 6 } postmaster_address = hostmaster at mail02.mailspooler.com protocols = imap pop3 lmtp sieve service anvil { client_limit = 1000 } service auth-worker { user = root } service auth { client_limit = 1000 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service dict { extra_groups = vmail unix_listener dict { group = vmail mode = 0660 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 8 service_count = 1 vsz_limit = 256 M } service imap-postlogin { executable = script-login /usr/local/bin/postlogin_imap.sh } service imap { executable = imap imap-postlogin process_limit = 1024 vsz_limit = 256 M } service lmtp { process_min_avail = 1 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } vsz_limit = 256 M } service managesieve-login { inet_listener { address = 127.0.0.1 212.52.224.210 212.52.224.206 port = 2000 } vsz_limit = 256 M } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } process_min_avail = 8 } service pop3-postlogin { executable = script-login /usr/local/bin/postlogin_pop3.sh } service pop3 { executable = pop3 pop3-postlogin process_limit = 1024 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { mode = 0666 user = vmail } user = vmail } ssl_ca = References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> Message-ID: <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> On 25.4.2012, at 17.03, Robert Schetterer wrote: > Program received signal SIGSEGV, Segmentation fault. > smtp_client_open (set=0x252b0b0, destination=0x254e410 > "ncitester2 at onmotive.de", return_path=0x24e4148 > "root at mail02.mailspooler.com", file_r=0x7fff2040a748) at smtp-client.c:146 > 146 smtp-client.c: Datei oder Verzeichnis nicht gefunden. > in smtp-client.c If you do the same and get here again, what does it say with: p *set From robert at schetterer.org Wed Apr 25 17:16:28 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 16:16:28 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> Message-ID: <4F98073C.3080406@schetterer.org> Am 25.04.2012 16:11, schrieb Timo Sirainen: > p *set hope doing it right (gdb) p *set $1 = {postmaster_address = 0xaa33d0 "P6\252", hostname = 0x0, submission_host = 0x0, sendmail_path = 0x7fbaf6e41f50 "H\211\\$\360H\211l$\370H\203\354\030\200=\207A%", rejection_subject = 0x7fbaf6e42b90 "H\211\\$\350L\211d$\370H\211\373H\211l$\360H\203\354(\200=?5%", rejection_reason = 0x7fbaf6e41ed0 "H\211\\$\360H\211l$\370H\203\354(\200=\aB%", deliver_log_format = 0x1
, recipient_delimiter = 0xa9c910 "\003", lda_original_recipient_header = 0xabd490 "\001", quota_full_tempfail = 16, lda_mailbox_autocreate = 208, lda_mailbox_autosubscribe = 171} -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From warden at geneseo.edu Wed Apr 25 17:20:41 2012 From: warden at geneseo.edu (David Warden) Date: Wed, 25 Apr 2012 10:20:41 -0400 Subject: [Dovecot] stats + fts squat plugins, indexer-worker error on message indexing Message-ID: On RHEL 6.2 + dovecot 2.1.5 with the stats and fts (squat) plugins enabled and each user having their own uid on the system I get the following error in the logs when the indexer-worker process indexes some messages: Apr 25 09:56:19 wardentest3 dovecot: imap-login: Login: user=warden, method=PLAIN, rip=137.238.60.164, lip=137.238.2.240, lport=1993, mpid=19464, encryption=TLS Apr 25 09:56:26 wardentest3 dovecot: service=indexer-worker user=warden rip= Indexed 12 messages in INBOX Apr 25 09:56:26 wardentest3 dovecot: service=indexer-worker user=warden rip= Error: open(/proc/self/io) failed: Permission denied Apr 25 09:56:29 wardentest3 dovecot: service=imap user=warden rip=137.238.60.164 Disconnected: Logged out bytes(in/out)=46/780 What I did to generate the above was remove the squat indexes for my Inbox then log in via telnet and search for a string in my Inbox, which forced an index of the messages in my Inbox. Based on http://wiki2.dovecot.org/Services#indexer-worker it seems like this is because the indexer-worker process is created as root but at the time of indexing the indexer-worker drops privs to the mail user. In /proc//, io is mode 0400 and owned by root, so it makes sense that the mail user would be unable to read that file. The strange thing is it looks like the stats are still collected (unless I am reading the output of the stats dump incorrectly): [root at wardentest3 mailtest]# doveadm stats dump -s /var/run/dovecot/mailtest/stats command user=warden cmd args session user last_update user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits select inbox 27fb87218302984f084c0000ae899e04 warden 1335362182.950250 0.000000 0.000000 172 0 46 0 0 12288 13 7883 4 381 0 0 0 0 0 search text "warden" 27fb87218302984f084c0000ae899e04 warden 1335362186.588023 0.001000 0.000000 32 0 4 1 0 0 4 108 1 43 0 0 0 0 0 logout 27fb87218302984f084c0000ae899e04 warden 1335362189.161790 0.000000 0.000000 1 0 0 3 0 0 1 108 3 157 0 0 0 0 0 This obviously isn't a show-stopper but if this is causing stats to be incorrect it would be great if this could be fixed. -David Warden My doveconf -n: # 2.1.5: /etc/dovecot/mailtest/dovecot.conf doveconf: Warning: service auth { client_limit=4096 } is lower than required under max. load (12288) doveconf: Warning: service anvil { client_limit=4096 } is lower than required under max. load (12291) # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.2 (Santiago) nfs auth_cache_negative_ttl = 0 auth_cache_size = 16 M auth_gssapi_hostname = $ALL auth_krb5_keytab = /etc/dovecot/mailtest.combined.keytab auth_master_user_separator = * auth_mechanisms = plain login gssapi auth_username_format = %Ln base_dir = /var/run/dovecot/mailtest/ default_client_limit = 4096 default_process_limit = 4096 deliver_log_format = msgid="%m" subject="%s" from="%f" size=%p result="%$" first_valid_uid = 0 hostname = mailtest.geneseo.edu instance_name = mailtest lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_log_format_elements = user=%u method=%m rip=%r lip=%l lport=%a mpid=%e encryption=%c mail_fsync = always mail_location = maildir:/Mail/mailhome/%Ln/mailtest/Maildir:CONTROL=/Mail/mailhome/%Ln/mailtest/.dovecot-control:INDEX=/Mail/mailhome/%Ln/mailtest/.dovecot-index mail_log_prefix = "service=%s user=%u rip=%r " mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = zlib quota mail_log notify fts fts_squat stats maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl mmap_disable = yes namespace default { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = no special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = . type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes } passdb { args = cache_key=%u dovecot driver = pam } plugin { fts = squat mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_fields = uid box msgid size from subject flags mail_log_group_events = yes quota = maildir:User quota quota_exceeded_message = Quota exceeded (mailbox for user is full). Please see http://go.geneseo.edu/emailoverquota for help deleting messages while over quota. quota_rule = *:storage=200M quota_rule2 = Trash:storage=+50M sieve = /Mail/mailhome/%Ln/mailtest/.filter.sieve sieve_dir = /Mail/mailhome/%Ln/mailtest/.sievedir sieve_max_redirects = 25 stats_memory_limit = 32 M stats_refresh = 30 secs stats_track_cmds = yes } postmaster_address = postmaster at geneseo.edu protocols = imap sieve lmtp quota_full_tempfail = yes service auth { unix_listener auth-exim { group = exim mode = 0660 } } service imap-login { inet_listener imap { port = 1143 } inet_listener imaps { port = 1993 } service_count = 0 vsz_limit = 256 M } service imap { process_limit = 4096 } service lmtp { inet_listener lmtp { port = 124 } } service managesieve-login { inet_listener sieve { port = 14190 } inet_listener sieve_deprecated { port = 12000 } } service pop3 { process_limit = 4096 } service stats { fifo_listener stats-mail { mode = 0666 } } ssl_ca = References: <4F95B3DD.1070609@allycomm.com> <4F962028.3060501@allycomm.com> <4EE7C62E-928C-499D-BE4E-9944D6013198@iki.fi> <4F96DD43.3070700@allycomm.com> <78F42D61-1402-408B-953B-F85254479761@iki.fi> Message-ID: <4F98091D.5070902@allycomm.com> On 04/24/2012 10:08 AM, Timo Sirainen wrote: > >>>> 25934 jeff 1 103 0 135M 102M CPU2 2 128:47 100.00% doveadm >>> Looks like it could be in an infinite loop. To find out where: >>> > That's a lot of getdirentries() calls in there, repeating all the time. So I don't think it's an infinite loop, just a lot of slowness repeating the same work over and over again. See if it helps if you add: > > maildir_very_dirty_syncs = yes I hope I understood correctly when I tried $ dsync -Dv -o maildir_very_dirty_syncs=yes backup maildir:~/Maildir into fresh directories (moving the previous Maildir aside). I didn't see any significant change (and it did not complete in several hours) on the usual UFS directory, and have also tried ZFS to see if it might be a filesystem issue. That has not completed either after over nine hours. From what I can see, I am guessing that it is "hanging" in index creation. Below, "mail" is the source directory, and "Maildir" is the destination of the sync. This is about nine hours of run time. [jeff at mail ~]$ ls -Al mail/.imap/cron_2010-07-10 total 9 -rw------- 1 jeff jeff 4910320 Feb 22 2011 dovecot.index -rw------- 1 jeff jeff 3600384 Apr 25 01:18 dovecot.index.cache -rw------- 1 jeff jeff 56 Feb 22 2011 dovecot.index.log [jeff at mail ~]$ ls -Al Maildir/.cron_2010-07-10 total 26 drwx------ 2 jeff jeff 102291 Apr 25 01:17 cur -rw------- 1 jeff jeff 36 Apr 25 01:18 dovecot-keywords -rw------- 1 jeff jeff 4122971 Apr 25 01:17 dovecot-uidlist -rw------- 1 jeff jeff 24 Apr 25 07:19 dovecot-uidlist.lock -rw------- 1 jeff jeff 1637008 Apr 25 07:03 dovecot.index -rw------- 1 jeff jeff 9442304 Apr 25 07:19 dovecot.index.cache -rw------- 1 jeff jeff 87984 Apr 25 07:19 dovecot.index.log -rw------- 1 jeff jeff 1048600 Apr 25 07:03 dovecot.index.log.2 -rw------- 1 jeff jeff 0 Apr 24 22:12 maildirfolder drwx------ 2 jeff jeff 2 Apr 24 22:12 new drwx------ 2 jeff jeff 2 Apr 25 01:17 tmp From tss at iki.fi Wed Apr 25 17:30:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 17:30:27 +0300 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F98073C.3080406@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> Message-ID: On 25.4.2012, at 17.16, Robert Schetterer wrote: > Am 25.04.2012 16:11, schrieb Timo Sirainen: >> p *set > > hope doing it right > > (gdb) p *set > $1 = {postmaster_address = 0xaa33d0 "P6\252", hostname = 0x0, > submission_host = 0x0, sendmail_path = 0x7fbaf6e41f50 > "H\211\\$\360H\211l$\370H\203\354\030\200=\207A%", > rejection_subject = 0x7fbaf6e42b90 > "H\211\\$\350L\211d$\370H\211\373H\211l$\360H\203\354(\200=?5%", > rejection_reason = 0x7fbaf6e41ed0 That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? From robert at schetterer.org Wed Apr 25 17:38:56 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 16:38:56 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> Message-ID: <4F980C80.2030908@schetterer.org> Am 25.04.2012 16:30, schrieb Timo Sirainen: > On 25.4.2012, at 17.16, Robert Schetterer wrote: > >> Am 25.04.2012 16:11, schrieb Timo Sirainen: >>> p *set >> >> hope doing it right >> >> (gdb) p *set >> $1 = {postmaster_address = 0xaa33d0 "P6\252", hostname = 0x0, >> submission_host = 0x0, sendmail_path = 0x7fbaf6e41f50 >> "H\211\\$\360H\211l$\370H\203\354\030\200=\207A%", >> rejection_subject = 0x7fbaf6e42b90 >> "H\211\\$\350L\211d$\370H\211\373H\211l$\360H\203\354(\200=?5%", >> rejection_reason = 0x7fbaf6e41ed0 > > That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? > its from latest http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ recompiled the debain way , never had bugged -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Wed Apr 25 17:43:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 17:43:06 +0300 Subject: [Dovecot] stats + fts squat plugins, indexer-worker error on message indexing In-Reply-To: References: Message-ID: <8F39E28F-3079-4EB1-BCAD-0A5C78DA05DC@iki.fi> On 25.4.2012, at 17.20, David Warden wrote: > Apr 25 09:56:26 wardentest3 dovecot: service=indexer-worker user=warden rip= Error: open(/proc/self/io) failed: Permission .. > Based on http://wiki2.dovecot.org/Services#indexer-worker it seems like this is because the indexer-worker process is created as root but at the time of indexing the indexer-worker drops privs to the mail user. In /proc//, io is mode 0400 and owned by root, so it makes sense that the mail user would be unable to read that file. Does the attached patch fix it? -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 2639 bytes Desc: not available URL: -------------- next part -------------- > The strange thing is it looks like the stats are still collected (unless I am reading the output of the stats dump incorrectly): > > [root at wardentest3 mailtest]# doveadm stats dump -s /var/run/dovecot/mailtest/stats command user=warden > cmd args session user last_update user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits > select inbox 27fb87218302984f084c0000ae899e04 warden 1335362182.950250 0.000000 0.000000 172 0 46 0 0 12288 13 7883 4 381 0 0 0 0 0 > search text "warden" 27fb87218302984f084c0000ae899e04 warden 1335362186.588023 0.001000 0.000000 32 0 4 1 0 0 4 108 1 43 0 0 0 0 0 > logout 27fb87218302984f084c0000ae899e04 warden 1335362189.161790 0.000000 0.000000 1 0 0 3 0 0 1 108 3 157 0 0 0 0 0 These are IMAP command statistics. The problem was only with indexer-worker. From xchris89x at googlemail.com Wed Apr 25 17:47:16 2012 From: xchris89x at googlemail.com (Chris) Date: Wed, 25 Apr 2012 16:47:16 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F980C80.2030908@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> <4F980C80.2030908@schetterer.org> Message-ID: 2012/4/25 Robert Schetterer : >> That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? >> > > its from latest > http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ > recompiled the debain way , never had bugged You are using packages compiled against debian sources under ubuntu??? -- Chris From robert at schetterer.org Wed Apr 25 17:50:16 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 16:50:16 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> <4F980C80.2030908@schetterer.org> Message-ID: <4F980F28.90205@schetterer.org> Am 25.04.2012 16:47, schrieb Chris: > 2012/4/25 Robert Schetterer : >>> That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? >>> >> >> its from latest >> http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ >> recompiled the debain way , never had bugged > > You are using packages compiled against debian sources under ubuntu??? > > -- > Chris jep that works since years -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From almarzuki2001 at hotmail.com Wed Apr 25 17:52:39 2012 From: almarzuki2001 at hotmail.com (Hadi Salem) Date: Wed, 25 Apr 2012 17:52:39 +0300 Subject: [Dovecot] dovecot-lda error Message-ID: Hi, im getting the fallowing error. from dovecot-lda log Apr 25 17:30:41 lda(jamal at example.com): Fatal: setgid(5003(vmail) from userdb lookup) failed with euid=8(mail), gid=12(mail), egid=12(mail): Operation not permitted (This binary should probably be called with process group set to 5003(vmail) instead of 12(mail)) this error occur if im using different uid and gid part from mail uid.. jamal at example.com:{PLAIN}123:5003:5003::/var/vmail/example.com/jamal/::userdb_mail=maildir:~/Maildir but if i use mail uid it work fine.. jamal at example.com:{PLAIN}123:8:12::/var/vmail/example.com/jamal/::userdb_mail=maildir:~/Maildir i tried using multiple uids as mention in wiki2.dovecot.org/LDA bye setting up setuid-root for dovecot-lda but im getting on-there error from sendmail :- Apr 25 17:30:41 host1 sendmail[31024]: q3PEUen2031022: to=, delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120359, relay=dovecot, dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/local/libexec/dovecot/dovecot-lda) exited with EX_TEMPFAIL im using sendmail. [root at host1 conf.d]# doveconf -n # 2.0.20: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-164.el5 x86_64 CentOS release 5.4 (Final) auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_verbose_passwords = plain debug_log_path = /var/log/dovecot.log disable_plaintext_auth = no first_valid_gid = 12 first_valid_uid = 8 hostname = host1.bigmama.com info_log_path = /var/log/dovecot.log last_valid_gid = 50000 last_valid_uid = 50000 log_path = /var/log/dovecot.log mail_debug = yes mail_gid = mail mail_location = maildir:~/Maildir mail_uid = mail passdb { args = scheme=PLAIN username_format=%u /etc/dovecot/users driver = passwd-file } postmaster_address = root at host1.bigmama.com service auth { unix_listener auth-userdb { group = mail mode = 0666 user = mail } } ssl = no userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol lda { info_log_path = /var/log/dovecot-lda.log log_path = /var/log/dovecot-lda-errors.log } From xchris89x at googlemail.com Wed Apr 25 17:54:03 2012 From: xchris89x at googlemail.com (Chris) Date: Wed, 25 Apr 2012 16:54:03 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F980F28.90205@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> <4F980C80.2030908@schetterer.org> <4F980F28.90205@schetterer.org> Message-ID: 2012/4/25 Robert Schetterer : > Am 25.04.2012 16:47, schrieb Chris: >> 2012/4/25 Robert Schetterer : >>>> That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? >>>> >>> >>> its from latest >>> http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ >>> recompiled the debain way , never had bugged >> >> You are using packages compiled against debian sources under ubuntu??? >> >> -- >> Chris > > jep that works since years Thats really a very bad idea. Please recompile it under ubuntu 10.04. -- Chris From robert at schetterer.org Wed Apr 25 17:59:21 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 16:59:21 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> <4F980C80.2030908@schetterer.org> <4F980F28.90205@schetterer.org> Message-ID: <4F981149.4070102@schetterer.org> Am 25.04.2012 16:54, schrieb Chris: > 2012/4/25 Robert Schetterer : >> Am 25.04.2012 16:47, schrieb Chris: >>> 2012/4/25 Robert Schetterer : >>>>> That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? >>>>> >>>> >>>> its from latest >>>> http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ >>>> recompiled the debain way , never had bugged >>> >>> You are using packages compiled against debian sources under ubuntu??? >>> >>> -- >>> Chris >> >> jep that works since years > > Thats really a very bad idea. Please recompile it under ubuntu 10.04. you miss understand i have it compiled under ubuntu 10.04 > > -- > Chris -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From joseba.torre at ehu.es Wed Apr 25 18:27:44 2012 From: joseba.torre at ehu.es (Joseba Torre) Date: Wed, 25 Apr 2012 17:27:44 +0200 Subject: [Dovecot] Error compiling pigeonhole 0.3.0 with dovecot-2.1.5 in openbsd 5.0 In-Reply-To: <4F968AF2.4090807@rename-it.nl> References: <4F967192.2060701@ehu.es> <4F968AF2.4090807@rename-it.nl> Message-ID: <4F9817F0.7030908@ehu.es> > > Looks like BSD doesn't know EPROTO (although it is POSIX). Dovecot uses > EIO elsewhere for such stream errors, so I adopted that for ManageSieve > as well. This should fix it: > > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/ef5045159e6b > > Regards, Perfect, thank you From robert at schetterer.org Wed Apr 25 18:48:20 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 17:48:20 +0200 Subject: [Dovecot] sieve vacation redirect problem / solved In-Reply-To: References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> Message-ID: <4F981CC4.3010600@schetterer.org> Am 25.04.2012 16:30, schrieb Timo Sirainen: > On 25.4.2012, at 17.16, Robert Schetterer wrote: > >> Am 25.04.2012 16:11, schrieb Timo Sirainen: >>> p *set >> >> hope doing it right >> >> (gdb) p *set >> $1 = {postmaster_address = 0xaa33d0 "P6\252", hostname = 0x0, >> submission_host = 0x0, sendmail_path = 0x7fbaf6e41f50 >> "H\211\\$\360H\211l$\370H\203\354\030\200=\207A%", >> rejection_subject = 0x7fbaf6e42b90 >> "H\211\\$\350L\211d$\370H\211\373H\211l$\360H\203\354(\200=?5%", >> rejection_reason = 0x7fbaf6e41ed0 > > That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? > jep it was broken , no idea why serveral new downloads give same result using http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz with recompile debian way ( same scripts etc ) to recent downloaded dovecot sources http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ works now no idea whats going on , but works now -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From weston.kris at ymail.com Wed Apr 25 18:52:12 2012 From: weston.kris at ymail.com (Kris Weston) Date: Wed, 25 Apr 2012 16:52:12 +0100 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <5B92F036-DDBE-42AB-94A1-DBF21F11DE14@iki.fi> References: <4F9796EE.5070907@ymail.com> <5B92F036-DDBE-42AB-94A1-DBF21F11DE14@iki.fi> Message-ID: <4F981DAC.8000106@ymail.com> On 25/04/12 11:48, Timo Sirainen wrote: > On 25.4.2012, at 9.17, Kris wrote: > >> but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user at domain): >> localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' > What exactly is doing this lookup? SMTP AUTH? Maybe the client's SMTP authentication is configured without the @domain part in username? > saslauthd via a query in dovecot-sql.conf.ext? after these suggestions im pretty sure nobody here has a clue about this, and this is not a known problem, i cant find anyone on the internet with it and the query is quite clearly coming up wrong and the unchangeable variable is quite clearly changing. the client can not be sending out different usernames every second surely. i put the username in myself in thunderbird, double, triple checked, made other accounts etc, the username changes by the second, ive watched the sql queries go in and they change for no obvious reason. From tss at iki.fi Wed Apr 25 18:55:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 18:55:40 +0300 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F981DAC.8000106@ymail.com> References: <4F9796EE.5070907@ymail.com> <5B92F036-DDBE-42AB-94A1-DBF21F11DE14@iki.fi> <4F981DAC.8000106@ymail.com> Message-ID: <8B6B7B68-278A-44C5-9013-AA14CCA991CD@iki.fi> On 25.4.2012, at 18.52, Kris Weston wrote: > On 25/04/12 11:48, Timo Sirainen wrote: >> On 25.4.2012, at 9.17, Kris wrote: >> >>> but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user at domain): >>> localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' >> What exactly is doing this lookup? SMTP AUTH? Maybe the client's SMTP authentication is configured without the @domain part in username? >> > saslauthd via a query in dovecot-sql.conf.ext? There is no saslauthd in Dovecot. > after these suggestions im pretty sure nobody here has a clue about this, and this is not a known problem, i cant find anyone on the internet with it and the query is quite clearly coming up wrong and the unchangeable variable is quite clearly changing. To me it looks like the client isn't sending the domain. > the client can not be sending out different usernames every second surely. i put the username in myself in thunderbird, double, triple checked, made other accounts etc, the username changes by the second, ive watched the sql queries go in and they change for no obvious reason. Show the full logs for a failing session, not just this one line. Even better would be full logs with auth_debug_passwords=yes, but in that case you'd better be using an unimportant password (some of the base64 encoded strings contain the password). From postmaster at mathiesen.info Wed Apr 25 20:56:18 2012 From: postmaster at mathiesen.info (postmaster) Date: Wed, 25 Apr 2012 19:56:18 +0200 Subject: [Dovecot] 2.1.5 problem with mail_location Message-ID: <4F983AC2.5010007@mathiesen.info> I've been advised to set : mail_location = maildir:~/ as the home directory is found through a sql request. this has been working perfectly up to and including 2.1.4 from 2.1.5 however this setting gives cause to a problem : instead of looking in the mail home eg /var/mail/domain/user\@domain/ dovecot is looking for mail in /var/mail/domain/user\@domain/~ the '~' directory is created upon the first request for mail and all future mail transactions in dovecot is done relative to this folder. is this behavior a bug or a feature ??? -- Bjarne D Mathiesen K?benhavn N ; Danmark ; Europa ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ denne besked er skrevet i et totalt M$-frit milj? MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3 From robert at schetterer.org Wed Apr 25 21:02:28 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 20:02:28 +0200 Subject: [Dovecot] sieve vacation redirect problem / solved / restested with lates hg pigeonhole broken In-Reply-To: <4F981CC4.3010600@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> <4F981CC4.3010600@schetterer.org> Message-ID: <4F983C34.4030605@schetterer.org> Am 25.04.2012 17:48, schrieb Robert Schetterer: > Am 25.04.2012 16:30, schrieb Timo Sirainen: >> On 25.4.2012, at 17.16, Robert Schetterer wrote: >> >>> Am 25.04.2012 16:11, schrieb Timo Sirainen: >>>> p *set >>> >>> hope doing it right >>> >>> (gdb) p *set >>> $1 = {postmaster_address = 0xaa33d0 "P6\252", hostname = 0x0, >>> submission_host = 0x0, sendmail_path = 0x7fbaf6e41f50 >>> "H\211\\$\360H\211l$\370H\203\354\030\200=\207A%", >>> rejection_subject = 0x7fbaf6e42b90 >>> "H\211\\$\350L\211d$\370H\211\373H\211l$\360H\203\354(\200=?5%", >>> rejection_reason = 0x7fbaf6e41ed0 >> >> That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? >> > > jep it was broken , no idea why serveral new downloads give same result > using > http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz > with recompile debian way ( same scripts etc ) to recent downloaded > dovecot sources > http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ > > works now > > no idea whats going on , but works now > just for info, i retested again with hg code latest hg clone http://hg.rename-it.nl/dovecot-2.1-pigeonhole/ compile runs ok... dpkg-buildpackage -rfakeroot -b ( on ubuntu 64 lucid latest patchlevel from today ) but at dpkg install it gives warning /usr/lib/dovecot/managesieve: symbol lookup error: /usr/lib/dovecot/managesieve: undefined symbol: sieve_file_script doveconf: Error: managesieve-login: dump-capability process returned 127 dont know if there is the bug reason however my test showes up described failures with latest hg code then sieve vacation redirect got broken it works if i use http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz for compile thx for your help and code anywhere, hope this results may help for debug -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Apr 25 21:16:58 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 20:16:58 +0200 Subject: [Dovecot] doveadm delete folders ? / unsubscribe ? In-Reply-To: <4F9795EE.4030604@schetterer.org> References: <4F900066.9030808@schetterer.org> <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> <4F906EE8.1030900@schetterer.org> <4F965B2E.2010103@schetterer.org> <283E4240-57E8-4BF2-8DCA-6DCC78C1D5A8@iki.fi> <4F9795EE.4030604@schetterer.org> Message-ID: <4F983F9A.5040900@schetterer.org> Am 25.04.2012 08:13, schrieb Robert Schetterer: > Am 24.04.2012 20:13, schrieb Timo Sirainen: >> On 24.4.2012, at 10.50, Robert Schetterer wrote: >> >>> doveadm expunge -d -u *@domain.example mailbox exampelfolder\* >>> savedbefore 10d >>> >>> works like charm, but the folders do not get unsubscribed >>> any chance to have i.e another add parameter ,or do it auto >>> cause empty and deleted folders should not stay subscribed in my eyes >> >> Yeah, I guess unsubscribing is always wanted: >> >> http://hg.dovecot.org/dovecot-2.1/rev/b1653b7bd165 >> > > Hi Timo, i will try after update and report, jep > unsubscribing should be the uni wanted behave > thx again ! tested works nice thx Timo ! -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From weston.kris at ymail.com Wed Apr 25 21:28:42 2012 From: weston.kris at ymail.com (Kris Weston) Date: Wed, 25 Apr 2012 19:28:42 +0100 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <8B6B7B68-278A-44C5-9013-AA14CCA991CD@iki.fi> References: <4F9796EE.5070907@ymail.com> <5B92F036-DDBE-42AB-94A1-DBF21F11DE14@iki.fi> <4F981DAC.8000106@ymail.com> <8B6B7B68-278A-44C5-9013-AA14CCA991CD@iki.fi> Message-ID: <4F98425A.6020908@ymail.com> >>>> but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user at domain): >>>> localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' >>> What exactly is doing this lookup? SMTP AUTH? Maybe the client's SMTP authentication is configured without the @domain part in username? >>> >> saslauthd via a query in dovecot-sql.conf.ext? > There is no saslauthd in Dovecot. thats correct, its not in dovecot, its installed on the same machine. but regardless, its salsauthd. >> after these suggestions im pretty sure nobody here has a clue about this, and this is not a known problem, i cant find anyone on the internet with it and the query is quite clearly coming up wrong and the unchangeable variable is quite clearly changing. > To me it looks like the client isn't sending the domain. so the client is changing its domain every coupla seconds for what reason, surely the thunderbird devs would have found this error? and even if thunderbird is sending it without the domain, the server should deal with thunderbird correctly, the user should not have to configure her thunderbird to work with dovecot rather than the other way round ? i dont believe this is the problem at all. >> the client can not be sending out different usernames every second surely. i put the username in myself in thunderbird, double, triple checked, made other accounts etc, the username changes by the second, ive watched the sql queries go in and they change for no obvious reason. > Show the full logs for a failing session, not just this one line. Even better would be full logs with auth_debug_passwords=yes, but in that case you'd better be using an unimportant password (some of the base64 encoded strings contain the password). thanks for trying, but forget it, its clear to me from answers ive received that nobody has a clue about this problem. guess im on me own. cheers bye! From warden at geneseo.edu Wed Apr 25 21:30:06 2012 From: warden at geneseo.edu (David Warden) Date: Wed, 25 Apr 2012 14:30:06 -0400 Subject: [Dovecot] stats + fts squat plugins, indexer-worker error on message indexing In-Reply-To: <8F39E28F-3079-4EB1-BCAD-0A5C78DA05DC@iki.fi> References: <8F39E28F-3079-4EB1-BCAD-0A5C78DA05DC@iki.fi> Message-ID: <86EED9D8-8445-49B4-9D72-C77883F926F9@geneseo.edu> On Apr 25, 2012, at 10:43 AM, Timo Sirainen wrote: > On 25.4.2012, at 17.20, David Warden wrote: > >> Apr 25 09:56:26 wardentest3 dovecot: service=indexer-worker user=warden rip= Error: open(/proc/self/io) failed: Permission > .. >> Based on http://wiki2.dovecot.org/Services#indexer-worker it seems like this is because the indexer-worker process is created as root but at the time of indexing the indexer-worker drops privs to the mail user. In /proc//, io is mode 0400 and owned by root, so it makes sense that the mail user would be unable to read that file. > > Does the attached patch fix it? > > The error is still logged, but is that intentional on your part? It looks like the patch logs the failure to open() when the error returned is anything other than ENOENT and in my case the error is EACCESS. Other than that it looks like it correctly disables accessing that file. >> The strange thing is it looks like the stats are still collected (unless I am reading the output of the stats dump incorrectly): >> >> [root at wardentest3 mailtest]# doveadm stats dump -s /var/run/dovecot/mailtest/stats command user=warden >> cmd args session user last_update user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits >> select inbox 27fb87218302984f084c0000ae899e04 warden 1335362182.950250 0.000000 0.000000 172 0 46 0 0 12288 13 7883 4 381 0 0 0 0 0 >> search text "warden" 27fb87218302984f084c0000ae899e04 warden 1335362186.588023 0.001000 0.000000 32 0 4 1 0 0 4 108 1 43 0 0 0 0 0 >> logout 27fb87218302984f084c0000ae899e04 warden 1335362189.161790 0.000000 0.000000 1 0 0 3 0 0 1 108 3 157 0 0 0 0 0 > > These are IMAP command statistics. The problem was only with indexer-worker. From tss at iki.fi Wed Apr 25 21:55:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 21:55:12 +0300 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F98425A.6020908@ymail.com> References: <4F9796EE.5070907@ymail.com> <5B92F036-DDBE-42AB-94A1-DBF21F11DE14@iki.fi> <4F981DAC.8000106@ymail.com> <8B6B7B68-278A-44C5-9013-AA14CCA991CD@iki.fi> <4F98425A.6020908@ymail.com> Message-ID: On 25.4.2012, at 21.28, Kris Weston wrote: >>>>> but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user at domain): >>>>> localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' >>>> What exactly is doing this lookup? SMTP AUTH? Maybe the client's SMTP authentication is configured without the @domain part in username? >>>> >>> saslauthd via a query in dovecot-sql.conf.ext? >> There is no saslauthd in Dovecot. > > thats correct, its not in dovecot, its installed on the same machine. > but regardless, its salsauthd. saslauthd doesn't authenticate via Dovecot. It might authenticate via IMAP though. > thanks for trying, but forget it, its clear to me from answers ive received that nobody has a clue about this problem. guess im on me own. cheers bye! The reason why nobody has a clue is because you haven't provided the full logs. We can only guess what the problem is. With full logs there would be no guessing necessary. From tss at iki.fi Wed Apr 25 22:09:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 22:09:09 +0300 Subject: [Dovecot] stats + fts squat plugins, indexer-worker error on message indexing In-Reply-To: <86EED9D8-8445-49B4-9D72-C77883F926F9@geneseo.edu> References: <8F39E28F-3079-4EB1-BCAD-0A5C78DA05DC@iki.fi> <86EED9D8-8445-49B4-9D72-C77883F926F9@geneseo.edu> Message-ID: On 25.4.2012, at 21.30, David Warden wrote: >>> Apr 25 09:56:26 wardentest3 dovecot: service=indexer-worker user=warden rip= Error: open(/proc/self/io) failed: Permission >> >> Does the attached patch fix it? > > The error is still logged, but is that intentional on your part? It looks like the patch logs the failure to open() when the error returned is anything other than ENOENT and in my case the error is EACCESS. Other than that it looks like it correctly disables accessing that file. The patch intended to open the /proc/self/io immediately at startup while still running as root. Anyway, now that I'm testing it again myself, I see that it's supposed to work even without the patch, because /proc/self/io's permissions change after seteuid(). I noticed similar errors happening somewhat randomly in another CentOS 6.2 system. Changing the kernel fixed it. (Downgrading at least worked, possibly upgrading might also.) From tss at iki.fi Wed Apr 25 23:46:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 23:46:08 +0300 Subject: [Dovecot] 2.1.5 problem with mail_location In-Reply-To: <4F983AC2.5010007@mathiesen.info> References: <4F983AC2.5010007@mathiesen.info> Message-ID: <1B191C98-90A2-40B6-ACDF-2420E9A5161B@iki.fi> On 25.4.2012, at 20.56, postmaster wrote: > I've been advised to set : > mail_location = maildir:~/ > as the home directory is found through a sql request. > > this has been working perfectly up to and including 2.1.4 > > from 2.1.5 however this setting gives cause to a problem : > instead of looking in the mail home eg /var/mail/domain/user\@domain/ dovecot is looking for mail in /var/mail/domain/user\@domain/~ > > the '~' directory is created upon the first request for mail and all future mail transactions in dovecot is done relative to this folder. Yeah, it's a bug. Fixed and added to tests so it won't break in future: http://hg.dovecot.org/dovecot-2.1/rev/249c1c89d9d3 From tss at iki.fi Thu Apr 26 00:16:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:16:45 +0300 Subject: [Dovecot] v2.1.5 released In-Reply-To: References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: On 24.4.2012, at 11.59, Gerhard Wiesinger wrote: > Hello Timo, > > Dovecot 2.1.5 seems to be broken. After upgrade from > 2.1.3 + patch http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e > I only see Inbox and Trash folder. > > Reverting => OK. > > Same config as before and disucssed per private E-mail and on the list. Guessing this comes again from: > mail_full_filesystem_access=yes I guess the problem was this: http://hg.dovecot.org/dovecot-2.1/rev/249c1c89d9d3 From Ralf.Hildebrandt at charite.de Thu Apr 26 00:26:10 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 25 Apr 2012 23:26:10 +0200 Subject: [Dovecot] v2.1.5 released In-Reply-To: References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <20120425212610.GA5653@charite.de> * Timo Sirainen : > On 24.4.2012, at 11.59, Gerhard Wiesinger wrote: > > > Hello Timo, > > > > Dovecot 2.1.5 seems to be broken. After upgrade from > > 2.1.3 + patch http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e > > I only see Inbox and Trash folder. > > > > Reverting => OK. > > > > Same config as before and disucssed per private E-mail and on the list. Guessing this comes again from: > > mail_full_filesystem_access=yes > > I guess the problem was this: http://hg.dovecot.org/dovecot-2.1/rev/249c1c89d9d3 2.1.5 doesn't seem to build with pigeonhole-sieve (or it's pigeonhole-sieve itself?): /bin/bash ../../libtool --tag=CC --mode=link gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -export-dynamic -g -Wl,--as-needed -o sievec sievec.o ../../src/lib-sieve/libdovecot-sieve.la ../../src/lib-sieve-tool/libsieve-tool.la /usr/src/dovecot-2.1/dovecot-2.1/src/lib-storage/libdovecot-storage.la -lssl -lcrypto /usr/src/dovecot-2.1/dovecot-2.1/src/lib-lda/libdovecot-lda.la /usr/src/dovecot-2.1/dovecot-2.1/src/lib-dovecot/libdovecot.la -export-dynamic -ldl -lrt libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -g -Wl,--as-needed -o .libs/sievec sievec.o -Wl,--export-dynamic ../../src/lib-sieve/.libs/libdovecot-sieve.so ../../src/lib-sieve-tool/.libs/libsieve-tool.a /usr/src/dovecot-2.1/dovecot-2.1/src/lib-storage/.libs/libdovecot-storage.so -lssl -lcrypto /usr/src/dovecot-2.1/dovecot-2.1/src/lib-lda/.libs/libdovecot-lda.so /usr/src/dovecot-2.1/dovecot-2.1/src/lib-dovecot/.libs/libdovecot.so -ldl -lrt -Wl,-rpath -Wl,/usr/local/dovecot-2.1/lib/dovecot ../../src/lib-sieve/.libs/libdovecot-sieve.so: error: undefined reference to 'sieve_file_script' ../../src/lib-sieve/.libs/libdovecot-sieve.so: error: undefined reference to 'sieve_dict_script' sievec.c:116: error: undefined reference to 'sieve_scriptfile_has_extension' collect2: ld returned 1 exit status make[3]: *** [sievec] Error 1 make[3]: Leaving directory /usr/src/dovecot-2.1/dovecot-2.1-pigeonhole/src/sieve-tools' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory /usr/src/dovecot-2.1/dovecot-2.1-pigeonhole/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory /usr/src/dovecot-2.1/dovecot-2.1-pigeonhole' make: *** [all] Error 2 Making install in . -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Thu Apr 26 00:27:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:27:25 +0300 Subject: [Dovecot] Permission problems when trying to create new folders with ACL plugin In-Reply-To: <20120424130005.0db3d425@legolas.home.ceotex.de> References: <20120424130005.0db3d425@legolas.home.ceotex.de> Message-ID: <7D3AF25B-FAE0-4187-B53C-DBB477A4AE79@iki.fi> On 24.4.2012, at 14.00, Markus Petri wrote: > I have the a problem with shared folders and the ACL plugin. The user > "markus" has full access to "Shared/shared-user/Data": > > 3 GETACL "Shared/shared-user/Data" > * ACL "Shared/shared-user/Data" "markus" akxeilprwtscd > 3 OK Getacl completed. > > but creating a new subfolder under "Data" fails: > > 4 CREATE "Shared/shared-user/Data/New1" > 4 NO [NOPERM] Permission denied Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e8b80e0767ac From tss at iki.fi Thu Apr 26 00:27:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:27:45 +0300 Subject: [Dovecot] acls not copied when creating subfolder of private INBOX In-Reply-To: <4F96B436.9090402@freenet.de> References: <4F96B436.9090402@freenet.de> Message-ID: <06DE7611-7783-4E5A-B58A-1448115802F5@iki.fi> On 24.4.2012, at 17.09, Andreas Helmcke wrote: > Using brand new dovecot 2.1.5 I still have a problem with the inheritance of acls. Maybe http://hg.dovecot.org/dovecot-2.1/rev/e8b80e0767ac fixes this as well? From tss at iki.fi Thu Apr 26 00:28:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:28:55 +0300 Subject: [Dovecot] Reading Mutt mboxes from Thunderbird In-Reply-To: <20120424160712.GB5048@leonardo-laptop> References: <20120424160712.GB5048@leonardo-laptop> Message-ID: On 24.4.2012, at 19.07, Leonardo M. Ram? wrote: > Hi, I'm a user of Mutt, and before moving to Thunderbird I'm trying to > share mboxes between them. To do this, I installed Dovecot to create an > IMAP server in my local machine, to serve mutt mboxes to be read by > Thunderbird. > > So far, I can read my inbox from /var/mail/%u, but in mutt, I have many > folders in ~/mail, like 2010-mails, 2011-mails, mailing-list-x, > mailing-list-y, and so on. How can I configure Dovecot to read them?. > > In /etc/dovecot/conf.d/10-mail.conf I have this: > > mail_location = mbox:~/Mail:INBOX=/var/mail/%u ~/mail vs. ~/Mail? From tss at iki.fi Thu Apr 26 00:29:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:29:51 +0300 Subject: [Dovecot] dovecot LDA error In-Reply-To: References: Message-ID: On 24.4.2012, at 20.41, Hadi Salem wrote: > Apr 24 20:25:46 lda(jamal at example.com): Fatal: > setgid(5000(vmail) from userdb lo > okup) failed with > euid=8(mail), gid=12(mail), egid=12(mail): Operation not permi > tted (This binary should probably be called with process group set to 5000(vmail > ) instead of 12(mail)) Configure your MTA to run dovecot-lda as vmail instead of as mail. From tss at iki.fi Thu Apr 26 00:31:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:31:29 +0300 Subject: [Dovecot] dovecot-lda error In-Reply-To: References: Message-ID: <1353D406-AEF1-46A1-ADB8-DFC159EC9946@iki.fi> On 25.4.2012, at 17.52, Hadi Salem wrote: > i tried using multiple uids as mention in wiki2.dovecot.org/LDA bye setting up setuid-root for dovecot-lda but im getting on-there error from sendmail :- > > Apr 25 17:30:41 host1 sendmail[31024]: q3PEUen2031022: to=, delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120359, relay=dovecot, dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/local/libexec/dovecot/dovecot-lda) exited with EX_TEMPFAIL What do the Dovecot logs say about this? > protocol lda { > info_log_path = /var/log/dovecot-lda.log > log_path = /var/log/dovecot-lda-errors.log > } If there is nothing in dovecot-lda-errors.log, make sure dovecot-lda has permissions to write there (e.g. temporarily make it world-writable). From tss at iki.fi Thu Apr 26 00:33:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:33:13 +0300 Subject: [Dovecot] v2.1.5 released In-Reply-To: <20120425212610.GA5653@charite.de> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> <20120425212610.GA5653@charite.de> Message-ID: On 26.4.2012, at 0.26, Ralf Hildebrandt wrote: > 2.1.5 doesn't seem to build with pigeonhole-sieve (or it's > pigeonhole-sieve itself?): Builds against Pigeonhole 0.3.0 tarball at least. From Ralf.Hildebrandt at charite.de Thu Apr 26 00:35:20 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 25 Apr 2012 23:35:20 +0200 Subject: [Dovecot] v2.1.5 released In-Reply-To: References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> <20120425212610.GA5653@charite.de> Message-ID: <20120425213520.GB5653@charite.de> * Timo Sirainen : > On 26.4.2012, at 0.26, Ralf Hildebrandt wrote: > > > 2.1.5 doesn't seem to build with pigeonhole-sieve (or it's > > pigeonhole-sieve itself?): > > Builds against Pigeonhole 0.3.0 tarball at least. Thought so. I'm rebuilding against that one while I write this :O -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Thu Apr 26 00:58:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:58:13 +0300 Subject: [Dovecot] v2.1.5 released In-Reply-To: <20120425212610.GA5653@charite.de> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> <20120425212610.GA5653@charite.de> Message-ID: <05BAF7A2-FFEC-4684-BEEC-B4B22F684114@iki.fi> On 26.4.2012, at 0.26, Ralf Hildebrandt wrote: > * Timo Sirainen : > 2.1.5 doesn't seem to build with pigeonhole-sieve (or it's > pigeonhole-sieve itself?): > > ../../src/lib-sieve/.libs/libdovecot-sieve.so: error: undefined reference to 'sieve_file_script' > ../../src/lib-sieve/.libs/libdovecot-sieve.so: error: undefined reference to 'sieve_dict_script' That's from hg, and you didn't run autogen.sh? ./configure --enable-maintainer-mode also helps avoiding this. From Ralf.Hildebrandt at charite.de Thu Apr 26 01:02:42 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Thu, 26 Apr 2012 00:02:42 +0200 Subject: [Dovecot] v2.1.5 released In-Reply-To: <05BAF7A2-FFEC-4684-BEEC-B4B22F684114@iki.fi> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> <20120425212610.GA5653@charite.de> <05BAF7A2-FFEC-4684-BEEC-B4B22F684114@iki.fi> Message-ID: <20120425220242.GY5653@charite.de> * Timo Sirainen : > On 26.4.2012, at 0.26, Ralf Hildebrandt wrote: > > > * Timo Sirainen : > > 2.1.5 doesn't seem to build with pigeonhole-sieve (or it's > > pigeonhole-sieve itself?): > > > > ../../src/lib-sieve/.libs/libdovecot-sieve.so: error: undefined reference to 'sieve_file_script' > > ../../src/lib-sieve/.libs/libdovecot-sieve.so: error: undefined reference to 'sieve_dict_script' > > That's from hg, yes > and you didn't run autogen.sh? ./configure --enable-maintainer-mode > also helps avoiding this. Damn, forgot that in the pigeonhole dir (I have that for the dovecot sources, but not for the pigeonhole sources)... -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From daniel.parthey at informatik.tu-chemnitz.de Thu Apr 26 01:51:16 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 26 Apr 2012 00:51:16 +0200 Subject: [Dovecot] How to remove duplicate GUID messages from storage? In-Reply-To: References: <20120421202928.GA12998@daniel.localdomain> Message-ID: <20120425225116.GB10062@daniel.localdomain> Timo Sirainen schrieb: > On 21.4.2012, at 23.29, Daniel Parthey wrote: > > > The following command > > doveadm -v force-resync -u username at example.org INBOX > > outputs lots of messages like this one, with different GUID > > doveadm(username at example.org): Error: mdbox /mail/dovecot/example.org/username/mail/storage: Duplicate GUID 0b8a032d66a0924fb42c0000de5f8128 in m.55:45484041 and m.14:52173045 > > I think this is a force-resync bug and it shouldn't really complain about duplicates. > Although I'm not entirely sure why with you it's complaining about them at all. I have anyway in TODO to look into this.. Maybe this is because I "manually" deleted all the mailboxes directories (rm -rf) containing the duplicates, which one should better avoid with mdbox mailboxes... :( So there is no "meta information" anymore, just the duplicate messages in the storage. and dovecot would need to "guess" where these messages from the store belong. > > How can I manually remove these identical, duplicate messages > > from the storage to save space? Dovecot does not do it > > automatically. > > Perhaps force-resync + purge should do that, but currently it doesn't. I already tried both, and purge is even running as a nightly cronjob. > Also it would be nice if doveadm import didn't add duplicates in the first place. > This is also something for which I have vague plans, because it would help dsync as well. Thanks for looking into this. I would really appreciate this idea of "duplicate GUID prevention" while duplicate messages (with different GUIDs) should probably be allowed. Maybe someone wants to store the same message several times... Regards, Daniel From daniel.parthey at informatik.tu-chemnitz.de Thu Apr 26 01:56:04 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 26 Apr 2012 00:56:04 +0200 Subject: [Dovecot] Reading Mutt mboxes from Thunderbird In-Reply-To: References: <20120424160712.GB5048@leonardo-laptop> Message-ID: <20120425225604.GA10291@daniel.localdomain> Timo Sirainen schrieb: > On 24.4.2012, at 19.07, Leonardo M. Ram? wrote: > > > Hi, I'm a user of Mutt, and before moving to Thunderbird I'm trying to > > share mboxes between them. To do this, I installed Dovecot to create an > > IMAP server in my local machine, to serve mutt mboxes to be read by > > Thunderbird. > > > > So far, I can read my inbox from /var/mail/%u, but in mutt, I have many > > folders in ~/mail, like 2010-mails, 2011-mails, mailing-list-x, > > mailing-list-y, and so on. How can I configure Dovecot to read them?. > > > > In /etc/dovecot/conf.d/10-mail.conf I have this: > > > > mail_location = mbox:~/Mail:INBOX=/var/mail/%u > > ~/mail vs. ~/Mail? On 2012-04-25 04:26:27 +0200, Daniel Parthey wrote: > Are you sure, dovecot does not read them, or are you just not "seeing" > your folders in thunderbird? Please try to right click your INBOX and > "subscribe" to the folders. On serverside, the subscribed folders are > listed in $mail_location/subscriptions, so you might auto-generate > this file by listing your mutt mailboxes. > > Regards, > Daniel You are right!, I just had to subscribe to the folders. Regards, Leonardo From dovecot at allycomm.com Thu Apr 26 02:16:29 2012 From: dovecot at allycomm.com (Jeff Kletsky) Date: Wed, 25 Apr 2012 16:16:29 -0700 Subject: [Dovecot] 2.1.x changes and dsync -- Wiki suggestion Message-ID: <4F9885CD.2000800@allycomm.com> There are a couple things that show up on the Upgrading/2.1 wiki page that would be useful to have on pages describing dsync (Tools/Dsync, Migration/MailFormat, Tools, perhaps others): * dsync was merged into doveadm * You can't dsync 2.0.x to/from 2.1.x That second point is especially "interesting" to me as it seems that Apple MacOS X Server is still on 2.0.16, at least from what I see on opensource.apple.com. I was planning on migrating mail off the current server onto my friend's new MacMini, but I'll have to figure out how to "downgrade" the mail jail to a 2.0 version, it seems. Thanks for all the great work, Jeff From jeff at j-simmons.net Thu Apr 26 04:34:49 2012 From: jeff at j-simmons.net (Jeff Simmons) Date: Wed, 25 Apr 2012 18:34:49 -0700 Subject: [Dovecot] dsync: convert only mbox in /var/mail to maildir? In-Reply-To: <04D01B3D-CD58-4E0A-8C1A-36B01E4792E7@iki.fi> References: <201204201226.41710.jeff@j-simmons.net> <04D01B3D-CD58-4E0A-8C1A-36B01E4792E7@iki.fi> Message-ID: <201204251834.50001.jeff@j-simmons.net> On Friday, April 20, 2012 02:27:59 pm Timo Sirainen wrote: > On 20.4.2012, at 22.26, Jeff Simmons wrote: > > I'm migrating an old UW pop server to a new Dovecot server. The old > > server did not allow mail to be stored on the server, so only mbox files > > exist in /var/mail, with nothing in /home/$user. I'd like to copy the > > /var/mail directory over to the new server and then use dsync to convert > > the mbox files and put them in /home/$user/Maildir, but every > > permutation of the command line I've tried dies with a variation on > > can't find/write either /var/mail/mail or /home/$user/mail, seems it > > won't deal with mbox files without an associated mail directory. Can > > dsync be used for this kind of conversion, or should I look elsewhere > > for conversion utilities? > > Assuming in destination server the mbox is in /var/mail/$user and you want > the destination to ~/Maildir: > > in dovecot.conf set: > mail_location = maildir.~/Maildir > > rm -rf /tmp/emptydir > dsync mirror mbox:/tmp/emptydir:/var/mail/$user Actually: dsync mirror mbox:/tmp/emptydir:INBOX=/var/mail/$user :) And be sure that /var/mail/$user has the same ownership:group as the user (i.e. $user:$user NOT $user:mail). > be sure to delete the emptydir between users. -- Jeff Simmons jeff at j-simmons.net Simmons Consulting - Network Engineering, Administration, Security From me at junc.org Thu Apr 26 04:53:08 2012 From: me at junc.org (Benny Pedersen) Date: Thu, 26 Apr 2012 03:53:08 +0200 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F97BACC.4080902@ymail.com> References: <4F9796EE.5070907@ymail.com> <20120425082615.GA14759@laptop-dg.leere.eu> <4F97BACC.4080902@ymail.com> Message-ID: <9d5bd8f6f2e4efc1c576f4a0246e03fb@junc.org> Den 2012-04-25 10:50, Kris Weston skrev: >> # auth_default_realm = > ahhh, ok. only i have multiple domains, so how does that werk ? > just put em all in ? if you use @ in imap/pop3 logins then the default for this setting is fine, if logins is without @ then you need to set it to the hostname of the machine where users is on, it cant be multiple hostnames on 127.0.0.1 :=) use localhost.example.org if unsure saslauthtest -u user -p passwd -r example.org OK or saslauthtest -u user at example.org -p passwd OK From jtam.home at gmail.com Thu Apr 26 05:14:38 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 25 Apr 2012 19:14:38 -0700 (PDT) Subject: [Dovecot] dovecot LDA with sendmail In-Reply-To: References: Message-ID: Timo, in response to Hadi, wrote: >> Apr 24 20:25:46 lda(jamal at example.com): Fatal: >> setgid(5000(vmail) from userdb lo >> okup) failed with >> euid=8(mail), gid=12(mail), egid=12(mail): Operation not permi >> tted (This binary should probably be called with process group set to 5000(vmail >> ) instead of 12(mail)) > > Configure your MTA to run dovecot-lda as vmail instead of as mail. Yeah, what he said. Someone helpfully added documentation on how to do this at the end of Dovecot1's wiki (http://wiki.dovecot.org/LDA/Sendmail), which probably should be copied to Dovecot 2's wiki. So that's what I did. Joseph Tam From abruce at tumnus.co.nz Thu Apr 26 08:35:00 2012 From: abruce at tumnus.co.nz (Bruce, Andrew) Date: Thu, 26 Apr 2012 17:35:00 +1200 Subject: [Dovecot] LDAP Lookup not returning value in maxStorage In-Reply-To: <4F74B274.8000600@tumnus.co.nz> References: <20120327091425.73963576@jimbo> <20120328092534.5690fa40@jimbo> <4F74B274.8000600@tumnus.co.nz> Message-ID: On 30 March 2012 08:05, Andrew Bruce wrote: > > On 28/03/2012 19:25, Nikita Koshikov wrote: >> >> On Wed, 28 Mar 2012 09:39:37 +1300 >> Bruce, Andrew wrote: >> >>> On 28 March 2012 09:36, Bruce, Andrew ?wrote: >>>> >>>> On 27 March 2012 19:14, Nikita Koshikov ?wrote: >>>>> >>>>> On Tue, 27 Mar 2012 13:57:04 +1300 >>>>> Bruce, Andrew wrote: >>>>> >>>>> Hi there, >>>>> >>>>> We're setting up a Dovecot virtual email setup - we've got everything >>>>> working perfect with LDAP logins authenticating against AD and so >>>>> forth, but we're having issues with retrieving the maxStorage value >>>>> from AD (this is a pre-setup field in AD that we'd like to use to set >>>>> per user quotas). >>>>> >>>>> In our LDAP lookup, we have the maxStorage entry listed under >>>>> user_attrs for the quota (user_attrs = >>>>> maxStorage=quota_rule=*:storage=%$M), and in the debug logs, can see >>>>> it trying to get the entry, but it fails with: >>>>> Mar 27 13:19:27 auth: Debug: ldap(username at site,192.168.1.5): user >>>>> search: base=dc=site,dc=local scope=subtree >>>>> filter=(&(objectClass=person)(| (userPrincipalName=username at site) >>>>> (|(mail=username at site)(samAccountName=username at site)))) >>>>> fields=maxStorage >>>>> Mar 27 13:19:27 auth: Debug: ldap(username at site,192.168.1.5): no >>>>> fields returned by the server >>>>> >>>>> At this point, we then see the default quota applied. >>>>> >>>> Try to change your quota rule to be like: >>>> maxStorage=quota_rule=*:bytes=%$ >>>> ? ? ? ? ? ? ? ? ? ? ? ?^^^^^^^^^ >>>> And put the value in bytes to maxStorage - if I remember correct - this >>>> is integer field and no K\M\G values is valid here. >>>> >>>> PS We successfully using maxStorage field to obtain non-default quota >>>> from AD, dovecot version 2.0.x >>>>> >>>>> If we change the name of the field from maxStorage to instanceType we >>>>> see the value show up in the logs and passed through to the quota >>>>> system and applied successfully: >>>>> Mar 27 11:09:01 auth: Debug: ldap(username at site,192.168.1.5): user >>>>> search: base=dc=site,dc=local scope=subtree >>>>> filter=(&(objectClass=person)(| (userPrincipalName=username at site) >>>>> (|(mail=username at site)(samAccountName=username at site)))) >>>>> fields=instanceType >>>>> Mar 27 11:09:01 auth: Debug: ldap(username at site,192.168.1.5): result: >>>>> instanceType(quota_rule=*:storage=%$M)=*:storage=4M >>>>> Mar 27 11:09:01 auth: Debug: master out: USER ? 3901227009 >>>>> username at site ? ?quota_rule=*:storage=4M >>>>> >>>>> >>>>> Which seems a bit weird. >>>>> >>>>> If we use ldapsearch and pass it the same search string and look for >>>>> the field maxStorage, we clearly see the field and the value being >>>>> returned. ?The result looks the same if we also lookup instanceType. >>>>> >>>>> We're using Dovecot 2.0.9. >>>>> >>>>> Does anyone have any idea as to why we can't use this field? >>>>> >>>>> Thanks, >>>>> >>>>> Andrew >>> >>> Tried your suggestion Nikita, no joy unfortunately. ?It still looks >>> like the value never gets returned from the LDAP server to Dovecot. >>> It definitely has something in the field (equivalent of 10GB, but in >>> bytes as suggested) and I changed the user_attrs also, but still get >>> the same "no fields returned by the server" error message. >>> >>> Modifying the user_attrs to lookup from a different field >>> (instanceType) definitely works. >>> >>> What exact version are you using - perhaps it's a problem with our >>> copy of 2.0.9. >>> >>> Thanks, >>> >>> Andrew >> >> maybe you met restriction of ldap port >> 3268?(http://wiki2.dovecot.org/AuthDatabase/LDAP) >> > Dead on - it was a restriction of ldap port 3268 - as soon as we pointed > ldapsearch at the same port, we got the same result - some of the fields > were missing. ?It all makes perfect sense and I wish I noticed that earlier. > > Now need to work out why Dovecot can get the fields and username back from > ldap on port 389, but it can't do the auth through it like it could with > 3268. > > Thanks Nikita for your help. > > Andrew Just to update the solution for us in the end... We couldn't get auth to work on port 389, but in the end we managed to get the maxStorage field to be presented via the query on port 3268. What we needed to do was to ensure that it was added as an attribute to the Global Catalog on our Windows Servers. We could then get this entry back via our query. There's instructions on how to do this on the following page: http://technet.microsoft.com/en-us/library/cc737521(v=ws.10).aspx Thanks for the help. Regards, Andrew From lists at wiesinger.com Thu Apr 26 09:26:34 2012 From: lists at wiesinger.com (Gerhard Wiesinger) Date: Thu, 26 Apr 2012 08:26:34 +0200 Subject: [Dovecot] v2.1.5 released In-Reply-To: References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <4F98EA9A.9080909@wiesinger.com> Hello Timo, Yes, fixed it. Thnx. I also had a coredump with 2.1.3 + http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e A02 LIST "" "*" imap(username: Panic: file mail-storage.c: line 628 (mailbox_alloc): assertion failed: (uni_utf8_str_is_valid(vname)) Also fixed with 2.1.5 + http://hg.dovecot.org/dovecot-2.1/rev/249c1c89d9d3 Ciao, Gerhard -- http://www.wiesinger.com/ On 25.04.2012 23:16, Timo Sirainen wrote: > On 24.4.2012, at 11.59, Gerhard Wiesinger wrote: > >> Hello Timo, >> >> Dovecot 2.1.5 seems to be broken. After upgrade from >> 2.1.3 + patch http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e >> I only see Inbox and Trash folder. >> >> Reverting => OK. >> >> Same config as before and disucssed per private E-mail and on the list. Guessing this comes again from: >> mail_full_filesystem_access=yes > I guess the problem was this: http://hg.dovecot.org/dovecot-2.1/rev/249c1c89d9d3 > > From markus at mpetri.org Thu Apr 26 09:34:44 2012 From: markus at mpetri.org (Markus Petri) Date: Thu, 26 Apr 2012 08:34:44 +0200 Subject: [Dovecot] Permission problems when trying to create new folders with ACL plugin In-Reply-To: <7D3AF25B-FAE0-4187-B53C-DBB477A4AE79@iki.fi> References: <20120424130005.0db3d425@legolas.home.ceotex.de> <7D3AF25B-FAE0-4187-B53C-DBB477A4AE79@iki.fi> Message-ID: <20120426083444.50be1185@legolas.home.ceotex.de> On Thu, 26 Apr 2012 00:27:25 +0300 Timo Sirainen wrote: > On 24.4.2012, at 14.00, Markus Petri wrote: > > > I have the a problem with shared folders and the ACL plugin. The > > user "markus" has full access to "Shared/shared-user/Data": > > > > 3 GETACL "Shared/shared-user/Data" > > * ACL "Shared/shared-user/Data" "markus" akxeilprwtscd > > 3 OK Getacl completed. > > > > but creating a new subfolder under "Data" fails: > > > > 4 CREATE "Shared/shared-user/Data/New1" > > 4 NO [NOPERM] Permission denied > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e8b80e0767ac > That fixed it. Thanks. From tomislav.mihalicek at gmail.com Thu Apr 26 13:37:32 2012 From: tomislav.mihalicek at gmail.com (Tomislav Mihalicek) Date: Thu, 26 Apr 2012 03:37:32 -0700 (PDT) Subject: [Dovecot] dovecot 2:2.1.5-0~auto+6 on debian squeeze Message-ID: <33752072.post@talk.nabble.com> After upgrade from latest 2.1.4 to 2.1.5 occasional coredumps on lmtp/lda daemon happens. It results in deffering mail in previous hop server and mail delay [1383350.925251] lmtp[22130]: segfault at 0 ip 00007f8829e2b4c5 sp 00007fff29fd37e0 error 4 in libdovecot-lda.so.0.0.0[7f8829e23000+b000] [1383351.038095] lmtp[22121]: segfault at 0 ip 00007f18863084c5 sp 00007fff65d87b20 error 4 in libdovecot-lda.so.0.0.0[7f1886300000+b000] [1383366.682285] lmtp[22101]: segfault at 0 ip 00007fe87a36a4c5 sp 00007fffb391b430 error 4 in libdovecot-lda.so.0.0.0[7fe87a362000+b000] [1383530.705974] lmtp[22126]: segfault at 0 ip 00007f0a7caa74c5 sp 00007ffff5bd0050 error 4 in libdovecot-lda.so.0.0.0[7f0a7ca9f000+b000] [1383530.844101] lmtp[17613]: segfault at 0 ip 00007f43aa1e84c5 sp 00007fff977ffd70 error 4 in libdovecot-lda.so.0.0.0[7f43aa1e0000+b000] [1383544.463542] lmtp[17576]: segfault at 0 ip 00007fbec6bc64c5 sp 00007fff3e347910 error 4 in libdovecot-lda.so.0.0.0[7fbec6bbe000+b000] -- View this message in context: http://old.nabble.com/dovecot-2%3A2.1.5-0%7Eauto%2B6-on-debian-squeeze-tp33752072p33752072.html Sent from the Dovecot mailing list archive at Nabble.com. From tomislav.mihalicek at gmail.com Thu Apr 26 13:39:22 2012 From: tomislav.mihalicek at gmail.com (Tomislav Mihalicek) Date: Thu, 26 Apr 2012 03:39:22 -0700 (PDT) Subject: [Dovecot] segfaults with dovecot 2:2.1.5-0~auto+6 on debian squeeze Message-ID: <33752072.post@talk.nabble.com> After upgrade from latest 2.1.4 to 2.1.5 occasional coredumps on lmtp/lda daemon happens. It results in deffering mail in previous hop server and mail delay [1383350.925251] lmtp[22130]: segfault at 0 ip 00007f8829e2b4c5 sp 00007fff29fd37e0 error 4 in libdovecot-lda.so.0.0.0[7f8829e23000+b000] [1383351.038095] lmtp[22121]: segfault at 0 ip 00007f18863084c5 sp 00007fff65d87b20 error 4 in libdovecot-lda.so.0.0.0[7f1886300000+b000] [1383366.682285] lmtp[22101]: segfault at 0 ip 00007fe87a36a4c5 sp 00007fffb391b430 error 4 in libdovecot-lda.so.0.0.0[7fe87a362000+b000] [1383530.705974] lmtp[22126]: segfault at 0 ip 00007f0a7caa74c5 sp 00007ffff5bd0050 error 4 in libdovecot-lda.so.0.0.0[7f0a7ca9f000+b000] [1383530.844101] lmtp[17613]: segfault at 0 ip 00007f43aa1e84c5 sp 00007fff977ffd70 error 4 in libdovecot-lda.so.0.0.0[7f43aa1e0000+b000] [1383544.463542] lmtp[17576]: segfault at 0 ip 00007fbec6bc64c5 sp 00007fff3e347910 error 4 in libdovecot-lda.so.0.0.0[7fbec6bbe000+b000] -- View this message in context: http://old.nabble.com/segfaults-with-dovecot-2%3A2.1.5-0%7Eauto%2B6-on-debian-squeeze-tp33752072p33752072.html Sent from the Dovecot mailing list archive at Nabble.com. From interfasys at gmail.com Thu Apr 26 17:23:25 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 26 Apr 2012 15:23:25 +0100 Subject: [Dovecot] LDA crashes on delivery (using sieve) Message-ID: <4F995A5D.7070708@gmail.com> Hello, The current version of LDA can crash (seg fault 11) under certain circumstances. I don't know why it happens only with some accounts, but here is a backtrace. The user has a sieve filter in place : (gdb) bt full #0 0x0000000800443434 in smtp_client_open () from /usr/lib/dovecot/libdovecot-lda.so.0 No symbol table info available. #1 0x00000008009943ab in lda_sieve_smtp_open.2406.2210 () from /usr/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #2 0x0000000800e80532 in sieve_smtp_open () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #3 0x0000000800e6157d in act_redirect_send.23018 () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #4 0x0000000800e7e175 in act_redirect_commit.23025 () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #5 0x0000000800e5f2bd in sieve_result_execute () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #6 0x0000000800e5ee7b in sieve_multiscript_execute.25305 () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #7 0x0000000800e496f8 in sieve_multiscript_run () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #8 0x0000000800994652 in lda_sieve_multiscript_execute.2399.2297 () from /usr/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #9 0x00000008009942c7 in lda_sieve_deliver_mail.2402.2206 () from /usr/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #10 0x00000008004435d8 in mail_deliver () from /usr/lib/dovecot/libdovecot-lda.so.0 No symbol table info available. #11 0x00000000004029f7 in main (argc=7, argv=0x7fffffffe830) at main.c:434 set_roots = {0x404520, 0x0} ctx = {pool = 0x800a0d820, set = 0x800a3cc98, session = 0x800a0d840, dup_ctx = 0x800abf0c0, session_id = 0x0, src_mail = 0x800a34840, src_envelope_sender = 0x800a0d858 "mailtest at example.com", dest_user = 0x800a89040, dest_addr = 0x7fffffffebfd "test at example.com", final_dest_addr = 0x7fffffffebfd "test at example.com", dest_mailbox_name = 0x40356b "INBOX", dest_mail = 0x0, var_expand_table = 0x0, tried_default_save = false, saved_mail = false, save_dest_mail = false, mailbox_full = false, dsn = false} service_flags = 1027 user = (const __unknown__ *) 0x7fffffffebcc "test at example.com" errstr = (const __unknown__ *) 0x0 path = (const __unknown__ *) 0x0 storage_service = (struct mail_storage_service_ctx *) 0x800a30840 service_user = (struct mail_storage_service_user *) 0x800a3c0a0 service_input = {module = 0x403567 "lda", service = 0x403567 "lda", username = 0x7fffffffebcc "test at example.com", session_id = 0x0, local_ip = {family = 0, u = {ip6 = {__u6_addr = {__u6_addr8 = '\0' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__u6_addr = {__u6_addr8 = '\0' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0} storage = (struct mail_storage *) 0x0 user_source = (const __unknown__ *) 0x4034c0 "" destaddr_source = (const __unknown__ *) 0x403576 "-a parameter" process_euid = 0 stderr_rejection = false ret = 1 c = -1 error = MAIL_ERROR_NONE Cheers, Olivier From interfasys at gmail.com Thu Apr 26 18:32:23 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 26 Apr 2012 16:32:23 +0100 Subject: [Dovecot] LDA crashes on delivery (using sieve) Message-ID: <4F996A87.90701@gmail.com> Hello, The current version of LDA can crash (seg fault 11) under certain circumstances. I don't know why it happens only with some accounts, but here is a backtrace. The user has a sieve filter in place : (gdb) bt full #0 0x0000000800443434 in smtp_client_open () from /usr/lib/dovecot/libdovecot-lda.so.0 No symbol table info available. #1 0x00000008009943ab in lda_sieve_smtp_open.2406.2210 () from /usr/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #2 0x0000000800e80532 in sieve_smtp_open () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #3 0x0000000800e6157d in act_redirect_send.23018 () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #4 0x0000000800e7e175 in act_redirect_commit.23025 () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #5 0x0000000800e5f2bd in sieve_result_execute () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #6 0x0000000800e5ee7b in sieve_multiscript_execute.25305 () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #7 0x0000000800e496f8 in sieve_multiscript_run () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #8 0x0000000800994652 in lda_sieve_multiscript_execute.2399.2297 () from /usr/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #9 0x00000008009942c7 in lda_sieve_deliver_mail.2402.2206 () from /usr/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #10 0x00000008004435d8 in mail_deliver () from /usr/lib/dovecot/libdovecot-lda.so.0 No symbol table info available. #11 0x00000000004029f7 in main (argc=7, argv=0x7fffffffe830) at main.c:434 set_roots = {0x404520, 0x0} ctx = {pool = 0x800a0d820, set = 0x800a3cc98, session = 0x800a0d840, dup_ctx = 0x800abf0c0, session_id = 0x0, src_mail = 0x800a34840, src_envelope_sender = 0x800a0d858 "mailtest at example.com", dest_user = 0x800a89040, dest_addr = 0x7fffffffebfd "test at example.com", final_dest_addr = 0x7fffffffebfd "test at example.com", dest_mailbox_name = 0x40356b "INBOX", dest_mail = 0x0, var_expand_table = 0x0, tried_default_save = false, saved_mail = false, save_dest_mail = false, mailbox_full = false, dsn = false} service_flags = 1027 user = (const __unknown__ *) 0x7fffffffebcc "test at example.com" errstr = (const __unknown__ *) 0x0 path = (const __unknown__ *) 0x0 storage_service = (struct mail_storage_service_ctx *) 0x800a30840 service_user = (struct mail_storage_service_user *) 0x800a3c0a0 service_input = {module = 0x403567 "lda", service = 0x403567 "lda", username = 0x7fffffffebcc "test at example.com", session_id = 0x0, local_ip = {family = 0, u = {ip6 = {__u6_addr = {__u6_addr8 = '\0' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__u6_addr = {__u6_addr8 = '\0' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0} storage = (struct mail_storage *) 0x0 user_source = (const __unknown__ *) 0x4034c0 "" destaddr_source = (const __unknown__ *) 0x403576 "-a parameter" process_euid = 0 stderr_rejection = false ret = 1 c = -1 error = MAIL_ERROR_NONE Cheers, Olivier From tss at iki.fi Thu Apr 26 18:35:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 18:35:17 +0300 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <4F996A87.90701@gmail.com> References: <4F996A87.90701@gmail.com> Message-ID: <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> On 26.4.2012, at 18.32, interfaSys s?rl wrote: > The current version of LDA can crash (seg fault 11) under certain > circumstances. "The current" means what exactly? (Stephan's autobuilt .debs?) From feltrin at gmail.com Thu Apr 26 18:40:30 2012 From: feltrin at gmail.com (Jean Michel) Date: Thu, 26 Apr 2012 12:40:30 -0300 Subject: [Dovecot] Troubles with /.dovecot.lda-dupe on dovecot 2.1.5 Message-ID: Seems that is something with vacation message, Apr 26 12:20:02 server dovecot: lmtp(26937, user at domain): Error: file_dotlock_create(/storage/domain/users/user/.dovecot.lda-dupes) failed: Resource temporarily unavailable Apr 26 12:20:02 server dovecot: lmtp(26937, user at domain): Fatal: master: service(lmtp): child 26937 killed with signal 11 (core dumps disabled) And its created a .dovecot.lda-dupes.lock Any idea ? -- ------------------------------ Jean Michel Feltrin From interfasys at gmail.com Thu Apr 26 18:47:08 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 26 Apr 2012 16:47:08 +0100 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> References: <4F996A87.90701@gmail.com> <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> Message-ID: <4F996DFC.8070607@gmail.com> Latest revision from these repositories: dovecot-2.1 dovecot-2.1-pigeonhole On 26/04/2012 16:35, Timo Sirainen wrote: > On 26.4.2012, at 18.32, interfaSys s?rl wrote: > >> The current version of LDA can crash (seg fault 11) under certain >> circumstances. > > > "The current" means what exactly? (Stephan's autobuilt .debs?) > > > From goetz.reinicke at filmakademie.de Thu Apr 26 18:48:36 2012 From: goetz.reinicke at filmakademie.de (=?ISO-8859-15?Q?G=F6tz_Reinicke?=) Date: Thu, 26 Apr 2012 17:48:36 +0200 Subject: [Dovecot] OT but E-Mail related. save a copy off all mail as eml Message-ID: <4F996E54.3020706@filmakademie.de> Hi, I'm sorry to bug you with an not directly dovecot related question, but this is for me right now the place with the best mailsystem admins I know. I'm asked if there is a way to save a copy of all mail we send trough our sendmail server (Red Hat EL 5.x) in and out as an .eml file for further processing by a DMS system. The files should be saved on some network share. Any suggestion are welcome and thanks. Best regards G?tz -- G?tz Reinicke IT-Koordinator Tel. +49 7141 969 82 420 Fax +49 7141 969 55 420 E-Mail goetz.reinicke at filmakademie.de Filmakademie Baden-W?rttemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: J?rgen Walter MdL Staatssekret?r im Ministerium f?r Wissenschaft, Forschung und Kunst Baden-W?rttemberg Gesch?ftsf?hrer: Prof. Thomas Schadt -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5161 bytes Desc: S/MIME Kryptografische Unterschrift URL: From robert at schetterer.org Thu Apr 26 19:02:36 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 26 Apr 2012 18:02:36 +0200 Subject: [Dovecot] Troubles with /.dovecot.lda-dupe on dovecot 2.1.5 In-Reply-To: References: Message-ID: <4F99719C.40500@schetterer.org> Am 26.04.2012 17:40, schrieb Jean Michel: > Seems that is something with vacation message, > > Apr 26 12:20:02 server dovecot: lmtp(26937, user at domain): Error: > file_dotlock_create(/storage/domain/users/user/.dovecot.lda-dupes) failed: > Resource temporarily unavailable > Apr 26 12:20:02 server dovecot: lmtp(26937, user at domain): Fatal: master: > service(lmtp): child 26937 killed with signal 11 (core dumps disabled) > And its created a .dovecot.lda-dupes.lock > > Any idea ? jep thats from sieve ,see my my mails from yesterday -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From interfasys at gmail.com Thu Apr 26 19:03:31 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 26 Apr 2012 17:03:31 +0100 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> References: <4F996A87.90701@gmail.com> <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> Message-ID: <4F9971D3.4020107@gmail.com> And it's on FreeBSD 8.2, compiled from source, using GCC 4.6 and the gold linker. On 26/04/2012 16:35, Timo Sirainen wrote: > On 26.4.2012, at 18.32, interfaSys s?rl wrote: > >> The current version of LDA can crash (seg fault 11) under certain >> circumstances. > > > "The current" means what exactly? (Stephan's autobuilt .debs?) > > > From robert at schetterer.org Thu Apr 26 19:04:58 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 26 Apr 2012 18:04:58 +0200 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <4F996DFC.8070607@gmail.com> References: <4F996A87.90701@gmail.com> <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> <4F996DFC.8070607@gmail.com> Message-ID: <4F99722A.7080300@schetterer.org> Am 26.04.2012 17:47, schrieb interfaSys s?rl: > Latest revision from these repositories: > dovecot-2.1 > dovecot-2.1-pigeonhole dovecot-2.1-pigeonhole from hg ,is your problem use i.e http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz to recompile , this works > > > On 26/04/2012 16:35, Timo Sirainen wrote: >> On 26.4.2012, at 18.32, interfaSys s?rl wrote: >> >>> The current version of LDA can crash (seg fault 11) under certain >>> circumstances. >> >> >> "The current" means what exactly? (Stephan's autobuilt .debs?) >> >> >> -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Thu Apr 26 19:09:01 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 26 Apr 2012 18:09:01 +0200 Subject: [Dovecot] OT but E-Mail related. save a copy off all mail as eml In-Reply-To: <4F996E54.3020706@filmakademie.de> References: <4F996E54.3020706@filmakademie.de> Message-ID: <4F99731D.3010405@schetterer.org> Am 26.04.2012 17:48, schrieb G?tz Reinicke: > Hi, > > I'm sorry to bug you with an not directly dovecot related question, but > this is for me right now the place with the best mailsystem admins I know. > > I'm asked if there is a way to save a copy of all mail we send trough > our sendmail server (Red Hat EL 5.x) in and out as an .eml file for > further processing by a DMS system. The files should be saved on some > network share. you need archive solution, in postfix its done like i.e.this recipient_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre sender_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre then use i.e procmail for filter , but i wouldnt recommend .eml i may be not easy to build uniq file names from i.e mail subject storing in maildir should be good enough ( each mail is a file ) > > Any suggestion are welcome and thanks. Best regards G?tz -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Thu Apr 26 19:10:59 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 26 Apr 2012 18:10:59 +0200 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <4F9971D3.4020107@gmail.com> References: <4F996A87.90701@gmail.com> <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> <4F9971D3.4020107@gmail.com> Message-ID: <4F997393.5050306@schetterer.org> Am 26.04.2012 18:03, schrieb interfaSys s?rl: > And it's on FreeBSD 8.2, compiled from source, using GCC 4.6 and the > gold linker. > > On 26/04/2012 16:35, Timo Sirainen wrote: >> On 26.4.2012, at 18.32, interfaSys s?rl wrote: >> >>> The current version of LDA can crash (seg fault 11) under certain >>> circumstances. >> >> >> "The current" means what exactly? (Stephan's autobuilt .debs?) >> >> >> oh ,may be more complicated with bsd i think i saw some late patches to compile -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From interfasys at gmail.com Thu Apr 26 22:18:10 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 26 Apr 2012 20:18:10 +0100 Subject: [Dovecot] LDA crashes on delivery (using sieve) Message-ID: <4F999F72.1060603@gmail.com> I can confirm that the issues are gone when using pigeonhole-0.3.0 I thought the sievec crashes might re-appear, but this isn't the case. Thank you Robert. Olivier > Am 26.04.2012 17:47, schrieb interfaSys s?rl: > >/ Latest revision from these repositories: > />/ dovecot-2.1 > />/ dovecot-2.1-pigeonhole > / > dovecot-2.1-pigeonhole from hg ,is your problem > use i.e > > http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz > to recompile , this works > > >/ > />/ > />/ On 26/04/2012 16:35, Timo Sirainen wrote: > />>/ On 26.4.2012, at 18.32, interfaSys s?rl wrote: > />>/ > />>>/ The current version of LDA can crash (seg fault 11) under certain > />>>/ circumstances. > />>/ > />>/ > />>/ "The current" means what exactly? (Stephan's autobuilt .debs?) > />>/ > />>/ > />>/ > / > > -- > Best Regards > > MfG Robert Schetterer > > Germany/Munich/Bavaria From stephan at rename-it.nl Fri Apr 27 03:28:04 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 27 Apr 2012 02:28:04 +0200 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <4F996A87.90701@gmail.com> References: <4F996A87.90701@gmail.com> Message-ID: <4F99E814.3050509@rename-it.nl> On 4/26/2012 5:32 PM, interfaSys s?rl wrote: > Hello, > > The current version of LDA can crash (seg fault 11) under certain > circumstances. > > I don't know why it happens only with some accounts, but here is a > backtrace. The user has a sieve filter in place : Fixed: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/f146790b5db3 Regards, Stephan. From nick.urbanik at optusnet.com.au Fri Apr 27 09:34:31 2012 From: nick.urbanik at optusnet.com.au (Nick Urbanik) Date: Fri, 27 Apr 2012 16:34:31 +1000 Subject: [Dovecot] 2.1.5: Panic: file mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: (mailbox_list_is_valid_pattern(_list, name)) Message-ID: <20120427063431.GA12180@nick.optusnet.com.au> Dear Folks, We had this assertion failure on Dovecot 2.1.1 on a particular mailbox. I upgraded to 2.1.5, and we have the same assertion failure, on the same mailbox: Apr 27 16:02:03 imap(nu-imaptest00258 at example.com): Panic: file mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: (mailbox_list_is_valid_pattern(_list, name)) Apr 27 16:02:03 imap(nu-imaptest00258 at example.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x4246a) [0x7f310c28146a] -> /usr/lib64/dovecot/libdovecot.so.0(+0x424b6) +[0x7f310c2814b6] -> /usr/lib64/dovecot/libdovecot.so.0(+0x1950a) [0x7f310c25850a] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x889b5) [0x7f310c5539b5] -> +/usr/lib64/dovecot/lib10_quota_plugin.so(+0x8a30) [0x7f3109ccea30] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0x8cdf) [0x7f3109ccecdf] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0x93c8) +[0x7f3109ccf3c8] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0x9436) [0x7f3109ccf436] -> /usr/lib64/dovecot/lib10_quota_plugin.so(quota_get_resource+0x72) [0x7f3109cca792] -> +/usr/lib64/dovecot/lib10_quota_plugin.so(quota_test_alloc+0x1bd) [0x7f3109ccaaed] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0xa870) [0x7f3109cd0870] -> +/usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_save_begin+0x4d) [0x7f310c544b9d] -> dovecot/imap [nu-imaptest00258 at example.com 127.0.0.1 APPEND]() [0x408ac4] -> dovecot/imap +[nu-imaptest00258 at example.com 127.0.0.1 APPEND](cmd_append+0x139) [0x408d69] -> dovecot/imap [nu-imaptest00258 at example.com 127.0.0.1 APPEND](command_exec+0x3d) [0x410aed] -> dovecot/imap +[nu-imaptest00258 at example.com 127.0.0.1 APPEND]() [0x40fa2e] -> dovecot/imap [nu-imaptest00258 at example.com 127.0.0.1 APPEND]() [0x40fb1a] -> dovecot/imap [nu-imaptest00258 at example.com +127.0.0.1 APPEND](client_handle_input+0x135) [0x40fd45] -> dovecot/imap [nu-imaptest00258 at example.com 127.0.0.1 APPEND](client_input+0x5f) [0x41066f] -> +/usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f310c28d9d6] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) [0x7f310c28ea5f] -> +/usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f310c28d978] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f310c279c73] -> dovecot/imap [nu-imaptest00258 at example.com +127.0.0.1 APPEND](main+0x29d) [0x418b3d] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f310bebccdd] -> dovecot/imap [nu-imaptest00258 at example.com 127.0.0.1 APPEND]() [0x408449] Apr 27 16:02:03 imaptest01 perdition.imap4[15893]: Closing session: 211.29.132.209:42922->211.29.132.245:143 authorisation_id=NONE authentication_id="nu-imaptest00258" received=67 sent=53 Apr 27 16:02:03 imap(nu-imaptest00258 at example.com): Fatal: master: service(imap): child 15894 killed with signal 6 (core not dumped) Any suggestions welcome. I guess enabling core dumps is a good start, I'll do that, but has anyone seen this before? -- Nick Urbanik http://nicku.org 808-71011 nick.urbanik at optusnet.com.au GPG: 7FFA CDC7 5A77 0558 DC7A 790A 16DF EC5B BB9D 2C24 ID: BB9D2C24 I disclaim, therefore I am. From robert at schetterer.org Fri Apr 27 09:50:24 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 27 Apr 2012 08:50:24 +0200 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <4F99E814.3050509@rename-it.nl> References: <4F996A87.90701@gmail.com> <4F99E814.3050509@rename-it.nl> Message-ID: <4F9A41B0.8040302@schetterer.org> Am 27.04.2012 02:28, schrieb Stephan Bosch: > On 4/26/2012 5:32 PM, interfaSys s?rl wrote: >> Hello, >> >> The current version of LDA can crash (seg fault 11) under certain >> circumstances. >> >> I don't know why it happens only with some accounts, but here is a >> backtrace. The user has a sieve filter in place : > > Fixed: > > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/f146790b5db3 > > Regards, > > Stephan. Hi Stephan, i can confirm by small tests using deb source latest from http://hg.rename-it.nl your patch fixed the lmtp crash with sieve and redirect/vacation Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From fumiyas at osstech.jp Fri Apr 27 10:20:28 2012 From: fumiyas at osstech.jp (SATOH Fumiyasu) Date: Fri, 27 Apr 2012 16:20:28 +0900 Subject: [Dovecot] [PATCH] Dovecot 2.1 compiling fix for AIX Message-ID: <87vcklfxoj.wl%fumiyas@osstech.jp> Hi, I'm trying to build Dovecot 2.1.5 on AIX 6.1 and got the following build error: $ ./configure ... $ make ... In file included from stats-connection.c:9: stats-plugin.h:14: error: field 'user_cpu' has incomplete type stats-plugin.h:14: error: field 'sys_cpu' has incomplete type ... The attached patch fixes this problem. -- -- Name: SATOH Fumiyasu (fumiyas @ osstech co jp) -- Business Home: http://www.OSSTech.co.jp/ -- Personal Home: https://github.com/fumiyas/ -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-2.1-stats-plugin-build-on-AIX.patch Type: application/octet-stream Size: 821 bytes Desc: not available URL: From fumiyas at osstech.jp Fri Apr 27 10:49:51 2012 From: fumiyas at osstech.jp (SATOH Fumiyasu) Date: Fri, 27 Apr 2012 16:49:51 +0900 Subject: [Dovecot] [PATCH] Dovecot 2.1 compiling fix for AIX In-Reply-To: <87vcklfxoj.wl%fumiyas@osstech.jp> References: <87vcklfxoj.wl%fumiyas@osstech.jp> Message-ID: <87sjfpfwbk.wl%fumiyas@osstech.jp> Revised patch. Sorry. At Fri, 27 Apr 2012 16:20:28 +0900, SATOH Fumiyasu wrote: > I'm trying to build Dovecot 2.1.5 on AIX 6.1 and got the following > build error: > > $ ./configure > ... > $ make > ... > In file included from stats-connection.c:9: > stats-plugin.h:14: error: field 'user_cpu' has incomplete type > stats-plugin.h:14: error: field 'sys_cpu' has incomplete type > ... > > The attached patch fixes this problem. -- -- Name: SATOH Fumiyasu (fumiyas @ osstech co jp) -- Business Home: http://www.OSSTech.co.jp/ -- Personal Home: https://github.com/fumiyas/ -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-2.1-include-sys-time.h-to-build-on-AIX.patch Type: application/octet-stream Size: 1230 bytes Desc: not available URL: From interfasys at gmail.com Fri Apr 27 20:11:14 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Fri, 27 Apr 2012 18:11:14 +0100 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <4F99E814.3050509@rename-it.nl> References: <4F996A87.90701@gmail.com> <4F99E814.3050509@rename-it.nl> Message-ID: <4F9AD332.4080803@gmail.com> Cheers Stephan! :) On 27/04/2012 01:28, Stephan Bosch wrote: > On 4/26/2012 5:32 PM, interfaSys s?rl wrote: >> Hello, >> >> The current version of LDA can crash (seg fault 11) under certain >> circumstances. >> >> I don't know why it happens only with some accounts, but here is a >> backtrace. The user has a sieve filter in place : > > Fixed: > > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/f146790b5db3 > > Regards, > > Stephan. From almarzuki2001 at hotmail.com Sat Apr 28 03:45:08 2012 From: almarzuki2001 at hotmail.com (Hadi Salem) Date: Sat, 28 Apr 2012 03:45:08 +0300 Subject: [Dovecot] dovecot LDA error In-Reply-To: References: , Message-ID: Hi Timo Sirainen, It?s working with me fine. With no problem. After I configured my MTA (sendmail) to run dovecot-lda as root with chmod 4750 /local/user/libexce/dovecot/dovecot-lda. And create group called secmail. I had to change dovecot-lda group to secmail. And then added mail to secmail group. It works with me. Before when I changed dovecot-lda group direct to mail doesn?t work. But it should to work.!!! I think that was the error. Later after creating separate group and adding mail to it, it?s working fine.. Any idea here? why doesn?t work if I changed dovecot-lda direct to mail group instead of creating new group and adding mail to it???. Hadi- > From: tss at iki.fi > Date: Thu, 26 Apr 2012 00:29:51 +0300 > To: almarzuki2001 at hotmail.com > CC: dovecot at dovecot.org > Subject: Re: [Dovecot] dovecot LDA error > > On 24.4.2012, at 20.41, Hadi Salem wrote: > > > Apr 24 20:25:46 lda(jamal at example.com): Fatal: > > setgid(5000(vmail) from userdb lo > > okup) failed with > > euid=8(mail), gid=12(mail), egid=12(mail): Operation not permi > > tted (This binary should probably be called with process group set to 5000(vmail > > ) instead of 12(mail)) > > Configure your MTA to run dovecot-lda as vmail instead of as mail. > From daniel.parthey at informatik.tu-chemnitz.de Sat Apr 28 14:00:23 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 28 Apr 2012 13:00:23 +0200 Subject: [Dovecot] lmtp proxy timeout while waiting for reply to DATA reply Message-ID: <20120428110023.GA9236@daniel.localdomain> Hi, we are experiencing similar sporadic data timeout issues with dovecot 2.0.20 as in http://dovecot.org/pipermail/dovecot/2011-June/059807.html at least once a week. Some mails get temporarily deferred in the postfix queue since dovecot director lmtp refuses them and the mails are delivered at a later time. The problem applies to mails delivered to single users as well as distribution lists where the mails gets delivered to several mailboxes by postfix. The postfix log shows the following output: postfix.log.7.gz:Apr 21 18:48:46 10.129.3.233 postfix/lmtp[15868]: 88062C08E: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=35, delays=5.4/0.01/0/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) The dovecot.log (mailbox and director log there via syslog) shows the following output: Apr 21 18:46:53 10.129.3.233 dovecot: lmtp(15869): Debug: none: root=, index=, control=, inbox=, alt= Apr 21 18:46:53 10.129.3.233 dovecot: lmtp(15869): Connect from 127.0.0.1 Apr 21 18:46:53 10.129.3.233 dovecot: lmtp(15869): Debug: auth input: user=username at example.org proxy port=19024 host=10.129.3.190 proxy_refresh=450 Apr 21 18:46:53 10.129.3.233 dovecot: lmtp(15869): Disconnect from 127.0.0.1: Client quit (in reset) Apr 21 18:47:29 10.129.3.233 dovecot: lmtp(15869): Debug: none: root=, index=, control=, inbox=, alt= Apr 21 18:47:29 10.129.3.233 dovecot: lmtp(15869): Connect from 127.0.0.1 Apr 21 18:47:29 10.129.3.233 dovecot: lmtp(15869): Debug: auth input: user=username at example.org proxy port=19024 host=10.129.3.190 proxy_refresh=450 Apr 21 18:47:29 10.129.3.233 dovecot: lmtp(15869): Disconnect from 127.0.0.1: Client quit (in reset) Apr 21 18:48:15 10.129.3.233 dovecot: lmtp(15869): Debug: none: root=, index=, control=, inbox=, alt= Apr 21 18:48:15 10.129.3.233 dovecot: lmtp(15869): Connect from 127.0.0.1 Apr 21 18:48:15 10.129.3.233 dovecot: lmtp(15869): Debug: auth input: user=username at example.org proxy port=19024 host=10.129.3.190 proxy_refresh=450 Apr 21 18:48:46 10.129.3.233 dovecot: lmtp(15869): Disconnect from 127.0.0.1: Client quit (in reset) One cannot identify the lmtp log origin from the log_format prefixes, but it seems that the "lmtp($PID)" log originated from the director, since proxy port 19024 is mentioned in the logs, which is our dovecot mailbox backend. Config for dovecot (mailbox) and director (proxy) is attached. mail_debug is yes for both dovecot and director. How can we find out more information? Shall I apply the following patches from http://dovecot.org/pipermail/dovecot/2011-June/059987.html or are they already included with version 2.0.20? Regards, Daniel -------------- next part -------------- # 2.0.20: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-40-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_cache_ttl = 1 mins auth_verbose = yes auth_verbose_passwords = sha1 deliver_log_format = mailbox: deliver: msgid=%m from=%f: %$ dict { quota = mysql:/etc/dovecot/conf.d/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = Mailbox login_log_format = mailbox: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "mailbox: mail: %s(%u): " mail_plugins = quota mail_privileged_group = vmail mail_uid = vmail managesieve_implementation_string = Sieve managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 50 M mmap_disable = yes passdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve service dict { unix_listener dict { group = vmail mode = 0660 } } service imap-login { inet_listener imap { port = 19143 } } service lmtp { inet_listener lmtp { address = * port = 19024 } } service managesieve-login { inet_listener sieve { port = 19200 } } service pop3-login { inet_listener pop3 { port = 19110 } } service quota-warning { executable = script /usr/local/bin/quota-warning unix_listener quota-warning { user = vmail } user = dovecot } ssl = no userdb { driver = prefetch } userdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_plugins = quota imap_quota } protocol lmtp { mail_plugins = quota sieve } -------------- next part -------------- # 2.0.20: /etc/dovecot-director/dovecot-director.conf # OS: Linux 2.6.32-40-server x86_64 Ubuntu 10.04.4 LTS auth_verbose = yes auth_verbose_passwords = sha1 base_dir = /var/run/dovecot-director deliver_log_format = director: deliver: msgid=%m from=%f: %$ director_mail_servers = 10.129.3.193 10.129.3.192 10.129.3.191 10.129.3.190 director_servers = 10.129.3.193 10.129.3.192 10.129.3.191 10.129.3.190 lmtp_proxy = yes login_greeting = Mail Balancer login_log_format = director: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "director: mail: %s(%u): " mail_privileged_group = vmail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes passdb { args = proxy=y nopassword=y user=%n at dovecotmail.%d driver = static } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director inet_listener imap { port = 20143 } inet_listener imaps { port = 20993 ssl = yes } } service lmtp { inet_listener lmtp { address = * port = 20024 } } service managesieve-login { inet_listener sieve { port = 20200 } } service pop3-login { executable = pop3-login director inet_listener pop3 { port = 20110 } inet_listener pop3s { port = 20995 ssl = yes } } ssl_cert = -- View this message in context: http://old.nabble.com/ERR-Attentication-error-dovecot-tp33763251p33763251.html Sent from the Dovecot mailing list archive at Nabble.com. From tss at iki.fi Sat Apr 28 17:53:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Apr 2012 17:53:12 +0300 Subject: [Dovecot] Session IDs Message-ID: <3AFABAEC-9FB7-449D-9473-64714C2038A7@iki.fi> v2.1.2 added support for "session IDs", which is a 9 year unique identifier for that specific IMAP/POP3 connection. Initially I thought it would only be useful for tracking connections going through Dovecot proxies to backends (the session ID is forwarded), but then I thought it could be useful for everyone if auth process also logs the session ID. So I thought for v2.1.6 I'd change the defaults: a) Failed login: Apr 28 17:37:23 auth: Debug: client in: AUTH 1 PLAIN service=imap secured session=XtG7KL6+jQB/AAAB lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=42125 resp= Apr 28 17:37:23 auth: Debug: passwd-file(foo,127.0.0.1,): lookup: user=foo file=/usr/local/etc/passwd.imap Apr 28 17:37:23 auth: Info: passwd-file(foo,127.0.0.1,): unknown user Apr 28 17:37:25 auth: Debug: client out: FAIL 1 user=foo Apr 28 17:37:50 imap-login: Info: Aborted login (auth failed, 1 attempts in 27 secs): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, session= b) Successful login: Apr 28 17:38:27 auth: Debug: client in: AUTH 2 PLAIN service=imap secured session=rNuHLL6+jgB/AAAB lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=42126 resp= Apr 28 17:38:27 auth: Debug: passwd-file(tss,127.0.0.1,): lookup: user=tss file=/usr/local/etc/passwd.imap Apr 28 17:38:27 auth: Debug: client out: OK 2 user=tss Apr 28 17:38:27 auth: Debug: master in: REQUEST 1375993857 14321 2 0f2f78deedcb7e38a278cf79819360c5 Apr 28 17:38:27 auth: Debug: passwd-file(tss,127.0.0.1,): lookup: user=tss file=/usr/local/etc/passwd.imap Apr 28 17:38:27 auth: Debug: master out: USER 1375993857 tss uid=1000 gid=1000 home=/home/tss Apr 28 17:38:27 imap-login: Info: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=14338, secured, session= .. Apr 28 17:47:15 imap(tss,): Info: Disconnected: Logged out in=8 out=369 The last line requires a modified mail_log_prefix, I wasn't planning on changing its default. I'm mainly wondering: 1) Does this change break someone's auth log line parsing? 2) Does the auth prefix look prettier with or without <> around session id? :) 3) Anything else I should do about this now that I'm changing it anyway? From dieterknopf at googlemail.com Sat Apr 28 19:07:08 2012 From: dieterknopf at googlemail.com (Dieter Knopf) Date: Sat, 28 Apr 2012 18:07:08 +0200 Subject: [Dovecot] Refiltering mails without loosing flags In-Reply-To: <4EE5E8FF.8080306@rename-it.nl> References: <4EE5E8FF.8080306@rename-it.nl> Message-ID: 2011/12/12 Stephan Bosch : > Since last week, the new sieve-filter command line tool, which is intended > for just such a task, is part of the normal compile output for Pigeonhole > 0.3.x for Dovecot v2.1. Unfortunately, it is not released yet, so I guess > you're stuck with your old getmail-based solution for now. > > The tool is still experimental and I must say that I have not tested the > preservation of flags for some time and many changes were performed since > then. I will verify that later today. Also, the interaction with the new > editheader extension needs attention. Is there any update about that? Would be really nice to have an nice and easy way like with procmail. Just via command line not via getmail/imap/.... Thanks From dieterknopf at googlemail.com Sat Apr 28 19:09:10 2012 From: dieterknopf at googlemail.com (Dieter Knopf) Date: Sat, 28 Apr 2012 18:09:10 +0200 Subject: [Dovecot] SETANNOTATION for Dovecot Message-ID: Hello, i'm using kmail2 and for Folder Settings the SETANNOATION command is needed. Is there any way to add this command to Dovecot? I already found a dovecot-metadata-plugin but there is no Package available for Debian. Thanks Dieter From daniel.parthey at informatik.tu-chemnitz.de Sat Apr 28 20:16:54 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 28 Apr 2012 19:16:54 +0200 Subject: [Dovecot] Session IDs In-Reply-To: <3AFABAEC-9FB7-449D-9473-64714C2038A7@iki.fi> References: <3AFABAEC-9FB7-449D-9473-64714C2038A7@iki.fi> Message-ID: <20120428171654.GA19481@daniel.localdomain> Timo Sirainen wrote: > v2.1.2 added support for "session IDs", which is a 9 year unique identifier > for that specific IMAP/POP3 connection. Initially I thought it would only be > useful for tracking connections going through Dovecot proxies to backends > (the session ID is forwarded) Sounds like an interesting debugging feature. > Apr 28 17:38:27 imap-login: Info: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=14338, secured, session= > The last line requires a modified mail_log_prefix, I wasn't planning on changing its default. What does your mail_log_prefix look like? Can you document the needed variable at http://wiki2.dovecot.org/Variables please? > 1) Does this change break someone's auth log line parsing? We don't do any parsing, just sending logs to a central syslog server for debugging purposes. > 2) Does the auth prefix look prettier with or without <> around session id? :) It should be separated using a fixed name like session=... for easy parsing. Regards, Daniel From jeep at rahul.net Sat Apr 28 23:54:45 2012 From: jeep at rahul.net (Jeff Lacki) Date: Sat, 28 Apr 2012 13:54:45 -0700 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query Message-ID: <20120428205446.0C08D1298B0@aqua.rahul.net> Hello- Sorry if this is a noob question, but I cant seem to get my password_query to work with dovecot + mysql. Im using 'Password verification by SQL server' at: http://wiki2.dovecot.org/AuthDatabase/SQL trying to modify it to work with my encrypted passwords in the DB. Im using the following which isnt working: password_query = SELECT NULL AS password, \ 'Y' as nopassword, userid AS user \ FROM users WHERE userid='%u' AND AES_DECRYPT(password, 'mykey')=password 1. Is it even possible to do this via 'password_query'? 2. If so, what am I doing wrong? Thanks, Jeff /mf/home/jeep/shell/.signature From gedalya at gedalya.net Sun Apr 29 00:16:59 2012 From: gedalya at gedalya.net (Gedalya) Date: Sat, 28 Apr 2012 17:16:59 -0400 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <20120428205446.0C08D1298B0@aqua.rahul.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> Message-ID: <4F9C5E4B.5080704@gedalya.net> On 04/28/2012 04:54 PM, Jeff Lacki wrote: > Hello- > > Sorry if this is a noob question, but I cant seem to get my password_query to > work with dovecot + mysql. Im using 'Password verification by SQL server' at: > > http://wiki2.dovecot.org/AuthDatabase/SQL > > trying to modify it to work with my encrypted passwords in the DB. > > Im using the following which isnt working: > > password_query = SELECT NULL AS password, \ > 'Y' as nopassword, userid AS user \ > FROM users WHERE userid='%u' AND AES_DECRYPT(password, 'mykey')=password > > 1. Is it even possible to do this via 'password_query'? > > 2. If so, what am I doing wrong? > > Thanks, > Jeff > > /mf/home/jeep/shell/.signature The wiki says: "The password is in %w variable" Maybe you meant password_query = SELECT NULL AS password, \ 'Y' as nopassword, userid AS user \ FROM users WHERE userid='%u' AND AES_DECRYPT(password, 'mykey')='%w' From jeep at rahul.net Sun Apr 29 00:25:36 2012 From: jeep at rahul.net (Jeff Lacki) Date: Sat, 28 Apr 2012 14:25:36 -0700 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <4F9C5E4B.5080704@gedalya.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <4F9C5E4B.5080704@gedalya.net> Message-ID: <20120428212537.20EDF12954C@aqua.rahul.net> > > > > /mf/home/jeep/shell/.signature > The wiki says: "The password is in %w variable" > Maybe you meant > > password_query = SELECT NULL AS password, \ > 'Y' as nopassword, userid AS user \ > FROM users WHERE userid='%u' AND AES_DECRYPT(password, 'mykey')='%w' > > Thanks, but when I do that, I get an empty value for '%w', not sure why? Apr 28 14:23:48 mydomain dovecot: auth-worker(13349): Debug: sql(jeff,127.0.0.1): query: SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users WHERE userid='jeff' AND AES_DECRYPT(password, 'mykey')='' /mf/home/jeep/shell/.signature From daniel.parthey at informatik.tu-chemnitz.de Sun Apr 29 01:11:29 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 29 Apr 2012 00:11:29 +0200 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <20120428205446.0C08D1298B0@aqua.rahul.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> Message-ID: <20120428221129.GA24819@daniel.localdomain> Hi Jeff, Jeff Lacki wrote: > Sorry if this is a noob question, but I cant seem to get my password_query to > work with dovecot + mysql. Im using 'Password verification by SQL server' at: > > http://wiki2.dovecot.org/AuthDatabase/SQL > > trying to modify it to work with my encrypted passwords in the DB. > > Im using the following which isn't working: > > password_query = SELECT NULL AS password, \ > 'Y' as nopassword, userid AS user \ > FROM users WHERE userid='%u' AND AES_DECRYPT(password, 'mykey')=password > > 1. Is it even possible to do this via 'password_query'? Please provide your dovecot version and output of the following command: doveconf -n and the complete external sql query files without passwords. You might alsolet the SQL server compare the encrypted password in the database with the encrypted string: password_query = SELECT NULL AS password, \ 'Y' as nopassword, userid AS user \ FROM users WHERE userid='%u' AND AES_ENCRYPT('%w','mykey')=password Regards, Daniel From gedalya at gedalya.net Sun Apr 29 01:25:18 2012 From: gedalya at gedalya.net (Gedalya) Date: Sat, 28 Apr 2012 18:25:18 -0400 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <20120428212537.20EDF12954C@aqua.rahul.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <4F9C5E4B.5080704@gedalya.net> <20120428212537.20EDF12954C@aqua.rahul.net> Message-ID: <4F9C6E4E.6030608@gedalya.net> On 04/28/2012 05:25 PM, Jeff Lacki wrote: >>> /mf/home/jeep/shell/.signature >> The wiki says: "The password is in %w variable" >> Maybe you meant >> >> password_query = SELECT NULL AS password, \ >> 'Y' as nopassword, userid AS user \ >> FROM users WHERE userid='%u' AND AES_DECRYPT(password, 'mykey')='%w' >> >> > Thanks, but when I do that, I get an empty value for '%w', not sure why? > > Apr 28 14:23:48 mydomain dovecot: auth-worker(13349): Debug: sql(jeff,127.0.0.1): query: SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users WHERE userid='jeff' AND AES_DECRYPT(password, 'mykey')='' > > > /mf/home/jeep/shell/.signature Are you using a plaintext authentication mechanism? From jeep at rahul.net Sun Apr 29 01:28:43 2012 From: jeep at rahul.net (Jeff Lacki) Date: Sat, 28 Apr 2012 15:28:43 -0700 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <20120428221129.GA24819@daniel.localdomain> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <20120428221129.GA24819@daniel.localdomain> Message-ID: <20120428222843.C0F44129904@aqua.rahul.net> > > > > 1. Is it even possible to do this via 'password_query'? > > Please provide your dovecot version and output of the following command: > doveconf -n > and the complete external sql query files without passwords. > > You might alsolet the SQL server compare the encrypted > password in the database with the encrypted string: > > password_query = SELECT NULL AS password, \ > 'Y' as nopassword, userid AS user \ > FROM users WHERE userid='%u' AND AES_ENCRYPT('%w','mykey')=password > > Regards, > Daniel Thank you Daniel. I downloaded and compiled 2.1.5 yesterday. The problem seems to be that '%w' evaulates to an empty string: Debug: sql(jeff,127.0.0.1): query: SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users WHERE userid='jeff' AND AES_DECRYPT('', 'key')=password I also just noticed that version 2.0.15 in my output below is coming from somewhere? I did try setting things up under 2.0.15 initially last week, but wanted to be up to date so downloaded the latest yesterday. I never did get it all working under 2.0.15 either btw. dovecot -n -c /opt/dovecot/etc/dovecot/dovecot.conf # 2.0.15: /opt/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.35.14-106.fc14.x86_64 x86_64 Fedora release 14 (Laughlin) ext4 auth_debug = yes auth_debug_passwords = yes auth_mechanisms = cram-md5 auth_verbose = yes auth_verbose_passwords = plain default_client_limit = 225 first_valid_uid = 1000 listen = * lock_method = flock mail_location = mbox:/var/mail/%d/%1n/%n:INDEX=/var/indexes/%d/%1n/%n mail_privileged_group = mail mbox_lock_timeout = 1 mins mbox_write_locks = fcntl namespace { inbox = yes location = prefix = separator = . type = private } passdb { args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } protocols = imap service auth { inet_listener { port = 12345 } unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } user = $default_internal_user } service imap-login { inet_listener imap { port = 143 } service_count = 1 } ssl_cert = References: <20120428205446.0C08D1298B0@aqua.rahul.net> <20120428221129.GA24819@daniel.localdomain> <20120428222843.C0F44129904@aqua.rahul.net> Message-ID: <4F9C7160.6030904@gedalya.net> On 04/28/2012 06:28 PM, Jeff Lacki wrote: >>> 1. Is it even possible to do this via 'password_query'? >> Please provide your dovecot version and output of the following command: >> doveconf -n >> and the complete external sql query files without passwords. >> >> You might alsolet the SQL server compare the encrypted >> password in the database with the encrypted string: >> >> password_query = SELECT NULL AS password, \ >> 'Y' as nopassword, userid AS user \ >> FROM users WHERE userid='%u' AND AES_ENCRYPT('%w','mykey')=password >> >> Regards, >> Daniel > Thank you Daniel. I downloaded and compiled 2.1.5 yesterday. > The problem seems to be that '%w' evaulates to an empty string: > > Debug: sql(jeff,127.0.0.1): query: SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users WHERE userid='jeff' AND AES_DECRYPT('', 'key')=password > > I also just noticed that version 2.0.15 in my output below is coming from > somewhere? I did try setting things up under 2.0.15 initially last week, > but wanted to be up to date so downloaded the latest yesterday. I never did > get it all working under 2.0.15 either btw. > > dovecot -n -c /opt/dovecot/etc/dovecot/dovecot.conf > # 2.0.15: /opt/dovecot/etc/dovecot/dovecot.conf > # OS: Linux 2.6.35.14-106.fc14.x86_64 x86_64 Fedora release 14 (Laughlin) ext4 > auth_debug = yes > auth_debug_passwords = yes > auth_mechanisms = cram-md5 > auth_verbose = yes > auth_verbose_passwords = plain > default_client_limit = 225 > first_valid_uid = 1000 > listen = * > lock_method = flock > mail_location = mbox:/var/mail/%d/%1n/%n:INDEX=/var/indexes/%d/%1n/%n > mail_privileged_group = mail > mbox_lock_timeout = 1 mins > mbox_write_locks = fcntl > namespace { > inbox = yes > location = > prefix = > separator = . > type = private > } > passdb { > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > driver = sql > } > protocols = imap > service auth { > inet_listener { > port = 12345 > } > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0666 > user = postfix > } > user = $default_internal_user > } > service imap-login { > inet_listener imap { > port = 143 > } > service_count = 1 > } > ssl_cert = ssl_key = userdb { > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > driver = sql > } > userdb { > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > driver = sql > } > userdb { > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > driver = sql > } > protocol imap { > imap_idle_notify_interval = 1 mins > imap_max_line_length = 64 k > mail_max_userip_connections = 5 > } > > > /mf/home/jeep/shell/.signature Yeap, you seem to only allow cram-md5. In this case, you client isn't transmitting the actual password that the user is typing, so dovecot simply doesn't have the password you want it to put in %w. It rather has a digest of it. The only way to use a non-plaintext auth mechanism is to provide dovecot the correct password from the database in plaintext. http://wiki2.dovecot.org/Authentication/Mechanisms From jeep at rahul.net Sun Apr 29 02:02:42 2012 From: jeep at rahul.net (Jeff Lacki) Date: Sat, 28 Apr 2012 16:02:42 -0700 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <4F9C7160.6030904@gedalya.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <20120428221129.GA24819@daniel.localdomain> <20120428222843.C0F44129904@aqua.rahul.net> <4F9C7160.6030904@gedalya.net> Message-ID: <20120428230243.BCF50129947@aqua.rahul.net> Gedalya wrote: > On 04/28/2012 06:28 PM, Jeff Lacki wrote: > >>> 1. Is it even possible to do this via 'password_query'? > >> Please provide your dovecot version and output of the following command: > >> doveconf -n > >> and the complete external sql query files without passwords. > >> > >> You might alsolet the SQL server compare the encrypted > >> password in the database with the encrypted string: > >> > >> password_query = SELECT NULL AS password, \ > >> 'Y' as nopassword, userid AS user \ > >> FROM users WHERE userid='%u' AND AES_ENCRYPT('%w','mykey')=password > >> > >> Regards, > >> Daniel > > Thank you Daniel. I downloaded and compiled 2.1.5 yesterday. > > The problem seems to be that '%w' evaulates to an empty string: > > > > Debug: sql(jeff,127.0.0.1): query: SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users WHERE userid='jeff' AND AES_DECRYPT('', 'key')=password > > > > I also just noticed that version 2.0.15 in my output below is coming from > > somewhere? I did try setting things up under 2.0.15 initially last week, > > but wanted to be up to date so downloaded the latest yesterday. I never did > > get it all working under 2.0.15 either btw. > > > > dovecot -n -c /opt/dovecot/etc/dovecot/dovecot.conf > > # 2.0.15: /opt/dovecot/etc/dovecot/dovecot.conf > > # OS: Linux 2.6.35.14-106.fc14.x86_64 x86_64 Fedora release 14 (Laughlin) ext4 > > auth_debug = yes > > auth_debug_passwords = yes > > auth_mechanisms = cram-md5 > > auth_verbose = yes > > auth_verbose_passwords = plain > > default_client_limit = 225 > > first_valid_uid = 1000 > > listen = * > > lock_method = flock > > mail_location = mbox:/var/mail/%d/%1n/%n:INDEX=/var/indexes/%d/%1n/%n > > mail_privileged_group = mail > > mbox_lock_timeout = 1 mins > > mbox_write_locks = fcntl > > namespace { > > inbox = yes > > location = > > prefix = > > separator = . > > type = private > > } > > passdb { > > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > > driver = sql > > } > > protocols = imap > > service auth { > > inet_listener { > > port = 12345 > > } > > unix_listener /var/spool/postfix/private/auth { > > group = postfix > > mode = 0666 > > user = postfix > > } > > user = $default_internal_user > > } > > service imap-login { > > inet_listener imap { > > port = 143 > > } > > service_count = 1 > > } > > ssl_cert = > ssl_key = > userdb { > > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > > driver = sql > > } > > userdb { > > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > > driver = sql > > } > > userdb { > > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > > driver = sql > > } > > protocol imap { > > imap_idle_notify_interval = 1 mins > > imap_max_line_length = 64 k > > mail_max_userip_connections = 5 > > } > > > > > > /mf/home/jeep/shell/.signature > > Yeap, you seem to only allow cram-md5. In this case, you client isn't > transmitting the actual password that the user is typing, so dovecot > simply doesn't have the password you want it to put in %w. It rather has > a digest of it. > The only way to use a non-plaintext auth mechanism is to provide dovecot > the correct password from the database in plaintext. > > http://wiki2.dovecot.org/Authentication/Mechanisms Ok thank you both for your help. I see why I didnt get anything for %w now. After reading the docs the past few days Im a little unsure of what setup I should use then for security purposes (primarily potential sniffing). Obviously using DIGEST-MD5 (per the docs) is better than CRAM-MD5, but is that possible to use here if I am trying to match passwords out of my mysql DB? I certainly do not want to use plaintext (and quite honestly Im not sure what 'login' really means either, but thats another issue). My desire is: - Only virtual users - All users info stored in mysql - login is via squirrelmail ATM, and later imap/iphone or other. Security is my #1 focus right now. Can someone explain the best solution? Or is the best solution to just get an SSL cert and use plaintext? (which is actually my future plan). Thanks! Jeff /mf/home/jeep/shell/.signature From gedalya at gedalya.net Sun Apr 29 02:31:07 2012 From: gedalya at gedalya.net (Gedalya) Date: Sat, 28 Apr 2012 19:31:07 -0400 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <20120428230243.BCF50129947@aqua.rahul.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <20120428221129.GA24819@daniel.localdomain> <20120428222843.C0F44129904@aqua.rahul.net> <4F9C7160.6030904@gedalya.net> <20120428230243.BCF50129947@aqua.rahul.net> Message-ID: <4F9C7DBB.2000604@gedalya.net> On 04/28/2012 07:02 PM, Jeff Lacki wrote: > Security is my #1 focus right now. > > Can someone explain the best solution? Or is the best solution to just get > an SSL cert and use plaintext? (which is actually my future plan). You absolutely must use SSL if you want security. A non-plaintext authentication mechanism only obfuscates the password itself during the login stage. The IMAP session itself (email content) needs to be secured and that can be more important than the email password (people emailing to each other passwords to more interesting things). Getting your certificate signed by a recognized CA helps your clients to verify that the server they are talking to is the server they want to be talking to. It doesn't make the encryption any stronger. If your clients are willing to click "I know what I'm doing, I trust this certificate", then you have the same results. You can try to get a free certificate here - http://www.startssl.com/ - their certificates are trusted by Mozilla and Microsoft products but not by RIM (blackberry) or java. Anyway, given your current setup: you're not using SSL, you want to AES-encrypt your passwords in mysql (you don't trust your database server) and keep your encryption key in the dovecot configuration (you do trust your dovecot server), you can just do: password_query = SELECT AES_DECRYPT(password, 'mykey') AS password, \ userid AS user \ FROM users WHERE userid='%u' This would allow you to use a digest-based authentication mechanism. However, you still have the liability of having your users' passwords in a reversibly encrypted format, with the key available nearby. Once you get SSL set up, it would be better to store the passwords in a salted hash format such as SSHA, and use plaintext auth (over SSL, of course). From jeep at rahul.net Sun Apr 29 05:29:49 2012 From: jeep at rahul.net (Jeff Lacki) Date: Sat, 28 Apr 2012 19:29:49 -0700 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <4F9C7DBB.2000604@gedalya.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <20120428221129.GA24819@daniel.localdomain> <20120428222843.C0F44129904@aqua.rahul.net> <4F9C7160.6030904@gedalya.net> <20120428230243.BCF50129947@aqua.rahul.net> <4F9C7DBB.2000604@gedalya.net> Message-ID: <20120429022949.9559D129606@aqua.rahul.net> > salted hash format such as SSHA, and use plaintext auth (over SSL, of > course). Thank you so much for your in depth reponse Gedalya, I appreciate it! So now that I changed things up and my conf is: auth_mechanisms = plain login disable_plaintext_auth = no I get the following, which I dont get, because now it seems to be telling me that my decrypted AES password needs to be MD5-CRYPT or PLAIN-MD5? Debug: sql(jeff,127.0.0.1): query: SELECT AES_DECRYPT(password, 'mykey') AS password, userid AS user FROM users WHERE userid='jeff' AND AES_DECRYPT(password, 'mykey')='mypass' Error: sql(jeff,127.0.0.1): Invalid password 'mypass' in passdb: Not a valid MD5-CRYPT or PLAIN-MD5 password Debug: sql(jeff,127.0.0.1): MD5(mypass) != 'mypass', try PLAIN scheme instead Apr 28 19:18:56 mydomain dovecot: auth: Debug: client out: FAIL#0111#011user=jeff This is even more confusing as my mechanisms say plain, not any type of MD5? Thanks again! /mf/home/jeep/shell/.signature From gedalya at gedalya.net Sun Apr 29 05:53:20 2012 From: gedalya at gedalya.net (Gedalya) Date: Sat, 28 Apr 2012 22:53:20 -0400 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <20120429022949.9559D129606@aqua.rahul.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <20120428221129.GA24819@daniel.localdomain> <20120428222843.C0F44129904@aqua.rahul.net> <4F9C7160.6030904@gedalya.net> <20120428230243.BCF50129947@aqua.rahul.net> <4F9C7DBB.2000604@gedalya.net> <20120429022949.9559D129606@aqua.rahul.net> Message-ID: <4F9CAD20.9020706@gedalya.net> On 4/28/2012 10:29 PM, Jeff Lacki wrote: > Error: sql(jeff,127.0.0.1): Invalid password 'mypass' in passdb: Not a valid MD5-CRYPT or PLAIN-MD5 password > Debug: sql(jeff,127.0.0.1): MD5(mypass) != 'mypass', try PLAIN scheme instead In dovecot-sql.conf.ext, set: default_pass_scheme = PLAIN Dovecot needs to be told what format the stored password is in. From ml at well-adjusted.de Sun Apr 29 14:04:51 2012 From: ml at well-adjusted.de (Jochen Spieker) Date: Sun, 29 Apr 2012 13:04:51 +0200 Subject: [Dovecot] Sieve doesn't find user scripts Message-ID: <20120429110451.GF12090@wasteland.homelinux.net> Hi, I want to use Sieve filtering with my Dovecot 1.2 installation on Debian squeeze. I have a virtual domain setup using Portgresql. ManageSieve works fine so far, I can edit and activate/deactive scripts (using Thunderbird + Plugin) and they show up in the filesystem where I expect them to be, see below. The problem is that LDA doesn't find the script. From /var/log/dovecot-deliver.log: | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Loading modules from directory: /usr/lib/dovecot/modules/lda | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: uid=105 | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: gid=109 | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Quota root: name=storage=512000 backend=maildir args= | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: maildir: data=/srv/mail/well-adjusted.de/jrspieker | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: maildir++: root=/srv/mail/well-adjusted.de/jrspieker, index=, control=, inbox= | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: sieve: user has no valid personal script | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: sieve: no scripts to execute: reverting to default delivery. AFAICS, the sieve module is loaded and the home directory is correct. The content of this directory looks like this: | $ ls -ltRa /srv/mail/home/well-adjusted.de/jrspieker/ | /srv/mail/home/well-adjusted.de/jrspieker/: | total 12 | drwx------ 3 vmail vmail 4096 Apr 29 12:22 . | lrwxrwxrwx 1 vmail vmail 16 Apr 29 12:22 .dovecot.sieve -> sieve/test.sieve | drwx------ 3 vmail vmail 4096 Apr 28 22:31 sieve | drwx------ 3 vmail vmail 4096 Apr 27 20:55 .. | | /srv/mail/home/well-adjusted.de/jrspieker/sieve: | total 16 | drwx------ 3 vmail vmail 4096 Apr 29 12:22 .. | drwx------ 3 vmail vmail 4096 Apr 28 22:31 . | drwx------ 2 vmail vmail 4096 Apr 28 22:31 tmp | -rw------- 1 vmail vmail 338 Apr 28 22:31 test.sieve | ? dovecot -n shows this for the plugin section: | plugin: | quota: maildir:storage=512000 | sieve_dir: ~/sieve/ | sieve: ~/.dovecot.sieve And finally, the content of the script: | require["fileinto"]; | | if header :is "subject" "test" { | fileinto "Test"; | } else { | keep; | } What am I missing here? Thanks, Jochen. -- Fashion is more important to me than war, famine, disease or art. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From giles at coochey.net Sun Apr 29 14:47:04 2012 From: giles at coochey.net (Giles Coochey) Date: Sun, 29 Apr 2012 12:47:04 +0100 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <20120429110451.GF12090@wasteland.homelinux.net> References: <20120429110451.GF12090@wasteland.homelinux.net> Message-ID: <4F9D2A38.1010400@coochey.net> On 29/04/2012 12:04, Jochen Spieker wrote: > Hi, > > I want to use Sieve filtering with my Dovecot 1.2 installation on Debian > squeeze. I have a virtual domain setup using Portgresql. > > ManageSieve works fine so far, I can edit and activate/deactive scripts (using > Thunderbird + Plugin) and they show up in the filesystem where I expect them to > be, see below. > > The problem is that LDA doesn't find the script. From > /var/log/dovecot-deliver.log: > > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Loading modules from directory: /usr/lib/dovecot/modules/lda > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: uid=105 > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: gid=109 > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Quota root: name=storage=512000 backend=maildir args= > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: maildir: data=/srv/mail/well-adjusted.de/jrspieker > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: maildir++: root=/srv/mail/well-adjusted.de/jrspieker, index=, control=, inbox= > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: sieve: user has no valid personal script > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: sieve: no scripts to execute: reverting to default delivery. > > AFAICS, the sieve module is loaded and the home directory is correct. The > content of this directory looks like this: > > | $ ls -ltRa /srv/mail/home/well-adjusted.de/jrspieker/ > | /srv/mail/home/well-adjusted.de/jrspieker/: > | total 12 > | drwx------ 3 vmail vmail 4096 Apr 29 12:22 . > | lrwxrwxrwx 1 vmail vmail 16 Apr 29 12:22 .dovecot.sieve -> sieve/test.sieve > | drwx------ 3 vmail vmail 4096 Apr 28 22:31 sieve > | drwx------ 3 vmail vmail 4096 Apr 27 20:55 .. > | > | /srv/mail/home/well-adjusted.de/jrspieker/sieve: > | total 16 > | drwx------ 3 vmail vmail 4096 Apr 29 12:22 .. > | drwx------ 3 vmail vmail 4096 Apr 28 22:31 . > | drwx------ 2 vmail vmail 4096 Apr 28 22:31 tmp > | -rw------- 1 vmail vmail 338 Apr 28 22:31 test.sieve > | ? > What user is deliver running as? -- Best Regards, Giles Coochey, CCNA Security, CCNA NetSecSpec Ltd giles.coochey at netsecspec.co.uk Tel: +44 (0) 7983 877 438 Live Messenger: giles at coochey.net http://www.netsecspec.co.uk http://www.coochey.net -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4928 bytes Desc: S/MIME Cryptographic Signature URL: From ml at well-adjusted.de Sun Apr 29 16:15:08 2012 From: ml at well-adjusted.de (Jochen Spieker) Date: Sun, 29 Apr 2012 15:15:08 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <4F9D2A38.1010400@coochey.net> References: <20120429110451.GF12090@wasteland.homelinux.net> <4F9D2A38.1010400@coochey.net> Message-ID: <20120429131508.GG12090@wasteland.homelinux.net> Giles Coochey: > On 29/04/2012 12:04, Jochen Spieker wrote: >> >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Loading modules from directory: /usr/lib/dovecot/modules/lda >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: uid=105 >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: gid=109 These IDs translate to user/group vmail: | $ grep 105 /etc/passwd | vmail:x:105:109::/home/vmail:/bin/false | | $ grep 109 /etc/group | vmail:x:109: > What user is deliver running as? Should be vmail: /etc/postfix/master.cf: | dovecot unix - n n - 20 pipe | flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} All mailboxes and home directories belong to this user and group. J. -- If nightclub doormen recognised me I would be more fulfilled. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From ml at well-adjusted.de Sun Apr 29 16:44:12 2012 From: ml at well-adjusted.de (Jochen Spieker) Date: Sun, 29 Apr 2012 15:44:12 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <20120429110451.GF12090@wasteland.homelinux.net> References: <20120429110451.GF12090@wasteland.homelinux.net> Message-ID: <20120429134411.GH12090@wasteland.homelinux.net> Jochen Spieker: > > The problem is that LDA doesn't find the script. Ok, using a global script works: deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: auth input: uid=105 deliver(jrspieker at well-adjusted.de): Info: auth input: gid=109 deliver(jrspieker at well-adjusted.de): Info: Quota root: name=storage=512000 backend=maildir args= deliver(jrspieker at well-adjusted.de): Info: maildir: data=/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: maildir++: root=/srv/mail/well-adjusted.de/jrspieker, index=, control=, inbox=/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: sieve: using sieve path for user's script: /srv/mail/home/global/sieve/global.sieve deliver(jrspieker at well-adjusted.de): Info: sieve: opening script /srv/mail/home/global/sieve/global.sieve deliver(jrspieker at well-adjusted.de): Info: sieve: executing compiled script /srv/mail/home/global/sieve/global.sieve deliver(jrspieker at well-adjusted.de): Info: Namespace : Using permissions from /srv/mail/well-adjusted.de/jrspieker: mode=0755 gid=-1 deliver(jrspieker at well-adjusted.de): Info: sieve: msgid=<20120429132328.D581B1E9 at manowar.wasteland.homelinux.net>: stored mail into mailbox 'INBOX' I suspect LDA doesn't look for the script where I expect it to look. Again, dovecot -n show this (now including the global settings): plugin: quota: maildir:storage=512000 sieve_dir: ~/sieve/ sieve: ~/.dovecot.sieve sieve_global_path: /srv/mail/home/global/sieve/global.sieve sieve_global_dir: /srv/mail/home/global/sieve/ I would expect ~ to translate to the userdb_home from the log entry above (/srv/mail/home/well-adjusted.de/jrspieker). ManageSieve's behaviour confirms that expectation. The wiki says "A relative path (or just a filename) will be interpreted to point under the user's home directory."? This works for ManageSieve just like the ~-notation, but LDA still doesn't find the script. J. ? http://wiki.dovecot.org/LDA/Sieve/Dovecot#Per-user_Sieve_script_location -- When I get home from the supermarket I don't know what to do with all the plastic. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From ml at well-adjusted.de Sun Apr 29 16:59:46 2012 From: ml at well-adjusted.de (Jochen Spieker) Date: Sun, 29 Apr 2012 15:59:46 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <20120429134411.GH12090@wasteland.homelinux.net> References: <20120429110451.GF12090@wasteland.homelinux.net> <20120429134411.GH12090@wasteland.homelinux.net> Message-ID: <20120429135945.GI12090@wasteland.homelinux.net> Someone asked for the complete dovecot -n: # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.34-xen-r4 i686 Debian 6.0.4 rootfs log_timestamp: %Y-%m-%d %H:%M:%S protocols: pop3s imaps managesieve ssl_cert_file: /etc/ssl/certs/mail.well-adjusted.de.cert ssl_key_file: /etc/ssl/private/mail.well-adjusted.de.key verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login login_processes_count: 1 login_max_processes_count: 32 first_valid_uid: 1 mail_privileged_group: vmail mail_location: maildir:/srv/mail/%Ld/%Ln fsync_disable: yes mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve lda: fsync_disable: no postmaster_address: postmaster at wasteland.xen-host.de log_path: /var/log/dovecot-deliver.log info_log_path: /var/log/dovecot-deliver.log mail_plugins: quota sieve auth_socket_path: /var/run/dovecot/auth-master mail_debug: yes auth default: mechanisms: plain login user: postgres username_chars: +*abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ username_format: %Lu passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: prefetch userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix master: path: /var/run/dovecot/auth-master mode: 432 user: vmail group: vmail plugin: quota: maildir:storage=512000 sieve_dir: sieve/ sieve: .dovecot.sieve sieve_global_path: /srv/mail/home/global/sieve/global.sieve sieve_global_dir: /srv/mail/home/global/sieve/ J. -- Tony Blair is a hypnotised self-seeking scarecrow just like all the rest. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From daniel.parthey at informatik.tu-chemnitz.de Mon Apr 30 01:49:28 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Mon, 30 Apr 2012 00:49:28 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <20120429135945.GI12090@wasteland.homelinux.net> References: <20120429110451.GF12090@wasteland.homelinux.net> <20120429134411.GH12090@wasteland.homelinux.net> <20120429135945.GI12090@wasteland.homelinux.net> Message-ID: <20120429224927.GA9001@daniel.localdomain> Hi Jochen, Jochen Spieker wrote: > mail_location: maildir:/srv/mail/%Ld/%Ln > plugin: > sieve_dir: sieve/ > sieve: .dovecot.sieve Sieve requires a mail_home directory. The sieve_dir is searched below mail_home, not mail_location where the actual mailboxes are stored. http://wiki.dovecot.org/VirtualUsers/Home Home vs. mail directory Regards, Daniel From snabb at epipe.com Sun Apr 29 21:09:43 2012 From: snabb at epipe.com (Janne Snabb) Date: Sun, 29 Apr 2012 18:09:43 +0000 (UTC) Subject: [Dovecot] setacl fails - does not find dovecot-acl file In-Reply-To: <1320442758.21919.222.camel@hurina> References: <4EA2C213.7030204@mike2k.de> <1320436059.21919.152.camel@hurina> <4EB44B1C.90505@mike2k.de> <1320442758.21919.222.camel@hurina> Message-ID: On Fri, 4 Nov 2011, Timo Sirainen wrote: > On Fri, 2011-11-04 at 21:29 +0100, Michael Stilkerich wrote: > > > Nov 4 16:29:03 keira dovecot: imap(isa): Error: fcntl(unlock) locking > > failed for file /home/dovecot/isa/dovecot.index.log: No such file or > > directory > > Nov 4 16:29:03 keira dovecot: imap(isa): Error: fstat() failed with > > file /home/dovecot/isa/dovecot.index.log: No such file or directory > > These simply shouldn't happen. I'd say it's a kernel bug. You're running > a default Ubuntu kernel? I wonder if other Ubuntu users have this > problem. I am seeing this same problem on Ubuntu 11.10 and 12.04 with stock kernels. The problem is clearly AppArmor related. The imap process seems to be using the "usr.sbin.dovecot" profile which prevents access to these files. There is a separate profile "usr.lib.dovecot.imap" but it seems that it does not get applied to the imap process for some odd reason. This is especially strange beacuse both profiles are enabled in "complain" and not in "enforce" mode, thus they should not enforce any of the rules. I am simultaneously getting messages similar to the following in my audit log: type=AVC msg=audit(1335712674.515:655016): apparmor="ALLOWED" operation="getattr" parent=10922 profile="/usr/sbin/dovecot//null-107//null-10b//null-118" name="/home/foobar/Maildir/.foobar/dovecot.index.log" pid=10937 comm="imap" requested_mask="r" denied_mask="r" fsuid=1000 ouid=1000 After disabling the "usr.sbin.dovecot" profile everything seems fine. Other dovecot related AppArmor profiles do not seem to cause problems. This looks like an issue in AppArmor to me... -- Janne Snabb / EPIPE Communications snabb at epipe.com - http://epipe.com/ From sdavies at sdc.com.au Mon Apr 30 04:08:37 2012 From: sdavies at sdc.com.au (Stephen Davies) Date: Mon, 30 Apr 2012 10:38:37 +0930 Subject: [Dovecot] Log messages Message-ID: <201204301038.37694.sdavies@sdc.com.au> Hello. I have been on leave for several weeks and have managed to lose all emails received since December due to a server crash. I asked about log synch error back in March and have (repeatedly) deleted all .imap files but the errors continue. # 2.1.1: /usr/etc/dovecot/dovecot.conf # OS: Linux 2.6.33.5-desktop-2mnb i686 Mandriva Linux 2010.2 mail_access_groups = mail mail_location = mbox:~/Mail:INBOX=/var/spool/mail/%u managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve sieve ssl_cert = References: <20120429110451.GF12090@wasteland.homelinux.net> <20120429134411.GH12090@wasteland.homelinux.net> <20120429135945.GI12090@wasteland.homelinux.net> <20120429224927.GA9001@daniel.localdomain> Message-ID: <20120430053946.GK12090@wasteland.homelinux.net> Daniel Parthey: > Jochen Spieker wrote: > >> mail_location: maildir:/srv/mail/%Ld/%Ln >> plugin: >> sieve_dir: sieve/ >> sieve: .dovecot.sieve > > Sieve requires a mail_home directory. Is this different from userdb_home? Both mail and home directory are returned from my user/password_queries and LDA reports exactly what I expect: deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker ManageSieve puts scripts under the directory reported as userdb_home: $ ls -ltra /srv/mail/home/well-adjusted.de/jrspieker total 12 drwx------ 3 vmail vmail 4096 Apr 27 20:55 .. drwx------ 3 vmail vmail 4096 Apr 28 22:31 sieve lrwxrwxrwx 1 vmail vmail 16 Apr 29 15:00 .dovecot.sieve -> sieve/test.sieve drwx------ 3 vmail vmail 4096 Apr 29 15:00 . J. -- I feel yawning hollowness whilst talking to people at parties. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From oni-neko at gmx.net Mon Apr 30 11:59:32 2012 From: oni-neko at gmx.net (oni-neko at gmx.net) Date: Mon, 30 Apr 2012 10:59:32 +0200 Subject: [Dovecot] restoring mails and directories Message-ID: <20120430085932.267600@gmx.net> Good day! is there an easy way to restore e.g. deleted mails and/or mail directories? let me specify: I'm running dovecot 1.2.9 on a current ubuntu lts. It is set to Maildir and I backup the folder with the users maildata nightly via rsync. now, when someone deletes a mail that they still need, and I copy it back from the backup, it will not be visible to mail clients. When I change the mailfile by a letter or so, it shows. that's easy enough, if not exactly awesome ;-) But, if said person deletes a whole folder of mails, this way to make restored mails show up becomes rather unreasonable. surely there is a better way to do this? I'd be glad for any and all hints! =) best regards silvana -- Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de From reuben-dovecot at reub.net Mon Apr 30 13:43:10 2012 From: reuben-dovecot at reub.net (Reuben Farrelly) Date: Mon, 30 Apr 2012 20:43:10 +1000 Subject: [Dovecot] Dovecot dsync mail replication issues Message-ID: <4F9E6CBE.4070609@reub.net> Hi, I'm trying to set up some dsync based replication between two hosts on my network. The current topology is a single server running Postfix with a single dovecot installation with a Maildir per user (only 4 users including myself). No NFS, just local system users on ext4. I am only using system users, ie no virtual users. I am using dovecot deliver to deliver mails into the Maildir's. This - being a very standard Maildir installation - all works just fine. I'm running dovecot-2.1.5 (release versions) x86_64 with Gentoo on both hosts and using key based ssh to transport the data using the root login. What I would like to do is extend the design so as to replicate the Maildirs across a second machine and in the future be able to connect via IMAP into either machine to sync mail. Two way dsync would be rather cool because I could then deliver via SMTP to either system and have changes automagically propagate - but initially just simple replication would be a good start. I was hoping that dsync would allow me to do this, but I've run into quite a number of problems while getting this to work. http://dovecot.org/list/dovecot/2012-March/064243.html ...was very useful and I've based my config on that. Initially I've tried to sync up the user Maildirs, and this has more or less worked: doveadm sync -u lyn remote:root at dustbowl.reub.net This succeeds without error on the initial sync. However if I try to run the re-sync again (such as a use case of if the sync of another Maildir takes 4 hours so I want to resync up the earlier ones again) I end up with a mysteriously named INBOX folder in both the source and destination Maildirs: drwx------ 5 lyn lyn 4096 Apr 30 19:32 .INBOX_7a86a62d465a974fb92f00003b258734 It has the basic structure of a Maildir but is empty in terms of mails: drwx------ 2 lyn lyn 4096 Apr 30 19:32 cur -rw------- 1 lyn lyn 220 Apr 30 19:32 dovecot.index.log -rw------- 1 lyn lyn 51 Apr 30 19:32 dovecot-uidlist -rw------- 1 lyn lyn 0 Apr 30 19:32 maildirfolder drwx------ 2 lyn lyn 4096 Apr 30 19:32 new drwx------ 2 lyn lyn 4096 Apr 30 19:32 tmp First question: why is this random named directory being created in the origin Maildir? Shouldn't the replication be more or less read-only in the origin Maildir? Second question: If I re-attempt a doveadm sync a second time I get this error: tornado Maildir # doveadm sync -u lyn remote:root at dustbowl.reub.net dsync-local(lyn): Error: Can't rename mailbox INBOX_7a86a62d465a974fb92f00003b258734 to INBOX: Target mailbox already exists dsync-local(lyn): Error: Can't rename mailbox INBOX to INBOX_eb15f30ea563be4b703200002bd68bb1: Renaming INBOX isn't supported. tornado Maildir # It's not clear if the second attempt has failed or succeeded, and it's a bit odd that it errors out on a directory that the dovecot sync process itself has created. Third question: Upon starting Dovecot lots of ugliness is logged in the mail log: Apr 30 19:44:59 tornado dovecot: master: Dovecot v2.1.5 starting up (core dumps disabled) Apr 30 19:44:59 tornado dovecot: doveadm(mozsync): Error: user mozsync: Initialization failed: Namespace '': mkdir(/var/www/xxx/server-full/Maildir) failed: Permission denied (euid=1016(mozsync) egid=1016() missing +w perm: /var/www/xxx/server-full, dir owned by 0:0 mode=0755) Apr 30 19:44:59 tornado dovecot: doveadm(mozsync): Error: sync: User init failed Apr 30 19:44:59 tornado dovecot: doveadm(mozsync): Warning: I/O leak: 0x414190 (line 102, fd 16) Apr 30 19:44:59 tornado dovecot: dsync-local(cisco): Error: remote: doveadm(cisco): Fatal: User doesn't exist Apr 30 19:44:59 tornado dovecot: dsync-local(cisco): Error: read() from worker server failed: EOF Users mozsync and cisco are not valid mail users and it's not appropriate that Dovecot tries to create a Maildir for either of them. The users are system unprivileged users only, and do not ever send or receive mail. And - I/O leak? ;) Also, user cisco is local to one box only, and does not exist (nor does it need to) on the remote host. So any complaints about this user are invalid and dovecot needs to ignore replication for this user anyway. Fourth question, upon starting dovecot again, mail.log then spews several hundred of these messages: Apr 30 19:45:06 tornado dovecot: dsync-local(reuben): Error: msg-get failed: box=Trash uid=114863 guid=1335382569.M98089P29952.tornado,S=6479,W=6625 Before aborting entirely with: Apr 30 19:45:09 tornado dovecot: imap: Server shutting down. in=328 out=2042 It seems to me that a a few of those problems logged could be solved by being able to specify which system users to synchronise, rather than Dovecot making a blind assumption that all users actually have valid Maildirs that need to be created, and all need to be sync'd between two hosts. Subsequent delivery based sync'ing fails silently (pending more investigation) but I'd like to try and fix the above listed problems first before worrying about the later stages of the sync not working. Has anyone else run into these problems or found better ways of making this type of setup all work? Thanks, Reuben From stephan at rename-it.nl Mon Apr 30 16:06:07 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 30 Apr 2012 15:06:07 +0200 Subject: [Dovecot] Refiltering mails without loosing flags In-Reply-To: References: <4EE5E8FF.8080306@rename-it.nl> Message-ID: <4F9E8E3F.1010004@rename-it.nl> On 4/28/2012 6:07 PM, Dieter Knopf wrote: > 2011/12/12 Stephan Bosch: > >> Since last week, the new sieve-filter command line tool, which is intended >> for just such a task, is part of the normal compile output for Pigeonhole >> 0.3.x for Dovecot v2.1. Unfortunately, it is not released yet, so I guess >> you're stuck with your old getmail-based solution for now. >> >> The tool is still experimental and I must say that I have not tested the >> preservation of flags for some time and many changes were performed since >> then. I will verify that later today. Also, the interaction with the new >> editheader extension needs attention. > Is there any update about that? Would be really nice to have an nice > and easy way > like with procmail. Just via command line not via getmail/imap/.... I didn't receive much feedback so far, so you may very well encounter some bugs. But I did test flag preservation just now and that just works. Pigeonhole v0.3.0 is released and contains all you need. Just experiment with it a bit before you start using it for real mail. Regards, Stephan. From thierry at odry.net Mon Apr 30 17:07:33 2012 From: thierry at odry.net (Thierry de Montaudry) Date: Mon, 30 Apr 2012 16:07:33 +0200 Subject: [Dovecot] restoring mails and directories In-Reply-To: <20120430085932.267600@gmx.net> References: <20120430085932.267600@gmx.net> Message-ID: <98F4E6D0-A89B-408A-ACC1-7BC15FAC5231@odry.net> On 30 Apr 2012, at 10:59, oni-neko at gmx.net wrote: > Good day! > > is there an easy way to restore e.g. deleted mails and/or mail directories? > let me specify: I'm running dovecot 1.2.9 on a current ubuntu lts. It is set to Maildir and I backup the folder with the users maildata nightly via rsync. > > now, when someone deletes a mail that they still need, and I copy it back from the backup, it will not be visible to mail clients. When I change the mailfile by a letter or so, it shows. that's easy enough, if not exactly awesome ;-) > > But, if said person deletes a whole folder of mails, this way to make restored mails show up becomes rather unreasonable. > > surely there is a better way to do this? I'd be glad for any and all hints! =) > > best regards > silvana > -- > Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir > belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de Hi, Not sure about the availability on version 1.2.9, but the following command should do the trick: doveadm force-resync -u Regards, Thierry From stephan at rename-it.nl Mon Apr 30 17:12:03 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 30 Apr 2012 16:12:03 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <20120429110451.GF12090@wasteland.homelinux.net> References: <20120429110451.GF12090@wasteland.homelinux.net> Message-ID: <4F9E9DB3.1000707@rename-it.nl> On 4/29/2012 1:04 PM, Jochen Spieker wrote: > Hi, > > I want to use Sieve filtering with my Dovecot 1.2 installation on Debian > squeeze. I have a virtual domain setup using Portgresql. > > ManageSieve works fine so far, I can edit and activate/deactive scripts (using > Thunderbird + Plugin) and they show up in the filesystem where I expect them to > be, see below. > > The problem is that LDA doesn't find the script. From > /var/log/dovecot-deliver.log: > > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Loading modules from directory: /usr/lib/dovecot/modules/lda > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: uid=105 > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: gid=109 > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Quota root: name=storage=512000 backend=maildir args= > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: maildir: data=/srv/mail/well-adjusted.de/jrspieker > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: maildir++: root=/srv/mail/well-adjusted.de/jrspieker, index=, control=, inbox= > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: sieve: user has no valid personal script > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: sieve: no scripts to execute: reverting to default delivery. > > AFAICS, the sieve module is loaded and the home directory is correct. The > content of this directory looks like this: Could you enable mail_debug in your configuration? This will provide more detail on why it is not finding a script. Regards, Stephan. From ml at well-adjusted.de Mon Apr 30 17:43:58 2012 From: ml at well-adjusted.de (Jochen Spieker) Date: Mon, 30 Apr 2012 16:43:58 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <4F9E9DB3.1000707@rename-it.nl> References: <20120429110451.GF12090@wasteland.homelinux.net> <4F9E9DB3.1000707@rename-it.nl> Message-ID: <20120430144358.GL12090@wasteland.homelinux.net> Stephan Bosch: >> >> AFAICS, the sieve module is loaded and the home directory is correct. The >> content of this directory looks like this: > > Could you enable mail_debug in your configuration? This will provide > more detail on why it is not finding a script. With globally enabled mail_debug and auth_debug I get this in mail.log: postfix/qmgr[30005]: E306113C1E1: from=, size=1987, nrcpt=1 (queue active) dovecot: auth(default): master in: USER 1 jrspieker at well-adjusted.de service=deliver dovecot: auth(default): prefetch(jrspieker at well-adjusted.de): passdb didn't return userdb entries, trying the next userdb dovecot: auth(default): sql(jrspieker at well-adjusted.de): SELECT u.storage || ':' || d.basedir || u.userid as mail, '/srv/mail/home/well-adjusted.de/jrspieker' as userdb_home, 105 as uid, 109 as gid FROM mailuser u, domain d WHERE u.userid = 'jrspieker' AND u.domain = 'well-adjusted.de' AND u.domain = d.name dovecot: auth(default): master out: USER 1 jrspieker at well-adjusted.de mail=maildir:/srv/mail/well-adjusted.de/jrspieker userdb_home=/srv/mail/home/well-adjusted.de/jrspieker uid=105 gid=109 postfix/smtpd[5421]: disconnect from e177130134.adsl.alicedsl.de[85.177.130.134] postfix/pipe[5428]: E306113C1E1: to=, relay=dovecot, delay=1.1, delays=0.6/0/0/0.5, dsn=2.0.0, status=sent (delivered via dovecot service) postfix/qmgr[30005]: E306113C1E1: removed and this in dovecot-deliver.log: deliver(jrspieker at well-adjusted.de): Info: Loading modules from directory: /usr/lib/dovecot/modules/lda deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: auth input: uid=105 deliver(jrspieker at well-adjusted.de): Info: auth input: gid=109 deliver(jrspieker at well-adjusted.de): Info: Quota root: name=storage=512000 backend=maildir args= deliver(jrspieker at well-adjusted.de): Info: maildir: data=/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: maildir++: root=/srv/mail/well-adjusted.de/jrspieker, index=, control=, inbox=/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: sieve: user has no valid personal script deliver(jrspieker at well-adjusted.de): Info: sieve: no scripts to execute: reverting to default delivery. deliver(jrspieker at well-adjusted.de): Info: Namespace : Using permissions from /srv/mail/well-adjusted.de/jrspieker: mode=0755 gid=-1 deliver(jrspieker at well-adjusted.de): Info: msgid=<20120430142818.354B31FE at manowar.wasteland.homelinux.net>: saved mail to INBOX Doesn't look different than before, I had already enabled mail_debug in the "protocol lda" section. J. -- I am not scared of death but terrified of people in Tommy Hilfiger sweatshirts. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From tss at iki.fi Mon Apr 30 18:18:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 30 Apr 2012 18:18:03 +0300 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <20120430144358.GL12090@wasteland.homelinux.net> References: <20120429110451.GF12090@wasteland.homelinux.net> <4F9E9DB3.1000707@rename-it.nl> <20120430144358.GL12090@wasteland.homelinux.net> Message-ID: <1B683452-BBC7-4D97-ACCF-CC1A6568F1A3@iki.fi> On 30.4.2012, at 17.43, Jochen Spieker wrote: > deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker It should say "home" here, not "userdb_home". From ml at well-adjusted.de Mon Apr 30 18:40:52 2012 From: ml at well-adjusted.de (Jochen Spieker) Date: Mon, 30 Apr 2012 17:40:52 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <1B683452-BBC7-4D97-ACCF-CC1A6568F1A3@iki.fi> References: <20120429110451.GF12090@wasteland.homelinux.net> <4F9E9DB3.1000707@rename-it.nl> <20120430144358.GL12090@wasteland.homelinux.net> <1B683452-BBC7-4D97-ACCF-CC1A6568F1A3@iki.fi> Message-ID: <20120430154052.GM12090@wasteland.homelinux.net> Timo Sirainen: > On 30.4.2012, at 17.43, Jochen Spieker wrote: > >> deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker > > It should say "home" here, not "userdb_home". Thanks, that's it! dovecot-deliver.log now reads: deliver(jrspieker at well-adjusted.de): Info: auth input: home=/srv/mail/home/well-adjusted.de/jrspieker ? deliver(jrspieker at well-adjusted.de): Info: sieve: using sieve path for user's script: /srv/mail/home/well-adjusted.de/jrspieker/.dovecot.sieve deliver(jrspieker at well-adjusted.de): Info: sieve: opening script /srv/mail/home/well-adjusted.de/jrspieker/.dovecot.sieve deliver(jrspieker at well-adjusted.de): Info: sieve: executing compiled script /srv/mail/home/well-adjusted.de/jrspieker/.dovecot.sieve Now I can finally start to migrate my .procmailrc. J. -- Driving behind lorries carrying hazardous chemicals makes me wish for a simpler life. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From andrei at lctax.ro Mon Apr 30 19:25:43 2012 From: andrei at lctax.ro (Michescu Andrei) Date: Mon, 30 Apr 2012 12:25:43 -0400 Subject: [Dovecot] Dovecot dsync mail replication issues In-Reply-To: <4F9E6CBE.4070609@reub.net> References: <4F9E6CBE.4070609@reub.net> Message-ID: Hello Reuben, I'm having a very similar setup. The 2 main differences: all my users are virtual and the "2nd server" is on a different continent (high latency sync). Unfortunately the dsync is not working for the moment. Timo is in the process of redesigning it. So once it is release will know about it. > > drwx------ 5 lyn lyn 4096 Apr 30 19:32 > .INBOX_7a86a62d465a974fb92f00003b258734 > > First question: why is this random named directory being created in the > origin Maildir? Shouldn't the replication be more or less read-only in > the origin Maildir? - the number it is not random, but rather it is the GUID of the folder on the other server. To get rid of this annoying problem you need to clean your source of all these newly created folders, rsync your folders in between the 2 machines, run dsync again (this time it will not mess up with your folder structure) > Second question: If I re-attempt a doveadm sync a second time I get > this error: > > tornado Maildir # doveadm sync -u lyn remote:root at dustbowl.reub.net > dsync-local(lyn): Error: Can't rename mailbox > INBOX_7a86a62d465a974fb92f00003b258734 to INBOX: Target mailbox already > exists > dsync-local(lyn): Error: Can't rename mailbox INBOX to > INBOX_eb15f30ea563be4b703200002bd68bb1: Renaming INBOX isn't supported. > tornado Maildir # > > It's not clear if the second attempt has failed or succeeded, and it's a > bit odd that it errors out on a directory that the dovecot sync process > itself has created. > do the fix at Q1 and you will not run into this... it is not a permission problem but rather a meta-info problem. The setup will run fine as long as you only update 1 server and the other one is backup. The current release does not handle well the master-master model (you'll endup with emails like the folders above: duplicated, with GUID appended to them etc etc)... Wish Timo good luck and inspiration! Best regards, Andrei From robert at schetterer.org Mon Apr 30 21:55:11 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 30 Apr 2012 20:55:11 +0200 Subject: [Dovecot] question dovecot Inheritance global acl vs userfolder acl Message-ID: <4F9EE00F.5050308@schetterer.org> Hi Timo my tests resulted in inheritance is given if a userfolder has set some acl to its new created subfolder , which is nice if some userfolder has its acl from global acl there is no inheritance to its new created subfolders, that subfolders will always created with full owner rights i am not really sure if its a good idea to have inheritance from global acl and if its hackabel what is your idea to this ? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From dmiller at amfes.com Mon Apr 30 22:42:00 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 30 Apr 2012 12:42:00 -0700 Subject: [Dovecot] Single Instance Storage management Message-ID: Was there a doveadm command that checks the SIS files and removes any not referenced? Will the "doveadm purge -A" catch that? -- Daniel From dmiller at amfes.com Mon Apr 30 22:43:22 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 30 Apr 2012 12:43:22 -0700 Subject: [Dovecot] mdbox packing Message-ID: Is there a way to manually force an mdbox storage to be rebuilt into new files? Particularly files of the maximum size? -- Daniel From dmiller at amfes.com Mon Apr 30 23:16:01 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 30 Apr 2012 13:16:01 -0700 Subject: [Dovecot] Dovecot/doveadm crash Message-ID: Having a problem with a mailbox. I've been trying to rebuild - but doveadm force-resync crashes. This is mdbox with sis. doveadm purge -u dmiller at amfes.com doveadm(dmiller at amfes.com): Panic: file istream.c: line 466 (i_stream_grow_buffer): assertion failed: (stream->max_buffer_size > 0) doveadm(dmiller at amfes.com): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x4167a) [0x7fb73465467a] -> /usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7fb734654762] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fb73462c2a0] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4949b) [0x7fb73465c49b] -> /usr/local/lib/dovecot/libdovecot.so.0(i_stream_get_buffer_space+0x82) [0x7fb73465c522] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4b1c8) [0x7fb73465e1c8] -> /usr/local/lib/dovecot/libdovecot.so.0(i_stream_read+0x69) [0x7fb73465bb69] -> /usr/local/lib/dovecot/libdovecot.so.0(i_stream_read_next_line+0x1d) [0x7fb73465c04d] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mdbox_purge+0x921) [0x7fb7348d7d71] -> doveadm() [0x40e1e6] -> doveadm() [0x40e018] -> doveadm() [0x40ea83] -> doveadm(doveadm_mail_try_run+0x141) [0x40eef1] -> doveadm(main+0x3c1) [0x415911] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xff) [0x7fb73429deff] -> doveadm() [0x40de39] -- Daniel From marcin at mejor.pl Mon Apr 30 23:48:50 2012 From: marcin at mejor.pl (=?ISO-8859-2?Q?Marcin_Miros=B3aw?=) Date: Mon, 30 Apr 2012 22:48:50 +0200 Subject: [Dovecot] mdbox packing In-Reply-To: References: Message-ID: <4F9EFAB2.8040303@mejor.pl> W dniu 2012-04-30 21:43, Daniel L. Miller pisze: > Is there a way to manually force an mdbox storage to be rebuilt into new > files? Particularly files of the maximum size? > Hello, probably doveadm purge should do it. Regards, Marcin From tss at iki.fi Sun Apr 1 00:28:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 1 Apr 2012 00:28:51 +0300 Subject: [Dovecot] Dovecot stones Message-ID: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> For the last few days I've been thinking about my company and what it really should do, and as much as the current plan seems reasonable, I think in good conscience I really can't help but to bring up an alternative plan: The support for Dovecot is quite good already, but wouldn't it be better if it was GUARANTEED that your email server was ALWAYS working with zero downtime or other problems? After some thinking, planning and consulting with my spiritual guides I realized I really should just go ahead and do it: From now on you can buy blessed Dovecot stones (http://www.dovecot.org/tmp/stone1.jpg), which are guaranteed to keep your email server running. I don't think there's really any further reason for me to provide commercial support. Probably the whole Dovecot mailing list will be pretty much useless after this as well, so I might as well stop posting there. If you have any problems, you'll just buy a Dovecot stone and the problem will go away. If for any reason that doesn't work, it means that your problem is more difficult than most and you need to buy a few more stones. Please remember that each shipment may take up to 3 weeks to reach you, so get your stones early and in sufficient quantities. A small email server will be happy enough with a single small Dovecot stone, but the more users you have the more stones you'll likely need. This is because most people are rather unhappy with their lives, and they project their negative energies towards their surroundings, including their email servers. To overcome this negative energy you need to have a counterbalancing positive energy, which the Dovecot stones will provide you. The Dovecot stones are hand painted from the finest materials available to the sister of the author of Dovecot that she could find in the local shops. Each stone contains such love and dedication towards the well being of Dovecot that you will shed tears when you lay your eyes upon their physical being. The images provided here unfortunately don't do justice to them. http://www.dovecot.org/tmp/stone2.jpg - This is how i keep my home desktop running all day every day. I have NEVER broken a single hard drive in my life. I don't even bother to keep backups, because I KNOW it is useless for me. A small Dovecot stone can be bought for 10 euros, a medium sized stone for 100 euros and a large stone for 500 euros. Each large stone can usually handle around 1000-2000 users before you need to buy more. If you have millions of users we'll need to find you a larger rock from my parents' backyard. There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago), so their prices need to be negotiated in advance. From pw at wk-serv.de Sun Apr 1 02:18:28 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Sun, 01 Apr 2012 01:18:28 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <4F7790C4.5080002@wk-serv.de> Timo Sirainen schrieb: > A small Dovecot stone can be bought for 10 euros, a medium sized stone for 100 euros and a large stone for 500 euros. Each large stone can usually handle around 1000-2000 users before you need to buy more. If you have millions of users we'll need to find you a larger rock from my parents' backyard. There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago), so their prices need to be negotiated in advance. =) From pw at wk-serv.de Sun Apr 1 02:21:45 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Sun, 01 Apr 2012 01:21:45 +0200 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F760B61.3090209@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> Message-ID: <4F779189.8000705@wk-serv.de> Patrick Westenberg schrieb: > Nick Warr schrieb: > >> I think some of the new Dovecot (director?) software is user aware, but >> I don't know if it's quite ready for production. > > Yes, with director it should be something like that: > > MTA --(lmtp)--\ /--(lmtp)--> backend1 --\ > -- director -- -- NFS > MTA --(lmtp)--/ \--(lmtp)--> backend2 --/ > > > IMAP-User --> frontend1 --\ /--(imap)--> backend1 --\ > -- director -- -- NFS > IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/ > > > So what happens if user1 at example.tld receives a mail? > > - The director decides to connect to backend1 which in turn stores > the mail on the NFS share and the index file locally? > > - Then, user1 at example.tld connects to one of the frontends. Does the > director know that, earlier, this user received a mail and proxies him > to backend1 too? Timo? This post is from March, right before you introduced the stones :D From robert at schetterer.org Sun Apr 1 09:43:13 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 01 Apr 2012 08:43:13 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <4F77F901.30605@schetterer.org> Am 31.03.2012 23:28, schrieb Timo Sirainen: > For the last few days I've been thinking about my company and what it really should do, and as much as the current plan seems reasonable, I think in good conscience I really can't help but to bring up an alternative plan: > > The support for Dovecot is quite good already, but wouldn't it be better if it was GUARANTEED that your email server was ALWAYS working with zero downtime or other problems? After some thinking, planning and consulting with my spiritual guides I realized I really should just go ahead and do it: > > From now on you can buy blessed Dovecot stones (http://www.dovecot.org/tmp/stone1.jpg), which are guaranteed to keep your email server running. I don't think there's really any further reason for me to provide commercial support. Probably the whole Dovecot mailing list will be pretty much useless after this as well, so I might as well stop posting there. If you have any problems, you'll just buy a Dovecot stone and the problem will go away. If for any reason that doesn't work, it means that your problem is more difficult than most and you need to buy a few more stones. Please remember that each shipment may take up to 3 weeks to reach you, so get your stones early and in sufficient quantities. > > A small email server will be happy enough with a single small Dovecot stone, but the more users you have the more stones you'll likely need. This is because most people are rather unhappy with their lives, and they project their negative energies towards their surroundings, including their email servers. To overcome this negative energy you need to have a counterbalancing positive energy, which the Dovecot stones will provide you. > > The Dovecot stones are hand painted from the finest materials available to the sister of the author of Dovecot that she could find in the local shops. Each stone contains such love and dedication towards the well being of Dovecot that you will shed tears when you lay your eyes upon their physical being. The images provided here unfortunately don't do justice to them. > > http://www.dovecot.org/tmp/stone2.jpg - This is how i keep my home desktop running all day every day. I have NEVER broken a single hard drive in my life. I don't even bother to keep backups, because I KNOW it is useless for me. > > A small Dovecot stone can be bought for 10 euros, a medium sized stone for 100 euros and a large stone for 500 euros. Each large stone can usually handle around 1000-2000 users before you need to buy more. If you have millions of users we'll need to find you a larger rock from my parents' backyard. There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago), so their prices need to be negotiated in advance. Hi Timo, perhaps the ghosts in the machines will now be very happy starting eat such bricks starting in the magical Ides of April so lets pray you ll never get out of Jedi power to bless the stones finally i am looking for a stone interface , meanwhile holy rays might have power enough, preparing them near the servers in the datacenter where i allready have prisoned evil trolls -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From anyaddress at gmx.net Sun Apr 1 10:28:40 2012 From: anyaddress at gmx.net (Tom Fernandes) Date: Sun, 1 Apr 2012 09:28:40 +0200 Subject: [Dovecot] namespace hierarchy Message-ID: <201204010928.41017.anyaddress@gmx.net> Hi, I'm migrating from Courier to Dovecot and would like to keep the original namespace and add two new ones on the new server. There's only one namespace in Courier which is "INBOX." . The seperator is "." In the new server I would like to have 3 namespaces (private, public, shared) and use "/" as separator. Is it possible to keep "INBOX." for private, so that the users mailboxes appear below the Inbox, but have public and shared on the same hierarchy-level like the Inbox? When I set prefix to "INBOX/" for private I get the users mailboxes below the Inbox, like I have it Courier now. But then at least the public, probably also the shared namespace doesn't work any more. I get "unknown subscription namespace" when trying to subscribe to some of the folders. My current configuration is: namespace: type: private separator: / inbox: yes list: yes subscriptions: yes namespace: type: public separator: / prefix: public/ location: maildir:/var/vmail/example.com/public list: yes namespace: type: shared separator: / prefix: shared/%%u/ location: maildir:%%h/Maildir list: yes warm regards, Tom From trashcan at odo.in-berlin.de Sun Apr 1 10:36:22 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sun, 1 Apr 2012 09:36:22 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: Nice ;-) On 31.03.2012, at 23:28, Timo Sirainen wrote: > A small Dovecot stone can be bought for 10 euros, a medium sized stone for 100 euros and a large stone for 500 euros. Each large stone can usually handle around 1000-2000 users before you need to buy more. If you have millions of users we'll need to find you a larger rock from my parents' backyard. There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago), so their prices need to be negotiated in advance. And sorry for the previous Mail. Mail.app from Apple isn't the best program ;-) Regards, Michael From marcin at mejor.pl Sun Apr 1 12:13:36 2012 From: marcin at mejor.pl (=?ISO-8859-2?Q?Marcin_Miros=B3aw?=) Date: Sun, 01 Apr 2012 11:13:36 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <4F781C40.40102@mejor.pl> W dniu 2012-03-31 23:28, Timo Sirainen pisze: > For the last few days I've been thinking about my company and what it really should do, and as much as the current plan seems reasonable, I think in good conscience I really can't help but to bring up an alternative plan: > > The support for Dovecot is quite good already, but wouldn't it be better if it was GUARANTEED that your email server was ALWAYS working with zero downtime or other problems? After some thinking, planning and consulting with my spiritual guides I realized I really should just go ahead and do it: > > From now on you can buy blessed Dovecot stones (http://www.dovecot.org/tmp/stone1.jpg), which are guaranteed to keep your email server running. I don't think there's really any further reason for me to provide commercial support. Probably the whole Dovecot mailing list will be pretty much useless after this as well, so I might as well stop posting there. If you have any problems, you'll just buy a Dovecot stone and the problem will go away. If for any reason that doesn't work, it means that your problem is more difficult than most and you need to buy a few more stones. Please remember that each shipment may take up to 3 weeks to reach you, so get your stones early and in sufficient quantities. > > A small email server will be happy enough with a single small Dovecot stone, but the more users you have the more stones you'll likely need. This is because most people are rather unhappy with their lives, and they project their negative energies towards their surroundings, including their email servers. To overcome this negative energy you need to have a counterbalancing positive energy, which the Dovecot stones will provide you. > > The Dovecot stones are hand painted from the finest materials available to the sister of the author of Dovecot that she could find in the local shops. Each stone contains such love and dedication towards the well being of Dovecot that you will shed tears when you lay your eyes upon their physical being. The images provided here unfortunately don't do justice to them. > > http://www.dovecot.org/tmp/stone2.jpg - This is how i keep my home desktop running all day every day. I have NEVER broken a single hard drive in my life. I don't even bother to keep backups, because I KNOW it is useless for me. > > A small Dovecot stone can be bought for 10 euros, a medium sized stone for 100 euros and a large stone for 500 euros. Each large stone can usually handle around 1000-2000 users before you need to buy more. If you have millions of users we'll need to find you a larger rock from my parents' backyard. There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago), so their prices need to be negotiated in advance. Hello! I'm using dovecot on vps, how can i use dovecot stone? Is it enough to use photo of stone as boot splash or should i start dovecot with LD_PRELOAD="/path_to_photo.jpg"? From patrickdk at patrickdk.com Sun Apr 1 17:22:30 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Sun, 01 Apr 2012 10:22:30 -0400 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <20120401102230.Horde.3VgjapLnE6FPeGSmb70RjgA@mail.patrickdk.com> Quoting Timo Sirainen : > For the last few days I've been thinking about my company and what > it really should do, and as much as the current plan seems > reasonable, I think in good conscience I really can't help but to > bring up an alternative plan: > > The support for Dovecot is quite good already, but wouldn't it be > better if it was GUARANTEED that your email server was ALWAYS > working with zero downtime or other problems? After some thinking, > planning and consulting with my spiritual guides I realized I really > should just go ahead and do it: > > From now on you can buy blessed Dovecot stones > (http://www.dovecot.org/tmp/stone1.jpg), which are guaranteed to > keep your email server running. I don't think there's really any > further reason for me to provide commercial support. Probably the > whole Dovecot mailing list will be pretty much useless after this as > well, so I might as well stop posting there. If you have any > problems, you'll just buy a Dovecot stone and the problem will go > away. If for any reason that doesn't work, it means that your > problem is more difficult than most and you need to buy a few more > stones. Please remember that each shipment may take up to 3 weeks > to reach you, so get your stones early and in sufficient quantities. > > A small email server will be happy enough with a single small > Dovecot stone, but the more users you have the more stones you'll > likely need. This is because most people are rather unhappy with > their lives, and they project their negative energies towards their > surroundings, including their email servers. To overcome this > negative energy you need to have a counterbalancing positive energy, > which the Dovecot stones will provide you. > > The Dovecot stones are hand painted from the finest materials > available to the sister of the author of Dovecot that she could find > in the local shops. Each stone contains such love and dedication > towards the well being of Dovecot that you will shed tears when you > lay your eyes upon their physical being. The images provided here > unfortunately don't do justice to them. > > http://www.dovecot.org/tmp/stone2.jpg - This is how i keep my home > desktop running all day every day. I have NEVER broken a single > hard drive in my life. I don't even bother to keep backups, because > I KNOW it is useless for me. > > A small Dovecot stone can be bought for 10 euros, a medium sized > stone for 100 euros and a large stone for 500 euros. Each large > stone can usually handle around 1000-2000 users before you need to > buy more. If you have millions of users we'll need to find you a > larger rock from my parents' backyard. There are only a limited > number of such rocks available (we disposed of most of them to our > neighbors' yards years ago), so their prices need to be negotiated > in advance. I'm unable to locate the order page. Or should I reply to the maillist with my creditcard info? From christian.rohmann at frittentheke.de Sun Apr 1 19:38:07 2012 From: christian.rohmann at frittentheke.de (Christian Rohmann) Date: Sun, 01 Apr 2012 18:38:07 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <4F781C40.40102@mejor.pl> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> <4F781C40.40102@mejor.pl> Message-ID: <4F78846F.5050909@frittentheke.de> Hey all, On 04/01/2012 11:13 AM, Marcin Miros?aw wrote: > I'm using dovecot on vps, how can i use dovecot stone? Is it enough to > use photo of stone as boot splash or should i start dovecot with > LD_PRELOAD="/path_to_photo.jpg"? Valid questions, what about Dovecot vStones? Apart from this practical issue, who is your marketing advisor? Timo in cloudy times like these, one needs product names that assure customers it's compatible to the Cloud: I suggest something like: Dovecot CloudProtector Stone 2000 S Dovecot CloudProtector Stone 4500 GT Dovecot CloudProtector Stone 6900 GTX turbo Also for such sheer value in less nightly calls on broken servers, you need to price the things higher. This will underline that the stones are of exceptional quality and energy. Regards Christian From robert at schetterer.org Sun Apr 1 20:51:13 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 01 Apr 2012 19:51:13 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <4F78846F.5050909@frittentheke.de> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> <4F781C40.40102@mejor.pl> <4F78846F.5050909@frittentheke.de> Message-ID: <4F789591.1000502@schetterer.org> Am 01.04.2012 18:38, schrieb Christian Rohmann: > Hey all, > > On 04/01/2012 11:13 AM, Marcin Miros?aw wrote: >> I'm using dovecot on vps, how can i use dovecot stone? Is it enough to >> use photo of stone as boot splash or should i start dovecot with >> LD_PRELOAD="/path_to_photo.jpg"? > > > Valid questions, what about Dovecot vStones? > > Apart from this practical issue, who is your marketing advisor? Timo in > cloudy times like these, one needs product names that assure customers > it's compatible to the Cloud: > > I suggest something like: > Dovecot CloudProtector Stone 2000 S > Dovecot CloudProtector Stone 4500 GT > Dovecot CloudProtector Stone 6900 GTX turbo > > > Also for such sheer value in less nightly calls on broken servers, you > need to price the things higher. This will underline that the stones are > of exceptional quality and energy. > > > > Regards > > Christian > by the way, are they kryptonite checked ? I dont wanna get in problems with Mr Kent ? http://en.wikipedia.org/wiki/Kryptonite -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From stsiol at yahoo.co.uk Mon Apr 2 00:54:40 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Sun, 1 Apr 2012 22:54:40 +0100 (BST) Subject: [Dovecot] Mail migration troubles Message-ID: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> Hello all, OK, I would really appreciate your help on this. I managed to do a pretty straight-forward and flawless migration on dovecot from one domain to another. However, I have this nagging problem with most of the thunderbird MTAs. When I try to delete some "test" messages I sent to make sure that everything works, it comes back with the following message on the bottom of the Thunderbird window : "The current operation on InBox did not succeed.? The mail server for account "Account at bloomingdomain.gr"? responded [TRYCREATE] mailbox doesn't exist : Trash" Now, I know it has to do something with automatically creating a? trash folder. But don't know anything more. Could someone shed some light on this ? Dovecot v1.2.15 AFAIK CentOS 5.5 x86 (32-bit) other stuff On Clients : Thunderbird v11.0.1 (XP Pro / 32-bit) s. ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis? From stsiol at yahoo.co.uk Mon Apr 2 00:59:25 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Sun, 1 Apr 2012 22:59:25 +0100 (BST) Subject: [Dovecot] strike my last Message-ID: <1333317565.13835.YahooMailNeo@web132204.mail.ird.yahoo.com> strike my last, looks like its a thunderbird issue Apologies... s. ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From jeetuindian at gmail.com Mon Apr 2 10:31:20 2012 From: jeetuindian at gmail.com (Jitendra Bhaskar) Date: Mon, 2 Apr 2012 13:01:20 +0530 Subject: [Dovecot] Dsync automation Message-ID: Hi, I am using dovecot-2.1.3 on centos 5.7. Its running fine. I have two systems for mail server one is for backup. Currently I am running rsync daily. But I want to use dsync. I went through all the http://wiki2.dovecot.org/Tools/Dsync and http://blog.dovecot.org/2012/02/dovecot-clustering-with-dsync-based.htmlbut I couldn't understood how to automatize dsync. Can anybody give me any idea ho to do it ? -- * Thanks & Regards * *Jitendra Kumar Bhaskar* Cell:- +91 7306311531 +91 8102997821 From stsiol at yahoo.co.uk Mon Apr 2 13:19:47 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 2 Apr 2012 11:19:47 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> Message-ID: <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> I have a suspicion that, as usual, no one is going to? answer my message. Because of the fact that this is a call for help and no-one ever bothers, I 'll just have to re-phrase my question : Here's what I found on the dovecot wiki : imapsync Here's an example of how to run imapsync for a single user: imapsync --syncinternaldates \??--host1 192.168.1.57 --authmech1 LOGIN --user1 leah at example.com --password1 secret \??--host2 127.0.0.1????--authmech2 LOGIN --user2 leah at example.com --password2 secret It is quite easy to script this for a number of users, assuming you have their passwords. Even if you do not, imapsync also supports logging in as an admin user that has the ability to copy message for sub users, and a variety of other authentication options. Is the above what I need if I migrate mailboxes FROM dovecot TO dovecot ? E.g. in my example, I only copy mailboxes from an old "user at domainA.gr" to "user at domainB.gr". Is the above all I need ? Is there a native dovecot tool for this or should I just use imapsync ? Thank you again, s. ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis ----- Original Message ----- > From: Spyros Tsiolis > To: Spyros Tsiolis > Cc: > Sent: Monday, 2 April 2012, 11:24 > Subject: Re: [Dovecot] Mail migration troubles > > Hello again, > > Well, it seems I have trouble with the migration after all. > After the migration and eventhough all folders and messages > are on their respective places, any new mail users get > inside their mailbox cannot be dragged-and-dropped to > their respective folders. > > So, if a user has a folder with my name on it and I send > him an e-mail, the user will try to drag-and-drop that > message to the folder with my name, but the message won't > move. > > Any gurus out there on this ? > Any help would be greatly appreciated. > > spyros > > > ? > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis > ? From CMarcus at Media-Brokers.com Mon Apr 2 13:30:06 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 02 Apr 2012 06:30:06 -0400 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> Message-ID: <4F797FAE.7030104@Media-Brokers.com> On 2012-04-02 6:19 AM, Spyros Tsiolis wrote: > Is the above what I need if I migrate mailboxes FROM dovecot TO dovecot ? > E.g. in my example, I only copy mailboxes from an old "user at domainA.gr" to > "user at domainB.gr". > > Is the above all I need ? > > Is there a native dovecot tool for this or should I just use imapsync ? Are these on ths same box? If so, why not just mv them?? -- Best regards, Charles From CMarcus at Media-Brokers.com Mon Apr 2 13:31:56 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 02 Apr 2012 06:31:56 -0400 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <4F79801C.8020804@Media-Brokers.com> On 2012-03-31 5:28 PM, Timo Sirainen wrote: > For the last few days I've been thinking about my company and what it > really should do, and as much as the current plan seems reasonable, I > think in good conscience I really can't help but to bring up an > alternative plan: > > The support for Dovecot is quite good already, but wouldn't it be > better if it was GUARANTEED that your email server was ALWAYS working > with zero downtime or other problems? After some thinking, planning > and consulting with my spiritual guides I realized I really should > just go ahead and do it: > > From now on you can buy blessed Dovecot stones > (http://www.dovecot.org/tmp/stone1.jpg), which are guaranteed to keep > your email server running. I don't think there's really any further > reason for me to provide commercial support. Alright Timo, where do I send the bill for my new keyboard? Mine is no ruined from the coffee I just spilled all over it... Rotflmao! Thanks, I needed that... -- Best regards, Charles From arne at drlinux.no Mon Apr 2 13:36:38 2012 From: arne at drlinux.no (Arne K. Haaje) Date: Mon, 02 Apr 2012 12:36:38 +0200 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> Message-ID: <5628413.EKgo53ll8l@trillian> Mandag 2. april 2012 11.19.47 skrev Spyros Tsiolis: [snip] > Is the above what I need if I migrate mailboxes FROM dovecot TO dovecot ? > E.g. in my example, I only copy mailboxes from an old "user at domainA.gr" to > "user at domainB.gr". > > Is the above all I need ? That worked for me when creating a backup copy of a mailbox, but with a different login. I copied mailbox A to mailbox B, then created an entry in the auth table that pointed to mailbox B. This was with maildir. I don't know if it works the same with all other types. Regards, Arne -- Arne K. Haaje http://www.drlinux.no/ LinkedIn: http://no.linkedin.com/pub/arne-haaje/27/189/bb From stsiol at yahoo.co.uk Mon Apr 2 13:51:53 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 2 Apr 2012 11:51:53 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <4F797FAE.7030104@Media-Brokers.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> Message-ID: <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> > On 2012-04-02 6:19 AM, Spyros Tsiolis wrote: >> Is the above what I need if I migrate mailboxes FROM dovecot TO dovecot ? >> E.g. in my example, I only copy mailboxes from an old? > "user at domainA.gr" to >> "user at domainB.gr". >>? >> Is the above all I need ? >>? >> Is there a native dovecot tool for this or should I just use imapsync ? >? > Are these on ths same box? If so, why not just mv them?? Hi Charles, Yes, on the same box, on the same drive, on the same partition/filesystem. I have a box with dovecot/XMail (pop3/smtp) and two domains; DomainA and DomainB The company bosses decided to move about 20 mailboxes from DomainA to DomainB. As far as I can tell everything went well except for one user (so far) that he tries to move fresh mail to (mv'd) folders and it doesn't get there. Also when he tries to delete messages, they don't get deleted. I did "cp -r" actually and kept the old mailboxes nearby just in case. Thanks, s. ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From CMarcus at Media-Brokers.com Mon Apr 2 13:55:12 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 02 Apr 2012 06:55:12 -0400 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <4F798590.6060701@Media-Brokers.com> On 2012-04-02 6:51 AM, Spyros Tsiolis wrote: > As far as I can tell everything went well except for one > user (so far) that he tries to move fresh mail to (mv'd) > folders and it doesn't get there. > Also when he tries to delete messages, they don't get > deleted. > I did "cp -r" actually and kept the old mailboxes nearby > just in case. What do the logs show when he tries this? Did you compare FS permissions on a working setup and the non-working one? -- Best regards, Charles From nmilas at noa.gr Mon Apr 2 14:44:48 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Mon, 02 Apr 2012 14:44:48 +0300 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <4F799130.2020806@noa.gr> On 2/4/2012 1:51 ??, Spyros Tsiolis wrote: > I did "cp -r" actually and kept the old mailboxes nearby > just in case. If on CentOS/RHEL (don't know for other systems), you could try using explicitly: /bin/cp -pr for better results, because cp is aliased. You might already be aware of this, but just in case... Nick From stsiol at yahoo.co.uk Mon Apr 2 15:05:23 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 2 Apr 2012 13:05:23 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <4F799130.2020806@noa.gr> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> <4F799130.2020806@noa.gr> Message-ID: <1333368323.2437.YahooMailNeo@web132204.mail.ird.yahoo.com> > On 2/4/2012 1:51 ??, Spyros Tsiolis wrote: >? >> I did "cp -r" actually and kept the old mailboxes nearby >> just in case. >? > If on CentOS/RHEL (don't know for other systems), you could try using? > explicitly: >? > ? /bin/cp -pr >? > for better results, because cp is aliased. >? > You might already be aware of this, but just in case... >? > Nick > Hi Nick, No I didn't know this. I mean I had it stored in the back of my head but? to tell you the truth I would never use it. Yes, thank you kindly for pointing that. I'll use that switch from now on. And , yes, I am using CentOS. Can't live without it, no matter what the people say :-) Thank you again, s. ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From stsiol at yahoo.co.uk Mon Apr 2 15:49:28 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 2 Apr 2012 13:49:28 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <4F798590.6060701@Media-Brokers.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> <4F798590.6060701@Media-Brokers.com> Message-ID: <1333370968.69235.YahooMailNeo@web132201.mail.ird.yahoo.com> ----- Original Message ----- > From: Charles Marcus > To: dovecot at dovecot.org > Cc: > Sent: Monday, 2 April 2012, 12:55 > Subject: Re: [Dovecot] Mail migration troubles > > On 2012-04-02 6:51 AM, Spyros Tsiolis wrote: >> As far as I can tell everything went well except for one >> user (so far) that he tries to move fresh mail to (mv'd) >> folders and it doesn't get there. >> Also when he tries to delete messages, they don't get >> deleted. >> I did "cp -r" actually and kept the old mailboxes nearby >> just in case. > > What do the logs show when he tries this? > > Did you compare FS permissions on a working setup and the non-working one? > > -- > Best regards, > > Charles > Hi Charles, Well, I checked the dovecot log files? (dovecot-info.log / dovecot.log)? It doesn't show something suspicious. Just the usual login logout and password chatter. At this point, I suspect it's thunderbird again. I asked the rest of the people how they are doing and they said they were? doing fine. So I suspect there's something wrong with this specific dovecot installation. Thank you though, s. ? ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From jdonovan at beth.k12.pa.us Mon Apr 2 16:01:04 2012 From: jdonovan at beth.k12.pa.us (jeff donovan) Date: Mon, 2 Apr 2012 09:01:04 -0400 Subject: [Dovecot] setting up ldap for sasl Message-ID: <98F8F008-100B-4CAF-BE0E-F2BBAC50B2D7@beth.k12.pa.us> Greetings im new to the list dovecot --version 2.0.13 ubuntu 11.10 i want to use dovecot to authenticate my ldap users. My local users can authenticate and send email. Using postfix sasl/tls dovecot pam. I have read many docs/tutorials, which all have a plethora of options and information. my file structure is different than most of the documentation. I have ../etc/dovecot.conf which points to. !include_try /usr/share/dovecot/protocols.d/*.protocol !include conf.d/*.conf !include_try local.conf which leads me to /etc/dovecot/conf.d, which is full of config files. All were traditionally located in dovecot.conf in previous versions. im using PAM to authenticate my local system users. I have read that could adjust PAM, to use LDAP. How do I configure dovecot to keep using pam for local users and ldap for remote ? Do i just add the passdb lines dovecot.conf: passdb ldap { args = /etc/dovecot-ldap.conf } userdb prefetch { } # The userdb below is used only by deliver. userdb ldap { args = /etc/dovecot-ldap.conf } dovecot-ldap.conf: pass_attrs = uid=user,userPassword=password, homeDirectory=userdb_home,uidNumber=userdb_uid,gidNumber=userdb_gid # For deliver: user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid any assistance would be helpful. -j -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2497 bytes Desc: not available URL: From CMarcus at Media-Brokers.com Mon Apr 2 16:20:49 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 02 Apr 2012 09:20:49 -0400 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333370968.69235.YahooMailNeo@web132201.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> <4F798590.6060701@Media-Brokers.com> <1333370968.69235.YahooMailNeo@web132201.mail.ird.yahoo.com> Message-ID: <4F79A7B1.4030908@Media-Brokers.com> On 2012-04-02 8:49 AM, Spyros Tsiolis wrote: > On2 April 2012, 12:55, Charles Marcus wrote: >> On 2012-04-02 6:51 AM, Spyros Tsiolis wrote: >>> As far as I can tell everything went well except for one >>> user (so far) that he tries to move fresh mail to (mv'd) >>> folders and it doesn't get there. >>> Also when he tries to delete messages, they don't get >>> deleted. >>> I did "cp -r" actually and kept the old mailboxes nearby >>> just in case. >> What do the logs show when he tries this? >> >> Did you compare FS permissions on a working setup and the >> non-working one? > Well, I checked the dovecot log files > (dovecot-info.log / dovecot.log) > It doesn't show something suspicious. > Just the usual login logout and > password chatter. Well, maybe one of the reasons people don't always try to help you is you often just don't respond with answers to questions... WHAT ARE THE FS PERMISSIONS ON THAT USERS MAIL FOLDERS? And don't just say 'they are ok' - SHOW them... And how about SHOWING SOME LOG EXCERPTS (and increase verbosity if needed), instead of just saying you don't see anything relevant. -- Best regards, Charles From jg at softjury.de Mon Apr 2 17:06:28 2012 From: jg at softjury.de (Jan Phillip Greimann) Date: Mon, 02 Apr 2012 16:06:28 +0200 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <4F79B264.9030007@softjury.de> STOP TALKING AND TAKE MY MONEY!!!!1111elf Am 31.03.2012 23:28, schrieb Timo Sirainen: > For the last few days I've been thinking about my company and what it really should do, and as much as the current plan seems reasonable, I think in good conscience I really can't help but to bring up an alternative plan: > > The support for Dovecot is quite good already, but wouldn't it be better if it was GUARANTEED that your email server was ALWAYS working with zero downtime or other problems? After some thinking, planning and consulting with my spiritual guides I realized I really should just go ahead and do it: > > From now on you can buy blessed Dovecot stones (http://www.dovecot.org/tmp/stone1.jpg), which are guaranteed to keep your email server running. I don't think there's really any further reason for me to provide commercial support. Probably the whole Dovecot mailing list will be pretty much useless after this as well, so I might as well stop posting there. If you have any problems, you'll just buy a Dovecot stone and the problem will go away. If for any reason that doesn't work, it means that your problem is more difficult than most and you need to buy a few more stones. Please remember that each shipment may take up to 3 weeks to reach you, so get your stones early and in sufficient quantities. > > A small email server will be happy enough with a single small Dovecot stone, but the more users you have the more stones you'll likely need. This is because most people are rather unhappy with their lives, and they project their negative energies towards their surroundings, including their email servers. To overcome this negative energy you need to have a counterbalancing positive energy, which the Dovecot stones will provide you. > > The Dovecot stones are hand painted from the finest materials available to the sister of the author of Dovecot that she could find in the local shops. Each stone contains such love and dedication towards the well being of Dovecot that you will shed tears when you lay your eyes upon their physical being. The images provided here unfortunately don't do justice to them. > > http://www.dovecot.org/tmp/stone2.jpg - This is how i keep my home desktop running all day every day. I have NEVER broken a single hard drive in my life. I don't even bother to keep backups, because I KNOW it is useless for me. > > A small Dovecot stone can be bought for 10 euros, a medium sized stone for 100 euros and a large stone for 500 euros. Each large stone can usually handle around 1000-2000 users before you need to buy more. If you have millions of users we'll need to find you a larger rock from my parents' backyard. There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago), so their prices need to be negotiated in advance. From wgrcunha at gmail.com Mon Apr 2 17:14:26 2012 From: wgrcunha at gmail.com (Francisco Wagner C. Freire) Date: Mon, 2 Apr 2012 11:14:26 -0300 Subject: [Dovecot] Backtrace on mdbox index Message-ID: Hello, I current using dovecot with mdbox and on one account i get this error: doveadm(user at domain): Panic: file mail-index-sync-ext.c: line 209 (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) doveadm(user at domain): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x3fd8a) [0x7fae684fdd8a] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7fae684fde72] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7fae684d719f] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9bdc3) [0x7fae68c1edc3] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_ext_intro+0x1e6) [0x7fae68c1fa76] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_record+0x49f) [0x7fae68c21a3f] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_map+0x234) [0x7fae68c225a4] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_map+0x86) [0x7fae68c13cd6] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x8b609) [0x7fae68c0e609] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_open+0x210) [0x7fae68c0eaa0] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x591c6) [0x7fae68bdc1c6] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_map_get_zero_ref_files+0x2d) [0x7fae68bdcd6d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_purge+0xbc) [0x7fae68bde7ac] -> doveadm() [0x40a135] -> doveadm() [0x40a813] -> doveadm(doveadm_mail_single_user+0x61) [0x40a8d1] -> doveadm() [0x40aafd] -> doveadm(doveadm_mail_try_run+0x141) [0x40af11] -> doveadm(main+0x381) [0x4109c1] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7fae67d6ec8d] -> doveadm() [0x409ff9] Aborted All doveadm commands fail on thie error, index / force-resync / search / fetch etc. Any help? From stsiol at yahoo.co.uk Mon Apr 2 17:29:39 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 2 Apr 2012 15:29:39 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <4F79A7B1.4030908@Media-Brokers.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333358682.80674.YahooMailNeo@web132201.mail.ird.yahoo.com> <1333361987.53819.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F797FAE.7030104@Media-Brokers.com> <1333363913.36523.YahooMailNeo@web132203.mail.ird.yahoo.com> <4F798590.6060701@Media-Brokers.com> <1333370968.69235.YahooMailNeo@web132201.mail.ird.yahoo.com> <4F79A7B1.4030908@Media-Brokers.com> Message-ID: <1333376979.98299.YahooMailNeo@web132205.mail.ird.yahoo.com> Hi again Charles, >>> Did you compare FS permissions on a working setup and the >>> non-working one? I don't have a non-working setup of dovecot >> Well, I checked the dovecot log files >> (dovecot-info.log / dovecot.log) >> It doesn't show something suspicious. >> Just the usual login logout and >> password chatter. >? > Well, maybe one of the reasons people don't always try to help you is you? > often just don't respond with answers to questions... Oh, I see. It's like what I ask questions about issues and? get answers on other issues... ? > WHAT ARE THE FS PERMISSIONS ON THAT USERS MAIL FOLDERS? And don't just say? > 'they are ok' - SHOW them... All users run with the virtual mail account (vmail) accounts (if I recall correctly) are stored in %maildir format. Do you want me to display user perms ? OK ! Fine. Here's the contents of the folder with the problem : [root at mailserver mailaccount]# ls -l total 8 drwx------ 6 vmail vmail 4096 Apr ?2 17:08 Maildir -rw-rw-rw- 1 vmail vmail ?141 Mar 30 17:21 user.tab Is this what you need ?? I didn't answer this not because I don't want to but because I don't know if this is what you want. There's no need to shout. ? > And how about SHOWING SOME LOG EXCERPTS (and increase verbosity if needed),? > instead of just saying you don't see anything relevant. Here goes : Apr 02 14:26:02 auth(default): Info: client out: OK1user=account at domain.gr Apr 02 14:26:02 auth(default): Info: master in: REQUEST1785294431 Apr 02 14:26:02 auth(default): Info: master out: USER1785account at domain.gruid=501gid=502home=/ho me/vmail/account at domain.gr Apr 02 14:26:02 imap-login: Info: Login: user=, method=PLAIN, rip=192.168.0.20, lip=192.168.1. 4, TLS Apr 02 14:26:02 auth(default): Info: new auth connection: pid=29450 Apr 02 14:27:02 auth(default): Info: client in: AUTH1PLAINservice=imapsecuredlip=192.168.1.4rip=192. 168.0.20lport=993rport=4494 Apr 02 14:27:02 auth(default): Info: client out: CONT1 Apr 02 14:27:02 auth(default): Info: client in: CONT1AHRpY2ldA29ZnzWzc9c5cgxTQA== Apr 02 14:27:02 auth(default): Info: passwd-file(account at domain.gr,192.168.0.20): lookup: user=account at domain.gr ?file=/etc/dovecot/passwd Apr 02 14:27:02 auth(default): Info: client out: OK1user=account at domain.gr Apr 02 14:27:02 auth(default): Info: master in: REQUEST1786294501 Apr 02 14:27:02 auth(default): Info: master out: USER1786account at domain.gruid=501gid=502home=/ho me/vmail/account at domain.gr Apr 02 14:27:02 imap-login: Info: Login: user=, method=PLAIN, rip=192.168.0.20, lip=192.168.1. 4, TLS Apr 02 14:27:02 auth(default): Info: new auth connection: pid=29453 s. ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From giom87 at libero.it Mon Apr 2 17:39:08 2012 From: giom87 at libero.it (Giovanni Mancuso) Date: Mon, 02 Apr 2012 16:39:08 +0200 Subject: [Dovecot] Connection queue full on dovecot 2.0.13 Message-ID: <4F79BA0C.1070104@libero.it> Hi, i have a problem with dovecot 2.0.13. I have one dovecot in Front-end that has only: passdb { driver = static args = proxy=y host=10.0.0.6 nopassword=y } In Back-end i have one dovecot that does auth and exports imap/pop3 ports. In dovecot's log of Front-end i see: /Apr 02 14:33:34 imap-login: Info: proxy(//user at example.com/ /): started proxying to 10.0.0.6:143: user=/>, method=PLAIN, rip=//xx.xx.xx.xx//, lip=xx.xx.xx.xx, TLS Apr 02 14:34:36 imap-login: Info: Disconnected: *Connection queue full *(auth failed, 1 attempts): user=/>, method=PLAIN, rip=//xx.xx.xx.xx//, lip=//xx.xx.xx.xx//, TLS/ I see this wiki page: http://wiki2.dovecot.org/LoginProcess but i read: It works by using a number of long running login processes, each handling a number of connections. This loses much of the security benefits of the login process design, because in case of a security hole (in Dovecot or SSL library) the attacker is now able to see other users logging in and steal their passwords, read their mails, etc. Is there another way? Thanks From zybi at talex.pl Mon Apr 2 18:45:25 2012 From: zybi at talex.pl (=?ISO-8859-2?Q?Artur_Zaprza=B3a?=) Date: Mon, 02 Apr 2012 17:45:25 +0200 Subject: [Dovecot] lmtp sometimes fails to deliver a message to all recipients Message-ID: <4F79C995.9060607@talex.pl> Dovecot version: 2.1.3 lmtp config: protocol lmtp { mail_plugins = zlib acl sieve } Running strace on lmtp reveals that when lmtp delivers a message to multiple recipients it saves the message to a file in the first mailbox and hard links that file to other mailboxes. But when in the meantime of delivery that file is deleted, lmtp fails to deliver the message to remaining mailboxes. For example a message for 46 recipients is delivered by lmtp to the first 12 only (fortunately postfix retries delivery for remaining recipients) and for the rest lmtp logs the following error messages: lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Message was expunged (guid) lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: script /vmail/domain/foo/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /vmail/domain/foo/.dovecot.sieve.log may reveal additional details) I'm seeing this error for about 10 messages per hour. Best regards, Artur Zaprza?a From adorca at uda.ad Mon Apr 2 19:11:06 2012 From: adorca at uda.ad (Aleix Dorca) Date: Mon, 2 Apr 2012 18:11:06 +0200 Subject: [Dovecot] imapsync, namespaces and Dovecot 2.x Message-ID: Hi people, I recently tried to migrate from Exchange using imapsync. It didn't work because it complained that dovecot had no namespace capablity. After looking through Dovecot's config files I found that namespaces could be defined, but if you did not a 'private' default one was created. In the 2.x wiki documentation (http://wiki2.dovecot.org/Migration) it says nothing about Namespaces. I don't know if I have to create one or not. If I add the parameters --sep2 . --prefix2 '' to imapsync then it works fine. I'm proxying using dovecot as frontend and backend, version 2.0.18 (The one that comes with Debian Wheezy). Any thoughts, please? Thanks, Aleix. From moseleymark at gmail.com Mon Apr 2 19:33:32 2012 From: moseleymark at gmail.com (Mark Moseley) Date: Mon, 2 Apr 2012 09:33:32 -0700 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: On Sat, Mar 31, 2012 at 2:28 PM, Timo Sirainen wrote: > For the last few days I've been thinking about my company and what it really should do, and as much as the current plan seems reasonable, I think in good conscience I really can't help but to bring up an alternative plan: > >... I'm slightly concerned that there's been no mention of what license these stones are going to be released under. GPL2? GPL3? Apache? I'm just hoping these aren't some sort of "open core" stones that will only work for basic features but that I'll end up needing to buy "Enterprise-grade stones" to cover large clusters. From kurt-dovecot at se.rit.edu Tue Apr 3 00:23:20 2012 From: kurt-dovecot at se.rit.edu (Kurt Mosiejczuk) Date: Mon, 02 Apr 2012 17:23:20 -0400 Subject: [Dovecot] imapdir still a configuration option? Message-ID: <4F7A18C8.3000904@se.rit.edu> I've been looking at moving an old IMAPdir-based solution to dovecot. I came across references that dovecot (to my pleasant surprise) added imapdir support in 1.1. However, in trying 2.0.13 (from an OpenBSD package) things start okay, but I see in the logs: "Initialization failed: Initializing mail storage from mail_location setting failed: Unknown mail storage driver imapdir" Poking around in the sources, I still see lots of references to imapdir, but see very little in the documentation. Is IMAPdir still supported in 2.0.x (or even 2.1.x)? I'd prefer not to have to migrate users via renaming, especially since they like the IMAPdir layout for their own searching. --Kurt From micah at riseup.net Tue Apr 3 01:26:43 2012 From: micah at riseup.net (Micah Anderson) Date: Mon, 02 Apr 2012 18:26:43 -0400 Subject: [Dovecot] Dsync automation References: Message-ID: <87k41xrcrg.fsf@algae.riseup.net> Jitendra Bhaskar writes: > I am using dovecot-2.1.3 on centos 5.7. Its running fine. I have two > systems for mail server one is for backup. Currently I am running rsync > daily. But I want to use dsync. I went through all the > http://wiki2.dovecot.org/Tools/Dsync and > http://blog.dovecot.org/2012/02/dovecot-clustering-with-dsync-based.htmlbut > I couldn't understood how to automatize dsync. Can anybody give me any > idea ho to do it ? As far as I can tell, doing a for loop around 'doveadm user \*' is the only way. That will do a dsync of all the users. On the backup server you will need to decide what to do with the data. I've been rotating it on a daily basis to try and get several daily backups, then some weekly, and monthly. I've been working on a backupninja[0] handler to do dsync backups, but its still rough. micah 0. https://labs.riseup.net/code/projects/backupninja -- From micah at riseup.net Tue Apr 3 01:34:07 2012 From: micah at riseup.net (Micah Anderson) Date: Mon, 02 Apr 2012 18:34:07 -0400 Subject: [Dovecot] 2.1.3: doveadm mailbox delete lost the -s option Message-ID: <874nt1rcf4.fsf@algae.riseup.net> Looks like the recently added -s (for unsubscribing) option to doveadm mailbox delete went missing: root at vireo# /usr/bin/doveadm mailbox delete -u micahtest -s restored delete: invalid option -- 's' doveadm mailbox delete [-u |-A] [-S ] [-s] [...] although the option still is shown as valid :) micah -- From zarf at klacto.net Tue Apr 3 02:02:17 2012 From: zarf at klacto.net (FZiegler) Date: Mon, 02 Apr 2012 19:02:17 -0400 Subject: [Dovecot] dovecot and unison Message-ID: <4F7A2FF9.7000204@klacto.net> I am successfully using dovecot purely as a personal local mail store on my desktop. (There is only one account, and it's only ever accessed by local mail clients on the machine. The point is to have a common store I can use with any client; plus, I prefer dovecot's Mailbox storage to Thunderbird's mboxes.) Now I'd like if possible, to replicate this setup on my laptop and keep both in sync with unison (http://www.cis.upenn.edu/~bcpierce/unison/), which I am already using to sync much of my home dir about once a day. I found at least one positive message regarding this topic (http://dovecot.org/list/dovecot/2010-April/048092.html), but I feel I could use some more advice. Namely, which of the ancillary files should I be syncing (or perhaps deleting before sync)? In addition to the mails themselves I'm seeing: -rw------- 1 fz 501 13 Apr 1 17:10 dovecot-keywords -rw------- 1 fz 501 2483 Apr 1 17:10 dovecot-uidlist -rw------- 1 fz 501 1040 Mar 29 14:50 dovecot.index -rw------- 1 fz 501 44032 Apr 1 17:10 dovecot.index.cache -rw------- 1 fz 501 9772 Apr 1 17:10 dovecot.index.log and at the top level (in .local/var/Maildir/): -rw------- 1 fz 501 8 Apr 1 17:10 dovecot-uidvalidity -r--r--r-- 1 fz 501 0 Apr 1 17:10 dovecot-uidvalidity.4efb323d -rw------- 1 fz 501 3288 Apr 1 17:10 dovecot.mailbox.log -rw------- 1 fz 501 4104 Mar 28 10:53 dovecot.mailbox.log.2 -rw------- 1 fz 501 5114 Apr 1 17:10 subscriptions In my naive trials I ended up at least once in a situation where Thunderbird would not see some subfolders (even though they were there). That's since fixed but I guess my question is, what's the safe recommended way to not have this happen? Thanks, FZiegler From stan at hardwarefreak.com Tue Apr 3 02:04:54 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 02 Apr 2012 18:04:54 -0500 Subject: [Dovecot] Dovecot stones In-Reply-To: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <4F7A3096.5050209@hardwarefreak.com> On 3/31/2012 4:28 PM, Timo Sirainen wrote: There are only a limited number of such rocks available (we disposed of most of them to our neighbors' yards years ago) :) -- Stan From micah at riseup.net Tue Apr 3 02:15:28 2012 From: micah at riseup.net (Micah Anderson) Date: Mon, 02 Apr 2012 19:15:28 -0400 Subject: [Dovecot] dsync redesign References: <936E95DA-E032-4F8F-A323-DEBB7AAC0E1B@iki.fi> <87obrixcyp.fsf@algae.riseup.net> <4F721650.4030901@Media-Brokers.com> Message-ID: <87zkatpvxr.fsf@algae.riseup.net> Charles Marcus writes: > On 2012-03-27 11:47 AM, Micah Anderson wrote: >> One would be the ability to perform *intelligent* incremental / >> rotated backups. I can do this now by running a dsync backup >> operation and then doing manual hardlinking or moving of the backup >> directories (daily.1, daily.2, weekly.1, monthly.1, etc.), but it >> would be more intelligent if this were baked into the backup process. > > There are already numerous tools that do this flawlessly - I've been using > rsnapshot (which uses rsync) for this for years. Are you snapshotting your filesystem (using LVM, or SAN, or similar) before doing rsnapshot? Because if you aren't then rsync will not assuredly get everything in a consistent state. > I don't know if Timo should be spending his time reinventing the wheel. dsync backup is already here, and it is quite useful. > I'm much more interested in dsync working flawlessly to keep one or more > secondary servers in sync, and leave backups to backup software. I'm not against that idea, I just have not yet found a good way to use any backup software in such a way to handle large numbers of user's mail. > Although, one interesting piece that I am hopeful I'll be able to implement soon > (with Timo's professional help) is the ability to easily and automatically map > my rsnapshot snapshots directory to a read-only 'Backups' namespace that > automatically shows the snapshots by date and time as they are produced. This > way users could 'go back in time' anytime they wanted without having to call > me... :) Interesting idea, would be a great one to share with the community if you decide to do so. micah From jdonovan at beth.k12.pa.us Tue Apr 3 03:56:42 2012 From: jdonovan at beth.k12.pa.us (jeff donovan) Date: Mon, 2 Apr 2012 20:56:42 -0400 Subject: [Dovecot] setting up ldap for sasl In-Reply-To: <98F8F008-100B-4CAF-BE0E-F2BBAC50B2D7@beth.k12.pa.us> References: <98F8F008-100B-4CAF-BE0E-F2BBAC50B2D7@beth.k12.pa.us> Message-ID: <7EEE40B1-2E79-4814-B3E0-F4AE0FB45831@beth.k12.pa.us> On Apr 2, 2012, at 9:01 AM, jeff donovan wrote: > Greetings > im new to the list > dovecot --version > 2.0.13 > ubuntu 11.10 > i want to use dovecot to authenticate my ldap users. My local users can authenticate and send email. Using postfix sasl/tls dovecot pam. > > I have read many docs/tutorials, which all have a plethora of options and information. my file structure is different than most of the documentation. > I have ../etc/dovecot.conf which points to. > > !include_try /usr/share/dovecot/protocols.d/*.protocol > !include conf.d/*.conf > !include_try local.conf i included /etc/dovecot/conf.d/dovecot-ldap.conf and I get errors from dovecot config. doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/dovecot-ldap.conf line 17: Unknown setting: hosts doveconf: Error: managesieve-login: dump-capability process returned 89 i remove it, then it goes to the next option i have set; doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/dovecot-ldap.conf line 21: Unknown setting: uris doveconf: Error: managesieve-login: dump-capability process returned 89 any help would be great -j -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2497 bytes Desc: not available URL: From jdonovan at beth.k12.pa.us Tue Apr 3 05:28:13 2012 From: jdonovan at beth.k12.pa.us (jeff donovan) Date: Mon, 2 Apr 2012 22:28:13 -0400 Subject: [Dovecot] setting up ldap for sasl::solved:: In-Reply-To: <7EEE40B1-2E79-4814-B3E0-F4AE0FB45831@beth.k12.pa.us> References: <98F8F008-100B-4CAF-BE0E-F2BBAC50B2D7@beth.k12.pa.us> <7EEE40B1-2E79-4814-B3E0-F4AE0FB45831@beth.k12.pa.us> Message-ID: <18E8FC85-525D-4B46-8246-7B9A6FDC113F@beth.k12.pa.us> I had to compile with ldap-- duh. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2497 bytes Desc: not available URL: From tss at iki.fi Tue Apr 3 09:32:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 09:32:37 +0300 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> Message-ID: <1333434757.21461.32.camel@innu> On Sun, 2012-04-01 at 22:54 +0100, Spyros Tsiolis wrote: > However, I have this nagging problem with most of the thunderbird > MTAs. When I try to delete some "test" messages I sent to make sure > that everything works, it comes back with the following message on > the bottom of the Thunderbird window : > > "The current operation on InBox did not succeed. > The mail server for account "Account at bloomingdomain.gr" > responded [TRYCREATE] mailbox doesn't exist : Trash" That error probably happens when deleting a message. > Dovecot v1.2.15 AFAIK dovecot -n output would have helped. Mainly: do you have a namespace prefix or not? That's about the only thing I can think of related to that error message (e.g. dovecot has non-empty namespace prefix, while TB has empty namespace prefix). From tss at iki.fi Tue Apr 3 09:33:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 09:33:29 +0300 Subject: [Dovecot] Backtrace on mdbox index In-Reply-To: References: Message-ID: <1333434809.21461.33.camel@innu> On Mon, 2012-04-02 at 11:14 -0300, Francisco Wagner C. Freire wrote: > I current using dovecot with mdbox and on one account i get this error: > > doveadm(user at domain): Panic: file mail-index-sync-ext.c: line 209 > (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) Dovecot version? I think I've fixed this in recent v2.1.x. From tss at iki.fi Tue Apr 3 09:44:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 09:44:10 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F760B61.3090209@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> Message-ID: <1333435450.21461.35.camel@innu> On Fri, 2012-03-30 at 21:37 +0200, Patrick Westenberg wrote: > Nick Warr schrieb: > > > I think some of the new Dovecot (director?) software is user aware, but > > I don't know if it's quite ready for production. > > Yes, with director it should be something like that: > > MTA --(lmtp)--\ /--(lmtp)--> backend1 --\ > -- director -- -- NFS > MTA --(lmtp)--/ \--(lmtp)--> backend2 --/ > > > IMAP-User --> frontend1 --\ /--(imap)--> backend1 --\ > -- director -- -- NFS > IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/ The director above must be the same one for both IMAP and MTA. > So what happens if user1 at example.tld receives a mail? > > - The director decides to connect to backend1 which in turn stores > the mail on the NFS share and the index file locally? > > - Then, user1 at example.tld connects to one of the frontends. Does the > director know that, earlier, this user received a mail and proxies him > to backend1 too? Yes. Director process actually doesn't know anything about mail protocols, it simply does username -> backend mapping. The regular Dovecot IMAP/POP3/LMTP/doveadm proxying code handles the rest. From lukas.mueller at newmedia.ch Tue Apr 3 10:24:58 2012 From: lukas.mueller at newmedia.ch (=?iso-8859-1?Q?M=FCller_Lukas?=) Date: Tue, 3 Apr 2012 07:24:58 +0000 Subject: [Dovecot] Dovecot 1.2.9. next_uid was lowered Message-ID: Hi There A while ago I posted here about a crash of dovecot, that had to do with nfs and was quite certainly due to bad luck. But still one problem persists: We have a Mailbox, that is accessed from multiple users using IMAP. Occasionally all the Emails in the mailbox seem to be missing, when accessing the Mailbox through IMAP, although they're still there on the server. This seems to happen, if the users are accessing the Mailbox on Server1 and at the same time an email gets delivered to the mailbox on server2. Since it happens quite seldom, there must be other conditions or circumstances that have to be met for the problem to occur. Unfortunately I don't know which. Shortly after the mail gets delivered on server2, I get the following errors on server1: mail01 dovecot: IMAP(akzidenz at triner.ch): fdatasync(/data/vmail/example.com/user/dovecot-uidlist) failed: Input/output error mail01 dovecot: IMAP(akzidenz at triner.ch): /data/vmail/example.com/user/dovecot-uidlist: next_uid was lowered (2143 -> 2142, hdr=2142) The new mail is than missing in the uidlist and as long as no new mail arrives all folders of the mailbox appear to be empty. If a new mail arrives, it will get the uid that should have been given to the pervious mail (the one that caused the error). The previous mail still won't be listed in the uidlist, but everything works as expected. Best regards Lukas M?ller Systems Engineer _______________________________________________ NEWMEDIA S?dostschweiz Newmedia AG http://www.newmedia.ch _______________________________________________ TYPO3 & Drupal - Wir wissen wie. Ihre professionelle Web Agentur in Chur, Ilanz, Glarus und Z?rich. From lee at standen.id.au Tue Apr 3 11:02:05 2012 From: lee at standen.id.au (Lee Standen) Date: Tue, 3 Apr 2012 16:02:05 +0800 Subject: [Dovecot] Dovecot 1.2.9. next_uid was lowered In-Reply-To: References: Message-ID: I'll just say what I'm sure someone else will shortly :) The problem is NFS: http://wiki2.dovecot.org/NFS The solution is 2.x Director: http://wiki2.dovecot.org/Director Although it looks like you're currently on 1.x so it'd be a fairly significant upgrade for you. On Tue, Apr 3, 2012 at 3:24 PM, M?ller Lukas wrote: > Shortly after the mail gets delivered on server2, I get the following > errors on server1: > mail01 dovecot: IMAP(akzidenz at triner.ch): fdatasync(/data/vmail/ > example.com/user/dovecot-uidlist) failed: Input/output error > mail01 dovecot: IMAP(akzidenz at triner.ch): /data/vmail/ > example.com/user/dovecot-uidlist: next_uid was lowered (2143 -> 2142, > hdr=2142) > > From CMarcus at Media-Brokers.com Tue Apr 3 14:33:39 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 03 Apr 2012 07:33:39 -0400 Subject: [Dovecot] dsync redesign In-Reply-To: <87zkatpvxr.fsf@algae.riseup.net> References: <936E95DA-E032-4F8F-A323-DEBB7AAC0E1B@iki.fi> <87obrixcyp.fsf@algae.riseup.net> <4F721650.4030901@Media-Brokers.com> <87zkatpvxr.fsf@algae.riseup.net> Message-ID: <4F7AE013.6040402@Media-Brokers.com> On 2012-04-02 7:15 PM, Micah Anderson wrote: > Charles Marcus writes: >> On 2012-03-27 11:47 AM, Micah Anderson wrote: >>> One would be the ability to perform *intelligent* incremental / >>> rotated backups. I can do this now by running a dsync backup >>> operation and then doing manual hardlinking or moving of the >>> backup directories (daily.1, daily.2, weekly.1, monthly.1, etc.), >>> but it would be more intelligent if this were baked into the >>> backup process. >> There are already numerous tools that do this flawlessly - I've >> beenusing rsnapshot (which uses rsync) for this for years. > Are you snapshotting your filesystem (using LVM, or SAN, or similar) > before doing rsnapshot? Because if you aren't then rsync will not > assuredly get everything in a consistent state. No, and you are correct... but I run it in the middle of the night, and the system is only barely utilized at the time, so the very minor inconsistencies are not a problem overall. I will, however, be changing this to using FS snapshots once I get my mailserver virtualized (already being planned for when our new office location comes online), so that will allow me to perform snapshots multiple times during the day (I'm thinking 4 times per day will be enough). >> I don't know if Timo should be spending his time reinventing the >> wheel. > dsync backup is already here, and it is quite useful. I'm not saying it isn't, I'm just saying that there are already *plenty* of different backup tools, and I don't see the sense in Timo spending lots of time on creating a new one just for dovecot. His time would be better spent just making it easier for any other backup tool to work better. >> Although, one interesting piece that I am hopeful I'll be able to >> implement soon (with Timo's professional help) is the ability to >> easily and automatically map my rsnapshot snapshots directory to a >> read-only 'Backups' namespace that automatically shows the >> snapshots by date and time as they are produced. This way users >> could 'go back in time' anytime they wanted without having to call >> me... :) > Interesting idea, would be a great one to share with the community > if you decide to do so. Absolutely - that is already on my list for when I pay Timo's company to do this - document it on the wiki. Hopefully if any code changes are needed to make it work right, they will be minor. -- Best regards, Charles From xnasx at yandex.ru Tue Apr 3 15:33:11 2012 From: xnasx at yandex.ru (xnasx) Date: Tue, 03 Apr 2012 16:33:11 +0400 Subject: [Dovecot] Courier to Dovecot migration script issue Message-ID: <125901333456391@web125.yandex.ru> Hi! Forgive my bad English. I trying to migrate to Dovecot 2.0.18 from Courier 4.10. I using a script 'courier-dovecot-migrate.pl' from oficial wiki - it works without errors and creating dovecot-uidlist, etc. . I running Dovecot on a different port: 11149. When I connecting to port 11149 from Thunderbird, all my message headers are reloading. Please help me to solve this issue. Best regards, Victor From tss at iki.fi Tue Apr 3 16:00:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 16:00:47 +0300 Subject: [Dovecot] Per-user flags/indexes for shared mailboxes Message-ID: <1333458047.21461.48.camel@innu> The attached patch adds support for doing e.g.: namespace { type = public prefix = Public/ location = mdbox:/var/lib/public/mdbox:INDEXPVT=~/mdbox/public list = yes } So the private flags are stored in indexes under ~/mdbox/public/, while the shared flags are stored under /var/lib/public/mdbox/. The patch currently hardcodes the shared flags as only MAIL_SEEN. I'm not really sure what would be the best place to configure which flags are private. Perhaps a per-namespace default, but how to configure it per-mailbox? Also per-user keywords aren't currently supported. And it's a little bit annoying that each private flag update must increase the shared modseq for the message, but that can't really be helped. (Alternative would be for each user to have their own private modseqs, which could get difficult.) I'll probably commit this to v2.1 after some more testing. -------------- next part -------------- A non-text attachment was scrubbed... Name: private-index.diff Type: text/x-patch Size: 28899 bytes Desc: not available URL: From feltrin at gmail.com Tue Apr 3 17:17:59 2012 From: feltrin at gmail.com (Jean Michel) Date: Tue, 3 Apr 2012 11:17:59 -0300 Subject: [Dovecot] Backtrace on mdbox index In-Reply-To: <1333434809.21461.33.camel@innu> References: <1333434809.21461.33.camel@innu> Message-ID: I tried with dovecot 2.1.3 and got the same error, any ideia ? 2012/4/3 Timo Sirainen > On Mon, 2012-04-02 at 11:14 -0300, Francisco Wagner C. Freire wrote: > > > I current using dovecot with mdbox and on one account i get this error: > > > > doveadm(user at domain): Panic: file mail-index-sync-ext.c: line 209 > > (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) > > Dovecot version? I think I've fixed this in recent v2.1.x. > > > -- ------------------------------ Jean Michel Feltrin From tss at iki.fi Tue Apr 3 17:37:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 17:37:26 +0300 Subject: [Dovecot] Backtrace on mdbox index In-Reply-To: References: <1333434809.21461.33.camel@innu> Message-ID: <1333463846.21461.50.camel@innu> Could you privately send me your all of your dovecot.index, dovecot.index.log, dovecot.map.index and dovecot.map.index.log files under mdbox? None of those contain any sensitive data. On Tue, 2012-04-03 at 11:17 -0300, Jean Michel wrote: > I tried with dovecot 2.1.3 and got the same error, any ideia ? > > 2012/4/3 Timo Sirainen > > > On Mon, 2012-04-02 at 11:14 -0300, Francisco Wagner C. Freire wrote: > > > > > I current using dovecot with mdbox and on one account i get this error: > > > > > > doveadm(user at domain): Panic: file mail-index-sync-ext.c: line 209 > > > (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) > > > > Dovecot version? I think I've fixed this in recent v2.1.x. > > > > > > > > From tss at iki.fi Tue Apr 3 17:40:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 17:40:34 +0300 Subject: [Dovecot] imapdir still a configuration option? In-Reply-To: <4F7A18C8.3000904@se.rit.edu> References: <4F7A18C8.3000904@se.rit.edu> Message-ID: <1333464034.21461.51.camel@innu> On Mon, 2012-04-02 at 17:23 -0400, Kurt Mosiejczuk wrote: > I've been looking at moving an old IMAPdir-based solution to dovecot. I > came across references that dovecot (to my pleasant surprise) added > imapdir support in 1.1. However, in trying 2.0.13 (from an OpenBSD > package) things start okay, but I see in the logs: > > "Initialization failed: Initializing mail storage from mail_location > setting failed: Unknown mail storage driver imapdir" > > Poking around in the sources, I still see lots of references to imapdir, > but see very little in the documentation. > > Is IMAPdir still supported in 2.0.x (or even 2.1.x)? I'd prefer not to > have to migrate users via renaming, especially since they like the > IMAPdir layout for their own searching. mail_location = maildir:~/Maildir:LAYOUT=imapdir From tss at iki.fi Tue Apr 3 17:43:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Apr 2012 17:43:50 +0300 Subject: [Dovecot] Connection queue full on dovecot 2.0.13 In-Reply-To: <4F79BA0C.1070104@libero.it> References: <4F79BA0C.1070104@libero.it> Message-ID: <1333464230.21461.53.camel@innu> On Mon, 2012-04-02 at 16:39 +0200, Giovanni Mancuso wrote: > Apr 02 14:34:36 imap-login: Info: Disconnected: *Connection queue full > *(auth failed, 1 attempts): user= I see this wiki page: http://wiki2.dovecot.org/LoginProcess but i read: > > It works by using a number of long running login processes, each > handling a number of connections. This loses much of the security > benefits of the login process design, because in case of a security hole > (in Dovecot or SSL library) the attacker is now able to see other users > logging in and steal their passwords, read their mails, etc. > > Is there another way? Just increase the number of processes: service imap-login { process_limit = 10000 } From kurt-dovecot at se.rit.edu Tue Apr 3 17:50:59 2012 From: kurt-dovecot at se.rit.edu (Kurt Mosiejczuk) Date: Tue, 03 Apr 2012 10:50:59 -0400 Subject: [Dovecot] imapdir still a configuration option? In-Reply-To: <1333464034.21461.51.camel@innu> References: <4F7A18C8.3000904@se.rit.edu> <1333464034.21461.51.camel@innu> Message-ID: <4F7B0E53.50601@se.rit.edu> Timo Sirainen wrote: > mail_location = maildir:~/Maildir:LAYOUT=imapdir Excellent. That works *perfectly*. --Kurt From stsiol at yahoo.co.uk Tue Apr 3 17:55:54 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 3 Apr 2012 15:55:54 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333434757.21461.32.camel@innu> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333434757.21461.32.camel@innu> Message-ID: <1333464954.38480.YahooMailNeo@web132206.mail.ird.yahoo.com> Hi Timo,? >________________________________ > From: Timo Sirainen >To: Spyros Tsiolis ? >Cc: Dovecot ? >Sent: Tuesday, 3 April 2012, 8:32 >Subject: Re: [Dovecot] Mail migration troubles >? >On Sun, 2012-04-01 at 22:54 +0100, Spyros Tsiolis wrote: >> However, I have this nagging problem with most of the thunderbird >> MTAs. When I try to delete some "test" messages I sent to make sure >> that everything works, it comes back with the following message on >> the bottom of the Thunderbird window : >>? >> "The current operation on InBox did not succeed.? >> The mail server for account "Account at bloomingdomain.gr"? >> responded [TRYCREATE] mailbox doesn't exist : Trash" > >That error probably happens when deleting a message. Yes, it does Timo. Thank you. >> Dovecot v1.2.15 AFAIK Actually it's v1.2.16 >dovecot -n output would have helped. Mainly: do you have a namespace >prefix or not? That's about the only thing I can think of related to >that error message (e.g. dovecot has non-empty namespace prefix, while >TB has empty namespace prefix). Why do I keep forgetting this ? Damn the gods :-) !@#$ :-) Here's the output of "dovecot -n" : -------------------------------------------------------------------- [root at mailgate ~]# dovecot -n # 1.2.16: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 base_dir: /var/run/dovecot/ log_path: /var/log/dovecot/dovecot.log info_log_path: /var/log/dovecot/dovecot-info.log ssl_parameters_regenerate: 48 verbose_ssl: yes login_dir: /var/run/dovecot//login login_executable: /usr/local/dovecot/libexec/dovecot/imap-login login_greeting: * Dovecot ready * login_max_processes_count: 64 mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir mail_plugins: zlib auth default: ? verbose: yes ? debug: yes ? debug_passwords: yes ? passdb: ? ? driver: passwd-file ? ? args: /etc/dovecot/passwd ? passdb: ? ? driver: pam ? userdb: ? ? driver: static ? ? args: uid=vmail gid=vmail home=/home/vmail/%u ? userdb: ? ? driver: passwd [root at mailgate ~]#? -------------------------------------------------------------------- I suspect it's some freak thunderbird bug. It's the only explanation. I've migrated about 20 mailboxes and it's the only one acting up. Thanks again Timo, s. ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From stsiol at yahoo.co.uk Tue Apr 3 18:06:42 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 3 Apr 2012 16:06:42 +0100 (BST) Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333464954.38480.YahooMailNeo@web132206.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333434757.21461.32.camel@innu> <1333464954.38480.YahooMailNeo@web132206.mail.ird.yahoo.com> Message-ID: <1333465602.15453.YahooMailNeo@web132203.mail.ird.yahoo.com> Also, What do you mean by namespace ? I am sorry I didn't get that one : >> dovecot -n output would have helped. Mainly: do you have a namespace >> prefix or not? That's about the only thing I can think of related to >> that error message (e.g. dovecot has non-empty namespace prefix, while >> TB has empty namespace prefix). I missed the "namespace" thing. s. ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From nmilas at noa.gr Tue Apr 3 18:33:59 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Tue, 03 Apr 2012 18:33:59 +0300 Subject: [Dovecot] Mail migration troubles In-Reply-To: <1333465602.15453.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1333317280.78593.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333434757.21461.32.camel@innu> <1333464954.38480.YahooMailNeo@web132206.mail.ird.yahoo.com> <1333465602.15453.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <4F7B1867.3030608@noa.gr> On 3/4/2012 6:06 ??, Spyros Tsiolis wrote: > What do you mean by namespace ? See: http://wiki2.dovecot.org/Namespaces Best regards, Nick From trashcan at odo.in-berlin.de Tue Apr 3 20:29:41 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Tue, 3 Apr 2012 19:29:41 +0200 Subject: [Dovecot] dovecot and unison In-Reply-To: <4F7A2FF9.7000204@klacto.net> References: <4F7A2FF9.7000204@klacto.net> Message-ID: <8EAB42AF-BC4A-4235-9CF9-A444BE553B10@odo.in-berlin.de> Hi -- [Sorry for my private response, instead of public one] On 03.04.2012, at 01:02, FZiegler wrote: > Now I'd like if possible, to replicate this setup on my laptop and keep > both in sync with unison (http://www.cis.upenn.edu/~bcpierce/unison/), > which I am already using to sync much of my home dir about once a day. > > I found at least one positive message regarding this topic > (http://dovecot.org/list/dovecot/2010-April/048092.html), but I feel I > could use some more advice. I used to sync my two server's maildirs for some years, but I moved to dsync mirroring some months ago, and now I'm into using replicator/dsync (see archive of March 2012). Unsion mirroring is working in general, but sometimes you might realize some duplicates being produced. Not bad, but it happens. I did realize the same with using dsync mirror instead. It's much better with the new replicator/dsync mirroring scheme, but it's not perfect, yet. (Again, see ML archives of the last weeks). > Namely, which of the ancillary files should I be syncing (or perhaps > deleting before sync)? In addition to the mails themselves I'm seeing: > > -rw------- 1 fz 501 13 Apr 1 17:10 dovecot-keywords > -rw------- 1 fz 501 2483 Apr 1 17:10 dovecot-uidlist > -rw------- 1 fz 501 1040 Mar 29 14:50 dovecot.index > -rw------- 1 fz 501 44032 Apr 1 17:10 dovecot.index.cache > -rw------- 1 fz 501 9772 Apr 1 17:10 dovecot.index.log > > and at the top level (in .local/var/Maildir/): > > -rw------- 1 fz 501 8 Apr 1 17:10 dovecot-uidvalidity > -r--r--r-- 1 fz 501 0 Apr 1 17:10 dovecot-uidvalidity.4efb323d > -rw------- 1 fz 501 3288 Apr 1 17:10 dovecot.mailbox.log > -rw------- 1 fz 501 4104 Mar 28 10:53 dovecot.mailbox.log.2 I used ... ignore = Name dovecot* ignore = Regex .*/tmp$ ignore = Regex .*/\.Drafts$ ? to ignore all of them. When runnig mirroring for the very first time, I started with one empty maildir at one server, IIRC. You better check that with a test maildir beforehand. > -rw------- 1 fz 501 5114 Apr 1 17:10 subscriptions That became mirrored. Regards, Michael From dave-lists-dovecot at weller-fahy.com Tue Apr 3 21:44:32 2012 From: dave-lists-dovecot at weller-fahy.com (David J. Weller-Fahy) Date: Tue, 3 Apr 2012 14:44:32 -0400 Subject: [Dovecot] Dovecot stones In-Reply-To: References: <3DC1C1CE-B290-4B59-89A4-3B1541F52448@iki.fi> Message-ID: <20120403184431.GA792@weller-fahy.com> * Mark Moseley [2012-04-02 12:41 -0400]: > On Sat, Mar 31, 2012 at 2:28 PM, Timo Sirainen wrote: > > For the last few days I've been thinking about my company and what > > > >... > > I'm slightly concerned that there's been no mention of what license > these stones are going to be released under. GPL2? GPL3? Apache? I'm > just hoping these aren't some sort of "open core" stones that will > only work for basic features but that I'll end up needing to buy > "Enterprise-grade stones" to cover large clusters. From the description and images, I'd say they are probably binary blobs to be included in the firmware via hardware dongle. ;) -- dave [ please don't CC me ] -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From dm-list-email-dovecot at scs.stanford.edu Tue Apr 3 22:22:02 2012 From: dm-list-email-dovecot at scs.stanford.edu (dm-list-email-dovecot at scs.stanford.edu) Date: Tue, 03 Apr 2012 12:22:02 -0700 Subject: [Dovecot] dovecot and unison In-Reply-To: <4F7A2FF9.7000204@klacto.net> References: <4F7A2FF9.7000204@klacto.net> Message-ID: <87398kei3p.wl@ta.scs.stanford.edu> At Mon, 02 Apr 2012 19:02:17 -0400, FZiegler wrote: > > I am successfully using dovecot purely as a personal local mail store on > my desktop. (There is only one account, and it's only ever accessed by > local mail clients on the machine. The point is to have a common store I > can use with any client; plus, I prefer dovecot's Mailbox storage to > Thunderbird's mboxes.) > > Now I'd like if possible, to replicate this setup on my laptop and keep > both in sync with unison (http://www.cis.upenn.edu/~bcpierce/unison/), > which I am already using to sync much of my home dir about once a day. > > I found at least one positive message regarding this topic > (http://dovecot.org/list/dovecot/2010-April/048092.html), but I feel I > could use some more advice. I have a similar setup, but I use offlineimap instead of unison: http://offlineimap.org/ It seems to work pretty well. That's not to say that unison wouldn't work as well also. However, offlineimap has the advantage that it doesn't restrict you to a star topology. You can, for instance, since to your laptop at work and from your laptop at home. Note that offlineimap is slow if you don't use imap at both ends. Therefore, I use it on the local end. A simplified excerpt of my .offlineimaprc looks like this: ======== [general] accounts = DefaultAccount [Account DefaultAccount] localrepository = MyLocal remoterepository = MyRemote [Repository MyRemote] type = IMAP preauthtunnel = ssh -qax -oBatchMode=yes -oServerAliveInterval=60 MY-MAIL-SERVER 'exec env CONFIG_FILE=/PATH/TO/PRIVATE/dovecot.conf /usr/lib/dovecot/imap' [Repository MyLocal] type = IMAP preauthtunnel = CONFIG_FILE=$HOME/etc/dovecot.conf /usr/lib/dovecot/imap ======== Unfortunately, in dovecot 2.1, the full text search no longer seems to work in pre-auth mode, but I don't think that has anything to do with offlineimap. I think maybe dovecot is deprecating pre-auth mode or requires a more complicated setup. From tss at iki.fi Wed Apr 4 03:45:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 03:45:36 +0300 Subject: [Dovecot] Backtrace on mdbox index In-Reply-To: References: Message-ID: <7F642313-D6F2-4032-A913-63E9BDAB6848@iki.fi> On 2.4.2012, at 17.14, Francisco Wagner C. Freire wrote: > I current using dovecot with mdbox and on one account i get this error: > > doveadm(user at domain): Panic: file mail-index-sync-ext.c: line 209 > (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) Fixed: http://hg.dovecot.org/dovecot-2.1/rev/47526bf86c57 From tss at iki.fi Wed Apr 4 03:48:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 03:48:12 +0300 Subject: [Dovecot] 2.1.3: doveadm mailbox delete lost the -s option In-Reply-To: <874nt1rcf4.fsf@algae.riseup.net> References: <874nt1rcf4.fsf@algae.riseup.net> Message-ID: On 3.4.2012, at 1.34, Micah Anderson wrote: > Looks like the recently added -s (for unsubscribing) option to doveadm > mailbox delete went missing: > > root at vireo# /usr/bin/doveadm mailbox delete -u micahtest -s restored > delete: invalid option -- 's' > doveadm mailbox delete [-u |-A] [-S ] [-s] [...] > > although the option still is shown as valid :) Fixed: http://hg.dovecot.org/dovecot-2.1/rev/411344f9daf3 From tss at iki.fi Wed Apr 4 03:56:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 03:56:05 +0300 Subject: [Dovecot] dovecot 2.1 breaks FTS + pre-auth? In-Reply-To: <878vigok53.wl@ta.scs.stanford.edu> References: <878vigok53.wl@ta.scs.stanford.edu> Message-ID: <335CB623-DFF7-4676-AA0E-53D7AD42E5C0@iki.fi> On 31.3.2012, at 18.38, dm-list-email-dovecot at scs.stanford.edu wrote: > Hi. I use dovecot in the simplest possible way, as an IMAP server in > pre-auth mode over ssh or just locally over a unix-domain socket > (e.g., with offlineimap, which runs much faster using dovecot for the > local message store). Ideally I would like to avoid running any extra > daemons or setting up anything as root. Until recently, this has > worked fine by just setting the CONFIG_FILE environment variable to > something in my home directory. You can run Dovecot without root: http://wiki2.dovecot.org/HowTo/Rootless > Full text search used to work just fine with this configuration, and > still does on a machine I have running dovecot 2.0.13. However, on > the machine with 2.1, I get errors about /var/run/dovecot/index not > existing. > > $ printf "a select INBOX\nb search text xyzzy\nc logout\n" \ > | /usr/lib/dovecot/imap > * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE] Logged in as dm > imap(dm): Error: net_connect_unix(/var/run/dovecot/indexer) failed: No such file or directory I looked at the code and looks like there is no easy way to make it work the old way. The new way is the only way to make fts-lucene work reliably, and it's also better for fts-solr. And fts-squat is really on its way out to die. From tss at iki.fi Wed Apr 4 03:57:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 03:57:33 +0300 Subject: [Dovecot] Is it possible to migrating mail to dovecot using imapc? In-Reply-To: <695D3A65-CFBF-4DC3-9DAC-E0C299ED0E6D@tao.org.uk> References: <72230FBC-F639-415E-AD1D-7D49CE4C6287@tao.org.uk> <07E5D123-E233-4EB5-B5F8-99B90AF6A809@tao.org.uk> <76BAD212-0807-4EA6-9211-4C3D4F6518CC@iki.fi> <2F722E50-B9AC-4660-AAC2-39FBEEE14364@iki.fi> <25DE9E4A-478B-4D70-8B65-0A727B0DEFAF@tao.org.uk> <9F05AEAA-6D32-417E-85F2-240FF39D0426@iki.fi> <695D3A65-CFBF-4DC3-9DAC-E0C299ED0E6D@tao.org.uk> Message-ID: <1C12148A-959F-453C-AA98-15ED001F53B6@iki.fi> On 30.3.2012, at 23.42, Dr Josef Karthauser wrote: > % dsync -Dv -u joe at local.com -o imapc_host=mail.remoteimap.com -o imapc_port=143 -o imapc_username=joe@ remoteimap.com -o imapc_password='somepass' mirror imapc: > & /tmp/output > > produced the following output. > dsync(joe at email.com): Error: Mailbox INBOX changed its GUID (f38c263a4919764f1b560100c516a7a4 -> c92f64f79f0d1ed01e6d5b314f04886c) dsync + imapc requires indexes. So you need to use e.g. imapc:/tmp/dsync-imapc From tss at iki.fi Wed Apr 4 04:05:52 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 04:05:52 +0300 Subject: [Dovecot] Many messages clustered around the same date.saved value In-Reply-To: References: Message-ID: <31C97BDA-5B4E-4289-9087-9AC72F11F0F5@iki.fi> On 29.3.2012, at 5.41, Joseph Tam wrote: >> Ah, with mbox there isn't any usable fallback for date.saved. If it's >> not in dovecot.index.cache, the current time is used. > > I'm a little confused as to why it needed a fallback. In other words, > why wasn't date.saved put into the index as soon as the IMAP operation > copied it into "Trash"? > > If this data isn't set at that time, when does it get instantiated? > When I actually ask for it? Well..: - date.saved is stored only in dovecot.index.cache file - if it doesn't exist and is requested, the current time is returned and it's added to the cache - when date.saved has already fetched once (so it already exists in dovecot.index.cache file), and mail is saved via LDA/IMAP then it gets added there immediately when saving - dovecot.index.cache has caching decisions, and some old/unused fields may get dropped from it once in a while - maybe due to some bugs or whatever, the fields or the entire cache may get dropped for some other reason So it probably should have worked, but for some reason didn't. It would be possible to store date.saved in dovecot.index file, like mdbox does, so cache decisions wouldn't matter. But probably too much trouble to be worth it, very few mbox installations care about it. From tss at iki.fi Wed Apr 4 04:08:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 04:08:46 +0300 Subject: [Dovecot] Dovecot upgrade from 1.2.x to 2.0.x: roundcube/squirrelmail sent folder doesn't work any more In-Reply-To: References: <4F71F8D8.6040700@wiesinger.com> <95C664AD-0532-4F3B-A2EB-2FD25B79187B@iki.fi> Message-ID: On 29.3.2012, at 8.25, Gerhard Wiesinger wrote: >>> 2.0.x: with Prefix ~/Mail >>> A0003 LIST "" ~/Mail/sent >>> A0003 OK List completed.: >> > > mail_full_filesystem_access = yes > mail_location = mbox:~:INBOX=/var/mail/%u Yes, the problem happens only with mail_full_filesystem_access=yes. It already works in v2.1, so I don't think I'll bother looking into why it's not working in v2.0. From tss at iki.fi Wed Apr 4 04:11:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 04:11:04 +0300 Subject: [Dovecot] File/folder permission issues in 2.1.3 In-Reply-To: <8B296F70-22B8-487B-AD7A-47BEB8C84F62@roessner-network-solutions.com> References: <8B296F70-22B8-487B-AD7A-47BEB8C84F62@roessner-network-solutions.com> Message-ID: On 29.3.2012, at 13.24, Christian R??ner wrote: > I figured out that Dovecot does not honer secondary groups with auth/auth-worker (??), if doing LDAP/TLS stuff. I had to use file system acls to add the user "vmail" to /etc/ssl/private and to the corresponding key file: service auth { extra_groups = any groups you want auth process to have } Don't give secondary groups to any Dovecot related users. From tss at iki.fi Wed Apr 4 04:35:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 04:35:13 +0300 Subject: [Dovecot] Dovecot allows creation of folders outside of a user's directory In-Reply-To: <4F759B08.1060603@in.tum.de> References: <4F759B08.1060603@in.tum.de> Message-ID: On 30.3.2012, at 14.37, Christoph Bu?enius wrote: > in our dovecot 2.0 setup with shared folders, users can make dovecot create directories outside their mail directory. Which is a bit scary imho. > > The following command: > > . create inbox.shared.abc123 > > or even > > . create "inbox.shared.strange &ANY- characters" > > -- even though it will fail with a "permission denied" error -- will create a directory like "/mail/users/strange &ANY- characters". That directory will only contain a subdirectory "Maildir" and therein dovecot-acl-list. Fixed: http://hg.dovecot.org/dovecot-2.0/rev/b15889b82258 From tss at iki.fi Wed Apr 4 04:47:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 04:47:33 +0300 Subject: [Dovecot] namespace hierarchy In-Reply-To: <201204010928.41017.anyaddress@gmx.net> References: <201204010928.41017.anyaddress@gmx.net> Message-ID: On 1.4.2012, at 10.28, Tom Fernandes wrote: > I'm migrating from Courier to Dovecot and would like to keep the original > namespace and add two new ones on the new server. > > There's only one namespace in Courier which is "INBOX." . The seperator is "." > > In the new server I would like to have 3 namespaces (private, public, shared) > and use "/" as separator. Is it possible to keep "INBOX." for private, so that > the users mailboxes appear below the Inbox, but have public and shared on the > same hierarchy-level like the Inbox? > > When I set prefix to "INBOX/" for private I get the users mailboxes below the > Inbox, like I have it Courier now. But then at least the public, probably also > the shared namespace doesn't work any more. I get "unknown subscription > namespace" when trying to subscribe to some of the folders. If you have any subscriptions=no namespaces, you need a parent namespace that has subscriptions=yes. You could have e.g.: namespace { prefix = INBOX/ list = no inbox = yes } namespace { prefix = list = no hidden = yes alias_for = INBOX/ } So now the prefix="" is used for the subscriptions but nothing else really. Anyway, INBOX/ isn't the same as INBOX. so if any client configs are using INBOX. as namespace prefix then they don't work with INBOX/. From tss at iki.fi Wed Apr 4 04:49:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 04:49:48 +0300 Subject: [Dovecot] imapsync, namespaces and Dovecot 2.x In-Reply-To: References: Message-ID: <65D3A596-EC48-430C-8F53-4A11F30437C4@iki.fi> On 2.4.2012, at 19.11, Aleix Dorca wrote: > I recently tried to migrate from Exchange using imapsync. It didn't work because it complained that dovecot had no namespace capablity. After looking through Dovecot's config files I found that namespaces could be defined, but if you did not a 'private' default one was created. The problem is that Dovecot doesn't advertise NAMESPACE capability before client has logged in. The better solution would be to change imapsync to figure this out automatically. The other solution would be to set in Dovecot: imap_capability = +NAMESPACE From tss at iki.fi Wed Apr 4 05:35:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 05:35:33 +0300 Subject: [Dovecot] lmtp sometimes fails to deliver a message to all recipients In-Reply-To: <4F79C995.9060607@talex.pl> References: <4F79C995.9060607@talex.pl> Message-ID: <1B1369FF-6748-46DE-AB94-AF2F32061757@iki.fi> On 2.4.2012, at 18.45, Artur Zaprza?a wrote: > Running strace on lmtp reveals that when lmtp delivers a message to multiple recipients it saves the message to a file in the first mailbox and hard links that file to other mailboxes. But when in the meantime of delivery that file is deleted, lmtp fails to deliver the message to remaining mailboxes. For example a message for 46 recipients is delivered by lmtp to the first 12 only (fortunately postfix retries delivery for remaining recipients) and for the rest lmtp logs the following error messages: > > lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Message was expunged (guid) > lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: script /vmail/domain/foo/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /vmail/domain/foo/.dovecot.sieve.log may reveal additional details) Fixed in hg. From tss at iki.fi Wed Apr 4 05:49:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 05:49:40 +0300 Subject: [Dovecot] dovecot-auth restaring and caching In-Reply-To: <4F6C696C.5030900@um.es> References: <4F6AF72E.9030206@um.es> <7B9D052D-5864-42A9-AE9A-6FCE858F48C0@iki.fi> <4F6C696C.5030900@um.es> Message-ID: On 23.3.2012, at 14.15, Angel L. Mateo wrote: > El 22/03/12 19:57, Timo Sirainen escribi?: >> On 22.3.2012, at 11.55, Angel L. Mateo wrote: >> >>> The problem I'm having is that if I have no activity in the server, dovecot stops its auth process and when another message is received, it restarted it, but with an empty cache. >> >> service auth { >> idle_kill = 0 >> } >> > > In a test server I have, this have solved the problem. In my productions servers it is still being restarted. Could it be another parameter involve in this? Oops, I remembered wrong. idle_kill=0 uses default_idle_kill limit. You could set for example: idle_kill = 4294967295s From tss at iki.fi Wed Apr 4 05:51:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 05:51:19 +0300 Subject: [Dovecot] dovecot 2.0.19 Panic: file mail-index-sync-ext.c: line 209 (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) In-Reply-To: <4F6A1BA7.7030208@uvm.edu> References: <4F69DD6E.1090502@uvm.edu> <94A1158A-2F42-406A-9212-C58A4D4FF879@iki.fi> <4F6A1BA7.7030208@uvm.edu> Message-ID: <18B56699-7AF4-4ED8-A21F-4EBA9E7C6752@iki.fi> On 21.3.2012, at 20.19, Jim Lawson wrote: > On 3/21/12 10:02 AM, Timo Sirainen wrote: >> On 21.3.2012, at 15.53, Jim Lawson wrote: >> >>> Had a user who couldn't access his INBOX: >>> >>>> Mar 21 09:21:17 penguina dovecot: imap([USER]): Panic: file >>>> mail-index-sync-ext.c: line 209 (sync_ext_reorder): assertion fai >>>> led: (offset < (uint16_t)-1) >> I kind of remember that this was fixed by http://hg.dovecot.org/dovecot-2.1/rev/b4d8e950eb9d but I'm not entirely sure. I guess I should have included in the commit the error message it fixed. > > > This applies cleanly against 2.0.19; should I try it on that version, or > not recommended? http://hg.dovecot.org/dovecot-2.1/rev/47526bf86c57 should fix this. From tss at iki.fi Wed Apr 4 05:57:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 05:57:03 +0300 Subject: [Dovecot] Courier to Dovecot migration script issue In-Reply-To: <125901333456391@web125.yandex.ru> References: <125901333456391@web125.yandex.ru> Message-ID: <9BBB03B5-4742-403B-8720-FBEFB554266D@iki.fi> On 3.4.2012, at 15.33, xnasx wrote: > I trying to migrate to Dovecot 2.0.18 from Courier 4.10. > I using a script 'courier-dovecot-migrate.pl' from oficial wiki - it works without errors and creating dovecot-uidlist, etc. . > > I running Dovecot on a different port: 11149. > When I connecting to port 11149 from Thunderbird, all my message headers are reloading. > > Please help me to solve this issue. telnet localhost 143 a login user pass b select inbox c status inbox (uidvalidity) d fetch 1:* uid telnet localhost 11149 a login user pass b select inbox c status inbox (uidvalidity) d fetch 1:* uid Check if the "c" and "d" outputs are the same for both servers. If they are, then the migration was successful and the problem is something else, such as maybe the client always redownloading messages because you changed the port (or something else). From tss at iki.fi Wed Apr 4 06:16:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 06:16:54 +0300 Subject: [Dovecot] ldap userdb warning in v2.1.1 In-Reply-To: References: <4F69989A.3000106@noa.gr> Message-ID: On 21.3.2012, at 14.06, Timo Sirainen wrote: > On 21.3.2012, at 11.00, Nikolaos Milas wrote: > >> Mar 21 10:07:23 imapserver dovecot: master: Dovecot v2.1.1 starting up (core dumps disabled) >> Mar 21 10:08:17 imapserverdovecot: auth: Warning: ldap: Ignoring changed user_attrs in /etc/dovecot/dovecot-passdb-ldap.conf, because userdb ldap not used. (If this is intentional, set userdb_warning_disable=yes) >> >> I didn't see such warnings in 2.0.13. >> >> I guess I should/could remove the "user_attrs" line from dovecot-passdb-ldap.conf because it's not needed? > > Hmm. Yes, if dovecot-usrdb-ldap.conf is a separate file from dovecot-passdb-ldap.conf you can just remove it. But this reminds me that in several places I've suggested to make one of them a symlink to the other, and you can't really do it then. Perhaps I'll need to remove this warning, or maybe make it recognize the symlink case. Removed, at least for now: http://hg.dovecot.org/dovecot-2.1/rev/324df4134049 From tss at iki.fi Wed Apr 4 06:25:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 04 Apr 2012 06:25:51 +0300 Subject: [Dovecot] Dovecot 2.1 with custom OpenSSL fails to build In-Reply-To: References: <1331816286.10319.23.camel@innu.invalid> Message-ID: <4F7BBF3F.9060103@iki.fi> On 03/15/2012 11:43 PM, Andreas M. Kirchwitz wrote: > Compilation works. Great! The binaries find all their libraries. > > But two libraries are not quite okay. They don't find their SSL libs: > > libdovecot-lda.so > libdovecot-storage.so > > Since libdovecot-lda.so doesn't contain the words libssl or libcrypto, > I guess that ldd just complains because it uses libdovecot-storage.so. > Thus, libdovecot-storage.so is the (only) one left with an incomplete > library search path. > > Luckily, all binaries use some additional libraries which come with > a proper library path. So the whole things works, but it's more like > some kind of magic. It would be great if libdovecot-storage.so could > be fixed as well to make things finally perfect. Maybe this fixes it? http://hg.dovecot.org/dovecot-2.1/rev/8b91367bc3e1 From tss at iki.fi Wed Apr 4 06:31:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 06:31:07 +0300 Subject: [Dovecot] Dovecot and scalable database storage In-Reply-To: References: Message-ID: <7CD2C788-D258-458B-A915-1360B861DEE6@iki.fi> On 23.3.2012, at 4.36, wrote: > I saw some interesting mails from TImo back in 2009 talking about the idea > of using something like Cassandra db or similar as a storage platform for > both email and index/logs. I was wondering if this has been discussed > since then, and if there are any plans to support something like this in > the future? I have been playing with Cassandra and found that their > RackAwareStrategy gives you the ability to replicate writes to as many > nodes as you would like, but more importantly what nodes and one of those > nodes could be defined by what rack it lives in or what data center it > lives in. This means multiple sites high available storage clusters, > seemingly a system that dovecot could benefit from in terms of performance > and redundancy and simplicity. Any takers? There are still plans, but not in near future. I'm still planning on implementing it by first making Dovecot's code simpler and more asynchronous, and then the key-value database support could be implemented in just a few hours. From tss at iki.fi Wed Apr 4 07:33:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 07:33:02 +0300 Subject: [Dovecot] doveadm user -f index In-Reply-To: <87pqc3l1xk.fsf@algae.riseup.net> References: <87pqc3l1xk.fsf@algae.riseup.net> Message-ID: <4C96B184-EF85-4301-AEFD-F9201A423D63@iki.fi> http://hg.dovecot.org/dovecot-2.1/rev/601014feade4 doveadm user -m -f home user at domain I'm still not sure if I implemented this the user friendliest way.. It's kind of annoying to have these existing "doveadm auth" and "doveadm user" commands. Perhaps "doveadm user -m" should have been the default, with some added fields, and then the current "doveadm user" would have been as an added option. Hm. Maybe I should change it to v2.2.. On 23.3.2012, at 18.25, Micah Anderson wrote: > > I've configured my mail_location to have a different location for > performance reasons so they aren't in the same location as the > mail_location. > > The 'doveadm user -f home' is useful to find where a user's home > directory is for various scripting purposes, but I can't seem to find a > way to determine the location of the user's indexes. > > I can do something with the output of dovecot -a to find the > mail_location and then look for a configured INDEX, but then I don't > have a good way of translating the %d/%1n/%n type string formatters into > their values for a user. > > thanks for any suggestions! > micah > > -- > > From xnasx at yandex.ru Wed Apr 4 10:36:07 2012 From: xnasx at yandex.ru (xnasx) Date: Wed, 04 Apr 2012 11:36:07 +0400 Subject: [Dovecot] Courier to Dovecot migration script issue In-Reply-To: <9BBB03B5-4742-403B-8720-FBEFB554266D@iki.fi> References: <125901333456391@web125.yandex.ru> <9BBB03B5-4742-403B-8720-FBEFB554266D@iki.fi> Message-ID: <331561333524967@web30.yandex.ru> Thank you! I really don't know what I made, but now all works fine with Thunderbird and Outlook 2007. But something strange: 'uidvalidity' are the same for both servers, but UIDs of mesages are not: Courier: d fetch 1:* uid * 1 FETCH (UID 64295) * 2 FETCH (UID 64296) * 3 FETCH (UID 64297) Dovecot: * 1 FETCH (UID 64296) * 2 FETCH (UID 64297) * 3 FETCH (UID 64298) There can be it because of that that the server is used? 04.04.2012, 06:57, "Timo Sirainen" : > On 3.4.2012, at 15.33, xnasx wrote: > >> ?I trying to migrate to Dovecot 2.0.18 from Courier 4.10. >> ?I using a script 'courier-dovecot-migrate.pl' from oficial wiki - it works without errors and creating dovecot-uidlist, etc. . >> >> ?I running Dovecot on a different port: 11149. >> ?When I connecting to port 11149 from Thunderbird, all my message headers are reloading. >> >> ?Please help me to solve this issue. > > telnet localhost 143 > a login user pass > b select inbox > c status inbox (uidvalidity) > d fetch 1:* uid > > telnet localhost 11149 > a login user pass > b select inbox > c status inbox (uidvalidity) > d fetch 1:* uid > > Check if the "c" and "d" outputs are the same for both servers. If they are, then the migration was successful and the problem is something else, such as maybe the client always redownloading messages because you changed the port (or something else). From adorca at uda.ad Wed Apr 4 13:28:36 2012 From: adorca at uda.ad (Aleix Dorca) Date: Wed, 4 Apr 2012 12:28:36 +0200 Subject: [Dovecot] imapsync, namespaces and Dovecot 2.x In-Reply-To: <65D3A596-EC48-430C-8F53-4A11F30437C4@iki.fi> References: <65D3A596-EC48-430C-8F53-4A11F30437C4@iki.fi> Message-ID: El 04/04/2012, a les 3:49, Timo Sirainen va escriure: > On 2.4.2012, at 19.11, Aleix Dorca wrote: > >> I recently tried to migrate from Exchange using imapsync. It didn't work because it complained that dovecot had no namespace capablity. After looking through Dovecot's config files I found that namespaces could be defined, but if you did not a 'private' default one was created. > > The problem is that Dovecot doesn't advertise NAMESPACE capability before client has logged in. The better solution would be to change imapsync to figure this out automatically. The other solution would be to set in Dovecot: > > imap_capability = +NAMESPACE > Dead on! Thanks a lot. Aleix. From feltrin at gmail.com Wed Apr 4 14:08:21 2012 From: feltrin at gmail.com (Jean Michel) Date: Wed, 4 Apr 2012 08:08:21 -0300 Subject: [Dovecot] Backtrace on mdbox index In-Reply-To: <7F642313-D6F2-4032-A913-63E9BDAB6848@iki.fi> References: <7F642313-D6F2-4032-A913-63E9BDAB6848@iki.fi> Message-ID: Almost 100%, while running force-resync got this message this time: doveadm(acc at domain): Error: Log synchronization error at seq=49,offset=3876 for /storage/3/bd/05/server00001/users/domain/cache/storage/dovecot.map.index: Broken extension introduction: Record alignment is too large doveadm(acc at domain): Warning: fscking index file /storage/3/bd/05/server00001/users/domain/cache/storage/dovecot.map.index doveadm(acc at domain): Warning: mdbox /storage/3/bd/05/server00001/users/domain/mdbox/storage: rebuilding indexes .... doveadm(acc at domain): Panic: file mail-storage.c: line 787 (mailbox_check_mismatching_separators): assertion failed: (strncmp(vname, ns->prefix, ns->prefix_len-1) == 0) doveadm(acc at domain): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x443da) [0x7fa9c7bbd3da] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7fa9c7bbd4c2] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7fa9c7b9451f] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_open_stream+0) [0x7fa9c7e813c0] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_rebuild_in_context+0xb2b) [0x7fa9c7e43aeb] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync_begin+0x7ec) [0x7fa9c7e41abc] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync+0x46) [0x7fa9c7e41b26] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_sync_init+0x87) [0x7fa9c7e41c07] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x31) [0x7fa9c7e7ff91] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x27) [0x7fa9c7e80aa7] -> doveadm() [0x4103d7] -> doveadm() [0x40f898] -> doveadm() [0x40fbf4] -> doveadm(doveadm_mail_try_run+0x141) [0x410071] -> doveadm(main+0x3a1) [0x416c31] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7fa9c7835c8d] -> doveadm() [0x40f1c9] I'm sending again a link with the indexes and more details on your email. 2012/4/3 Timo Sirainen > On 2.4.2012, at 17.14, Francisco Wagner C. Freire wrote: > > > I current using dovecot with mdbox and on one account i get this error: > > > > doveadm(user at domain): Panic: file mail-index-sync-ext.c: line 209 > > (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1) > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/47526bf86c57 > > > -- ------------------------------ Jean Michel Feltrin From mlists at edicom.eu Wed Apr 4 14:18:00 2012 From: mlists at edicom.eu (Miguel Tormo) Date: Wed, 4 Apr 2012 13:18:00 +0200 Subject: [Dovecot] Proxy and SSO (single sign-on) Message-ID: <201204041318.00760.mlists@edicom.eu> Hello, I have a running setup with a dovecot imap4/pop3 proxy to a few dovecot backend servers which actually store the mailboxes. This is running smoothly and allows me to transparently distribute mailboxes. I'm using some "extrafield" configured in the LDAP passdb. However, now I would like to use GSSAPI (preferred) and NTLM for single sign-on. Both are pretty straightforward to configure in a single instance environment, but I don't know if they would work with proxy. For example, with GSSAPI there are two cases: 1) Just use gssapi mechanism, without PAM. Then, it a user presents a ticket the passdb ldap is not used, so the extrafields are never read. 2) Use gssapi and PAM (thus allowing using a kerberos password). But the extrafields feature isn't available with PAM passdb driver, so again the proxy won't work. The case for NTLM would fall into the first case, I think. Am I right regarding this scenario? Is there a way I could make SSO and proxying work? I'm currently using dovecot 2.0.16 (had to patch it to increment LOGIN_MAX_INBUF_SIZE to 4096 for GSSAPI to work, as I read somewhere in this list), but I could upgrade to a newer version if that allows all this to work. Thanks! From tss at iki.fi Wed Apr 4 14:18:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 14:18:53 +0300 Subject: [Dovecot] Backtrace on mdbox index In-Reply-To: References: <7F642313-D6F2-4032-A913-63E9BDAB6848@iki.fi> Message-ID: <277DFE38-DBF3-4806-B73A-CA797763321F@iki.fi> On 4.4.2012, at 14.08, Jean Michel wrote: > doveadm(acc at domain): Panic: file mail-storage.c: line 787 > (mailbox_check_mismatching_separators): assertion failed: (strncmp(vname, > ns->prefix, ns->prefix_len-1) == 0) That's a bit strange. Can you get a core of it? http://dovecot.org/bugreport.html If you can access the crash with gdb, try commands like: fr 6 (or maybe 5, or maybe 4, or 7, or whatever makes the following commands work) p *box o *box.list.ns From tss at iki.fi Wed Apr 4 14:21:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 14:21:33 +0300 Subject: [Dovecot] Proxy and SSO (single sign-on) In-Reply-To: <201204041318.00760.mlists@edicom.eu> References: <201204041318.00760.mlists@edicom.eu> Message-ID: <48AD358F-ECC4-48F5-8CAB-9B91C8BDEEE3@iki.fi> On 4.4.2012, at 14.18, Miguel Tormo wrote: > I have a running setup with a dovecot imap4/pop3 proxy to a few dovecot backend servers which actually store the mailboxes. This is running smoothly and allows me to transparently distribute mailboxes. > I'm using some "extrafield" configured in the LDAP passdb. > > However, now I would like to use GSSAPI (preferred) and NTLM for single sign-on. Both are pretty straightforward to configure in a single instance environment, but I don't know if they would work with proxy. For example, with GSSAPI there are two cases: > 1) Just use gssapi mechanism, without PAM. Then, it a user presents a ticket the passdb ldap is not used, so the extrafields are never read. The patch in http://dovecot.org/list/dovecot/2012-March/064331.html makes this work I think. I still haven't managed to look into it much though. From pw at wk-serv.de Wed Apr 4 14:38:55 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Wed, 04 Apr 2012 13:38:55 +0200 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <1333435450.21461.35.camel@innu> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> Message-ID: <4F7C32CF.7010908@wk-serv.de> Timo Sirainen schrieb: > On Fri, 2012-03-30 at 21:37 +0200, Patrick Westenberg wrote: >> MTA --(lmtp)--\ /--(lmtp)--> backend1 --\ >> -- director -- -- NFS >> MTA --(lmtp)--/ \--(lmtp)--> backend2 --/ >> >> >> IMAP-User --> frontend1 --\ /--(imap)--> backend1 --\ >> -- director -- -- NFS >> IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/ > > The director above must be the same one for both IMAP and MTA. I know and it is meant to be the same in this drawing. However, in this scenario the director is a single point of failure. What would be the best way to get loadbalancing and high-availability? Patrick From tss at iki.fi Wed Apr 4 14:46:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 14:46:11 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F7C32CF.7010908@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> Message-ID: <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> On 4.4.2012, at 14.38, Patrick Westenberg wrote: > Timo Sirainen schrieb: >> On Fri, 2012-03-30 at 21:37 +0200, Patrick Westenberg wrote: >>> MTA --(lmtp)--\ /--(lmtp)--> backend1 --\ >>> -- director -- -- NFS >>> MTA --(lmtp)--/ \--(lmtp)--> backend2 --/ >>> >>> >>> IMAP-User --> frontend1 --\ /--(imap)--> backend1 --\ >>> -- director -- -- NFS >>> IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/ >> >> The director above must be the same one for both IMAP and MTA. > > I know and it is meant to be the same in this drawing. > > However, in this scenario the director is a single point of failure. > What would be the best way to get loadbalancing and high-availability? Another director. They're meant to connect to each others and do LB/HA. From mlists at edicom.eu Wed Apr 4 14:47:47 2012 From: mlists at edicom.eu (Miguel Tormo) Date: Wed, 4 Apr 2012 13:47:47 +0200 Subject: [Dovecot] Proxy and SSO (single sign-on) In-Reply-To: <48AD358F-ECC4-48F5-8CAB-9B91C8BDEEE3@iki.fi> References: <201204041318.00760.mlists@edicom.eu> <48AD358F-ECC4-48F5-8CAB-9B91C8BDEEE3@iki.fi> Message-ID: <201204041347.47817.mlists@edicom.eu> El Mi?rcoles, 4 de Abril de 2012 13:21:33 Timo Sirainen escribi?: > On 4.4.2012, at 14.18, Miguel Tormo wrote: > > > I have a running setup with a dovecot imap4/pop3 proxy to a few dovecot backend servers which actually store the mailboxes. This is running smoothly and allows me to transparently distribute mailboxes. > > I'm using some "extrafield" configured in the LDAP passdb. > > > > However, now I would like to use GSSAPI (preferred) and NTLM for single sign-on. Both are pretty straightforward to configure in a single instance environment, but I don't know if they would work with proxy. For example, with GSSAPI there are two cases: > > 1) Just use gssapi mechanism, without PAM. Then, it a user presents a ticket the passdb ldap is not used, so the extrafields are never read. > > The patch in http://dovecot.org/list/dovecot/2012-March/064331.html makes this work I think. I still haven't managed to look into it much though. > > It definitely is worth a look. I wonder if it would make ldap extrafields lookups work with gssapi auth, I will try it and post the results. Thank you! From pw at wk-serv.de Wed Apr 4 15:02:02 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Wed, 04 Apr 2012 14:02:02 +0200 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> Message-ID: <4F7C383A.2060401@wk-serv.de> Timo Sirainen schrieb: > Another director. They're meant to connect to each others and do LB/HA. But what about my MTAs? How can I tell my two postfix servers that there are two directors and it should/can use the other one if the first is down? Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one director. From tss at iki.fi Wed Apr 4 15:06:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 15:06:29 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F7C383A.2060401@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> Message-ID: <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> On 4.4.2012, at 15.02, Patrick Westenberg wrote: > Timo Sirainen schrieb: > >> Another director. They're meant to connect to each others and do LB/HA. > > But what about my MTAs? How can I tell my two postfix servers that there are two directors and it should/can use the other one if the > first is down? > > Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. > Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one director. I don't know if Postfix supports that. Typically people use a load balancer (cluster). From jerry at seibercom.net Wed Apr 4 15:29:21 2012 From: jerry at seibercom.net (Jerry) Date: Wed, 4 Apr 2012 08:29:21 -0400 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> Message-ID: <20120404082921.7d404024@scorpio> On Wed, 4 Apr 2012 15:06:29 +0300 Timo Sirainen articulated: > On 4.4.2012, at 15.02, Patrick Westenberg wrote: > > > Timo Sirainen schrieb: > > > >> Another director. They're meant to connect to each others and do > >> LB/HA. > > > > But what about my MTAs? How can I tell my two postfix servers that > > there are two directors and it should/can use the other one if the > > first is down? > > > > Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. > > Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one > > director. > > I don't know if Postfix supports that. Typically people use a load > balancer (cluster). Perhaps posting on the Postfix forum and asking Wietse Venema would be the logical course of action. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From tss at iki.fi Wed Apr 4 15:39:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 15:39:37 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <20120404082921.7d404024@scorpio> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> <20120404082921.7d404024@scorpio> Message-ID: <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> On 4.4.2012, at 15.29, Jerry wrote: >>> Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. >>> Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one >>> director. >> >> I don't know if Postfix supports that. Typically people use a load >> balancer (cluster). > > Perhaps posting on the Postfix forum and asking Wietse Venema would be > the logical course of action. I have a guess though, or alternatively a suggestion if it already doesn't work like that: lmtp:tcp:lmtp.example.com:24 where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places. From patrickdk at patrickdk.com Wed Apr 4 15:54:28 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Wed, 04 Apr 2012 08:54:28 -0400 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> <20120404082921.7d404024@scorpio> <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> Message-ID: <20120404085428.Horde.9RyhTJLnE6FPfESE0TaTcQA@kishi.patrickdk.com> Quoting Timo Sirainen : > On 4.4.2012, at 15.29, Jerry wrote: > >>>> Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. >>>> Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one >>>> director. >>> >>> I don't know if Postfix supports that. Typically people use a load >>> balancer (cluster). >> >> Perhaps posting on the Postfix forum and asking Wietse Venema would be >> the logical course of action. > > I have a guess though, or alternatively a suggestion if it already > doesn't work like that: > > lmtp:tcp:lmtp.example.com:24 > > where lmtp.example.com expands to your two IP addresses. DNS server > handles load balancing by returning IPs in round robin and Postfix > handles fallbacking to the second IP if the first one doesn't work. > Dovecot has similar behavior in several places. Personally I use MX dns entries, lowest mx entry is localhost, then others. From pw at wk-serv.de Wed Apr 4 16:04:11 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Wed, 04 Apr 2012 15:04:11 +0200 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> <20120404082921.7d404024@scorpio> <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> Message-ID: <4F7C46CB.3050709@wk-serv.de> Timo Sirainen schrieb: > where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places. As far as I understand, lmtp.example.com with out records would result in a MX-lookup. If postfix behaves for lmtp like it does for smtp, it will try all listed MX (my directors) in turn. From tss at iki.fi Wed Apr 4 16:09:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 16:09:46 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F7C46CB.3050709@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> <20120404082921.7d404024@scorpio> <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> <4F7C46CB.3050709@wk-serv.de> Message-ID: On 4.4.2012, at 16.04, Patrick Westenberg wrote: > Timo Sirainen schrieb: > >> where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places. > > As far as I understand, lmtp.example.com with out records would result in a MX-lookup. If postfix behaves for lmtp like it does for smtp, it will try all listed MX (my directors) in turn. MX lookups are intended for SMTP, I very much doubt LMTP uses them? From p at state-of-mind.de Wed Apr 4 16:10:38 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Wed, 4 Apr 2012 15:10:38 +0200 Subject: [Dovecot] Modifying LDAP search results for user_attrs Message-ID: <20120404131037.GB23371@state-of-mind.de> I need to authenticate users via LDAP either by mailaddress or login name. In both cases the mailbox location is /src/mail/%d/%n. I can easily deduct that path from the mailaddress, but not so from the login name. Can I get the mail address as part of fetching user_attrs and modify it on the fly? Something along the lines of this: user_attrs = mail:/srv/mail/%d/%n=home,uidNumber=uid,gidNumber=gid Or would I be able to modify this with a post-login script? p at rick -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From tss at iki.fi Wed Apr 4 16:16:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 16:16:22 +0300 Subject: [Dovecot] Modifying LDAP search results for user_attrs In-Reply-To: <20120404131037.GB23371@state-of-mind.de> References: <20120404131037.GB23371@state-of-mind.de> Message-ID: On 4.4.2012, at 16.10, Patrick Ben Koetter wrote: > I need to authenticate users via LDAP either by mailaddress or login name. > In both cases the mailbox location is /src/mail/%d/%n. I can easily deduct > that path from the mailaddress, but not so from the login name. If you have something like mailAddress=user in your pass_attrs and user_attrs, so that the username gets translated to the mailaddress then you can use %n/%d in dovecot.conf. > Can I get the mail address as part of fetching user_attrs and modify it on the > fly? Something along the lines of this: > > user_attrs = mail:/srv/mail/%d/%n=home,uidNumber=uid,gidNumber=gid > > Or would I be able to modify this with a post-login script? The correct syntax is: user_attrs = =home=mail:/srv/mail/%d/%n, .. but this assumes that the login is user at domain. Without changing the username to mail address you can't use %d/%n directly, and post-login script would be the only way to go. From patrickdk at patrickdk.com Wed Apr 4 16:57:01 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Wed, 04 Apr 2012 09:57:01 -0400 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> <20120404082921.7d404024@scorpio> <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> <4F7C46CB.3050709@wk-serv.de> Message-ID: <20120404095701.Horde.oqR4cpLnE6FPfFMtl5U1tdA@kishi.patrickdk.com> Quoting Timo Sirainen : > On 4.4.2012, at 16.04, Patrick Westenberg wrote: > >> Timo Sirainen schrieb: >> >>> where lmtp.example.com expands to your two IP addresses. DNS >>> server handles load balancing by returning IPs in round robin and >>> Postfix handles fallbacking to the second IP if the first one >>> doesn't work. Dovecot has similar behavior in several places. >> >> As far as I understand, lmtp.example.com with out records would >> result in a MX-lookup. If postfix behaves for lmtp like it does for >> smtp, it will try all listed MX (my directors) in turn. > > MX lookups are intended for SMTP, I very much doubt LMTP uses them? Yes, but postfix lmtp is the same binary as smtp, at least currently. From tss at iki.fi Wed Apr 4 17:10:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 4 Apr 2012 17:10:31 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <20120404095701.Horde.oqR4cpLnE6FPfFMtl5U1tdA@kishi.patrickdk.com> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> <27C1D6A5-E030-4BC4-B118-B562C464E307@iki.fi> <20120404082921.7d404024@scorpio> <57BC36BB-DB32-4138-AEE3-43798320C209@iki.fi> <4F7C46CB.3050709@wk-serv.de> <20120404095701.Horde.oqR4cpLnE6FPfFMtl5U1tdA@kishi.patrickdk.com> Message-ID: On 4.4.2012, at 16.57, Patrick Domack wrote: > Quoting Timo Sirainen : > >> On 4.4.2012, at 16.04, Patrick Westenberg wrote: >> >>> Timo Sirainen schrieb: >>> >>>> where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places. >>> >>> As far as I understand, lmtp.example.com with out records would result in a MX-lookup. If postfix behaves for lmtp like it does for smtp, it will try all listed MX (my directors) in turn. >> >> MX lookups are intended for SMTP, I very much doubt LMTP uses them? > > Yes, but postfix lmtp is the same binary as smtp, at least currently. That's about how to handle incoming lmtp vs. smtp connections. The outgoing smtp vs. lmtp connections are always handled by the same binary anyway. From zybi at talex.pl Wed Apr 4 19:09:50 2012 From: zybi at talex.pl (=?ISO-8859-2?Q?Artur_Zaprza=B3a?=) Date: Wed, 04 Apr 2012 18:09:50 +0200 Subject: [Dovecot] lmtp sometimes fails to deliver a message to all recipients In-Reply-To: <1B1369FF-6748-46DE-AB94-AF2F32061757@iki.fi> References: <4F79C995.9060607@talex.pl> <1B1369FF-6748-46DE-AB94-AF2F32061757@iki.fi> Message-ID: <4F7C724E.8040905@talex.pl> Timo Sirainen wrote: > On 2.4.2012, at 18.45, Artur Zaprza?a wrote: > >> Running strace on lmtp reveals that when lmtp delivers a message to multiple recipients it saves the message to a file in the first mailbox and hard links that file to other mailboxes. But when in the meantime of delivery that file is deleted, lmtp fails to deliver the message to remaining mailboxes. For example a message for 46 recipients is delivered by lmtp to the first 12 only (fortunately postfix retries delivery for remaining recipients) and for the rest lmtp logs the following error messages: >> >> lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Message was expunged (guid) >> lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: script /vmail/domain/foo/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /vmail/domain/foo/.dovecot.sieve.log may reveal additional details) > Fixed in hg. > Tested with Maildir. Works great. Thanks. The above problem was appearing when some recipients (including first one) had a sieve filter with discard action for current message. In this case, depending on the pattern of recipients having a sieve discard action, lmtp can create more than one instance of the message for a few dozen recipients. It would be nice if lmtp could create a single hardlinked instance of the message even in this case. Best regards, Artur Zaprza?a From property.of.mike.jones at gmail.com Wed Apr 4 20:52:27 2012 From: property.of.mike.jones at gmail.com (Mike Jones!) Date: Wed, 4 Apr 2012 13:52:27 -0400 Subject: [Dovecot] Listener not binding to port Message-ID: Hello. Dovecot is not binding or listening on port 993 when I start it. $ sudo netstat -tlnp | grep -c 993 0 Dovecot starts fine and runs without complaints. $ sudo service dovecot restart Restarting IMAP/POP3 mail server: dovecot. $ sudo tail /var/log/mail.log Apr 4 13:42:39 mwjones dovecot: master: Warning: Killed with signal 15 (by pid=16238 uid=0 code=kill) Apr 4 13:42:39 mwjones dovecot: master: Dovecot v2.0.18 starting up (core dumps disabled) $ sudo ps -ef | grep -i [d]ovecot root 16243 1 0 13:42 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf dovecot 16245 16243 0 13:42 ? 00:00:00 dovecot/anvil root 16246 16243 0 13:42 ? 00:00:00 dovecot/log root 16248 16243 0 13:42 ? 00:00:00 dovecot/config Still no listener :( $ sudo netstat -tlnp | grep -c 993 0 Other infos for your scrutiny. $ sudo dovecot --version 2.0.18 $ doveconf -n # 2.0.18: /etc/dovecot/dovecot.conf # OS: Linux 3.2.5-grsec x86_64 Debian wheezy/sid ext4 auth_debug = yes auth_mechanisms = plain login auth_verbose = yes first_valid_uid = 1000 log_timestamp = "%Y-%m-%d %H:%M:%S " mail_debug = yes mail_location = maildir:/home/vmail/%d/%n/Maildir passdb { args = /home/vmail/%d/etc/passwd driver = passwd-file } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } service imap-login { service_count = 1 } service ssl-params { type = startup } ssl_cert = References: <20120404131037.GB23371@state-of-mind.de> Message-ID: <6709704ac05c4ad9e121b73a76e5ae23@in.tum.de> On 2012-04-04 15:10, Patrick Ben Koetter wrote: > Or would I be able to modify this with a post-login script? Just one note: as far as I know, if your mail directory depends on a post-login script, it might be tricky to get LMTP or doveadm commands to work properly. Cheers, Christoph From eliezer at ngtech.co.il Thu Apr 5 06:44:18 2012 From: eliezer at ngtech.co.il (Eliezer Croitoru) Date: Thu, 05 Apr 2012 06:44:18 +0300 Subject: [Dovecot] Listener not binding to port In-Reply-To: References: Message-ID: <4F7D1512.5080102@ngtech.co.il> you should configure imap\pop3 services as far as i remeber... Regards, Eliezer On 04/04/2012 20:52, Mike Jones! wrote: > Hello. > > Dovecot is not binding or listening on port 993 when I start it. > > $ sudo netstat -tlnp | grep -c 993 > 0 > > Dovecot starts fine and runs without complaints. > > $ sudo service dovecot restart > Restarting IMAP/POP3 mail server: dovecot. > > $ sudo tail /var/log/mail.log > Apr 4 13:42:39 mwjones dovecot: master: Warning: Killed with signal > 15 (by pid=16238 uid=0 code=kill) > Apr 4 13:42:39 mwjones dovecot: master: Dovecot v2.0.18 starting up > (core dumps disabled) > > $ sudo ps -ef | grep -i [d]ovecot > root 16243 1 0 13:42 ? 00:00:00 /usr/sbin/dovecot -c > /etc/dovecot/dovecot.conf > dovecot 16245 16243 0 13:42 ? 00:00:00 dovecot/anvil > root 16246 16243 0 13:42 ? 00:00:00 dovecot/log > root 16248 16243 0 13:42 ? 00:00:00 dovecot/config > > Still no listener :( > > $ sudo netstat -tlnp | grep -c 993 > 0 > > Other infos for your scrutiny. > > $ sudo dovecot --version > 2.0.18 > > $ doveconf -n > # 2.0.18: /etc/dovecot/dovecot.conf > # OS: Linux 3.2.5-grsec x86_64 Debian wheezy/sid ext4 > auth_debug = yes > auth_mechanisms = plain login > auth_verbose = yes > first_valid_uid = 1000 > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_debug = yes > mail_location = maildir:/home/vmail/%d/%n/Maildir > passdb { > args = /home/vmail/%d/etc/passwd > driver = passwd-file > } > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > user = root > } > service imap-login { > service_count = 1 > } > service ssl-params { > type = startup > } > ssl_cert = ssl_key = userdb { > args = /home/vmail/%d/etc/passwd > driver = passwd-file > } > verbose_ssl = yes > > > What am I missing and how do I fix this? > > Thanks, > mwjones -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer ngtech.co.il From tss at iki.fi Thu Apr 5 09:08:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 5 Apr 2012 09:08:42 +0300 Subject: [Dovecot] Listener not binding to port In-Reply-To: References: Message-ID: <0EBD739C-2E2E-4CFB-B149-852AFD7279F4@iki.fi> On 4.4.2012, at 20.52, Mike Jones! wrote: > Dovecot is not binding or listening on port 993 when I start it. Try adding protocols=imap setting. From lists at wiesinger.com Thu Apr 5 09:16:33 2012 From: lists at wiesinger.com (Gerhard Wiesinger) Date: Thu, 5 Apr 2012 08:16:33 +0200 (CEST) Subject: [Dovecot] Dovecot upgrade from 1.2.x to 2.0.x: roundcube/squirrelmail sent folder doesn't work any more In-Reply-To: References: <4F71F8D8.6040700@wiesinger.com> <95C664AD-0532-4F3B-A2EB-2FD25B79187B@iki.fi> Message-ID: On Wed, 4 Apr 2012, Timo Sirainen wrote: > On 29.3.2012, at 8.25, Gerhard Wiesinger wrote: > >>>> 2.0.x: with Prefix ~/Mail >>>> A0003 LIST "" ~/Mail/sent >>>> A0003 OK List completed.: >>> >> >> mail_full_filesystem_access = yes >> mail_location = mbox:~:INBOX=/var/mail/%u > > Yes, the problem happens only with mail_full_filesystem_access=yes. It already works in v2.1, so I don't think I'll bother looking into why it's not working in v2.0. Hello Timo, As discussed per private mail you finally fixed it: http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e Thnx for your support. Ciao, Gerhard -- http://www.wiesinger.com/ From dvorak at cosign.jp Thu Apr 5 10:48:45 2012 From: dvorak at cosign.jp (Akihiko Sato) Date: Thu, 5 Apr 2012 16:48:45 +0900 Subject: [Dovecot] problem to configuration dovecot and postfix Message-ID: Hi, I'm having a problem with dovecot and postfix. I'm using a dovecot version 2.0.13. Postfix (Version: 2.8.5) config is changed according to the http://wiki2.dovecot.org/LDA/Postfix: I have thought to build a virtual user and virtual mailbox, and use the functionality of the Sieve. However, when the test set, the mail not received the mailbox. Record that there was a "temporary failure" in the log of Postfix. It delivered and to change the "virtual" to "dovecot", I think that's a matter of dovecot. To ensure that mail is delivered as intended, will How do I? Can anyone give me a hint what to do? # doveconf -n: # 2.0.13: /etc/dovecot/dovecot.conf # OS: Linux 3.0.0-12-server x86_64 Ubuntu 11.10 ext4 auth_debug = yes auth_debug_passwords = yes auth_verbose = yes disable_plaintext_auth = no info_log_path = /var/log/dovecot-info.log lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:/var/mail/vhosts/%d/%n managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables bo dy enotify environment mailbox date namespace { inbox = yes list = yes location = prefix = INBOX/ separator = / subscriptions = yes type = private } namespace { hidden = no list = yes location = maildir:/var/mail/shared/:INDEX=/var/mail/vhosts/%d/%n/shared/:CONTROL=/var/mail/vhosts/%d/%n/shared/ prefix = &UXFnCQ-/ separator = / subscriptions = yes type = public } passdb { args = /etc/auth/passwd.main driver = passwd-file } passdb { args = username_format=%n /etc/auth/%d/passwd driver = passwd-file } passdb { driver = pam } plugin { sieve = /var/mail/vhosts/%d/%n/_dovecot.sieve sieve_dir = /var/mail/vhosts/%d/%n/sieve/ } postmaster_address = postmaster at dom3.gs3 protocols = imap sieve service auth { user = root } ssl_cert = Apr 5 16:38:36 qbu postfix/cleanup[25500]: BDC3221212: message-id=<20120405073836.BDC3221212 at qbu> Apr 5 16:38:36 qbu postfix/qmgr[25471]: BDC3221212: from=, size=230, nrcpt=1 (queue active) Apr 5 16:38:36 qbu postfix/pipe[25502]: BDC3221212: to=, relay=dovecot, delay=0.04, delays=0.02/0/0/0.02, dsn=4.3.0, status=deferred (temporary failure) Regards, Akihiko From cor at xs4all.nl Thu Apr 5 11:15:09 2012 From: cor at xs4all.nl (Cor Bosman) Date: Thu, 5 Apr 2012 10:15:09 +0200 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F7C383A.2060401@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F760B61.3090209@wk-serv.de> <1333435450.21461.35.camel@innu> <4F7C32CF.7010908@wk-serv.de> <7BA0A5E9-091A-4FBD-B225-D458EBEC17FE@iki.fi> <4F7C383A.2060401@wk-serv.de> Message-ID: <20120405081509.GA1598@xs4all.nl> On Wed, Apr 04, 2012 at 02:02:02PM +0200, Patrick Westenberg wrote: > Timo Sirainen schrieb: > >> Another director. They're meant to connect to each others and do LB/HA. > > But what about my MTAs? How can I tell my two postfix servers that there > are two directors and it should/can use the other one if the > first is down? We use hardware loadbalancers in front of the directors. If a director goes down, the hardware loadbalancers sees that. We also use the poolmon script to tell the directors if a certain imap server is missing. Cor From tlx at leuxner.net Thu Apr 5 11:18:14 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 5 Apr 2012 10:18:14 +0200 Subject: [Dovecot] problem to configuration dovecot and postfix In-Reply-To: References: Message-ID: <20120405081814.GA7578@nihlus.leuxner.net> On Thu, Apr 05, 2012 at 04:48:45PM +0900, Akihiko Sato wrote: > # postconf -n: > mydestination = qbu.example.com, qbu, localhost.localdomain, localhost http://www.postfix.org/VIRTUAL_README.html [...] NEVER list a virtual MAILBOX domain name as a mydestination domain! > /var/log/mail.log: > Apr 5 16:38:36 qbu postfix/pickup[25470]: BDC3221212: uid=0 from= > Apr 5 16:38:36 qbu postfix/cleanup[25500]: BDC3221212: > message-id=<20120405073836.BDC3221212 at qbu> > Apr 5 16:38:36 qbu postfix/qmgr[25471]: BDC3221212: > from=, size=230, nrcpt=1 (queue active) > Apr 5 16:38:36 qbu postfix/pipe[25502]: BDC3221212: > to=, relay=dovecot, delay=0.04, delays=0.02/0/0/0.02, > dsn=4.3.0, status=deferred (temporary failure) Also with 2.x you may want to use LMTP rather than the LDA Piping. http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From p at state-of-mind.de Thu Apr 5 11:44:08 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Thu, 5 Apr 2012 10:44:08 +0200 Subject: [Dovecot] problem to configuration dovecot and postfix In-Reply-To: <20120405081814.GA7578@nihlus.leuxner.net> References: <20120405081814.GA7578@nihlus.leuxner.net> Message-ID: <20120405084408.GB10931@state-of-mind.de> * Thomas Leuxner : > On Thu, Apr 05, 2012 at 04:48:45PM +0900, Akihiko Sato wrote: > > > # postconf -n: > > mydestination = qbu.example.com, qbu, localhost.localdomain, localhost > > http://www.postfix.org/VIRTUAL_README.html > > [...] > NEVER list a virtual MAILBOX domain name as a mydestination domain! ... unless you know what you do and wish to benefit from the side-effects. p at rick -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 316 bytes Desc: Digital signature URL: From dvorak at cosign.jp Thu Apr 5 11:54:13 2012 From: dvorak at cosign.jp (Akihiko Sato) Date: Thu, 5 Apr 2012 17:54:13 +0900 Subject: [Dovecot] problem to configuration dovecot and postfix In-Reply-To: <20120405081814.GA7578@nihlus.leuxner.net> References: <20120405081814.GA7578@nihlus.leuxner.net> Message-ID: 2012/4/5 Thomas Leuxner : > On Thu, Apr 05, 2012 at 04:48:45PM +0900, Akihiko Sato wrote: > >> # postconf -n: >> mydestination = qbu.example.com, qbu, localhost.localdomain, localhost > > http://www.postfix.org/VIRTUAL_README.html > > [...] > NEVER list a virtual MAILBOX domain name as a mydestination domain! Oops, this is my careless mistake...mydestination is empty. But status does not change. Regards, Akihiko > > Also with 2.x you may want to use LMTP rather than the LDA Piping. > > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP > > Regards > Thomas > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > > iEYEARECAAYFAk99VUYACgkQ+meF/S97aXi/4wCgkLGKNByWuxYb0E7YIdZceHHU > UwQAnArESXtRggXYRbnk64WaVfItfAY9 > =N8Aq > -----END PGP SIGNATURE----- > From tlx at leuxner.net Thu Apr 5 12:31:20 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 5 Apr 2012 11:31:20 +0200 Subject: [Dovecot] problem to configuration dovecot and postfix In-Reply-To: References: <20120405081814.GA7578@nihlus.leuxner.net> Message-ID: <20120405093120.GB7578@nihlus.leuxner.net> On Thu, Apr 05, 2012 at 05:54:13PM +0900, Akihiko Sato wrote: > Oops, this is my careless mistake...mydestination is empty. > But status does not change. > > Regards, > Akihiko Hmm. Haven't used LDA since LMTP was introduced. Are there corresponding log entries on the Dovecot side? [...] protocol lda { postmaster_address = postmaster at domain.tld log_path = /var/log/dovecot-deliver.log info_log_path = /var/log/dovecot-deliver.log } Still would go the LMTP route rather than using Pipes: > > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP > > Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From dvorak at cosign.jp Thu Apr 5 13:10:07 2012 From: dvorak at cosign.jp (Akihiko Sato) Date: Thu, 5 Apr 2012 19:10:07 +0900 Subject: [Dovecot] problem to configuration dovecot and postfix In-Reply-To: <20120405093120.GB7578@nihlus.leuxner.net> References: <20120405081814.GA7578@nihlus.leuxner.net> <20120405093120.GB7578@nihlus.leuxner.net> Message-ID: 2012/4/5 Thomas Leuxner : > On Thu, Apr 05, 2012 at 05:54:13PM +0900, Akihiko Sato wrote: >> Oops, this is my careless mistake...mydestination is empty. >> But status does not change. >> >> Regards, >> Akihiko > > Hmm. Haven't used LDA since LMTP was introduced. Are there corresponding > log entries on the Dovecot side? > > [...] > protocol lda { > ? ? ? ?postmaster_address = postmaster at domain.tld > ? ? ? ?log_path = /var/log/dovecot-deliver.log > ? ? ? ?info_log_path = /var/log/dovecot-deliver.log > } I tried to restart the dovecot put these settings, the log was not generated. Existing log (dovecot.log, dovecot-info.log) that an error is also recorded in the was not. ;-( In this case, is it better to use LMTP? > > Still would go the LMTP route rather than using Pipes: > >> > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP >> > > > Thomas > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > > iEYEARECAAYFAk99ZmgACgkQ+meF/S97aXh1/ACgo/VaG3SP587MvPDwkXGRn/lG > o08An1nKrVecLm6yS9HuG60PULJgw0I3 > =hUSv > -----END PGP SIGNATURE----- > Regards, Akihiko From busseniu at in.tum.de Thu Apr 5 13:36:03 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Thu, 05 Apr 2012 12:36:03 +0200 Subject: [Dovecot] Dovecot allows creation of folders outside of a user's directory In-Reply-To: References: <4F759B08.1060603@in.tum.de> Message-ID: <4F7D7593.2090005@in.tum.de> On 04.04.2012 03:35, Timo Sirainen wrote: > On 30.3.2012, at 14.37, Christoph Bu?enius wrote: >> -- even though it will fail with a "permission denied" error -- will create a directory like "/mail/users/strange&ANY- characters". That directory will only contain a subdirectory "Maildir" and therein dovecot-acl-list. > > Fixed: http://hg.dovecot.org/dovecot-2.0/rev/b15889b82258 > Thanks, this fixed it. By the way, your tireless work on Dovecot is amazing :) -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From CMarcus at Media-Brokers.com Thu Apr 5 15:59:22 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 05 Apr 2012 08:59:22 -0400 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header Message-ID: <4F7D972A.8050406@Media-Brokers.com> On 2012-04-05 4:18 AM, Thomas Leuxner wrote: > Also with 2.x you may want to use LMTP rather than the LDA Piping. > > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP I am preparing to convert my main client's postfix_courier-imap setup to dovecot 2.1, which currently just uses the postfix virtual delivery agent... Does anyone know if the use of LMTP (or even the dovecot LDA) still loses the x-original-to header that the postfix vda adds and that I rely heavily on (since I use a lot of aliases), and if it does, is there any solution to get the original recipient added back in before final delivery? Everything I'm reading says that LMTP is better, but I really do need this header (or one like it) to be there so I know who the original recipient was (for filtering and other purposes). Thanks, -- Best regards, Charles From l.chelchowski at eurocar.pl Thu Apr 5 16:29:13 2012 From: l.chelchowski at eurocar.pl (l.chelchowski at eurocar.pl) Date: Thu, 05 Apr 2012 15:29:13 +0200 Subject: [Dovecot] Problem with polish characters and sieve fileinto Message-ID: <4F7D9E29.3050802@eurocar.pl> Hi, I have problems with sieve fileinto and polish characters in folder names: # rule:[??????????????????] elsif anyof (header :contains "Subject" "??????????????????") { fileinto "INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-"; } dovecot log: Apr 5 15:17:42 mailer dovecot: lmtp(54203, test at test1.com): rmHVJHabfU+70wAA9hAjDg: sieve: msgid=<4F7D9B68.4030206 at test2.com>: stored mail into mailbox 'INBOX' Apr 5 15:17:42 mailer dovecot: lmtp(54203, test at test1.com): Error: rmHVJHabfU+70wAA9hAjDg: sieve: execution of script /home/vmail/test1.com/test//.dovecot.sieve failed, but implicit keep was successful (user logfile /home/vmail/test1.com/test//.dovecot.sieve.log may reveal additional details) Apr 5 15:17:42 mailer postfix/lmtp[54202]: 941B1E2681F: to=, relay=m.test1.com[private/dovecot-lmtp], delay=0.04, delays=0/0.01/0/0.03, dsn=2.0.0, status=sent (250 2.0.0 rmHVJHabfU+70wAA9hAjDg Saved) dovecot.sieve.log: sieve: info: started log at Apr 05 15:17:42. error: msgid=<4F7D9B68.4030206 at test2.com>: failed to store into mailbox 'INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-' (INBOX/&-AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-): Mailbox doesn't exist: INBOX.&-AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-. Thanks for help. -- Pozdrawiam ?ukasz From zybi at talex.pl Thu Apr 5 16:45:24 2012 From: zybi at talex.pl (=?UTF-8?B?QXJ0dXIgWmFwcnphxYJh?=) Date: Thu, 05 Apr 2012 15:45:24 +0200 Subject: [Dovecot] Problem with polish characters and sieve fileinto In-Reply-To: <4F7D9E29.3050802@eurocar.pl> References: <4F7D9E29.3050802@eurocar.pl> Message-ID: <4F7DA1F4.40207@talex.pl> l.chelchowski at eurocar.pl wrote: > Hi, > > I have problems with sieve fileinto and polish characters in folder names: > # rule:[??????????????????] > elsif anyof (header :contains "Subject" "??????????????????") > { > fileinto "INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-"; > } > Sieve script expects UTF-8 folder names. This should work: fileinto "INBOX/??????????????????"; You can also use UTF-8 mailbox names on disk if you add UTF-8 option to mail_location: mail_location = maildir:~/Maildir:UTF-8 but you have to convert existing names to UTF-8 somehow. > > dovecot log: > Apr 5 15:17:42 mailer dovecot: lmtp(54203, test at test1.com): > rmHVJHabfU+70wAA9hAjDg: sieve: msgid=<4F7D9B68.4030206 at test2.com>: stored mail > into mailbox 'INBOX' > Apr 5 15:17:42 mailer dovecot: lmtp(54203, test at test1.com): Error: > rmHVJHabfU+70wAA9hAjDg: sieve: execution of script > /home/vmail/test1.com/test//.dovecot.sieve failed, but implicit keep was > successful (user logfile /home/vmail/test1.com/test//.dovecot.sieve.log may > reveal additional details) > Apr 5 15:17:42 mailer postfix/lmtp[54202]: 941B1E2681F: to=, > relay=m.test1.com[private/dovecot-lmtp], delay=0.04, delays=0/0.01/0/0.03, > dsn=2.0.0, status=sent (250 2.0.0 rmHVJHabfU+70wAA9hAjDg Saved) > > > dovecot.sieve.log: > sieve: info: started log at Apr 05 15:17:42. > error: msgid=<4F7D9B68.4030206 at test2.com>: failed to store into mailbox > 'INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-' > (INBOX/&-AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-): Mailbox doesn't > exist: INBOX.&-AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-. > > Thanks for help. -- Talex Sp??ka Akcyjna z siedzib? w Poznaniu adres: ul. Karpia 27d, 61-619 Pozna? NIP 782-00-21-045 zarejestrowana w S?dzie Rejonowym Pozna? ? Nowe Miasto i Wilda w Poznaniu VIII Wydzia? Gospodarczy - KRS pod nr 000048779 kapita? zak?adowy: 3.000.092,00 PLN (w ca?o?ci wp?acony) Uwaga: Niniejsza wiadomo??, w szczeg?lno?ci jej tre?? oraz za??czniki, mo?e by? poufna. W przypadku, gdy nie jest Pan/Pani zamierzonym jej adresatem, informujemy, ?e wszelkie rozpowszechnianie, dystrybucja lub powielanie powy?szej wiadomo?ci jest zabronione. Jednocze?nie prosimy o powiadomienie nadawcy oraz niezw?oczne usuni?cie powy?szej wiadomo?ci wraz z za??cznikami. Dzi?kujemy, Talex S.A. w Poznaniu. Confidentiality Notice: This email, particularly its content and any attached files, may be confidential. If you are not an intended recipient, any disclosure, distribution and reproduction of this message is prohibited. In this case please notify the sender immediately and then delete this message and any attachments. Thank you, Talex S.A., Poznan. From stephan at rename-it.nl Thu Apr 5 16:47:16 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 05 Apr 2012 15:47:16 +0200 Subject: [Dovecot] Problem with polish characters and sieve fileinto In-Reply-To: <4F7D9E29.3050802@eurocar.pl> References: <4F7D9E29.3050802@eurocar.pl> Message-ID: <4F7DA264.8020004@rename-it.nl> Op 4/5/2012 3:29 PM, l.chelchowski at eurocar.pl schreef: > Hi, > > I have problems with sieve fileinto and polish characters in folder > names: > # rule:[??????????????????] > elsif anyof (header :contains "Subject" "??????????????????") > { > fileinto "INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-"; > } Looks like your Sieve GUI uses the wrong Unicode format. The above is mUTF7 as used by IMAP. However, Sieve expects UTF-8. So it should use this instead: fileinto "INBOX/??????????????????"; Regards, Stephan. From xnasx at yandex.ru Thu Apr 5 16:51:32 2012 From: xnasx at yandex.ru (Victor) Date: Thu, 05 Apr 2012 17:51:32 +0400 Subject: [Dovecot] Username bug in '%%' variables selects Message-ID: <10301333633892@web94.yandex.ru> Hi! Forgive my bad English. I trying to set up shared folders, but I can't. I shared INBOX of 'victor at domain.com' for 'victor at testmail.domain.com' There is my debugging results: doveadm acl debug -u victor at testmail.domain.com INBOX.shared.victor at domain.com. doveadm(victor at testmail.domain.com): Error: Couldn't create namespace 'INBOX.shared.' for user victor at domain: userdb didn't return a home directory, but mail_location used it (%h): maildir:%h That's I seen in logs: auth-worker: Debug: sql(victor at domain): SELECT CONCAT('/var/spool/mail/', SUBSTRING_INDEX(email, '@', -1), '/', SUBSTRING_INDEX(email, '@', 1)) as home from `users` WHERE email='victor at domain' What happened with my username? 'victor at domain' is not correct value. My config: namespace { inbox = yes location = prefix = INBOX. separator = . type = private } namespace { hidden = no list = yes location = maildir:%%h prefix = INBOX.shared.%%u. separator = . subscriptions = no type = shared } password_query = \ SELECT email as username, password \ FROM users WHERE email = '%u' user_query = SELECT CONCAT('/var/spool/mail/', SUBSTRING_INDEX(email, '@', -1), '/', SUBSTRING_INDEX(email, '@', 1)) as home from `users` WHERE email='%u' Please, help me! -- Best regards, Victor From property.of.mike.jones at gmail.com Thu Apr 5 18:08:59 2012 From: property.of.mike.jones at gmail.com (Mike Jones!) Date: Thu, 5 Apr 2012 11:08:59 -0400 Subject: [Dovecot] Listener not binding to port In-Reply-To: <0EBD739C-2E2E-4CFB-B149-852AFD7279F4@iki.fi> References: <0EBD739C-2E2E-4CFB-B149-852AFD7279F4@iki.fi> Message-ID: Thanks so much all. I was missing a few items: protocols = imap # thanks to Timo Sirainen # thanks to Noel Butler service imap-login { inet_listener imap { port = 0 } inet_listener imaps { address = * port = 993 } service_count = 1 } Here is my final doveconf -n with listener successfully running on 993: # 2.0.18: /etc/dovecot/dovecot.conf # OS: Linux 3.2.5-grsec x86_64 Debian wheezy/sid ext4 auth_debug = yes auth_mechanisms = plain login auth_verbose = yes first_valid_gid = 1030 first_valid_uid = 1030 info_log_path = /var/log/dovecot-info.log last_valid_gid = 1030 last_valid_uid = 1030 log_path = /var/log/dovecot.log log_timestamp = "%Y-%m-%d %H:%M:%S " mail_debug = yes mail_location = maildir:/home/vmail/%d/%n/Maildir passdb { args = /home/vmail/%d/etc/passwd driver = passwd-file } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { address = * port = 993 } service_count = 1 } service ssl-params { type = startup } ssl_cert = Hi all, I'm planning on implementing this in my new upcoming dovecot instance, and would like to hear thoughts on how best to accomplish this. We will be paying Timo's support company to do the work, but obviously, the less work in the form of coding he has to do to get this working (I'm hoping it won't be a lot), the more money it will save us... ;) First - I currently use rsnapshot to backup emails, so that is the use-case I'm most interested in getting working. It is rsync based, and like other rsync based backup programs it uses hardlinks to save storage space - so you can have a *lot* of backups (going back months, or even years), where each snapshot only adds a little more to the total disk space being used. The snapshots are stored with the following filesystem layout: /path/to/snapshotsdir/hourly.0 ... /path/to/snapshotsdir/hourly.4 /path/to/snapshotsdir/daily.0 ... /path/to/snapshotsdir/daily.7 /path/to/snapshotsdir/weekly.0 ... /path/to/snapshotsdir/weekly.4 /path/to/snapshotsdir/monthly.0 ... /path/to/snapshotsdir/monthly.12 /path/to/snapshotsdir/yearly.0 ... /path/to/snapshotsdir/yearly.5 The 'names' (hourly, daily, weekly, monthly, yearly) are arbitrary (this is a bit confusing to people new to rsnapshot), and would *not* be used for displaying the mail folders to the users - it is the Date/Time stamps of each of the snapshot dirs above that would be used to display the folder names under the 'Time Machine' namespace. This is, I imagine, the part that will need some actual coding by Timo to get working - maybe just some new config variables added to the namespace code for mapping the date/time stamps of the directories to user friendly folder names in the namespace. That said, I'd like to design this and have it coded such that it will work with almost any type of backup storage that stores snapshots as date/time stamped directories like this (there must be others, right?). Also, it goes without saying that this code will be (if Timo is ok with it) part of the core dovecot code going forward, so anyone else will be able to benefit from it. What I'm envisioning is something like this... 1. Define a namespace - for this example we'll call it 'Time Machine' 2. Under this namespace, each user will see their, and *only* their snapshots So, each user would see something like this: My Mail Account Inbox Drafts Templates Sent Time Machine (sorted above user created folders if possible) -4/3/12, 8:00am (first subfolder) Inbox Drafts etc... (all other folders and sub-folders shown here) +4/3/12, 12:00pm (first subfolder) etc... Other User Folders ... Or even better, I'm thinking some magical code that can group them by Date, like: -4/3/12 (first subfolder) -8:00am (next sub-folder) Inbox Drafts Etc... (all folders and sub-folders shown here) +12:00pm +4:00pm +8:00pm +4/4/12 etc... Comments? Suggestions? Flames? -- Best regards, Charles From tom at whyscream.net Thu Apr 5 19:37:30 2012 From: tom at whyscream.net (Tom Hendrikx) Date: Thu, 05 Apr 2012 18:37:30 +0200 Subject: [Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups In-Reply-To: <4F7DBA3B.5090803@Media-Brokers.com> References: <4F7DBA3B.5090803@Media-Brokers.com> Message-ID: <4F7DCA4A.70009@whyscream.net> On 05-04-12 17:28, Charles Marcus wrote: > Hi all, > > I'm planning on implementing this in my new upcoming dovecot instance, > and would like to hear thoughts on how best to accomplish this. We will > be paying Timo's support company to do the work, but obviously, the less > work in the form of coding he has to do to get this working (I'm hoping > it won't be a lot), the more money it will save us... ;) > > First - I currently use rsnapshot to backup emails, so that is the > use-case I'm most interested in getting working. It is rsync based, and > like other rsync based backup programs it uses hardlinks to save storage > space - so you can have a *lot* of backups (going back months, or even > years), where each snapshot only adds a little more to the total disk > space being used. > > What I'm envisioning is something like this... > > 1. Define a namespace - for this example we'll call it 'Time Machine' > > 2. Under this namespace, each user will see their, and *only* their > snapshots > > So, each user would see something like this: > > My Mail Account > Inbox > Drafts > Templates > Sent > Time Machine (sorted above user created folders if possible) > -4/3/12, 8:00am (first subfolder) > Inbox > Drafts > etc... (all other folders and sub-folders shown here) > +4/3/12, 12:00pm (first subfolder) > etc... > Other User Folders > ... > > Or even better, I'm thinking some magical code that can group them by > Date, like: > > -4/3/12 (first subfolder) > -8:00am (next sub-folder) > Inbox > Drafts > Etc... (all folders and sub-folders shown here) > +12:00pm > +4:00pm > +8:00pm > +4/4/12 > etc... > > Comments? Suggestions? Flames? > The first interesting point I'd see with this, is that you supply the mail client with a near endless supply of folders, which would take a lot of caching space on the clients end, either (depending on the client and its configuration) from the moment that you enable this fort hem, or after someone starts searching in their 'time machine' for some old mail. I see my mail client on a new install working quite hard to download mail headers for 2 years of postfix/dovecot/etc mailing lists, so what happens if you provide a 'time machine' namespace going 1 month back, 4 with snapshots a day (i.e. 31x4 =~ 120 times more headers to download/index). -- Tom From CMarcus at Media-Brokers.com Thu Apr 5 20:02:51 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 05 Apr 2012 13:02:51 -0400 Subject: [Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups In-Reply-To: <4F7DCA4A.70009@whyscream.net> References: <4F7DBA3B.5090803@Media-Brokers.com> <4F7DCA4A.70009@whyscream.net> Message-ID: <4F7DD03B.30909@Media-Brokers.com> On 2012-04-05 12:37 PM, Tom Hendrikx wrote: > The first interesting point I'd see with this, is that you supply the > mail client with a near endless supply of folders, which would take a > lot of caching space on the clients end, either (depending on the client > and its configuration) from the moment that you enable this fort hem, or > after someone starts searching in their 'time machine' for some old mail. > > I see my mail client on a new install working quite hard to download > mail headers for 2 years of postfix/dovecot/etc mailing lists, so what > happens if you provide a 'time machine' namespace going 1 month back, 4 > with snapshots a day (i.e. 31x4 =~ 120 times more headers to > download/index). Interesting and valid point... hmmmm..... First, these folders would be read-only - a user could copy something from there back to one of his other folders, but couldn't write anything in them - so nothing would be changing under this namespace, except new snapshots magically appearing, which means that once they are indexed, the indexes would never need to be rebuilt (unless they got corrupted somehow). But, yeah, I can imagine some problems especially if someone has a ton of email. And while these would probably only be accessed rarely, in those cases where someone would want to access them, they would very likely want to be able to search, so disabling indexes wouldn't be a good idea... Since we use Thunderbird, I can of course disable offline mode for everyone, so the only time headers would be downloaded would be when the user selects (or performs a search on) one (or more) of the folders. Maybe Timo can think of something creative to minimize this problem... -- Best regards, Charles From zarf at klacto.net Thu Apr 5 20:37:37 2012 From: zarf at klacto.net (FZiegler) Date: Thu, 05 Apr 2012 13:37:37 -0400 Subject: [Dovecot] dovecot and unison In-Reply-To: <8EAB42AF-BC4A-4235-9CF9-A444BE553B10@odo.in-berlin.de> References: <4F7A2FF9.7000204@klacto.net> <8EAB42AF-BC4A-4235-9CF9-A444BE553B10@odo.in-berlin.de> Message-ID: <4F7DD861.7060303@klacto.net> Michael Grimm wrote: >> Namely, which of the ancillary files should I be syncing (or perhaps >> deleting before sync)? > I used ... > > ignore = Name dovecot* > ignore = Regex .*/tmp$ > ignore = Regex .*/\.Drafts$ > > ? to ignore all of them. Thanks Michael for this (and thanks also mazieres at stanford for your reply). I am now merrily ignoring these files too, and synced 400 MB of mail without seeing the disappearing subfolder issue again. Francois Z. From hsn at filez.com Thu Apr 5 21:46:50 2012 From: hsn at filez.com (Radim Kolar) Date: Thu, 05 Apr 2012 20:46:50 +0200 Subject: [Dovecot] create mailboxes in /var/mail via doveadm Message-ID: <4F7DE89A.4020409@filez.com> what usedid is used by doveadm for creating mailboxes? can it be setup to run as dovecot:mail? cant create mailbox from command line ponto# ls -ld /var/mail drwxrwxr-x 8 root mail 24 Apr 5 20:35 /var/mail ponto# doveadm mailbox create -u jboss INBOX doveadm(jboss): Error: Can't create mailbox INBOX: Permission denied mail_location = maildir:%h/Maildir:INBOX=/var/mail/%u From slusarz at curecanti.org Thu Apr 5 21:59:20 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Thu, 05 Apr 2012 12:59:20 -0600 Subject: [Dovecot] 2.1.3: Overly lax FETCH parsing Message-ID: <20120405125920.Horde.wazxEIF5lbhPfeuIm_gQ4uA@bigworm.curecanti.org> These commands both work in Dovecot (as of 2.1.3): 1 UID FETCH 1:* () (CHANGEDSINCE 1) * 1 FETCH (UID 30097 MODSEQ (19554)) [...] 1 OK Fetch completed. 2 FETCH 1:* () (CHANGEDSINCE 1) * 2 FETCH (MODSEQ (19554)) [...] 2 OK Fetch completed. However, RFC 3501 indicates that an empty FETCH attribute list is incorrect: fetch = "FETCH" SP sequence-set SP ("ALL" / "FULL" / "FAST" / fetch-att / "(" fetch-att *(SP fetch-att) ")") fetch-att = "ENVELOPE" / "FLAGS" / "INTERNALDATE" / "RFC822" [".HEADER" / ".SIZE" / ".TEXT"] / "BODY" ["STRUCTURE"] / "UID" / "BODY" section ["<" number "." nz-number ">"] / "BODY.PEEK" section ["<" number "." nz-number ">"] In other words, fetch-att is not allowed to be empty. (RFC 4466 does not alter this requirement when using FETCH modifiers). While useful that Dovecot is more liberal about what it receives, 3501 seems pretty clear that incorrect FETCH parameters must return a BAD. I can verify that the above commands fail on Cyrus. I don't disagree that the way Dovecot handles this is correct - or at least is the way that I wish the RFCs were written. Since this seems a bit redundant, but is necessary under the ABNF: 1 UID FETCH 1:* UID (CHANGEDSINCE 1) -or- 1 UID FETCH 1:* (UID) (CHANGEDSINCE 1) michael From centos.admin at gmail.com Thu Apr 5 23:02:17 2012 From: centos.admin at gmail.com (Emmanuel Noobadmin) Date: Fri, 6 Apr 2012 04:02:17 +0800 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? Message-ID: I'm trying to improve the setup of our Dovecot/Exim mail servers to handle the increasingly huge accounts (everybody thinks it's like infinitely growing storage like gmail and stores everything forever in their email accounts) by changing from Maildir to mdbox, and to take advantage of offloading older emails to alternative networked storage nodes. The question now is whether having a single large server or will a number of 1U servers with the same total capacity be better? Will be using RAID 1 pairs, likely XFS based on reading Hoeppner's recommendation on this and the mdadm list. Currently, I'm leaning towards multiple small servers because I think it should be better in terms of performance. At the very least even if one node gets jammed up, the rest should still be able serve up the emails for other accounts that is unless Dovecot will get locked up by that jammed transaction. Also, I could possibly arrange them in a sort of network raid 1 to gain redundancy over single machine failure. Would I be correct in these or do actual experiences say otherwise? From bruce.hammond at contractor.verio.net Fri Apr 6 01:05:53 2012 From: bruce.hammond at contractor.verio.net (Bruce Hammond) Date: Thu, 5 Apr 2012 18:05:53 -0400 Subject: [Dovecot] dovecot: auth: Error: ldap(...) failed: No such object Message-ID: <4803B289543E1244868D9FF82F670AA002E999AD@IAD-WPRD-XCHB01.corp.verio.net> I just installed the 2.1.3 version of dovecot. When I telnet to my dovecot linux server on the imap port (143) and try to login I get the error "No [UNAVAILABLE] Temporary authentication failure" on the telnet display and dovecot: auth: Error: ldap(bhammond_test at 131.103.212.38,128.121.121.193): ldap_search(base=uid=bhammond_test at 131.103.212.38,ou=users,dc=aspect filter=(&(objectClass=posixAccount)(productFeature=mail))) failed: No such object in the /var/log/maillog. I am not familiar with ldap and I haven't been able to find anything using google. Anyone have any ideas with the problem? Thanks. Here is my configuration: # 2.1.3: /usr/local/packages/dovecot2/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-274.3.1.el5.secure i686 Red Hat Enterprise Linux Server release 5.7 (Tikanga) auth_mechanisms = plain login auth_verbose = yes default_login_user = dovecot disable_plaintext_auth = no listen = * mail_location = maildir:~/mail passdb { args = /etc/devel/passdb.conf driver = ldap } pop3_lock_session = yes protocols = imap pop3 service auth { user = dovecot } service imap-login { process_limit = 128 process_min_avail = 30 service_count = 0 user = dovecot vsz_limit = 64 M } service pop3-login { process_limit = 128 process_min_avail = 30 service_count = 0 user = dovecot vsz_limit = 64 M } shutdown_clients = no ssl_cert = References: Message-ID: A timely topic as I was just mulling over ways to provide this to my users. Charles Marcus wrote: > The snapshots are stored with the following filesystem layout: > > /path/to/snapshotsdir/hourly.0 > ... This is familiar to NetApp users. > The 'names' (hourly, daily, weekly, monthly, yearly) are arbitrary (this > is a bit confusing to people new to rsnapshot), and would *not* be used > for displaying the mail folders to the users - it is the Date/Time > stamps of each of the snapshot dirs above that would be used to display > the folder names under the 'Time Machine' namespace. This is, I imagine, > the part that will need some actual coding by Timo to get working - > maybe just some new config variables added to the namespace code for > mapping the date/time stamps of the directories to user friendly folder > names in the namespace. > > That said, I'd like to design this and have it coded such that it will > work with almost any type of backup storage that stores snapshots as > date/time stamped directories like this (there must be others, right?). One idea is to take this complexity entirely out of dovecot and create a synthetic filesystem using hard or soft links (as rsnapshot has done) and create your own, with whatever weird and wonderful naming scheme you want. /path/to/TimeMachine// -> /path/to/snapshotsdir// namespace { prefix = TimeMachine location = maildir:/path/to/TimeMachine:INDEX=MEMORY ... } This might not be very scalable depending on how big your userbase is. I would probably define memory indices for this namespace, and take the performance hit on the assumption that access will be a once-in-a- while thing. On-disk indices will probably get out of date with each snapshot rollover, if if you have a lot of snapshots/mailboxes, it could consume a non-trivial amount of space without a lot of benefits. Or you could run a cron script to rename or remove old indices, but that seems more trouble than it's worth. One other consideration (at least for me) is if the INBOX and personal mail folders are stored in two separate FS's. It would be nice to fuse the two sets of backups under the same namespace, but I don't know how the namespace prefix matching works and whether you can define hierarchical namespaces like namespace { prefix = backup/inbox location = mbox:/path/to/inbox-snapdir/%u ... } namespace { prefix = backup/mail location = mbox:/path/to/mail-snapdir/%u ... } The above can also be accomplished with a synthetic filesystem. Joseph Tam From fxmulder at gmail.com Fri Apr 6 04:16:10 2012 From: fxmulder at gmail.com (James Devine) Date: Thu, 5 Apr 2012 19:16:10 -0600 Subject: [Dovecot] distributed mdbox In-Reply-To: <1ff190f691892c01a980927422f38753@mail.airstreamcomm.net> References: <1ff190f691892c01a980927422f38753@mail.airstreamcomm.net> Message-ID: On Fri, Mar 23, 2012 at 7:39 AM, wrote: > On Wed, 21 Mar 2012 09:56:12 -0600, James Devine > wrote: > > Anyone know how to setup dovecot with mdbox so that it can be used > through > > shared storage from multiple hosts? I've setup a gluster volume and am > > sharing it between 2 test clients. I'm using postfix/dovecot LDA for > > delivery and I'm using postal to send mail between 40 users. In doing > > this, I'm seeing these errors in the logs > > > > Mar 21 09:36:29 test-gluster-client2 dovecot: lda(testuser34): Error: > Fixed > > index file /mnt/testuser34/mdbox/storage/dovecot.map.index: > messages_count > > 272 -> 271 > > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: > Log > > synchronization error at seq=4,offset=3768 for > > /mnt/testuser28/mdbox/storage/dovecot.map.index: Append with UID 516, > but > > next_uid = 517 > > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: > Log > > synchronization error at seq=4,offset=4220 for > > /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update > > for invalid uid=517 > > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: > Log > > synchronization error at seq=4,offset=5088 for > > /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update > > for invalid uid=517 > > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Warning: > > fscking index file /mnt/testuser28/mdbox/storage/dovecot.map.index > > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser34): Warning: > > fscking index file /mnt/testuser34/mdbox/storage/dovecot.map.index > > > > > > This is my dovecot config currently: > > > > jdevine at test-gluster-client2:~> dovecot -n > > # 2.0.13: /etc/dovecot/dovecot.conf > > # OS: Linux 3.0.0-13-server x86_64 Ubuntu 11.10 > > lock_method = dotlock > > mail_fsync = always > > mail_location = mdbox:~/mdbox > > mail_nfs_index = yes > > mail_nfs_storage = yes > > mmap_disable = yes > > passdb { > > driver = pam > > } > > protocols = " imap" > > ssl_cert = > ssl_key = > userdb { > > driver = passwd > > } > > I was able to get dovecot working across a gluster cluster a few weeks ago > and it worked just fine. I would recommend using the native gluster mount > option (need to install gluster software on clients), and using distributed > replicated as your replication mechanism. If you're running two gluster > servers you should have a replica count of two with distributed replicated. > You should test first to make sure you can create a file in both mounts > and see it from every mount point in the cluster, as well as interact with > it. It's also very important to make sure your servers are running with > synchronized clocks from an NTP server. Very bad things happen to a > (dovecot or gluster) cluster out of sync with NTP. > > What storage method are you using? I'm able to produce errors within seconds of starting postal with more than one thread From thomasvoneyben at gmail.com Fri Apr 6 12:52:34 2012 From: thomasvoneyben at gmail.com (Thomas von Eyben) Date: Fri, 6 Apr 2012 11:52:34 +0200 Subject: [Dovecot] Outlook (2010) -> Dovecot (IMAP) >10x slower with high network load and many folders Message-ID: Hi, I am seeing a >10x as slow performance when trying to complete a "send/receive" from an Outlook 2010 client to Dovecot via IMAP, but only when the LAN is fully loaded with other traffic, EG file copying. It seems the problem is when outlook is trying to identify folders that have changed since last "send/receive" thus traversing the hierachy. Observations: - Apple's Mail.app does not have problems when exposed to the same environment. - Eliminating Outlook is not an option. - The tests have been performed during the easter vacation, thus almost no other clients are using the network/mailserver. Under normal load there are 2-300 users using the network. Description of the environment: Server sw: Mac OS X Server 10.6.8 running dovecot: 1.1.20apple0.5 (OS = fully updated 10.6.x) Server hw: Xserve Quadcore intel Xeon 2.26 GHz 12 GB RAM - Never having a cpu load > 20%, mail is stored on a Promise vtrack RAID connected via Fibre Channel Client: Windows 7, Outlook 2010 The client have maybe 50 folders (and each folder about a handfull of subfolders) the size of the mailbox is around 3-5 GB. Test results: CLIENT-1 is having the problems when CLIENT-2 is using all the (100Mbps) bandwidth eg. copying files to MAIL-SRV. If I move CLIENT-1 to CLIENT-3 then almost all the delay is gone. NB.: I have not (yet) tested if the problem also exists when CLIENT-2 generates traffic to MAIL-SRV as opposed to OTHER-SRV (but I am expecting the same problems). When dumping the traffic on CLIENT-1 (with Wireshark) one thing is catching my eyes for the 'bad' case: There is a very long delay after each "REQUEST IDLE" until the next "REQUEST DONE" ca. 0,3 seconds. Comparing to the 'good' setup the pause at the same place in the communication is less than 1 ms ! So why this delay? Where/how shall I continue my debugging? - Run dtruss/dtrace scripts on the server? - Get dovecot to output more debug info (I guess it's doing it's best so no problems will be seen there?)? Are there any dovecot configurations that can be altered to workaround [what I think is a limitation in Outlook's IMAP implementation]? The current dovecot configuration is Apple's defaults (+POP3 disabled)) I am seing one warning from dovecotd -n though: - But I see the same warning on a clean installed not-yet-configured OS X Server so I guess it's not 'that' bad: Warning: fd limit 256 is lower than what Dovecot can use under full load (more than 456). Either grow the limit or change login_max_processes_count and max_mail_processes settings Physical setup: +---------------+ ?+------------------------------+ ? ?+-----------------+ | CLIENT-1 ? ? ?| ?| CLIENT-2 ? ? ? ? ? ? ? ? ? ? | ? ?| CLIENT-3 ? ? ? ?| | Outlook "bad" | ?| Traffic generator, eg. Samba | ? ?| Outlook ?"good" | +---------------+ ?+------------------------------+ ? ?+-----------------+ ? ? ? ? | ? ? ? ? ? ? ? ? ? ? ? | ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| +-------------------------------------------------+ ? ? ? ? ? ? | | ? ? ? ? ? 100Mbps switch (clients) ? ? ? ? ? ? ?| ? ? ? ? ? ? | +-------------------------------------------------+ ? ? ? ? ? ? | ? ? ? ? ? ? ? ? ? ? ?| ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? | +------------------------------------------------------------------------+ | ? ? ? ? ? ? ? ? ? ? 1Gbps switch (for servers) ? ? ? ? ? ? ? ? ? ? ? ? | +------------------------------------------------------------------------+ ? ? ? ? ? ? ? | ? ? ? ? ? ? ? ? ? ? ? ? | +-----------------------------+ ? ?+-----------+ | MAIL-SRV: Dovecot, AFP, SMB | ? ?| OTHER-SRV | +-----------------------------+ ? ?+-----------+ Wireshark dump - IMAP communication: |Time ? ? | 10.211.55.3 ? ? ? ? ? ? ? ? ? ? ? ? ? | | ? ? ? ? | ? ? ? ? ? ? ? ? ? | 10.0.0.10 ? ? ? ? | |9.851 ? ?| ? ? ? ? Response: gatq OK L ? ? ? ? ? |IMAP: Response: gatq OK Logged in. | ? ? ? ? |(49433) ?<------------------ ?(143) ? ?| |9.852 ? ?| ? ? ? ? Request: o47u SELEC ? ? ? ? ? |IMAP: Request: o47u SELECT "1_GROUPS" | ? ? ? ? |(49433) ?------------------> ?(143) ? ?| |9.853 ? ?| ? ? ? ? Response: * FLAGS ( ? ? ? ? ? |IMAP: Response: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) | ? ? ? ? |(49433) ?<------------------ ?(143) ? ?| |9.854 ? ?| ? ? ? ? Request: 3y4b NOOP ? ? ? ? ? ?|IMAP: Request: 3y4b NOOP | ? ? ? ? |(49433) ?------------------> ?(143) ? ?| |9.854 ? ?| ? ? ? ? Response: 3y4b OK N ? ? ? ? ? |IMAP: Response: 3y4b OK NOOP completed. | ? ? ? ? |(49433) ?<------------------ ?(143) ? ?| |9.855 ? ?| ? ? ? ? Request: 4vlj IDLE ? ? ? ? ? ?|IMAP: Request: 4vlj IDLE | ? ? ? ? |(49433) ?------------------> ?(143) ? ?| |9.856 ? ?| ? ? ? ? Response: + idling ? ? ? ? ? ?|IMAP: Response: + idling | ? ? ? ? |(49433) ?<------------------ ?(143) ? ?| |10.108 ? | ? ? ? ? Request: DONE ? ? ? ? ? ? ? ? |IMAP: Request: DONE | ? ? ? ? |(49433) ?------------------> ?(143) ? ?| |10.108 ? | ? ? ? ? Response: 4vlj OK I ? ? ? ? ? |IMAP: Response: 4vlj OK Idle completed. | ? ? ? ? |(49433) ?<------------------ ?(143) ? ?| |10.108 ? | ? ? ? ? Request: wh89 SELEC ? ? ? ? ? |IMAP: Request: wh89 SELECT "1_GROUPS.Adm" BR Thomas von Eyben From adrian.minta at gmail.com Fri Apr 6 13:23:32 2012 From: adrian.minta at gmail.com (Adrian Minta) Date: Fri, 06 Apr 2012 13:23:32 +0300 Subject: [Dovecot] Outlook (2010) -> Dovecot (IMAP) >10x slower with high network load and many folders In-Reply-To: References: Message-ID: <4F7EC424.3010504@gmail.com> It seems that you have packets lost in the network. MAC and Windows have different network stacks so this may count for the different behavior. -- Best regards, Adrian Minta From xnasx at yandex.ru Fri Apr 6 14:21:53 2012 From: xnasx at yandex.ru (Victor) Date: Fri, 06 Apr 2012 15:21:53 +0400 Subject: [Dovecot] Username bug in '%%' variables selects In-Reply-To: <10301333633892@web94.yandex.ru> References: <10301333633892@web94.yandex.ru> Message-ID: <915031333711313@web74.yandex.ru> I solved this issue by changing a namespace to native type. namespace { separator = / prefix = inbox = yes } namespace { separator = . prefix = INBOX. inbox = no hidden = yes list = no alias_for = } namespace { type = shared separator = / prefix = shared/%%u/ list = yes subscriptions = no hidden = no location = maildir:%%h:INDEX=%h/shared/%%u } It did not turn out to make the transparent migration. :( Outlooks 2007 reloaded all scheme and frozen. It's seems a bug in Dovecot. From busseniu at in.tum.de Fri Apr 6 14:51:14 2012 From: busseniu at in.tum.de (=?UTF-8?B?Q2hyaXN0b3BoIEJ1w59lbml1cw==?=) Date: Fri, 06 Apr 2012 13:51:14 +0200 Subject: [Dovecot] Courier to Dovecot migration script issue In-Reply-To: <331561333524967@web30.yandex.ru> References: <125901333456391@web125.yandex.ru> <9BBB03B5-4742-403B-8720-FBEFB554266D@iki.fi> <331561333524967@web30.yandex.ru> Message-ID: <4F7ED8B2.6030809@in.tum.de> Hi, on 04/04/2012 09:36 AM, xnasx wrote: > 'uidvalidity' are the same for both servers, but UIDs of mesages are not: > > Courier: > d fetch 1:* uid > * 1 FETCH (UID 64295) > * 2 FETCH (UID 64296) > * 3 FETCH (UID 64297) > > Dovecot: > > * 1 FETCH (UID 64296) > * 2 FETCH (UID 64297) > * 3 FETCH (UID 64298) Does your Courier directory include a courierpop3sizelist file? The migration script places higher importance on the POP3 order than on IMAP UIDs. Also, in order to see if the old 64295 corresponds to the new 64298, you can use "fetch 1:* (uid rfc822.size body.peek[header.fields (message-id)])". Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From pw at wk-serv.de Fri Apr 6 16:58:18 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Fri, 06 Apr 2012 15:58:18 +0200 Subject: [Dovecot] Director (was: Hints for a NFS-Setup) In-Reply-To: <4F75A929.9090902@mobilia.it> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> Message-ID: <4F7EF67A.8060001@wk-serv.de> Hi again, I tried to setup a test invironemnt like this: MTA --(lmtp)--\ /--(lmtp)--> backend1 --\ -- director -- -- NFS MTA --(lmtp)--/ \--(lmtp)--> backend2 --/ IMAP-User --> frontend1 --\ /--(imap)--> backend1 --\ -- director -- -- NFS IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/ but now I'm very confused. Is it actually possible to setup a host (or two) as a director only or will I have to enable the director service on each frontend and MTA? Is anyone using a similar setup and could post his configuration for the different hosts? Happy Easter! Patrick From fxmulder at gmail.com Fri Apr 6 18:39:43 2012 From: fxmulder at gmail.com (James Devine) Date: Fri, 6 Apr 2012 09:39:43 -0600 Subject: [Dovecot] distributed mdbox In-Reply-To: References: <1ff190f691892c01a980927422f38753@mail.airstreamcomm.net> Message-ID: As it turns out I can duplicate this problem with a single dovecot server and a single gluster server using mdbox, so maybe not caching? This being the case I don't think director would help On Thu, Apr 5, 2012 at 7:16 PM, James Devine wrote: > > > On Fri, Mar 23, 2012 at 7:39 AM, wrote: > >> On Wed, 21 Mar 2012 09:56:12 -0600, James Devine >> wrote: >> > Anyone know how to setup dovecot with mdbox so that it can be used >> through >> > shared storage from multiple hosts? I've setup a gluster volume and am >> > sharing it between 2 test clients. I'm using postfix/dovecot LDA for >> > delivery and I'm using postal to send mail between 40 users. In doing >> > this, I'm seeing these errors in the logs >> > >> > Mar 21 09:36:29 test-gluster-client2 dovecot: lda(testuser34): Error: >> Fixed >> > index file /mnt/testuser34/mdbox/storage/dovecot.map.index: >> messages_count >> > 272 -> 271 >> > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: >> Log >> > synchronization error at seq=4,offset=3768 for >> > /mnt/testuser28/mdbox/storage/dovecot.map.index: Append with UID 516, >> but >> > next_uid = 517 >> > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: >> Log >> > synchronization error at seq=4,offset=4220 for >> > /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update >> > for invalid uid=517 >> > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: >> Log >> > synchronization error at seq=4,offset=5088 for >> > /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update >> > for invalid uid=517 >> > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Warning: >> > fscking index file /mnt/testuser28/mdbox/storage/dovecot.map.index >> > Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser34): Warning: >> > fscking index file /mnt/testuser34/mdbox/storage/dovecot.map.index >> > >> > >> > This is my dovecot config currently: >> > >> > jdevine at test-gluster-client2:~> dovecot -n >> > # 2.0.13: /etc/dovecot/dovecot.conf >> > # OS: Linux 3.0.0-13-server x86_64 Ubuntu 11.10 >> > lock_method = dotlock >> > mail_fsync = always >> > mail_location = mdbox:~/mdbox >> > mail_nfs_index = yes >> > mail_nfs_storage = yes >> > mmap_disable = yes >> > passdb { >> > driver = pam >> > } >> > protocols = " imap" >> > ssl_cert = > > ssl_key = > > userdb { >> > driver = passwd >> > } >> >> I was able to get dovecot working across a gluster cluster a few weeks ago >> and it worked just fine. I would recommend using the native gluster mount >> option (need to install gluster software on clients), and using >> distributed >> replicated as your replication mechanism. If you're running two gluster >> servers you should have a replica count of two with distributed >> replicated. >> You should test first to make sure you can create a file in both mounts >> and see it from every mount point in the cluster, as well as interact with >> it. It's also very important to make sure your servers are running with >> synchronized clocks from an NTP server. Very bad things happen to a >> (dovecot or gluster) cluster out of sync with NTP. >> >> What storage method are you using? I'm able to produce errors within > seconds of starting postal with more than one thread From stephan at rename-it.nl Fri Apr 6 20:08:17 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 06 Apr 2012 19:08:17 +0200 Subject: [Dovecot] Problem with polish characters and sieve fileinto In-Reply-To: <4F7DA264.8020004@rename-it.nl> References: <4F7D9E29.3050802@eurocar.pl> <4F7DA264.8020004@rename-it.nl> Message-ID: <4F7F2301.4020702@rename-it.nl> On 4/5/2012 3:47 PM, Stephan Bosch wrote: > Op 4/5/2012 3:29 PM, l.chelchowski at eurocar.pl schreef: >> Hi, >> >> I have problems with sieve fileinto and polish characters in folder >> names: >> # rule:[??????????????????] >> elsif anyof (header :contains "Subject" "??????????????????") >> { >> fileinto "INBOX/&AQUBBwEZAUIBRADzAVsBegF8AQQBBgEYAUEBQwDTAVoBeQF7-"; >> } > > Looks like your Sieve GUI uses the wrong Unicode format. The above is > mUTF7 as used by IMAP. However, Sieve expects UTF-8. So it should use > this instead: > > fileinto "INBOX/??????????????????"; Added item to the troubleshooting wiki page: http://master.wiki2.dovecot.org/Pigeonhole/Sieve/Troubleshooting#Mailbox_Names_with_non-ASCII_Characters_Cause_Problems By the looks of it, you are also using the RoundCube SieveRules plugin. The solution for RoundCube mentioned in the wiki should solve it for you. Regards, Stephan. From dmiller at amfes.com Fri Apr 6 21:53:31 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Fri, 06 Apr 2012 11:53:31 -0700 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: <4F7D972A.8050406@Media-Brokers.com> References: <4F7D972A.8050406@Media-Brokers.com> Message-ID: On 4/5/2012 5:59 AM, Charles Marcus wrote: > On 2012-04-05 4:18 AM, Thomas Leuxner wrote: > > Also with 2.x you may want to use LMTP rather than the LDA Piping. > > > > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP > > I am preparing to convert my main client's postfix_courier-imap setup > to dovecot 2.1, which currently just uses the postfix virtual delivery > agent... > > Does anyone know if the use of LMTP (or even the dovecot LDA) still > loses the x-original-to header that the postfix vda adds and that I > rely heavily on (since I use a lot of aliases), and if it does, is > there any solution to get the original recipient added back in before > final delivery? > > Everything I'm reading says that LMTP is better, but I really do need > this header (or one like it) to be there so I know who the original > recipient was (for filtering and other purposes). > I'm currently using Postfix 2.7, Dovecot 2.1, and the Dovecot LDA. I have a pure virtual user environment stored in LDAP. My messages include X-Original-To and Delivered-To headers. I had difficulty getting the LMTP transport to work previously - I may revisit that. -- Daniel From CMarcus at Media-Brokers.com Fri Apr 6 23:00:19 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 06 Apr 2012 16:00:19 -0400 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: References: <4F7D972A.8050406@Media-Brokers.com> Message-ID: <4F7F4B53.2090700@Media-Brokers.com> On 2012-04-06 2:53 PM, Daniel L. Miller wrote: > I'm currently using Postfix 2.7, Dovecot 2.1, and the Dovecot LDA. I > have a pure virtual user environment stored in LDAP. My messages include > X-Original-To and Delivered-To headers. Well that is great news... at least I'll be able to use the LDA, if not LMTP... Thanks! :) > I had difficulty getting the LMTP transport to work previously - I may > revisit that. If you do, by all means reply back on whether or not the headers are still there... Thanks again, -- Best regards, Charles From kgc at corp.sonic.net Sat Apr 7 02:40:01 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Fri, 6 Apr 2012 16:40:01 -0700 Subject: [Dovecot] POP3 dele to Trash? Message-ID: <20120406234001.GG89906@corp.sonic.net> To simplify recovery of accidentally deleted messages we'd like to send all messages that are deleted by pop clients to their Trash folder. (Which is auto expired already.) This allows a POP client who deletes an important email to restore it themselves by logging into our webmail client and move the lost message from Trash to the Inbox where it would be fetched by their client again. Has anyone already done this? Should this be possible via a plugin? I see the deleted-to-trash imap plugin. We are using Maildir if it makes a difference. -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From kgc at corp.sonic.net Sat Apr 7 03:10:26 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Fri, 06 Apr 2012 17:10:26 -0700 Subject: [Dovecot] POP3 dele to Trash? In-Reply-To: <20120406234001.GG89906@corp.sonic.net> References: <20120406234001.GG89906@corp.sonic.net> Message-ID: <4F7F85F2.3000909@corp.sonic.net> On 04/06/12 16:40, Kelsey Cummings wrote: > Has anyone already done this? Should this be possible via a plugin? > I see the deleted-to-trash imap plugin. We are using Maildir if it > makes a difference. Of course, this is exactly what the Lazy Expunge plugin does, isn't it? -K From wgillespie+dovecot at es2eng.com Sat Apr 7 04:16:14 2012 From: wgillespie+dovecot at es2eng.com (Willie Gillespie) Date: Fri, 06 Apr 2012 19:16:14 -0600 Subject: [Dovecot] Outlook (2010) -> Dovecot (IMAP) >10x slower with high network load and many folders In-Reply-To: References: Message-ID: <4F7F955E.2050508@es2eng.com> On 4/6/2012 3:52 AM, Thomas von Eyben wrote: > Test results: > CLIENT-1 is having the problems when CLIENT-2 is using all the > (100Mbps) bandwidth eg. copying files to MAIL-SRV. > If I move CLIENT-1 to CLIENT-3 then almost all the delay is gone. > NB.: I have not (yet) tested if the problem also exists when CLIENT-2 > generates traffic to MAIL-SRV as opposed to OTHER-SRV (but I am > expecting the same problems). So the link between your 100 Mbps switch and the 1 Gbps switch is saturated by CLIENT-2, so CLIENT-1 is just getting the leftovers? Since CLIENT-3 doesn't go through that 100 Mbps switch, it obviously doesn't see that issue. From plmb21 at gmail.com Sat Apr 7 06:48:43 2012 From: plmb21 at gmail.com (PL MB) Date: Fri, 6 Apr 2012 20:48:43 -0700 Subject: [Dovecot] Setting ACL for master user after login Message-ID: Hello, I'd like to log in to normal user accounts as a master user but retain the normal users' ACLs. The Master Users page on the Dovecot 1.x wiki (1) says that I can set the master user's ACLs in a postlogin script. The documentation for master users on the 2.x wiki (2) no longer has any statements about master user ACLs. Has something important in this regard changed? Can I no longer override the ACLs in a postlogin script? Thanks in advance. (1) http://wiki.dovecot.org/Authentication/MasterUsers (2) http://wiki2.dovecot.org/Authentication/MasterUsers From andy at xecu.net Sat Apr 7 10:13:42 2012 From: andy at xecu.net (Andy Dills) Date: Sat, 7 Apr 2012 03:13:42 -0400 (EDT) Subject: [Dovecot] Director pop-login and imap-login processes exiting on signal 11 Message-ID: <20120407030811.I46246@shell.xecu.net> We recently upgraded our cluster to 2.1.3, to enable director proxying. Everything appears to be working fine for the most part; the only odd thing is that I'm seeing a lot of entries in the logs like this: Apr 7 02:18:05 mail-out06 dovecot: pop3-login: Fatal: master: service(pop3-login): child 75029 killed with signal 11 (core not dumped - set service pop3-login { drop_priv_before_exec=yes }) This is on the proxy side, not that backend side. When I try to get a dump out of it, and add drop_prive_before_exec and chroot= to the pop3-login statement on the proxy, I keep running into permissions errors with the various service sockets. Any suggestions? Thanks, Andy --- Andy Dills Xecunet, Inc. www.xecu.net 301-682-9972 --- From stan at hardwarefreak.com Sat Apr 7 13:19:46 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 07 Apr 2012 05:19:46 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: References: Message-ID: <4F8014C2.3010606@hardwarefreak.com> On 4/5/2012 3:02 PM, Emmanuel Noobadmin wrote: Hi Emmanuel, > I'm trying to improve the setup of our Dovecot/Exim mail servers to > handle the increasingly huge accounts (everybody thinks it's like > infinitely growing storage like gmail and stores everything forever in > their email accounts) by changing from Maildir to mdbox, and to take > advantage of offloading older emails to alternative networked storage > nodes. I'll assume "networked storage nodes" means NFS, not FC/iSCSI SAN, in which case you'd have said "SAN". > The question now is whether having a single large server or will a > number of 1U servers with the same total capacity be better? Less complexity and cost is always better. CPU throughput isn't a factor in mail workloads--it's all about IO latency. A 1U NFS server with 12 drive JBOD is faster, cheaper, easier to setup and manage, sucks less juice and dissipates less heat than 4 1U servers each w/ 4 drives. I don't recall seeing your user load or IOPS requirements so I'm making some educated guesses WRT your required performance and total storage. I came up with the following system that should be close to suitable, for ~$10k USD. The 4 node system runs ~$12k USD. At $2k this isn't substantially higher. But when we double the storage of each architecture we're at ~$19k, vs ~$26k for an 8 node cluster, a difference of ~$7k. That's $1k shy of another 12 disk JBOD. Since CPU is nearly irrelevant for a mail workload, you can see it's much cheaper to scale capacity and IOPS with a single node w/fat storage than with skinny nodes w/thin storage. Ok, so here's the baseline config I threw together: http://h10010.www1.hp.com/wwpc/us/en/sm/WF06b/15351-15351-3328412-241644-3328421-4091396-4158470-4158440.html?dnr=1 8 cores is plenty, 2 boot drives mirrored on B110i, 16GB (4x4GB) http://www.lsi.com/products/storagecomponents/Pages/LSISAS9205-8e.aspx http://h10010.www1.hp.com/wwpc/us/en/sm/WF06b/12169-304616-3930445-3930445-3930445-3954787-4021626-4021628.html?dnr=1 w/ 12 2TB 7.2K SATA drives, configured as md concat+RAID1 pairs with 12 allocation groups, 12TB usable. Format the md device with the defaults: $ mkfs.xfs /dev/md0 Mount with inode64. No XFS stripe alignment to monkey with. No md chunk size or anything else to worry about. XFS' allocation group design is pure elegance here. If 12 TB isn't sufficient, or if you need more space later, you can daisy chain up to 3 additional D2600 JBODs for ~$8500 USD each, just add cables. This quadruples IOPS, throughput, and capacity--96TB total, 48TB net. Simply create 6 more mdraid1 devices and grow the linear array with them. Then do an xfs_growfs to bring the extra 12TB of free space into the filesystem. If you're budget conscious and/or simply prefer quality inexpensive whitebox/DIY type gear, as I do, you can get 24 x 2TB drives in one JBOD chassis for $7400 USD. That twice the drives, capacity, IOPS, for ~$2500 less than the HP JBOD. And unlike the HP 'enterprise SATA' drives, the 2TB WD Black series have a 5 year warranty, and work great with mdraid. Chassis and drives at Newegg: http://www.newegg.com/Product/Product.aspx?Item=N82E16816133047 http://www.newegg.com/Product/Product.aspx?Item=N82E16822136792 You can daisy chain 3 of these off one HBA SFF8088 port, 6 total on our LSI 9205-8e above, for a total of 144 2TB drives, 72 effective spindles in our concat+RAID1 setup, 144TB net space. > Will be > using RAID 1 pairs, likely XFS based on reading Hoeppner's > recommendation on this and the mdadm list. To be clear, the XFS configuration I recommend/promote for mailbox storage is very specific and layered. The layers must all be used together to get the performance. These layers consist of using multiple hardware or software RAID1 pairs and concatenating them with an md linear array. You then format that md device with the XFS defaults, or a specific agcount if you know how to precisely tune AG layout based on disk size and your anticipated concurrency level of writers. Putting XFS on a singe RAID1 pair, as you seem to be describing above for the multiple "thin" node case, and hitting one node with parallel writes to multiple user mail dirs, you'll get less performance than EXT3/4 on that mirror pair--possibly less than half, depending on the size of the disks and thus the number of AGs created. The 'secret' to XFS performance with this workload is concatenation of spindles. Without it you can't spread the AGs--thus directories, thus parallel file writes--horizontally across the spindles--and this is the key. By spreading AGs 'horizontally' across the disks in a concat, instead of 'vertically' down a striped array, you accomplish two important things: 1. You dramatically reduce disk head seeking by using the concat array. With XFS on a RAID10 array of 24 2TB disks you end up with 24 AGs evenly spaced vertically down each disk in the array, following the stripe pattern. Each user mailbox is stored in a different directory. Each directory was created in a different AG. So if you have 96 users writing their dovecot index concurrently, you have at worst case a minimum 192 head movements occurring back and forth across the entire platter of each disk, and likely not well optimized by TCQ/NCQ. Why 192 instead of 96? The modification time in the directory metadata must be updated for each index file, among other things. 2. Because we decrease seeks dramatically we also decrease response latency significantly. With the RAID1+concat+XFS we have 12 disks each with only 2 AGs spaced evenly down each platter. We have the same 4 user mail dirs in each AG, but in this case only 8 user mail dirs are contained on each disk instead of portions all 96. With the same 96 concurrent writes to indexes, in this case end up with only 16 seeks per drive--again, one to update each index file and one to update the metadata. Assuming these drives have a max seek rate of 150 which is the average for 7.2k drives, it will take 192/150 = 1.28 seconds for these operations on the RAID10 array. With the concat array it will only take 16/150 = 0.11 seconds. Extrapolating from that demonstrates that the concat array can handle 1.28/0.11 = 11.6*96 = 1,111 concurrent user index updates in the same time as the RAID10 array, just over 10 times more users. Granted, these are rough theoretical numbers--an index plus metadata update isn't always going to cause a seek on every chunk in a stripe, etc. But this does paint a very accurate picture of the differences in mailbox workload disk seek patterns between XFS on concat and RAID10 with the same hardware. In production one should be able to handle at minimum 2x more users, probably many more, with the RAID1+concat+XFS vs RAID10+XFS setup on the same hardware. > Currently, I'm leaning towards multiple small servers because I think > it should be better in terms of performance. This usually isn't the case with mail. It's impossible to split up the user files across the storage nodes in a way that balances block usage on each node and user access to those blocks. Hotspots are inevitable in both categories. You may achieve the same total performance of a single server, maybe slightly surpass it depending on user load, but you end up spending extra money on building resources that are idle most of the time, in the case of CPU and NICs, or under/over utilized, in the case of disk capacity in each node. Switch ports aren't horribly expensive today, but you're still wasting some with the farm setup. > At the very least even if > one node gets jammed up, the rest should still be able serve up the > emails for other accounts that is unless Dovecot will get locked up by > that jammed transaction. Some host failure redundancy is about all you'd gain from the farm setup. Dovecot shouldn't barf due to one NFS node being down, only hiccup. I.e. only imap process accessing files on the downed node would have trouble. > Also, I could possibly arrange them in a sort > of network raid 1 to gain redundancy over single machine failure. Now you're sounding like Charles Marcus, but worse. ;) Stay where you are, and brush your hair away from your forehead. I'm coming over with my branding iron that says "K.I.S.S" > Would I be correct in these or do actual experiences say otherwise? Oracles on Mount Interweb profess that 2^5 nodes wide scale out is the holy grail. IBM's mainframe evangelists tell us to put 5 million mail users on a SystemZ with hundreds of Linux VMs. I think bliss for most of us is found somewhere in the middle. -- Stan From centos.admin at gmail.com Sat Apr 7 17:43:09 2012 From: centos.admin at gmail.com (Emmanuel Noobadmin) Date: Sat, 7 Apr 2012 22:43:09 +0800 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F8014C2.3010606@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> Message-ID: On 4/7/12, Stan Hoeppner wrote: Firstly, thanks for the comprehensive reply. :) > I'll assume "networked storage nodes" means NFS, not FC/iSCSI SAN, in > which case you'd have said "SAN". I haven't decided on that but it would either be NFS or iSCSI over Gigabit. I don't exactly get a big budget for this. iSCSI because I planned to do md/mpath over two separate switches so that if one switch explodes, the email service would still work. > Less complexity and cost is always better. CPU throughput isn't a > factor in mail workloads--it's all about IO latency. A 1U NFS server > with 12 drive JBOD is faster, cheaper, easier to setup and manage, sucks > less juice and dissipates less heat than 4 1U servers each w/ 4 drives. My worry is that if that one server dies, everything is dead. With at least a pair of servers, I could keep it running, or if necessary, restore the accounts on the dead servers from backup, make some config changes and have everything back running while waiting for replacement hardware. > I don't recall seeing your user load or IOPS requirements so I'm making > some educated guesses WRT your required performance and total storage. I'm embarrassed to admit I don't have hard numbers on the user load except the rapidly dwindling disk space count and the fact when the web-based mail application try to list and check disk quota, it can bring the servers to a crawl. My lame excuse is that I'm just the web dev who got caught holding the server admin potato. > is nearly irrelevant for a mail workload, you can see it's much cheaper > to scale capacity and IOPS with a single node w/fat storage than with > skinny nodes w/thin storage. Ok, so here's the baseline config I threw > together: One of my concern is that heavy IO on the same server slow the overall performance even though the theoretical IOPS of the total drives are the same on 1 and on X servers. Right now, the servers are usually screeching to a halt, to the point of even locking out SSH access due to IOWait sending the load in top to triple digits. > Some host failure redundancy is about all you'd gain from the farm > setup. Dovecot shouldn't barf due to one NFS node being down, only > hiccup. I.e. only imap process accessing files on the downed node would > have trouble. But if I only have one big storage node and that went down, Dovecot would barf wouldn't it? Or would the mdbox format mean Dovecot would still use the local storage, just that users can't access the offloaded messages? >> Also, I could possibly arrange them in a sort >> of network raid 1 to gain redundancy over single machine failure. > > Now you're sounding like Charles Marcus, but worse. ;) Stay where you > are, and brush your hair away from your forehead. I'm coming over with > my branding iron that says "K.I.S.S" Lol, I have no idea who Charles is, but I always feel safer if there was some kind of backup. Especially since I don't have the time to dedicate myself to server administration, by the time I notice something is bad, it might be too late for anything but the backup. Of course management and clients don't agree with me since backup/redundancy costs money. :) From dmiller at amfes.com Sat Apr 7 21:06:48 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Sat, 07 Apr 2012 11:06:48 -0700 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: <4F7F4B53.2090700@Media-Brokers.com> References: <4F7D972A.8050406@Media-Brokers.com> <4F7F4B53.2090700@Media-Brokers.com> Message-ID: On 4/6/2012 1:00 PM, Charles Marcus wrote: > On 2012-04-06 2:53 PM, Daniel L. Miller wrote: >> I'm currently using Postfix 2.7, Dovecot 2.1, and the Dovecot LDA. I >> have a pure virtual user environment stored in LDAP. My messages include >> X-Original-To and Delivered-To headers. > > Well that is great news... at least I'll be able to use the LDA, if > not LMTP... > > Thanks! :) > >> I had difficulty getting the LMTP transport to work previously - I may >> revisit that. > > If you do, by all means reply back on whether or not the headers are > still there... > > Thanks again, > From the documentation... http://www.postfix.org/virtual.8.html The*virtual*(8) delivery agent prepends a "*From* /sender/ /time/*_*/stamp/" envelope header to each message, prepends a *Delivered-To:* message header with the envelope recipient address, prepends an*X-Original-To:* header with the recip- ient address as given to Postfix, prepends a*Return-Path:* message header with the envelope sender address, prepends a> character to lines beginning with "*From* ", and appends an empty line. Using the Postfix pipe agent, which is what is used with the Dovecot LDA, http://www.postfix.org/pipe.8.html *flags=BDFORXhqu.*> (optional) Optional message processing flags. By default, a message is copied unchanged. *B* Append a blank line at the end of each mes- sage. This is required by some mail user agents that recognize "*From* " lines only when preceded by a blank line. *D* Prepend a "*Delivered-To:* /recipient/" message header with the envelope recipient address. Note: for this to work, the/transport/*_desti-* *nation_recipient_limit* must be 1 (see SIN- GLE-RECIPIENT DELIVERY above for details). The*D* flag also enforces loop detection (Postfix 2.5 and later): if a message already contains a*Delivered-To:* header with the same recipient address, then the message is returned as undeliverable. The address comparison is case insensitive. This feature is available as of Postfix 2.0. *F* Prepend a "*From* /sender time/*_*/stamp/" envelope header to the message content. This is expected by, for example,*UUCP* software. *O* Prepend an "*X-Original-To:* /recipient/" mes- sage header with the recipient address as given to Postfix. Note: for this to work, the*/transport/_destination_recipient_limit * must be 1 (see SINGLE-RECIPIENT DELIVERY above for details). Unfortunately, the docs for the ltmp agent http://www.postfix.org/lmtp.8.html don't say anything about adding these headers. I tried asking on the Postfix list - didn't get much of an answer. -- Daniel From jerry at seibercom.net Sat Apr 7 21:30:38 2012 From: jerry at seibercom.net (Jerry) Date: Sat, 7 Apr 2012 14:30:38 -0400 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: References: <4F7D972A.8050406@Media-Brokers.com> <4F7F4B53.2090700@Media-Brokers.com> Message-ID: <20120407143038.5252acf9@scorpio> On Sat, 07 Apr 2012 11:06:48 -0700 Daniel L. Miller articulated: > Unfortunately, the docs for the ltmp agent > http://www.postfix.org/lmtp.8.html don't say anything about adding > these headers. I tried asking on the Postfix list - didn't get much > of an answer. I may be wrong; however, from what I have been able to understand in regards to the Postfix documentation, if it does not explicitly claim to have a feature, then that feature is not available. In other words, if it doesn't state it can do it, it can't. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From thomasvoneyben at gmail.com Sat Apr 7 23:13:15 2012 From: thomasvoneyben at gmail.com (Thomas von Eyben) Date: Sat, 7 Apr 2012 22:13:15 +0200 Subject: [Dovecot] Outlook (2010) -> Dovecot (IMAP) >10x slower with high network load and many folders In-Reply-To: <4F7F955E.2050508@es2eng.com> References: <4F7F955E.2050508@es2eng.com> Message-ID: On Sat, Apr 7, 2012 at 3:16 AM, Willie Gillespie wrote: > On 4/6/2012 3:52 AM, Thomas von Eyben wrote: >> >> Test results: >> CLIENT-1 is having the problems when CLIENT-2 is using all the >> (100Mbps) bandwidth eg. copying files to MAIL-SRV. >> If I move CLIENT-1 to CLIENT-3 then almost all the delay is gone. >> NB.: I have not (yet) tested if the problem also exists when CLIENT-2 >> generates traffic to MAIL-SRV as opposed to OTHER-SRV (but I am >> expecting the same problems). > > > So the link between your 100 Mbps switch and the 1 Gbps switch is saturated > by CLIENT-2, so CLIENT-1 is just getting the leftovers? > > Since CLIENT-3 doesn't go through that 100 Mbps switch, it obviously doesn't > see that issue. Yes - that's my current "workaround" (perhaps also solution), I'm wondering if the performance is really expected to be _so_ bad when other users are utilizing the LAN. (You seem to indicate that what I am observing is expected and is "just" caused by [un-intended] semi-bad behavior from other users?) BR TvE From dovecot at r.paypc.com Sat Apr 7 23:45:08 2012 From: dovecot at r.paypc.com (Robin) Date: Sat, 07 Apr 2012 13:45:08 -0700 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F8014C2.3010606@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> Message-ID: <4F80A754.4000703@r.paypc.com> > Putting XFS on a singe RAID1 pair, as you seem to be describing above > for the multiple "thin" node case, and hitting one node with parallel > writes to multiple user mail dirs, you'll get less performance than > EXT3/4 on that mirror pair--possibly less than half, depending on the > size of the disks and thus the number of AGs created. The 'secret' to > XFS performance with this workload is concatenation of spindles. > Without it you can't spread the AGs--thus directories, thus parallel > file writes--horizontally across the spindles--and this is the key. By > spreading AGs 'horizontally' across the disks in a concat, instead of > 'vertically' down a striped array, you accomplish two important things: > > 1. You dramatically reduce disk head seeking by using the concat array. > With XFS on a RAID10 array of 24 2TB disks you end up with 24 AGs > evenly spaced vertically down each disk in the array, following the > stripe pattern. Each user mailbox is stored in a different directory. > Each directory was created in a different AG. So if you have 96 users > writing their dovecot index concurrently, you have at worst case a > minimum 192 head movements occurring back and forth across the entire > platter of each disk, and likely not well optimized by TCQ/NCQ. Why 192 > instead of 96? The modification time in the directory metadata must be > updated for each index file, among other things. Does the XFS allocator automatically distribute AGs in this way even when disk usage is extremely light, i.e, a freshly formatted system with user directories initially created, and then the actual mailbox contents copied into them? If this is indeed the case, then what you describe is a wondrous revelation, since you're scaling out the number of simultaneous metadata reads+writes/second as you add RAID1 pairs, if my understanding of this is correct. I'm assuming of course, but should look at the code, that metadata locks imposed by the filesystem "distribute" as the number of pairs increase - if it's all just one Big Lock, then that wouldn't be the case. Forgive my laziness, as I could just experiment and take a look at the on-disk structures myself, but I don't have four empty drives handy to experiment. The bandwidth improvements due to striping (RAID0/5/6 style) are no help for metadata-intensive IO loads, and probably of little value for even mdbox loads too, I suspect, unless the mdbox max size is set to something pretty large, no? Have you tried other filesystems and seen if they distribute metadata in a similarly efficient and scalable manner across concatenated drive sets? Is there ANY point to using striping at all, a la "RAID10" in this? I'd have thought just making as many RAID1 pairs out of your drives as possible would be the ideal strategy - is this not the case? =R= From stan at hardwarefreak.com Sun Apr 8 03:46:20 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 07 Apr 2012 19:46:20 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F80A754.4000703@r.paypc.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F80A754.4000703@r.paypc.com> Message-ID: <4F80DFDC.2030703@hardwarefreak.com> On 4/7/2012 3:45 PM, Robin wrote: > >> Putting XFS on a singe RAID1 pair, as you seem to be describing above >> for the multiple "thin" node case, and hitting one node with parallel >> writes to multiple user mail dirs, you'll get less performance than >> EXT3/4 on that mirror pair--possibly less than half, depending on the >> size of the disks and thus the number of AGs created. The 'secret' to >> XFS performance with this workload is concatenation of spindles. >> Without it you can't spread the AGs--thus directories, thus parallel >> file writes--horizontally across the spindles--and this is the key. By >> spreading AGs 'horizontally' across the disks in a concat, instead of >> 'vertically' down a striped array, you accomplish two important things: >> >> 1. You dramatically reduce disk head seeking by using the concat array. >> With XFS on a RAID10 array of 24 2TB disks you end up with 24 AGs >> evenly spaced vertically down each disk in the array, following the >> stripe pattern. Each user mailbox is stored in a different directory. >> Each directory was created in a different AG. So if you have 96 users >> writing their dovecot index concurrently, you have at worst case a >> minimum 192 head movements occurring back and forth across the entire >> platter of each disk, and likely not well optimized by TCQ/NCQ. Why 192 >> instead of 96? The modification time in the directory metadata must be >> updated for each index file, among other things. > > Does the XFS allocator automatically distribute AGs in this way even > when disk usage is extremely light, i.e, a freshly formatted system with > user directories initially created, and then the actual mailbox contents > copied into them? It doesn't distribute AGs. There are a static number created during mkfs.xfs. The inode64 allocator round robins new directory creation across the AGs, and does the same with files created in those directories. Having the directory metadata and file extents in the same AG decreases head movement and thus seek latency for mixed metadata/extent high IOPS workloads. > If this is indeed the case, then what you describe is a wondrous > revelation, since you're scaling out the number of simultaneous metadata > reads+writes/second as you add RAID1 pairs, if my understanding of this > is correct. Correct. And adding more space and IOPS is uncomplicated. No chunk calculations, no restriping of the array. You simply grow the md linear array adding the new disk device. Then grow XFS to add the new free space to the filesystem. AFAIK this can be done infinitely, theoretically. I'm guessing md has a device count limit somewhere. If not your bash line buffer might. ;) > I'm assuming of course, but should look at the code, that > metadata locks imposed by the filesystem "distribute" as the number of > pairs increase - if it's all just one Big Lock, then that wouldn't be > the case. XFS locking is done as minimally as possibly and is insanely fast. I've not come across any reported performance issues relating to it. And yes, any single metadata lock will occur in a single AG on one mirror pair using the concat setup. > Forgive my laziness, as I could just experiment and take a look at the > on-disk structures myself, but I don't have four empty drives handy to > experiment. Don't sweat it. All of this stuff is covered in the XFS Filesystem Structure Guide, exciting reading if you enjoy a root canal while watching snales race: http://xfs.org/docs/xfsdocs-xml-dev/XFS_Filesystem_Structure//tmp/en-US/html/index.html > The bandwidth improvements due to striping (RAID0/5/6 style) are no help > for metadata-intensive IO loads, and probably of little value for even > mdbox loads too, I suspect, unless the mdbox max size is set to > something pretty large, no? The problem with striped parity RAID is not allocation, which takes place in free space and is pretty fast. The problem is the extra read seeks and bandwidth of the RMW cycle when you modify an existing stripe. Updating a single flag in a Dovecot index causes md or the hardware RAID controller to read the entire stripe into buffer space or RAID cache, modify the flag byte, recalculate parity, then write the whole stripe and parity block back out across all the disks. With a linear concat of RAID1 pairs we're simply rewriting a single 4KB filesystem block, maybe only a single 512B sector. I'm at the edge of my knowledge here. I don't know exactly how Timo does the index updates. Regardless of the method, the index update is light years faster with the concat setup as there is no RMW and full stripe writeback as with the RAID5/6 case. > Have you tried other filesystems and seen if they distribute metadata in > a similarly efficient and scalable manner across concatenated drive sets? EXT, any version, does not. ReiserFS does not. Both require disk striping to achieve any parallelism. With concat they both simply start writing at the beginning sectors of the first RAID1 pair and 4 years later maybe reach the last pair as they fill up the volume. ;) JFS has a more advanced allocation strategy that EXT or ReiserFS, not as advanced as XFS. I've never read of a concat example with JFS and I've never tested it. It's all but a dead filesystem at this point anyway, less than 2 dozen commits in 8 years last I checked, and these were simple bug fixes and changes to keep it building on new kernels. If it's not suffering bit rot now I'm sure it will be in the near future. > Is there ANY point to using striping at all, a la "RAID10" in this? I'd > have thought just making as many RAID1 pairs out of your drives as > possible would be the ideal strategy - is this not the case? If you're using XFS, and your workload is overwhelmingly mail, RAID1+concat is the only way to fly, and it flies. If the workload is not mail, say large file streaming writes, then you're limited to 100-200MB/s, a single drive of throughput, as each file is written to a single directory on a single AG on a single disk. For streaming write performance you'll need striping. If you have many concurrent large streaming writes, you'll want to concat multiple striped arrays. -- Stan From stan at hardwarefreak.com Sun Apr 8 21:21:47 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sun, 08 Apr 2012 13:21:47 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: References: <4F8014C2.3010606@hardwarefreak.com> Message-ID: <4F81D73B.9030901@hardwarefreak.com> On 4/7/2012 9:43 AM, Emmanuel Noobadmin wrote: > On 4/7/12, Stan Hoeppner wrote: > > Firstly, thanks for the comprehensive reply. :) > >> I'll assume "networked storage nodes" means NFS, not FC/iSCSI SAN, in >> which case you'd have said "SAN". > > I haven't decided on that but it would either be NFS or iSCSI over > Gigabit. I don't exactly get a big budget for this. iSCSI because I > planned to do md/mpath over two separate switches so that if one > switch explodes, the email service would still work. So it seems you have two courses of action: 1. Identify individual current choke points and add individual systems and storage to eliminate those choke points. 2. Analyze your entire workflow and all systems, identifying all choke points, then design a completely new well integrated storage architecture that solves all current problems and addresses future needs Adding an NFS server and moving infrequently (old) accessed files to alternate storage will alleviate your space problems. But it will probably not fix some of the other problems you mention, such as servers bogging down and becoming unresponsive, as that's not a space issue. The cause of that would likely be an IOPS issue, meaning you don't have enough storage spindles to service requests in a timely manner. >> Less complexity and cost is always better. CPU throughput isn't a >> factor in mail workloads--it's all about IO latency. A 1U NFS server >> with 12 drive JBOD is faster, cheaper, easier to setup and manage, sucks >> less juice and dissipates less heat than 4 1U servers each w/ 4 drives. > > My worry is that if that one server dies, everything is dead. With at > least a pair of servers, I could keep it running, or if necessary, > restore the accounts on the dead servers from backup, make some config > changes and have everything back running while waiting for replacement > hardware. You are a perfect candidate for VMware ESX. The HA feature will do exactly what you want. If one physical node in the cluster dies, HA automatically restarts the dead VMs on other nodes, transparently. Clients will will have to reestablish connections, but everything else will pretty much be intact. Worse case scenario will possibly be a few corrupted mailboxes that were being written when the hardware crashed. A SAN is required for such a setup. I had extensive experience with ESX and HA about 5 years ago and it works as advertised. After 5 years it can only have improved. It's not "cheap" but usually pays for itself due to being able to consolidate the workload of dozens of physical servers into just 2 or 3 boxes. >> I don't recall seeing your user load or IOPS requirements so I'm making >> some educated guesses WRT your required performance and total storage. > > I'm embarrassed to admit I don't have hard numbers on the user load > except the rapidly dwindling disk space count and the fact when the > web-based mail application try to list and check disk quota, it can > bring the servers to a crawl. Maybe just starting with a description of your current hardware setup and number of total users/mailboxes would be a good starting point. How many servers do you have, what storage is connected to each, percent of MUA POP/IMAP connections from user PCs versus those from webmail applications, etc, etc. Probably the single most important piece of information would be the hardware specs of your current Dovecot server, CPUs/RAM/disk array, etc, and what version of Dovecot you're running. The focus of your email is building a storage server strictly to offload old mail and free up space on the Dovecot server. From the sound of things, this may not be sufficient to solve all your problems. > My lame excuse is that I'm just the web > dev who got caught holding the server admin potato. Baptism by fire. Ouch. What doesn't kill you makes you stronger. ;) >> is nearly irrelevant for a mail workload, you can see it's much cheaper >> to scale capacity and IOPS with a single node w/fat storage than with >> skinny nodes w/thin storage. Ok, so here's the baseline config I threw >> together: > > One of my concern is that heavy IO on the same server slow the overall > performance even though the theoretical IOPS of the total drives are > the same on 1 and on X servers. Right now, the servers are usually > screeching to a halt, to the point of even locking out SSH access due > to IOWait sending the load in top to triple digits. If multiple servers are screeching to a halt due to iowait, either all of your servers individual disks are overloaded, or you already have shared storage. We really need more info on your current architecture. Right now we don't knw if we're talking about 4 servers or 40., 100 users or 10,000. >> Some host failure redundancy is about all you'd gain from the farm >> setup. Dovecot shouldn't barf due to one NFS node being down, only >> hiccup. I.e. only imap process accessing files on the downed node would >> have trouble. > > But if I only have one big storage node and that went down, Dovecot > would barf wouldn't it? > Or would the mdbox format mean Dovecot would still use the local > storage, just that users can't access the offloaded messages? If the big storage node is strictly alt storage, and it dies, Dovecot will still access its main mdbox storage just fine. It simply wouldn't be able to access the alt storage and would log errors for those requests. If you design a whole new architecture from scratch, going with ESX and an iSCSI SAN this whole line of thinking is moot as there is no SPOF. This can be done with as little as two physical servers and one iSCSI SAN array which has dual redundant controllers in the base config. Depending on your actual IOPS needs, you could possibly consolidate everything you have now into two physical servers and one iSCSI SAN array, for between $30-40K USD in hardware and $8-10K in ESX licenses. That just a guess on ESX as I don't know the current pricing. Even if it's that "high" it's far more than worth the price due to the capability. Such a setup allows you to run all of your Exim, webmail, Dovecot, etc servers on two machines, and you usually get much better performance than with individual boxes, especially if you manually place the VMs on the nodes for lowest network latency. For instance, if you place your webmail server VM on the same host as the Dovecot VM, TCP packet latency drops from the high micro/low milliscond range into the mid nanosecond range--a 1000x decrease in latency. Why? The packet transfer is now a memory-to-memory copy through the hypervisor. The packets never reach a physical network interface. You can do some of these things with free Linux hypervisors, but AFAIK the poor management interfaces for them make the price of ESX seem like a bargain. >>> Also, I could possibly arrange them in a sort >>> of network raid 1 to gain redundancy over single machine failure. >> >> Now you're sounding like Charles Marcus, but worse. ;) Stay where you >> are, and brush your hair away from your forehead. I'm coming over with >> my branding iron that says "K.I.S.S" > Lol, I have no idea who Charles is, but I always feel safer if there > was some kind of backup. Especially since I don't have the time to > dedicate myself to server administration, by the time I notice > something is bad, it might be too late for anything but the backup. Search the list archives for Charles' thread about bringing up a 2nd office site. His desire was/is to duplicate machines at the 2nd site for redundancy, when the proper thing to do is duplicate them at the primary site, and simply duplicate the network links between sites. My point to you and Charles is that you never add complexity for the sake of adding complexity. > Of course management and clients don't agree with me since > backup/redundancy costs money. :) So does gasoline, but even as the price has more than doubled in 3 years in the States, people haven't stopped buying it. Why? They have to have it. The case is the same for certain levels of redundancy. You simply have to have it. You job is properly explaining that need. Ask the CEO/CFO how much money the company will lose in productivity if nobody has email for 1 workday, as that is how long it will take to rebuild it from scratch and restore all the data when it fails. Then ask what the cost is if all the email is completely lost because they were to cheap to pay for a backup solution? To executives, money in the bank is like the family jewels in their trousers. Kicking the family jewels and generating that level of pain seriously gets their attention. Likewise, if a failed server plus rebuild/restore costs $50K in lost productivity, spending $20K on a solution to prevent that from happening is a good investment. Explain it in terms execs understand. Have industry data to back your position. There plenty of it available. -- Stan From dmiller at amfes.com Mon Apr 9 04:20:19 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Sun, 08 Apr 2012 18:20:19 -0700 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: <20120407143038.5252acf9@scorpio> References: <4F7D972A.8050406@Media-Brokers.com> <4F7F4B53.2090700@Media-Brokers.com> <20120407143038.5252acf9@scorpio> Message-ID: On Sat, 7 Apr 2012 14:30:38 -0400, Jerry wrote: > On Sat, 07 Apr 2012 11:06:48 -0700 > Daniel L. Miller articulated: > >> Unfortunately, the docs for the ltmp agent http://www.postfix.org/lmtp.8.html [1] don't say anything about adding these headers. I tried asking on the Postfix list - didn't get much of an answer. > > I may be wrong; however, from what I have been able to understand in > regards to the Postfix documentation, if it does not explicitly claim to > have a feature, then that feature is not available. In other words, if > it doesn't state it can do it, it can't. As I just stated on that list - even though a given feature may be documented, the possible uses of that feature may not be immediately apparent. And while the Postfix lda & virtual transports have the "flag" parameters, and the lmtp transport does not - the lmtp transport DOES have a whole slew of other parameters not available in the lda. So I was simply inquiring if there was a way to achieve my goal - given that my understanding of smtp handling in general, and Postfix in particular, are extremely limited. For some reason, I seem to irritate people with my polite questions - while others who are (in my opinion) downright rude and aggressive get assistance and acceptance. Maybe I need to start being more of a jerk on purpose... -- Daniel Links: ------ [1] http://www.postfix.org/lmtp.8.html From tss at iki.fi Mon Apr 9 09:47:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 09:47:11 +0300 Subject: [Dovecot] lmtp sometimes fails to deliver a message to all recipients In-Reply-To: <4F7C724E.8040905@talex.pl> References: <4F79C995.9060607@talex.pl> <1B1369FF-6748-46DE-AB94-AF2F32061757@iki.fi> <4F7C724E.8040905@talex.pl> Message-ID: <981BEDB0-63E3-43C1-906A-390E4BD053CA@iki.fi> On 4.4.2012, at 19.09, Artur Zaprza?a wrote: >>> lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Message was expunged (guid) >>> lmtp(3344, foo at domain): Error: RU1WMnueeU9QDQABxjIODQ: sieve: script /vmail/domain/foo/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /vmail/domain/foo/.dovecot.sieve.log may reveal additional details) >> Fixed in hg. >> > Tested with Maildir. Works great. Thanks. > > The above problem was appearing when some recipients (including first one) had a sieve filter with discard action for current message. In this case, depending on the pattern of recipients having a sieve discard action, lmtp can create more than one instance of the message for a few dozen recipients. It would be nice if lmtp could create a single hardlinked instance of the message even in this case. The problem here isn't the discard action, but that Sieve is used at all. The hard linking happens currently only for users who don't have Sieve scripts. I've a plan to fix this, but it's not a simple fix and it's pretty low priority currently. From tss at iki.fi Mon Apr 9 09:54:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 09:54:11 +0300 Subject: [Dovecot] 2.1.3: Overly lax FETCH parsing In-Reply-To: <20120405125920.Horde.wazxEIF5lbhPfeuIm_gQ4uA@bigworm.curecanti.org> References: <20120405125920.Horde.wazxEIF5lbhPfeuIm_gQ4uA@bigworm.curecanti.org> Message-ID: On 5.4.2012, at 21.59, Michael M Slusarz wrote: > While useful that Dovecot is more liberal about what it receives, 3501 seems pretty clear that incorrect FETCH parameters must return a BAD. I can verify that the above commands fail on Cyrus. It's a SHOULD, not a MUST: Servers SHOULD enforce the syntax outlined in this specification strictly. Any client command with a protocol syntax error, including (but not limited to) missing or extraneous spaces or arguments, SHOULD be rejected, and the client given a BAD server completion response. But since it's not much trouble to fix it: http://hg.dovecot.org/dovecot-2.1/rev/19e09ab09383 From tss at iki.fi Mon Apr 9 09:59:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 09:59:55 +0300 Subject: [Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups In-Reply-To: References: Message-ID: <558E2E13-E08D-46F3-B68A-0AA63DF43D38@iki.fi> On 6.4.2012, at 1.46, Joseph Tam wrote: > One other consideration (at least for me) is if the INBOX and > personal mail folders are stored in two separate FS's. It would be nice > to fuse the two sets of backups under the same namespace, but I don't > know how the namespace prefix matching works and whether you can define > hierarchical namespaces like > > namespace { > prefix = backup/inbox > location = mbox:/path/to/inbox-snapdir/%u > ... > } > > namespace { > prefix = backup/mail > location = mbox:/path/to/mail-snapdir/%u > ... > } You can define hierarchical namespaces, although they've probably not been used outside my few tests. Well, except shared// autocreated namespaces are already children to shared/ namespace, so I guess they should work. From tss at iki.fi Mon Apr 9 10:03:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:03:39 +0300 Subject: [Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups In-Reply-To: <4F7DD03B.30909@Media-Brokers.com> References: <4F7DBA3B.5090803@Media-Brokers.com> <4F7DCA4A.70009@whyscream.net> <4F7DD03B.30909@Media-Brokers.com> Message-ID: On 5.4.2012, at 20.02, Charles Marcus wrote: > On 2012-04-05 12:37 PM, Tom Hendrikx wrote: >> The first interesting point I'd see with this, is that you supply the >> mail client with a near endless supply of folders, which would take a >> lot of caching space on the clients end, either (depending on the client >> and its configuration) from the moment that you enable this fort hem, or >> after someone starts searching in their 'time machine' for some old mail. > > Since we use Thunderbird, I can of course disable offline mode for everyone, so the only time headers would be downloaded would be when the user selects (or performs a search on) one (or more) of the folders. Do they need to be accessible via Thunderbird, or maybe only via a webmail? Or perhaps a secondary (normally disabled?) TB account where you've specified a "backup/" namespace prefix (which is normally hidden)? From tss at iki.fi Mon Apr 9 10:07:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:07:40 +0300 Subject: [Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups In-Reply-To: <4F7DBA3B.5090803@Media-Brokers.com> References: <4F7DBA3B.5090803@Media-Brokers.com> Message-ID: On 5.4.2012, at 18.28, Charles Marcus wrote: > The snapshots are stored with the following filesystem layout: > > /path/to/snapshotsdir/hourly.0 > ... > /path/to/snapshotsdir/hourly.4 > /path/to/snapshotsdir/daily.0 .. > The 'names' (hourly, daily, weekly, monthly, yearly) are arbitrary (this is a bit confusing to people new to rsnapshot), and would *not* be used for displaying the mail folders to the users - it is the Date/Time stamps of each of the snapshot dirs above that would be used to display the folder names under the 'Time Machine' namespace. This is, I imagine, the part that will need some actual coding by Timo to get working - maybe just some new config variables added to the namespace code for mapping the date/time stamps of the directories to user friendly folder names in the namespace. I guess there could be kind of a "filter fs layout" that modifies the filesystem layout a bit and lets the underlying layout handle the rest: namespace { location = maildir:/path/to/snapshotsdir:LAYOUT=timestamp } Although it's annoying that it's not possible to have per-layout settings currently.. But I guess if this was implemented as plugin it would be enough to have: plugin { timestamp_layout = maildir++ } From tss at iki.fi Mon Apr 9 10:13:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:13:39 +0300 Subject: [Dovecot] Director (was: Hints for a NFS-Setup) In-Reply-To: <4F7EF67A.8060001@wk-serv.de> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F7EF67A.8060001@wk-serv.de> Message-ID: <613701AE-34D1-4366-887E-72F576A01A1D@iki.fi> On 6.4.2012, at 16.58, Patrick Westenberg wrote: > Hi again, > > I tried to setup a test invironemnt like this: > > MTA --(lmtp)--\ /--(lmtp)--> backend1 --\ > -- director -- -- NFS > MTA --(lmtp)--/ \--(lmtp)--> backend2 --/ > > > IMAP-User --> frontend1 --\ /--(imap)--> backend1 --\ > -- director -- -- NFS > IMAP-User --> frontend2 --/ \--(imap)--> backend2 --/ > > but now I'm very confused. Is it actually possible to setup a host (or two) as a director only or will I have to enable the director service on each frontend and MTA? The cleanest way to run director is to have 2 or more servers running only director itself. If you want to have less servers, it's also possible to place a Dovecot director configuration to any other servers as well, but that's conceptually more complex. For MTA you'd simply tell its LMTP client to connect to director servers, which could be one of: a) Load balancer's IP address b) Host name that expands to all directors' IP addresses. If the first one is down, the LMTP client (hopefully! verify!) connects to the second one. From tss at iki.fi Mon Apr 9 10:15:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:15:37 +0300 Subject: [Dovecot] distributed mdbox In-Reply-To: References: <1ff190f691892c01a980927422f38753@mail.airstreamcomm.net> Message-ID: <7864D800-17D2-48E3-832D-D44FA912FC5C@iki.fi> Yeah, not caching then. I know Glusterfs people implemented some fixes/workarounds to make Dovecot work better. I don't know if all of those fixes are in the public glusterfs. On 6.4.2012, at 18.39, James Devine wrote: > As it turns out I can duplicate this problem with a single dovecot server > and a single gluster server using mdbox, so maybe not caching? This being > the case I don't think director would help > > On Thu, Apr 5, 2012 at 7:16 PM, James Devine wrote: > >> >> >> On Fri, Mar 23, 2012 at 7:39 AM, wrote: >> >>> On Wed, 21 Mar 2012 09:56:12 -0600, James Devine >>> wrote: >>>> Anyone know how to setup dovecot with mdbox so that it can be used >>> through >>>> shared storage from multiple hosts? I've setup a gluster volume and am >>>> sharing it between 2 test clients. I'm using postfix/dovecot LDA for >>>> delivery and I'm using postal to send mail between 40 users. In doing >>>> this, I'm seeing these errors in the logs >>>> >>>> Mar 21 09:36:29 test-gluster-client2 dovecot: lda(testuser34): Error: >>> Fixed >>>> index file /mnt/testuser34/mdbox/storage/dovecot.map.index: >>> messages_count >>>> 272 -> 271 >>>> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: >>> Log >>>> synchronization error at seq=4,offset=3768 for >>>> /mnt/testuser28/mdbox/storage/dovecot.map.index: Append with UID 516, >>> but >>>> next_uid = 517 >>>> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: >>> Log >>>> synchronization error at seq=4,offset=4220 for >>>> /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update >>>> for invalid uid=517 >>>> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: >>> Log >>>> synchronization error at seq=4,offset=5088 for >>>> /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update >>>> for invalid uid=517 >>>> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Warning: >>>> fscking index file /mnt/testuser28/mdbox/storage/dovecot.map.index >>>> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser34): Warning: >>>> fscking index file /mnt/testuser34/mdbox/storage/dovecot.map.index >>>> >>>> >>>> This is my dovecot config currently: >>>> >>>> jdevine at test-gluster-client2:~> dovecot -n >>>> # 2.0.13: /etc/dovecot/dovecot.conf >>>> # OS: Linux 3.0.0-13-server x86_64 Ubuntu 11.10 >>>> lock_method = dotlock >>>> mail_fsync = always >>>> mail_location = mdbox:~/mdbox >>>> mail_nfs_index = yes >>>> mail_nfs_storage = yes >>>> mmap_disable = yes >>>> passdb { >>>> driver = pam >>>> } >>>> protocols = " imap" >>>> ssl_cert = >>> ssl_key = >>> userdb { >>>> driver = passwd >>>> } >>> >>> I was able to get dovecot working across a gluster cluster a few weeks ago >>> and it worked just fine. I would recommend using the native gluster mount >>> option (need to install gluster software on clients), and using >>> distributed >>> replicated as your replication mechanism. If you're running two gluster >>> servers you should have a replica count of two with distributed >>> replicated. >>> You should test first to make sure you can create a file in both mounts >>> and see it from every mount point in the cluster, as well as interact with >>> it. It's also very important to make sure your servers are running with >>> synchronized clocks from an NTP server. Very bad things happen to a >>> (dovecot or gluster) cluster out of sync with NTP. >>> >>> What storage method are you using? I'm able to produce errors within >> seconds of starting postal with more than one thread From tss at iki.fi Mon Apr 9 10:17:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:17:42 +0300 Subject: [Dovecot] POP3 dele to Trash? In-Reply-To: <4F7F85F2.3000909@corp.sonic.net> References: <20120406234001.GG89906@corp.sonic.net> <4F7F85F2.3000909@corp.sonic.net> Message-ID: <540057BA-FA72-4A34-B9C1-EBEA8458A210@iki.fi> On 7.4.2012, at 3.10, Kelsey Cummings wrote: > On 04/06/12 16:40, Kelsey Cummings wrote: >> Has anyone already done this? Should this be possible via a plugin? >> I see the deleted-to-trash imap plugin. We are using Maildir if it >> makes a difference. > > Of course, this is exactly what the Lazy Expunge plugin does, isn't it? Not exactly, the messages would go to lazyexpunge-namespace-prefix/INBOX. But maybe close enough? Otherwise would require writing a new plugin. From tss at iki.fi Mon Apr 9 10:27:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:27:04 +0300 Subject: [Dovecot] Setting ACL for master user after login In-Reply-To: References: Message-ID: <1F913824-E90D-4807-ABC4-78E119303CF0@iki.fi> On 7.4.2012, at 6.48, PL MB wrote: > I'd like to log in to normal user accounts as a master user but retain > the normal users' ACLs. > > The Master Users page on the Dovecot 1.x wiki (1) says that I can set > the master user's ACLs in a postlogin script. The documentation for > master users on the 2.x wiki (2) no longer has any statements about > master user ACLs. > > Has something important in this regard changed? Can I no longer > override the ACLs in a postlogin script? No, it's just that the ACL text was added there after wiki2 was forked. I updated now http://master.wiki2.dovecot.org/Authentication/MasterUsers#ACLs I'm pretty sure the userdb way works in v2.1, possibly also in v2.0 and probably not in v1.x. From tss at iki.fi Mon Apr 9 10:29:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:29:20 +0300 Subject: [Dovecot] Director pop-login and imap-login processes exiting on signal 11 In-Reply-To: <20120407030811.I46246@shell.xecu.net> References: <20120407030811.I46246@shell.xecu.net> Message-ID: <70AB2E58-42C9-49B6-BE13-C553893E19EF@iki.fi> On 7.4.2012, at 10.13, Andy Dills wrote: > Apr 7 02:18:05 mail-out06 dovecot: pop3-login: Fatal: master: > service(pop3-login): child 75029 killed with signal 11 (core not dumped - > set service pop3-login { drop_priv_before_exec=yes }) v2.1.3 proxying was buggy with SSL connections. Probably crashes because of that. I was supposed to release v2.1.4 already but.. From tss at iki.fi Mon Apr 9 10:33:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 10:33:18 +0300 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: <4F7D972A.8050406@Media-Brokers.com> References: <4F7D972A.8050406@Media-Brokers.com> Message-ID: On 5.4.2012, at 15.59, Charles Marcus wrote: > Does anyone know if the use of LMTP (or even the dovecot LDA) still loses the x-original-to header that the postfix vda adds and that I rely heavily on (since I use a lot of aliases), and if it does, is there any solution to get the original recipient added back in before final delivery? LMTP adds a new Delivered-To: header when there is a single RCPT TO. You can force a single RCPT TO from Postfix side by setting lmtp_destination_recipient_limit=1. LMTP doesn't add/remove/change X-Original-To: header. From stu at spacehopper.org Mon Apr 9 10:49:35 2012 From: stu at spacehopper.org (Stuart Henderson) Date: Mon, 9 Apr 2012 07:49:35 +0000 (UTC) Subject: [Dovecot] Outlook (2010) -> Dovecot (IMAP) >10x slower with high network load and many folders References: Message-ID: On 2012-04-06, Thomas von Eyben wrote: > I am seeing a >10x as slow performance when trying to complete a > "send/receive" from an Outlook 2010 client to Dovecot via IMAP, but > only when the LAN is fully loaded with other traffic, EG file copying. > It seems the problem is when outlook is trying to identify folders > that have changed since last "send/receive" thus traversing the > hierachy. Not sure why it would only affect Outlook clients, but if your switches are managed, you might like to check if flow control is enabled and, if so, try disabling it. From tss at iki.fi Mon Apr 9 11:19:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 11:19:03 +0300 Subject: [Dovecot] Director simplification? Message-ID: <71D6ACD2-8FF6-4E74-8C27-BF54DDACA3E5@iki.fi> An idea I just had: Director basically works by assigning the backend IP address by: ip = vhosts[ md5(username) mod vhosts_count ].ip The rest of director is about what happens when vhosts[] or vhosts_count changes. What about instead doing this on IP address level? ip = ip_pool[ md5(username) mod ip_pool_size ] When a backend dies, you'll reassign the backend's IPs to other backends. Each backend should have many IPs. The main restriction here is that the IP pool cannot change without stopping the entire Dovecot. But if you initially allocate enough IPs, that shouldn't be a problem. And the advantage of this over the current director? To guarantee that one director can't break others, because they don't need to communicate with each others. The disadvantage of course is that it's a little less flexible and requires more planning ahead. The IP address reassignment would also need some distro-specific scripts. This could be implemented as an alternative director-lite or something. The doveadm director status-related commands could still work with it. From tss at iki.fi Mon Apr 9 13:19:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 13:19:50 +0300 Subject: [Dovecot] v2.1.4 released Message-ID: http://dovecot.org/releases/2.1/dovecot-2.1.4.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.4.tar.gz.sig + Added mail_temp_scan_interval setting and changed its default value from 8 hours to 1 week. + Added pop3-migration plugin for easily doing a transparent IMAP+POP3 migration to Dovecot: http://wiki2.dovecot.org/Migration/Dsync + doveadm user: Added -m parameter to show some of the mail settings. - Proxying SSL connections crashed in v2.1.[23] - fts-solr: Indexing mail bodies was broken. - director: Several changes to significantly improve error handling - doveadm import didn't import messages' flags - mail_full_filesystem_access=yes was broken - Make sure IMAP clients can't create directories when accessing nonexistent users' mailboxes via shared namespace. - Dovecot auth clients authenticating via TCP socket could have failed with bogus "PID already in use" errors. From tss at iki.fi Mon Apr 9 13:19:52 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 13:19:52 +0300 Subject: [Dovecot] v2.0.20 released Message-ID: <3972C017-7380-4936-8231-AF660DEC9106@iki.fi> http://dovecot.org/releases/2.0/dovecot-2.0.20.tar.gz http://dovecot.org/releases/2.0/dovecot-2.0.20.tar.gz.sig + doveadm user: Added -m parameter to show some of the mail settings. - doveadm import didn't import messages' flags - Make sure IMAP clients can't create directories when accessing nonexistent users' mailboxes via shared namespace. - Dovecot auth clients authenticating via TCP socket could have failed with bogus "PID already in use" errors. From andy at xecu.net Mon Apr 9 13:58:33 2012 From: andy at xecu.net (Andy Dills) Date: Mon, 9 Apr 2012 06:58:33 -0400 (EDT) Subject: [Dovecot] Director pop-login and imap-login processes exiting on signal 11 In-Reply-To: <70AB2E58-42C9-49B6-BE13-C553893E19EF@iki.fi> References: <20120407030811.I46246@shell.xecu.net> <70AB2E58-42C9-49B6-BE13-C553893E19EF@iki.fi> Message-ID: <20120409065728.K14159@shell.xecu.net> On Mon, 9 Apr 2012, Timo Sirainen wrote: > On 7.4.2012, at 10.13, Andy Dills wrote: > > > Apr 7 02:18:05 mail-out06 dovecot: pop3-login: Fatal: master: > > service(pop3-login): child 75029 killed with signal 11 (core not dumped - > > set service pop3-login { drop_priv_before_exec=yes }) > > v2.1.3 proxying was buggy with SSL connections. Probably crashes because > of that. I was supposed to release v2.1.4 already but.. Thanks Timo. I can confirm this is fixed in 2.1.4. Andy --- Andy Dills Xecunet, Inc. www.xecu.net 301-682-9972 --- From CMarcus at Media-Brokers.com Mon Apr 9 15:50:42 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 09 Apr 2012 08:50:42 -0400 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: References: <4F7D972A.8050406@Media-Brokers.com> Message-ID: <4F82DB22.2050201@Media-Brokers.com> On 2012-04-09 3:33 AM, Timo Sirainen wrote: > On 5.4.2012, at 15.59, Charles Marcus wrote: > >> Does anyone know if the use of LMTP (or even the dovecot LDA) still >> loses the x-original-to header that the postfix vda adds and that I >> rely heavily on (since I use a lot of aliases), and if it does, is >> there any solution to get the original recipient added back in >> before final delivery? > LMTP adds a new Delivered-To: header when there is > a single RCPT TO. You can force a single RCPT TO from Postfix side by > setting lmtp_destination_recipient_limit=1. LMTP doesn't > add/remove/change X-Original-To: header. Ok, thanks Timo... but... Are you saying that this 'Delivered-To:' header can somehow be leveraged to provide the same info as the x-original-to header? If not, since it was the postfix virtual delivery agent that added the x-original-to, and since using lmtp means I would not be using the postfix vda, is the appropriate place to add this header in dovecot's lmtp implementation (and if so, how hard would it be)? Or would this need to be done somehow on the postfix side (if so, I'll go ask on the postfix list)? Sorry for my ignorance - but as I said, I rely on this header (I use a ton of aliases, and without it I can't see the original (alias) recipient), so I need to determine if I'm going to be able to use lmtp or not (obviously, I would much prefer to do so)... Thanks again Timo... -- Best regards, Charles From tss at iki.fi Mon Apr 9 15:53:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 15:53:56 +0300 Subject: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header In-Reply-To: <4F82DB22.2050201@Media-Brokers.com> References: <4F7D972A.8050406@Media-Brokers.com> <4F82DB22.2050201@Media-Brokers.com> Message-ID: On 9.4.2012, at 15.50, Charles Marcus wrote: >> LMTP adds a new Delivered-To: header when there is >> a single RCPT TO. You can force a single RCPT TO from Postfix side by >> setting lmtp_destination_recipient_limit=1. LMTP doesn't >> add/remove/change X-Original-To: header. > > Ok, thanks Timo... but... > > Are you saying that this 'Delivered-To:' header can somehow be leveraged to provide the same info as the x-original-to header? I guess X-Original-To is the same address as what Postfix sees as the original RCPT TO address before alias expansion and such? In that case, see my today's mail in Postfix list.. From pw at wk-serv.de Mon Apr 9 16:25:41 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Mon, 09 Apr 2012 15:25:41 +0200 Subject: [Dovecot] Director In-Reply-To: <613701AE-34D1-4366-887E-72F576A01A1D@iki.fi> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F7EF67A.8060001@wk-serv.de> <613701AE-34D1-4366-887E-72F576A01A1D@iki.fi> Message-ID: <4F82E355.8000901@wk-serv.de> Timo Sirainen schrieb: > b) Host name that expands to all directors' IP addresses. If the first one is down, the LMTP client (hopefully! verify!) connects to the second one. Yes, Postfix' LMTP works fine with a hostname expanding to two IPs. From pw at wk-serv.de Mon Apr 9 16:35:01 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Mon, 09 Apr 2012 15:35:01 +0200 Subject: [Dovecot] Problem with quota warning Message-ID: <4F82E585.7030201@wk-serv.de> Hi again, my test environment is working now and finally I tried to implement my quota warning but it is not working. LDA says the mailbox is full even it is not (only 81% and that is why the quota warning is executed). Further I used :noenforcing in my script. Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: Namespace INBOX.: Using permissions from /var/mail/k-team.info/test/mdbox: mode=0700 gid=-1 Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: Quota root: name=User quota backend=dict args=:file://mdbox/dovecot-quota:noenforcing Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: Quota warning: bytes=0 (90%) messages=0 reverse=no command=quota-warning 90 raw mail user Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: Quota warning: bytes=0 (75%) messages=0 reverse=no command=quota-warning 75 raw mail user Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: dict quota: user=raw mail user, uri=file://mdbox/dovecot-quota:noenforcing, noenforcing=0 Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: none: root=, index=, control=, inbox=, alt= Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): Debug: Destination address: test at k-team.info (source: user at hostname) Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): msgid=unspecified: save failed to INBOX: Quota exceeded (mailbox for user is full) Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): msgid=unspecified: rejected: Quota exceeded (mailbox for user is full) Apr 9 15:08:21 backend01 dovecot: lda(test at k-team.info): msgid=: Return-Path missing, rejection reason: Quota exceeded (mailbox for user is full) # 2.1.3: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.4 auth_debug = yes auth_debug_passwords = yes auth_verbose = yes listen = 172.17.99.96 mail_debug = yes mail_gid = vmail mail_home = /var/mail/%d/%n mail_location = mdbox:~/mdbox:INDEX=/var/indexes/%d/%n mail_plugins = quota mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { autocreate = INBOX.Sent autocreate2 = INBOX.Trash autocreate3 = INBOX.Drafts autocreate4 = INBOX.Spamverdacht autosubscribe = INBOX.Sent autosubscribe2 = INBOX.Trash autosubscribe3 = INBOX.Drafts autosubscribe4 = INBOX.Spamverdacht quota = dict:User quota::file:%h/mdbox/dovecot-quota quota_rule2 = INBOX.Trash:storage=+100M quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=75%% quota-warning 75 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_global_path = /usr/local/etc/dovecot/sieve/default.sieve } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { user = vmail } } service imap-login { service_count = 1 } service lmtp { inet_listener lmtp { address = 172.17.99.96 port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 } service quota-warning { executable = script /usr/local/etc/dovecot/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = required ssl_cert = References: <4F7D972A.8050406@Media-Brokers.com> <4F82DB22.2050201@Media-Brokers.com> Message-ID: <4F82E7CB.6010206@Media-Brokers.com> On 2012-04-09 8:53 AM, Timo Sirainen wrote: > I guess X-Original-To is the same address as what Postfix sees as the > original RCPT TO address before alias expansion and such? In that > case, see my today's mail in Postfix list. Yep... and hoping that you and Wietse can work out some way to support it... Thanks for participating in the discussion over there... :) -- Best regards, Charles From lists at luigirosa.com Mon Apr 9 16:44:46 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Mon, 09 Apr 2012 15:44:46 +0200 Subject: [Dovecot] mount Message-ID: <4F82E7CE.3070502@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I have a Dovecot installation on CentOS 5 where I sometimes mount external fs in /mnt Every Dovecot data is in local / file system, nothing is mounted elswhere After upgrading to 1.2.4 I rebooted the system for other reasons and at startup I got this on Dovecot log: master: Warning: /mnt is no longer mounted. If this is intentional, remove it with doveadm mount No /mnt entry in /etc/fstab and nothing pmounted under /mnt I THINK that the last time I used /mnt to mount something was few weeks ago to update VMware tools. Is there a way toi tell Dovecot to ignore /mnt ? Ciao, luigi - -- / +--[Luigi Rosa]-- \ $100 invested at 7% interest for 100 years will become $100,000, at which time it will be worth absolutely nothing. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk+C58gACgkQ3kWu7Tfl6ZRc0wCgl0Z4OtblYbfYwzvOp1/vUifV PqYAoIvfltvmq3cijvDbOEKV2Tai2rpu =hyrI -----END PGP SIGNATURE----- From =?UTF-8?Q?=D0=91=D1=80=D0=B0=D0=BD=D0=BA=D0=BE_=D0=9C=D0=B0=D1=98?= Mon Apr 9 16:51:49 2012 From: =?UTF-8?Q?=D0=91=D1=80=D0=B0=D0=BD=D0=BA=D0=BE_=D0=9C=D0=B0=D1=98?= (=?UTF-8?Q?=D0=91=D1=80=D0=B0=D0=BD=D0=BA=D0=BE_=D0=9C=D0=B0=D1=98?=) Date: Mon, 09 Apr 2012 15:51:49 +0200 Subject: [Dovecot] Username from rfc822Name subject alternative name Message-ID: Hello, I'm looking into adding support for extracting the username from client certificate's rfc822Name (from the subjectAltName extension). The question I have is what would be the best approach to do this? Current implementation has a kind of clean code since it just goes through the subject name, extracting the values with X509_NAME_get_text_by_NID (while NID is obtained with OBJ_txt2nid). If I were to add this, it's bound to make the code a little bit more complicated since SAN's can't be retrieved in the same way. So far in terms of options I have, I can see the following: 1. Create a distinct configuration option for the ssl_cert_username_field (i.e. specify something like "sanrfc822Name" to have Dovecot extract the username from the designated alternative name). 2. Make the current code fail-over to rfc822Name SAN if emailAddress is provided for ssl_cert_username (less invasion in code, but less flexibility as well). Any input/recommendation/directioning is welcome. I've wanted to actually first write a patch, and then submit it, but I think it might be better to check what would be preferable by Dovecot maintainers/devs. Best regards -- Branko Majic Jabber: branko at majic.rs Please use only Free formats when sending attachments to me. ?????? ????? ?????: branko at majic.rs ????? ??? ?? ??????? ?????? ????????? ? ????????? ?????????. From tss at iki.fi Mon Apr 9 16:57:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 16:57:48 +0300 Subject: [Dovecot] mount In-Reply-To: <4F82E7CE.3070502@luigirosa.com> References: <4F82E7CE.3070502@luigirosa.com> Message-ID: On 9.4.2012, at 16.44, Luigi Rosa wrote: > I have a Dovecot installation on CentOS 5 where I sometimes mount external fs > in /mnt > > Every Dovecot data is in local / file system, nothing is mounted elswhere .. > Is there a way toi tell Dovecot to ignore /mnt ? doveadm mount add /mnt ignore From lists at luigirosa.com Mon Apr 9 16:59:05 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Mon, 09 Apr 2012 15:59:05 +0200 Subject: [Dovecot] mount In-Reply-To: References: <4F82E7CE.3070502@luigirosa.com> Message-ID: <4F82EB29.8030709@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Timo Sirainen said the following on 09/04/12 15:57: >> Is there a way toi tell Dovecot to ignore /mnt ? > doveadm mount add /mnt ignore Thanks, next time I will RTFM first. Ciao, luigi - -- / +--[Luigi Rosa]-- \ [UNIX] n. A random quote, item of trivia, joke or maxim printed to the user's tty at login time or (less commonly) at logout time. Items from this jargon file have often been used as fortune cookies. --Jargon File -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk+C6ykACgkQ3kWu7Tfl6ZQI6QCgt4E3Imx1OeaB6SqjIjWDhjS0 xqUAoKizTRivIkvKkQE5SS7zwCtPlL9B =RCsz -----END PGP SIGNATURE----- From achekalin at lazurit.com Mon Apr 9 17:58:23 2012 From: achekalin at lazurit.com (Alexander Chekalin) Date: Mon, 09 Apr 2012 17:58:23 +0300 Subject: [Dovecot] Multiply mailboxes vs one huge Message-ID: <4F82F90F.8020902@lazurit.com> Hello, as I need to store a lot of messages on my IMAP server (order of 900K-1000K; this is an archive for some time, maybe a year or so), I see some "slowness" in dealing with such a huge amount. I mainly need to do searches like "get all messages from user1 at domain1.com to user2 at domain2.tld recieved between date1 and date2". So I really interested will it be wise to a) split all messages into several smaller mailboxes (per-month, or per-day, or create 2-level-structure like "month/day/") b) use dbox (vs currently used mbox) storage scheme (I'm afraid of mdbox as I still not sure I'll be able to parse it by scripts later "just in case") Dovecot is the latest one (2.1.3). No compression Dovecot-side, but it mails are in zfs volume with compression on. I ask this mainly due to my not fully understand how Dovecot indexes are working. I also test another approach: to use my own index somewhere outside Dovecot which will store reference between emails and UIDs, and dates and UIDs, so I'll simple query my index for things I need. But then, that's exactly what IMAP index can do, so I simple slow my search down, isn't it? The only reason I think about my own index is I won't use 'all header' as search scope, I need to deal only with From:, To:, Cc:, Bcc: (if any), Recieved (if nowehere else I see the from/to info), and date field(s) - I doubt IMAP will care for that for me. Yours, Alexander From tss at iki.fi Mon Apr 9 18:03:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 9 Apr 2012 18:03:38 +0300 Subject: [Dovecot] Multiply mailboxes vs one huge In-Reply-To: <4F82F90F.8020902@lazurit.com> References: <4F82F90F.8020902@lazurit.com> Message-ID: On 9.4.2012, at 17.58, Alexander Chekalin wrote: > as I need to store a lot of messages on my IMAP server (order of 900K-1000K; this is an archive for some time, maybe a year or so), I see some "slowness" in dealing with such a huge amount. I mainly need to do searches like "get all messages from user1 at domain1.com to user2 at domain2.tld recieved between date1 and date2". So by "received between date" you mean the IMAP INTERNALDATE as opposed to Date: header? These kind of searches are looked up from the index/cache files, and the performance should be exactly the same with all of the mailbox formats. It would be useful to figure out what exactly is causing the slowness. Is the SEARCH command slow? Something else? Is the slowness about user CPU, system CPU or disk IO? From andre.rodier at gmail.com Mon Apr 9 19:26:50 2012 From: andre.rodier at gmail.com (Andre Rodier) Date: Mon, 9 Apr 2012 17:26:50 +0100 Subject: [Dovecot] per user sieve after filters Message-ID: <20120409162650.GA29690@london.sagso.home> Hello, Thanks for dovecot, as it's still the best mail server. I'd like to use per users sieve_after scripts. Can I put in my dovecot config file, something like that: sieve_after = %h/Mails/Sieve/After/ It would be very useful for me, as I'd like to add vacation script to be executed from this place. Kind regards, Andr?. From stephan at rename-it.nl Mon Apr 9 21:01:42 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 09 Apr 2012 20:01:42 +0200 Subject: [Dovecot] per user sieve after filters In-Reply-To: <20120409162650.GA29690@london.sagso.home> References: <20120409162650.GA29690@london.sagso.home> Message-ID: <4F832406.8060108@rename-it.nl> On 4/9/2012 6:26 PM, Andre Rodier wrote: > Hello, > > Thanks for dovecot, as it's still the best mail server. > > I'd like to use per users sieve_after scripts. > > Can I put in my dovecot config file, something like that: > > sieve_after = %h/Mails/Sieve/After/ > > It would be very useful for me, as I'd like to add vacation script to be > executed from this place. I must say I've never tested something like that, but it should work. Regards, Stephan. From centos.admin at gmail.com Mon Apr 9 22:15:02 2012 From: centos.admin at gmail.com (Emmanuel Noobadmin) Date: Tue, 10 Apr 2012 03:15:02 +0800 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F81D73B.9030901@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> Message-ID: On 4/9/12, Stan Hoeppner wrote: > So it seems you have two courses of action: > 1. Identify individual current choke points and add individual systems > and storage to eliminate those choke points. > > 2. Analyze your entire workflow and all systems, identifying all choke > points, then design a completely new well integrated storage > architecture that solves all current problems and addresses future needs I started to do this and realize I have a serious mess on hand that makes delving in other people's uncommented source code seem like a joy :D Management added to this by deciding if we're going to offload the email storage to a network storage, we might as well consolidate everything into that shared storage system so we don't have TBs of un-utilized space. So I might not even be able to use your tested XFS + concat solution since it may not be optimal for VM images and databases. As the requirements' changed, I'll stop asking here as it's no longer really relevant just for Dovecot purposes. > You are a perfect candidate for VMware ESX. The HA feature will do > exactly what you want. If one physical node in the cluster dies, HA > automatically restarts the dead VMs on other nodes, transparently. > Clients will will have to reestablish connections, but everything else > will pretty much be intact. Worse case scenario will possibly be a few > corrupted mailboxes that were being written when the hardware crashed. > A SAN is required for such a setup. Thanks for the suggestion, I will need to find some time to look into this although we've mostly been using KVM for virtualization so far. Although the "SAN" part will probably prevent this from being accepted due to cost. >> My lame excuse is that I'm just the web >> dev who got caught holding the server admin potato. > > Baptism by fire. Ouch. What doesn't kill you makes you stronger. ;) True, but I'd hate to be the customer who get to pick up the pieces when things explode due to unintended negligence by a dev trying to level up by multi-classing as an admin. > physical network interface. You can do some of these things with free > Linux hypervisors, but AFAIK the poor management interfaces for them > make the price of ESX seem like a bargain. Unfortunately, the usual kind of customers we have here, spending that kind of budget isn't justifiable. The only reason we're providing email services is because customers wanted freebies and they felt there was no reason why we can't give them emails on our servers, they are all "servers" after all. So I have to make do with OTS commodity parts and free software for the most parts. From achekalin at lazurit.com Mon Apr 9 22:39:40 2012 From: achekalin at lazurit.com (Alexander Chekalin) Date: Mon, 09 Apr 2012 22:39:40 +0300 Subject: [Dovecot] Multiply mailboxes vs one huge In-Reply-To: References: <4F82F90F.8020902@lazurit.com> Message-ID: <4F833AFC.60301@lazurit.com> Hello, Timo, I feel a bit unsure about "which 'date' I mean", since I always consider the only date from Date: header. But which value is used as INTERNALDATE then? As soon as I use (for now) maildir storage type, all the metadata are stored in messages. So I expect Dovecot somehow parse and use Date: field itself, or I'm wrong with it? And also what's about messages without Date header at all? But the Date isn't the worst thing. Look, to have my archive work I setup server-side filter which redirect all messages it processed also to my archive mailbox. This way, each message (after such a redirect) targeted to 'archive at mydomain', instead of its original destination email. The only place I can find out the original recipient is to parse 'Recieved' field(-s). As I think I understand that none of these headers (Date or Received) are to be used for SEARCH anyway, and this was the idea behind creating my own index. But wait, is there any way I can make Dovecot also index additional fields (yes, I talk about 'Received') - then it'll be the best solution! Thank you, Timo, for your work, yours, Alexander 09.04.2012 18:03, Timo Sirainen ???????: > On 9.4.2012, at 17.58, Alexander Chekalin wrote: > >> as I need to store a lot of messages on my IMAP server (order of 900K-1000K; this is an archive for some time, maybe a year or so), I see some "slowness" in dealing with such a huge amount. I mainly need to do searches like "get all messages from user1 at domain1.com to user2 at domain2.tld recieved between date1 and date2". > So by "received between date" you mean the IMAP INTERNALDATE as opposed to Date: header? These kind of searches are looked up from the index/cache files, and the performance should be exactly the same with all of the mailbox formats. It would be useful to figure out what exactly is causing the slowness. Is the SEARCH command slow? Something else? Is the slowness about user CPU, system CPU or disk IO? > From marc at perkel.com Tue Apr 10 04:28:32 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 09 Apr 2012 18:28:32 -0700 Subject: [Dovecot] v2.1.4 released - broken In-Reply-To: References: Message-ID: <4F838CC0.2000900@perkel.com> I'm seeing this immediately after upgrading from 2.1.3 Apr 09 18:22:43 imap(chris at powerpage.org): Error: user chris at powerpage.org: Initialization failed: Initializing mail storage from mail_location setting failed: Home directory not set for user. Can't expand ~/ for mail root dir in: /vhome/powerpage.org/home/chris:INDEX=/email/imap-cache/powerpage.org-chris mail_location = maildir:/vhome/%d/home/%n:INDEX=/email/imap-cache/%d-%n From a.kostyrev at serverc.ru Tue Apr 10 05:37:27 2012 From: a.kostyrev at serverc.ru (=?koi8-r?B?68/T1NnSxdcg4czFy9PBzsTSIOHMxcvTxcXXyd4=?=) Date: Tue, 10 Apr 2012 13:37:27 +1100 Subject: [Dovecot] Authentication mechanism and Password scheme Message-ID: <213B51F00051AE48A9F0E112880177178F794E@Delta.sc.local> Good day! I'm just trying to figure out that my understanding of subject is correct. So, if I want to store passwords in my database encrypted with SSHA512 scheme, my only choice for Authentication mechanism is plaintext? Thanks in advance! -- ? ?????????, ???????? ????????? ????????? ????????????? ??? "??????-?????" ???.: (423) 262-02-62 (???. 2037) ????: (423) 262-02-10 From stan at hardwarefreak.com Tue Apr 10 08:00:19 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 10 Apr 2012 00:00:19 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> Message-ID: <4F83BE63.9050603@hardwarefreak.com> On 4/9/2012 2:15 PM, Emmanuel Noobadmin wrote: > Unfortunately, the usual kind of customers we have here, spending that > kind of budget isn't justifiable. The only reason we're providing > email services is because customers wanted freebies and they felt > there was no reason why we can't give them emails on our servers, they > are all "servers" after all. > > So I have to make do with OTS commodity parts and free software for > the most parts. OTS meaning you build your own systems from components? Too few in the business realm do so today. :( It sounds like budget overrides redundancy then. You can do an NFS cluster with SAN and GFS2, or two servers with their own storage and DRBD mirroring. Here's how to do the latter: http://www.howtoforge.com/high_availability_nfs_drbd_heartbeat The total cost is about the same for each solution as an iSCSI SAN array of drive count X is about the same cost as two JBOD disk arrays of count X*2. Redundancy in this case is expensive no matter the method. Given how infrequent host failures are, and the fact your storage is redundant, it may make more sense to simply keep spare components on hand and swap what fails--PSU, mobo, etc. Interestingly, I designed a COTS server back in January to handle at least 5k concurrent IMAP users, using best of breed components. If you or someone there has the necessary hardware skills, you could assemble this system and simply use it for NFS instead of Dovecot. The parts list: secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=17069985 In case the link doesn't work, the core components are: SuperMicro H8SGL G34 mobo w/dual Intel GbE, 2GHz 8-core Opteron 32GB Kingston REG ECC DDR3, LSI 9280-4i4e, Intel 24 port SAS expander 20 x 1TB WD RE4 Enterprise 7.2K SATA2 drives NORCO RPC-4220 4U 20 Hot-Swap Bays, SuperMicro 865W PSU All other required parts are in the Wish List. I've not written assembly instructions. I figure anyone who would build this knows what s/he is doing. Price today: $5,376.62 Configuring all 20 drives as a RAID10 LUN in the MegaRAID HBA would give you a 10TB net Linux device and 10 stripe spindles of IOPS and bandwidth. Using RAID6 would yield 18TB net and 18 spindles of read throughput, however parallel write throughput will be at least 3-6x slower than RAID10, which is why nobody uses RAID6 for transactional workloads. If you need more transactional throughput you could use 20 WD6000HLHX 600GB 10K RPM WD Raptor drives. You'll get 40% more throughput and 6TB net space with RAID10. They'll cost you $1200 more, or $6,576.62 total. Well worth the $1200 for 40% more throughput, if 6TB is enough. Both of the drives I've mentioned here are enterprise class drives, feature TLER, and are on the LSI MegaRAID SAS hardware compatibility list. The price of the 600GB Raptor has come down considerably since I designed this system, or I'd have used them instead. Anyway, lots of option out there. But $6,500 is pretty damn cheap for a quality box with 32GB RAM, enterprise RAID card, and 20x10K RPM 600GB drives. The MegaRAID 9280-4i4e has an external SFF8088 port For an additional $6,410 you could add an external Norco SAS expander JBOD chassis and 24 more 600GB 10K RPM Raptors, for 13.2TB of total net RAID10 space, and 22 10k spindles of IOPS performance from 44 total drives. That's $13K for a 5K random IOPS, 13TB, 44 drive NFS RAID COTS server solution, $1000/TB, $2.60/IOPS. Significantly cheaper than an HP, Dell, IBM solution of similar specs, each of which will set you back at least 20 large. Note the chassis I've spec'd have single PSUs, not the dual or triple redundant supplies you'll see on branded hardware. With a relatively stable climate controlled environment and a good UPS with filtering, quality single supplies are fine. In fact, in the 4U form factor single supplies are usually more reliable due to superior IC packaging and airflow through the heatsinks, not to mention much quieter. -- Stan From lists at wiesinger.com Tue Apr 10 08:46:31 2012 From: lists at wiesinger.com (Gerhard Wiesinger) Date: Tue, 10 Apr 2012 07:46:31 +0200 Subject: [Dovecot] v2.1.4 released - broken In-Reply-To: <4F838CC0.2000900@perkel.com> References: <4F838CC0.2000900@perkel.com> Message-ID: <4F83C937.1080707@wiesinger.com> On 10.04.2012 03:28, Marc Perkel wrote: > > I'm seeing this immediately after upgrading from 2.1.3 > > Apr 09 18:22:43 imap(chris at powerpage.org): Error: user > chris at powerpage.org: Initialization failed: Initializing mail storage > from mail_location setting failed: Home directory not set for user. > Can't expand ~/ for mail root dir in: > /vhome/powerpage.org/home/chris:INDEX=/email/imap-cache/powerpage.org-chris > > mail_location = maildir:/vhome/%d/home/%n:INDEX=/email/imap-cache/%d-%n > I'm guessing this occurs from the following bugfix: http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e Reverting this patch helps? What is your config (doveconf -n)? Ciao, Gerhard From centos.admin at gmail.com Tue Apr 10 09:09:18 2012 From: centos.admin at gmail.com (Emmanuel Noobadmin) Date: Tue, 10 Apr 2012 14:09:18 +0800 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F83BE63.9050603@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> Message-ID: On 4/10/12, Stan Hoeppner wrote: >> So I have to make do with OTS commodity parts and free software for >> the most parts. > > OTS meaning you build your own systems from components? Too few in the > business realm do so today. :( For the inhouse stuff and budget customers yes, in fact both the email servers are on seconded hardware that started life as something else. I spec HP servers for our app servers to customers who are willing to pay for their own colocated or onsite servers but still there are customers who balk at the cost and so go OTS or virtualized. > SuperMicro H8SGL G34 mobo w/dual Intel GbE, 2GHz 8-core Opteron > 32GB Kingston REG ECC DDR3, LSI 9280-4i4e, Intel 24 port SAS expander > 20 x 1TB WD RE4 Enterprise 7.2K SATA2 drives > NORCO RPC-4220 4U 20 Hot-Swap Bays, SuperMicro 865W PSU > All other required parts are in the Wish List. I've not written > assembly instructions. I figure anyone who would build this knows what > s/he is doing. > > Price today: $5,376.62 This price looks like something I might be able to push through although I'll probably have to go SATA instead of SAS due to cost of keeping spares. > Configuring all 20 drives as a RAID10 LUN in the MegaRAID HBA would give > you a 10TB net Linux device and 10 stripe spindles of IOPS and > bandwidth. Using RAID6 would yield 18TB net and 18 spindles of read > throughput, however parallel write throughput will be at least 3-6x > slower than RAID10, which is why nobody uses RAID6 for transactional > workloads. Not likely to go with RAID 5 or 6 due to concerns about the uncorrectable read errors risks on rebuild with large arrays. Is the MegaRAID being used as the actual RAID controller or just as a HBA? I have been avoiding hardware RAID because of a really bad experience with RAID 5 on an obsolete controller that eventually died without replacement and couldn't be recovered. Since then, it's always been RAID 1 and, after I discovered mdraid, using them as purely HBA with mdraid for the flexibility of being able to just pull the drives into a new system if necessary without having to worry about the controller. > Both of the drives I've mentioned here are enterprise class drives, > feature TLER, and are on the LSI MegaRAID SAS hardware compatibility > list. The price of the 600GB Raptor has come down considerably since I > designed this system, or I'd have used them instead. > > Anyway, lots of option out there. But $6,500 is pretty damn cheap for a > quality box with 32GB RAM, enterprise RAID card, and 20x10K RPM 600GB > drives. > > The MegaRAID 9280-4i4e has an external SFF8088 port For an additional > $6,410 you could add an external Norco SAS expander JBOD chassis and 24 > more 600GB 10K RPM Raptors, for 13.2TB of total net RAID10 space, and 22 > 10k spindles of IOPS performance from 44 total drives. That's $13K for > a 5K random IOPS, 13TB, 44 drive NFS RAID COTS server solution, > $1000/TB, $2.60/IOPS. Significantly cheaper than an HP, Dell, IBM > solution of similar specs, each of which will set you back at least 20 > large. Would this setup work well too for serving up VM images? I've been trying to find a solution for the virtualized app servers images as well but the distributed FSes currently are all bad with random reads/writes it seems. XFS seem to be good with large files like db and vm images with random internal write/read so given my time constraints, it would be nice to have a single configuration that works generally well for all the needs I have to oversee. > Note the chassis I've spec'd have single PSUs, not the dual or triple > redundant supplies you'll see on branded hardware. With a relatively > stable climate controlled environment and a good UPS with filtering, > quality single supplies are fine. In fact, in the 4U form factor single > supplies are usually more reliable due to superior IC packaging and > airflow through the heatsinks, not to mention much quieter. Same reason I do my best to avoid 1U servers, the space/heat issues worries me. Yes, I'm guilty of worrying too much but that had saved me on several occasions. From tss at iki.fi Tue Apr 10 10:10:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 10:10:28 +0300 Subject: [Dovecot] v2.1.4 released - broken In-Reply-To: <4F838CC0.2000900@perkel.com> References: <4F838CC0.2000900@perkel.com> Message-ID: On 10.4.2012, at 4.28, Marc Perkel wrote: > I'm seeing this immediately after upgrading from 2.1.3 > > Apr 09 18:22:43 imap(chris at powerpage.org): Error: user chris at powerpage.org: Initialization failed: Initializing mail storage from mail_location setting failed: Home directory not set for user. Can't expand ~/ for mail root dir in: /vhome/powerpage.org/home/chris:INDEX=/email/imap-cache/powerpage.org-chris > > mail_location = maildir:/vhome/%d/home/%n:INDEX=/email/imap-cache/%d-%n Now would be a good time to add home directories for users. :) Or apply: http://hg.dovecot.org/dovecot-2.1/rev/20e1aa322b1e From tss at iki.fi Tue Apr 10 10:11:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 10:11:05 +0300 Subject: [Dovecot] Authentication mechanism and Password scheme In-Reply-To: <213B51F00051AE48A9F0E112880177178F794E@Delta.sc.local> References: <213B51F00051AE48A9F0E112880177178F794E@Delta.sc.local> Message-ID: On 10.4.2012, at 5.37, ???????? ????????? ?????????? wrote: > Good day! > I'm just trying to figure out that my understanding of subject is correct. > > So, if I want to store passwords in my database encrypted with SSHA512 scheme, > my only choice for Authentication mechanism is plaintext? Yeah, that's correct. From tss at iki.fi Tue Apr 10 10:16:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 10:16:47 +0300 Subject: [Dovecot] Multiply mailboxes vs one huge In-Reply-To: <4F833AFC.60301@lazurit.com> References: <4F82F90F.8020902@lazurit.com> <4F833AFC.60301@lazurit.com> Message-ID: On 9.4.2012, at 22.39, Alexander Chekalin wrote: > Hello, Timo, > > I feel a bit unsure about "which 'date' I mean", since I always consider the only date from Date: header. But which value is used as INTERNALDATE then? As soon as I use (for now) maildir storage type, all the metadata are stored in messages. So I expect Dovecot somehow parse and use Date: field itself, or I'm wrong with it? The INTERNALDATE means the same as "received date", while the Date: header is the "sent date". With mbox format the received date is stored in the separating From-lines. IMAP supports searching and sorting messages by either INTERNALDATE or Date: header > And also what's about messages without Date header at all? The searching just doesn't match those messages then. Sorting falls back to using received date. > But the Date isn't the worst thing. Look, to have my archive work I setup server-side filter which redirect all messages it processed also to my archive mailbox. This way, each message (after such a redirect) targeted to 'archive at mydomain', instead of its original destination email. The only place I can find out the original recipient is to parse 'Recieved' field(-s). > > As I think I understand that none of these headers (Date or Received) are to be used for SEARCH anyway, and this was the idea behind creating my own index. But wait, is there any way I can make Dovecot also index additional fields (yes, I talk about 'Received') - then it'll be the best solution! If you do a SEARCH HEADER Received, then Dovecot adds the Received headers to dovecot.index.cache file and the subsequent searches should be quite fast, although the Received headers increase the cache file's size quite a lot. Also alternatively you can enable full text search indexes (Lucene or Solr) and the search is then done from them. From adrian.minta at gmail.com Tue Apr 10 13:22:18 2012 From: adrian.minta at gmail.com (Adrian Minta) Date: Tue, 10 Apr 2012 13:22:18 +0300 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F83BE63.9050603@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> Message-ID: <4F8409DA.1090202@gmail.com> On 04/10/12 08:00, Stan Hoeppner wrote: > Interestingly, I designed a COTS server back in January to handle at > least 5k concurrent IMAP users, using best of breed components. If you > or someone there has the necessary hardware skills, you could assemble > this system and simply use it for NFS instead of Dovecot. The parts > list: > secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=17069985 Don't forget the Battery Backup Unit for RAID card !!! From pw at wk-serv.de Tue Apr 10 13:44:08 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 10 Apr 2012 12:44:08 +0200 Subject: [Dovecot] Problem with quota warning In-Reply-To: <4F82E585.7030201@wk-serv.de> References: <4F82E585.7030201@wk-serv.de> Message-ID: <4F840EF8.7080109@wk-serv.de> Hi, I switched from dovecot-lda to lmtp for my quota warnings but there is something wrong with the parameters: /usr/local/libexec/dovecot/lmtp -d $USER -o "plugin/quota=dict:User quota::file:%h/mdbox/dovecot-quota:noenforcing" /usr/local/libexec/dovecot/lmtp: invalid option -- 'd' /usr/local/libexec/dovecot/lmtp -r $USER -o "plugin/quota=dict:User quota::file:%h/mdbox/dovecot-quota:noenforcing" /usr/local/libexec/dovecot/lmtp: invalid option -- 'r' Both parameters are listed on http://wiki2.dovecot.org/LMTP. So is this my fault or wrong information on the page? Patrick From tss at iki.fi Tue Apr 10 15:31:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 15:31:13 +0300 Subject: [Dovecot] Problem with quota warning In-Reply-To: <4F840EF8.7080109@wk-serv.de> References: <4F82E585.7030201@wk-serv.de> <4F840EF8.7080109@wk-serv.de> Message-ID: <3F52251E-6411-448E-B397-207EFD176478@iki.fi> On 10.4.2012, at 13.44, Patrick Westenberg wrote: > I switched from dovecot-lda to lmtp for my quota warnings but there is something wrong with the parameters: > > /usr/local/libexec/dovecot/lmtp -d $USER -o "plugin/quota=dict:User quota::file:%h/mdbox/dovecot-quota:noenforcing" This is not how to use LMTP. You need to tell your MTA to connect to it via UNIX socket or via TCP. You don't execute the LMTP. > Both parameters are listed on http://wiki2.dovecot.org/LMTP. > So is this my fault or wrong information on the page? It says "Compared to dovecot-lda parameters", so it's describing where dovecot-lda parameters are taken with LMTP. I guess it could be said in some clearer way, but I don't really know how. From pw at wk-serv.de Tue Apr 10 15:54:45 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 10 Apr 2012 14:54:45 +0200 Subject: [Dovecot] Problem with quota warning In-Reply-To: <3F52251E-6411-448E-B397-207EFD176478@iki.fi> References: <4F82E585.7030201@wk-serv.de> <4F840EF8.7080109@wk-serv.de> <3F52251E-6411-448E-B397-207EFD176478@iki.fi> Message-ID: <4F842D95.3070005@wk-serv.de> Timo Sirainen schrieb: > On 10.4.2012, at 13.44, Patrick Westenberg wrote: > >> I switched from dovecot-lda to lmtp for my quota warnings but there is something wrong with the parameters: >> >> /usr/local/libexec/dovecot/lmtp -d $USER -o "plugin/quota=dict:User quota::file:%h/mdbox/dovecot-quota:noenforcing" > > This is not how to use LMTP. You need to tell your MTA to connect to it via UNIX socket or via TCP. You don't execute the LMTP. There is no MTA installed on this system. This is the backend server. dovecot-lda doesn't work too, look at my first post with this subject. From tss at iki.fi Tue Apr 10 15:58:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 15:58:16 +0300 Subject: [Dovecot] Problem with quota warning In-Reply-To: <4F82E585.7030201@wk-serv.de> References: <4F82E585.7030201@wk-serv.de> Message-ID: <28D02445-BBDC-41F5-A7AA-2AD9C5E5093F@iki.fi> On 9.4.2012, at 16.35, Patrick Westenberg wrote: > cat << EOF | /usr/local/libexec/dovecot/dovecot-lda -d $USER -o "plugin/quota=dict:User quota::file:%h/mdbox/dovecot-quota:noenforcing" The :noenforcing isn't in the right location. dict:User quota::noenforcing:file:%h/... From pw at wk-serv.de Tue Apr 10 16:26:11 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 10 Apr 2012 15:26:11 +0200 Subject: [Dovecot] Problem with quota warning In-Reply-To: <28D02445-BBDC-41F5-A7AA-2AD9C5E5093F@iki.fi> References: <4F82E585.7030201@wk-serv.de> <28D02445-BBDC-41F5-A7AA-2AD9C5E5093F@iki.fi> Message-ID: <4F8434F3.3080708@wk-serv.de> Timo Sirainen schrieb: > The :noenforcing isn't in the right location. > > dict:User quota::noenforcing:file:%h/... Aww.. thank you! From luc at bigdata.nb.ca Tue Apr 10 16:45:13 2012 From: luc at bigdata.nb.ca (Luc Belliveau) Date: Tue, 10 Apr 2012 10:45:13 -0300 Subject: [Dovecot] old messages being delivered to blackberry Message-ID: <4F843969.6040806@bigdata.nb.ca> Several users are reporting old messages (sometime years old) are being delivered as new on their blackberry devices... I've confirmed that this happens to all blackberry users... I believe the problem is related to this error: Maildir: Expunged message reappeared, giving a new UID In fact, this is the only mail related error message on the server... the mail is stored on a glusterfs share, and I'm using director to prevent simultaneous access... although I wasn't able to get LMTP to work with Director... to simplify things, I've temporarily only enabled 1 dovecot server anyway, to get everything else working first... I'll fiddle with lmtp later... so right now only 1 server is accessing the mailboxes. Delivery is also done from that one server, from postfix. Possibly caused by GlusterFS, the indexes were still getting corrupted, even with only 1 server... so I've moved the indexes to a local directory on each server. I'm no longer getting index related errors... but the expunged error is still occuring... and the error seems to happen a few moments before the BB gets a bunch of messages... so it's very likely related. I would like to continue using GlusterFS if possible; I haven't found any indication that it is outright unsupported, as long as I'm using Director... I am using dovecot 2.0.16. Any help would be appreciated! Here is my configuration: # 2.0.16: /etc/dovecot/dovecot.conf # OS: Linux 3.1.9-1.4-desktop i686 openSUSE 12.1 (i586) auth_mechanisms = plain login disable_plaintext_auth = no log_timestamp = "%Y-%m-%d %H:%M:%S " mail_fsync = always mail_location = maildir:~/Maildir:INDEX=/var/indexes/%u mail_nfs_storage = yes mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve = /var/vmail/%d/%n/.sieve } postmaster_address = postmaster at bigdata.nb.ca service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } user = root } service lmtp { inet_listener lmtp { port = 24 } unix_listener /var/spool/postfix/private/dovecot-lmtp { mode = 0660 user = postfix } } ssl_ca = References: <4F843969.6040806@bigdata.nb.ca> Message-ID: <5CDED2FA-3654-4378-A2D3-8A0BFAF7B5CC@iki.fi> On 10.4.2012, at 16.45, Luc Belliveau wrote: > Several users are reporting old messages (sometime years old) are being delivered as new on their blackberry devices... I've confirmed that this happens to all blackberry users... I believe the problem is related to this error: > > Maildir: Expunged message reappeared, giving a new UID This means: 1. readdir() returns maildir file X 2. Later readdir() doesn't return file X, so Dovecot thinks the mail is expunged and also tells the client the mail is expunged 3. Later readdir() once again returns file X, so Dovecot can't really do anything except treat it as a new mail. > Possibly caused by GlusterFS, the indexes were still getting corrupted, even with only 1 server... so I've moved the indexes to a local directory on each server. I'm no longer getting index related errors... but the expunged error is still occuring... and the error seems to happen a few moments before the BB gets a bunch of messages... so it's very likely related. > > I would like to continue using GlusterFS if possible; I haven't found any indication that it is outright unsupported, as long as I'm using Director... If glusterfs can't do readdir() reliably, there's not a whole lot you can do.. Except maybe switch to another mailbox format. But if you're getting index corruption also with 1 Dovecot server, it doesn't look very promising. From apm at one.com Tue Apr 10 18:28:29 2012 From: apm at one.com (Peter Mogensen) Date: Tue, 10 Apr 2012 17:28:29 +0200 Subject: [Dovecot] 2.0/2.1 - different behavior for LIST-EXTENDED Message-ID: <4F84519D.9070104@one.com> Hi Timo, We are sitting here wondering if this difference in behaviour between dovecot 2.0.17 and 2.1.3 is intended. When you create a folder, subscribe to it and rename it (without changing the subscription) these are the behaviours: For 2.0.17: . list (SUBSCRIBED) "" "*" RETURN (STATUS (MESSAGES)) * LIST (\Subscribed \NonExistent) "." "INBOX.test" For 2.1.3: . list (SUBSCRIBED) "" "*" RETURN (STATUS (MESSAGES)) * LIST (\Subscribed) "." "INBOX.test" * NO Mailbox doesn't exist: test If you don't use rfc5819 the folder will just get silently ignored by dovecot 2.1.x, but if you actually try to get the number of messages you'll get the error. It seems to me from reading rfc5258 that the 2.0.x behaviour is the correct ?? /Peter From s.locatelli at hexanet.fr Tue Apr 10 18:29:45 2012 From: s.locatelli at hexanet.fr (=?ISO-8859-1?Q?St=E9phane_Locatelli?=) Date: Tue, 10 Apr 2012 17:29:45 +0200 Subject: [Dovecot] Per IP virtual domain Message-ID: Hello, My dovecot server (2.0.18 on FreeBSD 9) has multiple interface. I would like to configure the authentication (pop and imap) to append different domain according to the local IP address. I've tested this: local 10.0.0.1 { auth_default_realm = domain1.com } local 10.0.0.2 { auth_default_realm = domain2.com } But it doesn't work. I get this error message: doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf line 58: Auth settings not supported inside local/remote blocks: auth_default_realm -- Stephane Locatelli From tss at iki.fi Tue Apr 10 19:04:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 19:04:59 +0300 Subject: [Dovecot] 2.0/2.1 - different behavior for LIST-EXTENDED In-Reply-To: <4F84519D.9070104@one.com> References: <4F84519D.9070104@one.com> Message-ID: <73DC1DBA-930F-41F3-9220-68C2D53339C9@iki.fi> On 10.4.2012, at 18.28, Peter Mogensen wrote: > For 2.0.17: > . list (SUBSCRIBED) "" "*" RETURN (STATUS (MESSAGES)) > * LIST (\Subscribed \NonExistent) "." "INBOX.test" > > For 2.1.3: > . list (SUBSCRIBED) "" "*" RETURN (STATUS (MESSAGES)) > * LIST (\Subscribed) "." "INBOX.test" > * NO Mailbox doesn't exist: test This was fixed in v2.1.4. From tss at iki.fi Tue Apr 10 19:06:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 19:06:38 +0300 Subject: [Dovecot] Per IP virtual domain In-Reply-To: References: Message-ID: <39EBDB8E-C0E2-43D5-9548-6CDF274D51E3@iki.fi> On 10.4.2012, at 18.29, St?phane Locatelli wrote: > My dovecot server (2.0.18 on FreeBSD 9) has multiple interface. I would > like to configure the authentication (pop and imap) to append different > domain according to the local IP address. > > I've tested this: > > local 10.0.0.1 { > auth_default_realm = domain1.com > } > > local 10.0.0.2 { > auth_default_realm = domain2.com > } > > But it doesn't work. I get this error message: > > doveconf: Fatal: Error in configuration file > /usr/local/etc/dovecot/dovecot.conf line 58: Auth settings not supported > inside local/remote blocks: auth_default_realm As it says, currently they aren't supported. It would need some extra code. Depending on your passdb you may be able to do this in your passdb configuration. From tss at iki.fi Tue Apr 10 19:10:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 19:10:04 +0300 Subject: [Dovecot] Username from rfc822Name subject alternative name In-Reply-To: References: Message-ID: <578B8B92-3EC4-4137-B4D0-3B68A767CAB5@iki.fi> On 9.4.2012, at 16.51, ?????? ????? wrote: > I'm looking into adding support for extracting the username from client certificate's rfc822Name (from the subjectAltName extension). > > The question I have is what would be the best approach to do this? Current implementation has a kind of clean code since it just goes through the subject name, extracting the values with X509_NAME_get_text_by_NID (while NID is obtained with OBJ_txt2nid). If I were to add this, it's bound to make the code a little bit more complicated since SAN's can't be retrieved in the same way. > > So far in terms of options I have, I can see the following: > > 1. Create a distinct configuration option for the ssl_cert_username_field (i.e. specify something like "sanrfc822Name" to have Dovecot extract the username from the designated alternative name). I'm not sure if this is a good idea, but what about: ssl_cert_username_field = subjectAltName:rfc822Name > 2. Make the current code fail-over to rfc822Name SAN if emailAddress is provided for ssl_cert_username (less invasion in code, but less flexibility as well). Automatic failover seems dangerous. From s.locatelli at hexanet.fr Tue Apr 10 19:12:41 2012 From: s.locatelli at hexanet.fr (=?ISO-8859-1?Q?St=E9phane_Locatelli?=) Date: Tue, 10 Apr 2012 18:12:41 +0200 Subject: [Dovecot] Per IP virtual domain In-Reply-To: <39EBDB8E-C0E2-43D5-9548-6CDF274D51E3@iki.fi> References: <39EBDB8E-C0E2-43D5-9548-6CDF274D51E3@iki.fi> Message-ID: 2012/4/10 Timo Sirainen > > As it says, currently they aren't supported. It would need some extra > code. Depending on your passdb you may be able to do this in your passdb > configuration. Currently I'm using dovecot 1.x on my production server with mysql passdb. I have ugly SQL queries to do this and my idea was to upgrade to dovecot 2.x to avoid this. password_query = SELECT mbox AS user , passwd as password FROM mbox WHERE mbox=IF('%s' = 'smtp', '%u', IF('%u' like '%%@%%', '%u', concat('%u', case '%l' WHEN '10.0.0.1' THEN '@domain1.com' WHEN '10.0.0.2' THEN '@domain2.com' WHEN '10.0.0.3' THEN '@domain3.com' END ) ) ) AND passwd IS NOT NULL AND (status='active' OR status ='inactive'); -- Stephane Locatelli From lists at wildgooses.com Tue Apr 10 19:35:09 2012 From: lists at wildgooses.com (Ed W) Date: Tue, 10 Apr 2012 17:35:09 +0100 Subject: [Dovecot] Authentication mechanism and Password scheme In-Reply-To: References: <213B51F00051AE48A9F0E112880177178F794E@Delta.sc.local> Message-ID: <4F84613D.8040508@wildgooses.com> On 10/04/2012 08:11, Timo Sirainen wrote: > On 10.4.2012, at 5.37, ???????? ????????? ?????????? wrote: > >> Good day! >> I'm just trying to figure out that my understanding of subject is correct. >> >> So, if I want to store passwords in my database encrypted with SSHA512 scheme, >> my only choice for Authentication mechanism is plaintext? > Yeah, that's correct. > Does dovecot 2.0 also support SCRAM-SHA? I only mention because it's come up on my radar recently and as I understand it, it solves the issue of either having - plain text db of passwords, encrypted login - encrypted db of passwords, plaintext login With SCRAM you have both sides "encrypted". (Clearly it's also desirable that the hash algorithm is well chosen to be resistant to bruteforce, so some might argue that bcrypt/scrypt is even more desirable since there is not yet a GPU implementation - However, at least SHA is a decent stab at things) Can you confirm my understanding is correct? Next question is whether any current mail client supports SCRAM..? Regards Ed W From gwurster at rim.com Tue Apr 10 20:17:23 2012 From: gwurster at rim.com (Glenn Wurster) Date: Tue, 10 Apr 2012 17:17:23 +0000 Subject: [Dovecot] Bug in HIGHESTMODSEQ when MODSEQ tracking is not enabled for version 2.0.18. Message-ID: <48A61B0976102F488C64EF8A8F1E929D074BBA@XMB118CNC.rim.net> Greetings, I'm new to the list. I have encountered what I believe to be a bug in the return of HIGHESTMODSEQ when talking to a IMAP client. HIGHESTMODSEQ does not appear to be incrementing when a new message arrives. The exact steps I took are: 1) Create a new mail account 2) Send a mail to the account, check that you can receive the email. Send a second mail to the account. 3) perform a SELECT on the mailbox from a new session. PMLK4 SELECT INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 2 EXISTS * 1 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1334074920] UIDs valid * OK [UIDNEXT 3] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest PMLK4 OK [READ-WRITE] Select completed. 4) Retrieve the mail. Then perform another SELECT on the mailbox from yet another session. LLEP4 SELECT INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 2 EXISTS * 0 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1334074920] UIDs valid * OK [UIDNEXT 3] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest LLEP4 OK [READ-WRITE] Select completed. 5) Send another mail to the account, and check SELECT again. FKDA4 SELECT INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 3 EXISTS * 1 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1334074920] UIDs valid * OK [UIDNEXT 4] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest FKDA4 OK [READ-WRITE] Select completed. It appears as if HIGHESTMODSEQ is not being updated. I can get HIGHESTMODSEQ to start updating correctly if I send a "UID FETCH 1 MODSEQ" or similar command, which appears to enable MODSEQ tracking at the server (according to the comment around line 173 in file src/lib-index/mail-index-modseq.c), but until that command is sent, MODSEQ tracking is not enabled and hence HIGHESTMODSEQ is always going to return 1. According to RFC4551, the combination of HIGHESTMODSEQ and UIDVALIDITY should be sufficient to determine if the metadata associated with the mailbox has changed, but in this case looking at only those two parameters does not yield sufficient information about changes in the mailbox. The mail client I'm using relies on the combination of HIGHESTMODSEQ and UIDVALIDITY to determine if there are changes in the mailbox, and hence does not see new mail come in. It seems that Dovecot should not be returning HIGHESTMODSEQ in response to a command if MODSEQ tracking is not enabled, but I could be wrong. I've attached my configuration (it's Dovecot 2.0.18 running on Debian Stable). Let me know if you need more information. Glenn. --------------------------------------------------------------------- This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful. -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot.conf Type: application/octet-stream Size: 452 bytes Desc: dovecot.conf URL: From tss at iki.fi Tue Apr 10 21:21:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 21:21:25 +0300 Subject: [Dovecot] Authentication mechanism and Password scheme In-Reply-To: <4F84613D.8040508@wildgooses.com> References: <213B51F00051AE48A9F0E112880177178F794E@Delta.sc.local> <4F84613D.8040508@wildgooses.com> Message-ID: <8AB6491F-9277-45BF-8FF0-2C90D7941E8A@iki.fi> On 10.4.2012, at 19.35, Ed W wrote: > Does dovecot 2.0 also support SCRAM-SHA? v2.1 does. > I only mention because it's come up on my radar recently and as I understand it, it solves the issue of either having > - plain text db of passwords, encrypted login > - encrypted db of passwords, plaintext login > > With SCRAM you have both sides "encrypted". The same way as with DIGEST-MD5 and several others. Each mechanism requires that the server-side password is saved using a hash specific to that auth mechanism, none of them support generic MD5/SHA/etc hashes or other mechanisms' hashes. Looks like Dovecot's current SCRAM-SHA1 supports only plaintext passwords, but it would be possible to add SCRAM-SHA1 password scheme similar to others. From tss at iki.fi Tue Apr 10 21:42:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Apr 2012 21:42:28 +0300 Subject: [Dovecot] Bug in HIGHESTMODSEQ when MODSEQ tracking is not enabled for version 2.0.18. In-Reply-To: <48A61B0976102F488C64EF8A8F1E929D074BBA@XMB118CNC.rim.net> References: <48A61B0976102F488C64EF8A8F1E929D074BBA@XMB118CNC.rim.net> Message-ID: <11BD08AA-2E29-48DF-973D-4D5B4CCE2FF7@iki.fi> On 10.4.2012, at 20.17, Glenn Wurster wrote: > It appears as if HIGHESTMODSEQ is not being updated. I can get HIGHESTMODSEQ to start updating correctly if I send a "UID FETCH 1 MODSEQ" or similar command, which appears to enable MODSEQ tracking at the server (according to the comment around line 173 in file src/lib-index/mail-index-modseq.c), but until that command is sent, MODSEQ tracking is not enabled and hence HIGHESTMODSEQ is always going to return 1. Yes, modseqs aren't tracked in a mailbox until client expresses an interest for them. It would be a waste of disk space to save them since 99% of users don't need them. > According to RFC4551, the combination of HIGHESTMODSEQ and UIDVALIDITY should be sufficient to determine if the metadata associated with the mailbox has changed, but in this case looking at only those two parameters does not yield sufficient information about changes in the mailbox. The mail client I'm using relies on the combination of HIGHESTMODSEQ and UIDVALIDITY to determine if there are changes in the mailbox, and hence does not see new mail come in. Yeah, it does seem that the RFC says that.. > It seems that Dovecot should not be returning HIGHESTMODSEQ in response to a command if MODSEQ tracking is not enabled, but I could be wrong. I've attached my configuration (it's Dovecot 2.0.18 running on Debian Stable). RFC 4551 says that HIGHESTMODSEQ or NOMODSEQ MUST be returned. Hmm. Perhaps: 1) If the session is known to have modseqs enabled, immediately enable modseqs for newly created mailboxes 2) If a mailbox doesn't have modseqs enabled, return NOMODSEQ. This isn't ideal, but seems like the only possibility. From jeetuindian at gmail.com Tue Apr 10 22:07:48 2012 From: jeetuindian at gmail.com (Jitendra Bhaskar) Date: Wed, 11 Apr 2012 00:37:48 +0530 Subject: [Dovecot] Dsync test on 2.1.3 Message-ID: Hi, I am using dovecot version 2.1.3 on centos. I tested dsync from my live server to one of backup server. But I got that its working for most of the user and there is problem for few user. Some users got duplicated mail to their mailbox and I saw there is some extra folders i.e Sent Items_* , Draft_* and Trash_* in few users mail box. and I synced only from main server to backup server. I didn't enable mail_plugin in backup server. My configuration for testing of Dsync on master server is as: mail_plugins = $mail_plugins notify replication service aggregator { fifo_listener replication-notify-fifo { user = vmail mode = 0666 } unix_listener replication-notify { user = vmail mode = 0666 } } service replicator { # start replication at startup process_min_avail = 1 } plugin { #host1 replicates to host2 mail_replica = remote:root at backup_server } #dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -l%{lock_timeout} -n%{namespace} dsync_remote_cmd = ssh -l root %{host} doveadm dsync-server -u%{user} -l%{10} -n%{namespace} service doveadm { user = root } And in backup server : service doveadm { user = root } 1. Problem is I am not getting why some user getting lots of duplicate mails. 2. Why duplicate folders like Sent Items_* and Trash_* created for few users 3. I used only one way syncing then why both replicated. Note: i am using root only for testing purpose * Thanks & Regards * *Jitendra Kumar Bhaskar* Cell:- +91 7306311531 +91 8102997821 From fxmulder at gmail.com Tue Apr 10 23:19:26 2012 From: fxmulder at gmail.com (James Devine) Date: Tue, 10 Apr 2012 14:19:26 -0600 Subject: [Dovecot] lazy_expunge questions Message-ID: I'm messing around with lazy expunge and I just had a couple of questions. If I delete messages or folders with messages, I see them being moved to the expunge namespace, but if I delete an empty folder, all traces of it seem to disappear. Is there a way to move empty folders to the expunge namespace also? My second question is in regards to restoring folders. If I try to execute the imap command 'RENAME EXPUNGED/TEST INBOX/TEST' I get 'NO [CANNOT] Can't rename mailboxes to/from expunge namespace'. Just wondering if this is possible or do I need to recreate the folder and then move messages into it. From gwurster at rim.com Tue Apr 10 23:50:07 2012 From: gwurster at rim.com (Glenn Wurster) Date: Tue, 10 Apr 2012 20:50:07 +0000 Subject: [Dovecot] Bug in HIGHESTMODSEQ when MODSEQ tracking is not enabled for version 2.0.18. In-Reply-To: <11BD08AA-2E29-48DF-973D-4D5B4CCE2FF7@iki.fi> References: <48A61B0976102F488C64EF8A8F1E929D074BBA@XMB118CNC.rim.net> <11BD08AA-2E29-48DF-973D-4D5B4CCE2FF7@iki.fi> Message-ID: <48A61B0976102F488C64EF8A8F1E929D074E9F@XMB118CNC.rim.net> > Yes, modseqs aren't tracked in a mailbox until client expresses an > interest for them. It would be a waste of disk space to save them > since 99% of users don't need them. Makes sense, our mail client gets caught in the middle though, because it uses HIGHESTMODSEQ to track mailbox updates without using MODSEQ options on SELECT/FETCH to track message updates. > RFC 4551 says that HIGHESTMODSEQ or NOMODSEQ MUST be returned. Hmm. > Perhaps: > > 1) If the session is known to have modseqs enabled, immediately enable > modseqs for newly created mailboxes MODSEQ seems to be enabled on the mailbox, not on the session. For disconnected IMAP, there will frequently be new sessions established, and so we may not have a good indicator of whether the client is using MODSEQ (based on my reading). The current approach of looking for a MODSEQ related command seems to work in most cases, and so I'd be tempted to keep the current functionality (minus the incorrect reporting of HIGHESTMODSEQ if we can find a solution). > 2) If a mailbox doesn't have modseqs enabled, return NOMODSEQ. This > isn't ideal, but seems like the only possibility. The RFC also states that if we return NOMODSEQ we'd have to return a tagged BAD response to "UID FETCH 1 MODSEQ", which appears to one of the commands that enables MODSEQ for Dovecot ("SELECT INBOX (CONDSTORE)" also enables it...). What about returning a BAD response and at the same time start tracking MODSEQ so that future SELECT commands would return HIGHESTMODSEQ? Do we know what email clients are using CONDSTORE options and how they'd react to a mailbox suddenly having MODSEQ capabilities after we just told them it didn't? Glenn. --------------------------------------------------------------------- This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful. From branko at majic.rs Wed Apr 11 00:07:54 2012 From: branko at majic.rs (=?UTF-8?B?0JHRgNCw0L3QutC+INCc0LDRmNC40Zs=?=) Date: Tue, 10 Apr 2012 23:07:54 +0200 Subject: [Dovecot] Username from rfc822Name subject alternative name In-Reply-To: <578B8B92-3EC4-4137-B4D0-3B68A767CAB5@iki.fi> References: <578B8B92-3EC4-4137-B4D0-3B68A767CAB5@iki.fi> Message-ID: <20120410230754.1ab2831b@majic.rs> Another solution could be having another property that specifies whether the information should be pulled in from subject or subjectAltName. But I think what you've proposed would be ok as well, and it'd allow for future expansion too (if anyone decides to use some other SAN). I can get working on adding the support for this (might take a couple of days until I sort it out - getting quite busy here :). Would a patch both against the 1.x and 2.x be welcome? ???? Tue, 10 Apr 2012 19:10:04 +0300 Timo Sirainen ??????: > On 9.4.2012, at 16.51, ?????? ????? wrote: > > > I'm looking into adding support for extracting the username from > > client certificate's rfc822Name (from the subjectAltName extension). > > > > The question I have is what would be the best approach to do this? > > Current implementation has a kind of clean code since it just goes > > through the subject name, extracting the values with > > X509_NAME_get_text_by_NID (while NID is obtained with OBJ_txt2nid). > > If I were to add this, it's bound to make the code a little bit > > more complicated since SAN's can't be retrieved in the same way. > > > > So far in terms of options I have, I can see the following: > > > > 1. Create a distinct configuration option for the > > ssl_cert_username_field (i.e. specify something like > > "sanrfc822Name" to have Dovecot extract the username from the > > designated alternative name). > > I'm not sure if this is a good idea, but what about: > > ssl_cert_username_field = subjectAltName:rfc822Name > > > 2. Make the current code fail-over to rfc822Name SAN if > > emailAddress is provided for ssl_cert_username (less invasion in > > code, but less flexibility as well). > > Automatic failover seems dangerous. -- Branko Majic Jabber: branko at majic.rs Please use only Free formats when sending attachments to me. ?????? ????? ?????: branko at majic.rs ????? ??? ?? ??????? ?????? ????????? ? ????????? ?????????. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From robert at schetterer.org Wed Apr 11 00:08:56 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 10 Apr 2012 23:08:56 +0200 Subject: [Dovecot] sieve replace dot string for foldername Message-ID: <4F84A168.5090700@schetterer.org> Hi, i am playing about mail archive this is what allready works for me require ["variables","date","fileinto","mailbox","envelope","subaddress","regex"]; # Extract date info if currentdate :matches "year" "*" { set "year" "${1}"; } if currentdate :matches "month" "*" { set "month" "${1}"; } if currentdate :matches "day" "*" { set "day" "${1}"; } if envelope :detail :matches "to" "*" { set "to" "${1}"; } if true { fileinto :create "INBOX.${year}.${month}.${day}.${to}"; stop; } but ${to} may have a dot included, foldernames with dot are not allowed in my dove setup yet ( and i dont want to change it ) i am looking for a way ( guess regex ) for replace a dot in string replaced by a space char on the fly -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Wed Apr 11 00:44:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 00:44:04 +0300 Subject: [Dovecot] Username from rfc822Name subject alternative name In-Reply-To: <20120410230754.1ab2831b@majic.rs> References: <578B8B92-3EC4-4137-B4D0-3B68A767CAB5@iki.fi> <20120410230754.1ab2831b@majic.rs> Message-ID: <83E9DEB2-6986-4097-95EF-1532D1FB873F@iki.fi> I can add the patch to v2.1, but probably not to earlier versions. On 11.4.2012, at 0.07, ?????? ????? wrote: > Another solution could be having another property that specifies > whether the information should be pulled in from subject or > subjectAltName. > > But I think what you've proposed would be ok as well, and it'd allow > for future expansion too (if anyone decides to use some other SAN). I > can get working on adding the support for this (might take a couple > of days until I sort it out - getting quite busy here :). > > Would a patch both against the 1.x and 2.x be welcome? > > ???? Tue, 10 Apr 2012 19:10:04 +0300 > Timo Sirainen ??????: > >> On 9.4.2012, at 16.51, ?????? ????? wrote: >> >>> I'm looking into adding support for extracting the username from >>> client certificate's rfc822Name (from the subjectAltName extension). >>> >>> The question I have is what would be the best approach to do this? >>> Current implementation has a kind of clean code since it just goes >>> through the subject name, extracting the values with >>> X509_NAME_get_text_by_NID (while NID is obtained with OBJ_txt2nid). >>> If I were to add this, it's bound to make the code a little bit >>> more complicated since SAN's can't be retrieved in the same way. >>> >>> So far in terms of options I have, I can see the following: >>> >>> 1. Create a distinct configuration option for the >>> ssl_cert_username_field (i.e. specify something like >>> "sanrfc822Name" to have Dovecot extract the username from the >>> designated alternative name). >> >> I'm not sure if this is a good idea, but what about: >> >> ssl_cert_username_field = subjectAltName:rfc822Name >> >>> 2. Make the current code fail-over to rfc822Name SAN if >>> emailAddress is provided for ssl_cert_username (less invasion in >>> code, but less flexibility as well). >> >> Automatic failover seems dangerous. > > -- > Branko Majic > Jabber: branko at majic.rs > Please use only Free formats when sending attachments to me. > > ?????? ????? > ?????: branko at majic.rs > ????? ??? ?? ??????? ?????? ????????? ? ????????? ?????????. From stephan at rename-it.nl Wed Apr 11 01:06:48 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 11 Apr 2012 00:06:48 +0200 Subject: [Dovecot] sieve replace dot string for foldername In-Reply-To: <4F84A168.5090700@schetterer.org> References: <4F84A168.5090700@schetterer.org> Message-ID: <4F84AEF8.8000504@rename-it.nl> On 4/10/2012 11:08 PM, Robert Schetterer wrote: > Hi, i am playing about mail archive > > this is what allready works for me > > require > ["variables","date","fileinto","mailbox","envelope","subaddress","regex"]; > > # Extract date info > if currentdate :matches "year" "*" { set "year" "${1}"; } > if currentdate :matches "month" "*" { set "month" "${1}"; } > if currentdate :matches "day" "*" { set "day" "${1}"; } > if envelope :detail :matches "to" "*" { set "to" "${1}"; } > > if true { > fileinto :create "INBOX.${year}.${month}.${day}.${to}"; > stop; > } > > but ${to} may have a dot included, foldernames > with dot are not allowed in my dove setup yet ( and i dont want to > change it ) > > i am looking for a way ( guess regex ) > for replace a dot in string replaced by a space char > on the fly Hmm, no, I see no possibilities for that really. Sieve doesn't have such advanced string manipulation capabilities unfortunately. You could use the vnd.dovecot.execute extension (extprograms plugin) for that, but I wouldn't call that a nice solution by any means. Regards, Stephan. From robert at schetterer.org Wed Apr 11 01:44:33 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 11 Apr 2012 00:44:33 +0200 Subject: [Dovecot] sieve replace dot string for foldername In-Reply-To: <4F84AEF8.8000504@rename-it.nl> References: <4F84A168.5090700@schetterer.org> <4F84AEF8.8000504@rename-it.nl> Message-ID: <4F84B7D1.1050507@schetterer.org> Am 11.04.2012 00:06, schrieb Stephan Bosch: > On 4/10/2012 11:08 PM, Robert Schetterer wrote: >> Hi, i am playing about mail archive >> >> this is what allready works for me >> >> require >> ["variables","date","fileinto","mailbox","envelope","subaddress","regex"]; >> >> >> # Extract date info >> if currentdate :matches "year" "*" { set "year" "${1}"; } >> if currentdate :matches "month" "*" { set "month" "${1}"; } >> if currentdate :matches "day" "*" { set "day" "${1}"; } >> if envelope :detail :matches "to" "*" { set "to" "${1}"; } >> >> if true { >> fileinto :create "INBOX.${year}.${month}.${day}.${to}"; >> stop; >> } >> >> but ${to} may have a dot included, foldernames >> with dot are not allowed in my dove setup yet ( and i dont want to >> change it ) >> >> i am looking for a way ( guess regex ) >> for replace a dot in string replaced by a space char >> on the fly > > Hmm, no, I see no possibilities for that really. Sieve doesn't have such > advanced string manipulation capabilities unfortunately. > > You could use the vnd.dovecot.execute extension (extprograms plugin) for > that, but I wouldn't call that a nice solution by any means. > > Regards, > > Stephan. > > > Thx Stephan, so i dont have to search anymore perhaps i ll use listescape plugin for dot foldernames no idea if it breaks existing setup stuff yet -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From lee at standen.id.au Wed Apr 11 06:54:35 2012 From: lee at standen.id.au (Lee Standen) Date: Wed, 11 Apr 2012 11:54:35 +0800 Subject: [Dovecot] F5 SLB iRule - POP3 TLS Offload & Add Realm Message-ID: Hey Guys, I figured someone might have use of this at some point. I've created an iRule for our F5 Load Balancer which performs the following tasks: - Appends STLS to the POP3 Capability list (it does this blindly, so expects the backend POP3 server to not return this) - Watches for the STLS command and initiates SSL negotiation with the client (leaves the connection to the backend POP3 server unencrypted) - Watches for the USER command (login attempt) and appends a realm if the user hasn't supplied one - Stops watching once a successful login occurs It's designed to be attached to port 110 and port 995 at the same time, and will disable SSL if the user is connecting to port 110 until the STLS command is issued. This is a first pass, so there's probably room for improvement, but it does seem to work. This will require LTM v10 (at least), as the SSL::collect command didn't exist prior to that version. Here's the rule: when CLIENT_ACCEPTED { if { [TCP::local_port clientside] == "110" } { # Disable SSL if we're on port 110 (as we're doing TLS Offload) set secure 0 SSL::disable } else { set secure 1 } set realm "mydomain.com" set debug 0 set loggedin 0 } when SERVER_CONNECTED { TCP::collect } when CLIENTSSL_HANDSHAKE { SSL::collect set secure 1 } when CLIENTSSL_DATA { if { $debug } { log local0. "CLIENT DATA: [SSL::payload]" } set lcpayload [string tolower [SSL::payload]] if { $lcpayload starts_with "user" } { scan [SSL::payload] {%s%s} command user set pos [string first $user [SSL::payload] 0] SSL::payload replace $pos [string length $user] "$user@$realm" if { $debug } { log local0. [SSL::payload] } } SSL::release if { $loggedin == 0 } { SSL::collect } } when CLIENT_DATA { if { $debug } { log local0. "CLIENT DATA: [TCP::payload]" } set lcpayload [string tolower [TCP::payload]] if { $lcpayload starts_with "stls" } { TCP::respond "+OK Begin TLS negotiation\r\n" TCP::payload replace 0 [TCP::payload length] "" TCP::release SSL::enable return } elseif { $lcpayload starts_with "user" } { scan [TCP::payload] {%s%s} command user set pos [string first $user [TCP::payload] 0] TCP::payload replace $pos [string length $user] "$user@$realm" if { $debug } { log local0. [TCP::payload] } } TCP::release if { $loggedin == 0 } { TCP::collect } } when SERVER_DATA { if { $debug } { log local0. "SERVER DATA: [TCP::payload]" } set lcpayload [string tolower [TCP::payload]] if { $lcpayload starts_with "+ok logged in" } { TCP::release set loggedin 1 event disable if { $debug } { log local0. "POP3 Login Successful" } return } elseif { $lcpayload contains "capa" } { TCP::payload replace [expr [TCP::payload length] - 3] 0 "STLS\r\n" } if { $secure == 1 && $loggedin == 0 } { clientside { SSL::collect } } elseif { $secure == 0 && $loggedin == 0 } { clientside { TCP::collect } } TCP::release if { $loggedin == 0 } { TCP::collect } } From dmiller at amfes.com Wed Apr 11 08:06:34 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Tue, 10 Apr 2012 22:06:34 -0700 Subject: [Dovecot] Folder Aliases - Revisited Yet Again Four Years Later In-Reply-To: <1281704852.3046.120.camel@kurkku.sapo.corppt.com> References: <4C648BF7.4020209@amfes.com> <1281704852.3046.120.camel@kurkku.sapo.corppt.com> Message-ID: On 8/13/2010 6:07 AM, Timo Sirainen wrote: > On Thu, 2010-08-12 at 17:04 -0700, Daniel L. Miller wrote: >> If we can't get clients to agree on a "standard" set of folders >> (probably reasonably safe assumption) > [...] > > > I'm kind of annoyed with listescape though. Hopefully for v2.1 I can > figure out some API changes to make it much easier to modify mailbox > names, and I guess also make such aliasing plugin much easier to > implement. > Did you happen to look into this in your spare time? -- Daniel From tss at iki.fi Wed Apr 11 08:12:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 08:12:36 +0300 Subject: [Dovecot] Folder Aliases - Revisited Yet Again Four Years Later In-Reply-To: References: <4C648BF7.4020209@amfes.com> <1281704852.3046.120.camel@kurkku.sapo.corppt.com> Message-ID: On 11.4.2012, at 8.06, Daniel L. Miller wrote: > On 8/13/2010 6:07 AM, Timo Sirainen wrote: >> On Thu, 2010-08-12 at 17:04 -0700, Daniel L. Miller wrote: >>> If we can't get clients to agree on a "standard" set of folders >>> (probably reasonably safe assumption) >> [...] >> >> >> I'm kind of annoyed with listescape though. Hopefully for v2.1 I can >> figure out some API changes to make it much easier to modify mailbox >> names, and I guess also make such aliasing plugin much easier to >> implement. > > Did you happen to look into this in your spare time? Yes, this is much easier with v2.1 I think. The new listescape plugin is just a tiny wrapper that will probably just be replaced by a setting some day in future. I haven't actually tried to write such alias plugin though. From tss at iki.fi Wed Apr 11 08:55:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 08:55:04 +0300 Subject: [Dovecot] Folder Aliases - Revisited Yet Again Four Years Later In-Reply-To: References: <4C648BF7.4020209@amfes.com> <1281704852.3046.120.camel@kurkku.sapo.corppt.com> Message-ID: <91485582-FA24-41C8-B478-62B4EF174C15@iki.fi> On 11.4.2012, at 8.12, Timo Sirainen wrote: > Yes, this is much easier with v2.1 I think. The new listescape plugin is just a tiny wrapper that will probably just be replaced by a setting some day in future. I haven't actually tried to write such alias plugin though. Here: http://dovecot.org/patches/2.1/mailbox-alias-plugin.c But as it is, it doesn't really work in a useful way, since e.g.: x create real x OK Create completed. x list "" real x OK List completed. x list "" alias * LIST (\HasNoChildren) "/" "alias" x OK List completed. x create real x NO [ALREADYEXISTS] Mailbox already exists x status real (messages) * STATUS "real" (MESSAGES 0) x OK Status completed. Perhaps a more useful feature would be if LIST showed both alias and real? That would need a bit more code. From tss at iki.fi Wed Apr 11 09:01:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 09:01:16 +0300 Subject: [Dovecot] lazy_expunge questions In-Reply-To: References: Message-ID: <4791F1D6-6BF2-4FB8-82CE-7C99DB50B4EA@iki.fi> On 10.4.2012, at 23.19, James Devine wrote: > I'm messing around with lazy expunge and I just had a couple of questions. > > If I delete messages or folders with messages, I see them being moved to > the expunge namespace, but if I delete an empty folder, all traces of it > seem to disappear. Is there a way to move empty folders to the expunge > namespace also? If you give 3 namespaces to lazyexpunge, it uses another way of moving stuff, and that includes also moving empty folders. But with 1 namespace it doesn't currently move it. Perhaps it should. > My second question is in regards to restoring folders. If I try to execute > the imap command 'RENAME EXPUNGED/TEST INBOX/TEST' I get 'NO [CANNOT] Can't > rename mailboxes to/from expunge namespace'. Just wondering if this is > possible or do I need to recreate the folder and then move messages into it. You need to recreate + move. Renaming across namespaces is a little tricky. From tss at iki.fi Wed Apr 11 09:26:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 09:26:20 +0300 Subject: [Dovecot] Bug tracker Message-ID: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> I started thinking that perhaps I should move my TODO list to a bug tracker. But because of reasons I've explained a few times before, I don't want a full blown public bug tracking system. The requirements for it are: * I am the only person who can add new bugs. Everyone else reports bugs/requests to this mailing list as before. (Well, I guess Stephan could use this as well if he wants to.) * Everyone can comment existing bugs. * Dovecot mailing list integration: Commenting a bug sends a mail to the mailing list. Replies to those comments go back to bug tracker (probably based on some [#1234] tag in subject). I would have the option of adding a comment that doesn't go to the mailing list (= adding some internal comment that nobody else cares about). Notifications about new bugs won't go to the mailing list (most likely it was created due to a recent mailing list post). So the main difference to how things work now is that people would be able to easily browse existing bugs and add comments to them. I would add bugs there only when I'm not planning on fixing them within a few days. I wouldn't add each and every feature request there, only the things that I'm actually interested in developing. So the idea would be to actually get the bug tracker emptied at some point, not to be a graveyard of unimportant feature requests that about 1-2 people in the world would want. So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. From martin.rabl at rablnet.de Wed Apr 11 09:43:46 2012 From: martin.rabl at rablnet.de (Martin Rabl) Date: Wed, 11 Apr 2012 08:43:46 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: <8095D302-D3CE-4A4E-AB12-64DB2EF7BD96@rablnet.de> What about Mantis? --- Martin Rabl Am 11.04.2012 um 08:26 schrieb Timo Sirainen : > I started thinking that perhaps I should move my TODO list to a bug tracker. But because of reasons I've explained a few times before, I don't want a full blown public bug tracking system. The requirements for it are: > > * I am the only person who can add new bugs. Everyone else reports bugs/requests to this mailing list as before. (Well, I guess Stephan could use this as well if he wants to.) > > * Everyone can comment existing bugs. > > * Dovecot mailing list integration: Commenting a bug sends a mail to the mailing list. Replies to those comments go back to bug tracker (probably based on some [#1234] tag in subject). I would have the option of adding a comment that doesn't go to the mailing list (= adding some internal comment that nobody else cares about). Notifications about new bugs won't go to the mailing list (most likely it was created due to a recent mailing list post). > > So the main difference to how things work now is that people would be able to easily browse existing bugs and add comments to them. I would add bugs there only when I'm not planning on fixing them within a few days. I wouldn't add each and every feature request there, only the things that I'm actually interested in developing. So the idea would be to actually get the bug tracker emptied at some point, not to be a graveyard of unimportant feature requests that about 1-2 people in the world would want. > > So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. > From janfrode at tanso.net Wed Apr 11 09:44:19 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 11 Apr 2012 08:44:19 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: <20120411064419.GA25675@dibs.tanso.net> On Wed, Apr 11, 2012 at 09:26:20AM +0300, Timo Sirainen wrote: > > So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. > I didn't see open source as a requirement, so then I would give a plug for Jira, which is the nicest/prettiest thing :-) And they provide free hosted solution: http://www.atlassian.com/software/jira/pricing Apache/ASF is a heavy jira user, in case you're not familiar with it: http://wiki.apache.org/general/ApacheJira https://issues.apache.org/jira/ -jf From tss at iki.fi Wed Apr 11 09:46:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 09:46:47 +0300 Subject: [Dovecot] Bug tracker In-Reply-To: <8095D302-D3CE-4A4E-AB12-64DB2EF7BD96@rablnet.de> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> <8095D302-D3CE-4A4E-AB12-64DB2EF7BD96@rablnet.de> Message-ID: <65B699AA-10F7-4CF8-81F7-AA59895D54E9@iki.fi> Probably the ugliest/user-unfriendliest bug tracker UI that I've ever seen :) On 11.4.2012, at 9.43, Martin Rabl wrote: > What about Mantis? > > > --- > Martin Rabl > > Am 11.04.2012 um 08:26 schrieb Timo Sirainen : > >> I started thinking that perhaps I should move my TODO list to a bug tracker. But because of reasons I've explained a few times before, I don't want a full blown public bug tracking system. The requirements for it are: >> >> * I am the only person who can add new bugs. Everyone else reports bugs/requests to this mailing list as before. (Well, I guess Stephan could use this as well if he wants to.) >> >> * Everyone can comment existing bugs. >> >> * Dovecot mailing list integration: Commenting a bug sends a mail to the mailing list. Replies to those comments go back to bug tracker (probably based on some [#1234] tag in subject). I would have the option of adding a comment that doesn't go to the mailing list (= adding some internal comment that nobody else cares about). Notifications about new bugs won't go to the mailing list (most likely it was created due to a recent mailing list post). >> >> So the main difference to how things work now is that people would be able to easily browse existing bugs and add comments to them. I would add bugs there only when I'm not planning on fixing them within a few days. I wouldn't add each and every feature request there, only the things that I'm actually interested in developing. So the idea would be to actually get the bug tracker emptied at some point, not to be a graveyard of unimportant feature requests that about 1-2 people in the world would want. >> >> So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. >> > From tss at iki.fi Wed Apr 11 09:49:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 09:49:18 +0300 Subject: [Dovecot] Bug tracker In-Reply-To: <20120411064419.GA25675@dibs.tanso.net> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> <20120411064419.GA25675@dibs.tanso.net> Message-ID: On 11.4.2012, at 9.44, Jan-Frode Myklebust wrote: > On Wed, Apr 11, 2012 at 09:26:20AM +0300, Timo Sirainen wrote: >> >> So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. >> > > I didn't see open source as a requirement, so then I would give a plug > for Jira, which is the nicest/prettiest thing :-) I don't think it supports one of my requirements: > I would have the option of adding a comment that doesn't go to the mailing list Unless that's been added in a newer version. From ngu.antoine at gmail.com Wed Apr 11 09:49:45 2012 From: ngu.antoine at gmail.com (Antoine Nguyen) Date: Wed, 11 Apr 2012 08:49:45 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: Le 11 avril 2012 08:26, Timo Sirainen a ?crit : > I started thinking that perhaps I should move my TODO list to a bug > tracker. But because of reasons I've explained a few times before, I don't > want a full blown public bug tracking system. The requirements for it are: > > * I am the only person who can add new bugs. Everyone else reports > bugs/requests to this mailing list as before. (Well, I guess Stephan could > use this as well if he wants to.) > > * Everyone can comment existing bugs. > > * Dovecot mailing list integration: Commenting a bug sends a mail to the > mailing list. Replies to those comments go back to bug tracker (probably > based on some [#1234] tag in subject). I would have the option of adding a > comment that doesn't go to the mailing list (= adding some internal comment > that nobody else cares about). Notifications about new bugs won't go to the > mailing list (most likely it was created due to a recent mailing list post). > > So the main difference to how things work now is that people would be able > to easily browse existing bugs and add comments to them. I would add bugs > there only when I'm not planning on fixing them within a few days. I > wouldn't add each and every feature request there, only the things that I'm > actually interested in developing. So the idea would be to actually get the > bug tracker emptied at some point, not to be a graveyard of unimportant > feature requests that about 1-2 people in the world would want. > > So, any suggestions for what software could do these things? I think > Request Tracker has those features, but it's not really the > nicest/prettiest thing. > > Maybe Redmine ? (http://www.redmine.org/ It's more than just a bug tracker but I think it answers your needs. Antoine Modoboa developer (http://modoboa.org/) From martin.rabl at rablnet.de Wed Apr 11 09:53:24 2012 From: martin.rabl at rablnet.de (Martin Rabl) Date: Wed, 11 Apr 2012 08:53:24 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: <65B699AA-10F7-4CF8-81F7-AA59895D54E9@iki.fi> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> <8095D302-D3CE-4A4E-AB12-64DB2EF7BD96@rablnet.de> <65B699AA-10F7-4CF8-81F7-AA59895D54E9@iki.fi> Message-ID: ;-) Ok, next one: TRAC ? --- Martin Rabl Am 11.04.2012 um 08:46 schrieb Timo Sirainen : > Probably the ugliest/user-unfriendliest bug tracker UI that I've ever seen :) > > On 11.4.2012, at 9.43, Martin Rabl wrote: > >> What about Mantis? >> >> >> --- >> Martin Rabl >> >> Am 11.04.2012 um 08:26 schrieb Timo Sirainen : >> >>> I started thinking that perhaps I should move my TODO list to a bug tracker. But because of reasons I've explained a few times before, I don't want a full blown public bug tracking system. The requirements for it are: >>> >>> * I am the only person who can add new bugs. Everyone else reports bugs/requests to this mailing list as before. (Well, I guess Stephan could use this as well if he wants to.) >>> >>> * Everyone can comment existing bugs. >>> >>> * Dovecot mailing list integration: Commenting a bug sends a mail to the mailing list. Replies to those comments go back to bug tracker (probably based on some [#1234] tag in subject). I would have the option of adding a comment that doesn't go to the mailing list (= adding some internal comment that nobody else cares about). Notifications about new bugs won't go to the mailing list (most likely it was created due to a recent mailing list post). >>> >>> So the main difference to how things work now is that people would be able to easily browse existing bugs and add comments to them. I would add bugs there only when I'm not planning on fixing them within a few days. I wouldn't add each and every feature request there, only the things that I'm actually interested in developing. So the idea would be to actually get the bug tracker emptied at some point, not to be a graveyard of unimportant feature requests that about 1-2 people in the world would want. >>> >>> So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. >>> >> > From tlx at leuxner.net Wed Apr 11 09:54:28 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Wed, 11 Apr 2012 08:54:28 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: <20120411065428.GA3099@nihlus.leuxner.net> On Wed, Apr 11, 2012 at 09:26:20AM +0300, Timo Sirainen wrote: > So, any suggestions for what software could do these things? I think Request Tracker has those features, but it's not really the nicest/prettiest thing. > May be worth to take a look at http://www.thebuggenie.com/. Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From p at state-of-mind.de Wed Apr 11 10:10:07 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Wed, 11 Apr 2012 09:10:07 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: <20120411071006.GC28281@state-of-mind.de> * Antoine Nguyen : > > I started thinking that perhaps I should move my TODO list to a bug > > tracker. But because of reasons I've explained a few times before, I don't > > want a full blown public bug tracking system. The requirements for it are: > > > > * I am the only person who can add new bugs. Everyone else reports > > bugs/requests to this mailing list as before. (Well, I guess Stephan could > > use this as well if he wants to.) > > > > * Everyone can comment existing bugs. > > > > * Dovecot mailing list integration: Commenting a bug sends a mail to the > > mailing list. Replies to those comments go back to bug tracker (probably > > based on some [#1234] tag in subject). I would have the option of adding a > > comment that doesn't go to the mailing list (= adding some internal comment > > that nobody else cares about). Notifications about new bugs won't go to the > > mailing list (most likely it was created due to a recent mailing list post). > > > > So the main difference to how things work now is that people would be able > > to easily browse existing bugs and add comments to them. I would add bugs > > there only when I'm not planning on fixing them within a few days. I > > wouldn't add each and every feature request there, only the things that I'm > > actually interested in developing. So the idea would be to actually get the > > bug tracker emptied at some point, not to be a graveyard of unimportant > > feature requests that about 1-2 people in the world would want. > > > > So, any suggestions for what software could do these things? I think > > Request Tracker has those features, but it's not really the > > nicest/prettiest thing. > > > > > Maybe Redmine ? (http://www.redmine.org/ > > It's more than just a bug tracker but I think it answers your needs. +1 We moved from trac to redmine about a year ago and still are very happy about it. Our non-technical customers can deal with it well. Redmine allows to hide projects and it knows the concept of sub-projects. It's versatile in terms of repositories (mercurial supported) and ticket handling is fine too. Sometimes I miss a more comfortable wiki editor, but that's a minor tradeoff. p at rick -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From warren at decoy.co.za Wed Apr 11 10:18:28 2012 From: warren at decoy.co.za (Warren Baker) Date: Wed, 11 Apr 2012 09:18:28 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: <20120411071006.GC28281@state-of-mind.de> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> <20120411071006.GC28281@state-of-mind.de> Message-ID: On Wed, Apr 11, 2012 at 9:10 AM, Patrick Ben Koetter

wrote: > * Antoine Nguyen : >> >> It's more than just a bug tracker but I think it answers your needs. > > +1 > > We moved from trac to redmine about a year ago and still are very happy about > it. Our non-technical customers can deal with it well. Redmine allows to hide > projects and it knows the concept of sub-projects. It's versatile in terms of > repositories (mercurial supported) and ticket handling is fine too. > > Sometimes I miss a more comfortable wiki editor, but that's a minor tradeoff. +2 - we have been using it for another open source project for a number of years and it works well. We have it hooked up into github and it works without a glitch. thx -- .warren From stan at hardwarefreak.com Wed Apr 11 10:18:49 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 11 Apr 2012 02:18:49 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> Message-ID: <4F853059.1000706@hardwarefreak.com> On 4/10/2012 1:09 AM, Emmanuel Noobadmin wrote: > On 4/10/12, Stan Hoeppner wrote: >> SuperMicro H8SGL G34 mobo w/dual Intel GbE, 2GHz 8-core Opteron >> 32GB Kingston REG ECC DDR3, LSI 9280-4i4e, Intel 24 port SAS expander >> 20 x 1TB WD RE4 Enterprise 7.2K SATA2 drives >> NORCO RPC-4220 4U 20 Hot-Swap Bays, SuperMicro 865W PSU >> All other required parts are in the Wish List. I've not written >> assembly instructions. I figure anyone who would build this knows what >> s/he is doing. >> >> Price today: $5,376.62 > > This price looks like something I might be able to push through It's pretty phenomenally low considering what all you get, especially 20 enterprise class drives. > although I'll probably have to go SATA instead of SAS due to cost of > keeping spares. The 10K drives I mentioned are SATA not SAS. WD's 7.2k RE and 10k Raptor series drives are both SATA but have RAID specific firmware, better reliability, longer warranties, etc. The RAID specific firmware is why both are tested and certified by LSI with their RAID cards. >> Configuring all 20 drives as a RAID10 LUN in the MegaRAID HBA would give >> you a 10TB net Linux device and 10 stripe spindles of IOPS and >> bandwidth. Using RAID6 would yield 18TB net and 18 spindles of read >> throughput, however parallel write throughput will be at least 3-6x >> slower than RAID10, which is why nobody uses RAID6 for transactional >> workloads. > > Not likely to go with RAID 5 or 6 due to concerns about the > uncorrectable read errors risks on rebuild with large arrays. Is the Not to mention rebuild times for large width RAID5/6. > MegaRAID being used as the actual RAID controller or just as a HBA? It's a top shelf RAID controller, 512MB cache, up to 240 drives, SSD support, the works. It's an LSI "Feature Line" card: http://www.lsi.com/products/storagecomponents/Pages/6GBSATA_SASRAIDCards.aspx The specs: http://www.lsi.com/products/storagecomponents/Pages/MegaRAIDSAS9280-4i4e.aspx You'll need the cache battery module for safe write caching, which I forgot in the wish list (now added), $160: http://www.newegg.com/Product/Product.aspx?Item=N82E16816118163&Tpk=LSIiBBU08 With your workload and RAID10 you should run with all 512MB configured as write cache. Linux caches all reads so using any controller cache for reads is a waste. Using all 512MB for write cache will increase random write IOPS. Note the 9280 allows up to 64 LUNs, so you can do tiered storage within this 20 bay chassis. For spares management you'd probably not want to bother with two different sized drives. I didn't mention the 300GB 10K Raptors previously due to their limited capacity. Note they're only $15 more apiece than the 1TB RE4 drives in the original parts list. For a total of $300 more you get the same 40% increase in IOPs of the 600GB model, but you'll only have 3TB net space after RAID10. If 3TB is sufficient space for your needs, that extra 40% IOPS makes this config a no brainer. The decreased latency of the 10K drives will give a nice boost to VM read performance, especially when using NFS. Write performance probably won't be much different due to the generous 512MB write cache on the controller. I also forgot to mention that with BBWC enabled you can turn off XFS barriers, which will dramatically speed up Exim queues and Dovecot writes, all writes actually. Again, you probably don't want the spares management overhead of two different disk types on the shelf, but you could stick these 10K 300s in the first 16 slots, and put the 2TB RE4 drive in the last 4 slots, RAID10 on the 10K drives, RAID5 on the 2TB drives. This yields an 8 spindle high IOPS RAID10 of 2.4TB and a lower performance RAID5 of 6TB for near line storage such as your Dovecot alt storage, VM templates, etc, 8.4TB net, 1.6TB less than the original 10TB setup. Total additional cost is $920 for this setup. You'd have two XFS filesystems (with quite different mkfs parameters). > I have been avoiding hardware RAID because of a really bad experience > with RAID 5 on an obsolete controller that eventually died without > replacement and couldn't be recovered. Since then, it's always been > RAID 1 and, after I discovered mdraid, using them as purely HBA with > mdraid for the flexibility of being able to just pull the drives into > a new system if necessary without having to worry about the > controller. Assuming you have the right connector configuration for your drive/enclosure on the replacement card, you can usually swap out one LSI RAID card with any other LSI RAID card in the same, or newer, generation. It'll read the configuration metadata from the disks and be up an running in minutes. This feature has been around all the way back to the AMI/Mylex cards of the late 1990s. LSI acquired both companies, who were #1 and #2 in RAID, which is why LSI is so successful today. Back in those days LSI simply supplied the ASICs to AMI and Mylex. I have an AMI MegaRAID 428, top of the line in 1998, lying around somewhere. Still working when I retired it many years ago. FYI, LSI is the OEM provider of RAID and SAS/SATA HBA ASIC silicon for the tier 1 HBA and mobo down markets. Dell, HP, IBM, Intel, Oracle (Sun), Siemens/Fujitsu, all use LSI silicon and firmware. Some simply rebadge OEM LSI cards with their own model and part numbers. IBM and Dell specifically have been doing this rebadging for well over a decade, long before LSI acquired Mylex and AMI. The Dell PERC/2 is a rebadged AMI MegaRAID 428. Software and hardware RAID each have their pros and cons. I prefer hardware RAID for write cache performance and many administrative reasons, including SAF-TE enclosure management (fault LEDs, alarms, etc) so you know at a glance which drive has failed and needs replacing, email and SNMP notification of events, automatic rebuild, configurable rebuild priority, etc, etc, and good performance with striping and mirroring. Parity RAID performance often lags behind md with heavy workloads but not with light/medium. FWIW I rarely use parity RAID, due to the myriad performance downsides. For ultra high random IOPS workloads, or when I need a single filesystem space larger than the drive limit or practical limit for one RAID HBA, I'll stitch hardware RAID1 or small stripe width RAID 10 arrays (4-8 drives, 2-4 spindles) together with md RAID 0 or 1. >> Both of the drives I've mentioned here are enterprise class drives, >> feature TLER, and are on the LSI MegaRAID SAS hardware compatibility >> list. The price of the 600GB Raptor has come down considerably since I >> designed this system, or I'd have used them instead. >> >> Anyway, lots of option out there. But $6,500 is pretty damn cheap for a >> quality box with 32GB RAM, enterprise RAID card, and 20x10K RPM 600GB >> drives. >> >> The MegaRAID 9280-4i4e has an external SFF8088 port For an additional >> $6,410 you could add an external Norco SAS expander JBOD chassis and 24 >> more 600GB 10K RPM Raptors, for 13.2TB of total net RAID10 space, and 22 >> 10k spindles of IOPS performance from 44 total drives. That's $13K for >> a 5K random IOPS, 13TB, 44 drive NFS RAID COTS server solution, >> $1000/TB, $2.60/IOPS. Significantly cheaper than an HP, Dell, IBM >> solution of similar specs, each of which will set you back at least 20 >> large. > > Would this setup work well too for serving up VM images? I've been > trying to find a solution for the virtualized app servers images as > well but the distributed FSes currently are all bad with random > reads/writes it seems. XFS seem to be good with large files like db > and vm images with random internal write/read so given my time > constraints, it would be nice to have a single configuration that > works generally well for all the needs I have to oversee. Absolutely. If you setup these 20 drives as a single RAID10, soft/hard or hybrid, with the LSI cache set to 100% write-back, with a single XFS filesystem with 10 allocation groups and proper stripe alignment, you'll get maximum performance for pretty much any conceivable workload. Your only limitations will be possible NFS or TCP tuning issues, and maybe having only two GbE ports. For small random IOPS such as Exim queues, Dovecot store, VM image IO, etc, the two GbE ports are plenty. But if you add any large NFS file copies into the mix, such as copying new VM templates or ISO images over, etc, or do backups over NFS instead of directly on the host machine at the XFS level, then two bonded GbE ports might prove a bottleneck. The mobo has 2 PCIe x8 slots and one x4 slot. One of the x8 slots is an x16 physical connector. You'll put the LSI card in the x16 slot. If you mount the Intel SAS expander to the chassis as I do instead of in a slot, you have one free x8 and one free x4 slot. Given the $250 price, I'd simply ad an Intel quad port GbE NIC to the order. Link aggregate all 4 ports on day one and use one IP address for the NFS traffic. Use the two on board ports for management etc. This should give you a theoretical 400MB/s of peak NFS throughput, which should be plenty no matter what workload you throw at it. >> Note the chassis I've spec'd have single PSUs, not the dual or triple >> redundant supplies you'll see on branded hardware. With a relatively >> stable climate controlled environment and a good UPS with filtering, >> quality single supplies are fine. In fact, in the 4U form factor single >> supplies are usually more reliable due to superior IC packaging and >> airflow through the heatsinks, not to mention much quieter. > > Same reason I do my best to avoid 1U servers, the space/heat issues > worries me. Yes, I'm guilty of worrying too much but that had saved me > on several occasions. Just about every 1U server I've seen that's been racked for 3 or more years has warped under its own weight. I even saw an HPQ 2U that was warped this way, badly warped. In this instance the slide rail bolts had never been tightened down to the rack--could spin them by hand. Since the chassis side panels weren't secured, and there was lateral play, the weight of the 6 drives caused the side walls of the case to fold into a mild trapezoid, which allowed the bottom and top panels to bow. Let this be a lesson boys and girls: always tighten your rack bolts. :) -- Stan From tss at iki.fi Wed Apr 11 10:23:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 10:23:56 +0300 Subject: [Dovecot] Bug tracker In-Reply-To: References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: On 11.4.2012, at 9.49, Antoine Nguyen wrote: > Maybe Redmine ? (http://www.redmine.org/ I don't think it either supports: > I would have the option of adding a comment that doesn't go to the mailing list But I'm beginning to think that no BTS supports that in the way I want. And it's not a huge problem for me really, just means that this mailing list may get a bit more pointless notifications than necessary. Although I'm also not seeing much documentation about email notifications in general. Can it be configured to send comment updates, but no other updates? I guess this could also be kludged by dropping unwanted emails in MTA. From janfrode at tanso.net Wed Apr 11 10:25:22 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 11 Apr 2012 09:25:22 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> <20120411064419.GA25675@dibs.tanso.net> Message-ID: <20120411072522.GA26184@dibs.tanso.net> On Wed, Apr 11, 2012 at 09:49:18AM +0300, Timo Sirainen wrote: > > > > I didn't see open source as a requirement, so then I would give a plug > > for Jira, which is the nicest/prettiest thing :-) > > I don't think it supports one of my requirements: > > > I would have the option of adding a comment that doesn't go to the mailing list > > Unless that's been added in a newer version. > There is an option for restricting who can view your comment, plus "Email notifications will only be sent to people who have permission to view the relevant issue" http://confluence.atlassian.com/display/JIRA/Creating+a+Notification+Scheme so I would expect it to be possible to define that the mailinglist is not member of a group-b, while everyone else is, and restrict the comment to that group. But best would probably be to discuss it with atlassion support... -jf From lee at standen.id.au Wed Apr 11 10:59:28 2012 From: lee at standen.id.au (Lee Standen) Date: Wed, 11 Apr 2012 15:59:28 +0800 Subject: [Dovecot] Bug tracker In-Reply-To: <20120411072522.GA26184@dibs.tanso.net> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> <20120411064419.GA25675@dibs.tanso.net> <20120411072522.GA26184@dibs.tanso.net> Message-ID: We use JIRA internally and it's probably the best bug tracker I've ever seen. The permissions system is very flexible, and to the best of my knowledge it should support everything you've described in your original post. On Wed, Apr 11, 2012 at 3:25 PM, Jan-Frode Myklebust wrote: > On Wed, Apr 11, 2012 at 09:49:18AM +0300, Timo Sirainen wrote: > > > > > > I didn't see open source as a requirement, so then I would give a plug > > > for Jira, which is the nicest/prettiest thing :-) > > > > I don't think it supports one of my requirements: > > > > > I would have the option of adding a comment that doesn't go to the > mailing list > > > > Unless that's been added in a newer version. > > > > There is an option for restricting who can view your comment, plus > "Email notifications will only be sent to people who have permission to > view the relevant issue" > > > http://confluence.atlassian.com/display/JIRA/Creating+a+Notification+Scheme > > so I would expect it to be possible to define that the mailinglist is > not member of a group-b, while everyone else is, and restrict the comment > to that group. > > But best would probably be to discuss it with atlassion support... > > > -jf > From ngu.antoine at gmail.com Wed Apr 11 11:05:05 2012 From: ngu.antoine at gmail.com (Antoine Nguyen) Date: Wed, 11 Apr 2012 10:05:05 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: Le 11 avril 2012 09:23, Timo Sirainen a ?crit : > On 11.4.2012, at 9.49, Antoine Nguyen wrote: > > > Maybe Redmine ? (http://www.redmine.org/ > > I don't think it either supports: > > > I would have the option of adding a comment that doesn't go to the > mailing list > > But I'm beginning to think that no BTS supports that in the way I want. > And it's not a huge problem for me really, just means that this mailing > list may get a bit more pointless notifications than necessary. > > Although I'm also not seeing much documentation about email notifications > in general. Can it be configured to send comment updates, but no other > updates? I guess this could also be kludged by dropping unwanted emails in > MTA. > > AFAIK, yes. You can choose on which kind of event Redmine sends a notification. The documentation is not the best I've seen but the administration panel is pretty simple to understand. Antoine From warren at decoy.co.za Wed Apr 11 11:24:55 2012 From: warren at decoy.co.za (Warren Baker) Date: Wed, 11 Apr 2012 10:24:55 +0200 Subject: [Dovecot] Bug tracker In-Reply-To: References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: On Wed, Apr 11, 2012 at 10:05 AM, Antoine Nguyen wrote: > Le 11 avril 2012 09:23, Timo Sirainen a ?crit : >> > I would have the option of adding a comment that doesn't go to the >> mailing list >> >> But I'm beginning to think that no BTS supports that in the way I want. >> And it's not a huge problem for me really, just means that this mailing >> list may get a bit more pointless notifications than necessary. >> >> Although I'm also not seeing much documentation about email notifications >> in general. Can it be configured to send comment updates, but no other >> updates? I guess this could also be kludged by dropping unwanted emails in >> MTA. >> >> > AFAIK, yes. You can choose on which kind of event Redmine sends a > notification. The documentation is not the best I've seen but the > administration panel is pretty simple to understand. > There is a default notification option which is set to 'Only for things I watch or I'm involved in' - which in your case Timo you would receive everything since you would be the project owner. There are also global settings that you can configure which provide the ability to send notifications for certain events only: - Issue added - Issue updated - Note added - Status updated - Priority updated - News added - Comment added to a news - Document added - File added - Message added - Wiki page added - Wiki page updated I think there is also a plugin to disable email notifications entirely for a user. -- .warren From mlists at edicom.eu Wed Apr 11 12:35:25 2012 From: mlists at edicom.eu (Miguel Tormo) Date: Wed, 11 Apr 2012 11:35:25 +0200 Subject: [Dovecot] Proxy and SSO (single sign-on) In-Reply-To: <201204041347.47817.mlists@edicom.eu> References: <201204041318.00760.mlists@edicom.eu> <48AD358F-ECC4-48F5-8CAB-9B91C8BDEEE3@iki.fi> <201204041347.47817.mlists@edicom.eu> Message-ID: <201204111135.25742.mlists@edicom.eu> El Mi?rcoles, 4 de Abril de 2012 13:47:47 Miguel Tormo escribi?: > El Mi?rcoles, 4 de Abril de 2012 13:21:33 Timo Sirainen escribi?: > > On 4.4.2012, at 14.18, Miguel Tormo wrote: > > > > > I have a running setup with a dovecot imap4/pop3 proxy to a few dovecot backend servers which actually store the mailboxes. This is running smoothly and allows me to transparently distribute mailboxes. > > > I'm using some "extrafield" configured in the LDAP passdb. > > > > > > However, now I would like to use GSSAPI (preferred) and NTLM for single sign-on. Both are pretty straightforward to configure in a single instance environment, but I don't know if they would work with proxy. For example, with GSSAPI there are two cases: > > > 1) Just use gssapi mechanism, without PAM. Then, it a user presents a ticket the passdb ldap is not used, so the extrafields are never read. > > > > The patch in http://dovecot.org/list/dovecot/2012-March/064331.html makes this work I think. I still haven't managed to look into it much though. > > > > > > It definitely is worth a look. I wonder if it would make ldap extrafields lookups work with gssapi auth, I will try it and post the results. Well just in case someone is interested: I applied Sam Morris' patch and got this working. Very nice indeed. To keep in mind: - It's necessary to set a master password for the backend server so the proxy instance can impersonate any user when connecting to the backend instance. I needed to do this to make GSSAPI with proxy work (thus the proxy instance is who actually does the authentication), however NTLM worked without a masster password, it seems this authenticatiom mechanism can be forwarded as is. - In the LDAP configuration, "auth bind" must be set to "no". Obviously, when using sso you don't provide a password, so there is no way it can bind to the LDAP server with your credentials. There is still a problem that might be more related to the MUA, but still if anyone has a suggestion I would appreciate it. The problem is the following: if you are a roaming user (ie: with a laptop) when you are outside the LAN you cannot get a kerberos ticket thus the GSSAPI auth fails. Then you have to change your account configuration and select a password-based authentication mechanism. This is not very convenient. However, this is different if compared with kerberos authentication with a web browser: if you have a valid ticket, access is granted; if not, the browser prompts for user/password and then you are granted access if the supplied credentials are valid. Do you have any idea if something like this is possible to accomplish with IMAP? Thank you and regards, From CMarcus at Media-Brokers.com Wed Apr 11 13:20:47 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 11 Apr 2012 06:20:47 -0400 Subject: [Dovecot] Bug tracker In-Reply-To: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> References: <75774035-04FC-49F6-B0C1-24A462DCDC67@iki.fi> Message-ID: <4F855AFF.7050704@Media-Brokers.com> On 2012-04-11 2:26 AM, Timo Sirainen wrote: > Notifications about new bugs won't go to the mailing list (most > likely it was created due to a recent mailing list post). I actually would like to see these, and I imagine I'm not alone... If you'd prefer they don't go to the users list, maybe create a new 'notifications' (or maybe even a -dev) list that anyone could subscribe to to see these? > So, any suggestions for what software could do these things? I think > Request Tracker has those features, but it's not really the > nicest/prettiest thing. +1 to Redmine. I only used it for a short time, but I liked it, and the fact that you can integrate it with git or mercurial is a big plus. Yes, it is a full blown bug tracker, but if, in the future, you add more developers (maybe you get some angel funding), you already have a full blown bug tracker in place, including the extras it comes with. -- Best regards, Charles From interfasys at gmail.com Wed Apr 11 12:05:59 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Wed, 11 Apr 2012 10:05:59 +0100 Subject: [Dovecot] Bug tracker Message-ID: <4F854977.9000804@gmail.com> I second that. New version 3.2 supports sub-projects, editions and components and is user-friendly. Cheers, Olivier > On Wed, Apr 11, 2012 at 09:26:20AM +0300, Timo Sirainen wrote: > > So, any suggestions for what software could do these things? I think > Request > Tracker has those features, but it's not really the nicest/prettiest > thing. > > May be worth to take a look at http://www.thebuggenie.com/. > > Thomas > From helga.mayer at uni-hohenheim.de Wed Apr 11 17:13:21 2012 From: helga.mayer at uni-hohenheim.de (Helga Mayer) Date: Wed, 11 Apr 2012 16:13:21 +0200 Subject: [Dovecot] Problems with Apple Mail and attachments Message-ID: <20120411161321.165760epurnu7i4h@webmail.uni-hohenheim.de> Hello, has anyone experiences with the Apple mail client, to be precise : Mail 5.2? When I copy a message with an attachment from the inbox to a different folder using 'move' or 'copy' , Apple Mail does not properly display the attachment in the new folder. When I copy the same message using 'archive' the attachment shows up properly. There is no difference on the server side. We are using maildirs. The raw messages in the various folders are identical. Deleting plist on the Mac didn't help. After removing and recreation of the account (client side) the attachments are displayed correctly again. Is this an apple problem or has it got anything to do with dovecot.index.cache ? Kind Regards Helga Mayer From yzhou at medplus.com Wed Apr 11 17:49:05 2012 From: yzhou at medplus.com (Zhou, Yan) Date: Wed, 11 Apr 2012 14:49:05 +0000 Subject: [Dovecot] ldap idle connection timeout in DoveCot 1.0.13? Message-ID: Hi there, We are using DoveCot 1.0.13, it connects to LDAP server for authentication. It seems that DoveCot keeps the idle LDAP connection open. Our firewall is terminating these connections after some time of idle activity (2 hours), then, we run into authentication problem. If we restart either LDAP or DoveCot, then it is fine. Can we set some kind of LDAP idle connection timeout in DoveCot? /etc/dovecot-ldap.conf. I do not see any configuration available for 1.0.13. Another alternative is to set idle connection timeout in LDAP, but we prefer doing that in DoveCot. Thanks, Yan Confidentiality Notice: The information contained in this electronic transmission is confidential and may be legally privileged. It is intended only for the addressee(s) named above. If you are not an intended recipient, be aware that any disclosure, copying, distribution or use of the information contained in this transmission is prohibited and may be unlawful. If you have received this transmission in error, please notify us by telephone (513) 229-5500 or by email (postmaster at MedPlus.com). After replying, please erase it from your computer system. From fxmulder at gmail.com Wed Apr 11 18:34:54 2012 From: fxmulder at gmail.com (James Devine) Date: Wed, 11 Apr 2012 09:34:54 -0600 Subject: [Dovecot] lazy_expunge questions In-Reply-To: <4791F1D6-6BF2-4FB8-82CE-7C99DB50B4EA@iki.fi> References: <4791F1D6-6BF2-4FB8-82CE-7C99DB50B4EA@iki.fi> Message-ID: On Wed, Apr 11, 2012 at 12:01 AM, Timo Sirainen wrote: > On 10.4.2012, at 23.19, James Devine wrote: > > > I'm messing around with lazy expunge and I just had a couple of > questions. > > > > If I delete messages or folders with messages, I see them being moved to > > the expunge namespace, but if I delete an empty folder, all traces of it > > seem to disappear. Is there a way to move empty folders to the expunge > > namespace also? > > If you give 3 namespaces to lazyexpunge, it uses another way of moving > stuff, and that includes also moving empty folders. But with 1 namespace it > doesn't currently move it. Perhaps it should. > > > My second question is in regards to restoring folders. If I try to > execute > > the imap command 'RENAME EXPUNGED/TEST INBOX/TEST' I get 'NO [CANNOT] > Can't > > rename mailboxes to/from expunge namespace'. Just wondering if this is > > possible or do I need to recreate the folder and then move messages into > it. > > You need to recreate + move. Renaming across namespaces is a little tricky. > > It looks like 3 namespace support in the lazy expunge plugin was removed in 2.1+ ? From tss at iki.fi Wed Apr 11 19:10:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 19:10:36 +0300 Subject: [Dovecot] lazy_expunge questions In-Reply-To: References: <4791F1D6-6BF2-4FB8-82CE-7C99DB50B4EA@iki.fi> Message-ID: On 11.4.2012, at 18.34, James Devine wrote: > It looks like 3 namespace support in the lazy expunge plugin was removed in > 2.1+ ? Yeah, so it seems. Updated the wiki2. From tss at iki.fi Wed Apr 11 19:36:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 19:36:30 +0300 Subject: [Dovecot] ldap idle connection timeout in DoveCot 1.0.13? In-Reply-To: References: Message-ID: On 11.4.2012, at 17.49, Zhou, Yan wrote: > We are using DoveCot 1.0.13, it connects to LDAP server for authentication. It seems that DoveCot keeps the idle LDAP connection open. Yes. > Our firewall is terminating these connections after some time of idle activity (2 hours), then, we run into authentication problem. If we restart either LDAP or DoveCot, then it is fine. > > Can we set some kind of LDAP idle connection timeout in DoveCot? /etc/dovecot-ldap.conf. I do not see any configuration available for 1.0.13. No. But if you upgrade to a newer Dovecot (v2.x probably) this is solved by automatic transparent reconnection. From tss at iki.fi Wed Apr 11 19:37:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Apr 2012 19:37:59 +0300 Subject: [Dovecot] Problems with Apple Mail and attachments In-Reply-To: <20120411161321.165760epurnu7i4h@webmail.uni-hohenheim.de> References: <20120411161321.165760epurnu7i4h@webmail.uni-hohenheim.de> Message-ID: On 11.4.2012, at 17.13, Helga Mayer wrote: > has anyone experiences with the Apple mail client, to be precise : Mail 5.2? > When I copy a message with an attachment from the inbox to a different folder using 'move' or 'copy' , Apple Mail does not properly display the attachment in the new folder. .. > Is this an apple problem or has it got anything to do with dovecot.index.cache ? Highly unlikely to be a Dovecot problem. Of course, you can verify if it's a caching problem by disabling indexes and seeing if you can reproduce the problem: mail_location = maildir:~/Maildir:INDEX=MEMORY From thierry at odry.net Wed Apr 11 19:44:59 2012 From: thierry at odry.net (Thierry de Montaudry) Date: Wed, 11 Apr 2012 18:44:59 +0200 Subject: [Dovecot] Problems with Apple Mail and attachments In-Reply-To: <20120411161321.165760epurnu7i4h@webmail.uni-hohenheim.de> References: <20120411161321.165760epurnu7i4h@webmail.uni-hohenheim.de> Message-ID: On 11 Apr 2012, at 16:13, Helga Mayer wrote: > Hello, > > has anyone experiences with the Apple mail client, to be precise : Mail 5.2? > When I copy a message with an attachment from the inbox to a different folder using 'move' or 'copy' , Apple Mail does not properly display the attachment in the new folder. > > When I copy the same message using 'archive' the attachment shows up properly. > There is no difference on the server side. We are using maildirs. The raw > messages in the various folders are identical. > Deleting plist on the Mac didn't help. > After removing and recreation of the account (client side) the attachments are displayed correctly again. > > Is this an apple problem or has it got anything to do with dovecot.index.cache ? > > Kind Regards > Helga Mayer > > Hi Helga, I'm using Apple Mail 5.2 with a dovecot server (2.0.13, with maildir), and recently had no trouble like that, even though working with a mailbox that has about 10000 emails, a lot of folders, and moving mails around all the time. I've seen a similar problem a while ago (1 year maybe more), but used the mailbox Rebuild option on the client, which fixed it without having to delete and recreate the account. Your problem might just be a local index corruption, which can happen when loosing your Internet connection. Regards, Thierry From lists at wildgooses.com Wed Apr 11 19:50:09 2012 From: lists at wildgooses.com (Ed W) Date: Wed, 11 Apr 2012 17:50:09 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F853059.1000706@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> Message-ID: <4F85B641.5060103@wildgooses.com> Re XFS. Have you been watching BTRFS recently? I will concede that despite the authors considering it production ready I won't be using it for my servers just yet. However, it's benchmarking on single disk benchmarks fairly similarly to XFS and in certain cases (multi-threaded performance) can be somewhat better. I haven't yet seen any benchmarks on larger disk arrays yet, eg 6+ disks, so no idea how it scales up. Basically what I have seen seems "competitive" I don't have such hardware spare to benchmark, but I would be interested to hear from someone who benchmarks your RAID1+linear+XFS suggestion, especially if they have compared a cutting edge btrfs kernel on the same array? One of the snags of md RAID1 vs RAID6 is the lack of checksumming in the event of bad blocks. (I'm not sure what actually happens when md scrubbing finds a bad sector with raid1..?). For low performance requirements I have become paranoid and been using RAID6 vs RAID10, filesystems with sector checksums seem attractive... Regards Ed W From adrian.minta at gmail.com Wed Apr 11 23:48:00 2012 From: adrian.minta at gmail.com (Adrian Minta) Date: Wed, 11 Apr 2012 23:48:00 +0300 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F85B641.5060103@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> Message-ID: <4F85EE00.9030208@gmail.com> On 04/11/12 19:50, Ed W wrote: ... > One of the snags of md RAID1 vs RAID6 is the lack of checksumming in > the event of bad blocks. (I'm not sure what actually happens when md > scrubbing finds a bad sector with raid1..?). For low performance > requirements I have become paranoid and been using RAID6 vs RAID10, > filesystems with sector checksums seem attractive... > RAID6 is very slow for write operations. That's why is the worst choice for maildir. From CMarcus at Media-Brokers.com Wed Apr 11 21:50:11 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 11 Apr 2012 14:50:11 -0400 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F85EE00.9030208@gmail.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F85EE00.9030208@gmail.com> Message-ID: <4F85D263.3070201@Media-Brokers.com> On 2012-04-11 4:48 PM, Adrian Minta wrote: > On 04/11/12 19:50, Ed W wrote: >> One of the snags of md RAID1 vs RAID6 is the lack of checksumming in >> the event of bad blocks. (I'm not sure what actually happens when md >> scrubbing finds a bad sector with raid1..?). For low performance >> requirements I have become paranoid and been using RAID6 vs RAID10, >> filesystems with sector checksums seem attractive... > RAID6 is very slow for write operations. That's why is the worst choice > for maildir. He did say '"For *low* *performance* requirements..." ... ;) -- Best regards, Charles From stan at hardwarefreak.com Thu Apr 12 01:46:44 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 11 Apr 2012 17:46:44 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F8409DA.1090202@gmail.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F8409DA.1090202@gmail.com> Message-ID: <4F8609D4.2040205@hardwarefreak.com> On 4/10/2012 5:22 AM, Adrian Minta wrote: > On 04/10/12 08:00, Stan Hoeppner wrote: >> Interestingly, I designed a COTS server back in January to handle at >> least 5k concurrent IMAP users, using best of breed components. If you >> or someone there has the necessary hardware skills, you could assemble >> this system and simply use it for NFS instead of Dovecot. The parts >> list: >> secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=17069985 > > Don't forget the Battery Backup Unit for RAID card !!! Heh, thanks for the reminder Adrian. :) I got to your email a little late--already corrected the omission. Yes, battery or flash backup for the RAID cache is always a necessity when doing write-back caching. -- Stan From alietsantiesteban at gmail.com Thu Apr 12 02:03:04 2012 From: alietsantiesteban at gmail.com (Aliet Santiesteban Sifontes) Date: Wed, 11 Apr 2012 18:03:04 -0500 Subject: [Dovecot] ldap idle connection timeout in DoveCot 1.0.13? In-Reply-To: References: Message-ID: I had this problem running Dovecot 2.x where LDAP servers are located on another firewall zone, we use Juniper SSG550. The problem was that the firewall was dropping the ldap idle connections so client authentication was failing in dovecot for a while and after a time it reconnects, Dovecot/Openldap-Server never knows that the firewall has dropped the connection because this is the default, the firewall doesn't send TCP -Reset to the client and the server, in Juniper/Netscreen you can do a workaround to speed up the process by configuring the zone to send reset back to the client and the server. Check you have on the firewall: set flow tcp-mss unset flow no-tcp-seq-check set flow tcp-syn-check unset flow tcp-syn-bit-check set flow reverse-route clear-text prefer set flow reverse-route tunnel always Edit your zone and enable "If TCP non SYN, send RESET back" checkbox: This fixed the delay for us, it would be a nice feature at dovecot side... best regards El 11 de abril de 2012 11:36, Timo Sirainen escribi?: > On 11.4.2012, at 17.49, Zhou, Yan wrote: > > > We are using DoveCot 1.0.13, it connects to LDAP server for > authentication. It seems that DoveCot keeps the idle LDAP connection open. > > Yes. > > > Our firewall is terminating these connections after some time of idle > activity (2 hours), then, we run into authentication problem. If we restart > either LDAP or DoveCot, then it is fine. > > > > Can we set some kind of LDAP idle connection timeout in DoveCot? > /etc/dovecot-ldap.conf. I do not see any configuration available for > 1.0.13. > > No. But if you upgrade to a newer Dovecot (v2.x probably) this is solved > by automatic transparent reconnection. > > From stan at hardwarefreak.com Thu Apr 12 04:18:08 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 11 Apr 2012 20:18:08 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F85B641.5060103@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> Message-ID: <4F862D50.8000206@hardwarefreak.com> On 4/11/2012 11:50 AM, Ed W wrote: > Re XFS. Have you been watching BTRFS recently? > > I will concede that despite the authors considering it production ready > I won't be using it for my servers just yet. However, it's benchmarking > on single disk benchmarks fairly similarly to XFS and in certain cases > (multi-threaded performance) can be somewhat better. I haven't yet seen > any benchmarks on larger disk arrays yet, eg 6+ disks, so no idea how it > scales up. Basically what I have seen seems "competitive" Links? > I don't have such hardware spare to benchmark, but I would be interested > to hear from someone who benchmarks your RAID1+linear+XFS suggestion, > especially if they have compared a cutting edge btrfs kernel on the same > array? http://btrfs.boxacle.net/repository/raid/history/History_Mail_server_simulation._num_threads=128.html This is with an 8 wide LVM stripe over 8 17 drive hardware RAID0 arrays. If the disks had been setup as a concat of 68 RAID1 pairs, XFS would have turned in numbers significantly higher, anywhere from a 100% increase to 500%. It's hard to say because the Boxacle folks didn't show the XFG AG config they used. The concat+RAID1 setup can decrease disk seeks by many orders of magnitude vs striping. Everyone knows as seeks go down IOPS go up. Even with this very suboptimal disk setup, XFS still trounces everything but JFS which is a close 2nd. BTRFS is way down in the pack. It would be nice to see these folks update these results with a 3.2.6 kernel, as both BTRFS and XFS have improved significantly since 2.6.35. EXT4 and JFS have seen little performance work since. In fact JFS has seen no commits but bug fixes and changes to allow compiling with recent kernels. > One of the snags of md RAID1 vs RAID6 is the lack of checksumming in the > event of bad blocks. (I'm not sure what actually happens when md > scrubbing finds a bad sector with raid1..?). For low performance > requirements I have become paranoid and been using RAID6 vs RAID10, > filesystems with sector checksums seem attractive... Except we're using hardware RAID1 here and mdraid linear. Thus the controller takes care of sector integrity. RAID6 yields nothing over RAID10, except lower performance, and more usable space if more than 4 drives are used. -- Stan From centos.admin at gmail.com Thu Apr 12 05:23:19 2012 From: centos.admin at gmail.com (Emmanuel Noobadmin) Date: Thu, 12 Apr 2012 10:23:19 +0800 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F862D50.8000206@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> Message-ID: On 4/12/12, Stan Hoeppner wrote: > On 4/11/2012 11:50 AM, Ed W wrote: >> One of the snags of md RAID1 vs RAID6 is the lack of checksumming in the >> event of bad blocks. (I'm not sure what actually happens when md >> scrubbing finds a bad sector with raid1..?). For low performance >> requirements I have become paranoid and been using RAID6 vs RAID10, >> filesystems with sector checksums seem attractive... > > Except we're using hardware RAID1 here and mdraid linear. Thus the > controller takes care of sector integrity. RAID6 yields nothing over > RAID10, except lower performance, and more usable space if more than 4 > drives are used. How would the control ensure sector integrity unless it is writing additional checksum information to disk? I thought only a few filesystems like ZFS does the sector checksum to detect if any data corruption occurred. I suppose the controller could throw an error if the two drives returned data that didn't agree with each other but it wouldn't know which is the accurate copy but that wouldn't protect the integrity of the data, at least not directly without additional human intervention I would think. From tlx at leuxner.net Thu Apr 12 10:11:32 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 12 Apr 2012 09:11:32 +0200 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes Message-ID: <20120412071132.GA2583@nihlus.leuxner.net> Some change between bf5ae73e9475 and 584bd77c38fd seems to have broken something in the SSL Handshake. A previously valid server certificate is deemed invalid by various mail clients. http://hg.dovecot.org/dovecot-2.1/rev/bf5ae73e9475 works fine while http://hg.dovecot.org/dovecot-2.1/rev/584bd77c38fd does not. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From tss at iki.fi Thu Apr 12 10:43:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Apr 2012 10:43:22 +0300 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: <20120412071132.GA2583@nihlus.leuxner.net> References: <20120412071132.GA2583@nihlus.leuxner.net> Message-ID: On 12.4.2012, at 10.11, Thomas Leuxner wrote: > Some change between bf5ae73e9475 and 584bd77c38fd seems to have broken > something in the SSL Handshake. A previously valid server certificate is > deemed invalid by various mail clients. > > http://hg.dovecot.org/dovecot-2.1/rev/bf5ae73e9475 works fine while > http://hg.dovecot.org/dovecot-2.1/rev/584bd77c38fd does not. What kind of a certificate do you have? You have an intermediary cert that exists only in ssl_ca file? I couldn't reproduce this with a test. But anyway, reverted for now: http://hg.dovecot.org/dovecot-2.1/rev/f80f18d0ffa3 Now how do I fix the memory leak then?... From tss at iki.fi Thu Apr 12 10:49:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Apr 2012 10:49:12 +0300 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: References: <20120412071132.GA2583@nihlus.leuxner.net> Message-ID: On 12.4.2012, at 10.43, Timo Sirainen wrote: > On 12.4.2012, at 10.11, Thomas Leuxner wrote: > >> Some change between bf5ae73e9475 and 584bd77c38fd seems to have broken >> something in the SSL Handshake. A previously valid server certificate is >> deemed invalid by various mail clients. >> >> http://hg.dovecot.org/dovecot-2.1/rev/bf5ae73e9475 works fine while >> http://hg.dovecot.org/dovecot-2.1/rev/584bd77c38fd does not. > > What kind of a certificate do you have? You have an intermediary cert that exists only in ssl_ca file? I couldn't reproduce this with a test. But anyway, reverted for now: http://hg.dovecot.org/dovecot-2.1/rev/f80f18d0ffa3 > > Now how do I fix the memory leak then?... http://hg.dovecot.org/dovecot-2.1/rev/85ad4baedd43 ? From tlx at leuxner.net Thu Apr 12 11:16:45 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 12 Apr 2012 10:16:45 +0200 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: References: <20120412071132.GA2583@nihlus.leuxner.net> Message-ID: <20120412081645.GB2583@nihlus.leuxner.net> On Thu, Apr 12, 2012 at 10:43:22AM +0300, Timo Sirainen wrote: > What kind of a certificate do you have? You have an intermediary cert that exists only in ssl_ca file? I couldn't reproduce this with a test. But anyway, reverted for now: http://hg.dovecot.org/dovecot-2.1/rev/f80f18d0ffa3 > Thawte. They only do intermediates for some time now. $ openssl x509 -in /etc/ssl/certs/spectre_leuxner_net_2011.crt -noout -subject -issuer -dates subject= /O=spectre.leuxner.net/OU=Go to https://www.thawte.com/repository/index.html/OU=Thawte SSL123 certificate/OU=Domain Validated/CN=spectre.leuxner.net issuer= /C=US/O=Thawte, Inc./OU=Domain Validated SSL/CN=Thawte DV SSL CA notBefore=May 16 00:00:00 2011 GMT notAfter=Jun 14 23:59:59 2012 GMT [...] ssl_ca = From tss at iki.fi Thu Apr 12 11:17:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Apr 2012 11:17:50 +0300 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: <20120412081645.GB2583@nihlus.leuxner.net> References: <20120412071132.GA2583@nihlus.leuxner.net> <20120412081645.GB2583@nihlus.leuxner.net> Message-ID: <11B240E5-087D-43A2-A687-633AB3C4A367@iki.fi> On 12.4.2012, at 11.16, Thomas Leuxner wrote: > On Thu, Apr 12, 2012 at 10:43:22AM +0300, Timo Sirainen wrote: >> What kind of a certificate do you have? You have an intermediary cert that exists only in ssl_ca file? I couldn't reproduce this with a test. But anyway, reverted for now: http://hg.dovecot.org/dovecot-2.1/rev/f80f18d0ffa3 >> > > Thawte. They only do intermediates for some time now. But do you keep your intermediate cert in ssl_ca file or ssl_cert file? From tlx at leuxner.net Thu Apr 12 11:33:58 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 12 Apr 2012 10:33:58 +0200 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: <11B240E5-087D-43A2-A687-633AB3C4A367@iki.fi> References: <20120412071132.GA2583@nihlus.leuxner.net> <20120412081645.GB2583@nihlus.leuxner.net> <11B240E5-087D-43A2-A687-633AB3C4A367@iki.fi> Message-ID: <20120412083358.GA14349@nihlus.leuxner.net> On Thu, Apr 12, 2012 at 11:17:50AM +0300, Timo Sirainen wrote: > But do you keep your intermediate cert in ssl_ca file or ssl_cert file? Separate. Root and intermediate are in ssl_ca: $ cat /etc/ssl/certs/SSL123_CA_Bundle.pem -----BEGIN CERTIFICATE----- MIIEjzCCA3egAwIBAgIQdhASihe2grs6H50amjXAkjANBgkqhkiG9w0BAQUFADCB qTELMAkGA1UEBhMCVVMxFTATBgNVBAoTDHRoYXd0ZSwgSW5jLjEoMCYGA1UECxMf Q2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjE4MDYGA1UECxMvKGMpIDIw MDYgdGhhd3RlLCBJbmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxHzAdBgNV BAMTFnRoYXd0ZSBQcmltYXJ5IFJvb3QgQ0EwHhcNMTAwMjE4MDAwMDAwWhcNMjAw MjE3MjM1OTU5WjBeMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMVGhhd3RlLCBJbmMu MR0wGwYDVQQLExREb21haW4gVmFsaWRhdGVkIFNTTDEZMBcGA1UEAxMQVGhhd3Rl IERWIFNTTCBDQTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMuYyTY/ 0pzYFgfUSWP5g7DoAi3MXFp0l6YT7xMT3gV8p+bKACPaOfnvE89Sxa+a48q+84LZ iz2q4cyuiFBmoy3sYRR1SasOJPGsRFsLKKIzIHYeBmBqZwVxi7pmYhZ6s20Nx9CU QMaMPR6SDGI0DUSJ1feJ/intGI/2mysI92qr2EiXWvSf7Qx1UiL31V6EAJ/ASg0x d0xk0BLmDzrwocDVXB3nXy3C99Y2GNmVbkROyVgUTbaOu83eYh76W7W9GCuYrKyT P1Ba9RQLos+2855PWs1awzYj2hqvsE3WSiIDj0MCGb3qrN3EejUyFPFyLghVQAz0 B0FBrzg3hClCslUCAwEAAaOB/DCB+TAyBggrBgEFBQcBAQQmMCQwIgYIKwYBBQUH MAGGFmh0dHA6Ly9vY3NwLnRoYXd0ZS5jb20wEgYDVR0TAQH/BAgwBgEB/wIBADA0 BgNVHR8ELTArMCmgJ6AlhiNodHRwOi8vY3JsLnRoYXd0ZS5jb20vVGhhd3RlUENB LmNybDAOBgNVHQ8BAf8EBAMCAQYwKQYDVR0RBCIwIKQeMBwxGjAYBgNVBAMTEVZl cmlTaWduTVBLSS0yLTExMB0GA1UdDgQWBBSrRORd7IPH2cCFn/fhxpeQsIw/mDAf BgNVHSMEGDAWgBR7W0XPr87Lev0xkhpqtvNG61dIUDANBgkqhkiG9w0BAQUFAAOC AQEABLr7rLv8S1QRoy2Iszy9AG2KGraNxMGD+MdTKsEybjqBoVR92ho/OkVPNudC sApChZegrPvlh6eDT+ixt5tYZW4mgAuSTUdVuWEWUWXpK/Fo2Vi4A4HRt2Yc07zF pntfPsU4RnbndbSgDEvOosKpwcw2c3v7uSQkoF6n9vq7DChDnh3wTvA/2CSwIdxt Le6/Wjv6iJx0bK8h3ZLswxXvlHUmRtamP79mSKod790n5rdRiTh9E4QMQPzQtfHg 2/lPL0ActI5HImG4TJbe8F8Rfk8R2exQRyIOxR3iZEnnaGNFOorZcfRe8W63FE0+ bxQe3FL+vN8MvSk/dvsRX2hoFQ== -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- MIIERTCCA66gAwIBAgIQM2VQCHmtc+IwueAdDX+skTANBgkqhkiG9w0BAQUFADCB zjELMAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJ Q2FwZSBUb3duMR0wGwYDVQQKExRUaGF3dGUgQ29uc3VsdGluZyBjYzEoMCYGA1UE CxMfQ2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjEhMB8GA1UEAxMYVGhh d3RlIFByZW1pdW0gU2VydmVyIENBMSgwJgYJKoZIhvcNAQkBFhlwcmVtaXVtLXNl cnZlckB0aGF3dGUuY29tMB4XDTA2MTExNzAwMDAwMFoXDTIwMTIzMDIzNTk1OVow gakxCzAJBgNVBAYTAlVTMRUwEwYDVQQKEwx0aGF3dGUsIEluYy4xKDAmBgNVBAsT H0NlcnRpZmljYXRpb24gU2VydmljZXMgRGl2aXNpb24xODA2BgNVBAsTLyhjKSAy MDA2IHRoYXd0ZSwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MR8wHQYD VQQDExZ0aGF3dGUgUHJpbWFyeSBSb290IENBMIIBIjANBgkqhkiG9w0BAQEFAAOC AQ8AMIIBCgKCAQEArKDw+4BZ1JzHpM+doVlzCRBFDA0sbmjxbFtIaElZN/wLMxnC d3/MEC2VNBzm600JpxzSuMmXNgK3idQkXwbAzESUlI0CYm/rWt0RjSiaXISQEHoN vXRmL2o4oOLVVETrHQefB7pv7un9Tgsp9T6EoAHxnKv4HH6JpOih2HFlDaNRe+68 0iJgDblbnd+6/FFbC6+Ysuku6QToYofeK8jXTsFMZB7dz4dYukpPymgHHRydSsbV L5HMfHFyHMXAZ+sy/cmSXJTahcCbv1N9Kwn0jJ2RH5dqUsveCTakd9h7h1BE1T5u KWn7OUkmHgmlgHtALevoJ4XJ/mH9fuZ8lx3VnQIDAQABo4HCMIG/MA8GA1UdEwEB /wQFMAMBAf8wOwYDVR0gBDQwMjAwBgRVHSAAMCgwJgYIKwYBBQUHAgEWGmh0dHBz Oi8vd3d3LnRoYXd0ZS5jb20vY3BzMA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQU e1tFz6/Oy3r9MZIaarbzRutXSFAwQAYDVR0fBDkwNzA1oDOgMYYvaHR0cDovL2Ny bC50aGF3dGUuY29tL1RoYXd0ZVByZW1pdW1TZXJ2ZXJDQS5jcmwwDQYJKoZIhvcN AQEFBQADgYEAhKhMyT4qvJrizI8LsiV3xGGJiWNa1KMVQNT7Xj+0Q+pjFytrmXSe Cajd1FYVLnp5MV9jllMbNNkV6k9tcMq+9oKp7dqFd8x2HGqBCiHYQZl/Xi6Cweiq 95OBBaqStB+3msAHF/XLxrRMDtdW3HEgdDjWdMbWj2uvi42gbCkLYeA= -----END CERTIFICATE----- $ dovecot --version 2.1.4 (584bd77c38fd) Seems to have fixed it. Thanks. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From tss at iki.fi Thu Apr 12 11:35:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Apr 2012 11:35:48 +0300 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: <20120412083358.GA14349@nihlus.leuxner.net> References: <20120412071132.GA2583@nihlus.leuxner.net> <20120412081645.GB2583@nihlus.leuxner.net> <11B240E5-087D-43A2-A687-633AB3C4A367@iki.fi> <20120412083358.GA14349@nihlus.leuxner.net> Message-ID: On 12.4.2012, at 11.33, Thomas Leuxner wrote: > On Thu, Apr 12, 2012 at 11:17:50AM +0300, Timo Sirainen wrote: >> But do you keep your intermediate cert in ssl_ca file or ssl_cert file? > > Separate. Root and intermediate are in ssl_ca: The documentation tells to put the intermediary to ssl_cert though. I didn't even know it worked in ssl_ca. But I guess I won't intentionally break it.. From tlx at leuxner.net Thu Apr 12 11:41:52 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 12 Apr 2012 10:41:52 +0200 Subject: [Dovecot] SSL Certificate Anomalies with latest code changes In-Reply-To: References: <20120412071132.GA2583@nihlus.leuxner.net> <20120412081645.GB2583@nihlus.leuxner.net> <11B240E5-087D-43A2-A687-633AB3C4A367@iki.fi> <20120412083358.GA14349@nihlus.leuxner.net> Message-ID: <20120412084152.GB14349@nihlus.leuxner.net> On Thu, Apr 12, 2012 at 11:35:48AM +0300, Timo Sirainen wrote: > On 12.4.2012, at 11.33, Thomas Leuxner wrote: > > > On Thu, Apr 12, 2012 at 11:17:50AM +0300, Timo Sirainen wrote: > >> But do you keep your intermediate cert in ssl_ca file or ssl_cert file? > > > > Separate. Root and intermediate are in ssl_ca: > > The documentation tells to put the intermediary to ssl_cert though. I didn't even know it worked in ssl_ca. But I guess I won't intentionally break it.. Hmmm. I did emulate Thawte instructions though: https://search.thawte.com/support/ssl-digital-certificates/index?page=content&id=SO15464&actp=LIST&viewlocale=en_US https://search.thawte.com/library/VERISIGN/ALL_OTHER/thawte%20ca/SSL123_CA_Bundle.pem [...] SSLCertificateFile /usr/local/ssl/crt/domainname.crt SSLCertificateKeyFile /usr/local/ssl/private/server.key SSLCACertificateFile /usr/local/ssl/crt/cabundle.crt -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From stan at hardwarefreak.com Thu Apr 12 13:20:31 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 12 Apr 2012 05:20:31 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> Message-ID: <4F86AC6F.4070304@hardwarefreak.com> On 4/11/2012 9:23 PM, Emmanuel Noobadmin wrote: > On 4/12/12, Stan Hoeppner wrote: >> On 4/11/2012 11:50 AM, Ed W wrote: >>> One of the snags of md RAID1 vs RAID6 is the lack of checksumming in the >>> event of bad blocks. (I'm not sure what actually happens when md >>> scrubbing finds a bad sector with raid1..?). For low performance >>> requirements I have become paranoid and been using RAID6 vs RAID10, >>> filesystems with sector checksums seem attractive... >> >> Except we're using hardware RAID1 here and mdraid linear. Thus the >> controller takes care of sector integrity. RAID6 yields nothing over >> RAID10, except lower performance, and more usable space if more than 4 >> drives are used. > > How would the control ensure sector integrity unless it is writing > additional checksum information to disk? I thought only a few > filesystems like ZFS does the sector checksum to detect if any data > corruption occurred. I suppose the controller could throw an error if > the two drives returned data that didn't agree with each other but it > wouldn't know which is the accurate copy but that wouldn't protect the > integrity of the data, at least not directly without additional human > intervention I would think. When a drive starts throwing uncorrectable read errors, the controller faults the drive and tells you to replace it. Good hardware RAID controllers are notorious for their penchant to kick drives that would continue to work just fine in mdraid or as a single drive for many more years. The mindset here is that anyone would rather spent $150-$2500 dollars on a replacement drive than take a chance with his/her valuable data. Yes I typed $2500. EMC charges over $2000 for a single Seagate disk drive with an EMC label and serial# on it. The serial number is what prevents one from taking the same off the shelf Seagate drive at $300 and mounting it in a $250,000 EMC array chassis. The controller firmware reads the S/N from each connected drive and will not allow foreign drives to be used. HP, IBM, Oracle/Sun, etc do this as well. Which is why they make lots of profit, and is why I prefer open storage systems. -- Stan From lists at wildgooses.com Thu Apr 12 13:58:52 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 12 Apr 2012 11:58:52 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F86AC6F.4070304@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> Message-ID: <4F86B56C.6010201@wildgooses.com> On 12/04/2012 11:20, Stan Hoeppner wrote: > On 4/11/2012 9:23 PM, Emmanuel Noobadmin wrote: >> On 4/12/12, Stan Hoeppner wrote: >>> On 4/11/2012 11:50 AM, Ed W wrote: >>>> One of the snags of md RAID1 vs RAID6 is the lack of checksumming in the >>>> event of bad blocks. (I'm not sure what actually happens when md >>>> scrubbing finds a bad sector with raid1..?). For low performance >>>> requirements I have become paranoid and been using RAID6 vs RAID10, >>>> filesystems with sector checksums seem attractive... >>> Except we're using hardware RAID1 here and mdraid linear. Thus the >>> controller takes care of sector integrity. RAID6 yields nothing over >>> RAID10, except lower performance, and more usable space if more than 4 >>> drives are used. >> How would the control ensure sector integrity unless it is writing >> additional checksum information to disk? I thought only a few >> filesystems like ZFS does the sector checksum to detect if any data >> corruption occurred. I suppose the controller could throw an error if >> the two drives returned data that didn't agree with each other but it >> wouldn't know which is the accurate copy but that wouldn't protect the >> integrity of the data, at least not directly without additional human >> intervention I would think. > When a drive starts throwing uncorrectable read errors, the controller > faults the drive and tells you to replace it. Good hardware RAID > controllers are notorious for their penchant to kick drives that would > continue to work just fine in mdraid or as a single drive for many more > years. The mindset here is that anyone would rather spent $150-$2500 > dollars on a replacement drive than take a chance with his/her valuable > data. > I'm asking a subtlely different question. The claim by ZFS/BTRFS authors and others is that data silently "bit rots" on it's own. The claim is therefore that you can have a raid1 pair where neither drive reports a hardware failure, but each gives you different data? I can't personally claim to have observed this, so it remains someone else's theory... (for background my experience is simply: RAID10 for high performance arrays and RAID6 for all my personal data - I intend to investigate your linear raid idea in the future though) I do agree that if one drive reports a read error, then it's quite easy to guess which pair of the array is wrong... Just as an aside, I don't have a lot of failure experience. However, the few I have had (perhaps 6-8 events now) is that there is a massive correlation in failure time with RAID1, eg one pair I had lasted perhaps 2 years and then both failed within 6 hours of each other. I also had a bad experience with RAID 5 that wasn't being scrubbed regularly and when one drive started reporting errors (ie lack of monitoring meant it had been bad for a while), the rest of the array turned out to be a patchwork of read errors - linux raid then turns out to be quite fragile in the presence of a small number of read failures and it's extremely difficult to salvage the 99% of the array which is ok due to the disks getting kicked out... (of course regular scrubs would have prevented getting so deep into that situation - it was a small cheap nas box without such features) Ed W From tss at iki.fi Thu Apr 12 14:09:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Apr 2012 14:09:31 +0300 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F86B56C.6010201@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> Message-ID: <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> On 12.4.2012, at 13.58, Ed W wrote: > The claim by ZFS/BTRFS authors and others is that data silently "bit rots" on it's own. The claim is therefore that you can have a raid1 pair where neither drive reports a hardware failure, but each gives you different data? That's one reason why I planned on adding a checksum to each message in dbox. But I forgot to actually do that. I guess I could add it for new messages in some upcoming version. Then Dovecot could optionally verify the checksum before returning the message to client, and if it detects corruption perhaps automatically read it from some alternative location (e.g. if dsync replication is enabled ask from another replica). And Dovecot index files really should have had some small (8/16/32bit) checksums of stuff as well.. From lists at wildgooses.com Thu Apr 12 14:45:51 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 12 Apr 2012 12:45:51 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F862D50.8000206@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> Message-ID: <4F86C06F.3060109@wildgooses.com> On 12/04/2012 02:18, Stan Hoeppner wrote: > On 4/11/2012 11:50 AM, Ed W wrote: >> Re XFS. Have you been watching BTRFS recently? >> >> I will concede that despite the authors considering it production ready >> I won't be using it for my servers just yet. However, it's benchmarking >> on single disk benchmarks fairly similarly to XFS and in certain cases >> (multi-threaded performance) can be somewhat better. I haven't yet seen >> any benchmarks on larger disk arrays yet, eg 6+ disks, so no idea how it >> scales up. Basically what I have seen seems "competitive" > Links? http://btrfs.ipv5.de/index.php?title=Main_Page#Benchmarking See the regular Phoronix benchmarks in particular. However, I believe these are all single disk? >> I don't have such hardware spare to benchmark, but I would be interested >> to hear from someone who benchmarks your RAID1+linear+XFS suggestion, >> especially if they have compared a cutting edge btrfs kernel on the same >> array? > http://btrfs.boxacle.net/repository/raid/history/History_Mail_server_simulation._num_threads=128.html > > This is with an 8 wide LVM stripe over 8 17 drive hardware RAID0 arrays. > If the disks had been setup as a concat of 68 RAID1 pairs, XFS would > have turned in numbers significantly higher, anywhere from a 100% > increase to 500%. My instinct is that this is an irrelevant benchmark for BTRFS because its performance characteristics for these workloads have changed so significantly? I would be far more interested in a 3.2 and then a 3.6/3.7 benchmark in a years time In particular recent benchmarks on Phoronix show btrfs exceeding XFS performance on heavily threaded benchmarks - however, I doubt this is representative of performance on a multi-disk benchmark? > It would be nice to see these folks update these > results with a 3.2.6 kernel, as both BTRFS and XFS have improved > significantly since 2.6.35. EXT4 and JFS have seen little performance > work since. My understanding is that there was a significant multi-thread performance boost for EXT4 in the last year kind of timeframe? I don't have a link to hand, but someone did some work to reduce lock contention (??) which I seem to recall made a very large difference on multi-user or multi-cpu workloads? I seem to recall that the summary was that it allowed Ext4 to scale up to a good fraction of XFS performance on "medium sized" systems? (I believe that XFS still continues to scale far better than anything else on large systems) Point is that I think it's a bit unfair to say that little has changed on Ext4? It still seems to be developing faster than "maintenance only" However, well OT... The original question was: anyone tried very recent BTRFS on a multi-disk system. Seems like the answer is no. My proposal is that it may be worth watching in the future Cheers Ed W P.S. I have always been intrigued by the idea that a COW based filesystem could potentially implement much faster "RAID" parity, because it can avoid reading the whole stripe. The idea is that you treat unallocated space as "zero", which means you can compute the incremental parity with only a read/write of the checksum value (and with a COW filesystem you only ever update by rewriting to new "zero'd" space). I had in mind something like a fixed parity disk (RAID4?) and allowing the parity disk to be "write behind" cached in ram (ie exposed to risk of: power fails AND data disk fails at the same time). My code may not be following along for a while though... From zybi at talex.pl Thu Apr 12 14:47:49 2012 From: zybi at talex.pl (=?ISO-8859-2?Q?Artur_Zaprza=B3a?=) Date: Thu, 12 Apr 2012 13:47:49 +0200 Subject: [Dovecot] "doveadm mailbox status" destroys compressed messages without W= attribute Message-ID: <4F86C0E5.6090709@talex.pl> I still have some old messages from previous mail server. This messages have no W= attribute either in file name or dovecot-uidlist and are compressed with gzip. Running "doveadm mailbox status -A vsize \*" will result in the following messages: doveadm(foo at domain): Error: Cached message size larger than expected (2580 > 1451) doveadm(foo at domain): Error: Maildir filename has wrong S value, renamed the file from /vmail/domain/foo/Maildir/.Sent/cur/1206550323.M125837P13306V0000000000000812I0000809E.oldname,S=2580:2,SZ to /vmail/domain/foo/Maildir/.Sent/cur/1206550323.M125837P13306V0000000000000812I0000809E.oldname,S=1451:2,SZ doveadm(foo at domain): Error: Corrupted index cache file /vmail/domain/foo/Maildir/.Sent/dovecot.index.cache: Broken physical size for mail UID 2 doveadm(foo at domain): Error: Cached message size larger than expected (2580 > 1451) doveadm(foo at domain): Error: Corrupted index cache file /vmail/domain/foo/Maildir/.Sent/dovecot.index.cache: Broken physical size for mail UID 2 doveadm(foo at domain): Error: read(/vmail/domain/foo/Maildir/.Sent/cur/1206550323.M125837P13306V0000000000000812I0000809E.oldname,S=2580:2,SZ) failed: Input/output error (uid=2) (Size of uncompressed message is 2580 and compressed size is 1451) I have enabled zlib plugin for imap, pop3, lda and lmtp. But how to enable it for doveadm? -- Best regards, Artur Zaprza?a From tss at iki.fi Thu Apr 12 14:49:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 12 Apr 2012 14:49:44 +0300 Subject: [Dovecot] "doveadm mailbox status" destroys compressed messages without W= attribute In-Reply-To: <4F86C0E5.6090709@talex.pl> References: <4F86C0E5.6090709@talex.pl> Message-ID: <5CA9296A-87CB-4F47-AB1B-67533F866248@iki.fi> On 12.4.2012, at 14.47, Artur Zaprza?a wrote: > I have enabled zlib plugin for imap, pop3, lda and lmtp. But how to enable it for doveadm? Just set it globally: mail_plugins = zlib From lists at wildgooses.com Thu Apr 12 15:10:20 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 12 Apr 2012 13:10:20 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> Message-ID: <4F86C62C.8030305@wildgooses.com> On 12/04/2012 12:09, Timo Sirainen wrote: > On 12.4.2012, at 13.58, Ed W wrote: > >> The claim by ZFS/BTRFS authors and others is that data silently "bit rots" on it's own. The claim is therefore that you can have a raid1 pair where neither drive reports a hardware failure, but each gives you different data? > That's one reason why I planned on adding a checksum to each message in dbox. But I forgot to actually do that. I guess I could add it for new messages in some upcoming version. Then Dovecot could optionally verify the checksum before returning the message to client, and if it detects corruption perhaps automatically read it from some alternative location (e.g. if dsync replication is enabled ask from another replica). And Dovecot index files really should have had some small (8/16/32bit) checksums of stuff as well.. > I have to say - I haven't actually seen this happen... Do any of your big mailstore contacts observe this, eg rackspace, etc? I think it's worth thinking about the failure cases before implementing something to be honest? Just sticking in a checksum possibly doesn't help anyone unless it's on the right stuff and in the right place? Off the top of my head: - Someone butchers the file on disk (disk error or someone edits it with vi) - Restore of some files goes subtly wrong, eg tool tries to be clever and fails, snapshot taken mid-write, etc? - Filesystem crash (sudden power loss), how to deal with partial writes? Things I might like to do *if* there were some suitable "checksums" available: - Use the checksum as some kind of guid either for the whole message, the message minus the headers, or individual mime sections - Use the checksums to assist with replication speed/efficiency (dsync or custom imap commands) - File RFCs for new imap features along the "lemonde" lines which allow clients to have faster recovery from corrupted offline states... - Single instance storage (presumably already done, and of course this has some subtleties in the face of deliberate attack) - Possibly duplicate email suppression (but really this is an LDA problem...) - Storage backends where emails are redundantly stored and might not ALL be on a single server (find me the closest copy of email X) - derivations of this might be interesting for compliance archiving of messages? - Fancy key-value storage backends might use checksums as part of the key value (either for the whole or parts of the message) The mail server has always looked like a kind of key-value store to my eye. However, traditional key-value isn't usually optimised for "streaming reads", hence dovecot seems like a "key value store, optimised for sequential high speed streaming access to the key values"... Whilst it seems increasingly unlikely that a traditional key-value store will work well to replace say mdbox, I wonder if it's not worth looking at the replication strategies of key-value stores to see if those ideas couldn't lead to new features for mdbox? Cheers Ed W From dlie76 at yahoo.com.au Thu Apr 12 16:27:38 2012 From: dlie76 at yahoo.com.au (Daminto Lie) Date: Thu, 12 Apr 2012 06:27:38 -0700 (PDT) Subject: [Dovecot] vacation plugins for squirrelmail Message-ID: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> Hi, I am afraid I have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. Any help would be very much appreciated. Thank you From zybi at talex.pl Thu Apr 12 16:47:41 2012 From: zybi at talex.pl (=?UTF-8?B?QXJ0dXIgWmFwcnphxYJh?=) Date: Thu, 12 Apr 2012 15:47:41 +0200 Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> Message-ID: <4F86DCFD.5020007@talex.pl> Daminto Lie wrote: > Hi, > > I am afraid I have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. > > Any help would be very much appreciated. > > Thank you > I'm using Avelsieve 1.9.9 with a set of my own bugfixes: http://email.uoa.gr/avelsieve/ From dirk.jahnke-zumbusch at desy.de Thu Apr 12 17:08:31 2012 From: dirk.jahnke-zumbusch at desy.de (Dirk Jahnke-Zumbusch) Date: Thu, 12 Apr 2012 16:08:31 +0200 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F86C62C.8030305@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> <4F86C62C.8030305@wildgooses.com> Message-ID: <4F86E1DF.30800@desy.de> Hi there, > I have to say - I haven't actually seen this happen... Do any of your > big mailstore contacts observe this, eg rackspace, etc? Just to throw in to the discussion that with (silent) data corruption not only "the disk" is involved but many other parts of your systems. So perhaps you would like to have a look at https://indico.desy.de/getFile.py/access?contribId=65&sessionId=42&resId=0&materialId=slides&confId=257 http://indico.cern.ch/getFile.py/access?contribId=3&sessionId=0&resId=1&materialId=paper&confId=13797 The documents are from 2007 but the principals are still the same. Kind regards Dirk From helga.mayer at uni-hohenheim.de Thu Apr 12 18:31:35 2012 From: helga.mayer at uni-hohenheim.de (Helga Mayer) Date: Thu, 12 Apr 2012 17:31:35 +0200 Subject: [Dovecot] Problems with Apple Mail and attachments In-Reply-To: References: <20120411161321.165760epurnu7i4h@webmail.uni-hohenheim.de> Message-ID: <20120412173135.10654u9eo7r2pumv@webmail.uni-hohenheim.de> Quoting Thierry de Montaudry : > I've seen a similar problem a while ago (1 year maybe more), but > used the mailbox Rebuild option on the client, which fixed it > without having to delete and recreate the account. Your problem > might just be a local index corruption, which can happen when > loosing your Internet connection. Thank you, we will will try it. Regards Helga Helga Mayer Universit?t Hohenheim Kommunikations-, Informations- und Medienzentrum (630) IT-Dienste | Mail Schloss-Westhof-S?d | 70599 Stuttgart Tel.: +49 711 459-22838 | Fax: +49 711 459-23449 https://kim.uni-hohenheim.de From am at am.cx Thu Apr 12 18:35:15 2012 From: am at am.cx (Andrea Mistrali) Date: Thu, 12 Apr 2012 17:35:15 +0200 Subject: [Dovecot] Problems with master user Message-ID: Hi to all! I?m trying to setup master users, but I have some problems. Namely I can authenticate, but after it I cannot access INBOX or other mailboxes of the user. My configuration is: passdb { driver = ldap args = /etc/dovecot/ldap-passdb.conf } passdb { driver = sql args = /etc/dovecot/sql.conf } passdb { driver = passwd-file args = /etc/dovecot/passwd.masterusers master = yes pass = yes } userdb { driver = sql args = /etc/dovecot/sql.conf } (I look up for auth in LDAP server first, if it fails I look up in DB, else I check for master user) and relevant files are /etc/dovecot/sql.conf ?????????? password_query = SELECT fullusername as user, \ password, \ uid AS userdb_uid, \ gid AS userdb_gid, \ home AS userdb_home, \ mail AS userdb_mail, \ groups as userdb_acl_groups, \ quota_rule as userdb_quota_rule \ FROM pd_users_full WHERE \ username = '%n' AND \ domain = '%d' AND \ external_auth IS FALSE AND \ master_user IS FALSE AND \ %Ls_ok IS TRUE user_query = SELECT fullusername as user, \ uid, \ gid, \ home, \ mail, \ groups as acl_groups, \ quota_rule \ FROM pd_users_full WHERE \ username = '%n' AND \ domain = '%d' AND \ master_user IS FALSE iterate_query = SELECT fullusername as username, fullusername as user \ FROM pd_users_full where master_user IS FALSE ORDER BY domain,username /etc/dovecot/ldap-passdb.conf ?????????????? uris = ldap://dioniso.cube.lan base = cn=users,dc=cube,dc=lan auth_bind = yes auth_bind_userdn = uid=%n,cn=users,dc=cube,dc=lan pass_attrs = uid=username, \ userPassword=password, \ # uidNumber=userdb_uid, \ # =userdb_home=/var/mail/cubeholding.com/%Lu, \ # =userdb_domain=cubeholding.com, \ # =userdb_mail=maildir:~/maildir/:INBOX=~/maildir/INBOX:LAYOUT=fs:INDEX=~/indexes/ pass_filter = (&(objectClass=posixAccount)(uid=%n)(mail=*@%d)) # Attributes and filter to get a list of all users # iterate_attrs = uid=username iterate_attrs = uid=user iterate_filter = (&(objectClass=posixAccount)(mail=*@%d)) If I test with doveadm auth and doveadm user I receive this: # doveadm auth -x service=imap andre at am.cx\*master at am.cx XXXXXX passdb: andre at am.cx*master at am.cx auth succeeded extra fields: user=andre at am.cx # doveadm user andre at am.cx userdb: andre at am.cx uid : 10010 gid : 8 home : /var/mail/am.cx/andre mail : maildir:~/maildir:INBOX=~/maildir/INBOX:LAYOUT=fs:INDEX=~/indexes/ acl_groups: quota_rule: *:storage=10G and in log files I see: 20120412 17:31:26 auth: Info: passdb(master at am.cx,master): Master user logging in as andre at am.cx 20120412 17:31:26 auth: Info: ldap(andre at am.cx): invalid credentials (given password: XXXXXX) but if I try the real thing: # telnet localhost 143 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE ACL QUOTA NAMESPACE STARTTLS AUTH=PLAIN AUTH=LOGIN AUTH=GSSAPI] Dovecot ready. 0 login andre at am.cx*master at am.cx XXXXXX 0 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE ACL QUOTA NAMESPACE COMPRESS=DEFLATE QUOTA ACL RIGHTS=texk] Logged in 0 select INBOX 0 NO [SERVERBUG] Internal error occurred. Refer to server log for more information. [2012-04-12 17:33:15] and in log file I have: 20120412 17:34:25 auth: Info: passdb(master at am.cx,127.0.0.1,master): Master user logging in as andre at am.cx 20120412 17:34:25 auth: Info: ldap(andre at am.cx,127.0.0.1): invalid credentials (given password: silmaril) 20120412 17:34:25 imap-login: Info: Login: pid=1673, , 127.0.0.1/127.0.0.1, PLAIN, secured 20120412 17:34:47 imap(andre at am.cx): Error: Opening INBOX failed: Mailbox doesn't exist: INBOX Can someone tell me what is wrong in my setup? TIA A. From am at am.cx Thu Apr 12 19:11:36 2012 From: am at am.cx (Andrea Mistrali) Date: Thu, 12 Apr 2012 18:11:36 +0200 Subject: [Dovecot] Problems with master user In-Reply-To: <533C3A4E-2E3A-4B95-84DB-98A4903BBD2E@am.cx> References: <533C3A4E-2E3A-4B95-84DB-98A4903BBD2E@am.cx> Message-ID: Il giorno 12/apr/2012, alle ore 17.35, Andrea Mistrali ha scritto: > Hi to all! > I?m trying to setup master users, but I have some problems. Namely I can authenticate, but after it I cannot access INBOX or other mailboxes of the user. > snip > Can someone tell me what is wrong in my setup? Solved! It is a problem of ACL as stated at http://master.wiki2.dovecot.org/Authentication/MasterUsers#ACLs Sorry A. From root.kev at gmail.com Thu Apr 12 23:48:12 2012 From: root.kev at gmail.com (Root Kev) Date: Thu, 12 Apr 2012 16:48:12 -0400 Subject: [Dovecot] POP3 Dovecot Auth CPU usage 75%+ In-Reply-To: References: Message-ID: Hello all, I hope someone can help me, I have been testing out Dovecot to switch from popa3d which I use at the moment. When I get several users connecting and disconnection multiple times, the Dovecot process with command Auth uses 50-90% of the CPU for the period which they are connecting. I am wondering if there is something that I may have misconfigured, or if there is something that I can change so that this spike doesn't occur. If anyone could shed some light on the issue, I would appreciate it, Kevin /var/mail# dovecot -n # 2.1.4: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-33-generic-pae i686 Ubuntu 10.04.4 LTS ext4 auth_cache_size = 10 M auth_verbose = yes disable_plaintext_auth = no instance_name = Mail Popper 1 listen = 172.20.20.222 login_greeting = Mail Popper 1 Ready mail_location = mbox:/var/empty:INBOX=/var/mail/%u:INDEX=MEMORY mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = shadow } protocols = pop3 service pop3-login { service_count = 0 } ssl = no userdb { driver = passwd } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } From root.kev at gmail.com Thu Apr 12 23:46:46 2012 From: root.kev at gmail.com (Root Kev) Date: Thu, 12 Apr 2012 16:46:46 -0400 Subject: [Dovecot] POP3 Dovecot Auth CPU usage 75%+ Message-ID: Hello all, I hope someone can help me, I have been testing out Dovecot to switch from popa3d which I use at the moment. When I get several users connecting and disconnection multiple times, the Dovecot process with command Auth uses 50-90% of the CPU for the period which they are connecting. I am wondering if there is something that I may have misconfigured, or if there is something that I can change so that this spike doesn't occur. If anyone could shed some light on the issue, I would appreciate it, Kevin /var/mail# dovecot -n # 2.1.4: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-33-generic-pae i686 Ubuntu 10.04.4 LTS ext4 auth_cache_size = 10 M auth_verbose = yes disable_plaintext_auth = no instance_name = Mail Popper 1 listen = 172.20.20.222 login_greeting = Mail Popper 1 Ready mail_location = mbox:/var/empty:INBOX=/var/mail/%u:INDEX=MEMORY mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = shadow } protocols = pop3 service pop3-login { service_count = 0 } ssl = no userdb { driver = passwd } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } From michael at orlitzky.com Fri Apr 13 02:33:22 2012 From: michael at orlitzky.com (Michael Orlitzky) Date: Thu, 12 Apr 2012 19:33:22 -0400 Subject: [Dovecot] [OT] Outlook identities Message-ID: <4F876642.1040701@orlitzky.com> Nothing to do with Dovecot, but I figured this is the best place to ask. Do any of the newer versions of Outlook have proper identities support like Thunderbird, mutt, Roundcube, i.e. every other mail client on Earth? We have customers who set up ten different mailboxes for one person because otherwise Outlook won't Do the Right Thing. Is there some way to make it behave like the others? * When sending new mail, you can choose which address to use. * When replying to mail, it sends from the address that the message was sent to by default. * All mail winds up in one inbox. Outlook (2003, 2007) does do this if you set up different mail accounts, but we shouldn't have to do that. From stan at hardwarefreak.com Fri Apr 13 08:29:52 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 13 Apr 2012 00:29:52 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F86B56C.6010201@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> Message-ID: <4F87B9D0.5010300@hardwarefreak.com> On 4/12/2012 5:58 AM, Ed W wrote: > The claim by ZFS/BTRFS authors and others is that data silently "bit > rots" on it's own. The claim is therefore that you can have a raid1 pair > where neither drive reports a hardware failure, but each gives you > different data? You need to read those articles again very carefully. If you don't understand what they mean by "1 in 10^15 bits non-recoverable read error rate" and combined probability, let me know. And this has zero bearing on RAID1. And RAID1 reads don't work the way you describe above. I explained this in some detail recently. > I do agree that if one drive reports a read error, then it's quite easy > to guess which pair of the array is wrong... Been working that way for more than 2 decades Ed. :) Note that "RAID1" has that "1" for a reason. It was the first RAID level. It was in production for many many years before parity RAID hit the market. It is the most well understood of all RAID levels, and the simplest. -- Stan From centos.admin at gmail.com Fri Apr 13 09:12:48 2012 From: centos.admin at gmail.com (Emmanuel Noobadmin) Date: Fri, 13 Apr 2012 14:12:48 +0800 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F86AC6F.4070304@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> Message-ID: On 4/12/12, Stan Hoeppner wrote: > On 4/11/2012 9:23 PM, Emmanuel Noobadmin wrote: > I suppose the controller could throw an error if >> the two drives returned data that didn't agree with each other but it >> wouldn't know which is the accurate copy but that wouldn't protect the >> integrity of the data, at least not directly without additional human >> intervention I would think. > > When a drive starts throwing uncorrectable read errors, the controller > faults the drive and tells you to replace it. Good hardware RAID > controllers are notorious for their penchant to kick drives that would > continue to work just fine in mdraid or as a single drive for many more > years. What I meant wasn't the drive throwing uncorrectable read errors but the drives are returning different data that each think is correct or both may have sent the correct data but one of the set got corrupted on the fly. After reading the articles posted, maybe the correct term would be the controller receiving silently corrupted data, say due to bad cable on one. If the controller simply returns the fastest result, it could be the bad sector and that doesn't protect the integrity of the data right? if the controller gets 1st half from one drive and 2nd half from the other drive to speed up performance, we could still get the corrupted half and the controller itself still can't tell if the sector it got was corrupted isn't it? If the controller compares the two sectors from the drives, it may be able to tell us something is wrong but there isn't anyway for it to know which one of the sector was a good read and which isn't, or is there? From blevi.linux at gmail.com Fri Apr 13 09:22:34 2012 From: blevi.linux at gmail.com (Birta Levente) Date: Fri, 13 Apr 2012 09:22:34 +0300 Subject: [Dovecot] [OT] Outlook identities In-Reply-To: <4F876642.1040701@orlitzky.com> References: <4F876642.1040701@orlitzky.com> Message-ID: <4F87C62A.8060105@gmail.com> On 13/04/2012 02:33, Michael Orlitzky wrote: > Nothing to do with Dovecot, but I figured this is the best place to ask. > > Do any of the newer versions of Outlook have proper identities support > like Thunderbird, mutt, Roundcube, i.e. every other mail client on Earth? > > We have customers who set up ten different mailboxes for one person > because otherwise Outlook won't Do the Right Thing. Is there some way to > make it behave like the others? > > * When sending new mail, you can choose which address to use. > * When replying to mail, it sends from the address that the message > was sent to by default. > * All mail winds up in one inbox. > > Outlook (2003, 2007) does do this if you set up different mail accounts, > but we shouldn't have to do that. Control Panel / E-mail (Mail in win7) / Profiles ... I hope this help you .... Levi From robert at schetterer.org Fri Apr 13 09:35:53 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 13 Apr 2012 08:35:53 +0200 Subject: [Dovecot] [OT] Outlook identities In-Reply-To: <4F876642.1040701@orlitzky.com> References: <4F876642.1040701@orlitzky.com> Message-ID: <4F87C949.8010405@schetterer.org> Am 13.04.2012 01:33, schrieb Michael Orlitzky: > Nothing to do with Dovecot, but I figured this is the best place to ask. > > Do any of the newer versions of Outlook have proper identities support > like Thunderbird, mutt, Roundcube, i.e. every other mail client on Earth? > > We have customers who set up ten different mailboxes for one person > because otherwise Outlook won't Do the Right Thing. Is there some way to > make it behave like the others? > > * When sending new mail, you can choose which address to use. > * When replying to mail, it sends from the address that the message > was sent to by default. > * All mail winds up in one inbox. > > Outlook (2003, 2007) does do this if you set up different mail accounts, > but we shouldn't have to do that. you can do it with exchange, no wonder outlook isnt a internet mail client in first line its the client of exchange, so people should use real internet mail clients ( TB etc ), if m$ would more be compatible, the need of exchange may more less , but this should not be , as cashmakers should live forever *g http://www.msexchange.org/tutorials/Sending-As.html there are a few tricks to goal with profiles etc, but i saw nothing that was equal to thunderbird identities but there a many lists an payed support for outlook, you may asked there i also looked in my newest outlook version and didnt found something related i found this add on http://www.emailaddressmanager.com/outlook/email_accounts.html -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From am at am.cx Fri Apr 13 10:02:42 2012 From: am at am.cx (Andrea Mistrali) Date: Fri, 13 Apr 2012 09:02:42 +0200 Subject: [Dovecot] [OT] Outlook identities In-Reply-To: <4F876642.1040701@orlitzky.com> References: <4F876642.1040701@orlitzky.com> Message-ID: Il giorno 13/apr/2012, alle ore 01.33, Michael Orlitzky ha scritto: > Nothing to do with Dovecot, but I figured this is the best place to ask. > > Do any of the newer versions of Outlook have proper identities support > like Thunderbird, mutt, Roundcube, i.e. every other mail client on Earth? > > We have customers who set up ten different mailboxes for one person > because otherwise Outlook won't Do the Right Thing. Is there some way to > make it behave like the others? > > * When sending new mail, you can choose which address to use. > * When replying to mail, it sends from the address that the message > was sent to by default. > * All mail winds up in one inbox. > > Outlook (2003, 2007) does do this if you set up different mail accounts, > but we shouldn't have to do that. If it can comfort you even Apple Mail.app ha no support for full identities like Thunderbird and every other good mail client (I think of Eudora, back to 1999, that had full identities support) have, except for some little trick that does not do ?The Right Thing". We ended writing a little PyObjc application that adds support for full identities to Mail.app, but it is just a hack and dunno if it will work forever :/ A. From tss at iki.fi Fri Apr 13 14:51:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 13 Apr 2012 14:51:06 +0300 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F86C62C.8030305@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> <4F86C62C.8030305@wildgooses.com> Message-ID: <918556F5-6EE9-4651-A481-46720480FE46@iki.fi> On 12.4.2012, at 15.10, Ed W wrote: > On 12/04/2012 12:09, Timo Sirainen wrote: >> On 12.4.2012, at 13.58, Ed W wrote: >> >>> The claim by ZFS/BTRFS authors and others is that data silently "bit rots" on it's own. The claim is therefore that you can have a raid1 pair where neither drive reports a hardware failure, but each gives you different data? >> That's one reason why I planned on adding a checksum to each message in dbox. But I forgot to actually do that. I guess I could add it for new messages in some upcoming version. Then Dovecot could optionally verify the checksum before returning the message to client, and if it detects corruption perhaps automatically read it from some alternative location (e.g. if dsync replication is enabled ask from another replica). And Dovecot index files really should have had some small (8/16/32bit) checksums of stuff as well.. >> > > I have to say - I haven't actually seen this happen... Do any of your big mailstore contacts observe this, eg rackspace, etc? I haven't heard. But then again people don't necessarily notice if it has. > Things I might like to do *if* there were some suitable "checksums" available: > - Use the checksum as some kind of guid either for the whole message, the message minus the headers, or individual mime sections Messages already have a GUID. And the rest of that is kind of done with the single instance storage stuff.. I was thinking of using SHA1 of the entire message with headers as the checksum, and save it into dbox metadata field. I also thought about checksumming the metadata fields as well, but that would need another checksum as the first one can have other uses as well besides verifying the message integrity. > - Use the checksums to assist with replication speed/efficiency (dsync or custom imap commands) It would be of some use with dbox index rebuilding. I don't think it would help with dsync. > - File RFCs for new imap features along the "lemonde" lines which allow clients to have faster recovery from corrupted offline states... Too much trouble, no one would implement it :) > - Storage backends where emails are redundantly stored and might not ALL be on a single server (find me the closest copy of email X) - derivations of this might be interesting for compliance archiving of messages? > - Fancy key-value storage backends might use checksums as part of the key value (either for the whole or parts of the message) GUID would work for these as well, without the possibility of a hash collision. From tss at iki.fi Fri Apr 13 14:55:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 13 Apr 2012 14:55:07 +0300 Subject: [Dovecot] POP3 Dovecot Auth CPU usage 75%+ In-Reply-To: References: Message-ID: <0B6D0FE2-2831-460E-8B2B-C255C49DEBAE@iki.fi> On 12.4.2012, at 23.46, Root Kev wrote: > I hope someone can help me, I have been testing out Dovecot to switch from > popa3d which I use at the moment. When I get several users connecting and > disconnection multiple times, the Dovecot process with command Auth uses > 50-90% of the CPU for the period which they are connecting. I am wondering > if there is something that I may have misconfigured, or if there is > something that I can change so that this spike doesn't occur. So is it the "auth" process or "auth worker" process? What if you add: > passdb { > driver = shadow > } > userdb { > driver = passwd args = blocking=yes > } does that move the CPU usage from "auth" to "auth worker" process? Is it using /etc/shadow and /etc/passwd files? Are they large? Do you have enabled other weird stuff in /etc/nsswitch.conf (and were there some other files related to them as well?) From lists at wildgooses.com Fri Apr 13 15:17:19 2012 From: lists at wildgooses.com (Ed W) Date: Fri, 13 Apr 2012 13:17:19 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <918556F5-6EE9-4651-A481-46720480FE46@iki.fi> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> <4F86C62C.8030305@wildgooses.com> <918556F5-6EE9-4651-A481-46720480FE46@iki.fi> Message-ID: <4F88194F.4030807@wildgooses.com> On 13/04/2012 12:51, Timo Sirainen wrote: >> - Use the checksums to assist with replication speed/efficiency (dsync or custom imap commands) > It would be of some use with dbox index rebuilding. I don't think it would help with dsync. .. >> - File RFCs for new imap features along the "lemonde" lines which allow clients to have faster recovery from corrupted offline states... > Too much trouble, no one would implement it :) I presume you have seen that cyrus is working on various distributed options? Standardising this through imap might work if they also buy into it? >> - Storage backends where emails are redundantly stored and might not ALL be on a single server (find me the closest copy of email X) - derivations of this might be interesting for compliance archiving of messages? >> - Fancy key-value storage backends might use checksums as part of the key value (either for the whole or parts of the message) > GUID would work for these as well, without the possibility of a hash collision. I was thinking that the win for key-value store as a backend is if you can reduce the storage requirements or do better placement of the data (mail text replicated widely, attachments stored on higher latency storage?). Hence whilst I don't see this being a win with current options, if it were done then it would almost certainly be "per mime part", eg storing all large attachments in one place and the rest of the message somewhere else, perhaps with different redundancy levels per type OK, this is all completely pie in the sky. Please don't build it! All I meant was that these are the kind of things that someone might one day desire to do and hence they would have competing requirements for what to checksum... Cheers Ed W From tss at iki.fi Fri Apr 13 15:21:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 13 Apr 2012 15:21:49 +0300 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F88194F.4030807@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> <4F86C62C.8030305@wildgooses.com> <918556F5-6EE9-4651-A481-46720480FE46@iki.fi> <4F88194F.4030807@wildgooses.com> Message-ID: <6A4E7A1D-3B55-48DB-A4B0-EF5C01FA829A@iki.fi> On 13.4.2012, at 15.17, Ed W wrote: > On 13/04/2012 12:51, Timo Sirainen wrote: >>> - Use the checksums to assist with replication speed/efficiency (dsync or custom imap commands) >> It would be of some use with dbox index rebuilding. I don't think it would help with dsync. > .. >>> - File RFCs for new imap features along the "lemonde" lines which allow clients to have faster recovery from corrupted offline states... >> Too much trouble, no one would implement it :) > > I presume you have seen that cyrus is working on various distributed options? Standardising this through imap might work if they also buy into it? Probably more trouble than worth. I doubt anyone would want to run a cross-Dovecot/Cyrus cluster. >>> - Storage backends where emails are redundantly stored and might not ALL be on a single server (find me the closest copy of email X) - derivations of this might be interesting for compliance archiving of messages? >>> - Fancy key-value storage backends might use checksums as part of the key value (either for the whole or parts of the message) >> GUID would work for these as well, without the possibility of a hash collision. > > I was thinking that the win for key-value store as a backend is if you can reduce the storage requirements or do better placement of the data (mail text replicated widely, attachments stored on higher latency storage?). Hence whilst I don't see this being a win with current options, if it were done then it would almost certainly be "per mime part", eg storing all large attachments in one place and the rest of the message somewhere else, perhaps with different redundancy levels per type > > OK, this is all completely pie in the sky. Please don't build it! All I meant was that these are the kind of things that someone might one day desire to do and hence they would have competing requirements for what to checksum... That can almost be done already .. the attachments are saved and accessed via a lib-fs API. It wouldn't be difficult to write a backend for some key-value databases. So with about one day's coding you could already have Dovecot save all message attachments to a key-value db, and you can configure redundancy in the db's configs. From stan at hardwarefreak.com Fri Apr 13 15:33:19 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 13 Apr 2012 07:33:19 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> Message-ID: <4F881D0F.2090802@hardwarefreak.com> On 4/13/2012 1:12 AM, Emmanuel Noobadmin wrote: > On 4/12/12, Stan Hoeppner wrote: >> On 4/11/2012 9:23 PM, Emmanuel Noobadmin wrote: >> I suppose the controller could throw an error if >>> the two drives returned data that didn't agree with each other but it >>> wouldn't know which is the accurate copy but that wouldn't protect the >>> integrity of the data, at least not directly without additional human >>> intervention I would think. >> >> When a drive starts throwing uncorrectable read errors, the controller >> faults the drive and tells you to replace it. Good hardware RAID >> controllers are notorious for their penchant to kick drives that would >> continue to work just fine in mdraid or as a single drive for many more >> years. > > What I meant wasn't the drive throwing uncorrectable read errors but > the drives are returning different data that each think is correct or > both may have sent the correct data but one of the set got corrupted > on the fly. After reading the articles posted, maybe the correct term > would be the controller receiving silently corrupted data, say due to > bad cable on one. This simply can't happen. What articles are you referring to? If the author is stating what you say above, he simply doesn't know what he's talking about. > If the controller simply returns the fastest result, it could be the > bad sector and that doesn't protect the integrity of the data right? I already answered this in a previous post. > if the controller gets 1st half from one drive and 2nd half from the > other drive to speed up performance, we could still get the corrupted > half and the controller itself still can't tell if the sector it got > was corrupted isn't it? No, this is not correct. > If the controller compares the two sectors from the drives, it may be > able to tell us something is wrong but there isn't anyway for it to > know which one of the sector was a good read and which isn't, or is > there? Yes it can, and it does. Emmanuel, Ed, we're at a point where I simply don't have the time nor inclination to continue answering these basic questions about the base level functions of storage hardware. You both have serious misconceptions about how many things work. To answer the questions you're asking will require me to teach you the basics of hardware signaling protocols, SCSI, SATA, Fiber Channel, and Ethernet transmission error detection protocols, disk drive firmware error recovery routines, etc, etc, etc. I don't mind, and actually enjoy, passing knowledge. But the amount that seems to be required here to bring you up to speed is about 2^15 times above and beyond the scope of mailing list conversation. In closing, I'll simply say this: If hardware, whether a mobo-down SATA chip, or a $100K SGI SAN RAID controller, allowed silent data corruption or transmission to occur, there would be no storage industry, and we'll all still be using pen and paper. The questions you're asking were solved by hardware and software engineers decades ago. You're fretting and asking about things that were solved decades ago. -- Stan From jtl+dovecot at uvm.edu Fri Apr 13 16:12:02 2012 From: jtl+dovecot at uvm.edu (Jim Lawson) Date: Fri, 13 Apr 2012 09:12:02 -0400 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F881D0F.2090802@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> Message-ID: <4F882622.8040701@uvm.edu> On 04/13/2012 08:33 AM, Stan Hoeppner wrote: >> What I meant wasn't the drive throwing uncorrectable read errors but >> the drives are returning different data that each think is correct or >> both may have sent the correct data but one of the set got corrupted >> on the fly. After reading the articles posted, maybe the correct term >> would be the controller receiving silently corrupted data, say due to >> bad cable on one. > This simply can't happen. What articles are you referring to? If the > author is stating what you say above, he simply doesn't know what he's > talking about. ?! Stan, are you really saying that silent data corruption "simply can't happen"? People who have been studying this have been talking about it for years now. It can happen in the same way that Emmanuel describes. USENIX FAST08: http://static.usenix.org/event/fast08/tech/bairavasundaram.html CERN: http://storagemojo.com/2007/09/19/cerns-data-corruption-research/ http://fuji.web.cern.ch/fuji/talk/2007/kelemen-2007-C5-Silent_Corruptions.pdf LANL: http://institute.lanl.gov/resilience/conferences/2009/HPCResilience09_Michalak.pdf There are others if you search for it. This problem has been well-known in large (petabyte+) data storage systems for some time. Jim From lists at wildgooses.com Fri Apr 13 17:04:17 2012 From: lists at wildgooses.com (Ed W) Date: Fri, 13 Apr 2012 15:04:17 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <6A4E7A1D-3B55-48DB-A4B0-EF5C01FA829A@iki.fi> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <1BD04075-A624-46EA-B5F4-07544DB5CBA8@iki.fi> <4F86C62C.8030305@wildgooses.com> <918556F5-6EE9-4651-A481-46720480FE46@iki.fi> <4F88194F.4030807@wildgooses.com> <6A4E7A1D-3B55-48DB-A4B0-EF5C01FA829A@iki.fi> Message-ID: <4F883261.2040309@wildgooses.com> On 13/04/2012 13:21, Timo Sirainen wrote: > On 13.4.2012, at 15.17, Ed W wrote: > >> On 13/04/2012 12:51, Timo Sirainen wrote: >>>> - Use the checksums to assist with replication speed/efficiency (dsync or custom imap commands) >>> It would be of some use with dbox index rebuilding. I don't think it would help with dsync. >> .. >>>> - File RFCs for new imap features along the "lemonde" lines which allow clients to have faster recovery from corrupted offline states... >>> Too much trouble, no one would implement it :) >> I presume you have seen that cyrus is working on various distributed options? Standardising this through imap might work if they also buy into it? > Probably more trouble than worth. I doubt anyone would want to run a cross-Dovecot/Cyrus cluster. No definitely not. Sorry I just meant that you are both working on similar things. Standardising the basics that each use might be useful in the future > That can almost be done already .. the attachments are saved and accessed via a lib-fs API. It wouldn't be difficult to write a backend for some key-value databases. So with about one day's coding you could already have Dovecot save all message attachments to a key-value db, and you can configure redundancy in the db's configs. Hmm, super. Ed W From stan at hardwarefreak.com Fri Apr 13 17:20:29 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 13 Apr 2012 09:20:29 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F882622.8040701@uvm.edu> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F882622.8040701@uvm.edu> Message-ID: <4F88362D.9080400@hardwarefreak.com> On 4/13/2012 8:12 AM, Jim Lawson wrote: > On 04/13/2012 08:33 AM, Stan Hoeppner wrote: >>> What I meant wasn't the drive throwing uncorrectable read errors but >>> the drives are returning different data that each think is correct or >>> both may have sent the correct data but one of the set got corrupted >>> on the fly. After reading the articles posted, maybe the correct term >>> would be the controller receiving silently corrupted data, say due to >>> bad cable on one. >> This simply can't happen. What articles are you referring to? If the >> author is stating what you say above, he simply doesn't know what he's >> talking about. > > > ?! Stan, are you really saying that silent data corruption "simply > can't happen"? Yes, I did. Did you read the context in which I made that statement? > People who have been studying this have been talking > about it for years now. Yes, they have. Did you miss the paragraph where I stated exactly that? Did you also miss the part about the probably of such being dictated by total storage system size and access rate? > It can happen in the same way that Emmanuel > describes. No, it can't. Not in the way Emmanuel described. I already stated the reason, and all of this research backs my statement. You won't see this with a 2 drive mirror, or a 20 drive RAID10. Not until each drive has a capacity in the 15TB+ range, if not more, and again, depending on the total system size. This doesn't address the "RAID5", better known as "parity RAID" write hole, which is a separate issue. Which is also one of the reasons I don't use it. In lieu of an actual controller firmware bug, or mdraid or lvm bug, you'll never see this on small scale systems. > USENIX FAST08: > > http://static.usenix.org/event/fast08/tech/bairavasundaram.html > > CERN: > > http://storagemojo.com/2007/09/19/cerns-data-corruption-research/ > > http://fuji.web.cern.ch/fuji/talk/2007/kelemen-2007-C5-Silent_Corruptions.pdf > > LANL: > > http://institute.lanl.gov/resilience/conferences/2009/HPCResilience09_Michalak.pdf > > There are others if you search for it. This problem has been well-known > in large (petabyte+) data storage systems for some time. And again, this is the crux of it. One doesn't see this problem until one hits extreme scale, which I spent at least a paragraph or two explaining, referencing the same research. Please re-read my post at least twice, critically. Then tell me if I've stated anything substantively different than what any of these researches have. The statements "shouldn't" "wouldn't" and "can't" are based on probabilities. "Can't" or "won't" does not need equal probability 0. The probability of this type of silent data corruption occurring on a 2 disk or 20 disk array of today's drives is not zero over 10 years, but it is so low the effective statement is "can't" or "won't" see this corruption. As I said, when we reach 15-30TB+ disk drives, this may change for small count arrays. -- Stan From root.kev at gmail.com Fri Apr 13 17:32:34 2012 From: root.kev at gmail.com (Root Kev) Date: Fri, 13 Apr 2012 10:32:34 -0400 Subject: [Dovecot] POP3 Dovecot Auth CPU usage 75%+ In-Reply-To: <0B6D0FE2-2831-460E-8B2B-C255C49DEBAE@iki.fi> References: <0B6D0FE2-2831-460E-8B2B-C255C49DEBAE@iki.fi> Message-ID: I tried making the changes that you suggested but it didn't seem to make a noticeable difference. It should be using the shadow file directly. The shadow file has the default Ubuntu system accounts and 16 user accounts, so overall fairly small. The nsswitch.conf file is set as default: # /etc/nsswitch.conf # # Example configuration of GNU Name Service Switch functionality. # If you have the `glibc-doc-reference' and `info' packages installed, try: # `info libc "Name Service Switch"' for information about this file. passwd: compat group: compat shadow: compat hosts: files dns networks: files protocols: db files services: db files ethers: db files rpc: db files netgroup: nis An example of users connecting and the Auth process using alot of CPU (from top): Cpu(s): 87.4%us, 8.0%sy, 0.0%ni, 2.3%id, 0.0%wa, 0.7%hi, 1.7%si, 0.0%st Mem: 1026096k total, 533924k used, 492172k free, 60340k buffers Swap: 1757176k total, 0k used, 1757176k free, 414212k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 643 dovecot 20 0 3096 1616 1208 S 50.7 0.2 0:01.76 auth 644 root 20 0 3096 1524 1140 S 1.3 0.1 0:00.08 auth 642 dovenull 20 0 4276 1612 1256 S 1.0 0.2 0:00.03 pop3-login 623 root 20 0 2704 1020 772 S 0.7 0.1 0:00.02 dovecot 627 root 20 0 4344 2808 1056 S 0.7 0.3 0:00.03 config 631 syslog 20 0 33916 1924 1036 S 0.3 0.2 0:01.61 rsyslogd 696 serverma 20 0 5464 2564 2040 R 0.3 0.2 0:00.01 pop3 1 root 20 0 2652 1604 1216 S 0.0 0.2 0:01.59 init 2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd Thanks for any other ideas.... Kevin On Fri, Apr 13, 2012 at 7:55 AM, Timo Sirainen wrote: > On 12.4.2012, at 23.46, Root Kev wrote: > > So is it the "auth" process or "auth worker" process? What if you add: > > > passdb { > > driver = shadow > > } > > userdb { > > driver = passwd > args = blocking=yes > > } > > does that move the CPU usage from "auth" to "auth worker" process? Is it > using /etc/shadow and /etc/passwd files? Are they large? Do you have > enabled other weird stuff in /etc/nsswitch.conf (and were there some other > files related to them as well?) > > From tss at iki.fi Fri Apr 13 17:37:43 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 13 Apr 2012 17:37:43 +0300 Subject: [Dovecot] POP3 Dovecot Auth CPU usage 75%+ In-Reply-To: References: <0B6D0FE2-2831-460E-8B2B-C255C49DEBAE@iki.fi> Message-ID: <337BF2E0-9F19-4AB4-A81C-2DCE76484662@iki.fi> On 13.4.2012, at 17.32, Root Kev wrote: > 643 dovecot 20 0 3096 1616 1208 S 50.7 0.2 0:01.76 > auth Well, I can't think of any good reason for this. Try strace -p 643 (or whatever pid of the process eating CPU) for a few seconds and send its output? Also maybe get a couple of gdb backtraces to see where it's doing stuff: gdb -p 643 bt quit and repeat it a few times From f.bonnet at esiee.fr Fri Apr 13 18:01:59 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Fri, 13 Apr 2012 17:01:59 +0200 Subject: [Dovecot] French characters from 2.0.x to 2.1.3 Message-ID: <4F883FE7.9080401@esiee.fr> Hello Maybe it's a coincidence but it "seems" since I upgrade from 2.0.18 to 2.1.3 that some French characters ( a with accents by example ) are not well displayed anymore using thunderbird and are replaced by a "?" character Thanks for any info From lists at wildgooses.com Fri Apr 13 18:09:31 2012 From: lists at wildgooses.com (Ed W) Date: Fri, 13 Apr 2012 16:09:31 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F87B9D0.5010300@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F86B56C.6010201@wildgooses.com> <4F87B9D0.5010300@hardwarefreak.com> Message-ID: <4F8841AB.6010108@wildgooses.com> On 13/04/2012 06:29, Stan Hoeppner wrote: > On 4/12/2012 5:58 AM, Ed W wrote: > >> The claim by ZFS/BTRFS authors and others is that data silently "bit >> rots" on it's own. The claim is therefore that you can have a raid1 pair >> where neither drive reports a hardware failure, but each gives you >> different data? > You need to read those articles again very carefully. If you don't > understand what they mean by "1 in 10^15 bits non-recoverable read error > rate" and combined probability, let me know. OK, I'll bite. I only have an honours degree in mathematics from a well known university, so grateful if you could dumb it down appropriately? Lets start with what "those articles" are you referring to? I don't see any articles if I go literally up the chain from this email, but you might be talking about any one of the lots of other emails in this thread or even some other email thread? Wikipedia has it's faults, but it dumbs the "silent corruption" claim down to: http://en.wikipedia.org/wiki/ZFS "an undetected error for every 67TB" And a CERN study apparently claims "far higher than one in every 10^16 bits" Now, I'm NOT professing any experience of axe to grind here. I'm simply asking by what feature do you believe either software or hardware RAID1 is capable of detecting which pair is correct when both pairs of a raid one disk return different results and there is no hardware failure to clue us that one pair suffered a read error? Please don't respond with a maths pissing competition, it's an innocent question about what levels of data checking are done on each piece of the hardware chain? My (probably flawed) understanding is that popular RAID 1 implementations don't add any additional sector checksums over and above what the drives/filesystem/etc add already offer - is this the case? > And this has zero bearing on RAID1. And RAID1 reads don't work the way > you describe above. I explained this in some detail recently. Where? > Been working that way for more than 2 decades Ed. :) Note that "RAID1" > has that "1" for a reason. It was the first RAID level. What should I make of RAID0 then? Incidentally do you disagree with the history of RAID evolution on Wikipedia? http://en.wikipedia.org/wiki/RAID Regards Ed W From lists at wildgooses.com Fri Apr 13 18:31:35 2012 From: lists at wildgooses.com (Ed W) Date: Fri, 13 Apr 2012 16:31:35 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F881D0F.2090802@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> Message-ID: <4F8846D7.7020900@wildgooses.com> On 13/04/2012 13:33, Stan Hoeppner wrote: >> What I meant wasn't the drive throwing uncorrectable read errors but >> the drives are returning different data that each think is correct or >> both may have sent the correct data but one of the set got corrupted >> on the fly. After reading the articles posted, maybe the correct term >> would be the controller receiving silently corrupted data, say due to >> bad cable on one. > This simply can't happen. What articles are you referring to? If the > author is stating what you say above, he simply doesn't know what he's > talking about. It quite clearly can??! Just grab your drive, lever the connector off a little bit until it's a bit flaky and off you go? *THIS* type of problem I have heard of and you can find easy examples with a quick google search of any hobbyist storage board. Very common other examples are such problems due to failing PSUs and other interference driven examples causing explicit disk errors (and once the error rate goes up, some will make it past the checksum) Note this is NOT what I was originally asking about. My interest is more about when the hardware is working reliably and as you agree, the error levels are vastly lower. However, it would be incredibly foolish to claim that it's not trivial to construct a scenario where bad hardware causes plenty of silent corruption? >> If the controller simply returns the fastest result, it could be the >> bad sector and that doesn't protect the integrity of the data right? > I already answered this in a previous post. Not obviously?! I will also add my understanding that linux software RAID1,5&6 *DO NOT* read all disks and hence will not be aware when disks have different data. In fact with software raid you need to run a regular "scrub" job to check this consistency. I also believe that most commodity hardware raid implementations work exactly the same way and a background scrub is needed to detect inconsistent arrays. However, feel free to correct that understanding? >> if the controller gets 1st half from one drive and 2nd half from the >> other drive to speed up performance, we could still get the corrupted >> half and the controller itself still can't tell if the sector it got >> was corrupted isn't it? > No, this is not correct. I definitely think you are wrong and Emmanuel is right? If the controller gets a good read from the disk then it will trust that read and will NOT check the result with the other disk (or parity in the case of RAID5/6). If that read was incorrect for some reason then the data will be passed as good. >> If the controller compares the two sectors from the drives, it may be >> able to tell us something is wrong but there isn't anyway for it to >> know which one of the sector was a good read and which isn't, or is >> there? > Yes it can, and it does. No it definitely does not!! At least not with linux software raid and I don't believe on commodity hardware controllers either! (You would be able to tell because the disk IO would be doubled) Linux software raid 1 isn't that smart, but reads only one disk and trusts the answer if the read did not trigger an error. It does not check the other disk except during an explicit disk scrub. > Emmanuel, Ed, we're at a point where I simply don't have the time nor > inclination to continue answering these basic questions about the base > level functions of storage hardware. You mean those "answers" like: "I answered that in another thread" or "you need to read 'those' articles again" Referring to some unknown and hard to find previous emails is not the same as answering? Also you are wondering off at extreme tangents. The question is simple: - Disk 1 Read good, checksum = A - Disk 2 Read good, checksum = B Disks are a raid 1 pair. How do we know which disk is correct. Please specify raid 1 implementation and mechanism used with any answer > To answer the questions > you're asking will require me to teach you the basics of hardware > signaling protocols, SCSI, SATA, Fiber Channel, and Ethernet > transmission error detection protocols, disk drive firmware error > recovery routines, etc, etc, etc. I really think not... A simple statement of: - Each sector on disk has a certain sized checksum - Controller checks checksum on read - Sent back over SATA connection, with a certain sized checksum - After that you are on your own vs corruption ...Should cover it I think? > In closing, I'll simply say this: If hardware, whether a mobo-down SATA > chip, or a $100K SGI SAN RAID controller, allowed silent data corruption > or transmission to occur, there would be no storage industry, and we'll > all still be using pen and paper. The questions you're asking were > solved by hardware and software engineers decades ago. You're fretting > and asking about things that were solved decades ago. So why are so many people getting excited about it now? Note, there have been plenty of shoddy disk controller implementations before today - ie there exists hardware on sale with *known* defects. Despite that the industry continues without collapse. Now you claim that if corruption is silent and people only tend to notice it much later and under certain edge conditions that this can't be possible because it should cause the industry to collapse..??? ...Not buying your logic... Ed W From michael at orlitzky.com Fri Apr 13 19:08:31 2012 From: michael at orlitzky.com (Michael Orlitzky) Date: Fri, 13 Apr 2012 12:08:31 -0400 Subject: [Dovecot] [OT] Outlook identities In-Reply-To: <4F87C62A.8060105@gmail.com> References: <4F876642.1040701@orlitzky.com> <4F87C62A.8060105@gmail.com> Message-ID: <4F884F7F.5070002@orlitzky.com> On 04/13/12 02:22, Birta Levente wrote: >> >> Outlook (2003, 2007) does do this if you set up different mail accounts, >> but we shouldn't have to do that. > > Control Panel / E-mail (Mail in win7) / Profiles ... > > I hope this help you .... This is more work than setting up multiple accounts =) From michael at orlitzky.com Fri Apr 13 19:13:30 2012 From: michael at orlitzky.com (Michael Orlitzky) Date: Fri, 13 Apr 2012 12:13:30 -0400 Subject: [Dovecot] [OT] Outlook identities In-Reply-To: <4F87C949.8010405@schetterer.org> References: <4F876642.1040701@orlitzky.com> <4F87C949.8010405@schetterer.org> Message-ID: <4F8850AA.80903@orlitzky.com> On 04/13/12 02:35, Robert Schetterer wrote: > > you can do it with exchange, > no wonder outlook isnt a internet mail client in first line > its the client of exchange, so people should use real internet mail > clients ( TB etc ), if m$ would more be compatible, the need of exchange > may more less , but this should not be , as cashmakers should live > forever *g > > http://www.msexchange.org/tutorials/Sending-As.html > > there are a few tricks to goal with profiles etc, but i saw nothing > that was equal to thunderbird identities > Exchange... the cure is worse than the disease! This isn't looking good -- I guess I'll continue to do what I have been: telling people to switch off of Outlook if they want their mail client to not suck. From robert at schetterer.org Fri Apr 13 20:51:59 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 13 Apr 2012 19:51:59 +0200 Subject: [Dovecot] sieve pipe gpg archive mails Message-ID: <4F8867BF.7030207@schetterer.org> hi @ll would it be possible to use http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Pipe to gpg sign archived mail, incoming via lmtp the intention is to store all archived mail gpg signed to notice some changes the mails at possible restore case has anyone done it bevor , or something equal ? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Fri Apr 13 21:33:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 13 Apr 2012 21:33:05 +0300 Subject: [Dovecot] sieve pipe gpg archive mails In-Reply-To: <4F8867BF.7030207@schetterer.org> References: <4F8867BF.7030207@schetterer.org> Message-ID: On 13.4.2012, at 20.51, Robert Schetterer wrote: > would it be possible to use > > http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Pipe > > to gpg sign archived mail, incoming via lmtp > > the intention is to store all archived mail gpg signed > to notice some changes the mails at possible restore case > > has anyone done it bevor , or something equal ? What about mails saved via IMAP APPEND? From mcbdovecot at robuust.nl Fri Apr 13 22:10:04 2012 From: mcbdovecot at robuust.nl (Maarten Bezemer) Date: Fri, 13 Apr 2012 21:10:04 +0200 (CEST) Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F8846D7.7020900@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F8846D7.7020900@wildgooses.com> Message-ID: On Fri, 13 Apr 2012, Ed W wrote: > On 13/04/2012 13:33, Stan Hoeppner wrote: >>> What I meant wasn't the drive throwing uncorrectable read errors but >>> the drives are returning different data that each think is correct or >>> both may have sent the correct data but one of the set got corrupted >>> on the fly. After reading the articles posted, maybe the correct term >>> would be the controller receiving silently corrupted data, say due to >>> bad cable on one. >> This simply can't happen. What articles are you referring to? If the >> author is stating what you say above, he simply doesn't know what he's >> talking about. > It quite clearly can??! I totally agree with Ed here. Drives sure can and sometimes really do return different data, without reporting errors. Also, data can get corrupted on any of the busses or chips it passes through. The math about 10^15 or 10^16 and all that stuff is not only about array sizes. It's also about data transfer. I've seen silent corruption on a few systems myself. (Luckily, only 3 times in a couple years.) Those systems were only in the 2TB-5TB size category, which is substantially lower than the 67TB claimed elsewhere. Yet, statistically, it's well within normal probability levels. Linux mdraid only reads one mirror as long as the drives don't return an error. Easy to check, the read speeds are way beyond a single drive's read speed. When the kernel would have to read all (possibly more than two) mirrors, and compare them, and make a decision based on this comparison, things would be horribly slow. Hardware raid typically uses this exact same approach. This goes for Areca, 3ware, LSI, which cover most of the regular (i.e. non-SAN) professional hardware raid setups. If you don't believe it, just don't take my word for it but test it for yourself. Cleanly power down a raid1 array, take the individual drives, put them into a simple desktop machine, and write different data to both, using some raw disk writing tool like dd. Then put the drives back into the raid1 array, power it up, and re-read the information. You'll see data from both drives will be intermixed as parts of the reads come from one disk, and parts come from the other. Only when you order the raid array to do a verification pass, it'll start screaming and yelling. At least, I hope it will... But as explained elsewhere, silent corruption can occur at numerous places. If you don't have an explicit checksumming/checking mechanism, there are indeed cases that will haunt you if you don't do regular scrubbing or at least do regular verification runs. Heck, that's why Linux mdadm comes with cron jobs to do just that, and hardware raid controllers have similar scheduling capabilities. Of course, scrubbing/verification is not going to magically protect you from all problems. But you would at least get notifications if it detects problems. >>> If the controller compares the two sectors from the drives, it may be >>> able to tell us something is wrong but there isn't anyway for it to >>> know which one of the sector was a good read and which isn't, or is >>> there? >> Yes it can, and it does. > > No it definitely does not!! At least not with linux software raid and I don't > believe on commodity hardware controllers either! (You would be able to tell > because the disk IO would be doubled) Obviously there is no way to tell which versions of a story are correct if you are not biased to believe one of the storytellers and distrust the other. You would have to add a checksum layer for that. (And hope the checksum isn't the part that got corrupted!) >> To answer the questions >> you're asking will require me to teach you the basics of hardware >> signaling protocols, SCSI, SATA, Fiber Channel, and Ethernet >> transmission error detection protocols, disk drive firmware error >> recovery routines, etc, etc, etc. I'm quite familiar with the basics of these protocols. I'm also quite familiar with the flaws in several implementations of "seemingly straightforward protocols". More often than not, there's a pressing need to get new devices onto the market before the competition has something similar and you loose your advantage. More often than not, this results in suboptimal implementations of all those fine protocols and algorithms. And let's face it: flaws in error recovery routines often don't surface until someone actually needs those routines. As long as drives (or any other device) are functioning as expected, everything is all right. But as soon as something starts to get flaky, error recovery has to kick in but may just as well fail to do the right thing. Just consider the real-world analogy of politicians. They do or say something stupid every once in a while, and error recovery (a.k.a. damage control) has to kick in. But even though those well trained professionals, having decades of experience in the political arena, sometimes simply fail to do the right thing. They may have overlooked some pesky details, or they may take actions that don't have the expected outcome because... indeed, things work differently in damage control mode, and the only law you can trust is physics: you always go down when you can't stay on your feet. With hard drives, raid controllers, mainboards, data buses, it's exactly the same. If _something_ isn't working as it should, how should we know which part of it we _can_ trust? >> In closing, I'll simply say this: If hardware, whether a mobo-down SATA >> chip, or a $100K SGI SAN RAID controller, allowed silent data corruption >> or transmission to occur, there would be no storage industry, and we'll >> all still be using pen and paper. The questions you're asking were >> solved by hardware and software engineers decades ago. You're fretting >> and asking about things that were solved decades ago. Isn't it just "worked around" by adding more layers of checksuming and adding more redundancy into the mix? Don't believe this "storage industry" because they tell you it's OK. It simply is not OK. You might want to talk to people in the data and computing cluster business about their opinion on "storage industry professionals"... Timo's suggestion to add checksums to mailboxes/metadata could help to (at least) report these types of failures. Re-reading from different storage when available could also recover the data that got corrupted, but I'm not sure what would be the best way to handle these situations. If you know there is a corruption problem on one of your storage locations, you might want to switch that to read-only asap. Automagically trying to recover might not be the best thing to do. Given all kinds of different use cases, I think that should at least be configurable :-P -- Maarten From lists at necoro.eu Fri Apr 13 23:41:02 2012 From: lists at necoro.eu (=?UTF-8?B?UmVuw6kgTmV1bWFubg==?=) Date: Fri, 13 Apr 2012 22:41:02 +0200 Subject: [Dovecot] imapc / namespace question Message-ID: <4F888F5E.5030709@necoro.eu> Hi, I've got two questions regarding imapc and namespaces: 1) Is there a way of stating, that I only want part of the hierarchy on the end of an imapc connection to be available here? My "problem": On the other end the layout is: INBOX.Shared.{some,nice,folders} If I mirror this into my dovecot hierarchy with prefix "Gemeinsam" I get the following folder structure: Gemeinsam.INBOX.Shared.{some,nice,folders} But I want to ignore the two top-most parts of the hierarchy, i.e. I want it to look: Gemeinsam.{some,nice,folders} Is this possible? I tried with 'location=imapc:INBOX.Shared.' but this did not change anything. 2) "Private" namespaces only make sense, if location contains something user-specific like "%u" or "~". Is this correct? If yes: Is there some other way to make something like an imapc connection only appear for one user, if there is nothing in the location or imapc_user to denote the user it belongs to and hence I cannot use "private"? The only way I see currently is to use "shared" and set up ACLs to forbid all users except one to access the namespace (which gets slightly ugly as I have to use global ACLs, as -- I think -- there is no way to store ACLs for an imapc storage). Thanks, Ren? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From joe at tao.org.uk Sat Apr 14 00:59:36 2012 From: joe at tao.org.uk (Dr Josef Karthauser) Date: Fri, 13 Apr 2012 22:59:36 +0100 Subject: [Dovecot] How do I test if the anti spam plugin is working? In-Reply-To: <3779AD95-CA9E-484B-8B63-039F50B0426E@tao.org.uk> References: <3779AD95-CA9E-484B-8B63-039F50B0426E@tao.org.uk> Message-ID: <4692B327-5985-4227-82A4-C30B4A291C4E@tao.org.uk> On 30 Mar 2012, at 12:29, Dr Josef Karthauser wrote: > I've configured the dspam anti spam plugin, but it doesn't appear to be doing anything when I move mail between mailboxes. Can anyone help me determine what's going on? > > The plugin appears to be loading; at least if I don't define all the required configuration options I get a complaint in the log file. But beyond that I don't see any activity. > > [cut] > I don't even appear to be seeing any log entries from the plugin. > > I've moving an email from my main mailbox into a mailbox called 'SPAM', which is how I thought that it was supposed to be triggered. It turns out that I had the signature header configured incorrectly. :). Joe From stan at hardwarefreak.com Sat Apr 14 06:31:04 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 13 Apr 2012 22:31:04 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F8846D7.7020900@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F8846D7.7020900@wildgooses.com> Message-ID: <4F88EF78.7040202@hardwarefreak.com> On 4/13/2012 10:31 AM, Ed W wrote: > On 13/04/2012 13:33, Stan Hoeppner wrote: >> In closing, I'll simply say this: If hardware, whether a mobo-down SATA >> chip, or a $100K SGI SAN RAID controller, allowed silent data corruption >> or transmission to occur, there would be no storage industry, and we'll >> all still be using pen and paper. The questions you're asking were >> solved by hardware and software engineers decades ago. You're fretting >> and asking about things that were solved decades ago. > > So why are so many people getting excited about it now? "So many"? I know of one person "getting excited" about it. Data densities and overall storage sizes and complexity at the top end of the spectrum are increasing at a faster rate than the consistency/validation mechanisms. That's the entire point of the various academic studies on the issue. Note that the one study required a sample set of 1.5 million disk drives. If the phenomenon were a regular occurrence as you would have everyone here believe, they could have used a much smaller sample set. Ed, this is an academic exercise. Academia leads industry. Almost always has. Academia blows the whistle and waves hands, prompting industry to take action. There is nothing normal users need to do to address this problem. The hardware and software communities will make the necessary adjustments to address this issue before it filters down to the general user community in a half decade or more--when normal users have a 10-20 drive array of 500TB to 1PB or more. Having the prestigious degree that you do, you should already understand the relationship between academic research and industry, and the considerable lead times involved. -- Stan From stan at hardwarefreak.com Sat Apr 14 06:48:07 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 13 Apr 2012 22:48:07 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F8846D7.7020900@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F8846D7.7020900@wildgooses.com> Message-ID: <4F88F377.1070801@hardwarefreak.com> On 4/13/2012 10:31 AM, Ed W wrote: > You mean those "answers" like: > "you need to read 'those' articles again" > > Referring to some unknown and hard to find previous emails is not the > same as answering? No, referring to this: On 4/12/2012 5:58 AM, Ed W wrote: > The claim by ZFS/BTRFS authors and others is that data silently "bit > rots" on it's own. Is it not a correct assumption that you read this in articles? If you read this in books, scrolls, or chiseled tablets, my apologies for assuming it was articles. -- Stan From ngu.antoine at gmail.com Sat Apr 14 12:50:32 2012 From: ngu.antoine at gmail.com (Antoine Nguyen) Date: Sat, 14 Apr 2012 11:50:32 +0200 Subject: [Dovecot] Unseen messages question Message-ID: Hi list, this question is related to the IMAP protocol itself, not really to Dovecot. I'm trying to understand what is the more efficient way to maintain the number of unseen messages of the currently selected mailbox. RFC3501 says a client must not issue a STATUS command to the selected mailbox and that information sent by a SELECT is enough. My current idea follows these steps : * Issue a STATUS before the mailbox is selected => I know how many unseen messages it contains * SELECT the mailbox => I got the eventual first unseen message in this mailbox but I don't understand how this info can be useful * Maintain the unseen counter (on client side) according to what the user do * Send a NOOP command every X minutes and look at the RECENT response to see if there are new messages I think it works pretty well when the mailbox is opened only once. Let's imagine this mailbox is opened twice, by different clients. If one client marks a message as \Seen, how can the second client know about this change? Thanks for your help, Antoine Nguyen http://modoboa.org/ From lists at wildgooses.com Sat Apr 14 13:00:40 2012 From: lists at wildgooses.com (Ed W) Date: Sat, 14 Apr 2012 11:00:40 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F88F377.1070801@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F8846D7.7020900@wildgooses.com> <4F88F377.1070801@hardwarefreak.com> Message-ID: <4F894AC8.9060406@wildgooses.com> On 14/04/2012 04:48, Stan Hoeppner wrote: > On 4/13/2012 10:31 AM, Ed W wrote: > >> You mean those "answers" like: >> "you need to read 'those' articles again" >> >> Referring to some unknown and hard to find previous emails is not the >> same as answering? > No, referring to this: > > On 4/12/2012 5:58 AM, Ed W wrote: > >> The claim by ZFS/BTRFS authors and others is that data silently "bit >> rots" on it's own. > Is it not a correct assumption that you read this in articles? If you > read this in books, scrolls, or chiseled tablets, my apologies for > assuming it was articles. > WHAT?!! The original context was that you wanted me to learn some very specific thing that you accused me of misunderstanding, and then it turns out that the thing I'm supposed to learn comes from re-reading every email, every blog post, every video, every slashdot post, every wiki, every ... that mentions ZFS's reason for including end to end checksumming?!! Please stop wasting our time and get specific You have taken my email which contained a specific question, been asked of you multiple times now and yet you insist on only answering irrelevant details with a pointed and personal dig on each answer. The rudeness is unnecessary, and your evasiveness of answers does not fill me with confidence that you actually know the answer... For the benefit of anyone reading this via email archives or whatever, I think the conclusion we have reached is that: modern systems are now a) a complex sum of pieces, any of which can cause an error to be injected, b) the level of error correction which was originally specified as being sufficient is now starting to be reached in real systems, possibly even consumer systems. There is no "solution", however, the first step is to enhance "detection". Various solutions have been proposed, all increase cost, computation or have some disadvantage - however, one of the more promising detection mechanisms is an end to end checksum, which will then have the effect of augmenting ALL the steps in the chain, not just one specific step. As of today, only a few filesystems offer this, roll on more adopting it Regards Ed W From janfrode at tanso.net Sat Apr 14 13:04:22 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sat, 14 Apr 2012 12:04:22 +0200 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F881D0F.2090802@hardwarefreak.com> References: <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> Message-ID: <20120414100422.GA5353@dibs.tanso.net> On Fri, Apr 13, 2012 at 07:33:19AM -0500, Stan Hoeppner wrote: > > > > What I meant wasn't the drive throwing uncorrectable read errors but > > the drives are returning different data that each think is correct or > > both may have sent the correct data but one of the set got corrupted > > on the fly. After reading the articles posted, maybe the correct term > > would be the controller receiving silently corrupted data, say due to > > bad cable on one. > > This simply can't happen. What articles are you referring to? If the > author is stating what you say above, he simply doesn't know what he's > talking about. It has happened to me, with RAID5 not RAID1. It was a firmware bug in the raid controller that caused the RAID array to go silently corrupted. The HW reported everything green -- but the filesystem was reporting lots of strange errors.. This LUN was part of a larger filesystem striped over multiple LUNs, so parts of the fs was OK, while other parts was corrupt. It was this bug: http://delivery04.dhe.ibm.com/sar/CMA/SDA/02igj/7/ibm_fw1_ds4kfc_07605200_anyos_anycpu.chg - Fix 432525 - CR139339 Data corruption found on drive after reconstruct from GHSP (Global Hot Spare) > In closing, I'll simply say this: If hardware, whether a mobo-down SATA > chip, or a $100K SGI SAN RAID controller, allowed silent data corruption > or transmission to occur, there would be no storage industry, and we'll > all still be using pen and paper. The questions you're asking were > solved by hardware and software engineers decades ago. You're fretting > and asking about things that were solved decades ago. Look at the plans are for your favorite fs: http://www.youtube.com/watch?v=FegjLbCnoBw They're planning on doing metadata checksumming to be sure they don't receive corrupted metadata from the backend storage, and say that data validation is a storage subsystem *or* application problem. Hardly a solved problem.. -jf From lists at wildgooses.com Sat Apr 14 13:22:37 2012 From: lists at wildgooses.com (Ed W) Date: Sat, 14 Apr 2012 11:22:37 +0100 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F88EF78.7040202@hardwarefreak.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F8846D7.7020900@wildgooses.com> <4F88EF78.7040202@hardwarefreak.com> Message-ID: <4F894FED.70901@wildgooses.com> On 14/04/2012 04:31, Stan Hoeppner wrote: > On 4/13/2012 10:31 AM, Ed W wrote: >> On 13/04/2012 13:33, Stan Hoeppner wrote: >>> In closing, I'll simply say this: If hardware, whether a mobo-down SATA >>> chip, or a $100K SGI SAN RAID controller, allowed silent data corruption >>> or transmission to occur, there would be no storage industry, and we'll >>> all still be using pen and paper. The questions you're asking were >>> solved by hardware and software engineers decades ago. You're fretting >>> and asking about things that were solved decades ago. >> So why are so many people getting excited about it now? > "So many"? I know of one person "getting excited" about it. You love being vague don't you? Go on, I'll bite again, do you mean yourself? :-) > Data densities and overall storage sizes and complexity at the top end > of the spectrum are increasing at a faster rate than the > consistency/validation mechanisms. That's the entire point of the > various academic studies on the issue. Again, you love being vague. By your dismissive "academic studies" phrase, do you mean studies done on a major industrial player, ie NetApp in this case? Or do you mean that it's rubbish because they asked someone with some background in statistics to do the work, rather than asking someone sitting nearby in the office to do it? I don't think the researcher broke into NetApp to do this research, so we have to conclude that the industrial partner was onboard. NetApp seem to do a bunch of engineering of their own (got enough patents..) that I think we can safely assume they very much do their own research on this and it's not just "academic"... I doubt they publish all their own internal research, be thankful you got to see some of the results this way... > Note that the one study required > a sample set of 1.5 million disk drives. If the phenomenon were a > regular occurrence as you would have everyone here believe, they could > have used a much smaller sample set. Sigh... You could criticise the study if it had a small number of drives as being under-representive and now you criticise a large study for having too many observations... You cannot have "too many" observations when measuring a small and unpredictable phenomena... Where does it say that they could NOT have reproduced this study with just 10 drives? If you have 1.5 million available, why not use all the results?? > Ed, this is an academic exercise. Academia leads industry. Almost > always has. Academia blows the whistle and waves hands, prompting > industry to take action. Sigh... We are back to the start of the email thread again... Gosh you seem to love arguing and muddying the water for zero reason but to have the last word? It's *trivial* to do a google search and hit *lots* of reports of corruptions in various parts of the system, from corrupting drivers, to hardware which writes incorrectly, to operating system flaws. I just found a bunch more in the Redhat database today while looking for something else. You yourself are very vocal on avoiding certain brands of HD controller which have been rumoured to cause corrupted data... (and thankyou for revealing that kind of thing - it's very helpful) Don't veer off at a tangent now: The *original* email this has spawned is about a VERY specific point. RAID1 appears to offer less protection against a class of error conditions than does RAID6. Nothing more, nothing less. Don't veer off and talk about the minutiae of testing studies at universities, this is a straightforward claim that you have been jumping around and avoiding answering with claims of needing to educate me on SCSI protocols and other fatuous responses. Nor deviate and discuss that RAID6 is inappropriate for many situations - we all get that... > There is nothing normal users need to do to address this problem. ...except sit tight and hope they don't loose anything important! :-) > Having the prestigious degree that you do, you should already understand > the relationship between academic research and industry, and the > considerable lead times involved. I'm guessing you haven't attended higher education then? You are confusing graduate and post-graduate systems... Byee Ed W From jerry at seibercom.net Sat Apr 14 13:51:13 2012 From: jerry at seibercom.net (Jerry) Date: Sat, 14 Apr 2012 06:51:13 -0400 Subject: [Dovecot] [OT] Outlook identities In-Reply-To: <4F8850AA.80903@orlitzky.com> References: <4F876642.1040701@orlitzky.com> <4F87C949.8010405@schetterer.org> <4F8850AA.80903@orlitzky.com> Message-ID: <20120414065113.79eb4065@scorpio> On Fri, 13 Apr 2012 12:13:30 -0400 Michael Orlitzky articulated: > Exchange... the cure is worse than the disease! This isn't looking > good -- I guess I'll continue to do what I have been: telling people > to switch off of Outlook if they want their mail client to not suck. First of all, there are no existing RFC's that require any MUA to meet the requirements that you desire. So please, stop your wining and crying. It is embarrassing. Second, there are avenues available that can make Outlook behave in a fashion that should be acceptable to you. If you choose not to pursue them, then that is you business. I have had to endure hours of tedious nonsense to get a simple sound card to work under a *.nix environment when I could have simply plugged it into a machine running Microsoft Windows and had it working immediately. Your "the cure is worse than the disease" is just self-serving bull-shit. Outlook + MS Exchange offers features that no other MUA presently comes close to being able to duplicate in an office environment. If these don't fit your needs, then please find an MUA that does. No one is holding a gun to your head. However, your desire to force others to abandon something that works fine for them to simple suit your narrow view of what an MUA should or should not do stinks of fascism. I use Outlook at work and claws-mail at home. Each one fits perfectly into the environment I have chosen to use it in. By the way, after examining your original post, I cannot find a single thing that the proper use of filtering rules and plugins cannot easily accomplish. Instead of your customers using a different MUA, they should consider changing to a new service provider. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From amk at spamfence.net Sat Apr 14 14:04:17 2012 From: amk at spamfence.net (Andreas M. Kirchwitz) Date: Sat, 14 Apr 2012 11:04:17 +0000 (UTC) Subject: [Dovecot] Dovecot 2.1 with custom OpenSSL fails to build References: <1331816286.10319.23.camel@innu.invalid> <4F7BBF3F.9060103@iki.fi> Message-ID: Timo Sirainen wrote: >> But two libraries are not quite okay. They don't find their SSL libs: >> >> libdovecot-lda.so >> libdovecot-storage.so > > Maybe this fixes it? > > http://hg.dovecot.org/dovecot-2.1/rev/8b91367bc3e1 Works perfectly! Great, now all components find their libraries by themselves. Thanks a lot for fixing this issue which seemed quite complicated. Very good, thank you ... Andreas From kjonca at o2.pl Sat Apr 14 14:27:59 2012 From: kjonca at o2.pl (Kamil =?iso-8859-2?Q?Jo=F1ca?=) Date: Sat, 14 Apr 2012 13:27:59 +0200 Subject: [Dovecot] Compressed mbox - patch Message-ID: <871unq7dts.fsf@alfa.kjonca> Some time ago I complained about very slow access to compressed mboxes. Unfortunately it looks like that it is very little interest in it, so I have to investigate some things by myself. Firstly: some rationale. Why do I prefer use mbox/maildir over mdbox. Short answer "bus factor" for support mdbox (not only dovecot) Longer answer: if something goes wrong withm maildir/mbox i can use other tools (mutt, or formail or even text editor) and with mdbox ... I am not ISP, I use dovecot as a "gateway" to my (rather huge) mail archive. Most of these mails are rather valuable for me, so I prefer use something "well-known-and-tested". (I can't do like most ISP's do: write in "Terms of Service" that mail can be lost or damaged and we give no warranty :) ) So then: Below my patch. It contains 2 changes: 1. when buffer is compressed, we try to save last marked offset. 2. Increase temporary buffer for decompression. without these changes 1.5 GB of bzip compressed mbox with ~20K messages can't be open in 1.5 day After applying 1. change it can be open in ~1.5 h With both changes it was a few minutes. Maybe it is a good idea to add config parameter to specify size of decompress buffer? Patch is against v2.0.18 -------------- next part -------------- A non-text attachment was scrubbed... Name: compress.patch Type: text/x-diff Size: 3898 bytes Desc: not available URL: -------------- next part -------------- -- Gdyby kto? mia? zb?dny Toshiba G450 - to ch?tnie przejm? ;) < asuffield> a workstation is anything you can stick on somebodies desk and con them into using -- in #debian-devel From kjonca at o2.pl Sat Apr 14 15:13:58 2012 From: kjonca at o2.pl (Kamil =?iso-8859-2?Q?Jo=F1ca?=) Date: Sat, 14 Apr 2012 14:13:58 +0200 Subject: [Dovecot] Sieve pipe extension - can it retur something? Message-ID: <87ty0m5x4p.fsf@alfa.kjonca> I have a question about sieve pipe: can it return something to further processing? For example in procmail I can do: --8<---------------cut here---------------start------------->8--- :0 VIRUS=`$CLAMSCAN --mbox --disable-summary --stdout -` --8<---------------cut here---------------end--------------->8--- and then test VIRUS variable. Maybe I missing something, when read http://hg.rename-it.nl/pigeonhole-0.2-sieve-pipe/raw-file/tip/doc/rfc/spec-bosch-sieve-pipe.txt KJ -- http://sporothrix.wordpress.com/2011/01/16/usa-sie-krztusza-kto-nastepny/ Gloffing is a state of mine. From branko at majic.rs Sat Apr 14 16:30:06 2012 From: branko at majic.rs (=?UTF-8?B?0JHRgNCw0L3QutC+INCc0LDRmNC40Zs=?=) Date: Sat, 14 Apr 2012 15:30:06 +0200 Subject: [Dovecot] Dovecot 2.1.4 and client certificates Message-ID: <20120414153006.19ce7e3c@majic.rs> Version: 2.1.4 OS: Gentoo stable/amd64 OpenSSL version: 1.0.0h I'm having a slight problem with the client certificates in Dovecot 2.1.4. I've set-up the client certificate verification/authentication, and it seems that Dovecot is choking on the trustchain with CRL's that I'm providing to it (attached to this mail). When I enable the client authentication using certificates, and pick the certificate from my client (I've also tried it out with gnutls-cli as well), I get the following errors in Dovecot's log: imap-login: Info: Invalid certificate: Different CRL scope: /CN=Example Root CA/O=Example Inc./C=RS As per the wiki2 configuration page, I've set up the truststore in the following order (everything PEM-encoded): Example Person CA Certificate Example Person CA CRL Example Root CA Certificate Example Root CA CRL Person CA is the one issuing the end-entity certificates, of course. I'm also attaching the certificate I've used for testing. On additional note, the imap-login process also got stuck writing out the error message to the log file, refusing to die when receiving the SIGTERM (had to send SIGKILL). A similar set-up used to work under Dovecot in Debian Squeeze (version 1.2.15). The same file copied over to Dovecot 2.1.4's configuration won't work. I've compiled Dovecot by hand, and I'm not running it in any kind of chroot (this is a developer set-up so I could add support for rfc822Name username extraction I mentioned a week or so ago without messing around as root). Best regards -- Branko Majic Jabber: branko at majic.rs Please use only Free formats when sending attachments to me. ?????? ????? ?????: branko at majic.rs ????? ??? ?? ??????? ?????? ????????? ? ????????? ?????????. -------------- next part -------------- A non-text attachment was scrubbed... Name: trustchain.pem Type: application/x-x509-ca-cert Size: 6640 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: branko_majic.crt Type: application/x-x509-ca-cert Size: 1700 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From cor at xs4all.nl Sat Apr 14 19:24:24 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sat, 14 Apr 2012 18:24:24 +0200 Subject: [Dovecot] LMTP auth problem Message-ID: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> hey all, im getting the following error: Apr 14 14:29:44 lmtpdirector1 dovecot: auth: Error: passdb(scorpio,127.0.0.1): Auth client doesn't have permissions to do a PASS lookup: /var/run/dovecot/auth-userdb mode=0666, but not owned by UID 112(dovecot) Apr 14 14:29:44 lmtpdirector1 dovecot: lmtp(18298): Error: user scorpio: Auth PASS lookup failed My config. Director servers running both imap and lmtp with a matching set of real servers accepting imap/lmtp. Imap is working fine, and has been working fine for a while. Im trying to add lmtp to the director, but i cant seem to get that working. We're passing passdb on to the real servers. How does this work with lmtp? protocols = imap lmtp protocol lmtp { auth_socket_path = director-userdb } lmtp_proxy = yes # passdb check on real servers passdb { driver = static args = proxy=y nopassword=y } Cor From cor at xs4all.nl Sat Apr 14 19:52:40 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sat, 14 Apr 2012 18:52:40 +0200 Subject: [Dovecot] LMTP auth problem In-Reply-To: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> References: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> Message-ID: <20120414165240.GA31983@xs4all.nl> Of course the moment I post I seem to have figured it out.. service auth { unix_listener auth-userdb { mode = 0777 } } Is this safe if your servers are secure? Cor From tlx at leuxner.net Sat Apr 14 20:10:44 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sat, 14 Apr 2012 19:10:44 +0200 Subject: [Dovecot] LMTP auth problem In-Reply-To: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> References: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> Message-ID: <71BEFF67-4A0A-442A-9ED0-C4B1B23A4E7F@leuxner.net> Am 14.04.2012 um 18:24 schrieb Cor Bosman: > Apr 14 14:29:44 lmtpdirector1 dovecot: auth: Error: passdb(scorpio,127.0.0.1): Auth client doesn't have permissions to do a PASS lookup: /var/run/dovecot/auth-userdb mode=0666, but not owned by UID 112(dovecot) > Apr 14 14:29:44 lmtpdirector1 dovecot: lmtp(18298): Error: user scorpio: Auth PASS lookup failed I'd just try 'user = dovecot' rather than making it wide open because that's what the log basically says. $ doveconf -d | grep 'unix_listener auth-userdb' -A 4 unix_listener auth-userdb { group = mode = 0666 user = } Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From cor at xs4all.nl Sat Apr 14 20:21:35 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sat, 14 Apr 2012 19:21:35 +0200 Subject: [Dovecot] LMTP auth problem In-Reply-To: <71BEFF67-4A0A-442A-9ED0-C4B1B23A4E7F@leuxner.net> References: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> <71BEFF67-4A0A-442A-9ED0-C4B1B23A4E7F@leuxner.net> Message-ID: <20120414172135.GA32608@xs4all.nl> > > Apr 14 14:29:44 lmtpdirector1 dovecot: auth: Error: passdb(scorpio,127.0.0.1): Auth client doesn't have permissions to do a PASS lookup: /var/run/dovecot/auth-userdb mode=0666, but not owned by UID 112(dovecot) > > Apr 14 14:29:44 lmtpdirector1 dovecot: lmtp(18298): Error: user scorpio: Auth PASS lookup failed > > I'd just try 'user = dovecot' rather than making it wide open because that's what the log basically says. > > $ doveconf -d | grep 'unix_listener auth-userdb' -A 4 > unix_listener auth-userdb { > group = > mode = 0666 > user = > } > My config was the same as yours. That didnt work for me. But if I add user = dovecot mode = 0666 That does work. Of course, the difference between 777 and 666 is minimal. I think 666 is handled as a special case in the code? Cor From stephan at rename-it.nl Sun Apr 15 01:36:08 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 15 Apr 2012 00:36:08 +0200 Subject: [Dovecot] Sieve pipe extension - can it retur something? In-Reply-To: <87ty0m5x4p.fsf@alfa.kjonca> References: <87ty0m5x4p.fsf@alfa.kjonca> Message-ID: <4F89FBD8.9040501@rename-it.nl> Op 4/14/2012 2:13 PM, Kamil Jo?ca schreef: > > I have a question about sieve pipe: can it return something to further > processing? > For example in procmail I can do: > --8<---------------cut here---------------start------------->8--- > :0 > VIRUS=`$CLAMSCAN --mbox --disable-summary --stdout -` > --8<---------------cut here---------------end--------------->8--- > and then test VIRUS variable. > > Maybe I missing something, when read > http://hg.rename-it.nl/pigeonhole-0.2-sieve-pipe/raw-file/tip/doc/rfc/spec-bosch-sieve-pipe.txt For Pigeonhole 0.3/Dovecot 2.1 there is a new plugin called ExtPrograms. Apart from the 'pipe' extension it adds the 'execute' extension that should match just what you want: http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/raw-file/d4683490a878/doc/rfc/spec-bosch-sieve-extprograms.txt Regards, Stephan. From stan at hardwarefreak.com Sun Apr 15 01:39:55 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 14 Apr 2012 17:39:55 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <20120414100422.GA5353@dibs.tanso.net> References: <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <20120414100422.GA5353@dibs.tanso.net> Message-ID: <4F89FCBB.4070305@hardwarefreak.com> On 4/14/2012 5:04 AM, Jan-Frode Myklebust wrote: > On Fri, Apr 13, 2012 at 07:33:19AM -0500, Stan Hoeppner wrote: >>> >>> What I meant wasn't the drive throwing uncorrectable read errors but >>> the drives are returning different data that each think is correct or >>> both may have sent the correct data but one of the set got corrupted >>> on the fly. After reading the articles posted, maybe the correct term >>> would be the controller receiving silently corrupted data, say due to >>> bad cable on one. >> >> This simply can't happen. What articles are you referring to? If the >> author is stating what you say above, he simply doesn't know what he's >> talking about. > > It has happened to me, with RAID5 not RAID1. It was a firmware bug > in the raid controller that caused the RAID array to go silently > corrupted. The HW reported everything green -- but the filesystem was > reporting lots of strange errors.. This LUN was part of a larger > filesystem striped over multiple LUNs, so parts of the fs was OK, while > other parts was corrupt. > > It was this bug: > > http://delivery04.dhe.ibm.com/sar/CMA/SDA/02igj/7/ibm_fw1_ds4kfc_07605200_anyos_anycpu.chg > - Fix 432525 - CR139339 Data corruption found on drive after > reconstruct from GHSP (Global Hot Spare) Note my comments were specific to the RAID1 case, or a concatenated set of RAID1 devices. And note the discussion was framed around silent corruption in the absence of bugs and hardware failure, or should I say, where no bugs or hardware failures can be identified. > > >> In closing, I'll simply say this: If hardware, whether a mobo-down SATA >> chip, or a $100K SGI SAN RAID controller, allowed silent data corruption >> or transmission to occur, there would be no storage industry, and we'll >> all still be using pen and paper. The questions you're asking were >> solved by hardware and software engineers decades ago. You're fretting >> and asking about things that were solved decades ago. > > Look at the plans are for your favorite fs: > > http://www.youtube.com/watch?v=FegjLbCnoBw > > They're planning on doing metadata checksumming to be sure they don't > receive corrupted metadata from the backend storage, and say that data > validation is a storage subsystem *or* application problem. You can't made sure you don't receive corrupted data. You take steps to mitigate the negative effects of it if and when it happens. The XFS devs are planning this for the future. If the problem was here now, this work would have already been done. > Hardly a solved problem.. It has been up to this point. The issue going forward is that current devices don't employ sufficient consistency checking to meet future needs. And the disk drive makers apparently don't want to consume the additional bits required to properly do this in the drives. If they'd dedicate far more bits to ECC we may not have this issue. But since it appears this isn't going to change, kernel, filesystem and application developers are taking steps to mitigate it. Again, this "silent corruption" issue as described in the various academic papers is a future problem for most, not a current problem. It's only a current problem for those are the bleeding edge of large scale storage. Note that firmware bugs in individual products aren't part of this issue. Those will be with us forever in various products because humans make mistakes. No amount of filesystem or application code can mitigate those. The solution to that is standard best practices: snapshots, backups, or even mirroring all your storage across different vendor hardware. -- Stan From stan at hardwarefreak.com Sun Apr 15 03:05:19 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 14 Apr 2012 19:05:19 -0500 Subject: [Dovecot] Better to use a single large storage server or multiple smaller for mdbox? In-Reply-To: <4F894AC8.9060406@wildgooses.com> References: <4F8014C2.3010606@hardwarefreak.com> <4F81D73B.9030901@hardwarefreak.com> <4F83BE63.9050603@hardwarefreak.com> <4F853059.1000706@hardwarefreak.com> <4F85B641.5060103@wildgooses.com> <4F862D50.8000206@hardwarefreak.com> <4F86AC6F.4070304@hardwarefreak.com> <4F881D0F.2090802@hardwarefreak.com> <4F8846D7.7020900@wildgooses.com> <4F88F377.1070801@hardwarefreak.com> <4F894AC8.9060406@wildgooses.com> Message-ID: <4F8A10BF.3020706@hardwarefreak.com> On 4/14/2012 5:00 AM, Ed W wrote: > On 14/04/2012 04:48, Stan Hoeppner wrote: >> On 4/13/2012 10:31 AM, Ed W wrote: >> >>> You mean those "answers" like: >>> "you need to read 'those' articles again" >>> >>> Referring to some unknown and hard to find previous emails is not the >>> same as answering? >> No, referring to this: >> >> On 4/12/2012 5:58 AM, Ed W wrote: >> >>> The claim by ZFS/BTRFS authors and others is that data silently "bit >>> rots" on it's own. >> Is it not a correct assumption that you read this in articles? If you >> read this in books, scrolls, or chiseled tablets, my apologies for >> assuming it was articles. >> > > WHAT?!! The original context was that you wanted me to learn some very > specific thing that you accused me of misunderstanding, and then it > turns out that the thing I'm supposed to learn comes from re-reading > every email, every blog post, every video, every slashdot post, every > wiki, every ... that mentions ZFS's reason for including end to end > checksumming?!! No, the original context was your town crier statement that the sky is falling due to silent data corruption. I pointed out that this is not the case, currently, that most wouldn't see this until quite a few years down the road. I provided facts to back my statement, which you didn't seem to grasp or comprehend. I pointed this out and your top popped with a cloud of steam. > Please stop wasting our time and get specific Whose time am I wasting Ed? You're the primary person one on this list who wastes everyone's time with these drawn out threads, usually unrelated to Dovecot. I have been plenty specific. The problem is you lack the knowledge and understanding of hardware communication. You're upset because I'm not pointing out the knowledge you seem to lack? Is that not a waste of everyone's time? Is that not be even "more insulting"? Causing even more excited/heated emails from you? > You have taken my email which contained a specific question, been asked > of you multiple times now and yet you insist on only answering > irrelevant details with a pointed and personal dig on each answer. The > rudeness is unnecessary, and your evasiveness of answers does not fill > me with confidence that you actually know the answer... Ed, I have not been rude. I've been attempting to prevent you dragging us into the mud, which you've done, as you often do. How specific would you like me to get? This is what you seem to be missing: Drives perform per sector CRC before transmitting data to the HBA. ATA, SATA, SCSI, SAS, fiber channel devices and HBAs all perform CRC on wire data. The PCI/PCI-X/PCIe buses/channels and Southbridge all perform CRC on wire data. HyperTransport, and Intel's proprietary links also perform CRC on wire transmissions. Server memory is protected by ECC, some by ChipKill which can tolerate double bit errors. With today's systems and storage densities, with error correcting code on all data paths within the system, and on the drives themselves, "silent data corruption" is not an issue--in absence of defective hardware or a bug, which are not relevant to the discussion. > For the benefit of anyone reading this via email archives or whatever, I > think the conclusion we have reached is that: modern systems are now a) > a complex sum of pieces, any of which can cause an error to be injected, Errors occur all the time. And they're corrected nearly all of the time, on modern complex systems. Silent errors do not occur frequently, usually not at all, on most modern systems. > b) the level of error correction which was originally specified as being > sufficient is now starting to be reached in real systems, FSVO 'real systems'. The few occurrences of "silent data corruption" I'm aware of have been documented in academic papers published by researches working at taxpayer funded institutions. In the case of CERN, the problem was a firmware bug in the Western Digital drives that caused an issue with the 3Ware controllers. This kind of thing happens when using COTS DIY hardware in the absence of proper load validation testing. So this case doesn't really fit the Henny-penny silent data corruption scenario as a firmware bug caused it. One that should have been caught and corrected during testing. In the other cases I'm aware of, all were HPC systems which generated SDC under extended high loads, and these SDCs nearly all occurred somewhere other than the storage systems--CPUs, RAM, interconnect, etc. HPC apps tend to run the CPUs, interconnects, storage, etc, at full bandwidth for hours at a time, across tens of thousands of nodes, so the probability of SDC is much higher simply due to scale. > possibly even > consumer systems. Possibly? If you're going to post pure conjecture why not say "possibly even iPhones or Androids"? There's no data to back either claim. Stick to the facts. > There is no "solution", however, the first step is to > enhance "detection". Various solutions have been proposed, all increase > cost, computation or have some disadvantage - however, one of the more > promising detection mechanisms is an end to end checksum, which will > then have the effect of augmenting ALL the steps in the chain, not just > one specific step. As of today, only a few filesystems offer this, roll > on more adopting it So after all the steam blowing, we're back to where we started. I disagree with your assertion that this is an issue that we--meaning "average" users not possessing 1PB storage systems or massive clusters--need to be worried about TODAY. I gave sound reasons as to why this is the case. You've given us 'a couple of academic papers say the sky is falling so I'm repeating the sky is falling'. Without apparently truly understanding the issue. The data available and the experience of the vast majority of IT folks backs my position--which is why that's my position. There is little to no data supporting your position. I say this isn't going to be an issue for average users, if at all, for a few years to come. You say it's here now. That's a fairly minor point of disagreement to cause such a heated (on your part) lengthy exchange. BTW, if you see anything I've stated as rude you've apparently not been on the Interwebs long. ;) -- Stan From dovecot-user at spambox.dk Sun Apr 15 12:42:16 2012 From: dovecot-user at spambox.dk (Henrik Larsson) Date: Sun, 15 Apr 2012 11:42:16 +0200 Subject: [Dovecot] 2.1.2 Corrupted squat uidlist In-Reply-To: <4F7164DC.7010706@unict.it> References: <4F7164DC.7010706@unict.it> Message-ID: <4F8A97F8.70808@spambox.dk> On 27-03-2012 08:57, Luca Palazzo wrote: > Hi Timo and All, > after upgrading to 2.1.2 i'm getting a lot of these messages: > Error: Corrupted squat uidlist file XXXXXX wrong indexid After an upgrade to 2.1.3 i see this as well. Any thoughts? Apr 15 03:43:43 imap(xxxx): Error: Corrupted squat uidlist file /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid Apr 15 06:30:27 imap(xxxx): Error: Corrupted squat uidlist file /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid Apr 15 07:43:55 pop3(xxxx): Error: Corrupted squat uidlist file /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid Apr 15 09:00:01 imap(xxxx): Error: Corrupted squat uidlist file /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid # doveconf -n # 2.1.3: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 8.2-STABLE amd64 auth_mechanisms = plain login digest-md5 cram-md5 first_valid_uid = 125 listen = * log_path = /var/log/dovecot mail_plugins = fts fts_squat zlib mail_privileged_group = postfix mail_temp_dir = /var/db/dovecot managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = prefix = separator = . type = private } namespace { hidden = yes inbox = no list = no location = prefix = INBOX. separator = . type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { fts = squat fts_squat = partial=4 full=10 sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve service auth-worker { user = $default_internal_user } service auth { unix_listener /home/mail/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } service lmtp { executable = lmtp -L unix_listener /home/mail/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } ssl_cert = Hi Timo the virtual setup got broke after update to 2.1 ( its still working under 2.0 ) on other loadbalanced server namespace default { separator = / prefix = "" list = yes subscriptions = yes hidden = no } namespace virtual { prefix = "virtual/" separator = / location = virtual:/etc/dovecot/virtual:LAYOUT=maildir++:INDEX=~/virtual hidden = yes list = no subscriptions= no } namespace real { prefix = "RealMails/" separator = / list = no hidden = yes } after upgrade i have Error: Couldn't open INBOX: Virtual mailbox open failed because of mailbox /RealMails/: Invalid mailbox name --------------------- also i wanted to include namespace inbox { #mailbox name { # auto=create will automatically create this mailbox. # auto=subscribe will both create and subscribe to the mailbox. #auto = no # Space separated list of IMAP SPECIAL-USE attributes as specified by # RFC 6154: \All \Archive \Drafts \Flagged \Junk \Sent \Trash #special_use = #} # These mailboxes are widely used and could perhaps be created automatically: mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Trash { special_use = \Trash } # For \Sent mailboxes there are two widely used names. We'll mark both of # them as \Sent. User typically deletes one of them if duplicates are created. mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } # If you have a virtual "All messages" mailbox: mailbox virtual/All { special_use = \All } # If you have a virtual "Flagged" mailbox: mailbox virtual/Flagged { special_use = \Flagged } } but i found no working namespace combination with virtual -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Sun Apr 15 16:14:25 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 15 Apr 2012 15:14:25 +0200 Subject: [Dovecot] http://xi.rename-it.nl upgrade 2.0 - 2.1 dovecot-common (<< 2:2.1.alpha1-0~auto+47) Message-ID: <4F8AC9B1.2030307@schetterer.org> Hi, i got depency problem like dovecot-common (<< 2:2.1.alpha1-0~auto+47) dpkg -i dovecot-common_2.1.4-0~auto+3_all.de after all dove seems to run without install dovecot-common basicly -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From mhlavink at redhat.com Mon Apr 16 12:55:28 2012 From: mhlavink at redhat.com (Michal Hlavinka) Date: Mon, 16 Apr 2012 11:55:28 +0200 Subject: [Dovecot] dovecot and systemd In-Reply-To: <1331820329.10319.32.camel@innu> References: <4F61EFE8.1000901@redhat.com> <1331820329.10319.32.camel@innu> Message-ID: <4F8BEC90.8060504@redhat.com> On 03/15/2012 03:05 PM, Timo Sirainen wrote: > On Thu, 2012-03-15 at 14:34 +0100, Michal Hlavinka wrote: >> What exactly should happen when >> dovecot.conf does not match dovecot.socket configuration? > > Dovecot's systemd code was written by one of you Redhat guys. I had some > similar thoughts when I applied the patch, but didn't really know what > to do about it, so I didn't do anything. So: I don't know. Maybe some > other project has solved this somehow already? Seems other projects did not solve this yet. Most projects provide just one service. In that case, they serve any connection they get, because they know what to do. I've discussed this with systemd upstream and we've decided that the best solution is to log error message and close that socket. > Dovecot anyway needs its own internal UNIX listeners. Should all > internal inet listeners be disabled? Could Dovecot somehow talk to > systemd and ask what listeners it's using for Dovecot and log warnings > if they don't match? I don't understand this question completely. What it does already is that during start up, when dovecot creates sockets, it checks what sockets already exist and creates only the missing ones. Systemd provides following functions: sd_is_fifo (3) - Check the type of a file descriptor sd_is_mq (3) - Check the type of a file descriptor sd_is_socket (3) - Check the type of a file descriptor sd_is_socket_inet (3) - Check the type of a file descriptor sd_is_socket_unix (3) - Check the type of a file descriptor sd_listen_fds (3) - Check for file descriptors passed by the init system. http://0pointer.de/public/systemd-man/sd_listen_fds.html I wrote simple patch that close the extra sockets. It's tested and works fine. You'll maybe want to move that function to different place and/or change wording of error messages. Michal -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-2.0.19-systemdfix.patch Type: text/x-patch Size: 2669 bytes Desc: not available URL: From root.kev at gmail.com Mon Apr 16 17:55:54 2012 From: root.kev at gmail.com (Root Kev) Date: Mon, 16 Apr 2012 10:55:54 -0400 Subject: [Dovecot] POP3 Dovecot Auth CPU usage 75%+ In-Reply-To: <337BF2E0-9F19-4AB4-A81C-2DCE76484662@iki.fi> References: <0B6D0FE2-2831-460E-8B2B-C255C49DEBAE@iki.fi> <337BF2E0-9F19-4AB4-A81C-2DCE76484662@iki.fi> Message-ID: I think my last email may have been bounced due to attachment size, I have put a snippet of the captures below. The CPU is still going to high percent of usage when my test mailboxes are used. An ideas on how to bring down the Auth CPU usage would be greatly appreciated! Thanks, Kevin Stace on the Auth process: epoll_wait(13, {{EPOLLIN, {u32=150109008, u64=150109008}}}, 29, 149958) = 1 gettimeofday({1334328634, 21072}, NULL) = 0 read(29, "VERSION\t1\t1\nREQUEST\t1011351553\t3"..., 1024) = 72 time(NULL) = 1334328634 writev(29, [{"USER\t1011351553\tservermailbox1\ts"..., 108}, {"\n", 1}], 2) = 109 gettimeofday({1334328634, 27993}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=149927248, u64=149927248}}}, 29, 149992) = 1 gettimeofday({1334328634, 32215}, NULL) = 0 accept(11, {sa_family=AF_FILE, NULL}, [2]) = 30 fcntl64(30, F_GETFL) = 0x2 (flags O_RDWR) fcntl64(30, F_SETFL, O_RDWR|O_NONBLOCK) = 0 gettimeofday({1334328634, 32342}, NULL) = 0 fstat64(30, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0 _llseek(30, 0, 0xbffd24c0, SEEK_CUR) = -1 ESPIPE (Illegal seek) getsockname(30, {sa_family=AF_FILE, path="/usr/local/var/run/dovecot"}, [41]) = 0 epoll_ctl(13, EPOLL_CTL_ADD, 30, {EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP, {u32=150123496, u64=150123496}}) = 0 write(30, "VERSION\t1\t1\nSPID\t2093\n", 22) = 22 gettimeofday({1334328634, 32625}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=150123496, u64=150123496}}}, 29, 1000) = 1 gettimeofday({1334328634, 32721}, NULL) = 0 read(30, "VERSION\t1\t1\n", 1024) = 12 gettimeofday({1334328634, 32792}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=150123496, u64=150123496}}}, 29, 1000) = 1 gettimeofday({1334328634, 32883}, NULL) = 0 read(30, "REQUEST\t3624009729\t3062\t16\tbe004"..., 1012) = 60 time(NULL) = 1334328634 writev(30, [{"USER\t3624009729\tservermailbox\tsy"..., 105}, {"\n", 1}], 2) = 106 gettimeofday({1334328634, 33062}, NULL) = 0 epoll_wait(13, {{EPOLLIN|EPOLLHUP, {u32=150094520, u64=150094520}}}, 29, 999) = 1 gettimeofday({1334328634, 33766}, NULL) = 0 read(28, "", 6243) = 0 epoll_ctl(13, EPOLL_CTL_DEL, 28, {0, {u32=150094520, u64=150094520}}) = 0 close(28) = 0 epoll_wait(13, {{EPOLLIN|EPOLLHUP, {u32=150109008, u64=150109008}}}, 29, -1) = 1 gettimeofday({1334328634, 40036}, NULL) = 0 read(29, "", 952) = 0 epoll_ctl(13, EPOLL_CTL_DEL, 29, {0, {u32=150109008, u64=150109008}}) = 0 close(29) = 0 gettimeofday({1334328634, 40163}, NULL) = 0 gettimeofday({1334328634, 40197}, NULL) = 0 epoll_wait(13, {{EPOLLIN|EPOLLHUP, {u32=150123496, u64=150123496}}}, 29, 1000) = 1 gettimeofday({1334328634, 44007}, NULL) = 0 read(30, "", 952) = 0 epoll_ctl(13, EPOLL_CTL_DEL, 30, {0, {u32=150123496, u64=150123496}}) = 0 close(30) = 0 gettimeofday({1334328634, 44148}, NULL) = 0 gettimeofday({1334328634, 44184}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=150065544, u64=150065544}}}, 29, 1000) = 1 gettimeofday({1334328634, 52466}, NULL) = 0 read(26, "AUTH\t1\tPLAIN\tservice=pop3\tlip=17"..., 8170) = 122 gettimeofday({1334328634, 52582}, NULL) = 0 writev(12, [{"PENALTY-GET\t172.20.20.110", 25}, {"\n", 1}], 2) = 26 gettimeofday({1334328634, 52698}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=149924840, u64=149924840}}}, 29, 992) = 1 gettimeofday({1334328634, 52760}, NULL) = 0 read(12, "0 0\n", 424) = 4 time(NULL) = 1334328634 gettimeofday({1334328634, 93200}, NULL) = 0 writev(26, [{"OK\t1\tuser=servermailbox1", 24}, {"\n", 1}], 2) = 25 read(12, 0x8f36c14, 420) = -1 EAGAIN (Resource temporarily unavailable) gettimeofday({1334328634, 93651}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=150065544, u64=150065544}}}, 29, 951) = 1 gettimeofday({1334328634, 93715}, NULL) = 0 read(26, "AUTH\t2\tPLAIN\tservice=pop3\tlip=17"..., 8048) = 118 gettimeofday({1334328634, 93808}, NULL) = 0 writev(12, [{"PENALTY-GET\t172.20.20.110", 25}, {"\n", 1}], 2) = 26 gettimeofday({1334328634, 93919}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=149924840, u64=149924840}}}, 29, 951) = 1 gettimeofday({1334328634, 93980}, NULL) = 0 read(12, "0 0\n", 420) = 4 time(NULL) = 1334328634 gettimeofday({1334328634, 133578}, NULL) = 0 writev(26, [{"OK\t2\tuser=servermailbox", 23}, {"\n", 1}], 2) = 24 read(12, 0x8f36c18, 416) = -1 EAGAIN (Resource temporarily unavailable) gettimeofday({1334328634, 133998}, NULL) = 0 epoll_wait(13, {{EPOLLIN, {u32=149927248, u64=149927248}}}, 29, 911) = 1 gettimeofday({1334328634, 134064}, NULL) = 0 accept(11, {sa_family=AF_FILE, NULL}, [2]) = 28 fcntl64(28, F_GETFL) = 0x2 (flags O_RDWR) fcntl64(28, F_SETFL, O_RDWR|O_NONBLOCK) = 0 gettimeofday({1334328634, 134200}, NULL) = 0 fstat64(28, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0 _llseek(28, 0, 0xbffd24c0, SEEK_CUR) = -1 ESPIPE (Illegal seek) getsockname(28, {sa_family=AF_FILE, path="/usr/local/var/run/dovecot"}, [41]) = 0 epoll_ctl(13, EPOLL_CTL_ADD, 28, {EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP, {u32=150094520, u64=150094520}}) = 0 Debug on the auth process: Apr 13 11:01:58 devsmtp dovecot: auth: Debug: cache(qamailbox,172.20.20.222): miss Apr 13 11:01:58 devsmtp dovecot: auth-worker(3432): Debug: Loading modules from directory: /usr/local/lib/dovecot/auth Apr 13 11:01:58 devsmtp dovecot: auth-worker(3432): Debug: shadow(qamailbox,172.20.20.222): lookup Apr 13 11:01:58 devsmtp dovecot: auth: Debug: client out: OK#0111#011user=qamailbox Apr 13 11:01:58 devsmtp dovecot: auth: Debug: master in: REQUEST#0111220673537#0113397#0111#0115609887f745a84903ce3699d23e7b886 Apr 13 11:01:58 devsmtp dovecot: auth: Debug: userdb-cache(qamailbox,172.20.20.222): miss Apr 13 11:01:58 devsmtp dovecot: auth-worker(3432): Debug: passwd(qamailbox,172.20.20.222): lookup Apr 13 11:01:58 devsmtp dovecot: auth: Debug: master out: USER#0111220673537#011qamailbox#011system_groups_user=qamailbox#011uid=1002#011gid=1002#011home=/home/qamailbox Apr 13 11:01:58 devsmtp dovecot: pop3-login: Login: user=, method=PLAIN, rip=172.20.20.222, lip=172.20.20.222, mpid=3433, secured Apr 13 11:02:05 devsmtp dovecot: pop3(qamailbox): Disconnected: Logged out top=0/0, retr=1/15637, del=0/50, size=779917 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: auth client connected (pid=3397) Apr 13 11:05:31 devsmtp dovecot: auth: Debug: client in: AUTH#0112#011PLAIN#011service=pop3#011lip=172.20.20.222#011rip=172.20.20.110#011lport=110#011rport=53254#011resp=AHNlcnZlcm1haWxib3gxADEyMzQ1Ng== Apr 13 11:05:31 devsmtp dovecot: auth: Debug: cache(servermailbox1,172.20.20.110): miss Apr 13 11:05:31 devsmtp dovecot: auth-worker(3459): Debug: Loading modules from directory: /usr/local/lib/dovecot/auth Apr 13 11:05:31 devsmtp dovecot: auth-worker(3459): Debug: shadow(servermailbox1,172.20.20.110): lookup Apr 13 11:05:31 devsmtp dovecot: auth: Debug: client out: OK#0112#011user=servermailbox1 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: master in: REQUEST#011358219777#0113397#0112#01132ecf6b93729a90ece98bbb643446ea6 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: userdb-cache(servermailbox1,172.20.20.110): miss Apr 13 11:05:31 devsmtp dovecot: auth-worker(3459): Debug: passwd(servermailbox1,172.20.20.110): lookup Apr 13 11:05:31 devsmtp dovecot: auth: Debug: master out: USER#011358219777#011servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:31 devsmtp dovecot: pop3-login: Login: user=, method=PLAIN, rip=172.20.20.110, lip=172.20.20.222, mpid=3460 Apr 13 11:05:31 devsmtp dovecot: pop3(servermailbox1): Disconnected: Logged out top=5/6281, retr=5/77906, del=0/50, size=778637 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: client in: AUTH#0113#011PLAIN#011service=pop3#011lip=172.20.20.222#011rip=172.20.20.110#011lport=110#011rport=53255#011resp=AHNlcnZlcm1haWxib3gxADEyMzQ1Ng== Apr 13 11:05:31 devsmtp dovecot: auth: Debug: cache(servermailbox1,172.20.20.110): hit: {CRYPT}$6$bpFXtlOP$zczdubFhGyRjJA0PD9lr/QbWAdweg3jThwkxkfCEGgxPxFpualm1ea.8rECmM1UZ0YuTNKWpiGPHwLhys1luy0#011user=servermailbox1#011user=servermailbox1 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: client out: OK#0113#011user=servermailbox1 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: master in: REQUEST#0112057699329#0113397#0113#01132ecf6b93729a90ece98bbb643446ea6 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: userdb-cache(servermailbox1,172.20.20.110): hit: servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:31 devsmtp dovecot: auth: Debug: master out: USER#0112057699329#011servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:31 devsmtp dovecot: pop3-login: Login: user=, method=PLAIN, rip=172.20.20.110, lip=172.20.20.222, mpid=3461 Apr 13 11:05:31 devsmtp dovecot: pop3(servermailbox1): Disconnected: Logged out top=0/0, retr=0/0, del=0/50, size=778637 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=pop3#011lip=172.20.20.222#011rip=172.20.20.110#011lport=110#011rport=53261#011resp=AHNlcnZlcm1haWxib3gxADEyMzQ1Ng== Apr 13 11:05:33 devsmtp dovecot: auth: Debug: cache(servermailbox1,172.20.20.110): hit: {CRYPT}$6$bpFXtlOP$zczdubFhGyRjJA0PD9lr/QbWAdweg3jThwkxkfCEGgxPxFpualm1ea.8rECmM1UZ0YuTNKWpiGPHwLhys1luy0#011user=servermailbox1#011user=servermailbox1 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: client out: OK#0111#011user=servermailbox1 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: master in: REQUEST#0113081109505#0113386#0111#01146c7d95e8b31022008fef693a1ef018c Apr 13 11:05:33 devsmtp dovecot: auth: Debug: userdb-cache(servermailbox1,172.20.20.110): hit: servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: master out: USER#0113081109505#011servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:33 devsmtp dovecot: pop3-login: Login: user=, method=PLAIN, rip=172.20.20.110, lip=172.20.20.222, mpid=3462 Apr 13 11:05:33 devsmtp dovecot: pop3(servermailbox1): Disconnected: Logged out top=0/0, retr=0/0, del=1/50, size=778637 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: client in: AUTH#0114#011PLAIN#011service=pop3#011lip=172.20.20.222#011rip=172.20.20.110#011lport=110#011rport=53262#011resp=AHNlcnZlcm1haWxib3gxADEyMzQ1Ng== Apr 13 11:05:33 devsmtp dovecot: auth: Debug: cache(servermailbox1,172.20.20.110): hit: {CRYPT}$6$bpFXtlOP$zczdubFhGyRjJA0PD9lr/QbWAdweg3jThwkxkfCEGgxPxFpualm1ea.8rECmM1UZ0YuTNKWpiGPHwLhys1luy0#011user=servermailbox1#011user=servermailbox1 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: client out: OK#0114#011user=servermailbox1 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: master in: REQUEST#0112632187905#0113397#0114#01132ecf6b93729a90ece98bbb643446ea6 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: userdb-cache(servermailbox1,172.20.20.110): hit: servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: master out: USER#0112632187905#011servermailbox1#011system_groups_user=servermailbox1#011uid=1007#011gid=1007#011home=/home/servermailbox1 Apr 13 11:05:33 devsmtp dovecot: pop3-login: Login: user=, method=PLAIN, rip=172.20.20.110, lip=172.20.20.222, mpid=3463 Apr 13 11:05:33 devsmtp dovecot: pop3(servermailbox1): Disconnected: Logged out top=0/0, retr=0/0, del=0/49, size=763044 Apr 13 11:05:33 devsmtp dovecot: auth: Debug: client in: AUTH#0115#011PLAIN#011service=pop3#011lip=172.20.20.222#011rip=172.20.20.110#011lport=110#011rport=53263#011resp=AHNlcnZlcm1haWxib3gxADEyMzQ1Ng== Apr 13 11:05:33 devsmtp dovecot: auth: Debug: cache(servermailbox1,172.20.20.110): hit: {CRYPT}$6$bpFXtlOP$zczdubFhGyRjJA0PD9lr/QbWAdweg3jThwkxkfCEGgxPxFpualm1ea.8rECmM1UZ0YuTNKWpiGPHwLhys1luy0#011user=servermailbox1#011user=servermailbox1 From robert at schetterer.org Mon Apr 16 18:48:13 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 16 Apr 2012 17:48:13 +0200 Subject: [Dovecot] problems upgrade 2.0 - 2.1 virtual inbox broken / special use namespace In-Reply-To: <4F8AC473.7030307@schetterer.org> References: <4F8AC473.7030307@schetterer.org> Message-ID: <4F8C3F3D.3030103@schetterer.org> Am 15.04.2012 14:52, schrieb Robert Schetterer: > Hi Timo > > the virtual setup got broke after update to 2.1 > ( its still working under 2.0 ) on other loadbalanced server > > namespace default { > separator = / > prefix = "" > list = yes > subscriptions = yes > hidden = no > } > > namespace virtual { > prefix = "virtual/" > separator = / > location = virtual:/etc/dovecot/virtual:LAYOUT=maildir++:INDEX=~/virtual > hidden = yes > list = no > subscriptions= no > } > > namespace real { > prefix = "RealMails/" > separator = / > list = no > hidden = yes > } > > > after upgrade i have > > Error: Couldn't open INBOX: Virtual mailbox open failed because of > mailbox /RealMails/: Invalid mailbox name > > --------------------- ok i identified the problem but didnt find a working other solution +RealMails/ << not longer work in 2.1 +RealMails/* -RealMails/Trash -RealMails/Trash/* -RealMails/Sent -RealMails/Sent/* -RealMails/Templates -RealMails/Templates/* -RealMails/Drafts -RealMails/Drafts/* -RealMails/Archives -RealMails/Archives/* all > > also > > i wanted > to include > > namespace inbox { > > #mailbox name { > # auto=create will automatically create this mailbox. > # auto=subscribe will both create and subscribe to the mailbox. > #auto = no > > # Space separated list of IMAP SPECIAL-USE attributes as specified by > # RFC 6154: \All \Archive \Drafts \Flagged \Junk \Sent \Trash > #special_use = > #} > > # These mailboxes are widely used and could perhaps be created > automatically: > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Trash { > special_use = \Trash > } > > # For \Sent mailboxes there are two widely used names. We'll mark both of > # them as \Sent. User typically deletes one of them if duplicates are > created. > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > > # If you have a virtual "All messages" mailbox: > mailbox virtual/All { > special_use = \All > } > > # If you have a virtual "Flagged" mailbox: > mailbox virtual/Flagged { > special_use = \Flagged > } > } > > but i found no working namespace combination with virtual -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Mon Apr 16 22:01:26 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 16 Apr 2012 21:01:26 +0200 Subject: [Dovecot] problems upgrade 2.0 - 2.1 virtual inbox broken / special use namespace /solved In-Reply-To: <4F8C3F3D.3030103@schetterer.org> References: <4F8AC473.7030307@schetterer.org> <4F8C3F3D.3030103@schetterer.org> Message-ID: <4F8C6C86.7000707@schetterer.org> Am 16.04.2012 17:48, schrieb Robert Schetterer: > Am 15.04.2012 14:52, schrieb Robert Schetterer: >> Hi Timo >> >> the virtual setup got broke after update to 2.1 >> ( its still working under 2.0 ) on other loadbalanced server >> >> namespace default { >> separator = / >> prefix = "" >> list = yes >> subscriptions = yes >> hidden = no >> } >> >> namespace virtual { >> prefix = "virtual/" >> separator = / >> location = virtual:/etc/dovecot/virtual:LAYOUT=maildir++:INDEX=~/virtual >> hidden = yes >> list = no >> subscriptions= no >> } >> >> namespace real { >> prefix = "RealMails/" >> separator = / >> list = no >> hidden = yes >> } >> >> >> after upgrade i have >> >> Error: Couldn't open INBOX: Virtual mailbox open failed because of >> mailbox /RealMails/: Invalid mailbox name >> >> --------------------- > > ok i identified the problem > but didnt find a working other solution > > > > +RealMails/ << not longer work in 2.1 > +RealMails/* > -RealMails/Trash > -RealMails/Trash/* > -RealMails/Sent > -RealMails/Sent/* > -RealMails/Templates > -RealMails/Templates/* > -RealMails/Drafts > -RealMails/Drafts/* > -RealMails/Archives > -RealMails/Archives/* > all > > > >> >> also >> >> i wanted >> to include >> >> namespace inbox { >> >> #mailbox name { >> # auto=create will automatically create this mailbox. >> # auto=subscribe will both create and subscribe to the mailbox. >> #auto = no >> >> # Space separated list of IMAP SPECIAL-USE attributes as specified by >> # RFC 6154: \All \Archive \Drafts \Flagged \Junk \Sent \Trash >> #special_use = >> #} >> >> # These mailboxes are widely used and could perhaps be created >> automatically: >> mailbox Drafts { >> special_use = \Drafts >> } >> mailbox Junk { >> special_use = \Junk >> } >> mailbox Trash { >> special_use = \Trash >> } >> >> # For \Sent mailboxes there are two widely used names. We'll mark both of >> # them as \Sent. User typically deletes one of them if duplicates are >> created. >> mailbox Sent { >> special_use = \Sent >> } >> mailbox "Sent Messages" { >> special_use = \Sent >> } >> >> # If you have a virtual "All messages" mailbox: >> mailbox virtual/All { >> special_use = \All >> } >> >> # If you have a virtual "Flagged" mailbox: >> mailbox virtual/Flagged { >> special_use = \Flagged >> } >> } >> >> but i found no working namespace combination with virtual > > for small test this setting should work with 2.1 and special use namespace notice default namespace virtual example in the wiki must change to namespace inbox so the i.e sql lookup must change CASE '%s' WHEN 'pop3' THEN NULL ELSE 'yes' END AS 'namespace/inbox /inbox', \ CASE '%s' WHEN 'pop3' THEN 'yes' ELSE NULL END AS 'namespace/virtual/inbox', \ 10-mail.conf .. namespace inbox { prefix = separator = / list = yes } 15-mailboxes.conf as you like --------------------------------- the virtual fix at my setup is RealMails +RealMails/* -RealMails/Trash -RealMails/Trash/* -RealMails/Sent -RealMails/Sent/* -RealMails/Templates -RealMails/Templates/* -RealMails/Drafts -RealMails/Drafts/* -RealMails/Archives -RealMails/Archives/* all -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From dlie76 at yahoo.com.au Tue Apr 17 05:31:56 2012 From: dlie76 at yahoo.com.au (Daminto Lie) Date: Mon, 16 Apr 2012 19:31:56 -0700 (PDT) Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> <4F86DCA7.1060806@talex.pl> <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> Message-ID: <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> Thank you folks for your replies. I realised that the dovecot 1.2.9 that I installed via ubuntu synaptic package manager has already come with sieve plugins. After making some modifications on the dovecot.conf file, I went to download and install avelsieve 1.0.1 (stable version) onto the squirrelmail. Then, I run in squirrelmail conf.pl and added the plugin. I logged in and could see "Filter" option right on the top header row. I went to click it and nothing came up. I checked in the mail.log but did not see any errors there. Here is my snapshot of dovecot configuration # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS ext4 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap listen: *:143 ssl: no disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login verbose_proctitle: yes first_valid_uid: 1001 last_valid_uid: 600 mail_privileged_group: mail mail_location: maildir:/var/vmail/%u/Maildir mbox_write_locks: fcntl dotlock imap_client_workarounds: outlook-idle delay-newmail netscape-eoh tb-extra-mailbox-sep lda: ? postmaster_address: postmaster at ourcompany.com ? mail_plugins: sieve ? quota_full_tempfail: yes ? deliver_log_format: msgid=%m: %$ ? sendmail_path: /usr/lib/sendmail ? rejection_reason: Your message to <%t> was automatically rejected:%n%r ? auth_socket_path: /var/run/dovecot-auth-master auth default: ? mechanisms: plain login ? username_format: %Lu ? verbose: yes ? debug: yes ? debug_passwords: yes ? passdb: ??? driver: pam ? passdb: ??? driver: ldap ??? args: /etc/dovecot/dovecot-ldap.conf ? userdb: ??? driver: passwd ? userdb: ??? driver: static ??? args: uid=1001 gid=1001 home=/var/vmail/%u allow_all_users=yes ? socket: ??? type: listen ??? client: ????? path: /var/spool/postfix/private/auth ????? mode: 432 ????? user: postfix ????? group: mail ??? master: ????? path: /var/run/dovecot-auth-master ????? mode: 432 ????? user: vmail ????? group: vmail plugin: ? sieve: /var/vmail/%u/.sieve Also, I have restarted dovecot after making changes to the dovecot.conf. Any help would be very much appreciated. Thank you ? ________________________________ From: Artur Zaprza?a To: Dovecot Mailing List Cc: Daminto Lie Sent: Thursday, 12 April 2012 11:46 PM Subject: Re: [Dovecot] vacation plugins for squirrelmail Daminto Lie wrote: > Hi, > > I am afraid I have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. > > Any help would be very much appreciated. > > Thank you > I'm using Avelsieve 1.9.9 with a set of my own bugfixes: http://email.uoa.gr/avelsieve/ -- Talex Sp??ka Akcyjna z siedzib? w Poznaniu adres: ul. Karpia 27d, 61-619 Pozna? NIP 782-00-21-045 zarejestrowana w S?dzie Rejonowym Pozna? ? Nowe Miasto i Wilda w Poznaniu VIII Wydzia? Gospodarczy - KRS pod nr 000048779 kapita? zak?adowy: 3.000.092,00 PLN (w ca?o?ci wp?acony) Uwaga: Niniejsza wiadomo??, w szczeg?lno?ci jej tre?? oraz za??czniki, mo?e by? poufna. W przypadku, gdy nie jest Pan/Pani zamierzonym jej adresatem, informujemy, ?e wszelkie rozpowszechnianie, dystrybucja lub powielanie powy?szej wiadomo?ci jest zabronione. Jednocze?nie prosimy o powiadomienie nadawcy oraz niezw?oczne usuni?cie powy?szej wiadomo?ci wraz z za??cznikami. Dzi?kujemy, Talex S.A. w Poznaniu. Confidentiality Notice: This email, particularly its content and any attached files, may be confidential. If you are not an intended recipient, any disclosure, distribution and reproduction of this message is prohibited. In this case please notify the sender immediately and then delete this message and any attachments. Thank you, Talex S.A., Poznan. From dovecot-list at mohtex.net Tue Apr 17 06:25:44 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Tue, 17 Apr 2012 10:25:44 +0700 Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> <4F86DCA7.1060806@talex.pl> <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> Message-ID: <4F8CE2B8.3080801@mohtex.net> Show us your avelsieve-config in your-squirrelmail-dir/plugins/avelsieve/config/config.php Daminto Lie wrote the following on 17.04.2012 09:31: > Thank you folks for your replies. > > > I realised that the dovecot 1.2.9 that I installed via ubuntu synaptic package manager has already come with sieve plugins. After making some modifications on the dovecot.conf file, I went to download and install avelsieve 1.0.1 (stable version) onto the squirrelmail. Then, I run in squirrelmail conf.pl and added the plugin. I logged in and could see "Filter" option right on the top header row. I went to click it and nothing came up. I checked in the mail.log but did not see any errors there. > > Here is my snapshot of dovecot configuration > # 1.2.9: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS ext4 > log_timestamp: %Y-%m-%d %H:%M:%S > protocols: imap > listen: *:143 > ssl: > no > disable_plaintext_auth: no > login_dir: /var/run/dovecot/login > login_executable: /usr/lib/dovecot/imap-login > verbose_proctitle: yes > first_valid_uid: 1001 > last_valid_uid: 600 > mail_privileged_group: mail > mail_location: maildir:/var/vmail/%u/Maildir > mbox_write_locks: fcntl dotlock > imap_client_workarounds: outlook-idle delay-newmail netscape-eoh tb-extra-mailbox-sep > lda: > postmaster_address: postmaster at ourcompany.com > mail_plugins: sieve > quota_full_tempfail: yes > deliver_log_format: msgid=%m: %$ > sendmail_path: /usr/lib/sendmail > rejection_reason: Your message to<%t> was automatically rejected:%n%r > auth_socket_path: /var/run/dovecot-auth-master > auth default: > mechanisms: plain login > username_format: %Lu > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: > pam > passdb: > driver: ldap > args: /etc/dovecot/dovecot-ldap.conf > userdb: > driver: passwd > userdb: > driver: static > args: uid=1001 gid=1001 home=/var/vmail/%u allow_all_users=yes > socket: > type: listen > client: > path: /var/spool/postfix/private/auth > mode: 432 > user: postfix > group: mail > master: > path: /var/run/dovecot-auth-master > mode: 432 > user: vmail > group: vmail > plugin: > sieve: /var/vmail/%u/.sieve > > Also, I have restarted dovecot after making changes to the dovecot.conf. > > Any help would be very much appreciated. > > Thank you > > > > > ________________________________ > From: Artur Zaprza?a > To: Dovecot Mailing List > Cc: Daminto Lie > Sent: Thursday, 12 April 2012 11:46 PM > Subject: Re: [Dovecot] vacation plugins for squirrelmail > > Daminto Lie wrote: >> Hi, >> >> I am afraid I > have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. >> Any help would be very much appreciated. >> >> Thank you >> > I'm using Avelsieve 1.9.9 with a set of my own bugfixes: > http://email.uoa.gr/avelsieve/ > > From dlie76 at yahoo.com.au Tue Apr 17 09:14:57 2012 From: dlie76 at yahoo.com.au (Daminto Lie) Date: Mon, 16 Apr 2012 23:14:57 -0700 (PDT) Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <4F8CE2B8.3080801@mohtex.net> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> <4F86DCA7.1060806@talex.pl> <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> <4F8CE2B8.3080801@mohtex.net> Message-ID: <1334643297.87733.YahooMailNeo@web113417.mail.gq1.yahoo.com> Thank you for your reply. Here it is as follows if(!defined('AVELSIEVE_DEBUG')) { ??? define('AVELSIEVE_DEBUG', 0); } global $avelsieve_backend; $avelsieve_backend = 'File'; /*global $sieveport; $sieveport = 2000;*/ global $sieve_preferred_sasl_mech; $sieve_preferred_sasl_mech = 'PLAIN'; global $avelsieve_disabletls; $avelsieve_disabletls = false; global $avelsieve_file_backend_options, $data_dir, $username; $avelsieve_file_backend_options = array( ??? 'avelsieve_default_file' => "$data_dir/$username.sievesource" ); /*global $avelsieve_oldcyrus; $avelsieve_oldcyrus = true;*/ global $avelsieve_enable_envelope_auth; $avelsieve_enable_envelope_auth = true; global $avelsieve_custom_sieve_implementation; $avelsieve_custom_sieve_implementation = ''; global $avelsieve_striproot; $avelsieve_striproot = ''; global $avelsieve_hardcoded_capabilities; $avelsieve_hardcoded_capabilities = array( ??? 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' ); /*global $avelsieve_imapproxymode, $avelsieve_imapproxyserv; $avelsieve_imapproxymode = false; $avelsieve_imapproxyserv = array( ??? 'localhost' => 'imap.example.org' );*/ global $avelsieve_ldapuserdatamode; $avelsieve_ldapuserdatamode = false; /*global $avelsieve_cyrusadmins_map; $avelsieve_cyrusadmins_map = array( ??? 'cyrusimap' => 'cyrussieve' );*/ $conservative = false; $useimages = true; global $translate_return_msgs; $translate_return_msgs = false; $imagetheme = 'famfamfam'; //$imagetheme = 'bluecurve_24x24'; //$imagetheme = 'bluecurve_16x16'; global $startitems; $startitems = 3; global $maxitems; $maxitems = 10; global $headers; $headers = array( ?'From', 'To', 'Cc', 'Bcc', 'Subject', 'Reply-To', 'Sender', 'List-Id', ?'MailingList', 'Mailing-List', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', ?'Resent-From',? 'Resent-To', 'X-Mailer', 'X-Mailing-List', ?'X-Spam-Flag', 'X-Spam-Status', ?'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', ?'Return-Path', 'Received', 'Auto-Submitted', ?'X-Spam-Flag', 'X-Spam-Status','X-Spam-Tests' ?); global $notifymethods; $notifymethods = array( 'mailto', 'sms' ); //$notifymethods = false; // $disable_avelsieve_capabilities = array("notify"); global $disable_avelsieve_capabilities; $disable_avelsieve_capabilities = array(); global $avelsieveheaderlink; $avelsieveheaderlink = true; global $avelsieve_default_mode; $avelsieve_default_mode = 'terse'; global $avelsieve_enable_rules; $avelsieve_enable_rules = array(); global $avelsieve_rules_settings; $avelsieve_rules_settings = array(); foreach($avelsieve_enable_rules as $r) { ??? if(file_exists(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php')) { ??????? require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php'); ??? } else { ??????? require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.default.php'); ??? } } $avelsieve_spam_highlight_enable = false; Hope this help you to help me. Thank you ________________________________ From: Tamsy To: "dovecot at dovecot.org" Sent: Tuesday, 17 April 2012 1:25 PM Subject: Re: [Dovecot] vacation plugins for squirrelmail Show us your avelsieve-config in your-squirrelmail-dir/plugins/avelsieve/config/config.php Daminto Lie wrote the following on 17.04.2012 09:31: > Thank you folks for your replies. > > > I realised that the dovecot 1.2.9 that I installed via ubuntu synaptic package manager has already come with sieve plugins. After making some modifications on the dovecot.conf file, I went to download and install avelsieve 1.0.1 (stable version) onto the squirrelmail. Then, I run in squirrelmail conf.pl and added the plugin. I logged in and could see "Filter" option right on the top header row. I went to click it and nothing came up. I checked in the mail.log but did not see any errors there. > > Here is my snapshot of dovecot configuration > # 1.2.9: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS ext4 > log_timestamp: %Y-%m-%d %H:%M:%S > protocols: imap > listen: *:143 > ssl: >? no > disable_plaintext_auth: no > login_dir: /var/run/dovecot/login > login_executable: /usr/lib/dovecot/imap-login > verbose_proctitle: yes > first_valid_uid: 1001 > last_valid_uid: 600 > mail_privileged_group: mail > mail_location: maildir:/var/vmail/%u/Maildir > mbox_write_locks: fcntl dotlock > imap_client_workarounds: outlook-idle delay-newmail netscape-eoh tb-extra-mailbox-sep > lda: >? ? postmaster_address: postmaster at ourcompany.com >? ? mail_plugins: sieve >? ? quota_full_tempfail: yes >? ? deliver_log_format: msgid=%m: %$ >? ? sendmail_path: /usr/lib/sendmail >? ? rejection_reason: Your message to<%t>? was automatically rejected:%n%r >? ? auth_socket_path: /var/run/dovecot-auth-master > auth default: >? ? mechanisms: plain login >? ? username_format: %Lu >? ? verbose: yes >? ? debug: yes >? ? debug_passwords: yes >? ? passdb: >? ? ? driver: >? pam >? ? passdb: >? ? ? driver: ldap >? ? ? args: /etc/dovecot/dovecot-ldap.conf >? ? userdb: >? ? ? driver: passwd >? ? userdb: >? ? ? driver: static >? ? ? args: uid=1001 gid=1001 home=/var/vmail/%u allow_all_users=yes >? ? socket: >? ? ? type: listen >? ? ? client: >? ? ? ? path: /var/spool/postfix/private/auth >? ? ? ? mode: 432 >? ? ? ? user: postfix >? ? ? ? group: mail >? ? ? master: >? ? ? ? path: /var/run/dovecot-auth-master >? ? ? ? mode: 432 >? ? ? ? user: vmail >? ? ? ? group: vmail > plugin: >? ? sieve: /var/vmail/%u/.sieve > > Also, I have restarted dovecot after making changes to the dovecot.conf. > > Any help would be very much appreciated. > > Thank you > >? > > > ________________________________ >? From: Artur Zaprza?a > To: Dovecot Mailing List > Cc: Daminto Lie > Sent: Thursday, 12 April 2012 11:46 PM > Subject: Re: [Dovecot] vacation plugins for squirrelmail > > Daminto Lie wrote: >> Hi, >> >> I am afraid I >? have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. >> Any help would be very much appreciated. >> >> Thank you >> > I'm using Avelsieve 1.9.9 with a set of my own bugfixes: > http://email.uoa.gr/avelsieve/ > > From dovecot-list at mohtex.net Tue Apr 17 09:51:11 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Tue, 17 Apr 2012 13:51:11 +0700 Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <1334643297.87733.YahooMailNeo@web113417.mail.gq1.yahoo.com> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> <4F86DCA7.1060806@talex.pl> <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> <4F8CE2B8.3080801@mohtex.net> <1334643297.87733.YahooMailNeo@web113417.mail.gq1.yahoo.com> Message-ID: <4F8D12DF.9020508@mohtex.net> Hi Daminto, That looks pretty much unconfigured. No wonder nothing is coming up. I have attached the avelsieve config file we are using on one of our domains. Don't copy it blindly. At least you should configure $sieveport to the port you are using on your server for Sieve. And put your domain name where it reads YOURDOMAINNAMEHERE.com. For the arrays at $headers you should tune the filters you are using with your headers. Hope this helps (working fine with our domain and Squirrel-Mail). Rgds Tamsy Daminto Lie wrote the following on 17.04.2012 13:14: > Thank you for your reply. > > Here it is as follows > > > if(!defined('AVELSIEVE_DEBUG')) { > define('AVELSIEVE_DEBUG', 0); > } > > global $avelsieve_backend; > $avelsieve_backend = 'File'; > > /*global $sieveport; > $sieveport = 2000;*/ > > global $sieve_preferred_sasl_mech; > $sieve_preferred_sasl_mech = 'PLAIN'; > > global $avelsieve_disabletls; > $avelsieve_disabletls = false; > > global $avelsieve_file_backend_options, $data_dir, $username; > $avelsieve_file_backend_options = array( > 'avelsieve_default_file' => "$data_dir/$username.sievesource" > ); > > /*global $avelsieve_oldcyrus; > $avelsieve_oldcyrus = true;*/ > > global $avelsieve_enable_envelope_auth; > $avelsieve_enable_envelope_auth = true; > > global $avelsieve_custom_sieve_implementation; > $avelsieve_custom_sieve_implementation = ''; > > global $avelsieve_striproot; > $avelsieve_striproot = ''; > > global $avelsieve_hardcoded_capabilities; > $avelsieve_hardcoded_capabilities = array( > 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' > ); > > /*global $avelsieve_imapproxymode, $avelsieve_imapproxyserv; > $avelsieve_imapproxymode = false; > $avelsieve_imapproxyserv = array( > 'localhost' => 'imap.example.org' > );*/ > > global $avelsieve_ldapuserdatamode; > $avelsieve_ldapuserdatamode = false; > > /*global $avelsieve_cyrusadmins_map; > $avelsieve_cyrusadmins_map = array( > 'cyrusimap' => 'cyrussieve' > );*/ > > $conservative = false; > > $useimages = true; > > global $translate_return_msgs; > $translate_return_msgs = false; > > $imagetheme = 'famfamfam'; > //$imagetheme = 'bluecurve_24x24'; > //$imagetheme = 'bluecurve_16x16'; > > global $startitems; > $startitems = 3; > > global $maxitems; > $maxitems = 10; > > global $headers; > $headers = array( > 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Reply-To', 'Sender', 'List-Id', > 'MailingList', 'Mailing-List', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', > 'Resent-From', 'Resent-To', 'X-Mailer', 'X-Mailing-List', > 'X-Spam-Flag', 'X-Spam-Status', > 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', > 'Return-Path', 'Received', 'Auto-Submitted', > 'X-Spam-Flag', 'X-Spam-Status','X-Spam-Tests' > ); > > global $notifymethods; > $notifymethods = array( > 'mailto', 'sms' > ); > > //$notifymethods = false; > > // $disable_avelsieve_capabilities = array("notify"); > global $disable_avelsieve_capabilities; > $disable_avelsieve_capabilities = array(); > > global $avelsieveheaderlink; > $avelsieveheaderlink = true; > > global $avelsieve_default_mode; > $avelsieve_default_mode = 'terse'; > > global $avelsieve_enable_rules; > $avelsieve_enable_rules = array(); > > global $avelsieve_rules_settings; > $avelsieve_rules_settings = array(); > foreach($avelsieve_enable_rules as $r) { > if(file_exists(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php')) { > require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php'); > } else { > require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.default.php'); > } > } > > $avelsieve_spam_highlight_enable = false; > > > Hope this help you to help me. > > Thank you > > > > > ________________________________ > From: Tamsy > To: "dovecot at dovecot.org" > Sent: Tuesday, 17 April 2012 1:25 PM > Subject: Re: [Dovecot] vacation plugins for squirrelmail > > Show us your avelsieve-config in > your-squirrelmail-dir/plugins/avelsieve/config/config.php > > > Daminto Lie wrote the following on 17.04.2012 09:31: >> Thank you folks for your replies. >> >> >> I realised that the dovecot 1.2.9 that I installed via ubuntu synaptic package manager has already come with sieve plugins. After making some modifications on the dovecot.conf file, I went to download and install avelsieve 1.0.1 (stable version) onto the squirrelmail. Then, I run in squirrelmail conf.pl and added the plugin. I logged in and could see "Filter" option right on the top header row. I went to click it and nothing came up. I checked in the mail.log but did not see any errors there. >> >> Here is my snapshot of dovecot configuration >> # 1.2.9: /etc/dovecot/dovecot.conf >> # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS ext4 >> log_timestamp: %Y-%m-%d %H:%M:%S >> protocols: imap >> listen: *:143 >> ssl: >> no >> disable_plaintext_auth: no >> login_dir: /var/run/dovecot/login >> login_executable: /usr/lib/dovecot/imap-login >> verbose_proctitle: yes >> first_valid_uid: 1001 >> last_valid_uid: 600 >> mail_privileged_group: mail >> mail_location: maildir:/var/vmail/%u/Maildir >> mbox_write_locks: fcntl dotlock >> imap_client_workarounds: outlook-idle delay-newmail netscape-eoh tb-extra-mailbox-sep >> lda: >> postmaster_address: postmaster at ourcompany.com >> mail_plugins: sieve >> quota_full_tempfail: yes >> deliver_log_format: msgid=%m: %$ >> sendmail_path: /usr/lib/sendmail >> rejection_reason: Your message to<%t> was automatically rejected:%n%r >> auth_socket_path: /var/run/dovecot-auth-master >> auth default: >> mechanisms: plain login >> username_format: %Lu >> verbose: yes >> debug: yes >> debug_passwords: yes >> passdb: >> driver: >> pam >> passdb: >> driver: ldap >> args: /etc/dovecot/dovecot-ldap.conf >> userdb: >> driver: passwd >> userdb: >> driver: static >> args: uid=1001 gid=1001 home=/var/vmail/%u allow_all_users=yes >> socket: >> type: listen >> client: >> path: /var/spool/postfix/private/auth >> mode: 432 >> user: postfix >> group: mail >> master: >> path: /var/run/dovecot-auth-master >> mode: 432 >> user: vmail >> group: vmail >> plugin: >> sieve: /var/vmail/%u/.sieve >> >> Also, I have restarted dovecot after making changes to the dovecot.conf. >> >> Any help would be very much appreciated. >> >> Thank you >> >> >> >> >> ________________________________ >> From: Artur Zaprza?a >> To: Dovecot Mailing List >> Cc: Daminto Lie >> Sent: Thursday, 12 April 2012 11:46 PM >> Subject: Re: [Dovecot] vacation plugins for squirrelmail >> >> Daminto Lie wrote: >>> Hi, >>> >>> I am afraid I >> have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. >>> Any help would be very much appreciated. >>> >>> Thank you >>> >> I'm using Avelsieve 1.9.9 with a set of my own bugfixes: >> http://email.uoa.gr/avelsieve/ >> >> From dovecot-list at mohtex.net Tue Apr 17 10:00:23 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Tue, 17 Apr 2012 14:00:23 +0700 Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <1334643297.87733.YahooMailNeo@web113417.mail.gq1.yahoo.com> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> <4F86DCA7.1060806@talex.pl> <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> <4F8CE2B8.3080801@mohtex.net> <1334643297.87733.YahooMailNeo@web113417.mail.gq1.yahoo.com> Message-ID: <4F8D1507.7060609@mohtex.net> Seems not to be attached to my former mail. Here it comes again, inline: avelsieve config.php ############################### * @copyright 2002-2004 Alexandros Vellis * @package plugins * @subpackage avelsieve */ /** * Debug Mode. Enable this (change to 1) if you need to send a bug report, * or to 2 or 3 if you are a developer! * * Valid values: * 0 = No debugging output * 1 = Normal debugging output * 2 = Firebug-enhanced debugging output * 3 = Enhanced debugging output and no Sieve capabilities checks - enables * all UI features! */ if(!defined('AVELSIEVE_DEBUG')) { define('AVELSIEVE_DEBUG', 1); } /* ======================================================================== */ /* =================== IMAP Server / SIEVE Setup ========================= */ /* ======================================================================== */ /** @var string Backend to use. Available backends are: * 'ManageSieve': Uses the ManageSieve protocol. e.g. Cyrus * 'File': Writes files straight to disk. e.g. Exim4, Dovecot LDA. */ global $avelsieve_backend; $avelsieve_backend = 'ManageSieve'; /* ======================================================================== */ /* =================== ManageSieve Backend Options ======================== */ /* ======================================================================== */ /* Port where ManageSieve servers (e.g. Cyrus IMAP's timsieved) listens to. * Default is 2000. Note that another possible port that your server might * listen to, is 4190, which is now the official (IANA-assigned) port for * Sieve. */ global $sieveport; $sieveport = 4190; /** * @var string Space separated list of preferred SASL mechanisms for the * authentication to ManageSieve server. e.g. "PLAIN DIGEST-MD5";*/ global $sieve_preferred_sasl_mech; $sieve_preferred_sasl_mech = 'PLAIN'; /** * @var boolean Disable STARTTLS for ManageSieve. You can set this to true, * if you do not wish to use encryption via TLS mechanisms (i.e. the server * is not configured properly, or this is a local connection and TLS is not * needed. * Note that STARTTLS is supported only in PHP5+. In PHP4 this option will * have no effect and STARTTLS will be disabled anyway. */ global $avelsieve_disabletls; $avelsieve_disabletls = false; /* ======================================================================== */ /* ======================= File Backend Options =========================== */ /* ======================================================================== */ global $avelsieve_file_backend_options, $data_dir, $username; $avelsieve_file_backend_options = array( 'avelsieve_default_file' => "$data_dir/$username.sievesource" ); /* ======================================================================== */ /* ====== Implementation- and Server-Specific Options ==================== */ /* ======================================================================== */ /* In Cyrus 2.3+, the notification action is a bit more complex than the * others. The oldcyrus variable is for supporting the partially implemented * notify extension implementation of Cyrus < 2.3. If you have Cyrus < 2.3, * just set this to true. * * This only changes the informational / help text displayed in avelsieve. * * Cyrus < 2.3 : $from$, $env-from$, $subject$ * Cyrus 2.3+ : $from$, $env-from$, $subject$, $text$, $text[n]$ */ global $avelsieve_oldcyrus; $avelsieve_oldcyrus = true; /* If you have Cyrus with an lmtpd that can understand the "auth" argument to * the :envelope test as the SMTP/LMTP auth, or any other Sieve implementation, * then you can enable this to provide this functionality to the user. * * This was not clarified in the base spec of RFC 3028. It will be done * correctly in a new version of Cyrus, based on a new draft / spec. */ global $avelsieve_enable_envelope_auth; $avelsieve_enable_envelope_auth = true; /* Some Implementations of Sieve need certain things in order to operate * correctly. If you use any of the following server implementations, you MUST * set this variable to the corresponding value for the filtering to work * correctly. * Valid values are: * - Any RFC3028-mostly-compatible implementation: '' (empty). * - Exim MTA: 'exim' * - MFL (as supported by mvmf): 'mfl' */ global $avelsieve_custom_sieve_implementation; $avelsieve_custom_sieve_implementation = ''; /* For delivery agents that don't know how to handle some mailbox prefixes, you * can enable this option. Example for Dovecot LDA: 'INBOX.'; */ global $avelsieve_striproot; $avelsieve_striproot = 'INBOX.'; /* If the backend does not support capabilities reporting, such as the File * Backend, then you should define which capabilities are used by the server * implementation. * * The following are the capabilities supported by Exim4 as of Exim version * 4.60, according to README.SIEVE. You can change them if a new version of * Exim provides more functionality: * 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' * * The following are the capabilities that are suported by MFM, according to: * http://www.mvmf.org/mfl/language.shtml#sieve * 'envelope', 'fileinto', 'reject', 'relational', 'subaddress', 'regex', * 'editheader', 'copy', 'vacation', 'comparator-i;ascii-casemap', * 'comparator-i;octet' * * The following capabilities have been reported to work with Dovecot LDA: * 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric', * 'imapflags', 'subaddress','relational','regex' * */ /* global $avelsieve_hardcoded_capabilities; * $avelsieve_hardcoded_capabilities = array( * 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' * ); */ global $avelsieve_hardcoded_capabilities; $avelsieve_hardcoded_capabilities = array( 'fileinto', 'reject', 'envelope', 'encoded-character', 'vacation', 'subaddress', 'comparator-i;ascii-numeric', 'relational', 'regex', 'imap4flags', 'copy', 'include', 'variables', 'body', 'enotify', 'environment', 'mailbox', 'date' ); /** @var boolean Enable ImapProxy mode. * If you use imapproxy, because imapproxy cannot understand and proxy the * SIEVE protocol, you must connect to the SIEVE daemon (usually on the IMAP * server) itself. So you need to set $imapproxymode to true, and define a * mapping, from the imapproxy host (usually localhost) to your real IMAP * server (usually the same that is defined on Imapproxy's configuration). * * This will not work if you use a perdition-style proxy, where different users * go to different IMAP servers; it applies mostly to people running imapproxy * for speed and want a quick hack. */ global $avelsieve_imapproxymode, $avelsieve_imapproxyserv; $avelsieve_imapproxymode = false; $avelsieve_imapproxyserv = array( 'localhost' => 'YOURDOMAINNAMEHERE.com' ); /** @var boolean Ldapuserdata mode: Gets user's email addresses (including * mailAlternate & mailAuthorized) from LDAP Prefs Backend plugin's cache */ global $avelsieve_ldapuserdatamode; $avelsieve_ldapuserdatamode = false; /** @var array Map of cyrus administrator users, for proxy authentication */ global $avelsieve_cyrusadmins_map; $avelsieve_cyrusadmins_map = array( 'cyrusimap' => 'cyrussieve' ); /* ======================================================================== */ /* =============== Avelsieve Interface / Behavior Setup ================== */ /* ======================================================================== */ /* Be conservative to our updates on the SIEVE server? If true, a button * entitled "Save Changes" will appear, which will give the user the * functionality to register her changes. 'false' is recommended. */ $conservative = false; /* Use images for the move up / down, delete rule buttons and STOP? */ $useimages = true; /* Translate the messages returned by the "Reject" and "Vacation" actions? The * default behaviour since 0.9 is not to translate them. Change to true if in * an intranet environment or in a same-language environment. */ global $translate_return_msgs; $translate_return_msgs = false; /* Theme to use for the images. A directory with the same name must exist under * plugins/avelsieve/$imagetheme, that contains the files: up.png, down.png, * del.png, dup.png, edit.png, top.png, bottom.png. */ $imagetheme = 'famfamfam'; //$imagetheme = 'bluecurve_24x24'; //$imagetheme = 'bluecurve_16x16'; /* Number of items to display _initially_, when displaying the header match * rule */ global $startitems; $startitems = 3; /* Maximum number of items to allow in one header match rule. */ global $maxitems; $maxitems = 10; /* Headers to display in listbox widget, when adding a new header rule. */ /* global $headers; * $headers = array( * 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Reply-To', 'Sender', 'List-Id', * 'MailingList', 'Mailing-List', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', * 'Resent-From', 'Resent-To', 'X-Mailer', 'X-Mailing-List', * 'X-Spam-Flag', 'X-Spam-Status', * 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', * 'Return-Path', 'Received', 'Auto-Submitted', * 'X-Spam-Flag', 'X-Spam-Status','X-Spam-Tests' * ); */ global $headers; $headers = array( 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Sender', 'Reply-To', 'List-Id', 'MailingList', 'Mailing-List', 'X-DKIM-Authentication-Results', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', 'Resent-From', 'Resent-To', 'X-Mailer', 'X-Mailing-List', 'X-SpamTest-Status-Extended', 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', 'Return-Path', 'Received', 'Auto-Submitted', ); /* Available :method's for the :notify extension (if applicable) */ global $notifymethods; $notifymethods = array( 'mailto', 'sms' ); /* use the value "false" if you want to provide a simple input box so that * users can edit the method themselves : */ //$notifymethods = false; /* Capabilities to disable. If you would like to force avelsieve not to display * certain features, even though there _is_ a capability for them in Sieve * server, you should specify these here. For instance, if you would * like to disable the notify extension, even though ManageSieve advertises it, * you should add 'notify' in this array: $force_disable_avelsieve_capability = * array("notify");. This will still leave the defined feature on, and if the * user can upload her own scripts then she can use that feature; this option * just disables the GUI of it. Leave as-is (empty array) if you do not need * that. * * Look in $implemented_capabilities array in include/constants.inc.php for * valid values */ // $disable_avelsieve_capabilities = array("notify"); global $disable_avelsieve_capabilities; $disable_avelsieve_capabilities = array(); /* Display Filters link in the top Squirrelmail header? */ global $avelsieveheaderlink; $avelsieveheaderlink = true; /* Default rules table display mode, one of 'verbose' or 'terse' */ global $avelsieve_default_mode; $avelsieve_default_mode = 'terse'; /* ======================================================================== */ /* ========================= Custom rules Configuration =================== */ /* ======================================================================== */ /** * @var array * List of additional rules to enable. This must be the numeric id of the * rule. * * 10: Spam Rule (as existed in Avelsieve) * 11: Junk-Mail Rule (new-style SPAM Rule, only one exists per Sieve script) * 12: Global Whitelist (only one exists per Sieve script) * 13: Custom Sieve Code (allows users to enter whatever Sieve code they like - * Alpha/Experimental) * * Example: * array(11, 12); */ global $avelsieve_enable_rules; $avelsieve_enable_rules = array(); global $avelsieve_rules_settings; $avelsieve_rules_settings = array(); foreach($avelsieve_enable_rules as $r) { if(file_exists(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php')) { require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php'); } else { require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.default.php'); } } /* Please keep the following setting false; it is alpha + needs Squirrelmail * to be patched in three or four places. */ $avelsieve_spam_highlight_enable = false; ############################### Hi Daminto, That looks pretty much unconfigured. No wonder nothing is coming up. I have attached the avelsieve config file we are using on one of our domains. Don't copy it blindly. At least you should configure $sieveport to the port you are using on your server for Sieve. And put your domain name where it reads YOURDOMAINNAMEHERE.com. For the arrays at $headers you should tune the filters you are using with your headers. Hope this helps (working fine with our domain and Squirrel-Mail). Rgds Tamsy Daminto Lie wrote the following on 17.04.2012 13:14: > Thank you for your reply. > > Here it is as follows > > > if(!defined('AVELSIEVE_DEBUG')) { > define('AVELSIEVE_DEBUG', 0); > } > > global $avelsieve_backend; > $avelsieve_backend = 'File'; > > /*global $sieveport; > $sieveport = 2000;*/ > > global $sieve_preferred_sasl_mech; > $sieve_preferred_sasl_mech = 'PLAIN'; > > global $avelsieve_disabletls; > $avelsieve_disabletls = false; > > global $avelsieve_file_backend_options, $data_dir, $username; > $avelsieve_file_backend_options = array( > 'avelsieve_default_file' => "$data_dir/$username.sievesource" > ); > > /*global $avelsieve_oldcyrus; > $avelsieve_oldcyrus = true;*/ > > global $avelsieve_enable_envelope_auth; > $avelsieve_enable_envelope_auth = true; > > global $avelsieve_custom_sieve_implementation; > $avelsieve_custom_sieve_implementation = ''; > > global $avelsieve_striproot; > $avelsieve_striproot = ''; > > global $avelsieve_hardcoded_capabilities; > $avelsieve_hardcoded_capabilities = array( > 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' > ); > > /*global $avelsieve_imapproxymode, $avelsieve_imapproxyserv; > $avelsieve_imapproxymode = false; > $avelsieve_imapproxyserv = array( > 'localhost' => 'imap.example.org' > );*/ > > global $avelsieve_ldapuserdatamode; > $avelsieve_ldapuserdatamode = false; > > /*global $avelsieve_cyrusadmins_map; > $avelsieve_cyrusadmins_map = array( > 'cyrusimap' => 'cyrussieve' > );*/ > > $conservative = false; > > $useimages = true; > > global $translate_return_msgs; > $translate_return_msgs = false; > > $imagetheme = 'famfamfam'; > //$imagetheme = 'bluecurve_24x24'; > //$imagetheme = 'bluecurve_16x16'; > > global $startitems; > $startitems = 3; > > global $maxitems; > $maxitems = 10; > > global $headers; > $headers = array( > 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Reply-To', 'Sender', 'List-Id', > 'MailingList', 'Mailing-List', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', > 'Resent-From', 'Resent-To', 'X-Mailer', 'X-Mailing-List', > 'X-Spam-Flag', 'X-Spam-Status', > 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', > 'Return-Path', 'Received', 'Auto-Submitted', > 'X-Spam-Flag', 'X-Spam-Status','X-Spam-Tests' > ); > > global $notifymethods; > $notifymethods = array( > 'mailto', 'sms' > ); > > //$notifymethods = false; > > // $disable_avelsieve_capabilities = array("notify"); > global $disable_avelsieve_capabilities; > $disable_avelsieve_capabilities = array(); > > global $avelsieveheaderlink; > $avelsieveheaderlink = true; > > global $avelsieve_default_mode; > $avelsieve_default_mode = 'terse'; > > global $avelsieve_enable_rules; > $avelsieve_enable_rules = array(); > > global $avelsieve_rules_settings; > $avelsieve_rules_settings = array(); > foreach($avelsieve_enable_rules as $r) { > if(file_exists(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php')) { > require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php'); > } else { > require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.default.php'); > } > } > > $avelsieve_spam_highlight_enable = false; > > > Hope this help you to help me. > > Thank you > > > > > ________________________________ > From: Tamsy > To: "dovecot at dovecot.org" > Sent: Tuesday, 17 April 2012 1:25 PM > Subject: Re: [Dovecot] vacation plugins for squirrelmail > > Show us your avelsieve-config in > your-squirrelmail-dir/plugins/avelsieve/config/config.php > > > Daminto Lie wrote the following on 17.04.2012 09:31: >> Thank you folks for your replies. >> >> >> I realised that the dovecot 1.2.9 that I installed via ubuntu synaptic package manager has already come with sieve plugins. After making some modifications on the dovecot.conf file, I went to download and install avelsieve 1.0.1 (stable version) onto the squirrelmail. Then, I run in squirrelmail conf.pl and added the plugin. I logged in and could see "Filter" option right on the top header row. I went to click it and nothing came up. I checked in the mail.log but did not see any errors there. >> >> Here is my snapshot of dovecot configuration >> # 1.2.9: /etc/dovecot/dovecot.conf >> # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS ext4 >> log_timestamp: %Y-%m-%d %H:%M:%S >> protocols: imap >> listen: *:143 >> ssl: >> no >> disable_plaintext_auth: no >> login_dir: /var/run/dovecot/login >> login_executable: /usr/lib/dovecot/imap-login >> verbose_proctitle: yes >> first_valid_uid: 1001 >> last_valid_uid: 600 >> mail_privileged_group: mail >> mail_location: maildir:/var/vmail/%u/Maildir >> mbox_write_locks: fcntl dotlock >> imap_client_workarounds: outlook-idle delay-newmail netscape-eoh tb-extra-mailbox-sep >> lda: >> postmaster_address: postmaster at ourcompany.com >> mail_plugins: sieve >> quota_full_tempfail: yes >> deliver_log_format: msgid=%m: %$ >> sendmail_path: /usr/lib/sendmail >> rejection_reason: Your message to<%t> was automatically rejected:%n%r >> auth_socket_path: /var/run/dovecot-auth-master >> auth default: >> mechanisms: plain login >> username_format: %Lu >> verbose: yes >> debug: yes >> debug_passwords: yes >> passdb: >> driver: >> pam >> passdb: >> driver: ldap >> args: /etc/dovecot/dovecot-ldap.conf >> userdb: >> driver: passwd >> userdb: >> driver: static >> args: uid=1001 gid=1001 home=/var/vmail/%u allow_all_users=yes >> socket: >> type: listen >> client: >> path: /var/spool/postfix/private/auth >> mode: 432 >> user: postfix >> group: mail >> master: >> path: /var/run/dovecot-auth-master >> mode: 432 >> user: vmail >> group: vmail >> plugin: >> sieve: /var/vmail/%u/.sieve >> >> Also, I have restarted dovecot after making changes to the dovecot.conf. >> >> Any help would be very much appreciated. >> >> Thank you >> >> >> >> >> ________________________________ >> From: Artur Zaprza?a >> To: Dovecot Mailing List >> Cc: Daminto Lie >> Sent: Thursday, 12 April 2012 11:46 PM >> Subject: Re: [Dovecot] vacation plugins for squirrelmail >> >> Daminto Lie wrote: >>> Hi, >>> >>> I am afraid I >> have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. >>> Any help would be very much appreciated. >>> >>> Thank you >>> >> I'm using Avelsieve 1.9.9 with a set of my own bugfixes: >> http://email.uoa.gr/avelsieve/ >> >> From f.bonnet at esiee.fr Tue Apr 17 10:54:35 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Tue, 17 Apr 2012 09:54:35 +0200 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) Message-ID: <4F8D21BB.2030407@esiee.fr> Hello I need some feedbacks advices of experienced admins I will have to setup in few monthes an email system for approx 50K "intensives" users. The only mandatory thing will be I must use HP proliant servers The operating system will be FreeBSD or Linux Thank you for any advices From dchenusa at yahoo.com Tue Apr 17 12:16:22 2012 From: dchenusa at yahoo.com (D Chen) Date: Tue, 17 Apr 2012 02:16:22 -0700 (PDT) Subject: [Dovecot] Dovecot and Outlook Express "plaintext authentication not allowed..." Message-ID: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> Server: Ubuntu 11.10 server with postfix (MTA), dovecot (MDA) Mail client: Outlook Express in Windows XP with IMAP on port 143, SMTP on port 25 When tried to set up a new Outlook Express mail acct, the following error encountered: ? ? ?Your IMAP server wishes to alert you to the following: ? ? ?plaintext authentication not allowed without SSL/TLS, but your client did it anyway. ?If ? ? ?anyone was listening, the password was exposed. When asked "would you like to download folders from the mail server you added?" then got the following error: The server has rejected your login. ?Please verify that your user name and password are correct. I re-entered several time with the correct username and password, but still responded the same error. Any insights would be greatly appreciated. Here is the message from the /var/log/mail.log: Apr 17 01:54:17 server dovecot: imap-login: Disconnected (tried to use disabled plaintext auth): rip= 192.168.20.51, lip=192.168.20.100Apr 17 01:54:28 server dovecot: imap-login: Disconnected (tried to use disabled plaintext auth): rip= 192.168.20.51, lip=192.168.20.100 What might be wrong in my dovecot.conf file. ?Below is my dovecot.conf file: # 2.0.13: /etc/dovecot/dovecot.conf # OS: Linux 3.0.0-16-server x86_64 Ubuntu 11.10? log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:~/Maildir mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date passdb { ? driver = pam } passdb { ? driver = pam } plugin { ? sieve = ~/.dovecot.sieve ? sieve_dir = ~/sieve } protocols = imap pop3 sieve service auth { ? unix_listener /var/spool/postfix/private/auth-client { ? ? group = postfix ? ? mode = 0660 ? ? user = postfix ? } ? unix_listener /var/spool/postfix/private/dovecot-auth { ? ? group = postfix ? ? mode = 0660 ? ? user = postfix ? } ? user = root } ssl_cert = was automatically rejected:%n%r } From CMarcus at Media-Brokers.com Tue Apr 17 13:26:37 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 17 Apr 2012 06:26:37 -0400 Subject: [Dovecot] Dovecot and Outlook Express "plaintext authentication not allowed..." In-Reply-To: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> References: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> Message-ID: <4F8D455D.9060805@Media-Brokers.com> On 2012-04-17 5:16 AM, D Chen wrote: > When tried to set up a new Outlook Express mail acct, the following error encountered: > > Your IMAP server wishes to alert you to the following: > plaintext authentication not allowed without SSL/TLS, but your client did it anyway. Ummm... hello? McFly? ;) You need to configure SSL on Outlook Express (why on gods green earth you are using that is beyond me), as the error plainly says. Until you do this it doesn't matter what you enter for a username/password. Also, you should be using SSL on port 993, because I'm fairly certain that Outlook Express doesn't support STARTTLS on port 143. And off topic but in the same vein, for postfix/SMTP relay you should be using the submission port (STARTTLS on port 587) for most clients, but I think Outlook Express again doesn't work with it, so you will need to use the deprecated SSL on port 465 (these two need to be enabled in postfix/master.cf - just uncomment the examples). -- Best regards, Charles From blevi.linux at gmail.com Tue Apr 17 13:47:26 2012 From: blevi.linux at gmail.com (Birta Levente) Date: Tue, 17 Apr 2012 13:47:26 +0300 Subject: [Dovecot] Dovecot and Outlook Express "plaintext authentication not allowed..." In-Reply-To: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> References: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> Message-ID: <4F8D4A3E.8000401@gmail.com> On 17/04/2012 12:16, D Chen wrote: > Server: Ubuntu 11.10 server with postfix (MTA), dovecot (MDA) > Mail client: Outlook Express in Windows XP with IMAP on port 143, SMTP on port 25 > > When tried to set up a new Outlook Express mail acct, the following error encountered: > > Your IMAP server wishes to alert you to the following: > plaintext authentication not allowed without SSL/TLS, but your client did it anyway. If > anyone was listening, the password was exposed. > > When asked "would you like to download folders from the mail server you added?" then got the > following error: > > The server has rejected your login. Please verify that your user name and password are correct. > > I re-entered several time with the correct username and password, but still responded the same error. > > Any insights would be greatly appreciated. > > > Here is the message from the /var/log/mail.log: > > > Apr 17 01:54:17 server dovecot: imap-login: Disconnected (tried to use disabled plaintext auth): rip= > 192.168.20.51, lip=192.168.20.100Apr 17 01:54:28 server dovecot: imap-login: Disconnected (tried to use disabled plaintext auth): rip= > 192.168.20.51, lip=192.168.20.100 > > What might be wrong in my dovecot.conf file. Below is my dovecot.conf file: > > # 2.0.13: /etc/dovecot/dovecot.conf > # OS: Linux 3.0.0-16-server x86_64 Ubuntu 11.10 > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_location = maildir:~/Maildir > mail_privileged_group = mail > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date > passdb { > driver = pam > } > passdb { > driver = pam > } > plugin { > sieve = ~/.dovecot.sieve > sieve_dir = ~/sieve > } > protocols = imap pop3 sieve > service auth { > unix_listener /var/spool/postfix/private/auth-client { > group = postfix > mode = 0660 > user = postfix > } > unix_listener /var/spool/postfix/private/dovecot-auth { > group = postfix > mode = 0660 > user = postfix > } > user = root > } > ssl_cert = ssl_cipher_list = ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM > ssl_key = userdb { > driver = passwd > } > userdb { > driver = passwd > } > protocol imap { > imap_client_workarounds = delay-newmail > mail_max_userip_connections = 10 > } > protocol pop3 { > mail_max_userip_connections = 10 > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > pop3_uidl_format = %08Xu%08Xv > } > protocol lda { > deliver_log_format = msgid=%m: %$ > mail_plugins = sieve > postmaster_address = postmaster > quota_full_tempfail = yes > rejection_reason = Your message to<%t> was automatically rejected:%n%r > } > Maybe auth_mechanisms = plain login cram-md5 From robert at schetterer.org Tue Apr 17 14:47:57 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 17 Apr 2012 13:47:57 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken Message-ID: <4F8D586D.3010706@schetterer.org> Hi Tim, after upgrade to 2.1 acl with shared namespace seems to be broken it looks like depend to No lookup right to mailbox: shared/ it runs perfect under 2.0.20 -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From mlopez at gtdinternet.com Tue Apr 17 14:54:15 2012 From: mlopez at gtdinternet.com (=?ISO-8859-1?Q?Mauricio_L=F3pez_Riffo?=) Date: Tue, 17 Apr 2012 08:54:15 -0300 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8D21BB.2030407@esiee.fr> References: <4F8D21BB.2030407@esiee.fr> Message-ID: <4F8D59E7.6010403@gtdinternet.com> Frank, Here we have approx. 200K users with 4000 concurrent connections (90% POP3 users) All servers in virtual environment Vmware, supermicro servers and Netapp Metrocluster storage solutions (nfs storage with 10G ethernet network) POP3 sessions take betwen 40 and 300 milisecons at connect, auth and list. All accounts lives in LDAP, CentOS 5 and exim like a mta relay. Regards El 17-04-2012 4:54, Frank Bonnet escribi?: > Hello > > I need some feedbacks advices of experienced admins > I will have to setup in few monthes an email system > for approx 50K "intensives" users. > > The only mandatory thing will be I must use HP proliant servers > > The operating system will be FreeBSD or Linux > > Thank you for any advices > > -- Mauricio L?pez Riffo Red Hat Certified Engineer 804006455319519 Administrador de Servicios Internet ?rea Ingenier?a Gtd Internet S.A. http://www.grupogtd.com/ Moneda 920, Oficina 602 - Fono : +562 4139742 From janfrode at tanso.net Tue Apr 17 15:42:41 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 17 Apr 2012 14:42:41 +0200 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8D59E7.6010403@gtdinternet.com> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> Message-ID: <20120417124241.GA23120@dibs.tanso.net> On Tue, Apr 17, 2012 at 08:54:15AM -0300, Mauricio L?pez Riffo wrote: > > Here we have approx. 200K users with 4000 concurrent connections > (90% POP3 users) How do you measure "concurrent" POP3 users? > All servers in virtual environment Vmware, > supermicro servers and Netapp Metrocluster storage solutions (nfs > storage with 10G ethernet network) POP3 sessions take betwen 40 and > 300 milisecons at connect, auth and list. All accounts lives in > LDAP, CentOS 5 and exim like a mta relay. Very interesting config. We're close to 1M accounts, GPFS cluster fs, LDAP, RHEL5/6 and postfix + dovecot director for pop/imap/lmtp, and moving from maildir to mdbox. What mailbox-format are you using? Do you have a director, or accounts sticky to a server some other way? How's the NFS performance? I've always bean weary that NFS works terribly with many small files (i.e. maildir).. What does the metrocluster give you? Is it for disaster recovery on second location, or do you have two active locations working against the same filesystem? -jf From f.bonnet at esiee.fr Tue Apr 17 16:01:31 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Tue, 17 Apr 2012 15:01:31 +0200 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8D59E7.6010403@gtdinternet.com> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> Message-ID: <4F8D69AB.5090706@esiee.fr> hello Thanks for your answer , MY problem will be IMAPS connections I dunno how much I will have but it would be possible that we'll have 4000/6000 imaps concurent connections during working hours . POP3 users will be very few Le 17/04/2012 13:54, Mauricio L?pez Riffo a ?crit : > Frank, > > Here we have approx. 200K users with 4000 concurrent connections > (90% POP3 users) All servers in virtual environment Vmware, > supermicro servers and Netapp Metrocluster storage solutions (nfs > storage with 10G ethernet network) POP3 sessions take betwen 40 and > 300 milisecons at connect, auth and list. All accounts lives in LDAP, > CentOS 5 and exim like a mta relay. > > Regards > > > > El 17-04-2012 4:54, Frank Bonnet escribi?: >> Hello >> >> I need some feedbacks advices of experienced admins >> I will have to setup in few monthes an email system >> for approx 50K "intensives" users. >> >> The only mandatory thing will be I must use HP proliant servers >> >> The operating system will be FreeBSD or Linux >> >> Thank you for any advices >> >> > From mlopez at gtdinternet.com Tue Apr 17 16:10:02 2012 From: mlopez at gtdinternet.com (=?ISO-8859-1?Q?Mauricio_L=F3pez_Riffo?=) Date: Tue, 17 Apr 2012 10:10:02 -0300 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <20120417124241.GA23120@dibs.tanso.net> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <20120417124241.GA23120@dibs.tanso.net> Message-ID: <4F8D6BAA.7060501@gtdinternet.com> Jan, >> How do you measure "concurrent" POP3 users? We use cacti for metrics like concurrent connections or pop3 delay and Zabbix for alarms. >> What mailbox-format are you using? Do you have a director, or accounts >> sticky to a server some other way? Maildir like a mailformat and currently without a director, but we testing a new environment with director to reduce number of servers (7 virtual servers with 4 Vcpu, 6gb ram) At the meantime a lvs piranha do the work of steaky connections but its not enought, for this reason will use a director. >> How's the NFS performance? I've always bean weary that NFS works terribly with many small files (i.e. maildir).. At peak hours our storage traffic about 10 thousend total ops (bewten read and write, 80/20) and cpu mark 70% of use. >> What does the metrocluster give you? Is it for disaster recovery on >> second location, or do you have two active locations working against the >>same filesystem? Our storage have two missions, first hight availability because we have two datacenter (environment N +1) and posibility of use two storages in cluster for the same filesystems en this two datacenters (Vmware and Mail storage resides in NFS filesystem) >> Very interesting config. We're close to 1M accounts, GPFS cluster fs, LDAP, >> RHEL5/6 and postfix + dovecot director for pop/imap/lmtp, and moving from >> maildir to mdbox. 1M = 1 milion ? How many servers you have? hardware? Any help o contribution, you welcome :) Regards El 17-04-2012 9:42, Jan-Frode Myklebust escribi?: > On Tue, Apr 17, 2012 at 08:54:15AM -0300, Mauricio L?pez Riffo wrote: >> Here we have approx. 200K users with 4000 concurrent connections >> (90% POP3 users) > How do you measure "concurrent" POP3 users? > >> All servers in virtual environment Vmware, >> supermicro servers and Netapp Metrocluster storage solutions (nfs >> storage with 10G ethernet network) POP3 sessions take betwen 40 and >> 300 milisecons at connect, auth and list. All accounts lives in >> LDAP, CentOS 5 and exim like a mta relay. > Very interesting config. We're close to 1M accounts, GPFS cluster fs, LDAP, > RHEL5/6 and postfix + dovecot director for pop/imap/lmtp, and moving from > maildir to mdbox. > > What mailbox-format are you using? Do you have a director, or accounts > sticky to a server some other way? > > How's the NFS performance? I've always bean weary that NFS works > terribly with many small files (i.e. maildir).. > > What does the metrocluster give you? Is it for disaster recovery on > second location, or do you have two active locations working against the > same filesystem? > > > -jf -- Mauricio L?pez Riffo Red Hat Certified Engineer 804006455319519 Administrador de Servicios Internet ?rea Ingenier?a Gtd Internet S.A. http://www.grupogtd.com/ Moneda 920, Oficina 602 - Fono : +562 4139742 From goetz.reinicke at filmakademie.de Tue Apr 17 17:21:39 2012 From: goetz.reinicke at filmakademie.de (=?ISO-8859-15?Q?G=F6tz_Reinicke?=) Date: Tue, 17 Apr 2012 16:21:39 +0200 Subject: [Dovecot] 487 imap-login processes - is this ok Message-ID: <4F8D7C73.9090700@filmakademie.de> Hi, recently I checked our mailserver for some more tuning and optimizing and noticed, that currently I do have 487 imap-login processes and I'm asking myself if that's o.k. We do have about 1.000 user in total and not everybody is currently logged in. May be this is o.k. but if not, what might I check? We run the dovecot package provided by redhat EL 5.8 dovecot-1.0.7-7.el5_7.1 Thanks for any comment or suggestion. Regards . G?tz -- G?tz Reinicke IT-Koordinator Tel. +49 7141 969 420 Fax +49 7141 969 55 420 E-Mail goetz.reinicke at filmakademie.de Filmakademie Baden-W?rttemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: J?rgen Walter MdL Staatssekret?r im Ministerium f?r Wissenschaft, Forschung und Kunst Baden-W?rttemberg Gesch?ftsf?hrer: Prof. Thomas Schadt -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5161 bytes Desc: S/MIME Kryptografische Unterschrift URL: From robert at schetterer.org Tue Apr 17 17:40:19 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 17 Apr 2012 16:40:19 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken In-Reply-To: <4F8D586D.3010706@schetterer.org> References: <4F8D586D.3010706@schetterer.org> Message-ID: <4F8D80D3.2050505@schetterer.org> Am 17.04.2012 13:47, schrieb Robert Schetterer: > Hi Tim, i should get some "o" > after upgrade to 2.1 > acl with shared namespace seems to be broken > it looks like depend to > > No lookup right to mailbox: shared/ > > it runs perfect under 2.0.20 looks its depended to bug handling doveadm acl debug shows on both versions Fatal: ACL shared dict iteration failed but dove 2.1 does handle this in another way means, results in no rights , and no list for that -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From jd.beaubien at gmail.com Tue Apr 17 19:19:49 2012 From: jd.beaubien at gmail.com (Jean-Daniel Beaubien) Date: Tue, 17 Apr 2012 12:19:49 -0400 Subject: [Dovecot] Howto reindex with solr? Message-ID: Hi, I am currently testing a 2.1 dovecot setup with fts-solr. Search speed is simply amazing. I decided to try to delete and rebuild the solr indexes, but i can't seem to rebuild them thru dovecot. At first, when telnetting to the imap server, the search initially built the index, and then subsequent searches are blazingly fast. 1- I deleted all the solr indexes with the following query: *:* 2- Flag the mailbox to be re-indexed: doveadm force-resync INBOX At this point, it doesn't work. When I telnet into the imap server, the searches always come back empty 3- I also tried a few other commands: doveadm fts rescan, doveadm index INBOX. Still no search results. Nothing seems to be appearing in the logs when I issue all the doveadm commands. What did I miss? Thanks, -JD From cor at xs4all.nl Tue Apr 17 20:57:01 2012 From: cor at xs4all.nl (Cor Bosman) Date: Tue, 17 Apr 2012 19:57:01 +0200 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8D59E7.6010403@gtdinternet.com> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> Message-ID: <20120417175701.GA28791@xs4all.nl> > Here we have approx. 200K users with 4000 concurrent connections > (90% POP3 users) All servers in virtual environment Vmware, supermicro > servers and Netapp Metrocluster storage solutions (nfs storage with 10G > ethernet network) POP3 sessions take betwen 40 and 300 milisecons at > connect, auth and list. All accounts lives in LDAP, CentOS 5 and exim > like a mta relay. Similar setup here. Maybe 15-20K concurrent connections, imap only (pop is not handled by dovecot yet) about 800K mailboxes. We have all bare metal servers, linux. Currently 35 servers handling the load easily. We could run on 1/3rd of the load probably. In front of the 35 servers are 3 directors, handling imap only, although im in the process of adding LMTP/sieve to the mix. Backend storage is NetApp Metrocluster over 2 datacenters. Cor From joe.beaubien at gmail.com Tue Apr 17 21:36:33 2012 From: joe.beaubien at gmail.com (Joe Beaubien) Date: Tue, 17 Apr 2012 14:36:33 -0400 Subject: [Dovecot] Re-indexing of fts-solr not working Message-ID: Hi, I am currently testing a 2.1 dovecot setup with fts-solr. Search speed is simply amazing. I decided to try to delete and rebuild the solr indexes, but i can't seem to rebuild them thru dovecot. At first, when telnetting to the imap server, the search initially built the index, and then subsequent searches are blazingly fast. 1- I deleted all the solr indexes with the following query: *:* 2- Flag the mailbox to be re-indexed: doveadm force-resync INBOX At this point, it doesn't work. When I telnet into the imap server, the searches always come back empty 3- I also tried a few other commands: doveadm fts rescan, doveadm index INBOX. Still no search results. Nothing seems to be appearing in the logs when I issue all the doveadm commands. What did I miss? Thanks, Joe From janfrode at tanso.net Tue Apr 17 23:08:20 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 17 Apr 2012 22:08:20 +0200 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8D6BAA.7060501@gtdinternet.com> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <20120417124241.GA23120@dibs.tanso.net> <4F8D6BAA.7060501@gtdinternet.com> Message-ID: <20120417200820.GA30743@dibs.tanso.net> On Tue, Apr 17, 2012 at 10:10:02AM -0300, Mauricio L?pez Riffo wrote: > > 1M = 1 milion ? 976508 to be exact :-) but it's very much a useless number. Lots and lots of these are inactive. A better number is probably that we're seeing about 80 logins/second for the last hour.. (just checked now, not sure if this is the most busy hour or not). > How many servers you have? hardware? 7 backend dovecot servers (two IBM x336, three x346 and two x3550, with a 8 GB for the x336/x346 and 16GB memory memory for the x3550's). 2 frontend dovecot directors (IBM x3550). None of these are really very busy, so we could probably reduce the number of backends a bit if we wanted. Our struggle is the number of iops we're able to get from the backend storage (IBM DS4800), mostly a problem when we have storms of incoming marketing messages in addition to the pop/imap traffic. -jf From dchenusa at yahoo.com Tue Apr 17 23:41:20 2012 From: dchenusa at yahoo.com (D Chen) Date: Tue, 17 Apr 2012 13:41:20 -0700 (PDT) Subject: [Dovecot] Dovecot and Outlook Express "plaintext authentication not allowed..." In-Reply-To: <4F8D455D.9060805@Media-Brokers.com> References: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> <4F8D455D.9060805@Media-Brokers.com> Message-ID: <1334695280.2248.YahooMailNeo@web161606.mail.bf1.yahoo.com> Hi Charles, Thanks for the response. Per your suggestion, in Outlook Express I changed to use SSL on port 993 for IMAP, and port 465 for SMTP,?the creation of the new account succeeded without error! ?However, when tried to send to my yahoo mail or gmail acct, ?got the following error: (apparently failed to send!) "The message could not be sent because one of the recipients was rejected by the server. ?The rejected email address was 'dchenusa at yahoo.com.' ?Subject: 'test out to yahoo mail' Account:'dchen192.168.20.100' Server: 192.168.20.100 Protocol: SMTP, server response: 554 5.7.1 : client host rejected access denied, port: 465, Secured [SSL]: Yes, server Error: 554, Error number: 0x800CCC79" PS. in the same client PC, using Thunderbird client, IMAP and SMTP configured with either with SSL or without SSL, both outgoing mails were sent successfully! Any thought? Thx again, Dennis ________________________________ From: Charles Marcus To: dovecot at dovecot.org Sent: Tuesday, April 17, 2012 3:26 AM Subject: Re: [Dovecot] Dovecot and Outlook Express "plaintext authentication not allowed..." On 2012-04-17 5:16 AM, D Chen wrote: > When tried to set up a new Outlook Express mail acct, the following error encountered: > >? ? ? Your IMAP server wishes to alert you to the following: >? ? ? plaintext authentication not allowed without SSL/TLS, but your client did it anyway. Ummm... hello? McFly? ;) You need to configure SSL on Outlook Express (why on gods green earth you are using that is beyond me), as the error plainly says. Until you do this it doesn't matter what you enter for a username/password. Also, you should be using SSL on port 993, because I'm fairly certain that Outlook Express doesn't support STARTTLS on port 143. And off topic but in the same vein, for postfix/SMTP relay you should be using the submission port (STARTTLS on port 587) for most clients, but I think Outlook Express again doesn't work with it, so you will need to use the deprecated SSL on port 465 (these two need to be enabled in postfix/master.cf - just uncomment the examples). -- Best regards, Charles From jerry at seibercom.net Wed Apr 18 00:27:18 2012 From: jerry at seibercom.net (Jerry) Date: Tue, 17 Apr 2012 17:27:18 -0400 Subject: [Dovecot] Dovecot and Outlook Express "plaintext authentication not allowed..." In-Reply-To: <1334695280.2248.YahooMailNeo@web161606.mail.bf1.yahoo.com> References: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> <4F8D455D.9060805@Media-Brokers.com> <1334695280.2248.YahooMailNeo@web161606.mail.bf1.yahoo.com> Message-ID: <20120417172718.3af8127d@scorpio> On Tue, 17 Apr 2012 13:41:20 -0700 (PDT) D Chen articulated: > Per your suggestion, in Outlook Express I changed to use SSL on port > 993 for IMAP, and port 465 for SMTP,?the creation of the new account > succeeded without error! ?However, when tried to send to my yahoo > mail or gmail acct, ?got the following error: (apparently failed to > send!) > > "The message could not be sent because one of the recipients was > rejected by the server. ?The rejected email address was > 'dchenusa at yahoo.com.' ?Subject: 'test out to yahoo mail' > Account:'dchen192.168.20.100' Server: 192.168.20.100 > Protocol: SMTP, server response: 554 5.7.1 > : client host rejected access denied, port: > 465, Secured [SSL]: Yes, server Error: 554, Error number: 0x800CCC79" > > PS. in the same client PC, using Thunderbird client, IMAP and SMTP > configured with either with SSL or without SSL, both outgoing mails > were sent successfully! > > Any thought? Well, my first thought is that you should lost the tendency to "Top Post". Second, why are you screwing around with the SMTP settings? Dovecot could not care less about them. Third, Outlook Express has been abandoned years ago. Why are you still using it? If you really need it, try Windows Live Essentials . I know users who have it working with Dovecot just fine. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From kgc at corp.sonic.net Wed Apr 18 01:29:17 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Tue, 17 Apr 2012 15:29:17 -0700 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> Message-ID: <4F8DEEBD.9080206@corp.sonic.net> On 03/30/12 07:35, Timo Sirainen wrote: > IP stickiness isn't enough if user uses more than one IMAP client, which is pretty common nowadays. And doesn't help at all with LDA. Timo, having the LDA flow through the director with maildir and NFS is only an issue if you want to use the dovecot delivery agent since it manipulates the indexes on delivery, right? This shouldn't be a problem if you are running a completely distinct delivery cluster running a LDA like procmail. -K From noel.butler at ausics.net Wed Apr 18 05:24:29 2012 From: noel.butler at ausics.net (Noel Butler) Date: Wed, 18 Apr 2012 12:24:29 +1000 Subject: [Dovecot] 487 imap-login processes - is this ok In-Reply-To: <4F8D7C73.9090700@filmakademie.de> References: <4F8D7C73.9090700@filmakademie.de> Message-ID: <1334715869.4078.3.camel@tardis> Older versions of dovecot (all of v1) can under some circumstances screwup and leave hung login processes, this more often than not occurs if you have network issues, requiring a restart of dovecot to clear them. Timo has apparently, IIRC, worked around this for v2.x and it should not happen. On Tue, 2012-04-17 at 16:21 +0200, G?tz Reinicke wrote: > Hi, > > recently I checked our mailserver for some more tuning and optimizing > and noticed, that currently I do have 487 imap-login processes and I'm > asking myself if that's o.k. > > We do have about 1.000 user in total and not everybody is currently > logged in. > > May be this is o.k. but if not, what might I check? > > We run the dovecot package provided by redhat EL 5.8 dovecot-1.0.7-7.el5_7.1 > > Thanks for any comment or suggestion. > > Regards . G?tz -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From dlie76 at yahoo.com.au Wed Apr 18 04:12:49 2012 From: dlie76 at yahoo.com.au (Daminto Lie) Date: Tue, 17 Apr 2012 18:12:49 -0700 (PDT) Subject: [Dovecot] vacation plugins for squirrelmail In-Reply-To: <4F8D1507.7060609@mohtex.net> References: <1334237258.31736.YahooMailNeo@web113408.mail.gq1.yahoo.com> <4F86DCA7.1060806@talex.pl> <1334629802.53381.YahooMailNeo@web113419.mail.gq1.yahoo.com> <1334629916.64023.YahooMailNeo@web113406.mail.gq1.yahoo.com> <4F8CE2B8.3080801@mohtex.net> <1334643297.87733.YahooMailNeo@web113417.mail.gq1.yahoo.com> <4F8D1507.7060609@mohtex.net> Message-ID: <1334711569.21467.YahooMailNeo@web113409.mail.gq1.yahoo.com> Thank you Tamsy for sharing with me your config file. Initially, I thought I tried not to use managesieve and that's why I skipped that part there. But after seeing your config, I'd try to use managesieve. I would need to modify my dovecot config file first to incorporate the managesieve. And I also learnt that I need to put in my domain name. In this case, I would try this again next week as this server has not been made as a production server just yet. I thought if I could get all the things I need to work before the day of putting it online to the Internet. That would save me a lot of work. Thank you once again ________________________________ From: Tamsy To: Dovecot Mailing List Sent: Tuesday, 17 April 2012 5:00 PM Subject: Re: [Dovecot] vacation plugins for squirrelmail Seems not to be attached to my former mail. Here it comes again, inline: avelsieve config.php ############################### * @copyright 2002-2004 Alexandros Vellis * @package plugins * @subpackage avelsieve */ /** * Debug Mode. Enable this (change to 1) if you need to send a bug report, * or to 2 or 3 if you are a developer! * * Valid values: * 0 = No debugging output * 1 = Normal debugging output * 2 = Firebug-enhanced debugging output * 3 = Enhanced debugging output and no Sieve capabilities checks - enables *? ? all UI features! */ if(!defined('AVELSIEVE_DEBUG')) { ? ? define('AVELSIEVE_DEBUG', 1); } /* ======================================================================== */ /* =================== IMAP Server / SIEVE Setup? ========================= */ /* ======================================================================== */ /** @var string Backend to use. Available backends are: * 'ManageSieve': Uses the ManageSieve protocol. e.g. Cyrus * 'File': Writes files straight to disk. e.g. Exim4, Dovecot LDA. */ global $avelsieve_backend; $avelsieve_backend = 'ManageSieve'; /* ======================================================================== */ /* =================== ManageSieve Backend Options ======================== */ /* ======================================================================== */ /* Port where ManageSieve servers (e.g. Cyrus IMAP's timsieved) listens to. * Default is 2000. Note that another possible port that your server might * listen to, is 4190, which is now the official (IANA-assigned) port for * Sieve. */ global $sieveport; $sieveport = 4190; /** * @var string Space separated list of preferred SASL mechanisms for the * authentication to ManageSieve server. e.g. "PLAIN DIGEST-MD5";*/ global $sieve_preferred_sasl_mech; $sieve_preferred_sasl_mech = 'PLAIN'; /** * @var boolean Disable STARTTLS for ManageSieve. You can set this to true, * if you do not wish to use encryption via TLS mechanisms (i.e. the server * is not configured properly, or this is a local connection and TLS is not * needed. * Note that STARTTLS is supported only in PHP5+. In PHP4 this option will * have no effect and STARTTLS will be disabled anyway. */ global $avelsieve_disabletls; $avelsieve_disabletls = false; /* ======================================================================== */ /* ======================= File Backend Options =========================== */ /* ======================================================================== */ global $avelsieve_file_backend_options, $data_dir, $username; $avelsieve_file_backend_options = array( ? ? 'avelsieve_default_file' => "$data_dir/$username.sievesource" ); /* ======================================================================== */ /* ====== Implementation- and Server-Specific? Options ==================== */ /* ======================================================================== */ /* In Cyrus 2.3+, the notification action is a bit more complex than the * others. The oldcyrus variable is for supporting the partially implemented * notify extension implementation of Cyrus < 2.3. If you have Cyrus < 2.3, * just set this to true. * * This only changes the informational / help text displayed in avelsieve. * * Cyrus < 2.3 : $from$, $env-from$, $subject$ * Cyrus 2.3+? : $from$, $env-from$, $subject$, $text$, $text[n]$ */ global $avelsieve_oldcyrus; $avelsieve_oldcyrus = true; /* If you have Cyrus with an lmtpd that can understand the "auth" argument to * the :envelope test as the SMTP/LMTP auth, or any other Sieve implementation, * then you can enable this to provide this functionality to the user. * * This was not clarified in the base spec of RFC 3028. It will be done * correctly in a new version of Cyrus, based on a new draft / spec. */ global $avelsieve_enable_envelope_auth; $avelsieve_enable_envelope_auth = true; /* Some Implementations of Sieve need certain things in order to operate * correctly. If you use any of the following server implementations, you MUST * set this variable to the corresponding value for the filtering to work * correctly. * Valid values are: * - Any RFC3028-mostly-compatible implementation: '' (empty). * - Exim MTA: 'exim' * - MFL (as supported by mvmf): 'mfl' */ global $avelsieve_custom_sieve_implementation; $avelsieve_custom_sieve_implementation = ''; /* For delivery agents that don't know how to handle some mailbox prefixes, you * can enable this option. Example for Dovecot LDA: 'INBOX.'; */ global $avelsieve_striproot; $avelsieve_striproot = 'INBOX.'; /* If the backend does not support capabilities reporting, such as the File * Backend, then you should define which capabilities are used by the server * implementation. * * The following are the capabilities supported by Exim4 as of Exim version * 4.60, according to README.SIEVE. You can change them if a new version of * Exim provides more functionality: *? 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' * * The following are the capabilities that are suported by MFM, according to: * http://www.mvmf.org/mfl/language.shtml#sieve *? 'envelope', 'fileinto', 'reject', 'relational', 'subaddress', 'regex', *? 'editheader', 'copy', 'vacation', 'comparator-i;ascii-casemap', *? 'comparator-i;octet' * * The following capabilities have been reported to work with Dovecot LDA: *? 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric', *? 'imapflags', 'subaddress','relational','regex' * */ /* global $avelsieve_hardcoded_capabilities; * $avelsieve_hardcoded_capabilities = array( *? ? 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' * ); */ global $avelsieve_hardcoded_capabilities; $avelsieve_hardcoded_capabilities = array( ? ? 'fileinto', 'reject', 'envelope', 'encoded-character', 'vacation', ? ? 'subaddress', 'comparator-i;ascii-numeric', 'relational', 'regex', ? ? 'imap4flags', 'copy', 'include', 'variables', 'body', 'enotify', ? ? 'environment', 'mailbox', 'date' ); /** @var boolean Enable ImapProxy mode. * If you use imapproxy, because imapproxy cannot understand and proxy the * SIEVE protocol, you must connect to the SIEVE daemon (usually on the IMAP * server) itself. So you need to set $imapproxymode to true, and define a * mapping, from the imapproxy host (usually localhost) to your real IMAP * server (usually the same that is defined on Imapproxy's configuration). * * This will not work if you use a perdition-style proxy, where different users * go to different IMAP servers; it applies mostly to people running imapproxy * for speed and want a quick hack. */ global $avelsieve_imapproxymode, $avelsieve_imapproxyserv; $avelsieve_imapproxymode = false; $avelsieve_imapproxyserv = array( ? ? 'localhost' => 'YOURDOMAINNAMEHERE.com' ); /** @var boolean Ldapuserdata mode: Gets user's email addresses (including * mailAlternate & mailAuthorized) from LDAP Prefs Backend plugin's cache */ global $avelsieve_ldapuserdatamode; $avelsieve_ldapuserdatamode = false; /** @var array Map of cyrus administrator users, for proxy authentication */ global $avelsieve_cyrusadmins_map; $avelsieve_cyrusadmins_map = array( ? ? 'cyrusimap' => 'cyrussieve' ); /* ======================================================================== */ /* =============== Avelsieve Interface / Behavior Setup? ================== */ /* ======================================================================== */ /* Be conservative to our updates on the SIEVE server? If true, a button * entitled "Save Changes" will appear, which will give the user the * functionality to register her changes. 'false' is recommended. */ $conservative = false; /* Use images for the move up / down, delete rule buttons and STOP? */ $useimages = true; /* Translate the messages returned by the "Reject" and "Vacation" actions? The * default behaviour since 0.9 is not to translate them. Change to true if in * an intranet environment or in a same-language environment. */ global $translate_return_msgs; $translate_return_msgs = false; /* Theme to use for the images. A directory with the same name must exist under * plugins/avelsieve/$imagetheme, that contains the files: up.png, down.png, * del.png, dup.png, edit.png, top.png, bottom.png. */ $imagetheme = 'famfamfam'; //$imagetheme = 'bluecurve_24x24'; //$imagetheme = 'bluecurve_16x16'; /* Number of items to display _initially_, when displaying the header match * rule */ global $startitems; $startitems = 3; /* Maximum number of items to allow in one header match rule. */ global $maxitems; $maxitems = 10; /* Headers to display in listbox widget, when adding a new header rule. */ /* global $headers; * $headers = array( *? 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Reply-To', 'Sender', 'List-Id', *? 'MailingList', 'Mailing-List', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', *? 'Resent-From',? 'Resent-To', 'X-Mailer', 'X-Mailing-List', *? 'X-Spam-Flag', 'X-Spam-Status', *? 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', *? 'Return-Path', 'Received', 'Auto-Submitted', *? 'X-Spam-Flag', 'X-Spam-Status','X-Spam-Tests' *? ); */ global $headers; $headers = array( 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Sender', 'Reply-To', 'List-Id', 'MailingList', 'Mailing-List', 'X-DKIM-Authentication-Results', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', 'Resent-From',? 'Resent-To', 'X-Mailer', 'X-Mailing-List', 'X-SpamTest-Status-Extended', 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', 'Return-Path', 'Received', 'Auto-Submitted', ); /* Available :method's for the :notify extension (if applicable) */ global $notifymethods; $notifymethods = array( 'mailto', 'sms' ); /* use the value "false" if you want to provide a simple input box so that * users can edit the method themselves : */ //$notifymethods = false; /* Capabilities to disable. If you would like to force avelsieve not to display * certain features, even though there _is_ a capability for them in Sieve * server, you should specify these here. For instance, if you would * like to disable the notify extension, even though ManageSieve advertises it, * you should add 'notify' in this array: $force_disable_avelsieve_capability = * array("notify");. This will still leave the defined feature on, and if the * user can upload her own scripts then she can use that feature; this option * just disables the GUI of it. Leave as-is (empty array) if you do not need * that. * * Look in $implemented_capabilities array in include/constants.inc.php for * valid values */ // $disable_avelsieve_capabilities = array("notify"); global $disable_avelsieve_capabilities; $disable_avelsieve_capabilities = array(); /* Display Filters link in the top Squirrelmail header? */ global $avelsieveheaderlink; $avelsieveheaderlink = true; /* Default rules table display mode, one of 'verbose' or 'terse' */ global $avelsieve_default_mode; $avelsieve_default_mode = 'terse'; /* ======================================================================== */ /* ========================= Custom rules Configuration =================== */ /* ======================================================================== */ /** * @var array * List of additional rules to enable. This must be the numeric id of the * rule. * * 10: Spam Rule (as existed in Avelsieve) * 11: Junk-Mail Rule (new-style SPAM Rule, only one exists per Sieve script) * 12: Global Whitelist (only one exists per Sieve script) * 13: Custom Sieve Code (allows users to enter whatever Sieve code they like - *? ? Alpha/Experimental) * * Example: * array(11, 12); */ global $avelsieve_enable_rules; $avelsieve_enable_rules = array(); global $avelsieve_rules_settings; $avelsieve_rules_settings = array(); foreach($avelsieve_enable_rules as $r) { ? ? if(file_exists(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php')) { ? ? ? ? require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php'); ? ? } else { ? ? ? ? require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.default.php'); ? ? } } /* Please keep the following setting false; it is alpha + needs Squirrelmail * to be patched in three or four places. */ $avelsieve_spam_highlight_enable = false; ############################### Hi Daminto, That looks pretty much unconfigured. No wonder nothing is coming up. I have attached the avelsieve config file we are using on one of our domains. Don't copy it blindly. At least you should configure $sieveport to the port you are using on your server for Sieve. And put your domain name where it reads YOURDOMAINNAMEHERE.com. For the arrays at $headers you should tune the filters you are using with your headers. Hope this helps (working fine with our domain and Squirrel-Mail). Rgds Tamsy Daminto Lie wrote the following on 17.04.2012 13:14: > Thank you for your reply. > > Here it is as follows > > > if(!defined('AVELSIEVE_DEBUG')) { >? ? ? define('AVELSIEVE_DEBUG', 0); > } > > global $avelsieve_backend; > $avelsieve_backend = 'File'; > > /*global $sieveport; > $sieveport = 2000;*/ > > global $sieve_preferred_sasl_mech; > $sieve_preferred_sasl_mech = 'PLAIN'; > > global $avelsieve_disabletls; > $avelsieve_disabletls = false; > > global $avelsieve_file_backend_options, $data_dir, $username; > $avelsieve_file_backend_options = array( >? ? ? 'avelsieve_default_file' =>? "$data_dir/$username.sievesource" > ); > > /*global $avelsieve_oldcyrus; > $avelsieve_oldcyrus = true;*/ > > global $avelsieve_enable_envelope_auth; > $avelsieve_enable_envelope_auth = true; > > global $avelsieve_custom_sieve_implementation; > $avelsieve_custom_sieve_implementation = ''; > > global $avelsieve_striproot; > $avelsieve_striproot = ''; > > global $avelsieve_hardcoded_capabilities; > $avelsieve_hardcoded_capabilities = array( >? ? ? 'envelope', 'fileinto', 'copy', 'vacation', 'comparator-i;ascii-numeric' > ); > > /*global $avelsieve_imapproxymode, $avelsieve_imapproxyserv; > $avelsieve_imapproxymode = false; > $avelsieve_imapproxyserv = array( >? ? ? 'localhost' =>? 'imap.example.org' > );*/ > > global $avelsieve_ldapuserdatamode; > $avelsieve_ldapuserdatamode = false; > > /*global $avelsieve_cyrusadmins_map; > $avelsieve_cyrusadmins_map = array( >? ? ? 'cyrusimap' =>? 'cyrussieve' > );*/ > > $conservative = false; > > $useimages = true; > > global $translate_return_msgs; > $translate_return_msgs = false; > > $imagetheme = 'famfamfam'; > //$imagetheme = 'bluecurve_24x24'; > //$imagetheme = 'bluecurve_16x16'; > > global $startitems; > $startitems = 3; > > global $maxitems; > $maxitems = 10; > > global $headers; > $headers = array( >? 'From', 'To', 'Cc', 'Bcc', 'Subject', 'Reply-To', 'Sender', 'List-Id', >? 'MailingList', 'Mailing-List', 'X-ML-Name', 'X-List', 'X-List-Name', 'X-Mailing-List', >? 'Resent-From',? 'Resent-To', 'X-Mailer', 'X-Mailing-List', >? 'X-Spam-Flag', 'X-Spam-Status', >? 'X-Priority', 'Importance', 'X-MSMail-Priority', 'Precedence', >? 'Return-Path', 'Received', 'Auto-Submitted', >? 'X-Spam-Flag', 'X-Spam-Status','X-Spam-Tests' >? ); > > global $notifymethods; > $notifymethods = array( > 'mailto', 'sms' > ); > > //$notifymethods = false; > > // $disable_avelsieve_capabilities = array("notify"); > global $disable_avelsieve_capabilities; > $disable_avelsieve_capabilities = array(); > > global $avelsieveheaderlink; > $avelsieveheaderlink = true; > > global $avelsieve_default_mode; > $avelsieve_default_mode = 'terse'; > > global $avelsieve_enable_rules; > $avelsieve_enable_rules = array(); > > global $avelsieve_rules_settings; > $avelsieve_rules_settings = array(); > foreach($avelsieve_enable_rules as $r) { >? ? ? if(file_exists(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php')) { >? ? ? ? ? require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.php'); >? ? ? } else { >? ? ? ? ? require(SM_PATH . 'plugins/avelsieve/config/rule.'.$r.'.default.php'); >? ? ? } > } > > $avelsieve_spam_highlight_enable = false; > > > Hope this help you to help me. > > Thank you > > > > > ________________________________ >? From: Tamsy > To: "dovecot at dovecot.org" > Sent: Tuesday, 17 April 2012 1:25 PM > Subject: Re: [Dovecot] vacation plugins for squirrelmail > > Show us your avelsieve-config in > your-squirrelmail-dir/plugins/avelsieve/config/config.php > > > Daminto Lie wrote the following on 17.04.2012 09:31: >> Thank you folks for your replies. >> >> >> I realised that the dovecot 1.2.9 that I installed via ubuntu synaptic package manager has already come with sieve plugins. After making some modifications on the dovecot.conf file, I went to download and install avelsieve 1.0.1 (stable version) onto the squirrelmail. Then, I run in squirrelmail conf.pl and added the plugin. I logged in and could see "Filter" option right on the top header row. I went to click it and nothing came up. I checked in the mail.log but did not see any errors there. >> >> Here is my snapshot of dovecot configuration >> # 1.2.9: /etc/dovecot/dovecot.conf >> # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS ext4 >> log_timestamp: %Y-%m-%d %H:%M:%S >> protocols: imap >> listen: *:143 >> ssl: >>? ? no >> disable_plaintext_auth: no >> login_dir: /var/run/dovecot/login >> login_executable: /usr/lib/dovecot/imap-login >> verbose_proctitle: yes >> first_valid_uid: 1001 >> last_valid_uid: 600 >> mail_privileged_group: mail >> mail_location: maildir:/var/vmail/%u/Maildir >> mbox_write_locks: fcntl dotlock >> imap_client_workarounds: outlook-idle delay-newmail netscape-eoh tb-extra-mailbox-sep >> lda: >>? ? ? postmaster_address: postmaster at ourcompany.com >>? ? ? mail_plugins: sieve >>? ? ? quota_full_tempfail: yes >>? ? ? deliver_log_format: msgid=%m: %$ >>? ? ? sendmail_path: /usr/lib/sendmail >>? ? ? rejection_reason: Your message to<%t>? was automatically rejected:%n%r >>? ? ? auth_socket_path: /var/run/dovecot-auth-master >> auth default: >>? ? ? mechanisms: plain login >>? ? ? username_format: %Lu >>? ? ? verbose: yes >>? ? ? debug: yes >>? ? ? debug_passwords: yes >>? ? ? passdb: >>? ? ? ? driver: >>? ? pam >>? ? ? passdb: >>? ? ? ? driver: ldap >>? ? ? ? args: /etc/dovecot/dovecot-ldap.conf >>? ? ? userdb: >>? ? ? ? driver: passwd >>? ? ? userdb: >>? ? ? ? driver: static >>? ? ? ? args: uid=1001 gid=1001 home=/var/vmail/%u allow_all_users=yes >>? ? ? socket: >>? ? ? ? type: listen >>? ? ? ? client: >>? ? ? ? ? path: /var/spool/postfix/private/auth >>? ? ? ? ? mode: 432 >>? ? ? ? ? user: postfix >>? ? ? ? ? group: mail >>? ? ? ? master: >>? ? ? ? ? path: /var/run/dovecot-auth-master >>? ? ? ? ? mode: 432 >>? ? ? ? ? user: vmail >>? ? ? ? ? group: vmail >> plugin: >>? ? ? sieve: /var/vmail/%u/.sieve >> >> Also, I have restarted dovecot after making changes to the dovecot.conf. >> >> Any help would be very much appreciated. >> >> Thank you >> >>? >> >> ________________________________ >>? ? From: Artur Zaprza?a >> To: Dovecot Mailing List >> Cc: Daminto Lie >> Sent: Thursday, 12 April 2012 11:46 PM >> Subject: Re: [Dovecot] vacation plugins for squirrelmail >> >> Daminto Lie wrote: >>> Hi, >>> >>> I am afraid I >>? ? have a question to ask of you all. I have just completed setting up a mail server running on Ubuntu Server 10.04. It has postfix, dovecot 1.2.19, LDAP and squirrelmail as the webmail. I have also created virtual users accounts on the system through LDAP. I can send and receive mails which is great. Now, what I am trying to do is to set up a vacation auto-reply on the squirrelmail so that users who are about to go on vacation can set it up themselves. I was looking around for the vacation plugins for dovecot that I can incorporate it into squirrelmail. >>> Any help would be very much appreciated. >>> >>> Thank you >>> >> I'm using Avelsieve 1.9.9 with a set of my own bugfixes: >> http://email.uoa.gr/avelsieve/ >> >> From cor at xs4all.nl Wed Apr 18 11:22:32 2012 From: cor at xs4all.nl (Cor Bosman) Date: Wed, 18 Apr 2012 10:22:32 +0200 Subject: [Dovecot] trash plugin Message-ID: <6332F986-C1C4-421E-9EFC-362D429047F4@xs4all.nl> The trash plugin docs say: "Normally if a message can't be saved/copied because it would bring user over quota, the save/copy fails with "Quota exceeded" error. The trash plugin can be used to avoid such situations by making Dovecot automatically expunge oldest messages from configured mailboxes until the message can be saved" This leads me to believe that if a message cant be stored, the plugin would avoid returning a quota error, but instead delete/expunge messages until the original message can fit, and then complete the command. But I see something different, at least using the plugin in imap (havent tried it on lmtp). When I drag a message to a folder thats almost full, and the message doesnt fit, I get an actual quota error: 23 uid copy 454343 "Spam" 23 NO [OVERQUOTA] Quota exceeded (mailbox for user is full) After my client sees this error, I see the trash plugin creating space. But then it's too late right? The client already has a visible error. Am I just misunderstanding how the trash plugin is supposed to work? Dovecot 2.1.4, config is on a set of test servers: namespace { type = private separator = / inbox = yes } namespace { type = private separator = / inbox = no prefix = Spam/ location = maildir:/var/spool/mail/%1u/%1.1u/%u/.Spam:INDEX=/var/spool/mail/dovecot-control/indexes/%1u/%2u/%u/.Spam:CONTROL=/var/spool/mail/dovecot-control/%1u/%2u/%u/.Spam subscriptions = no list = yes hidden = yes } plugin { quota = maildir:Spam quota:ns=Spam/ quota_rule = *:storage=5000K } My goal is to create a FIFO spam folder that never returns a quota error, but instead just deletes the oldest emails as new spam comes in. Regards, Cor From cor at xs4all.nl Wed Apr 18 11:53:34 2012 From: cor at xs4all.nl (Cor Bosman) Date: Wed, 18 Apr 2012 10:53:34 +0200 Subject: [Dovecot] returning a namespace location from userdb? Message-ID: <567E4A7D-930E-4C57-B79D-8C94554D7C72@xs4all.nl> hey all, is it possible to return the location of namespace from the userdb lookup? The code is a bit unclear about it. There seems to be a part of the docs saying: If you want to override settings inside sections, you can separate the section name and key with '/'. For example: namespace default { inbox = yes } The inbox setting can be overridden by returning namespace/default/inbox=no extra field I realize you can set 'location=' in the namespace, but we have mailbox locations that cant be expressed by the normal variable substitution vars in dovecot (dont ask). So we have a userdb that returns the correct mail location. This works fine. But im currently trying to implement a second namespace, and Im having trouble returning a mailbox location for this specific namespace from our userdb. Is this possible? If so, what would the userdb have to return exactly? namespace spam { type = private separator = / inbox = no prefix = Spam/ subscriptions = no list = yes hidden = yes } Regards, Cor From pw at wk-serv.de Wed Apr 18 12:49:14 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Wed, 18 Apr 2012 11:49:14 +0200 Subject: [Dovecot] Missing 20-managesieve.conf in 2.1.4? Message-ID: <4F8E8E1A.3080408@wk-serv.de> Hello, I installed Dovecot 2.1.4 and Pigeonhole 0.3.0 and wondered that there is no 20-managesieve.conf file anymore. It is no problem to use one of my old files. I just want to know whether this is a bug or not. Regards Patrick From CMarcus at Media-Brokers.com Wed Apr 18 13:22:27 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 18 Apr 2012 06:22:27 -0400 Subject: [Dovecot] OT Re: Dovecot and Outlook Express "plaintext authentication not allowed..." In-Reply-To: <1334695280.2248.YahooMailNeo@web161606.mail.bf1.yahoo.com> References: <1334654182.6879.YahooMailNeo@web161605.mail.bf1.yahoo.com> <4F8D455D.9060805@Media-Brokers.com> <1334695280.2248.YahooMailNeo@web161606.mail.bf1.yahoo.com> Message-ID: <4F8E95E3.6020500@Media-Brokers.com> On 2012-04-17 4:41 PM, D Chen wrote: > Per your suggestion, in Outlook Express I changed to use SSL on port > 993 for IMAP, and port 465 for SMTP, the creation of the new account > succeeded without error! However, when tried to send to my yahoo mail or > gmail acct, got the following error: (apparently failed to send!) Changed subject to OT because this is not relevant to dovecot... > "The message could not be sent because one of the recipients was rejected by the server. The > rejected email address was 'dchenusa at yahoo.com.' Subject: 'test out to yahoo mail' > Account:'dchen192.168.20.100' > Server: 192.168.20.100 > Protocol: SMTP, server response: 554 5.7.1 That is a 'Relay access denied' error (assuming yahoo uses correct error codes, which is probably a bad assumption)... Either Yahoo doesn't support port 465 (try the submission port for your yahoo smtp server - 587)... or you have a free account which doesn't allow relaying using their servers (so you'd have to use something else for sending - what does the Thunderbird client use for sending for that account?)... But as suggested... don't use Outlook Express, use a modern mail client (Thunderbird is my preference, but someone else suggested Windows Live Mail which does suck much less than Outlook Express)... -- Best regards, Charles From stephan at rename-it.nl Wed Apr 18 13:34:15 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 18 Apr 2012 12:34:15 +0200 Subject: [Dovecot] Missing 20-managesieve.conf in 2.1.4? In-Reply-To: <4F8E8E1A.3080408@wk-serv.de> References: <4F8E8E1A.3080408@wk-serv.de> Message-ID: <4F8E98A7.2040305@rename-it.nl> On 4/18/2012 11:49 AM, Patrick Westenberg wrote: > Hello, > > I installed Dovecot 2.1.4 and Pigeonhole 0.3.0 and wondered that there > is no 20-managesieve.conf file anymore. > > It is no problem to use one of my old files. I just want to know whether > this is a bug or not. I wouldn't know why it is missing in your installation. It is still contained in the tarball. How did you install it exactly? Regards, Stephan. From pw at wk-serv.de Wed Apr 18 14:06:34 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Wed, 18 Apr 2012 13:06:34 +0200 Subject: [Dovecot] Missing 20-managesieve.conf in 2.1.4? In-Reply-To: <4F8E98A7.2040305@rename-it.nl> References: <4F8E8E1A.3080408@wk-serv.de> <4F8E98A7.2040305@rename-it.nl> Message-ID: <4F8EA03A.7000601@wk-serv.de> Stephan Bosch schrieb: > I wouldn't know why it is missing in your installation. It is still > contained in the tarball. How did you install it exactly? I installed it the same way as usual: ./configure make make install No errors reported. I checked the configure options and also tried --with-managesieve but the file is still missing. However, this is my first installation on Scientific Linux. Patrick From john.robinson at anonymous.org.uk Wed Apr 18 14:36:51 2012 From: john.robinson at anonymous.org.uk (John Robinson) Date: Wed, 18 Apr 2012 12:36:51 +0100 Subject: [Dovecot] mbox folders on-disc layout Message-ID: <4F8EA753.4070605@anonymous.org.uk> I'd like to change the on-disk layout for my mboxes from being ~/mail/foo/bar to ~/mail/foo.bar so that I can have folders containing both messages and subfolders, without having subfolders beginning with . and then having to do all the other fiddling with locations of index files etc. that goes along with LAYOUT=maildir++, and also leaving the filesystem layout looking sensible so my users see their mailboxes instead of the index and control files. Is this possible or do I need to patch something? (CentOS 6, dovecot 2.0.9) Cheers, John. From stephan at rename-it.nl Wed Apr 18 15:42:14 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 18 Apr 2012 14:42:14 +0200 Subject: [Dovecot] http://xi.rename-it.nl upgrade 2.0 - 2.1 dovecot-common (<< 2:2.1.alpha1-0~auto+47) In-Reply-To: <4F8AC9B1.2030307@schetterer.org> References: <4F8AC9B1.2030307@schetterer.org> Message-ID: <4F8EB6A6.6010206@rename-it.nl> Op 4/15/2012 3:14 PM, Robert Schetterer schreef: > Hi, > i got > depency problem like dovecot-common (<< 2:2.1.alpha1-0~auto+47) > > dpkg -i dovecot-common_2.1.4-0~auto+3_all.de > > after all dove seems to run without install dovecot-common basicly You should be able to drop dovecot-common entirely. It is nothing but a dummy package now. Regards, Stephan. From tss at iki.fi Wed Apr 18 16:45:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 16:45:45 +0300 Subject: [Dovecot] Statistics scripts Message-ID: <42DEDB45-6B9A-47BB-BFB1-B82ECC086839@iki.fi> I wrote two scripts, which you can use with the stats plugin enabled to get a better "doveadm stats top" than it currently is: http://dovecot.org/tools/stats-top.pl http://dovecot.org/tools/stats.pl From tss at iki.fi Wed Apr 18 19:29:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 19:29:58 +0300 Subject: [Dovecot] trash plugin In-Reply-To: <6332F986-C1C4-421E-9EFC-362D429047F4@xs4all.nl> References: <6332F986-C1C4-421E-9EFC-362D429047F4@xs4all.nl> Message-ID: <14B830E2-9EFD-45CA-B0F7-BEDDF0FCDFF1@iki.fi> On 18.4.2012, at 11.22, Cor Bosman wrote: > The trash plugin docs say: > > "Normally if a message can't be saved/copied because it would bring user over quota, the save/copy fails with "Quota exceeded" error. The trash plugin can be used to avoid such situations by making Dovecot automatically expunge oldest messages from configured mailboxes until the message can be saved" > > This leads me to believe that if a message cant be stored, the plugin would avoid returning a quota error, but instead delete/expunge messages until the original message can fit, and then complete the command. Yes. > But I see something different, at least using the plugin in imap (havent tried it on lmtp). When I drag a message to a folder thats almost full, and the message doesnt fit, I get an actual quota error: > > 23 uid copy 454343 "Spam" > 23 NO [OVERQUOTA] Quota exceeded (mailbox for user is full) > > After my client sees this error, I see the trash plugin creating space. But then it's too late right? The client already has a visible error. Am I just misunderstanding how the trash plugin is supposed to work? Fixed: http://hg.dovecot.org/dovecot-2.1/rev/51b40cffb628 From tss at iki.fi Wed Apr 18 19:42:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 19:42:21 +0300 Subject: [Dovecot] returning a namespace location from userdb? In-Reply-To: <567E4A7D-930E-4C57-B79D-8C94554D7C72@xs4all.nl> References: <567E4A7D-930E-4C57-B79D-8C94554D7C72@xs4all.nl> Message-ID: On 18.4.2012, at 11.53, Cor Bosman wrote: > hey all, is it possible to return the location of namespace from the userdb lookup? Yes. > But im currently trying to implement a second namespace, and Im having trouble returning a mailbox location for this specific namespace from our userdb. Is this possible? If so, what would the userdb have to return exactly? > > namespace spam { namespace/spam/location=maildir:/something/different From tss at iki.fi Wed Apr 18 19:45:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 19:45:13 +0300 Subject: [Dovecot] mbox folders on-disc layout In-Reply-To: <4F8EA753.4070605@anonymous.org.uk> References: <4F8EA753.4070605@anonymous.org.uk> Message-ID: <35C2B470-438E-4CEC-A42A-0BC2C39A0159@iki.fi> On 18.4.2012, at 14.36, John Robinson wrote: > I'd like to change the on-disk layout for my mboxes from being > ~/mail/foo/bar > to > ~/mail/foo.bar > so that I can have folders containing both messages and subfolders, without having subfolders beginning with . and then having to do all the other fiddling with locations of index files etc. that goes along with LAYOUT=maildir++, and also leaving the filesystem layout looking sensible so my users see their mailboxes instead of the index and control files. > > Is this possible or do I need to patch something? Not possible without patching. There's also the other possibility of doing something like: # Trick mbox configuration which allows a mail folder which contains both # messages and sub-folders mail_location = mbox:~/mail/mailboxes:DIRNAME=mBoX-MeSsAgEs:INDEX=~/mail/index:CONTROL=~/mail/control from http://wiki2.dovecot.org/MboxChildFolders From tss at iki.fi Wed Apr 18 19:46:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 19:46:42 +0300 Subject: [Dovecot] 487 imap-login processes - is this ok In-Reply-To: <4F8D7C73.9090700@filmakademie.de> References: <4F8D7C73.9090700@filmakademie.de> Message-ID: <74640698-B3E5-46BB-88FC-98B088CF0330@iki.fi> On 17.4.2012, at 17.21, G?tz Reinicke wrote: > recently I checked our mailserver for some more tuning and optimizing > and noticed, that currently I do have 487 imap-login processes and I'm > asking myself if that's o.k. > > We do have about 1.000 user in total and not everybody is currently > logged in. > > May be this is o.k. but if not, what might I check? Normal in "high-security mode": http://wiki.dovecot.org/LoginProcess Most of those 487 processes are probably handling SSL traffic. From tss at iki.fi Wed Apr 18 19:47:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 19:47:11 +0300 Subject: [Dovecot] Hints for a NFS-Setup In-Reply-To: <4F8DEEBD.9080206@corp.sonic.net> References: <4F7599AE.9080300@wk-serv.de> <4F75A929.9090902@mobilia.it> <4F8DEEBD.9080206@corp.sonic.net> Message-ID: <8B60CEC2-4945-4513-9873-CCEEA0344C7A@iki.fi> On 18.4.2012, at 1.29, Kelsey Cummings wrote: > On 03/30/12 07:35, Timo Sirainen wrote: >> IP stickiness isn't enough if user uses more than one IMAP client, which is pretty common nowadays. And doesn't help at all with LDA. > > Timo, having the LDA flow through the director with maildir and NFS is only an issue if you want to use the dovecot delivery agent since it manipulates the indexes on delivery, right? This shouldn't be a problem if you are running a completely distinct delivery cluster running a LDA like procmail. Right. If indexes or dovecot-uidlist isn't updated then it doesn't matter. From tss at iki.fi Wed Apr 18 19:57:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 19:57:26 +0300 Subject: [Dovecot] Re-indexing of fts-solr not working In-Reply-To: References: Message-ID: <1766B7F6-4A32-41DA-9655-13E0AE43A8C7@iki.fi> On 17.4.2012, at 21.36, Joe Beaubien wrote: > I am currently testing a 2.1 dovecot setup with fts-solr. Search speed is > simply amazing. I decided to try to delete and rebuild the solr indexes, > but i can't seem to rebuild them thru dovecot. With Maildir you could just delete dovecot.index* files. But with dbox that's not a good idea. Anyway, implemented now the minimum needed to get doveadm fts rescan to work: http://hg.dovecot.org/dovecot-2.1/rev/d145ea6f7061 From tss at iki.fi Wed Apr 18 20:01:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 20:01:14 +0300 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken In-Reply-To: <4F8D80D3.2050505@schetterer.org> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> Message-ID: On 17.4.2012, at 17.40, Robert Schetterer wrote: > doveadm acl debug > > shows on both versions > > Fatal: ACL shared dict iteration failed Doesn't it show any other error message before this? Either to terminal or to error log? What's your doveconf -n? From tss at iki.fi Wed Apr 18 20:06:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 20:06:45 +0300 Subject: [Dovecot] LMTP auth problem In-Reply-To: <20120414172135.GA32608@xs4all.nl> References: <7EC18FE2-6612-4BCC-A68A-EF79FA82E80F@xs4all.nl> <71BEFF67-4A0A-442A-9ED0-C4B1B23A4E7F@leuxner.net> <20120414172135.GA32608@xs4all.nl> Message-ID: On 14.4.2012, at 20.21, Cor Bosman wrote: > My config was the same as yours. That didnt work for me. But if I add > > user = dovecot > mode = 0666 Hmm. Perhaps I should make this the default. > That does work. Of course, the difference between 777 and 666 is > minimal. I think 666 is handled as a special case in the code? Yes, it's handled specially. 0666 is now a safe default for multi-user systems, since it returns data for a user only if the calling process's UID matches the returned "uid". 0777 makes it truly world-accessible. From news0104.6.weezy at spamgourmet.com Wed Apr 18 20:00:48 2012 From: news0104.6.weezy at spamgourmet.com (Andreas Helmcke) Date: Wed, 18 Apr 2012 19:00:48 +0200 Subject: [Dovecot] namespace hierarchy In-Reply-To: References: <201204010928.41017.anyaddress@gmx.net> Message-ID: <4F8EF340.4010407@spamgourmet.com> Am 04.04.2012 03:47, schrieb Timo Sirainen: > On 1.4.2012, at 10.28, Tom Fernandes wrote: > >> In the new server I would like to have 3 namespaces (private, public, shared) >> and use "/" as separator. Is it possible to keep "INBOX." for private, so that >> the users mailboxes appear below the Inbox, but have public and shared on the >> same hierarchy-level like the Inbox? >> > > If you have any subscriptions=no namespaces, you need a parent namespace that has subscriptions=yes. You could have e.g.: > > namespace { > prefix = INBOX/ > list = no > inbox = yes > } > namespace { > prefix = > list = no > hidden = yes > alias_for = INBOX/ > } > > So now the prefix="" is used for the subscriptions but nothing else really. I tried this, but sadly with dovecot V2.1.1 it is not working. Steps to reproduce: + create empty mail directory + create one folder (Allgemeines/Tabel) in public namespace with doveadm + start mail-client (in my case thunderbird) + try to subscribe to the new public folder Error message in log: Apr 18 18:19:34 elablnmail02 dovecot: imap(ahelmcke): Warning: Subscriptions file /home/mail/user/ahelmcke/subscriptions: Removing invalid entry: Allgemeines/Tabel doveconf -n: # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) ext4 auth_master_user_separator = * first_valid_uid = 400 mail_gid = vmail mail_location = maildir:/home/mail/user/%u mail_uid = vmail mbox_write_locks = fcntl namespace Allgemeines { location = maildir:/home/mail/Allgemeines:INDEX=/home/mail/user/%u/SEEN.Allgemeines prefix = Allgemeines/ separator = / subscriptions = no type = public } namespace dummy { alias_for = INBOX/ hidden = yes list = no location = prefix = separator = / type = private } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile sieve = /var/lib/dovecot/sieve/%u.sieve sieve_global_dir = /var/lib/dovecot/sieve/global/ sieve_global_path = /var/lib/dovecot/sieve/default.sieve } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } service imap-login { inet_listener imap { address = * } inet_listener imaps { address = * } } ssl = required ssl_cert = References: <201204010928.41017.anyaddress@gmx.net> <4F8EF340.4010407@spamgourmet.com> Message-ID: <305E07F6-1D75-4B6D-862A-D7697442A770@iki.fi> On 18.4.2012, at 20.00, Andreas Helmcke wrote: >> If you have any subscriptions=no namespaces, you need a parent namespace that has subscriptions=yes. You could have e.g.: >> >> namespace { >> prefix = INBOX/ >> list = no >> inbox = yes >> } >> namespace { >> prefix = >> list = no >> hidden = yes >> alias_for = INBOX/ >> } >> >> So now the prefix="" is used for the subscriptions but nothing else really. > > I tried this, but sadly with dovecot V2.1.1 it is not working. Oh, I hadn't thought of one problem: It's now using the same subscriptions file for both prefix="" and prefix=INBOX/ namespaces. This confuses it. So you need to place the prefix="" namespace somewhere else, for example: > namespace dummy { > alias_for = INBOX/ location = maildir:/home/mail/user/%u/public > hidden = yes > list = no > location = > prefix = > separator = / > type = private > } From robert at schetterer.org Wed Apr 18 20:44:23 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 18 Apr 2012 19:44:23 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken In-Reply-To: References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> Message-ID: <4F8EFD77.90103@schetterer.org> Am 18.04.2012 19:01, schrieb Timo Sirainen: > On 17.4.2012, at 17.40, Robert Schetterer wrote: > >> doveadm acl debug >> >> shows on both versions >> >> Fatal: ACL shared dict iteration failed > > Doesn't it show any other error message before this? Either to terminal or to error log? What's your doveconf -n? Hi Timo, i didnt found more usefull errors before all are like Debug: acl: No lookup right to mailbox: but that istn true, i checked it in the filesystem and tried new acls etc ( and they working under 2.0.20 ) to me it looks like some acl check/list problem perhaps depend on stuff like in i.e http://hg.dovecot.org/dovecot-2.1/rev/65a75939ac2c as i said the old 2.0.20 works still fine on the other server the only change i did was with virtual and namespace inbox to get special_use working > this is from the dove version # 2.1.4 (85ad4baedd43): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-37-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 50 M auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes auth_verbose_passwords = plain auth_worker_max_count = 300 dict { acl = mysql:/etc/dovecot/dovecot-dict-acl-sql.conf.ext expire = mysql:/etc/dovecot/dovecot-dict-expire-sql.conf.ext quotadict = mysql:/etc/dovecot/dovecot-dict-quota-sql.conf.ext } disable_plaintext_auth = no first_valid_gid = 1001 first_valid_uid = 1001 hostname = mail01.example.com last_valid_gid = 1001 last_valid_uid = 1001 listen = * login_greeting = imap, pop ready mail_access_groups = vmail mail_debug = yes mail_fsync = always mail_gid = 1001 mail_location = maildir:~/ mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = " stats zlib expire virtual fts fts_squat mail_log autocreate notify acl listescape" mail_privileged_group = vmail mail_uid = 1001 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags notify mmap_disable = yes namespace { list = yes location = maildir:/usr/local/virtual/%%d/%%u:INDEX=~/shared/%%d/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { hidden = no list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } mailbox name { special_use = \Drafts \Junk \Sent \Trash } prefix = separator = / subscriptions = yes } namespace real { hidden = yes list = no location = prefix = RealMails/ separator = / } namespace virtual { hidden = yes list = no location = virtual:/etc/dovecot/virtual2:LAYOUT=maildir++:INDEX=~/virtual prefix = virtual/ separator = / } passdb { args = /etc/dovecot/dovecot-sql-master.conf.ext driver = sql master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl autocreate = Trash autocreate2 = Junk autocreate3 = Sent autocreate4 = Drafts autocreate5 = Templates autocreate6 = Hostmaster-Backup autocreate7 = archiv-backup-mailspooler autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Drafts autosubscribe5 = Templates expire = Trash expire2 = Trash/* expire3 = Junk expire4 = Junk/* expire5 = Hostmaster-Backup expire6 = Hostmaster-Backup/* expire7 = archiv-backup-mailspooler/* expire_dict = proxy::expire fts = squat home = /usr/local/virtual/%d/%u mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size mail_log_group_events = yes quota = dict:::proxy::quotadict quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /usr/local/virtual/%d/%u/dovecot.sieve sieve_before = /etc/dovecot/sieve/global.sieve sieve_dir = /usr/local/virtual/%d/%u/sieve sieve_execute_bin_dir = /usr/lib/dovecot/sieve-execute sieve_execute_socket_dir = sieve-execute sieve_extensions = +notify +imapflags sieve_filter_bin_dir = /usr/lib/dovecot/sieve-filter sieve_filter_socket_dir = sieve-filter sieve_global_dir = /etc/dovecot/sieve/ sieve_global_extensions = +vnd.dovecot.execute +vnd.dovecot.pipe +vnd.dovecot.filter sieve_global_path = /etc/dovecot/sieve/global.sieve sieve_pipe_bin_dir = /usr/lib/dovecot/sieve-pipe sieve_pipe_socket_dir = sieve-pipe sieve_plugins = sieve_extprograms stats_refresh = 30 secs stats_track_cmds = yes zlib_save = gz zlib_save_level = 6 } postmaster_address = hostmaster at mail01.example.com protocols = imap pop3 lmtp sieve sendmail_path = /usr/lib/sendmail service anvil { client_limit = 1000 } service auth-worker { user = root } service auth { client_limit = 1000 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service dict { extra_groups = vmail unix_listener dict { group = vmail mode = 0660 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 8 service_count = 1 } service imap-postlogin { executable = script-login /usr/local/bin/postlogin_imap.sh } service imap { executable = imap imap-postlogin process_limit = 1024 vsz_limit = 256 M } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } vsz_limit = 256 M } service managesieve-login { inet_listener { address = 127.0.0.1 212.52.224.210 212.52.224.205 port = 2000 } vsz_limit = 256 M } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service pop3-postlogin { executable = script-login /usr/local/bin/postlogin_pop3.sh } service pop3 { executable = pop3 pop3-postlogin process_limit = 1024 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { mode = 0666 user = vmail } user = vmail } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl_ca = References: <1766B7F6-4A32-41DA-9655-13E0AE43A8C7@iki.fi> Message-ID: Awesome, thanks. Will this appear in the 2.1.5 release? On Wed, Apr 18, 2012 at 12:57 PM, Timo Sirainen wrote: > On 17.4.2012, at 21.36, Joe Beaubien wrote: > > > I am currently testing a 2.1 dovecot setup with fts-solr. Search speed is > > simply amazing. I decided to try to delete and rebuild the solr indexes, > > but i can't seem to rebuild them thru dovecot. > > With Maildir you could just delete dovecot.index* files. But with dbox > that's not a good idea. Anyway, implemented now the minimum needed to get > doveadm fts rescan to work: > > http://hg.dovecot.org/dovecot-2.1/rev/d145ea6f7061 > > From tss at iki.fi Wed Apr 18 22:43:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Apr 2012 22:43:46 +0300 Subject: [Dovecot] Re-indexing of fts-solr not working In-Reply-To: References: <1766B7F6-4A32-41DA-9655-13E0AE43A8C7@iki.fi> Message-ID: Sure. Everything in hg goes to the next release. On 18.4.2012, at 22.41, Joe Beaubien wrote: > Awesome, thanks. > > Will this appear in the 2.1.5 release? > > > On Wed, Apr 18, 2012 at 12:57 PM, Timo Sirainen wrote: > >> On 17.4.2012, at 21.36, Joe Beaubien wrote: >> >>> I am currently testing a 2.1 dovecot setup with fts-solr. Search speed is >>> simply amazing. I decided to try to delete and rebuild the solr indexes, >>> but i can't seem to rebuild them thru dovecot. >> >> With Maildir you could just delete dovecot.index* files. But with dbox >> that's not a good idea. Anyway, implemented now the minimum needed to get >> doveadm fts rescan to work: >> >> http://hg.dovecot.org/dovecot-2.1/rev/d145ea6f7061 >> >> From jtam.home at gmail.com Thu Apr 19 02:11:17 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 18 Apr 2012 16:11:17 -0700 (PDT) Subject: [Dovecot] alias_for Message-ID: I noticed the addition of a new namespace configuration "alias_for". What pracical difference is adding this over and above making namespace aliases hidden and disabling list, as in older versions of dovecot configurations? Joseph Tam From janfrode at tanso.net Thu Apr 19 11:22:45 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 19 Apr 2012 10:22:45 +0200 Subject: [Dovecot] migrate 15K users to new domain name Message-ID: <20120419082245.GA4050@dibs.tanso.net> I need to migrate 15K users to a new domain name, and plan to use "dsync mirror" in the transition phase. Could someone confirm that this should work: Before giving users access to new-domain do a first sync to get all the stale data over: for user in $old-domain; do dsync mirror $user at old-domain $user at new-domain done Configure sieve vacation filter to forward all messages from $user at old-domain to $user at new-domain, and notify sender of changed address. Give users access to both new-domain and old-domain, and do a final sync. for user in $old-domain; do dsync mirror $user at old-domain $user at new-domain dsync mirror $user at old-domain $user at new-domain # twice in case the first was slow drop all messages for $user at old-domain Leave notice message for $user at old-domain saying he should use new-domain done Wait a few weeks/months, and then drop all users at old-domain. Does this look sensible? -jf From dchenusa at yahoo.com Thu Apr 19 11:44:35 2012 From: dchenusa at yahoo.com (D Chen) Date: Thu, 19 Apr 2012 01:44:35 -0700 (PDT) Subject: [Dovecot] IMAP SSL incoming test need help ! Message-ID: <1334825075.56841.YahooMailNeo@web161604.mail.bf1.yahoo.com> Ubuntu 11.10 server with postfix/dovecot/squirrelmail configured. Outgoing SMTP server seemed worked fine, however, the incoming IMAP with SSL on port 993 doesn't seem to receive any mail at all !? ?Here is the mail.log during the test session from an external yahoo mail (dchenusa at yahoo.com) to the internal server testmail.biokeyinc.com (dchen at testmail.biokeyinc.com) The log did NOT show any error ! ?The sender showed sent mail! but no mail reach to the destination! Any help would be appreciated. Apr 19 01:20:02 testmail postfix/smtpd[4103]: connect from nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:03 testmail postfix/smtpd[4103]: 091ED200973: client=nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:03 testmail postfix/cleanup[4108]: 091ED200973: message-id=<32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com> Apr 19 01:20:03 testmail postfix/qmgr[2007]: 091ED200973: from=, size=2840, nrcpt=1 (queue active) Apr 19 01:20:03 testmail postfix/smtpd[4103]: disconnect from nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:12 testmail postfix/smtpd[4113]: connect from localhost[127.0.0.1] Apr 19 01:20:12 testmail postfix/smtpd[4113]: 33AE9200AE9: client=localhost[127.0.0.1] Apr 19 01:20:12 testmail postfix/cleanup[4108]: 33AE9200AE9: message-id=<32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com> Apr 19 01:20:12 testmail postfix/qmgr[2007]: 33AE9200AE9: from=, size=3536, nrcpt=1 (queue active) Apr 19 01:20:12 testmail postfix/smtpd[4113]: disconnect from localhost[127.0.0.1] Apr 19 01:20:12 testmail amavis[2042]: (02042-06) Passed CLEAN, [98.138.91.180] [98.138.87.4] -> , Message-ID: <32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com>, mail_id: rqyVZiQDwSsG, Hits: -0.109, size: 2837, queued_as: 33AE9200AE9, dkim_id=dchenusa at yahoo.com, at yahoo.com, 8647 ms Apr 19 01:20:12 testmail postfix/smtp[4109]: 091ED200973: to=, relay=127.0.0.1[127.0.0.1]:10024, delay=9.5, delays=0.87/0.01/0/8.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 33AE9200AE9) Apr 19 01:20:12 testmail postfix/qmgr[2007]: 091ED200973: removed Apr 19 01:20:12 testmail postfix/local[4114]: 33AE9200AE9: to=, relay=local, delay=0.18, delays=0.1/0/0/0.07, dsn=2.0.0, status=sent (delivered to command: procmail -a "$EXTENSION") Apr 19 01:20:12 testmail postfix/qmgr[2007]: 33AE9200AE9: removed Apr 19 01:21:30 testmail dovecot: imap-login: Login: user=, method=PLAIN, rip=63.195.90.22, lip=192.168.20.100, mpid=4122, TLS Apr 19 01:23:13 testmail dovecot: imap(dchen): Disconnected: Disconnected in IDLE bytes=99/708 Apr 19 01:23:23 testmail postfix/anvil[4105]: statistics: max connection rate 1/60s for (smtp:98.138.91.180) at Apr 19 01:20:02 Apr 19 01:23:23 testmail postfix/anvil[4105]: statistics: max connection count 1 for (smtp:98.138.91.180) at Apr 19 01:20:02 Apr 19 01:23:23 testmail postfix/anvil[4105]: statistics: max cache size 1 at Apr 19 01:20:02 From pw at wk-serv.de Thu Apr 19 12:34:50 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 19 Apr 2012 11:34:50 +0200 Subject: [Dovecot] =?utf-8?q?Missing_20-managesieve=2Econf_in_2=2E1=2E4=3F?= In-Reply-To: <4F8E98A7.2040305@rename-it.nl> References: <4F8E8E1A.3080408@wk-serv.de> <4F8E98A7.2040305@rename-it.nl> Message-ID: Hi Stephan, after re-installing the whole system the files have been installed properly. Patrick From stan at hardwarefreak.com Thu Apr 19 12:40:43 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 19 Apr 2012 04:40:43 -0500 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8D69AB.5090706@esiee.fr> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <4F8D69AB.5090706@esiee.fr> Message-ID: <4F8FDD9B.6060007@hardwarefreak.com> On 4/17/2012 8:01 AM, Frank Bonnet wrote: > have 4000/6000 imaps concurent connections during working hours . > > POP3 users will be very few How much disk space do you plan to offer per user mail directory? Will you be using quotas? >>> I need some feedbacks advices of experienced admins >>> I will have to setup in few monthes an email system >>> for approx 50K "intensives" users. >>> >>> The only mandatory thing will be I must use HP proliant servers >>> >>> The operating system will be FreeBSD or Linux Quite a coincidence Frank. It's a shame it has to be an HP solution. I just finished designing a high quality high performance 4U 72 drive server yesterday that will easily handle 15K concurrent IMAP users, for only ~$24K USD, $0.48/user @50K users. So it may not be of interest to you, but maybe to others. It is capable of ~7K random 4KB r/w IOPS sustained, has 10TB net space for an average ~200MB/user mail directory assuming 50K users. The parts for this machine run ~$24K USD at Newegg. I just made the wishlist public so it should be available tomorrow or Friday. I'll provide the link when it's available. All components used are top quality, best available in the channel. The reliability of the properly assembled server will rival that of any HP/Dell/IBM machine. For those not familiar with SuperMicro, they manufacture many of Intel's retail boards and have for a decade+. The majority of the COTS systems used in large academic HPC clusters are built with SuperMicro chassis and motherboards, as well as some 1000+ node US DOE clusters. Here are the basics: 72x 2.5" bay 4U chassis, 3x SAS backplanes each w/redundant expanders: http://www.newegg.com/Product/Product.aspx?Item=N82E16811152212 78x Seagate 10K SAS 300GB drives--includes 6 spares Triple LSI 9261-8i dual port 512MB BBWC RAID controllers each with 2 redundant load balanced connections to a backplane 24 drives per controller for lowest latency, maximum throughput, 1.5GB total write cache, a rebuild affects only one controller, etc SuperMicro mainboard, 2x 6-core 3.3GHz AMD Interlagos Opteron CPUs 64GB Reg ECC DDR3-1066, 8x8GB DIMMs, 34GB/s aggregate bandwidth Dual Intel Quad port GbE NICs, 10 total Intel GbE ports Use the 2 mobo ports for redundant management links Aggregate 4 ports, 2 on each quad NIC, for mail traffic Aggregate the remaining 4 for remote backup, future connection to an iSCSI SAN array, etc Or however works best--having 8 GbEs gives flexibility and these two cards are only $500 of the total 2x Intel 20GB SSD internal fixed drives, hardware mirrored by the onboard LSI SAS chip, for boot/OS The key to performance, and yielding a single file tree, is once again using XFS to take advantage of this large spindle count across 3 RAID controllers. Unlike previous configurations where I recommended using a straight md concatenation of hardware RAID1 pairs, in this case we're going to use a concatenation of 6 hardware RAID10 arrays. There are a couple of reasons for doing so in this case: 1. Using 36 device names in a single md command line is less than intuitive and possibly error prone. Using 6 is more manageable. 2. We have 3 BBWC RAID controllers w/24 drives each. This is a high performance server and will see a high IO load in production. In many cases one would use an external filesystem journal, which we could easily do and get great performance with our mirrored SSDs. However, the SSDs are not backed by BBWC, so a UPS failure or system crash could hose the log journal. So we'll go with the default internal journal which will be backed by the BBWC. Going internal with the log in this mail scenario can cause a serious amount of extra IOPS on the filesystem data section, this being Allocation Group 0. If we did the "normal" RAID1 concat, all the log IO would hit the first RAID1 pair. On this system, the load may hit that spindle pretty hard, making access to mailboxes in AG0 slower than others. With 6 RAID10 arrays in a concat, the internal log writes will be striped across 6 spindles in the first array. With 512MB BBWC backing that array and optimizing writeout, and with delaylog, this will yield optimal log write performance without slowing down mailbox file access in AG0. To create such a setup we'd do something like this, assuming the mobo LSI controller yields sd[ab], and the 6 array devices on the PCIe LSI cards yield sd[cdefgh] 1. Create two RAID10 arrays, each of 12 drives, in the WebBIOS GUI of each LSI card, using a strip size of 32KB which should yield good random r/w performance for any mailbox format. Use the following policies for each array: RW, Normal, Wback, Direct, Disable, No, and use the full size. Create the concatenated md device: $ mdadm -C /dev/md0 -l linear -n 6 /dev/sd[cdefgh] Then we format it with XFS, optimizing the AG layout for our mailbox workload, and allocation write stripe alignment to each hardware array: $ mkfs.xfs -d agcount=24 su=32k sw=6 /dev/md0 This yields 4 AGs per RAID10 array which will minimize the traditional inode64 head seeking overhead on striped arrays, while still yielding fantastic allocation parallelism with 24 AGs. Optimal fstab for MTA queue/mailbox workload, assuming kernel 2.6.39+: /dev/md0 /mail xfs defaults,inode64,nobarrier 0 0 We disable write barriers as we have BBWC. And that 1.5GB of BBWC will yield extremely low Dovecot write latency and throughput. Given the throughput available, if you're running Postfix on this box, you will want to create a directory on this filesystem for the Postfix spool. Postfix puts the spool files in many dozens, hundreds of subdirectories, so you'll get 100% parallelism across all AGs, thus all disks. It's very likely none of you will decide to build this system. My hope is that some of the design concepts and components used, along with the low cost but high performance of this machine, may be educational or simply give people new ideas, steer them in directions they may not have previously considered. -- Stan From odhiambo at gmail.com Thu Apr 19 14:01:44 2012 From: odhiambo at gmail.com (Odhiambo Washington) Date: Thu, 19 Apr 2012 14:01:44 +0300 Subject: [Dovecot] migrate 15K users to new domain name In-Reply-To: <20120419082245.GA4050@dibs.tanso.net> References: <20120419082245.GA4050@dibs.tanso.net> Message-ID: On Thu, Apr 19, 2012 at 11:22, Jan-Frode Myklebust wrote: > I need to migrate 15K users to a new domain name, and plan to use "dsync > mirror" in the transition phase. Could someone confirm that this should > work: > > Before giving users access to new-domain do a first sync to get all the > stale data over: > > for user in $old-domain; do > dsync mirror $user at old-domain $user at new-domain > done > > Configure sieve vacation filter to forward all messages from > $user at old-domain to $user at new-domain, and notify sender of changed > address. > > Give users access to both new-domain and old-domain, and do a final > sync. > > for user in $old-domain; do > dsync mirror $user at old-domain $user at new-domain > dsync mirror $user at old-domain $user at new-domain # twice in > case the first was slow > drop all messages for $user at old-domain > Leave notice message for $user at old-domain saying he should > use new-domain > done > > Wait a few weeks/months, and then drop all users at old-domain. > > > Does this look sensible? > > > -jf > What do you mean by a "new domain" in this context? Is the server changing? Is the storage changing? In my thinking, a domain change is as simple as using a rewrite rule in your MTA. I am sure I didn't understand your requirement and as such my response is equally vague. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I can't hear you -- I'm using the scrambler. Please consider the environment before printing this email. -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 652 bytes Desc: not available URL: From robert at schetterer.org Thu Apr 19 15:09:10 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 14:09:10 +0200 Subject: [Dovecot] doveadm delete folders ? Message-ID: <4F900066.9030808@schetterer.org> Hi , is there a feature like doveadm expunge savedbefore ... for imap folder delete ? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From luc at bigdata.nb.ca Thu Apr 19 15:14:16 2012 From: luc at bigdata.nb.ca (Luc Belliveau) Date: Thu, 19 Apr 2012 09:14:16 -0300 Subject: [Dovecot] old messages being delivered to blackberry In-Reply-To: <5CDED2FA-3654-4378-A2D3-8A0BFAF7B5CC@iki.fi> References: <4F843969.6040806@bigdata.nb.ca> <5CDED2FA-3654-4378-A2D3-8A0BFAF7B5CC@iki.fi> Message-ID: <4F900198.1080703@bigdata.nb.ca> I've switched to NFS instead of GlusterFS, and the problems have disappeared. It's too bad... GlusterFS had some nice features ... On 04/10/2012 12:11 PM, Timo Sirainen wrote: > On 10.4.2012, at 16.45, Luc Belliveau wrote: > >> Several users are reporting old messages (sometime years old) are being delivered as new on their blackberry devices... I've confirmed that this happens to all blackberry users... I believe the problem is related to this error: >> >> Maildir: Expunged message reappeared, giving a new UID > This means: > > 1. readdir() returns maildir file X > 2. Later readdir() doesn't return file X, so Dovecot thinks the mail is expunged and also tells the client the mail is expunged > 3. Later readdir() once again returns file X, so Dovecot can't really do anything except treat it as a new mail. > >> Possibly caused by GlusterFS, the indexes were still getting corrupted, even with only 1 server... so I've moved the indexes to a local directory on each server. I'm no longer getting index related errors... but the expunged error is still occuring... and the error seems to happen a few moments before the BB gets a bunch of messages... so it's very likely related. >> >> I would like to continue using GlusterFS if possible; I haven't found any indication that it is outright unsupported, as long as I'm using Director... > If glusterfs can't do readdir() reliably, there's not a whole lot you can do.. Except maybe switch to another mailbox format. But if you're getting index corruption also with 1 Dovecot server, it doesn't look very promising. > From gedalya at gedalya.net Thu Apr 19 16:19:54 2012 From: gedalya at gedalya.net (Gedalya) Date: Thu, 19 Apr 2012 15:19:54 +0200 Subject: [Dovecot] doveadm delete folders ? In-Reply-To: <4F900066.9030808@schetterer.org> References: <4F900066.9030808@schetterer.org> Message-ID: <4F9010FA.8000704@gedalya.net> On 4/19/2012 2:09 PM, Robert Schetterer wrote: > Hi , > is there a feature like > doveadm expunge savedbefore ... > for imap folder delete ? I was wondering the same thing - when a user moves a whole folder into Trash, doveadm expunge mailbox "Trash.*" savedbefore would eventually remove all the messages, but the folder itself remains a subfolder of Trash. It would make sense to be able to remove subfolders which have been there for more than x days and are now empty. From janfrode at tanso.net Thu Apr 19 16:28:03 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 19 Apr 2012 15:28:03 +0200 Subject: [Dovecot] migrate 15K users to new domain name In-Reply-To: References: <20120419082245.GA4050@dibs.tanso.net> Message-ID: <20120419132803.GA9396@dibs.tanso.net> On Thu, Apr 19, 2012 at 02:01:44PM +0300, Odhiambo Washington wrote: > > What do you mean by a "new domain" in this context? The user's email addresses are changing from username at old.domain to username at new-domain. > Is the server changing? No. > Is the storage changing? The user's home directory is based on the user's email address, so this is changing. > In my thinking, a domain change is as simple as using a rewrite rule in > your MTA. Also the user's login-names needs to change from old to new domain, and all their data needs to move from old to new domain. -jf From cor at xs4all.nl Thu Apr 19 16:37:25 2012 From: cor at xs4all.nl (Cor Bosman) Date: Thu, 19 Apr 2012 15:37:25 +0200 Subject: [Dovecot] multiple Return-Path headers added using lmtp director Message-ID: <72F28E8A-EE19-48B9-AD36-0147DACD73B2@xs4all.nl> Emails arrive with 2 Return-Paths when using lmtp director. Is this something configurable in the director or is this a bug? Return-Path: Delivered-To: Received: from lmtpdirector1.xs4all.net ([194.109.26.176]) by lmtp2.xs4all.net (Dovecot) with LMTP id gOe2E3YSkE/aEAAAYH/+MA for ; Thu, 19 Apr 2012 15:26:14 +0200 Return-Path: Received: from mxdrop-builder.xs4all.nl ([194.109.23.200]) by lmtpdirector1.xs4all.net (Dovecot) with LMTP id CnOvKTkzjU9sJQAAbddgHA ; Thu, 19 Apr 2012 15:26:14 +0200 Regards From robert at schetterer.org Thu Apr 19 16:57:21 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 15:57:21 +0200 Subject: [Dovecot] doveadm delete folders ? In-Reply-To: <4F9010FA.8000704@gedalya.net> References: <4F900066.9030808@schetterer.org> <4F9010FA.8000704@gedalya.net> Message-ID: <4F9019C1.4070106@schetterer.org> Am 19.04.2012 15:19, schrieb Gedalya: > On 4/19/2012 2:09 PM, Robert Schetterer wrote: >> Hi , >> is there a feature like >> doveadm expunge savedbefore ... >> for imap folder delete ? > I was wondering the same thing - when a user moves a whole folder into > Trash, doveadm expunge mailbox "Trash.*" savedbefore would eventually > remove all the messages, but the folder itself remains a subfolder of > Trash. It would make sense to be able to remove subfolders which have > been there for more than x days and are now empty. > jep i.e for all empty and not longer needed archive date folders -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From odhiambo at gmail.com Thu Apr 19 17:03:01 2012 From: odhiambo at gmail.com (Odhiambo Washington) Date: Thu, 19 Apr 2012 17:03:01 +0300 Subject: [Dovecot] migrate 15K users to new domain name In-Reply-To: <20120419132803.GA9396@dibs.tanso.net> References: <20120419082245.GA4050@dibs.tanso.net> <20120419132803.GA9396@dibs.tanso.net> Message-ID: On Thu, Apr 19, 2012 at 16:28, Jan-Frode Myklebust wrote: > On Thu, Apr 19, 2012 at 02:01:44PM +0300, Odhiambo Washington wrote: > > > > What do you mean by a "new domain" in this context? > > The user's email addresses are changing from username at old.domain to > username at new-domain. > That can be handled by address rewrites within the MTA. > > > Is the server changing? > > No. > > > Is the storage changing? > > The user's home directory is based on the user's email address, so this is > changing. > In my setup, I have virtual users. So the home directory is in the /var/spool/virtual/$domain/$user/mdbox How is yours setup? If the domain name changed, from domain1 to domain2, I believe it would be easy to change as follows: cd /var/spool/virtual/ mv $domain1 $domain2 Well, it the size of $domain1 is in TBs, I'd probably do it another way as doing mv is a bit dangerous with very large datasets: cd /var/spool/virtual mkdir domain2 cd domain1 for d in `ls -1`; do mv $d domain2/$d; done [ or something closer] > > In my thinking, a domain change is as simple as using a rewrite rule in > > your MTA. > > Also the user's login-names needs to change from old to new domain, and > all their data needs to move from old to new domain. > > And the login names are stored in a flatfile or db?? Either way, you can do a rename. Dump the database and just use vi to rename old-domain to new-domain, then drop the db and import the dump. mysqldump dbname dbname.sql vi dbname.sql :g/old-domain/s//new-domain/g mysqladmin drop dbname mysqladmin create dbname mysql dbname < dbname.sql You can also edit the flatfile to s/old-domain/new-domain/g cat flatfile | xargs sed -i.BAK 's/old-domain/new-domain/g' Maybe I still don't understand you:-) -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I can't hear you -- I'm using the scrambler. Please consider the environment before printing this email. -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 652 bytes Desc: not available URL: From janfrode at tanso.net Thu Apr 19 17:19:30 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 19 Apr 2012 16:19:30 +0200 Subject: [Dovecot] migrate 15K users to new domain name In-Reply-To: References: <20120419082245.GA4050@dibs.tanso.net> <20120419132803.GA9396@dibs.tanso.net> Message-ID: <20120419141930.GB10069@dibs.tanso.net> On Thu, Apr 19, 2012 at 05:03:01PM +0300, Odhiambo Washington wrote: > > > > In my setup, I have virtual users. So the home directory is in the > /var/spool/virtual/$domain/$user/mdbox > > How is yours setup? mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln > If the domain name changed, from domain1 to domain2, I > believe it would be easy to change as follows: > > cd /var/spool/virtual/ > mv $domain1 $domain2 If I could figure out what the %256LRHu hash is, mv would probably be a very good solution.. > > And the login names are stored in a flatfile or db?? LDAP > Either way, you can do a rename. No, we need to keep the old username/password working, so that all users will get notified of the changed -- even if they take off on a 6 month vacation the day before the change. > > Maybe I still don't understand you:-) You seem to be understanding perfectly well. I've been looking myself blind on dsync mirror, when a simple "mv" probably will work just as well :-) -jf From robert at schetterer.org Thu Apr 19 17:45:34 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 16:45:34 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works In-Reply-To: <4F8EFD77.90103@schetterer.org> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> Message-ID: <4F90250E.3000200@schetterer.org> Am 18.04.2012 19:44, schrieb Robert Schetterer: > Am 18.04.2012 19:01, schrieb Timo Sirainen: >> On 17.4.2012, at 17.40, Robert Schetterer wrote: >> >>> doveadm acl debug >>> >>> shows on both versions >>> >>> Fatal: ACL shared dict iteration failed >> >> Doesn't it show any other error message before this? Either to terminal or to error log? What's your doveconf -n? > > > Hi Timo, > i didnt found more usefull errors before > all are like Debug: acl: No lookup right to mailbox: > but that istn true, i checked it in the filesystem > and tried new acls etc > ( and they working under 2.0.20 ) > > to me it looks like some acl check/list problem > perhaps depend on stuff like in i.e > http://hg.dovecot.org/dovecot-2.1/rev/65a75939ac2c > > as i said the old 2.0.20 works still fine > on the other server > the only change i did was with virtual and namespace inbox > to get special_use working > >> > this is from the dove version > > # 2.1.4 (85ad4baedd43): /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-37-server x86_64 Ubuntu 10.04.4 LTS > auth_cache_negative_ttl = 0 > auth_cache_size = 50 M > auth_debug = yes > auth_debug_passwords = yes > auth_master_user_separator = * > auth_mechanisms = plain login > auth_socket_path = /var/run/dovecot/auth-userdb > auth_verbose = yes > auth_verbose_passwords = plain > auth_worker_max_count = 300 > dict { > acl = mysql:/etc/dovecot/dovecot-dict-acl-sql.conf.ext > expire = mysql:/etc/dovecot/dovecot-dict-expire-sql.conf.ext > quotadict = mysql:/etc/dovecot/dovecot-dict-quota-sql.conf.ext > } > disable_plaintext_auth = no > first_valid_gid = 1001 > first_valid_uid = 1001 > hostname = mail01.example.com > last_valid_gid = 1001 > last_valid_uid = 1001 > listen = * > login_greeting = imap, pop ready > mail_access_groups = vmail > mail_debug = yes > mail_fsync = always > mail_gid = 1001 > mail_location = maildir:~/ > mail_nfs_index = yes > mail_nfs_storage = yes > mail_plugins = " stats zlib expire virtual fts fts_squat mail_log > autocreate notify acl listescape" > mail_privileged_group = vmail > mail_uid = 1001 > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date ihave imapflags notify > mmap_disable = yes > namespace { > list = yes > location = maildir:/usr/local/virtual/%%d/%%u:INDEX=~/shared/%%d/%%u > prefix = shared/%%u/ > separator = / > subscriptions = no > type = shared > } > namespace inbox { > hidden = no > list = yes > location = > mailbox Drafts { > auto = subscribe > special_use = \Drafts > } > mailbox Junk { > auto = subscribe > special_use = \Junk > } > mailbox Sent { > auto = subscribe > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > auto = subscribe > special_use = \Trash > } > mailbox name { > special_use = \Drafts \Junk \Sent \Trash > } > prefix = > separator = / > subscriptions = yes > } > namespace real { > hidden = yes > list = no > location = > prefix = RealMails/ > separator = / > } > namespace virtual { > hidden = yes > list = no > location = virtual:/etc/dovecot/virtual2:LAYOUT=maildir++:INDEX=~/virtual > prefix = virtual/ > separator = / > } > passdb { > args = /etc/dovecot/dovecot-sql-master.conf.ext > driver = sql > master = yes > pass = yes > } > passdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > plugin { > acl = vfile:/etc/dovecot/global-acls:cache_secs=300 > acl_shared_dict = proxy::acl > autocreate = Trash > autocreate2 = Junk > autocreate3 = Sent > autocreate4 = Drafts > autocreate5 = Templates > autocreate6 = Hostmaster-Backup > autocreate7 = archiv-backup-mailspooler > autosubscribe = Trash > autosubscribe2 = Junk > autosubscribe3 = Sent > autosubscribe4 = Drafts > autosubscribe5 = Templates > expire = Trash > expire2 = Trash/* > expire3 = Junk > expire4 = Junk/* > expire5 = Hostmaster-Backup > expire6 = Hostmaster-Backup/* > expire7 = archiv-backup-mailspooler/* > expire_dict = proxy::expire > fts = squat > home = /usr/local/virtual/%d/%u > mail_log_events = delete undelete expunge copy mailbox_delete > mailbox_rename > mail_log_fields = uid box msgid size > mail_log_group_events = yes > quota = dict:::proxy::quotadict > quota_warning = storage=95%% quota-warning 95 %u > quota_warning2 = storage=80%% quota-warning 80 %u > sieve = /usr/local/virtual/%d/%u/dovecot.sieve > sieve_before = /etc/dovecot/sieve/global.sieve > sieve_dir = /usr/local/virtual/%d/%u/sieve > sieve_execute_bin_dir = /usr/lib/dovecot/sieve-execute > sieve_execute_socket_dir = sieve-execute > sieve_extensions = +notify +imapflags > sieve_filter_bin_dir = /usr/lib/dovecot/sieve-filter > sieve_filter_socket_dir = sieve-filter > sieve_global_dir = /etc/dovecot/sieve/ > sieve_global_extensions = +vnd.dovecot.execute +vnd.dovecot.pipe > +vnd.dovecot.filter > sieve_global_path = /etc/dovecot/sieve/global.sieve > sieve_pipe_bin_dir = /usr/lib/dovecot/sieve-pipe > sieve_pipe_socket_dir = sieve-pipe > sieve_plugins = sieve_extprograms > stats_refresh = 30 secs > stats_track_cmds = yes > zlib_save = gz > zlib_save_level = 6 > } > postmaster_address = hostmaster at mail01.example.com > protocols = imap pop3 lmtp sieve > sendmail_path = /usr/lib/sendmail > service anvil { > client_limit = 1000 > } > service auth-worker { > user = root > } > service auth { > client_limit = 1000 > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0666 > user = postfix > } > unix_listener auth-userdb { > group = vmail > mode = 0600 > user = vmail > } > } > service dict { > extra_groups = vmail > unix_listener dict { > group = vmail > mode = 0660 > } > } > service imap-login { > inet_listener imap { > port = 143 > } > inet_listener imaps { > port = 993 > ssl = yes > } > process_min_avail = 8 > service_count = 1 > } > service imap-postlogin { > executable = script-login /usr/local/bin/postlogin_imap.sh > } > service imap { > executable = imap imap-postlogin > process_limit = 1024 > vsz_limit = 256 M > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0660 > user = postfix > } > vsz_limit = 256 M > } > service managesieve-login { > inet_listener { > address = 127.0.0.1 212.52.224.210 212.52.224.205 > port = 2000 > } > vsz_limit = 256 M > } > service pop3-login { > inet_listener pop3 { > port = 110 > } > inet_listener pop3s { > port = 995 > ssl = yes > } > } > service pop3-postlogin { > executable = script-login /usr/local/bin/postlogin_pop3.sh > } > service pop3 { > executable = pop3 pop3-postlogin > process_limit = 1024 > } > service quota-warning { > executable = script /usr/local/bin/quota-warning.sh > unix_listener quota-warning { > mode = 0666 > user = vmail > } > user = vmail > } > service stats { > fifo_listener stats-mail { > mode = 0600 > user = vmail > } > } > ssl_ca = ssl_cert = ssl_key = userdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > userdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > verbose_proctitle = yes > protocol lda { > mail_plugins = quota sieve virtual acl mail_log notify expire listescape > } > protocol imap { > imap_client_workarounds = delay-newmail > imap_id_log = * > imap_id_send = * > imap_idle_notify_interval = 2 mins > imap_logout_format = bytes=%i/%o > imap_max_line_length = 128 k > mail_max_userip_connections = 100 > mail_plugins = imap_acl acl fts fts_squat mail_log autocreate notify > expire virtual quota imap_quota zlib imap_zlib listescape stats imap_stats > } > protocol lmtp { > mail_plugins = quota sieve virtual acl fts fts_squat mail_log notify > expire listescape > } > protocol sieve { > mail_max_userip_connections = 100 > } > protocol pop3 { > mail_max_userip_connections = 100 > mail_plugins = expire virtual quota mail_log fts fts_squat notify > listescape > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > pop3_enable_last = yes > pop3_fast_size_lookups = yes > pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s > pop3_no_flag_updates = no > pop3_reuse_xuidl = no > pop3_uidl_format = %g > } > > > Hi Timo, i did some more tests in shared namespace prefix shared/%%u/ is not working shared/%%n/ is working i cant use that cause of mutiple domains naming convention ,users are user at domain.de etc -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Thu Apr 19 17:59:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 17:59:39 +0300 Subject: [Dovecot] migrate 15K users to new domain name In-Reply-To: <20120419141930.GB10069@dibs.tanso.net> References: <20120419082245.GA4050@dibs.tanso.net> <20120419132803.GA9396@dibs.tanso.net> <20120419141930.GB10069@dibs.tanso.net> Message-ID: On 19.4.2012, at 17.19, Jan-Frode Myklebust wrote: > mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln > > If I could figure out what the %256LRHu hash is, mv would probably be a > very good solution.. With v2.1.4 you could do something like: doveadm -c dummy.conf user -m user at domain where dummy.conf contains the minimum configuration needed: mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln ssl = no It still does a userdb lookup though, so either the user at domain needs to exist in userdb or you need to create a separate dummy Dovecot installation using userdb static that the dummy.conf uses. From tss at iki.fi Thu Apr 19 18:16:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 18:16:37 +0300 Subject: [Dovecot] doveadm delete folders ? In-Reply-To: <4F900066.9030808@schetterer.org> References: <4F900066.9030808@schetterer.org> Message-ID: <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> On 19.4.2012, at 15.09, Robert Schetterer wrote: > Hi , > is there a feature like > doveadm expunge savedbefore ... > for imap folder delete ? Here: http://hg.dovecot.org/dovecot-2.1/rev/2d8bafd11569 http://hg.dovecot.org/dovecot-2.1/rev/782570f644f7 From tss at iki.fi Thu Apr 19 18:23:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 18:23:22 +0300 Subject: [Dovecot] multiple Return-Path headers added using lmtp director In-Reply-To: <72F28E8A-EE19-48B9-AD36-0147DACD73B2@xs4all.nl> References: <72F28E8A-EE19-48B9-AD36-0147DACD73B2@xs4all.nl> Message-ID: On 19.4.2012, at 16.37, Cor Bosman wrote: > Emails arrive with 2 Return-Paths when using lmtp director. Is this something configurable in the director or is this a bug? Fixed: http://hg.dovecot.org/dovecot-2.1/rev/85b3975c9008 From zhbmaillistonly at gmail.com Thu Apr 19 18:30:17 2012 From: zhbmaillistonly at gmail.com (Zhang Huangbin) Date: Thu, 19 Apr 2012 23:30:17 +0800 Subject: [Dovecot] Custom LDAP filter for doveadm-mailbox? Message-ID: <1E42E3B7EAAF4EC4A4D8D70AC3AFB045@gmail.com> Dear Timo, With Dovecot-2.0.16, 'doveadm mailbox list -A' will perform LDAP search with filter "(objectClass=posixAccount)", and get value of attribute 'uid' as mail user. Is it possible to custom the ldap filter and returned attribute name? For example: - LDAP filter: (&(objectClass=mailUser)(accountStatus=active)) - Returned attribute name: mail Did i miss something? Thanks. ---- Zhang Huangbin iRedMail: Open Source Mail Server Solution for Red Hat Enterprise Linux, CentOS, Scientific Linux, Debian, Ubuntu, Mint, Gentoo, openSUSE, FreeBSD: http://www.iredmail.org/ From tss at iki.fi Thu Apr 19 18:41:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 18:41:28 +0300 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works In-Reply-To: <4F90250E.3000200@schetterer.org> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> <4F90250E.3000200@schetterer.org> Message-ID: <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> On 19.4.2012, at 17.45, Robert Schetterer wrote: > Hi Timo, > i did some more tests > in shared namespace > prefix > > shared/%%u/ is not working > > > shared/%%n/ is working > > i cant use that cause of mutiple domains > naming convention ,users are user at domain.de etc I can't really think of why that would make a difference. I did a few tests and couldn't reproduce the problem. Try if changing the location in the shared namespace makes a difference: namespace { list = yes location = maildir:%%h:INDEX=~/shared/%%d/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } BTW. You could remove autocreate plugin by moving the rest of the autocreates to mailbox { auto=create } settings. From tss at iki.fi Thu Apr 19 18:44:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 18:44:07 +0300 Subject: [Dovecot] alias_for In-Reply-To: References: Message-ID: <49931079-28AE-4A18-AE32-1F38735D7255@iki.fi> On 19.4.2012, at 2.11, Joseph Tam wrote: > I noticed the addition of a new namespace configuration "alias_for". > What pracical difference is adding this over and above making namespace > aliases hidden and disabling list, as in older versions of dovecot > configurations? It's used by doveadm and dsync to skip duplicate mailboxes. I've also been thinking that it could be made automatic by comparing which namespaces have identical "location" settings. From tss at iki.fi Thu Apr 19 18:44:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 18:44:56 +0300 Subject: [Dovecot] Custom LDAP filter for doveadm-mailbox? In-Reply-To: <1E42E3B7EAAF4EC4A4D8D70AC3AFB045@gmail.com> References: <1E42E3B7EAAF4EC4A4D8D70AC3AFB045@gmail.com> Message-ID: <41EDF5DE-49DF-4900-86CA-52F8A6B05717@iki.fi> On 19.4.2012, at 18.30, Zhang Huangbin wrote: > With Dovecot-2.0.16, 'doveadm mailbox list -A' will perform LDAP search with filter "(objectClass=posixAccount)", and get value of attribute 'uid' as mail user. Is it possible to custom the ldap filter and returned attribute name? For example: > > - LDAP filter: (&(objectClass=mailUser)(accountStatus=active)) > - Returned attribute name: mail > > Did i miss something? See iterate_* settings in example dovecot-ldap.conf.ext. From news1204.10.weezy at spamgourmet.com Wed Apr 18 20:09:58 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Wed, 18 Apr 2012 19:09:58 +0200 Subject: [Dovecot] namespace hierarchy In-Reply-To: References: <201204010928.41017.anyaddress@gmx.net> Message-ID: Am 04.04.2012 03:47, schrieb Timo Sirainen: > On 1.4.2012, at 10.28, Tom Fernandes wrote: > >> In the new server I would like to have 3 namespaces (private, public, shared) >> and use "/" as separator. Is it possible to keep "INBOX." for private, so that >> the users mailboxes appear below the Inbox, but have public and shared on the >> same hierarchy-level like the Inbox? >> > > If you have any subscriptions=no namespaces, you need a parent namespace that has subscriptions=yes. You could have e.g.: > > namespace { > prefix = INBOX/ > list = no > inbox = yes > } > namespace { > prefix = > list = no > hidden = yes > alias_for = INBOX/ > } > > So now the prefix="" is used for the subscriptions but nothing else really. I tried this, but sadly with dovecot V2.1.1 it is not working. Steps to reproduce: + create empty mail directory + create one folder (Allgemeines/Tabel) in public namespace with doveadm + start mail-client (in my case thunderbird) + try to subscribe to the new public folder Error message in log: Apr 18 18:19:34 elablnmail02 dovecot: imap(ahelmcke): Warning: Subscriptions file /home/mail/user/ahelmcke/subscriptions: Removing invalid entry: Allgemeines/Tabel doveconf -n: # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) ext4 auth_master_user_separator = * first_valid_uid = 400 mail_gid = vmail mail_location = maildir:/home/mail/user/%u mail_uid = vmail mbox_write_locks = fcntl namespace Allgemeines { location = maildir:/home/mail/Allgemeines:INDEX=/home/mail/user/%u/SEEN.Allgemeines prefix = Allgemeines/ separator = / subscriptions = no type = public } namespace dummy { alias_for = INBOX/ hidden = yes list = no location = prefix = separator = / type = private } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile sieve = /var/lib/dovecot/sieve/%u.sieve sieve_global_dir = /var/lib/dovecot/sieve/global/ sieve_global_path = /var/lib/dovecot/sieve/default.sieve } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } service imap-login { inet_listener imap { address = * } inet_listener imaps { address = * } } ssl = required ssl_cert = Hello, after finally having managed to setup up dovecot with virtual users and public folders I tried to get it connected to postfix. The problem is, that I can't get postix to deliver mail via dovecot-lda. After some testing it seems to come down to dovecot-lda behaving different when called as root then when called as user vmail. Basic setting: - dovecot 2.1.1 - all mailboxes owned by user vmail (using mail_uid and mail_gid) Test 1: as user root: /usr/libexec/dovecot/dovecot-lda -d ahelmcke < some_mail working as expected; some_mail gets stored in users INBOX Test 2: same command executed as user vmail. dovecot-lda terminates immediately. syslog-output with mail_debug = yes: Test 1: Apr 19 16:38:04 elablnmail02 dovecot: lda: Debug: auth input: ahelmcke Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: Effective uid=494, gid=491, home=/home/mail/user/ahelmcke Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=/home/mail/user/ahelmcke, alt= Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke/public, index=, control=, inbox=, alt= Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace Allgemeines: type=public, prefix=Allgemeines/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no location=maildir:/home/mail/Allgemeines:INDEX=/home/mail/user/ahelmcke/SEEN.Allgemeines Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/Allgemeines, index=/home/mail/user/ahelmcke/SEEN.Allgemeines, control=, inbox=, alt= Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: none: root=, index=, control=, inbox=, alt= Apr 19 16:51:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Destination address: ahelmcke at elablnmail02.ela-bln.local (source: user at hostname) Apr 19 16:51:37 elablnmail02 dovecot: lda(ahelmcke): msgid=unspecified: saved mail to INBOX Test 2: Apr 19 16:37:37 elablnmail02 dovecot: lda: Debug: auth input: ahelmcke Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Effective uid=494, gid=491, home=/home/mail/user/ahelmcke Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=/home/mail/user/ahelmcke, alt= Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke/public, index=, control=, inbox=, alt= Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, subscriptions=yes location= Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mdbox: access(/home/mail/user/ahelmcke/mdbox, rwx): failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mdbox: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: sdbox: access(/home/mail/user/ahelmcke/sdbox, rwx): failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: sdbox: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir: access(/home/mail/user/ahelmcke/Maildir, rwx): failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has .imap/: stat(/home/mail/user/ahelmcke/mail/.imap) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has inbox: stat(/home/mail/user/ahelmcke/mail/inbox) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has mbox: stat(/home/mail/user/ahelmcke/mail/mbox) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has .imap/: stat(/home/mail/user/ahelmcke/Mail/.imap) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has inbox: stat(/home/mail/user/ahelmcke/Mail/inbox) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has mbox: stat(/home/mail/user/ahelmcke/Mail/mbox) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: sdbox: access(/home/mail/user/ahelmcke/sdbox, rwx): failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: sdbox: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Error: user ahelmcke: Initialization failed: Namespace '': Mail storage autodetection failed with home=/home/mail/user/ahelmcke Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Fatal: Invalid user settings. Refer to server log for more information. doveconf -n # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) auth_master_user_separator = * auth_verbose = yes first_valid_uid = 400 mail_debug = yes mail_gid = vmail mail_home = /home/mail/user/%u mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl namespace Allgemeines { location = maildir:/home/mail/Allgemeines:INDEX=/home/mail/user/%u/SEEN.Allgemeines prefix = Allgemeines/ separator = / subscriptions = no type = public } namespace dummy { alias_for = INBOX/ hidden = yes list = no location = maildir:/home/mail/user/%u/public/ prefix = separator = / type = private } namespace inbox { inbox = yes location = maildir:/home/mail/user/%u prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile sieve = /home/mail/user/%u/sieve.active sieve_dir = /home/mail/user/%u/sieve.scripts sieve_global_dir = /var/lib/dovecot/sieve/global/ sieve_global_path = /var/lib/dovecot/sieve/default.sieve } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = * } inet_listener imaps { address = * } } service managesieve-login { inet_listener sieve { address = * port = 4190 } } ssl = required ssl_cert = References: <1E42E3B7EAAF4EC4A4D8D70AC3AFB045@gmail.com> <41EDF5DE-49DF-4900-86CA-52F8A6B05717@iki.fi> Message-ID: <16B8C19B02364F77908C1521F76E4AA8@gmail.com> On Thursday, April 19, 2012 at 11:44 PM, Timo Sirainen wrote: > See iterate_* settings in example dovecot-ldap.conf.ext. It works as expected. Thanks. :) ---- Zhang Huangbin iRedMail: Open Source Mail Server Solution for Red Hat Enterprise Linux, CentOS, Scientific Linux, Debian, Ubuntu, Mint, Gentoo, openSUSE, FreeBSD: http://www.iredmail.org/ From tss at iki.fi Thu Apr 19 18:53:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 19 Apr 2012 18:53:32 +0300 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: References: Message-ID: On 19.4.2012, at 17.56, Andreas Helmcke wrote: > namespace inbox { > inbox = yes > location = maildir:/home/mail/user/%u > prefix = INBOX/ > separator = / > type = private > } Move the location from the above namespace to global: mail_location = maildir:/home/mail/user/%u Although it probably still shouldn't have failed.. From news1204.10.weezy at spamgourmet.com Thu Apr 19 19:09:52 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Thu, 19 Apr 2012 18:09:52 +0200 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: References: Message-ID: <2vp569xe2o.ln2@mail.ela-soft.com> Am 19.04.2012 17:53, schrieb Timo Sirainen: > On 19.4.2012, at 17.56, Andreas Helmcke wrote: > >> namespace inbox { >> location = maildir:/home/mail/user/%u >> .... > > Move the location from the above namespace to global: > > mail_location = maildir:/home/mail/user/%u Actually it /had/ been there and I moved it to inbox while trying to find the problem. So, change reverted (excerpt from doveconf -n): mail_location = maildir:/home/mail/user/%u namespace inbox { inbox = yes location = prefix = INBOX/ separator = / type = private } Now dovecot-lda is complaining about duplicate namespaces (still only when run as vmail; run it as root and it is working): Apr 19 17:58:07 elablnmail02 dovecot: lda: Debug: auth input: ahelmcke Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Effective uid=494, gid=491, home=/home/mail/user/ahelmcke Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=/home/mail/user/ahelmcke, alt= Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke/public, index=, control=, inbox=, alt= Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=, alt= Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Error: user ahelmcke: Initialization failed: namespace configuration error: Duplicate namespace prefix: "" Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Fatal: Invalid user settings. Refer to server log for more information. Greetings Andreas From robert at schetterer.org Thu Apr 19 19:30:57 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 18:30:57 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works In-Reply-To: <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> <4F90250E.3000200@schetterer.org> <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> Message-ID: <4F903DC1.5070501@schetterer.org> Am 19.04.2012 17:41, schrieb Timo Sirainen: > On 19.4.2012, at 17.45, Robert Schetterer wrote: > >> Hi Timo, >> i did some more tests >> in shared namespace >> prefix >> >> shared/%%u/ is not working >> >> >> shared/%%n/ is working >> >> i cant use that cause of mutiple domains >> naming convention ,users are user at domain.de etc > > I can't really think of why that would make a difference. I did a few tests and couldn't reproduce the problem. Try if changing the location in the shared namespace makes a difference: > > namespace { > list = yes > location = maildir:%%h:INDEX=~/shared/%%d/%%u hi Timo, > prefix = shared/%%u/ is not working !!! under 2.1 / it works under 2.0.20 prefix = shared/%%n/ is working but then i see only users of my domain acl shared with their user part of i.e user at domain.de --shared | user ( of my own domain ) i cant use this because i have to share cross domain and users may have same names in different domains what i need is --shared | user at domainwhatever.de > separator = / > subscriptions = no > type = shared > } > > BTW. You could remove autocreate plugin by moving the rest of the autocreates to mailbox { auto=create } settings. i know this nevertheless i try location change -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From almarzuki2001 at hotmail.com Thu Apr 19 21:10:33 2012 From: almarzuki2001 at hotmail.com (Hadi Salem) Date: Thu, 19 Apr 2012 21:10:33 +0300 Subject: [Dovecot] dovecot LDA with sendmail Message-ID: Hi, Im configuring sendmail with dovecot for virtual users using password file and file for user name. on centos. dovecot-1.0.7-7.el5 sendmail-8.13.8-2.el5 dovecot ?n # 1.0.7: /etc/dovecot.conf base_dir: /var/run/dovecot/ log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot.log ssl_disable: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login default_mail_env: maildir:/home/vmail/%d/%n mail_location: maildir:/home/vmail/%d/%n mail_debug: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib64/dovecot/imap mail_plugin_dir(imap): /usr/lib64/dovecot/imap mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3 auth default: verbose: yes debug: yes debug_passwords: yes passdb: driver: passwd-file args: /etc/dovecot/passwd userdb: driver: passwd-file args: /etc/dovecot/users socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: root group: root I have also added dovecot LDA for sendmail /usr/share/sendmail-cf/mailer/dovecot.m4 ######################*****############## ### DOVECOT Mailer specification ### ##################*****################## Mdovecot, P=/usr/libexec/dovecot/deliver, F=DFMPhnu9,S=EnvFromSMTP/HdrFromSMTP, R=EnvToSMTP/HdrFromSMTP,T=DNS/RFC822/X-Unix,A=/usr/libexec/dovecot/deliver -d $u And also [root at host1 mail]# cat /etc/mail/mailertable example.com dovecot:dovecot however when im sending email to the virtual user im getting this error, Apr 19 20:52:13 host1 sendmail[22311]: q3JHqDAR022309: to=, ctladdr= (0/0), delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120292, relay=dovecot, dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/libexec/dovecot/deliver) exited with EX_TEMPFAIL Dovecot LDA can?t deliver the mail to the user!!! From robert at schetterer.org Thu Apr 19 21:36:28 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 20:36:28 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works In-Reply-To: <4F903DC1.5070501@schetterer.org> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> <4F90250E.3000200@schetterer.org> <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> <4F903DC1.5070501@schetterer.org> Message-ID: <4F905B2C.8040204@schetterer.org> Am 19.04.2012 18:30, schrieb Robert Schetterer: > Am 19.04.2012 17:41, schrieb Timo Sirainen: >> On 19.4.2012, at 17.45, Robert Schetterer wrote: >> >>> Hi Timo, >>> i did some more tests >>> in shared namespace >>> prefix >>> >>> shared/%%u/ is not working >>> >>> >>> shared/%%n/ is working >>> >>> i cant use that cause of mutiple domains >>> naming convention ,users are user at domain.de etc >> >> I can't really think of why that would make a difference. I did a few tests and couldn't reproduce the problem. Try if changing the location in the shared namespace makes a difference: >> >> namespace { >> list = yes >> location = maildir:%%h:INDEX=~/shared/%%d/%%u > > hi Timo, > >> prefix = shared/%%u/ > is not working !!! under 2.1 / it works under 2.0.20 > > prefix = shared/%%n/ > is working > > > > but then i see only users of my domain acl shared > with their user part of i.e user at domain.de > > --shared > | > user ( of my own domain ) > > i cant use this because i have to share cross domain > and users may have same names in different domains > > > what i need is > > --shared > | > user at domainwhatever.de > >> separator = / >> subscriptions = no >> type = shared >> } >> > >> BTW. You could remove autocreate plugin by moving the rest of the autocreates to mailbox { auto=create } settings. > i know this > > nevertheless i try location change > just for info location change to location = maildir:%%h:INDEX=~/shared/%%d/%%u does not lead to get it work prefix = shared/%%u/ simply dont works i have reverted back to exact namespace config of the 2.0.20 server, same result, so something broken in 2.1 -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Thu Apr 19 23:00:40 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 22:00:40 +0200 Subject: [Dovecot] doveadm delete folders ? In-Reply-To: <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> References: <4F900066.9030808@schetterer.org> <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> Message-ID: <4F906EE8.1030900@schetterer.org> Am 19.04.2012 17:16, schrieb Timo Sirainen: > On 19.4.2012, at 15.09, Robert Schetterer wrote: > >> Hi , >> is there a feature like >> doveadm expunge savedbefore ... >> for imap folder delete ? > > Here: > > http://hg.dovecot.org/dovecot-2.1/rev/2d8bafd11569 > http://hg.dovecot.org/dovecot-2.1/rev/782570f644f7 > thx Timo ! -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Thu Apr 19 23:27:21 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Apr 2012 22:27:21 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works / solved by remove listescape In-Reply-To: <4F905B2C.8040204@schetterer.org> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> <4F90250E.3000200@schetterer.org> <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> <4F903DC1.5070501@schetterer.org> <4F905B2C.8040204@schetterer.org> Message-ID: <4F907529.2020403@schetterer.org> > > prefix = shared/%%u/ simply dont works > > i have reverted back to exact namespace config > of the 2.0.20 server, same result, so something broken in 2.1 > > Hi Timo problem solved by removing listescape plugin the acl paths did / instead of dot in domainname any chance to get listescape fixed ? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From stan at hardwarefreak.com Fri Apr 20 03:31:13 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 19 Apr 2012 19:31:13 -0500 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <20120417200820.GA30743@dibs.tanso.net> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <20120417124241.GA23120@dibs.tanso.net> <4F8D6BAA.7060501@gtdinternet.com> <20120417200820.GA30743@dibs.tanso.net> Message-ID: <4F90AE51.8080407@hardwarefreak.com> On 4/17/2012 3:08 PM, Jan-Frode Myklebust wrote: > Our struggle is the number of > iops we're able to get from the backend storage (IBM DS4800), mostly > a problem when we have storms of incoming marketing messages in addition > to the pop/imap traffic. This issue has come up twice on the Postfix list in less than a month. You can fix this specific problem very easily. Only marketing servers and busy/misconfigured list servers make many parallel connections to your MX hosts. Allowing them to blast all those messages over parallel connections is what bogs down your spool storage. The fix is simple: limit all SMTP clients to a small number of parallel connections. This will slow down marketing and list server blasts without affecting normal sending MTAs. To do so, add this to /etc/postfix/main.cf: smtpd_client_connection_count_limit = 4* The default Postfix process limit is 100. The concurrent connection limit is 1/2 the process limit, so 50 parallel connections per client IP are allowed by default. If remote hosts also do connection caching, they can force feed your MTA many hundreds of messages/sec. Limiting concurrent connections will decrease their mail rate to a small fraction of what you're seeing now, reducing IOPS load on your spool storage significantly. * This is a good starting point. You may need to tweak it up a little bit. Some list servers (such as XFS) will unsub members if their multiple connections keep getting refused, so tweak this value until you find your sweet spot. -- Stan From dchenusa at yahoo.com Fri Apr 20 03:38:01 2012 From: dchenusa at yahoo.com (Dennis Chen) Date: Thu, 19 Apr 2012 17:38:01 -0700 Subject: [Dovecot] Fwd: IMAP SSL incoming test need help ! References: <1334876809.51362.YahooMailNeo@web161602.mail.bf1.yahoo.com> Message-ID: <82BE5266-7A1D-4E1F-9862-F503668D6EAF@yahoo.com> Sent from my iPhone Begin forwarded message: > From: D Chen > Date: April 19, 2012 4:06:49 PM PDT > To: lists-dovecot > Subject: Re: [Dovecot] IMAP SSL incoming test need help ! > Reply-To: D Chen > > Thanks for nice pointing out in my mail.log! > > I could not find where is the procmail log located under /var/log ! > > As procmail is "suspicious", I recalled that I selected either use procmail or not use for local delivery while I re-configure the postfix. I ran "sudo dpkg-reconfigure postfix" again, and selected NOT use "Procmail" for local delivery, restart postfix and ran the tested again, it worked now ! examined the mail.log, it said "delivered to maildir" instead of procmail. > > why works NOT using procmail ? thx. > > From: lists-dovecot > To: D Chen > Sent: Thursday, April 19, 2012 5:22 AM > Subject: Re: [Dovecot] IMAP SSL incoming test need help ! > > > > ------------ Original Message ------------ > > Date: Thursday, April 19, 2012 01:44:35 AM -0700 > > From: D Chen > > To: Dovecot Mailing List > > Subject: [Dovecot] IMAP SSL incoming test need help ! > > > > Ubuntu 11.10 server with postfix/dovecot/squirrelmail configured. > > > > Outgoing SMTP server seemed worked fine, however, the incoming > > IMAP with SSL on port 993 doesn't seem to receive any mail at all > > !? Here is the mail.log during the test session from an external > > yahoo mail (dchenusa at yahoo.com) to the internal server > > testmail.biokeyinc.com (dchen at testmail.biokeyinc.com) The log did > > NOT show any error ! The sender showed sent mail! but no mail > > reach to the destination! Any help would be appreciated. > > > > Apr 19 01:20:02 testmail postfix/smtpd[4103]: connect from > > nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:03 > > testmail postfix/smtpd[4103]: 091ED200973: > > client=nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 > > 01:20:03 testmail postfix/cleanup[4108]: 091ED200973: > > message-id=<32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com> Apr 19 > > 01:20:03 testmail postfix/qmgr[2007]: 091ED200973: > > from=, size=2840, nrcpt=1 (queue active) Apr > > 19 01:20:03 testmail postfix/smtpd[4103]: disconnect from > > nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:12 > > testmail postfix/smtpd[4113]: connect from localhost[127.0.0.1] > > Apr 19 01:20:12 testmail postfix/smtpd[4113]: 33AE9200AE9: > > client=localhost[127.0.0.1] Apr 19 01:20:12 testmail > > postfix/cleanup[4108]: 33AE9200AE9: > > message-id=<32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com> Apr 19 > > 01:20:12 testmail postfix/qmgr[2007]: 33AE9200AE9: > > from=, size=3536, nrcpt=1 (queue active) Apr > > 19 01:20:12 testmail postfix/smtpd[4113]: disconnect from > > localhost[127.0.0.1] Apr 19 01:20:12 testmail amavis[2042]: > > (02042-06) Passed CLEAN, [98.138.91.180] [98.138.87.4] > > -> , > > Message-ID: <32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com>, > > mail_id: rqyVZiQDwSsG, Hits: -0.109, size: 2837, queued_as: > > 33AE9200AE9, dkim_id=dchenusa at yahoo.com, at yahoo.com, 8647 ms Apr 19 > > 01:20:12 testmail postfix/smtp[4109]: 091ED200973: > > to=, > > relay=127.0.0.1[127.0.0.1]:10024, delay=9.5, > > delays=0.87/0.01/0/8.6, dsn=2.0.0, status=sent (250 2.0.0 from > > MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 33AE9200AE9) Apr > > 19 01:20:12 testmail postfix/qmgr[2007]: 091ED200973: removed Apr > > 19 01:20:12 testmail postfix/local[4114]: 33AE9200AE9: > > to=, relay=local, delay=0.18, > > delays=0.1/0/0/0.07, dsn=2.0.0, status=sent (delivered to command: > > procmail -a "$EXTENSION") Apr 19 01:20:12 testmail > > postfix/qmgr[2007]: 33AE9200AE9: removed Apr 19 01:21:30 testmail > > dovecot: imap-login: Login: user=, method=PLAIN, > > rip=63.195.90.22, lip=192.168.20.100, mpid=4122, TLS Apr 19 > > 01:23:13 testmail dovecot: imap(dchen): Disconnected: Disconnected > > in IDLE bytes=99/708 Apr 19 01:23:23 testmail postfix/anvil[4105]: > > statistics: max connection rate 1/60s for (smtp:98.138.91.180) at > > Apr 19 01:20:02 Apr 19 01:23:23 testmail postfix/anvil[4105]: > > statistics: max connection count 1 for (smtp:98.138.91.180) at Apr > > 19 01:20:02 Apr 19 01:23:23 testmail postfix/anvil[4105]: > > statistics: max cache size 1 at Apr 19 01:20:02 > > ------------ End Original Message ------------ > > > I'm going to bet that procmail is eating the inbound message: > > Apr 19 01:20:12 testmail postfix/local[4114]: 33AE9200AE9: > to=, relay=local, delay=0.18, > delays=0.1/0/0/0.07, dsn=2.0.0, status=sent (delivered to command: > > procmail -a "$EXTENSION") > > and it's not getting to the user's inbox. If a message doesn't get > to the user's inbox then it's not dovecot's issue that it can't be > retrieved. > > fyi -- imap/dovecot have nothing to do your inbound delivery. > > > however, the incoming IMAP with SSL on port 993 doesn't > > seem to receive any mail > > that's handled by your MTA, which in your case is postfix, with > procmail in the mix. > > I would try looking at the procmail log to see what's happening > there. > > - Richard > > > > > > > > > > > From jtam.home at gmail.com Fri Apr 20 04:45:38 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Thu, 19 Apr 2012 18:45:38 -0700 (PDT) Subject: [Dovecot] dovecot LDA with sendmail In-Reply-To: References: Message-ID: Hadi Salem writes: > Im configuring sendmail with dovecot for virtual users using > password file and file for user name. on centos. > > dovecot-1.0.7-7.el5 Maybe I can beat Charles Marcus to the punch and recommend you upgrade to the latest version, otherwise you'll run into bugs that have already been fixed. > Mdovecot, > P=/usr/libexec/dovecot/deliver, F=DFMPhnu9,S=EnvFromSMTP/HdrFromSMTP, > R=EnvToSMTP/HdrFromSMTP,T=DNS/RFC822/X-Unix,A=/usr/libexec/dovecot/deliver -d > $u > ... > Apr 19 20:52:13 host1 sendmail[22311]: q3JHqDAR022309: > to=, ctladdr= (0/0), > delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120292, relay=dovecot, > dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/libexec/dovecot/deliver) exited > with EX_TEMPFAIL Is there any logs from deliver? If not, look at this page to configure logging http://wiki.dovecot.org/LDA That's your best chance at finding out the problem. I don't have virtual users, so I can't venture to say whether this is part of the problem. I know I had to fiddle with the mailer flags (you have "DFMPhnu9") to get my setup working, but I don't remember what I did or why. Try adding the "S" flag and see if that helps. Joseph Tam From dovecot-user at spambox.dk Fri Apr 20 10:40:02 2012 From: dovecot-user at spambox.dk (Henrik Larsson) Date: Fri, 20 Apr 2012 08:40:02 +0100 Subject: [Dovecot] 2.1.3 Corrupted squat uidlist In-Reply-To: <4F8A97F8.70808@spambox.dk> References: <4F7164DC.7010706@unict.it> <4F8A97F8.70808@spambox.dk> Message-ID: <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> This problem is still showing up every day in the logs. I tried to delete the dovecot.index.search and dovecot.index.search.uids files for all users, but the errors are still there after a re-build of the search index with "/usr/local/bin/doveadm search -A TEXT xyzabczzz". If you need any further details, please let me know. Best regards Henrik Larsson On 15-04-2012 10:42, Henrik Larsson wrote: > On 27-03-2012 08:57, Luca Palazzo wrote: >> Hi Timo and All, >> after upgrading to 2.1.2 i'm getting a lot of these messages: >> Error: Corrupted squat uidlist file XXXXXX wrong indexid > After an upgrade to 2.1.3 i see this as well. Any thoughts? > > Apr 15 03:43:43 imap(xxxx): Error: Corrupted squat uidlist file > /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong > indexid > Apr 15 06:30:27 imap(xxxx): Error: Corrupted squat uidlist file > /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong > indexid > Apr 15 07:43:55 pop3(xxxx): Error: Corrupted squat uidlist file > /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong > indexid > Apr 15 09:00:01 imap(xxxx): Error: Corrupted squat uidlist file > /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong > indexid > > # doveconf -n > # 2.1.3: /usr/local/etc/dovecot/dovecot.conf > # OS: FreeBSD 8.2-STABLE amd64 > auth_mechanisms = plain login digest-md5 cram-md5 > first_valid_uid = 125 > listen = * > log_path = /var/log/dovecot > mail_plugins = fts fts_squat zlib > mail_privileged_group = postfix > mail_temp_dir = /var/db/dovecot > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date ihave > namespace { > inbox = yes > location = > prefix = > separator = . > type = private > } > namespace { > hidden = yes > inbox = no > list = no > location = > prefix = INBOX. > separator = . > type = private > } > passdb { > args = /usr/local/etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > plugin { > fts = squat > fts_squat = partial=4 full=10 > sieve = ~/.dovecot.sieve > sieve_dir = ~/sieve > } > protocols = imap pop3 lmtp sieve > service auth-worker { > user = $default_internal_user > } > service auth { > unix_listener /home/mail/postfix/private/dovecot-auth { > group = postfix > mode = 0660 > user = postfix > } > } > service lmtp { > executable = lmtp -L > unix_listener /home/mail/postfix/private/dovecot-lmtp { > group = postfix > mode = 0660 > user = postfix > } > } > ssl_cert = ssl_key = userdb { > args = /usr/local/etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > protocol lmtp { > log_path = /var/log/dovecot-deliver > mail_plugins = fts fts_squat zlib sieve > } > protocol lda { > mail_plugins = fts fts_squat zlib sieve > } > protocol imap { > mail_plugins = fts fts_squat zlib imap_zlib > } > > > Best regards > Henrik Larsson From dchenusa at yahoo.com Fri Apr 20 11:29:31 2012 From: dchenusa at yahoo.com (Dennis Chen) Date: Fri, 20 Apr 2012 01:29:31 -0700 Subject: [Dovecot] Fwd: IMAP SSL incoming test need help ! References: <82BE5266-7A1D-4E1F-9862-F503668D6EAF@yahoo.com> Message-ID: <485DD6C9-EFCB-4644-9BFA-D0C0CCA224EA@yahoo.com> Sent from my iPhone Begin forwarded message: > From: Dennis Chen > Date: April 19, 2012 5:38:01 PM PDT > To: dovecot at dovecot.org > Subject: Fwd: [Dovecot] IMAP SSL incoming test need help ! > > > > Sent from my iPhone > > Begin forwarded message: > >> From: D Chen >> Date: April 19, 2012 4:06:49 PM PDT >> To: lists-dovecot >> Subject: Re: [Dovecot] IMAP SSL incoming test need help ! >> Reply-To: D Chen >> > >> Thanks for nice pointing out in my mail.log! >> >> I could not find where is the procmail log located under /var/log ! >> >> As procmail is "suspicious", I recalled that I selected either use procmail or not use for local delivery while I re-configure the postfix. I ran "sudo dpkg-reconfigure postfix" again, and selected NOT use "Procmail" for local delivery, restart postfix and ran the tested again, it worked now ! examined the mail.log, it said "delivered to maildir" instead of procmail. >> >> why works NOT using procmail ? thx. >> >> From: lists-dovecot >> To: D Chen >> Sent: Thursday, April 19, 2012 5:22 AM >> Subject: Re: [Dovecot] IMAP SSL incoming test need help ! >> >> >> >> ------------ Original Message ------------ >> > Date: Thursday, April 19, 2012 01:44:35 AM -0700 >> > From: D Chen >> > To: Dovecot Mailing List >> > Subject: [Dovecot] IMAP SSL incoming test need help ! >> > >> > Ubuntu 11.10 server with postfix/dovecot/squirrelmail configured. >> > >> > Outgoing SMTP server seemed worked fine, however, the incoming >> > IMAP with SSL on port 993 doesn't seem to receive any mail at all >> > !? Here is the mail.log during the test session from an external >> > yahoo mail (dchenusa at yahoo.com) to the internal server >> > testmail.biokeyinc.com (dchen at testmail.biokeyinc.com) The log did >> > NOT show any error ! The sender showed sent mail! but no mail >> > reach to the destination! Any help would be appreciated. >> > >> > Apr 19 01:20:02 testmail postfix/smtpd[4103]: connect from >> > nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:03 >> > testmail postfix/smtpd[4103]: 091ED200973: >> > client=nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 >> > 01:20:03 testmail postfix/cleanup[4108]: 091ED200973: >> > message-id=<32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com> Apr 19 >> > 01:20:03 testmail postfix/qmgr[2007]: 091ED200973: >> > from=, size=2840, nrcpt=1 (queue active) Apr >> > 19 01:20:03 testmail postfix/smtpd[4103]: disconnect from >> > nm20-vm4.bullet.mail.ne1.yahoo.com[98.138.91.180] Apr 19 01:20:12 >> > testmail postfix/smtpd[4113]: connect from localhost[127.0.0.1] >> > Apr 19 01:20:12 testmail postfix/smtpd[4113]: 33AE9200AE9: >> > client=localhost[127.0.0.1] Apr 19 01:20:12 testmail >> > postfix/cleanup[4108]: 33AE9200AE9: >> > message-id=<32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com> Apr 19 >> > 01:20:12 testmail postfix/qmgr[2007]: 33AE9200AE9: >> > from=, size=3536, nrcpt=1 (queue active) Apr >> > 19 01:20:12 testmail postfix/smtpd[4113]: disconnect from >> > localhost[127.0.0.1] Apr 19 01:20:12 testmail amavis[2042]: >> > (02042-06) Passed CLEAN, [98.138.91.180] [98.138.87.4] >> > -> , >> > Message-ID: <32C9C558-95DA-417D-8687-41ABE1296D61 at yahoo.com>, >> > mail_id: rqyVZiQDwSsG, Hits: -0.109, size: 2837, queued_as: >> > 33AE9200AE9, dkim_id=dchenusa at yahoo.com, at yahoo.com, 8647 ms Apr 19 >> > 01:20:12 testmail postfix/smtp[4109]: 091ED200973: >> > to=, >> > relay=127.0.0.1[127.0.0.1]:10024, delay=9.5, >> > delays=0.87/0.01/0/8.6, dsn=2.0.0, status=sent (250 2.0.0 from >> > MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 33AE9200AE9) Apr >> > 19 01:20:12 testmail postfix/qmgr[2007]: 091ED200973: removed Apr >> > 19 01:20:12 testmail postfix/local[4114]: 33AE9200AE9: >> > to=, relay=local, delay=0.18, >> > delays=0.1/0/0/0.07, dsn=2.0.0, status=sent (delivered to command: >> > procmail -a "$EXTENSION") Apr 19 01:20:12 testmail >> > postfix/qmgr[2007]: 33AE9200AE9: removed Apr 19 01:21:30 testmail >> > dovecot: imap-login: Login: user=, method=PLAIN, >> > rip=63.195.90.22, lip=192.168.20.100, mpid=4122, TLS Apr 19 >> > 01:23:13 testmail dovecot: imap(dchen): Disconnected: Disconnected >> > in IDLE bytes=99/708 Apr 19 01:23:23 testmail postfix/anvil[4105]: >> > statistics: max connection rate 1/60s for (smtp:98.138.91.180) at >> > Apr 19 01:20:02 Apr 19 01:23:23 testmail postfix/anvil[4105]: >> > statistics: max connection count 1 for (smtp:98.138.91.180) at Apr >> > 19 01:20:02 Apr 19 01:23:23 testmail postfix/anvil[4105]: >> > statistics: max cache size 1 at Apr 19 01:20:02 >> >> ------------ End Original Message ------------ >> >> >> I'm going to bet that procmail is eating the inbound message: >> >> Apr 19 01:20:12 testmail postfix/local[4114]: 33AE9200AE9: >> to=, relay=local, delay=0.18, >> delays=0.1/0/0/0.07, dsn=2.0.0, status=sent (delivered to command: >> >> procmail -a "$EXTENSION") >> >> and it's not getting to the user's inbox. If a message doesn't get >> to the user's inbox then it's not dovecot's issue that it can't be >> retrieved. >> >> fyi -- imap/dovecot have nothing to do your inbound delivery. >> >> > however, the incoming IMAP with SSL on port 993 doesn't >> > seem to receive any mail >> >> that's handled by your MTA, which in your case is postfix, with >> procmail in the mix. >> >> I would try looking at the procmail log to see what's happening >> there. >> >> - Richard >> >> >> >> >> >> >> >> >> >> >> From john.robinson at anonymous.org.uk Fri Apr 20 12:01:45 2012 From: john.robinson at anonymous.org.uk (John Robinson) Date: Fri, 20 Apr 2012 10:01:45 +0100 Subject: [Dovecot] mbox folders on-disc layout In-Reply-To: <35C2B470-438E-4CEC-A42A-0BC2C39A0159@iki.fi> References: <4F8EA753.4070605@anonymous.org.uk> <35C2B470-438E-4CEC-A42A-0BC2C39A0159@iki.fi> Message-ID: <4F9125F9.6060601@anonymous.org.uk> On 18/04/2012 17:45, Timo Sirainen wrote: > On 18.4.2012, at 14.36, John Robinson wrote: > >> I'd like to change the on-disk layout for my mboxes from being >> ~/mail/foo/bar >> to >> ~/mail/foo.bar >> so that I can have folders containing both messages and subfolders, without having subfolders beginning with . and then having to do all the other fiddling with locations of index files etc. that goes along with LAYOUT=maildir++, and also leaving the filesystem layout looking sensible so my users see their mailboxes instead of the index and control files. >> >> Is this possible or do I need to patch something? > > Not possible without patching. There's also the other possibility of doing something like: > > # Trick mbox configuration which allows a mail folder which contains both > # messages and sub-folders > mail_location = mbox:~/mail/mailboxes:DIRNAME=mBoX-MeSsAgEs:INDEX=~/mail/index:CONTROL=~/mail/control > > from http://wiki2.dovecot.org/MboxChildFolders Yes, I read all that, and thought all the options were pretty ugly - and the comment that it's a "trick" configuration would seem to suggest that the original author thought so to. And I think Maildir++ is pretty ugly too. If I wanted to try writing a new LAYOUT, where should I begin? Cheers, John. From CMarcus at Media-Brokers.com Fri Apr 20 14:07:57 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 20 Apr 2012 07:07:57 -0400 Subject: [Dovecot] dovecot LDA with sendmail In-Reply-To: References: Message-ID: <4F91438D.3040200@Media-Brokers.com> On 2012-04-19 9:45 PM, Joseph Tam wrote: > Hadi Salem writes: >> Im configuring sendmail with dovecot for virtual users using >> password file and file for user name. on centos. >> dovecot-1.0.7-7.el5 > Maybe I can beat Charles Marcus to the punch and recommend you upgrade > to the latest version, otherwise you'll run into bugs that have already > been fixed. Lol... I stopped reading at '...sendmail with dovecot...', otherwise I'd have beat you to it... ;) -- Best regards, Charles From news1204.10.weezy at spamgourmet.com Fri Apr 20 14:11:09 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Fri, 20 Apr 2012 13:11:09 +0200 Subject: [Dovecot] default mail quota when using per user quota Message-ID: Hello, using dovecot 2.1 and per user mail quota via ldap is there a way to have a default quota which gets used, if the "quota" field in ldap is not set? I tried with: plugin { quota = maildir:User quota quota_rule = *:storage=3G } userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } and in dovecot-ldap.conf: user_attrs = dcMailQuota=quota_rule=*:storage=%$ but with this configuration the mail quota /allways/ gets loaded from ldap resulting in users without dcMailQuota attribute having no quota at all. Greetings Andreas From nmilas at noa.gr Fri Apr 20 15:26:51 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Fri, 20 Apr 2012 15:26:51 +0300 Subject: [Dovecot] default mail quota when using per user quota In-Reply-To: References: Message-ID: <4F91560B.5040009@noa.gr> On 20/4/2012 2:11 ??, Andreas Helmcke wrote: > using dovecot 2.1 and per user mail quota via ldap is there a way to have a default quota which gets used, if the > "quota" field in ldap is not set? You may see: http://www.mail-archive.com/dovecot at dovecot.org/msg44010.html Regards, Nick From campbell at cnpapers.com Fri Apr 20 15:30:27 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Fri, 20 Apr 2012 08:30:27 -0400 Subject: [Dovecot] Error in logs indicating broken files Message-ID: <4F9156E3.4050801@cnpapers.com> Still new to dovecot, I'm seeing a couple of errors in my log file indicating corruption of index files. Error: Corrupted index cache file/home/xxxxxx/mail/.imap/sent-mail/dovecot.index.cache: Broken virtual size for mail UID 67: 1 Time(s) Error: FETCH [] for mailbox ~/mail/sent-mail UID 67 got too little data: 6203 vs 7478: 1 Time(s) This account is used by multiple users, but it is not a shared account (haven't set any of those up yet - still new to me), so I would expect corruption possibilities when multiple users modify the account folders in some manner. Until I get some experience under my belt and make this a truly "shared" account, is there any way to resolve the corruption of the files mentioned above? Maybe just delete them? thanks steve campbell From ml at smtp.fakessh.eu Fri Apr 20 15:25:01 2012 From: ml at smtp.fakessh.eu (ml) Date: Fri, 20 Apr 2012 14:25:01 +0200 Subject: [Dovecot] 2.1.3 Corrupted squat uidlist In-Reply-To: <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> References: <4F7164DC.7010706@unict.it> <4F8A97F8.70808@spambox.dk> <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> Message-ID: Le 2012-04-20 09:40, Henrik Larsson a ?crit?: > This problem is still showing up every day in the logs. > > I tried to delete the dovecot.index.search and > dovecot.index.search.uids files for all users, but the errors are > still there after a re-build of the search index with > "/usr/local/bin/doveadm search -A TEXT xyzabczzz". > > If you need any further details, please let me know. > > > Best regards > Henrik Larsson > > > On 15-04-2012 10:42, Henrik Larsson wrote: >> On 27-03-2012 08:57, Luca Palazzo wrote: >>> Hi Timo and All, >>> after upgrading to 2.1.2 i'm getting a lot of these messages: >>> Error: Corrupted squat uidlist file XXXXXX wrong indexid >> After an upgrade to 2.1.3 i see this as well. Any thoughts? >> >> Apr 15 03:43:43 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> Apr 15 06:30:27 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> Apr 15 07:43:55 pop3(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> Apr 15 09:00:01 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> >> # doveconf -n >> # 2.1.3: /usr/local/etc/dovecot/dovecot.conf >> # OS: FreeBSD 8.2-STABLE amd64 >> auth_mechanisms = plain login digest-md5 cram-md5 >> first_valid_uid = 125 >> listen = * >> log_path = /var/log/dovecot >> mail_plugins = fts fts_squat zlib >> mail_privileged_group = postfix >> mail_temp_dir = /var/db/dovecot >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date ihave >> namespace { >> inbox = yes >> location = >> prefix = >> separator = . >> type = private >> } >> namespace { >> hidden = yes >> inbox = no >> list = no >> location = >> prefix = INBOX. >> separator = . >> type = private >> } >> passdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> plugin { >> fts = squat >> fts_squat = partial=4 full=10 >> sieve = ~/.dovecot.sieve >> sieve_dir = ~/sieve >> } >> protocols = imap pop3 lmtp sieve >> service auth-worker { >> user = $default_internal_user >> } >> service auth { >> unix_listener /home/mail/postfix/private/dovecot-auth { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> service lmtp { >> executable = lmtp -L >> unix_listener /home/mail/postfix/private/dovecot-lmtp { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> ssl_cert = > ssl_key = > userdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> protocol lmtp { >> log_path = /var/log/dovecot-deliver >> mail_plugins = fts fts_squat zlib sieve >> } >> protocol lda { >> mail_plugins = fts fts_squat zlib sieve >> } >> protocol imap { >> mail_plugins = fts fts_squat zlib imap_zlib >> } >> >> >> Best regards >> Henrik Larsson comparable error exists for corrupted Maildir with the wrong filename i don't no the exact name of error i do not use zlib for this reason i thinks a patch from timo fix this feature later -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation From news1204.10.weezy at spamgourmet.com Fri Apr 20 17:31:33 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Fri, 20 Apr 2012 16:31:33 +0200 Subject: [Dovecot] default mail quota when using per user quota In-Reply-To: <4F91560B.5040009@noa.gr> References: <4F91560B.5040009@noa.gr> Message-ID: Am 20.04.2012 14:26, schrieb Nikolaos Milas: > > You may see: http://www.mail-archive.com/dovecot at dovecot.org/msg44010.html I do not see any relevant differences to my setup. Do you /really/ have a default quota of 4G if roomNumber is empty or rather have unlimited quota then? Greetings Andreas From cor at xs4all.nl Fri Apr 20 19:05:20 2012 From: cor at xs4all.nl (Cor Bosman) Date: Fri, 20 Apr 2012 18:05:20 +0200 Subject: [Dovecot] quota not being calculated Message-ID: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> It looks like my quota isnt being calculated properly after I started setting quota to a specific folder. The quota in that folder is always starting out at 0, and only new email is being added to the quota. If I remove the maildirsize file, and recalculate, it still starts at 0. Once email arrives in the folder, it gets added to maildirsize. This is with maildirquota, in perhaps a bit of an exotic config. The idea is to set a specific quota to Spam folder, using the trash plugin to keep it at a certain size. . getquotaroot "Spam" * QUOTAROOT "Spam" "User quota" "Spam quota" * QUOTA "User quota" (STORAGE 40848 1228800) * QUOTA "Spam quota" (STORAGE 0 20000) . OK Getquotaroot completed. Once 1 email gets added: .Spam# cat maildirsize 20480000S 0 0 2017 1 namespace { hidden = no ignore_on_failure = no inbox = yes list = yes location = maildir:%h:INDEX=/var/spool/mail/dovecot-control/indexes/%1u/%2u/%u:CONTROL=/var/spool/mail/dovecot-control/%1u/%2u/%u/INBOX prefix = separator = / subscriptions = yes type = private } namespace spam { hidden = yes ignore_on_failure = no inbox = no list = yes location = maildir:%h/.Spam:INDEX=/var/spool/mail/dovecot-control/indexes/%1u/%2u/%u/.Spam:CONTROL=/var/spool/mail/dovecot-control/%1u/%2u/%u/.Spam prefix = Spam/ separator = / subscriptions = no type = private } plugin { quota = fs:User quota quota2 = maildir:Spam quota:ns=Spam/ quota2_rule = *:storage=20000K sieve = /var/spool/mail/dovecot-control/sieve/%1u/%2u/%u/dovecot.sieve sieve_before = /etc/sieve/before sieve_dir = /var/spool/mail/dovecot-control/sieve/%1u/%2u/%u/scripts trash = /etc/dovecot/conf.d/dovecot-trash.conf.ext } (full config: http://pastebin.com/Mui4X7Zh) From tim.ruehsen at gmx.de Fri Apr 20 17:27:54 2012 From: tim.ruehsen at gmx.de (Tim Ruehsen) Date: Fri, 20 Apr 2012 16:27:54 +0200 Subject: [Dovecot] array code issue ? Message-ID: <201204201627.54213.tim.ruehsen@gmx.de> Hi, I just took a look into the dovecot 2.1 sources and just saw a possible issue in array.h. This code snippet as an example: #static inline void * #array_get_modifiable_i(struct array *array, unsigned int *count_r) #{ # *count_r = array->buffer->used / array->element_size; # return buffer_get_modifiable_data(array->buffer, NULL); #} array->buffer->used and array->element_size are of type 'size_t' which is 64bit on amd64 and others while 'count_r' is a 32bit value. At least, I see ugly warnings with -Wconversion (which I personally like to use). I know, it is unlikely that 'array->buffer->used / array->element_size' exceeds 32bit range. But then, dovecot's source is so well written, that the above code seems to disturb dovecot's code aesthetics. And who knows... in a few years (when we have THz and TBytes on our desktops) emails (and array sizes) might exceed everything that we think of today. Tim From mafonso at hangas.net Fri Apr 20 19:19:48 2012 From: mafonso at hangas.net (Miguel Afonso) Date: Fri, 20 Apr 2012 17:19:48 +0100 Subject: [Dovecot] dbox vs. mdbox In-Reply-To: References: <4D69FCB5.3090100@wildgooses.com> <58BED0A6-9EA0-4B4D-9065-69B22033D627@iki.fi> Message-ID: Hi, I finally deployed a migration test plant and done some conversions last night, so I took the chance to collect some data of these tests. I've set up a virtual machine with a dovecot 2.0.13 installation in order test the mailbox conversion. This machine has two adicional disks, one with the source data and other to store the conversion results. The source data is a restore from the latest production Maildir backup and I did a conversion to sdbox, and then to mdbox. I attach DiskIO and CPU plots for both conversions orange Line is read from source disk, blue line is write to destination. sdbox on the left and mdbox on the right. Ok, this is a conversion process and not actual mailbox usage. And lacks info on how does dbox behaves in read operations.. but this is what I got for now. [image: Inline image 1] [image: Inline image 2] Miguel On Thu, Mar 29, 2012 at 12:16 PM, Miguel Afonso wrote: > > > On Thu, Mar 29, 2012 at 12:30 AM, Timo Sirainen wrote: > >> >> The main problem is that it's difficult to do any "real world" tests with >> IMAP, especially when users are using many different kinds of IMAP clients. >> So I'm very interested in hearing some numbers (and disk IO graphs for a >> few weeks would be great) before your migration and after your migration, >> but the numbers for your tests might not mean all that much. > > > I was considering using the imaptest tool to simulate IMAP activity. I > would keep the same machine configuration, only varying the mailbox format > while running imaptest against each setup for a few hours/days. > > I'm now converting the original Maildir format to both dbox formats and > I'll give it a try. I'll share some graphs afterwards. > > -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 24202 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 27261 bytes Desc: not available URL: From tss at iki.fi Fri Apr 20 20:13:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 20 Apr 2012 20:13:29 +0300 Subject: [Dovecot] array code issue ? In-Reply-To: <201204201627.54213.tim.ruehsen@gmx.de> References: <201204201627.54213.tim.ruehsen@gmx.de> Message-ID: On 20.4.2012, at 17.27, Tim Ruehsen wrote: > I just took a look into the dovecot 2.1 sources and just saw a possible issue > in array.h. > > This code snippet as an example: > #static inline void * > #array_get_modifiable_i(struct array *array, unsigned int *count_r) > #{ > # *count_r = array->buffer->used / array->element_size; > # return buffer_get_modifiable_data(array->buffer, NULL); > #} > > array->buffer->used and array->element_size are of type 'size_t' which is > 64bit on amd64 and others while 'count_r' is a 32bit value. At least, I see > ugly warnings with -Wconversion (which I personally like to use). I've been planning on trying out some of clang's warning flags. Last time I used -Wconversion with gcc it was giving way too many warnings to be usable, but clang's -Wconversion looked better when I quickly looked at it. > I know, it is unlikely that 'array->buffer->used / array->element_size' > exceeds 32bit range. But then, dovecot's source is so well written, that the > above code seems to disturb dovecot's code aesthetics. :) Yeah, I intentionally decided to use unsigned int here. It's a bit of wasteful and ugly to use size_t everywhere.. I guess the code could be made something like: size_t count = array->buffer->used / array->element_size; I_assert(count < UINT_MAX); *count_r = (unsigned int)count; Or something like that. Although these array functions are sometimes in performance critical paths, so adding extra code isn't very good either. Perhaps a simple cast to make the warning go away.. Probably the element_size could also be changed to be unsigned int. > And who knows... in a few years (when we have THz and TBytes on our desktops) > emails (and array sizes) might exceed everything that we think of today. The email sizes yes, but probably not the number of emails in a mailbox. From nmilas at noa.gr Fri Apr 20 21:01:39 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Fri, 20 Apr 2012 21:01:39 +0300 Subject: [Dovecot] default mail quota when using per user quota In-Reply-To: References: <4F91560B.5040009@noa.gr> Message-ID: <4F91A483.8000406@noa.gr> On 20/4/2012 5:31 ??, Andreas Helmcke wrote: > Do you/really/ have a default quota of 4G if roomNumber is empty or rather have unlimited quota then? Yes, default quota is 4G for all mailboxes; if roomNumber is defined, then that value is used instead. In practice we are using non-default values for very few maiboxes. We have a relatively small number of users/mailboxes (<300), so we have the luxury of enjoying relatively large mailboxes (with some "overbooking" - because most users rarely use more than 1G). Nick From CMarcus at Media-Brokers.com Fri Apr 20 21:05:42 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 20 Apr 2012 14:05:42 -0400 Subject: [Dovecot] Error in logs indicating broken files In-Reply-To: <4F9156E3.4050801@cnpapers.com> References: <4F9156E3.4050801@cnpapers.com> Message-ID: <4F91A576.1090709@Media-Brokers.com> On 2012-04-20 8:30 AM, Steve Campbell wrote: > Still new to dovecot, > > I'm seeing a couple of errors in my log file indicating corruption of > index files. It is customary to provide at least some bare minimal system details, like, dovecot -n output (which includes the version), platform/OS, etc... -- Best regards, Charles From jeff at j-simmons.net Fri Apr 20 22:26:41 2012 From: jeff at j-simmons.net (Jeff Simmons) Date: Fri, 20 Apr 2012 12:26:41 -0700 Subject: [Dovecot] dsync: convert only mbox in /var/mail to maildir? Message-ID: <201204201226.41710.jeff@j-simmons.net> I'm migrating an old UW pop server to a new Dovecot server. The old server did not allow mail to be stored on the server, so only mbox files exist in /var/mail, with nothing in /home/$user. I'd like to copy the /var/mail directory over to the new server and then use dsync to convert the mbox files and put them in /home/$user/Maildir, but every permutation of the command line I've tried dies with a variation on can't find/write either /var/mail/mail or /home/$user/mail, seems it won't deal with mbox files without an associated mail directory. Can dsync be used for this kind of conversion, or should I look elsewhere for conversion utilities? Thanks for helping a noob who can't traverse a wiki without leaving a trail of breadcrumbs. -- Jeff Simmons jeff at j-simmons.net Simmons Consulting - Network Engineering, Administration, Security From victormanuelo at gmail.com Fri Apr 20 23:40:18 2012 From: victormanuelo at gmail.com (=?UTF-8?Q?Victor_O=C3=B1ate?=) Date: Fri, 20 Apr 2012 16:10:18 -0430 Subject: [Dovecot] sieve against ldap in Dovecot 1.2 vacation messeage Message-ID: Hi. I need configure "sieve" against ldap this is my file configuration dovecot.conf plugin { quota = dirsize:user sieve_dir = ~/sieve } protocol lda { mail_plugins = sieve } auth default { mechanisms = plain passdb pam { } userdb ldap { args = /etc/dovecot/dovecot-ldap.conf } user = nobody } I dont know what I have to put in dovecot-ldap.conf, ofcourse I have a atribut in LDAP by user with a vacation messeage "I'm out of office, please contact Mr. Black" this is dovecot-ldap.conf hosts = server auth_bind = yes ldap_version = 3 base = dc=dominio,dc=uk deref = never scope = subtree user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid,mail=Maildir,mailQuotaSize=quota_rule=*:storage=%$ user_filter = (&(objectClass=posixAccount)(uid=%u)) The principal idea is keep the control the message by user with LDAP and Dovecot(IMAP). without touching the postfix configuration file is that possible, or am I dreaming. Thanks.. Victor O?ate. From tss at iki.fi Sat Apr 21 00:18:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Apr 2012 00:18:24 +0300 Subject: [Dovecot] default mail quota when using per user quota In-Reply-To: References: Message-ID: <553C5242-5931-47B1-9FA9-0B64A7216197@iki.fi> On 20.4.2012, at 14.11, Andreas Helmcke wrote: > using dovecot 2.1 and per user mail quota via ldap is there a way to have a default quota which gets used, if the > "quota" field in ldap is not set? By "not set" you mean it doesn't exist at all, or it exists but is empty? > I tried with: > > plugin { > quota = maildir:User quota > quota_rule = *:storage=3G > } > > userdb { > args = /etc/dovecot/dovecot-ldap.conf > driver = ldap > } > > and in dovecot-ldap.conf: > user_attrs = dcMailQuota=quota_rule=*:storage=%$ If dcMailQuota isn't returned, then the global quota_rule is used. If it is returned as empty, I guess it gets treated as unlimited quota. There's currently no easy way avoid this (a difficult way could be e.g. post-login scripting, but that works only for imap/pop3). From tss at iki.fi Sat Apr 21 00:23:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Apr 2012 00:23:02 +0300 Subject: [Dovecot] mbox folders on-disc layout In-Reply-To: <4F9125F9.6060601@anonymous.org.uk> References: <4F8EA753.4070605@anonymous.org.uk> <35C2B470-438E-4CEC-A42A-0BC2C39A0159@iki.fi> <4F9125F9.6060601@anonymous.org.uk> Message-ID: <73A197D0-6D8A-4C7D-B6DE-F6B89EC40A0B@iki.fi> On 20.4.2012, at 12.01, John Robinson wrote: > Yes, I read all that, and thought all the options were pretty ugly - and the comment that it's a "trick" configuration would seem to suggest that the original author thought so to. And I think Maildir++ is pretty ugly too. If I wanted to try writing a new LAYOUT, where should I begin? src/lib-storage/mailbox-list-private.h has the API you need to implement src/lib-storage/list/mailbox-list-fs* is the "fs" implementation src/lib-storage/list/mailbox-list-maildir* is the "maildir++" implementation So decide which one looks more like the one you want, and either copy&paste it or start extending it. For example "imapdir" is nearly the same implementation as "maildir++" but almost all of the code is shared. From tss at iki.fi Sat Apr 21 00:27:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Apr 2012 00:27:59 +0300 Subject: [Dovecot] dsync: convert only mbox in /var/mail to maildir? In-Reply-To: <201204201226.41710.jeff@j-simmons.net> References: <201204201226.41710.jeff@j-simmons.net> Message-ID: <04D01B3D-CD58-4E0A-8C1A-36B01E4792E7@iki.fi> On 20.4.2012, at 22.26, Jeff Simmons wrote: > I'm migrating an old UW pop server to a new Dovecot server. The old server did > not allow mail to be stored on the server, so only mbox files exist in > /var/mail, with nothing in /home/$user. I'd like to copy the /var/mail > directory over to the new server and then use dsync to convert the mbox files > and put them in /home/$user/Maildir, but every permutation of the command line > I've tried dies with a variation on can't find/write either /var/mail/mail or > /home/$user/mail, seems it won't deal with mbox files without an associated > mail directory. Can dsync be used for this kind of conversion, or should I > look elsewhere for conversion utilities? Assuming in destination server the mbox is in /var/mail/$user and you want the destination to ~/Maildir: in dovecot.conf set: mail_location = maildir.~/Maildir rm -rf /tmp/emptydir dsync mirror mbox:/tmp/emptydir:/var/mail/$user be sure to delete the emptydir between users. From tss at iki.fi Sat Apr 21 00:29:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Apr 2012 00:29:40 +0300 Subject: [Dovecot] quota not being calculated In-Reply-To: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> References: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> Message-ID: <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> On 20.4.2012, at 19.05, Cor Bosman wrote: > It looks like my quota isnt being calculated properly after I started setting quota to a specific folder. The quota in that folder is always starting out at 0, and only new email is being added to the quota. If I remove the maildirsize file, and recalculate, it still starts at 0. Once email arrives in the folder, it gets added to maildirsize. This is with maildirquota, in perhaps a bit of an exotic config. The idea is to set a specific quota to Spam folder, using the trash plugin to keep it at a certain size. I'll try later, but try if it would work with dict quota file? http://wiki2.dovecot.org/Quota/Dict From news1204.10.weezy at spamgourmet.com Sat Apr 21 00:47:44 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Fri, 20 Apr 2012 23:47:44 +0200 Subject: [Dovecot] default mail quota when using per user quota (news1204: message 6 of 20) In-Reply-To: <553C5242-5931-47B1-9FA9-0B64A7216197@iki.fi> References: <553C5242-5931-47B1-9FA9-0B64A7216197@iki.fi> Message-ID: <4F91D980.8070307@spamgourmet.com> Am 20.04.2012 23:18, schrieb Timo Sirainen: > By "not set" you mean it doesn't exist at all, or it exists but is empty? I tried both (at least I do think I tried both). > If dcMailQuota isn't returned, then the global quota_rule is used. If it is returned as empty, I guess it gets treated as unlimited quota. There's currently no easy way avoid this (a difficult way could be e.g. post-login scripting, but that works only for imap/pop3). > Maybe openldap does not distingish between this cases. I will try again on monday, when I am back at work. From tss at iki.fi Sat Apr 21 00:54:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Apr 2012 00:54:01 +0300 Subject: [Dovecot] default mail quota when using per user quota (news1204: message 6 of 20) In-Reply-To: <4F91D980.8070307@spamgourmet.com> References: <553C5242-5931-47B1-9FA9-0B64A7216197@iki.fi> <4F91D980.8070307@spamgourmet.com> Message-ID: On 21.4.2012, at 0.47, Andreas Helmcke wrote: >> If dcMailQuota isn't returned, then the global quota_rule is used. If it is returned as empty, I guess it gets treated as unlimited quota. There's currently no easy way avoid this (a difficult way could be e.g. post-login scripting, but that works only for imap/pop3). >> > > Maybe openldap does not distingish between this cases. I will try again on > monday, when I am back at work. Hm. Or maybe I broke it in v2.1 when I changed this code. I'll also check later. :) From stan at hardwarefreak.com Sat Apr 21 03:22:05 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 20 Apr 2012 19:22:05 -0500 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F8FDD9B.6060007@hardwarefreak.com> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <4F8D69AB.5090706@esiee.fr> <4F8FDD9B.6060007@hardwarefreak.com> Message-ID: <4F91FDAD.7050102@hardwarefreak.com> On 4/19/2012 4:40 AM, Stan Hoeppner wrote: > On 4/17/2012 8:01 AM, Frank Bonnet wrote: > >> have 4000/6000 imaps concurent connections during working hours . >>>> for approx 50K "intensives" users. >>>> >>>> The only mandatory thing will be I must use HP proliant servers >>>> >>>> The operating system will be FreeBSD or Linux > I just made the wishlist public so it should be available tomorrow or > Friday. I'll provide the link when it's available. And here it is: http://secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=16797311 Since your requirement is for an HP solution, following is an HP server and storage system solution of roughly identical performance and redundancy to the SuperMicro based system I detailed. The HP system solution is $44,263, almost double the cost at $20,000 more. Due to the stupidity of Newegg requiring all wish lists to be reviewed before going live, I'll simply provide the links to all the products. Yes boys and girls, Newegg isn't just consumer products. They carry nearly the entire line of HP Proliant servers and storage, including the 4-way 48-core Opteron DL585 G7 w/64GB, the P2000 fiber channel array, and much more. In this case they sell every product needed to assemble this complete mail server solution: 1x http://www.newegg.com/Product/Product.aspx?Item=N82E16859105807 8x http://www.newegg.com/Product/Product.aspx?Item=N82E16820326150 3x http://www.newegg.com/Product/Product.aspx?Item=N82E16816401143 80x http://www.newegg.com/Product/Product.aspx?Item=N82E16822332061 3x http://www.newegg.com/Product/Product.aspx?Item=N82E16816118109 3x http://www.newegg.com/Product/Product.aspx?Item=N82E16816118163 2x http://www.newegg.com/Product/Product.aspx?Item=N82E16816133048 2x http://www.newegg.com/Product/Product.aspx?Item=N82E16833106050 The 9280-8e RAID controllers are identical to 9261-8i boards but have 2 external vs internal x4 6Gb SAS ports. I spec them instead of the Smart Array boards as they're far cheaper, easier to work with, and offer equal or superior performance. Thus everything written below is valid for this system as well, with the exception that you would configure 1 global hot spare in each chassis since these units have 25 drive bays instead of 24. The D2700 units come with 20" 8088 cables. I an additional spec'd two 3ft cables to make sure we reach all 3 disk chassis from the server, thinking the sever would be on top with the 3 disk chassis below. I hope this and my previous post are helpful in one aspect or another to Frank and anyone else. I spent more than a few minutes on these designs. ;) Days in fact on the SuperMicro design, only a couple of hours on the HP. It wouldn't have taken quite so long if all PCIe slots were created equal (x8), which they're not, or if modern servers didn't require 4 different types of DIMMs depending on how many slots you want to fill and how much expansion capacity you need without having to throw out all the previous memory, which many folks end up doing out of ignorance. Memory configuration is simply too darn complicated with high cap servers containing 8 channels and 24 slots. > The key to performance, and yielding a single file tree, is once again > using XFS to take advantage of this large spindle count across 3 RAID > controllers. Unlike previous configurations where I recommended using a > straight md concatenation of hardware RAID1 pairs, in this case we're > going to use a concatenation of 6 hardware RAID10 arrays. There are a > couple of reasons for doing so in this case: > > 1. Using 36 device names in a single md command line is less than > intuitive and possibly error prone. Using 6 is more manageable. > > 2. We have 3 BBWC RAID controllers w/24 drives each. This is a high > performance server and will see a high IO load in production. In many > cases one would use an external filesystem journal, which we could > easily do and get great performance with our mirrored SSDs. However, > the SSDs are not backed by BBWC, so a UPS failure or system crash could > hose the log journal. So we'll go with the default internal journal > which will be backed by the BBWC. > > Going internal with the log in this mail scenario can cause a serious > amount of extra IOPS on the filesystem data section, this being > Allocation Group 0. If we did the "normal" RAID1 concat, all the log IO > would hit the first RAID1 pair. On this system, the load may hit that > spindle pretty hard, making access to mailboxes in AG0 slower than > others. With 6 RAID10 arrays in a concat, the internal log writes will > be striped across 6 spindles in the first array. With 512MB BBWC > backing that array and optimizing writeout, and with delaylog, this will > yield optimal log write performance without slowing down mailbox file > access in AG0. To create such a setup we'd do something like this, > assuming the mobo LSI controller yields sd[ab], and the 6 array devices > on the PCIe LSI cards yield sd[cdefgh] > > 1. Create two RAID10 arrays, each of 12 drives, in the WebBIOS GUI of > each LSI card, using a strip size of 32KB which should yield good random > r/w performance for any mailbox format. Use the following policies for > each array: RW, Normal, Wback, Direct, Disable, No, and use the full > size. > > Create the concatenated md device: > $ mdadm -C /dev/md0 -l linear -n 6 /dev/sd[cdefgh] > > Then we format it with XFS, optimizing the AG layout for our mailbox > workload, and allocation write stripe alignment to each hardware array: > $ mkfs.xfs -d agcount=24 su=32k sw=6 /dev/md0 > > This yields 4 AGs per RAID10 array which will minimize the traditional > inode64 head seeking overhead on striped arrays, while still yielding > fantastic allocation parallelism with 24 AGs. > > Optimal fstab for MTA queue/mailbox workload, assuming kernel 2.6.39+: > /dev/md0 /mail xfs defaults,inode64,nobarrier 0 0 > > We disable write barriers as we have BBWC. And that 1.5GB of BBWC will > yield extremely low Dovecot write latency and throughput. > > Given the throughput available, if you're running Postfix on this box, > you will want to create a directory on this filesystem for the Postfix > spool. Postfix puts the spool files in many dozens, hundreds of > subdirectories, so you'll get 100% parallelism across all AGs, thus all > disks. > > It's very likely none of you will decide to build this system. My hope > is that some of the design concepts and components used, along with the > low cost but high performance of this machine, may be educational or > simply give people new ideas, steer them in directions they may not have > previously considered. -- Stan From jtam.home at gmail.com Sat Apr 21 06:32:30 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Fri, 20 Apr 2012 20:32:30 -0700 (PDT) Subject: [Dovecot] Error in logs indicating broken files In-Reply-To: References: Message-ID: Steve Campbell writes: > I'm seeing a couple of errors in my log file indicating corruption of > index files. > > Error: Corrupted index cache file/home/xxxxxx/mail/.imap/sent-mail/dovecot.index.cache: Broken virtual size for mail UID 67: 1 Time(s) > Error: FETCH [] for mailbox ~/mail/sent-mail UID 67 got too little data: 6203 vs 7478: 1 Time(s) > > This account is used by multiple users, but it is not a shared account > (haven't set any of those up yet - still new to me), so I would expect > corruption possibilities when multiple users modify the account folders > in some manner. This is just one dovecot IMAP client process moaning after another user client process pulled the rug out from under it by modifying the mailbox. I get a few dozen log entries of this sort per week with ~200 IMAP users without shared access. Unless your users actually complain about a problem, I think it's safe to ignore them. Joseph Tam From lgruen at gmx.ch Sat Apr 21 09:20:32 2012 From: lgruen at gmx.ch (=?ISO-8859-15?Q?=22L=2E_Gr=FCn=22?=) Date: Sat, 21 Apr 2012 08:20:32 +0200 Subject: [Dovecot] Correction of home and mail directory config Message-ID: <4F9251B0.5030501@gmx.ch> Hello, after running dovecot for a while, I stumbled accross the warning, not to have mail and home directory settings point to one and same directory. Here is the output of dovecot -n: > dovecot -n # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686 i686 Debian 6.0.4 ext3 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imaps imap disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login mail_max_userip_connections: 100 mail_privileged_group: mail mail_uid: 5000 mail_gid: 5000 mail_location: maildir:/var/vmail/%d/%n/Maildir mail_full_filesystem_access: yes mbox_write_locks: fcntl dotlock namespace: type: private separator: . inbox: yes list: yes subscriptions: yes lda: auth_socket_path: /var/run/dovecot/auth-master postmaster_address: xxxx at xxxx.org mail_plugins: sieve log_path: auth default: mechanisms: plain login passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: static args: uid=5000 gid=5000 home=/var/vmail/%d/%n/Maildir allow_all_users=yes socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail In "args: uid=5000 gid=5000 home=/var/vmail/%d/%n/Maildir allow_all_users=yes" I should delete "Maildir". My question is: What files do I have to copy from /Maildir to the home directory /var/vmail/%d/%n/ ? Are there other side effects, I have to consider? Thank you very much for your help! Stefan From cor at xs4all.nl Sat Apr 21 11:01:51 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sat, 21 Apr 2012 10:01:51 +0200 Subject: [Dovecot] quota not being calculated In-Reply-To: <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> References: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> Message-ID: > >> It looks like my quota isnt being calculated properly after I started setting quota to a specific folder. The quota in that folder is always starting out at 0, and only new email is being added to the quota. If I remove the maildirsize file, and recalculate, it still starts at 0. Once email arrives in the folder, it gets added to maildirsize. This is with maildirquota, in perhaps a bit of an exotic config. The idea is to set a specific quota to Spam folder, using the trash plugin to keep it at a certain size. > > I'll try later, but try if it would work with dict quota file? http://wiki2.dovecot.org/Quota/Dict > This almost works, but maybe there's a small bug. Config (2.1.4) namespace spam { type = private separator = / inbox = no prefix = Spam/ location = maildir:%h/.Spam:INDEX=/var/spool/mail/dovecot-control/indexes/%1u/%2u/%u/.Spam:CONTROL=/var/spool/mail/dovecot-control/%1u/%2u/%u/.Spam subscriptions = no list = yes hidden = yes } plugin { quota = fs:User quota quota2 = dict:Spam Quota::ns=spam:file:%h/spam-quota quota2_rule = *:storage=20M } Now I see: . getquotaroot "Spam" * QUOTAROOT "Spam" "User quota" "Spam Quota" * QUOTA "User quota" (STORAGE 10128 1228800) * QUOTA "Spam Quota" (STORAGE 8727 20480) The problem is that the storage calculated for the spam ns is wrong. It is less than 1MB. The 8727 seems to be the whole maildir including all folders. This seems to be caused by: Apr 21 10:00:11 lmtp1 dovecot: imap(cor): Error: quota: Unknown namespace: spam Regards, Cor From janfrode at tanso.net Sat Apr 21 12:52:57 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sat, 21 Apr 2012 11:52:57 +0200 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <4F90AE51.8080407@hardwarefreak.com> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <20120417124241.GA23120@dibs.tanso.net> <4F8D6BAA.7060501@gtdinternet.com> <20120417200820.GA30743@dibs.tanso.net> <4F90AE51.8080407@hardwarefreak.com> Message-ID: <20120421095257.GA19877@dibs.tanso.net> On Thu, Apr 19, 2012 at 07:31:13PM -0500, Stan Hoeppner wrote: > > This issue has come up twice on the Postfix list in less than a month. Oh, thanks! I'll look into those list posts.. I had mostly given up solving this by rate limits and decided to throw hardware at the problem when I saw the log entries for sender *.anpdm.com.. Seems to be a newsletter sender, which I found as 203 different mailserver ip-addresses in our incoming mailserver logs, from 53 different B-nets and 8 different A-nets. Will give smtpd_client_connection_count_limit a try.. -jf From janfrode at tanso.net Sat Apr 21 13:22:52 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sat, 21 Apr 2012 12:22:52 +0200 Subject: [Dovecot] migrate 15K users to new domain name In-Reply-To: References: <20120419082245.GA4050@dibs.tanso.net> <20120419132803.GA9396@dibs.tanso.net> <20120419141930.GB10069@dibs.tanso.net> Message-ID: <20120421102252.GA20159@dibs.tanso.net> On Thu, Apr 19, 2012 at 05:59:39PM +0300, Timo Sirainen wrote: > > With v2.1.4 you could do something like: > > doveadm -c dummy.conf user -m user at domain > > where dummy.conf contains the minimum configuration needed: > > mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln > ssl = no > Thanks! Works perfect. -jf From tss at iki.fi Sat Apr 21 13:32:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Apr 2012 13:32:21 +0300 Subject: [Dovecot] quota not being calculated In-Reply-To: References: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> Message-ID: <2D57E521-3DB3-41D9-B8AB-F280DE546061@iki.fi> On 21.4.2012, at 11.01, Cor Bosman wrote: > prefix = Spam/ .. > quota2 = dict:Spam Quota::ns=spam:file:%h/spam-quota .. > Apr 21 10:00:11 lmtp1 dovecot: imap(cor): Error: quota: Unknown namespace: spam Oh. It would make more sense to have ns=spam, but unfortunately I hadn't thought of namespace names before v2.1. You need to use ns=Spam/ From cor at xs4all.nl Sat Apr 21 13:42:35 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sat, 21 Apr 2012 12:42:35 +0200 Subject: [Dovecot] quota not being calculated In-Reply-To: <2D57E521-3DB3-41D9-B8AB-F280DE546061@iki.fi> References: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> <2D57E521-3DB3-41D9-B8AB-F280DE546061@iki.fi> Message-ID: On Apr 21, 2012, at 12:32 PM, Timo Sirainen wrote: > On 21.4.2012, at 11.01, Cor Bosman wrote: > >> prefix = Spam/ > . >> quota2 = dict:Spam Quota::ns=spam:file:%h/spam-quota > . >> Apr 21 10:00:11 lmtp1 dovecot: imap(cor): Error: quota: Unknown namespace: spam > > Oh. It would make more sense to have ns=spam, but unfortunately I hadn't thought of namespace names before v2.1. You need to use ns=Spam/ Exact same issue with dict quota. It starts out at 0 (even though there are 116 messages in the Spam folder. Then when 1 email gets saved to the Spam folder, the counter is increased by the size of that 1 email. Cor From nmilas at noa.gr Sat Apr 21 13:53:45 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Sat, 21 Apr 2012 13:53:45 +0300 Subject: [Dovecot] default mail quota when using per user quota (news1204: message 6 of 20) In-Reply-To: References: <553C5242-5931-47B1-9FA9-0B64A7216197@iki.fi> <4F91D980.8070307@spamgourmet.com> Message-ID: <4F9291B9.4080701@noa.gr> On 21/4/2012 12:54 ??, Timo Sirainen wrote: > Hm. Or maybe I broke it in v2.1 when I changed this code. I'll also check later. I'm afraid that's possible. The same configuration in 2.0.13 produces (doveadm quota get -u tester): Quota name Type Value Limit % User quota STORAGE 237803 4194304 5 User quota MESSAGE 531 - 0 and in 2.1.1: Quota name Type Value Limit % User quota STORAGE 0 - 0 User quota MESSAGE 0 - 0 (In the latter test server value should be 0 - because there are no messages there - but limit should be as above. Things do not change after: doveadm quota recalc -u tester) Regards, Nick From almarzuki2001 at hotmail.com Sat Apr 21 17:51:55 2012 From: almarzuki2001 at hotmail.com (Hadi Salem) Date: Sat, 21 Apr 2012 17:51:55 +0300 Subject: [Dovecot] dovecot LDA with sendmail In-Reply-To: <4F91438D.3040200@Media-Brokers.com> References: , , <4F91438D.3040200@Media-Brokers.com> Message-ID: Hi, i installed new version 2.0.20. but its the same problem # 2.0.20: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-164.el5 x86_64 CentOS release 5.4 (Final) auth_debug = yes auth_debug_passwords = yes auth_verbose = yes debug_log_path = /var/log/dovecot.log default_login_user = vmail disable_plaintext_auth = no hostname = host1.bigmama.com info_log_path = /var/log/dovecot.log log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:~/Maildir mail_privileged_group = vmail passdb { args = scheme=PLAIN username_format=%u /etc/dovecot/users driver = passwd-file } postmaster_address = root at host1.bigmama.com service auth { unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } ssl = no userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } Apr 21 17:47:16 host1 sendmail[32561]: q3LElF79032559: to=, ctladdr= (0/0), delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120292, relay=dovecot, dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/local/libexec/dovecot/dovecot-lda) exited with EX_TEMPFAIL > Date: Fri, 20 Apr 2012 07:07:57 -0400 > From: CMarcus at Media-Brokers.com > To: dovecot at dovecot.org > Subject: Re: [Dovecot] dovecot LDA with sendmail > > On 2012-04-19 9:45 PM, Joseph Tam wrote: > > Hadi Salem writes: > >> Im configuring sendmail with dovecot for virtual users using > >> password file and file for user name. on centos. > >> dovecot-1.0.7-7.el5 > > > Maybe I can beat Charles Marcus to the punch and recommend you upgrade > > to the latest version, otherwise you'll run into bugs that have already > > been fixed. > > Lol... I stopped reading at '...sendmail with dovecot...', otherwise I'd > have beat you to it... ;) > > -- > > Best regards, > > Charles From stan at hardwarefreak.com Sat Apr 21 22:59:44 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 21 Apr 2012 14:59:44 -0500 Subject: [Dovecot] Configuration advices for a 50000 mailboxes server(s) In-Reply-To: <20120421095257.GA19877@dibs.tanso.net> References: <4F8D21BB.2030407@esiee.fr> <4F8D59E7.6010403@gtdinternet.com> <20120417124241.GA23120@dibs.tanso.net> <4F8D6BAA.7060501@gtdinternet.com> <20120417200820.GA30743@dibs.tanso.net> <4F90AE51.8080407@hardwarefreak.com> <20120421095257.GA19877@dibs.tanso.net> Message-ID: <4F9311B0.7000503@hardwarefreak.com> On 4/21/2012 4:52 AM, Jan-Frode Myklebust wrote: > On Thu, Apr 19, 2012 at 07:31:13PM -0500, Stan Hoeppner wrote: >> >> This issue has come up twice on the Postfix list in less than a month. > > Oh, thanks! I'll look into those list posts.. I had mostly given up > solving this by rate limits and decided to throw hardware at the problem > when I saw the log entries for sender *.anpdm.com.. Seems to be a newsletter > sender, which I found as 203 different mailserver ip-addresses in our > incoming mailserver logs, from 53 different B-nets and 8 different A-nets. Yeah, they're a newsletter service provider. > Will give smtpd_client_connection_count_limit a try.. Setting this to 1 or 2 should severely slow their delivery rate. You can also do rate limiting at a much more fine grained level with a Postfix policy daemon such as postfwd (Postfix firewall daemon), though the setup is a bit more complicated. -- Stan From daniel.parthey at informatik.tu-chemnitz.de Sat Apr 21 23:29:28 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 21 Apr 2012 22:29:28 +0200 Subject: [Dovecot] How to remove duplicate GUID messages from storage? Message-ID: <20120421202928.GA12998@daniel.localdomain> Hi, I tried to import messages like this: doveadm import -u username at example.org mdbox:/mail/dovecot/example.org/username/mail mbox:/root/mail.txt all This seemed to have imported the same messages in the mail storage, as they were already there. And now I have got a lot of duplicates. Then I deleted the mbox:/root/mail mail stuff from "mailboxes" folder and the storage remained, with duplicate GUIDs. The following command doveadm -v force-resync -u username at example.org INBOX outputs lots of messages like this one, with different GUID doveadm(username at example.org): Error: mdbox /mail/dovecot/example.org/username/mail/storage: Duplicate GUID 0b8a032d66a0924fb42c0000de5f8128 in m.55:45484041 and m.14:52173045 The messages at m.55:45484041 and m.14:52173045 have the same content, since they were seemingly imported from the mail store itself, they are in the storage twice now and require twice as much disk space as before. How can I manually remove these identical, duplicate messages from the storage to save space? Dovecot does not do it automatically. Kind regards, Daniel From jtam.home at gmail.com Sun Apr 22 07:36:57 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Sat, 21 Apr 2012 21:36:57 -0700 (PDT) Subject: [Dovecot] dovecot LDA with sendmail In-Reply-To: References: Message-ID: > i installed new version 2.0.20. but its the same problem > > args = username_format=%u /etc/dovecot/users ... > > Apr 21 17:47:16 host1 sendmail[32561]: q3LElF79032559: to=, ctladdr= (0/0), delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120292, relay=dovecot, dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/local/libexec/dovecot/dovecot-lda) exited with EX_TEMPFAIL Any logs for dovecot? Same sendmail configs as before or did you try the extra flag I suggested? Joseph Tam From dovecot at webrz.net Sun Apr 22 11:03:55 2012 From: dovecot at webrz.net (Jos Chrispijn) Date: Sun, 22 Apr 2012 10:03:55 +0200 Subject: [Dovecot] Dovecot upgrade Message-ID: <4F93BB6B.7000501@webrz.net> Can someone tell me how I can upgrade from Dovecot 1.x to 2.x best? thanks for your reply, Jos Chrispijn From cor at xs4all.nl Sun Apr 22 11:08:44 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sun, 22 Apr 2012 10:08:44 +0200 Subject: [Dovecot] Dovecot upgrade In-Reply-To: <4F93BB6B.7000501@webrz.net> References: <4F93BB6B.7000501@webrz.net> Message-ID: <0BC7794E-1F1E-45BF-8539-61D5D8BAEE74@xs4all.nl> On Apr 22, 2012, at 10:03 AM, Jos Chrispijn wrote: > Can someone tell me how I can upgrade from Dovecot 1.x to 2.x best? > thanks for your reply, > Jos Chrispijn Have you read this? http://wiki2.dovecot.org/Upgrading/2.0 Cor From tlx at leuxner.net Sun Apr 22 11:08:52 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sun, 22 Apr 2012 10:08:52 +0200 Subject: [Dovecot] Dovecot upgrade In-Reply-To: <4F93BB6B.7000501@webrz.net> References: <4F93BB6B.7000501@webrz.net> Message-ID: Am 22.04.2012 um 10:03 schrieb Jos Chrispijn: > Can someone tell me how I can upgrade from Dovecot 1.x to 2.x best? > thanks for your reply, > Jos Chrispijn http://wiki2.dovecot.org/Upgrading/2.0 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dg at dguhl.org Sun Apr 22 12:06:20 2012 From: dg at dguhl.org (Dennis Guhl) Date: Sun, 22 Apr 2012 11:06:20 +0200 Subject: [Dovecot] sieve against ldap in Dovecot 1.2 vacation messeage In-Reply-To: References: Message-ID: <20120422090620.GA22644@laptop-dg.leere.eu> On Fri, Apr 20, 2012 at 04:10:18PM -0430, Victor O?ate wrote: > Hi. > I need configure "sieve" against ldap You can't configure sieve against LDAP. Sieve is a filter language, which is called by the LDA and configured by files in your home dir. For more information about sieve (with Dovecot 1) look here: http://wiki.dovecot.org/LDA/Sieve and here: http://wiki.dovecot.org/LDA/Sieve/Dovecot [..] > The principal idea is keep the control the message by user with LDAP > and Dovecot(IMAP). without touching the postfix configuration file > is that possible, or am I dreaming. What you want is ManageSieve: http://wiki.dovecot.org/ManageSieve more specifically: http://wiki.dovecot.org/ManageSieve/Clients Dennis From dg at dguhl.org Sun Apr 22 12:10:18 2012 From: dg at dguhl.org (Dennis Guhl) Date: Sun, 22 Apr 2012 11:10:18 +0200 Subject: [Dovecot] Correction of home and mail directory config In-Reply-To: <4F9251B0.5030501@gmx.ch> References: <4F9251B0.5030501@gmx.ch> Message-ID: <20120422091018.GB22644@laptop-dg.leere.eu> On Sat, Apr 21, 2012 at 08:20:32AM +0200, "L. Gr?n" wrote: > Hello, > after running dovecot for a while, I stumbled accross the warning, > not to have mail and home directory settings point to one and same > directory. > Here is the output of dovecot -n: > > > dovecot -n > # 1.2.15: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-5-686 i686 Debian 6.0.4 ext3 [..] > mail_location: maildir:/var/vmail/%d/%n/Maildir [..] > auth default: > mechanisms: plain login > passdb: > driver: sql > args: /etc/dovecot/dovecot-sql.conf > userdb: > driver: static > args: uid=5000 gid=5000 home=/var/vmail/%d/%n/Maildir [..] > In "args: uid=5000 gid=5000 home=/var/vmail/%d/%n/Maildir > allow_all_users=yes" I should delete "Maildir". Yes, exactly. > My question is: > What files do I have to copy from /Maildir to the home directory > /var/vmail/%d/%n/ ? Everything concerning sieve should be in the home dir. > Are there other side effects, I have to consider? Less warnings ;) Dennis From cor at xs4all.nl Sun Apr 22 12:33:03 2012 From: cor at xs4all.nl (Cor Bosman) Date: Sun, 22 Apr 2012 11:33:03 +0200 Subject: [Dovecot] Correction of home and mail directory config In-Reply-To: <20120422091018.GB22644@laptop-dg.leere.eu> References: <4F9251B0.5030501@gmx.ch> <20120422091018.GB22644@laptop-dg.leere.eu> Message-ID: <98FF7560-C0B6-495E-9E9C-DA9D1C742085@xs4all.nl> > Everything concerning sieve should be in the home dir. Why? It can be anywhere you want as long as it doesnt conflict with the names of your mailstore. Cor From dovecot-user at spambox.dk Mon Apr 23 11:30:06 2012 From: dovecot-user at spambox.dk (Henrik Larsson) Date: Mon, 23 Apr 2012 09:30:06 +0100 Subject: [Dovecot] 2.1.4 Corrupted squat uidlist In-Reply-To: <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> References: <4F7164DC.7010706@unict.it> <4F8A97F8.70808@spambox.dk> <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> Message-ID: <829b37b74c07acbf148f7bee6a1e175e@larsson.as> Just to let you know, 2.1.4 didn't solve this either. If you need any futher info to get this solved, please let me know. Best regards Henrik Larsson > This problem is still showing up every day in the logs. > > I tried to delete the dovecot.index.search and > dovecot.index.search.uids files for all users, but the errors are > still there after a re-build of the search index with > "/usr/local/bin/doveadm search -A TEXT xyzabczzz". > > If you need any further details, please let me know. > > > Best regards > Henrik Larsson > > > On 15-04-2012 10:42, Henrik Larsson wrote: >> On 27-03-2012 08:57, Luca Palazzo wrote: >>> Hi Timo and All, >>> after upgrading to 2.1.2 i'm getting a lot of these messages: >>> Error: Corrupted squat uidlist file XXXXXX wrong indexid >> After an upgrade to 2.1.3 i see this as well. Any thoughts? >> >> Apr 15 03:43:43 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> Apr 15 06:30:27 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> Apr 15 07:43:55 pop3(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> Apr 15 09:00:01 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong >> indexid >> >> # doveconf -n >> # 2.1.3: /usr/local/etc/dovecot/dovecot.conf >> # OS: FreeBSD 8.2-STABLE amd64 >> auth_mechanisms = plain login digest-md5 cram-md5 >> first_valid_uid = 125 >> listen = * >> log_path = /var/log/dovecot >> mail_plugins = fts fts_squat zlib >> mail_privileged_group = postfix >> mail_temp_dir = /var/db/dovecot >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date ihave >> namespace { >> inbox = yes >> location = >> prefix = >> separator = . >> type = private >> } >> namespace { >> hidden = yes >> inbox = no >> list = no >> location = >> prefix = INBOX. >> separator = . >> type = private >> } >> passdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> plugin { >> fts = squat >> fts_squat = partial=4 full=10 >> sieve = ~/.dovecot.sieve >> sieve_dir = ~/sieve >> } >> protocols = imap pop3 lmtp sieve >> service auth-worker { >> user = $default_internal_user >> } >> service auth { >> unix_listener /home/mail/postfix/private/dovecot-auth { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> service lmtp { >> executable = lmtp -L >> unix_listener /home/mail/postfix/private/dovecot-lmtp { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> ssl_cert = > ssl_key = > userdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> protocol lmtp { >> log_path = /var/log/dovecot-deliver >> mail_plugins = fts fts_squat zlib sieve >> } >> protocol lda { >> mail_plugins = fts fts_squat zlib sieve >> } >> protocol imap { >> mail_plugins = fts fts_squat zlib imap_zlib >> } >> >> >> Best regards >> Henrik Larsson From tss at iki.fi Mon Apr 23 13:20:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 13:20:24 +0300 Subject: [Dovecot] quota not being calculated In-Reply-To: References: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> <2D57E521-3DB3-41D9-B8AB-F280DE546061@iki.fi> Message-ID: <96E3D11C-8F71-4726-816A-6C448554BC70@iki.fi> On 21.4.2012, at 13.42, Cor Bosman wrote: > On Apr 21, 2012, at 12:32 PM, Timo Sirainen wrote: > >> On 21.4.2012, at 11.01, Cor Bosman wrote: >> >>> prefix = Spam/ >> . >>> quota2 = dict:Spam Quota::ns=spam:file:%h/spam-quota >> . >>> Apr 21 10:00:11 lmtp1 dovecot: imap(cor): Error: quota: Unknown namespace: spam >> >> Oh. It would make more sense to have ns=spam, but unfortunately I hadn't thought of namespace names before v2.1. You need to use ns=Spam/ > > Exact same issue with dict quota. It starts out at 0 (even though there are 116 messages in the Spam folder. Then when 1 email gets saved to the Spam folder, the counter is increased by the size of that 1 email. http://hg.dovecot.org/dovecot-2.1/rev/4c8f79d1f9f1 should fix it with dict quota. From tss at iki.fi Mon Apr 23 13:34:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 13:34:09 +0300 Subject: [Dovecot] default mail quota when using per user quota (news1204: message 6 of 20) In-Reply-To: <4F9291B9.4080701@noa.gr> References: <553C5242-5931-47B1-9FA9-0B64A7216197@iki.fi> <4F91D980.8070307@spamgourmet.com> <4F9291B9.4080701@noa.gr> Message-ID: On 21.4.2012, at 13.53, Nikolaos Milas wrote: > On 21/4/2012 12:54 ??, Timo Sirainen wrote: > >> Hm. Or maybe I broke it in v2.1 when I changed this code. I'll also check later. Just tried, it works as expected. > I'm afraid that's possible. The same configuration in 2.0.13 produces (doveadm quota get -u tester): > > Quota name Type Value Limit % > User quota STORAGE 237803 4194304 5 > User quota MESSAGE 531 - 0 > > and in 2.1.1: > > Quota name Type Value Limit % > User quota STORAGE 0 - 0 > User quota MESSAGE 0 - 0 > > (In the latter test server value should be 0 - because there are no messages there - but limit should be as above. > Things do not change after: doveadm quota recalc -u tester) What do you get in logs with auth_debug=yes? From tss at iki.fi Mon Apr 23 13:35:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 13:35:46 +0300 Subject: [Dovecot] Error in logs indicating broken files In-Reply-To: References: Message-ID: <53563F64-C80D-4E98-B722-E7154BC0FAB3@iki.fi> On 21.4.2012, at 6.32, Joseph Tam wrote: >> Error: Corrupted index cache file/home/xxxxxx/mail/.imap/sent-mail/dovecot.index.cache: Broken virtual size for mail UID 67: 1 Time(s) >> Error: FETCH [] for mailbox ~/mail/sent-mail UID 67 got too little data: 6203 vs 7478: 1 Time(s) > > This is just one dovecot IMAP client process moaning after another user > client process pulled the rug out from under it by modifying the mailbox. > I get a few dozen log entries of this sort per week with ~200 IMAP users > without shared access. Well, it is a bug and it shouldn't happen. But since it happens only with mbox, I haven't bothered to look into it. From tss at iki.fi Mon Apr 23 13:52:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 13:52:49 +0300 Subject: [Dovecot] 2.1.3 Corrupted squat uidlist In-Reply-To: <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> References: <4F7164DC.7010706@unict.it> <4F8A97F8.70808@spambox.dk> <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> Message-ID: <96F0AD8C-9832-4CA5-B1DA-3936F9E2F0FB@iki.fi> So the doveadm search itself succeeds without errors, but if you immediately do another doveadm search it fails? Or do you need to do something else also in the middle? I couldn't reproduce this with a few tests. On 20.4.2012, at 10.40, Henrik Larsson wrote: > This problem is still showing up every day in the logs. > > I tried to delete the dovecot.index.search and dovecot.index.search.uids files for all users, but the errors are still there after a re-build of the search index with "/usr/local/bin/doveadm search -A TEXT xyzabczzz". > > If you need any further details, please let me know. > > > Best regards > Henrik Larsson > > > On 15-04-2012 10:42, Henrik Larsson wrote: >> On 27-03-2012 08:57, Luca Palazzo wrote: >>> Hi Timo and All, >>> after upgrading to 2.1.2 i'm getting a lot of these messages: >>> Error: Corrupted squat uidlist file XXXXXX wrong indexid >> After an upgrade to 2.1.3 i see this as well. Any thoughts? >> >> Apr 15 03:43:43 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid >> Apr 15 06:30:27 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid >> Apr 15 07:43:55 pop3(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid >> Apr 15 09:00:01 imap(xxxx): Error: Corrupted squat uidlist file >> /home/mail/xxxx/xxxx//Maildir/dovecot.index.search.uids: wrong indexid >> >> # doveconf -n >> # 2.1.3: /usr/local/etc/dovecot/dovecot.conf >> # OS: FreeBSD 8.2-STABLE amd64 >> auth_mechanisms = plain login digest-md5 cram-md5 >> first_valid_uid = 125 >> listen = * >> log_path = /var/log/dovecot >> mail_plugins = fts fts_squat zlib >> mail_privileged_group = postfix >> mail_temp_dir = /var/db/dovecot >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date ihave >> namespace { >> inbox = yes >> location = >> prefix = >> separator = . >> type = private >> } >> namespace { >> hidden = yes >> inbox = no >> list = no >> location = >> prefix = INBOX. >> separator = . >> type = private >> } >> passdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> plugin { >> fts = squat >> fts_squat = partial=4 full=10 >> sieve = ~/.dovecot.sieve >> sieve_dir = ~/sieve >> } >> protocols = imap pop3 lmtp sieve >> service auth-worker { >> user = $default_internal_user >> } >> service auth { >> unix_listener /home/mail/postfix/private/dovecot-auth { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> service lmtp { >> executable = lmtp -L >> unix_listener /home/mail/postfix/private/dovecot-lmtp { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> ssl_cert = > ssl_key = > userdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> protocol lmtp { >> log_path = /var/log/dovecot-deliver >> mail_plugins = fts fts_squat zlib sieve >> } >> protocol lda { >> mail_plugins = fts fts_squat zlib sieve >> } >> protocol imap { >> mail_plugins = fts fts_squat zlib imap_zlib >> } >> >> >> Best regards >> Henrik Larsson > From tss at iki.fi Mon Apr 23 13:59:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 13:59:44 +0300 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: <2vp569xe2o.ln2@mail.ela-soft.com> References: <2vp569xe2o.ln2@mail.ela-soft.com> Message-ID: <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> On 19.4.2012, at 19.09, Andreas Helmcke wrote: > Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace > dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, > subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ Here's the dummy namespace with prefix="". > Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace : > type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, > subscriptions=yes location=maildir:/home/mail/user/ahelmcke But what is this? You appear to have an empty namespace {} in your configs? > Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Error: user > ahelmcke: Initialization failed: namespace configuration error: Duplicate > namespace prefix: "" That's why it fails with this. From tss at iki.fi Mon Apr 23 14:12:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 14:12:25 +0300 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works / solved by remove listescape In-Reply-To: <4F907529.2020403@schetterer.org> References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> <4F90250E.3000200@schetterer.org> <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> <4F903DC1.5070501@schetterer.org> <4F905B2C.8040204@schetterer.org> <4F907529.2020403@schetterer.org> Message-ID: On 19.4.2012, at 23.27, Robert Schetterer wrote: >> prefix = shared/%%u/ simply dont works >> >> i have reverted back to exact namespace config >> of the 2.0.20 server, same result, so something broken in 2.1 >> >> > Hi Timo > problem solved by removing listescape plugin > the acl paths did / instead of dot in domainname > > any chance to get listescape fixed ? Ah, that makes it easy to fix: http://hg.dovecot.org/dovecot-2.1/rev/63af3274fb6f From tss at iki.fi Mon Apr 23 14:16:41 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 14:16:41 +0300 Subject: [Dovecot] imapc / namespace question In-Reply-To: <4F888F5E.5030709@necoro.eu> References: <4F888F5E.5030709@necoro.eu> Message-ID: <23ABA092-7511-4DF5-98AA-932E7338A994@iki.fi> On 13.4.2012, at 23.41, Ren? Neumann wrote: > I've got two questions regarding imapc and namespaces: > > 1) Is there a way of stating, that I only want part of the hierarchy on > the end of an imapc connection to be available here? My "problem": hg has now imapc_list_prefix. > 2) "Private" namespaces only make sense, if location contains something > user-specific like "%u" or "~". Is this correct? > If yes: Is there some other way to make something like an imapc > connection only appear for one user, if there is nothing in the location > or imapc_user to denote the user it belongs to and hence I cannot use > "private"? > The only way I see currently is to use "shared" and set up ACLs to > forbid all users except one to access the namespace (which gets slightly > ugly as I have to use global ACLs, as -- I think -- there is no way to > store ACLs for an imapc storage). Well, it's not necessarily a problem to use a private namespace for anything you want. If you set it hidden=yes (probably should do anyway) then IMAP clients won't even know that you've set it as private. Internally the main difference between private and public namespaces is the default ACLs. From tss at iki.fi Mon Apr 23 14:18:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 14:18:20 +0300 Subject: [Dovecot] French characters from 2.0.x to 2.1.3 In-Reply-To: <4F883FE7.9080401@esiee.fr> References: <4F883FE7.9080401@esiee.fr> Message-ID: <912BFCAA-3ECE-42AA-884F-0FE0BEDB4BBC@iki.fi> On 13.4.2012, at 18.01, Frank Bonnet wrote: > Maybe it's a coincidence but it "seems" > since I upgrade from 2.0.18 to 2.1.3 > that some French characters ( a with accents by example ) > are not well displayed anymore using thunderbird and are > replaced by a "?" character Dovecot doesn't send decoded characters to IMAP clients. Of course there might be something else going on, but I can't really guess what it could be. From news1204.10.weezy at spamgourmet.com Mon Apr 23 14:40:11 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Mon, 23 Apr 2012 13:40:11 +0200 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> References: <2vp569xe2o.ln2@mail.ela-soft.com> <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> Message-ID: <4F953F9B.10307@spamgourmet.com> Am 23.04.2012 12:59, schrieb Timo Sirainen: > On 19.4.2012, at 19.09, Andreas Helmcke wrote: > >> Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace : >> type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, >> subscriptions=yes location=maildir:/home/mail/user/ahelmcke > > But what is this? You appear to have an empty namespace {} in your configs? > nope. You might check yourself; complete dovecot -n: # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) ext4 auth_master_user_separator = * auth_verbose = yes first_valid_uid = 400 mail_debug = yes mail_gid = vmail mail_home = /home/mail/user/%u mail_location = maildir:/home/mail/user/%u mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl namespace Allgemeines { location = maildir:/home/mail/Allgemeines:INDEX=/home/mail/user/%u/SEEN.Allgemeines prefix = Allgemeines/ separator = / subscriptions = no type = public } namespace dummy { alias_for = INBOX/ hidden = yes list = no location = maildir:/home/mail/user/%u/public/ prefix = separator = / type = private } namespace inbox { inbox = yes location = prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { quota = maildir:User quota quota_rule = *:storage=3G } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = * } inet_listener imaps { address = * } } ssl = required ssl_cert = References: <4F883FE7.9080401@esiee.fr> <912BFCAA-3ECE-42AA-884F-0FE0BEDB4BBC@iki.fi> Message-ID: <4F954017.50702@esiee.fr> On 04/23/2012 01:18 PM, Timo Sirainen wrote: > On 13.4.2012, at 18.01, Frank Bonnet wrote: > >> Maybe it's a coincidence but it "seems" >> since I upgrade from 2.0.18 to 2.1.3 >> that some French characters ( a with accents by example ) >> are not well displayed anymore using thunderbird and are >> replaced by a "?" character > Dovecot doesn't send decoded characters to IMAP clients. Of course there might be something else going on, but I can't really guess what it could be. > OK thanks , maybe an update of thunderbird ? From tss at iki.fi Mon Apr 23 14:51:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 14:51:50 +0300 Subject: [Dovecot] Bug in HIGHESTMODSEQ when MODSEQ tracking is not enabled for version 2.0.18. In-Reply-To: <48A61B0976102F488C64EF8A8F1E929D074E9F@XMB118CNC.rim.net> References: <48A61B0976102F488C64EF8A8F1E929D074BBA@XMB118CNC.rim.net> <11BD08AA-2E29-48DF-973D-4D5B4CCE2FF7@iki.fi> <48A61B0976102F488C64EF8A8F1E929D074E9F@XMB118CNC.rim.net> Message-ID: <8A3B9DEC-6E45-4A2B-82FB-1A39BABF2247@iki.fi> On 10.4.2012, at 23.50, Glenn Wurster wrote: >> Yes, modseqs aren't tracked in a mailbox until client expresses an >> interest for them. It would be a waste of disk space to save them >> since 99% of users don't need them. > > Makes sense, our mail client gets caught in the middle though, because it uses HIGHESTMODSEQ to track mailbox updates without using MODSEQ options on SELECT/FETCH to track message updates. It would be actually possible for Dovecot to always keep track of highestmodseq, even if individual modseqs weren't tracked. I almost implemented it, but keeping it backwards compatible with old versions would have needed to make it more complex. Maybe v2.2 could do this. >> 2) If a mailbox doesn't have modseqs enabled, return NOMODSEQ. This >> isn't ideal, but seems like the only possibility. > > The RFC also states that if we return NOMODSEQ we'd have to return a tagged BAD response to "UID FETCH 1 MODSEQ", which appears to one of the commands that enables MODSEQ for Dovecot ("SELECT INBOX (CONDSTORE)" also enables it...). What about returning a BAD response and at the same time start tracking MODSEQ so that future SELECT commands would return HIGHESTMODSEQ? Do we know what email clients are using CONDSTORE options and how they'd react to a mailbox suddenly having MODSEQ capabilities after we just told them it didn't? That's kind of an annoying part of the RFC that it says the commands MUST fail with BAD.. I don't think there was really any good reason to add that text. Also Dovecot hasn't failed those commands earlier also with mailbox formats that don't support modseqs at all. So at least for now I simply made it return NOMODSEQ when modseqs aren't enabled, and the rest of the behavior is the same. From tss at iki.fi Mon Apr 23 14:55:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 14:55:21 +0300 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: <4F953F9B.10307@spamgourmet.com> References: <2vp569xe2o.ln2@mail.ela-soft.com> <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> <4F953F9B.10307@spamgourmet.com> Message-ID: On 23.4.2012, at 14.40, Andreas Helmcke wrote: > Am 23.04.2012 12:59, schrieb Timo Sirainen: >> On 19.4.2012, at 19.09, Andreas Helmcke wrote: >> >>> Apr 19 17:58:07 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace : >>> type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, >>> subscriptions=yes location=maildir:/home/mail/user/ahelmcke >> >> But what is this? You appear to have an empty namespace {} in your configs? >> > nope. > > You might check yourself; complete dovecot -n: See if this changes the behavior: service config { unix_listener config { mode = 0666 } } I'd guess that makes the vmail work also? Although it still doesn't explain the difference between asking config from config process vs. getting it by executing doveconf. From dovecot-user at spambox.dk Mon Apr 23 14:56:13 2012 From: dovecot-user at spambox.dk (Henrik Larsson) Date: Mon, 23 Apr 2012 12:56:13 +0100 Subject: [Dovecot] 2.1.3 Corrupted squat uidlist In-Reply-To: <96F0AD8C-9832-4CA5-B1DA-3936F9E2F0FB@iki.fi> References: <4F7164DC.7010706@unict.it> <4F8A97F8.70808@spambox.dk> <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> <96F0AD8C-9832-4CA5-B1DA-3936F9E2F0FB@iki.fi> Message-ID: <9ca41dc4b98ee3c8d0e38f4c2f5d187d@larsson.as> > So the doveadm search itself succeeds without errors, but if you > immediately do another doveadm search it fails? Or do you need to do > something else also in the middle? I couldn't reproduce this with a > few tests. Is there any way to produce debug logs that will show what goes wrong. It is consistent in such a way that I will have several errors within a 24 hour period. I will be willing to add code and rebuild dovecot to produce the logs you need. Best regards Henrik Larsson From tss at iki.fi Mon Apr 23 15:02:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 15:02:37 +0300 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: <4F953F9B.10307@spamgourmet.com> References: <2vp569xe2o.ln2@mail.ela-soft.com> <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> <4F953F9B.10307@spamgourmet.com> Message-ID: <2F3EF8E1-2851-418A-8B43-EE0D91A321C8@iki.fi> Oh, right: On 23.4.2012, at 14.40, Andreas Helmcke wrote: > namespace Allgemeines { Lowercase it: namespace allgemeines { This is a bit annoying bug, but I'm not entirely sure how to fix it properly. From tss at iki.fi Mon Apr 23 15:05:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 15:05:58 +0300 Subject: [Dovecot] 2.1.3 Corrupted squat uidlist In-Reply-To: <9ca41dc4b98ee3c8d0e38f4c2f5d187d@larsson.as> References: <4F7164DC.7010706@unict.it> <4F8A97F8.70808@spambox.dk> <3413bb6dea04bdca4f5ad1f3d2ded934@larsson.as> <96F0AD8C-9832-4CA5-B1DA-3936F9E2F0FB@iki.fi> <9ca41dc4b98ee3c8d0e38f4c2f5d187d@larsson.as> Message-ID: <3B05C40A-4FC7-479D-8EF3-F21C5DBEEA90@iki.fi> On 23.4.2012, at 14.56, Henrik Larsson wrote: >> So the doveadm search itself succeeds without errors, but if you >> immediately do another doveadm search it fails? Or do you need to do >> something else also in the middle? I couldn't reproduce this with a >> few tests. > > Is there any way to produce debug logs that will show what goes wrong. It is consistent in such a way that I will have several errors within a 24 hour period. > > I will be willing to add code and rebuild dovecot to produce the logs you need. Well, Squat is currently rather low priority to me .. I even considered removing it entirely, since I don't think it's very usable. fts-lucene would work much better. From l.chelchowski at eurocar.pl Mon Apr 23 15:27:08 2012 From: l.chelchowski at eurocar.pl (l.chelchowski at eurocar.pl) Date: Mon, 23 Apr 2012 14:27:08 +0200 Subject: [Dovecot] questions about SPECIAL-USE and virtual Message-ID: <4F954A9C.6000604@eurocar.pl> Hi, I have some questions about Special-Use and virtual namespace: I want to create a virtual namespace with mailboxes: Sent, Unread, Flagged, All. 1. Is it enough to put in the config file: mailbox virtual/Sent { auto = subscribe special_use = \Sent } mailbox virtual/Unread { auto = subscribe special_use = \Unread #Is it correct? } mailbox virtual/Flagged { auto = subscribe special_use = \Flagged } mailbox virtual/All { auto = subscribe special_use = \All } and it would displays mail from every namespace I have subscribed (Shared, Public)? OR I have to make also dovecot-virtual files? 2. In dovecot-virtual file can I put for mailbox virtual/Sent something like: * \Sent 3. Is the http://wiki2.dovecot.org/Plugins/Virtual full documented, if not , where can I find it? 4. Is there a documentation for Special-Use? -- ?ukasz From tss at iki.fi Mon Apr 23 15:33:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 15:33:05 +0300 Subject: [Dovecot] questions about SPECIAL-USE and virtual In-Reply-To: <4F954A9C.6000604@eurocar.pl> References: <4F954A9C.6000604@eurocar.pl> Message-ID: On 23.4.2012, at 15.27, l.chelchowski at eurocar.pl wrote: > I have some questions about Special-Use and virtual namespace: > > I want to create a virtual namespace with mailboxes: Sent, Unread, Flagged, All. > > 1. Is it enough to put in the config file: > mailbox virtual/Sent { > auto = subscribe > special_use = \Sent > } You have to put the mailbox {}es inside the namespace {} you want. Otherwise it just gives "unknown section" error. > OR I have to make also dovecot-virtual files? You have to do this also. The mailbox {} sections don't do anything magically different. They simply add the \Sent flag. The auto=subscribe can automatically subscribe the virtual mailbox, but it can't automatically create it, because new virtual mailboxes can't be created by IMAP clients. > 2. In dovecot-virtual file can I put for mailbox virtual/Sent something like: > * > \Sent No. I'm not sure what you'd want that to do. There is no \Sent message flag. > 3. Is the http://wiki2.dovecot.org/Plugins/Virtual full documented, if not , where can I find it? The only other documentation is the code. > 4. Is there a documentation for Special-Use? http://tools.ietf.org/html/rfc6154 From CMarcus at Media-Brokers.com Mon Apr 23 15:47:04 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 23 Apr 2012 08:47:04 -0400 Subject: [Dovecot] questions about SPECIAL-USE and virtual In-Reply-To: References: <4F954A9C.6000604@eurocar.pl> Message-ID: <4F954F48.7010409@Media-Brokers.com> On 2012-04-23 8:33 AM, Timo Sirainen wrote: > The auto=subscribe can automatically subscribe the virtual mailbox, > but it can't automatically create it, because new virtual mailboxes > can't be created by IMAP clients. Hmmm... Thunderbird has the ability to create virtual mailboxes too, but obviously they are created and handled locally, not on the server... How much trouble would it be to add an API so that an IMAP client *could* create and manage them on a dovecot server? And Maybe this would be a good candidate for an RFC spec? -- Best regards, Charles From tss at iki.fi Mon Apr 23 15:50:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 15:50:36 +0300 Subject: [Dovecot] questions about SPECIAL-USE and virtual In-Reply-To: <4F954F48.7010409@Media-Brokers.com> References: <4F954A9C.6000604@eurocar.pl> <4F954F48.7010409@Media-Brokers.com> Message-ID: <8D2C7878-B981-4DC9-8091-E0058BD361BA@iki.fi> On 23.4.2012, at 15.47, Charles Marcus wrote: > On 2012-04-23 8:33 AM, Timo Sirainen wrote: >> The auto=subscribe can automatically subscribe the virtual mailbox, >> but it can't automatically create it, because new virtual mailboxes >> can't be created by IMAP clients. > > Hmmm... > > Thunderbird has the ability to create virtual mailboxes too, but obviously they are created and handled locally, not on the server... > > How much trouble would it be to add an API so that an IMAP client *could* create and manage them on a dovecot server? And Maybe this would be a good candidate for an RFC spec? There have been a few drafts about virtual mailboxes, but they died at some point. I guess it could be implemented, but I'm less certain that there's actual need for it. From news1204.10.weezy at spamgourmet.com Mon Apr 23 16:30:39 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Mon, 23 Apr 2012 15:30:39 +0200 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: References: <2vp569xe2o.ln2@mail.ela-soft.com> <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> <4F953F9B.10307@spamgourmet.com> Message-ID: <4F95597F.2080908@spamgourmet.com> Am 23.04.2012 13:55, schrieb Timo Sirainen: > > See if this changes the behavior: > > service config { > unix_listener config { > mode = 0666 > } > } > > I'd guess that makes the vmail work also? Although it still doesn't explain the difference between asking config from config process vs. getting it by executing doveconf. Yes, with this entry sending as user vmail is working. Thanks. From news1204.10.weezy at spamgourmet.com Mon Apr 23 16:39:08 2012 From: news1204.10.weezy at spamgourmet.com (Andreas Helmcke) Date: Mon, 23 Apr 2012 15:39:08 +0200 Subject: [Dovecot] error when using dovecot-lda as non root (vmail) user In-Reply-To: <2F3EF8E1-2851-418A-8B43-EE0D91A321C8@iki.fi> References: <2vp569xe2o.ln2@mail.ela-soft.com> <5F377954-1D5A-4DED-A2A3-40469CC35D92@iki.fi> <4F953F9B.10307@spamgourmet.com> <2F3EF8E1-2851-418A-8B43-EE0D91A321C8@iki.fi> Message-ID: <4F955B7C.2070009@spamgourmet.com> Am 23.04.2012 14:02, schrieb Timo Sirainen: > On 23.4.2012, at 14.40, Andreas Helmcke wrote: > >> namespace Allgemeines { > > Lowercase it: namespace allgemeines { > Did the trick, also. Sending as user vmail without the service config entry, which you recommended in the other mail, is working. From tss at iki.fi Mon Apr 23 16:49:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Apr 2012 16:49:12 +0300 Subject: [Dovecot] v2.1.5 released Message-ID: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz.sig A few announcements first: I'm now going to start being more strict about not answering Dovecot questions sent to me privately (especially support requests, but also bug reports, feature requests, etc). Often those questions could be answered by other people in the mailing list as well, so they just waste my time. Even if you're sure that I'm the only person who can possibly answer it, it may be something that others are interested of as well and replying privately denies them the possibility of reading about it. If you really do not want to send something publicly, you still have an option for private communication: just buy one of the Dovecot support packages. This is also the first Dovecot release that has run through automated release testing. I've been planning on doing that for last 5 years at least, and finally it's done. The testing includes running a lot of tests to verify that the IMAP server gives proper responses, stress testing doesn't break it, memory isn't leaked and several other things. The tests are run for many different kinds of configurations. Some bugs show up only for some specific combination of settings, but unfortunately because there's such a large number of them it's impossible to test every combination. So here's another advantage that Dovecot support customers have: they can have their specific configuration included in the automated tests. This release already fixes one bug I found only after testing a customer's config. Oh, and looks like I got accepted as a speaker to LinuxTag 2012. The plan is to talk about some new things in Dovecot and dsync replication (looks like I have to hurry up with the dsync rewrite). And finally, the most important changes since v2.1.4: * IMAP: When neither the session nor the mailbox has modseq tracking enabled, return the mailbox as having NOMODSEQ in SELECT/EXAMINE reply. Old versions in this situation always simply returned HIGHESTMODSEQ as 1, which could have broken some clients. + dict file: Added optional fcntl/flock locking (default is dotlock) + fts-solr: doveadm fts rescan now resets indexes, which allows reindexing mails. (This isn't a full rescan implementation like fts-lucene has.) + doveadm expunge: Added -d parameter to delete mailbox if it's empty after expunging. - IMAP: Several fixes related to mailbox listing in some configs - director: A lot of fixes and performance improvements - v2.1.4 didn't work without a mail home directory set - mbox: Deleting a mailbox didn't delete its index files. - pop3c: TOP command was sent incorrectly - trash plugin didn't work properly - LMTP: Don't add a duplicate Return-Path: header when proxying. - listescape: Don't unescape namespace prefixes. From stsiol at yahoo.co.uk Mon Apr 23 17:09:17 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 23 Apr 2012 15:09:17 +0100 (BST) Subject: [Dovecot] v2.1.5 released In-Reply-To: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <1335190157.12813.YahooMailNeo@web132202.mail.ird.yahoo.com> >Oh, and looks like I got accepted as a speaker to LinuxTag 2012. The plan is to talk about some new things in Dovecot and dsync replication (looks like I have to hurry up with the dsync rewrite). > I am quite amazed they haven't called you earlier. From what it seems you are doing a great job and dovecot is a rock-solid piece of software. Just my thoughts, s. ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From giles at coochey.net Mon Apr 23 17:12:50 2012 From: giles at coochey.net (Giles Coochey) Date: Mon, 23 Apr 2012 15:12:50 +0100 Subject: [Dovecot] v2.1.5 released In-Reply-To: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <4F956362.7010507@coochey.net> On 23/04/2012 14:49, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz > http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz.sig > > A few announcements first: > > I'm now going to start being more strict about not answering Dovecot questions sent to me privately (especially support requests, but also bug reports, feature requests, etc). Often those questions could be answered by other people in the mailing list as well, so they just waste my time. Even if you're sure that I'm the only person who can possibly answer it, it may be something that others are interested of as well and replying privately denies them the possibility of reading about it. If you really do not want to send something publicly, you still have an option for private communication: just buy one of the Dovecot support packages. > I'd just like to say thanks for the amount of time you put in not just developing the software, but supporting your userbase. I am sure everyone here is well aware of when Timo signs in each day as a flood of replies to queries on the dovecot mailing list starts to arrive to our mailboxes. It is very much appreciated, there are many OSS mailing-lists out there, but not many as vibrant as dovecots! -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4928 bytes Desc: S/MIME Cryptographic Signature URL: From l.chelchowski at eurocar.pl Mon Apr 23 17:27:54 2012 From: l.chelchowski at eurocar.pl (l.chelchowski at eurocar.pl) Date: Mon, 23 Apr 2012 16:27:54 +0200 Subject: [Dovecot] virtual plugin and shared namespace In-Reply-To: References: <4F4235B8.6090107@eurocar.pl> Message-ID: <4F9566EA.6040109@eurocar.pl> W dniu 2012-02-21 01:44, Timo Sirainen pisze: > On 20.2.2012, at 13.59, l.chelchowski at eurocar.pl wrote: > >> I have a problem with virtual plugin and geting mails form shared namespace. >> dovecot-virtual: >> ---- >> Shared/* >> Public/* >> * > .. >> When I enter the virtual mailbox Unread I get this debug info: >> virtual mailbox Virtual/Unread: Skipping non-existing mailbox Shared/test at domain.pl/ > * matches only within namespace. Shared/* matches only the entries directly under Shared/, i.e. Shared/test at domain.pl itself. Shared/test at domain.pl/* would again be a separate namespace. > >> Is it a bug or misconfig? > Maybe a "missing feature" :) I guess Shared/* should match all autocreated child namespaces.. This should help: > http://hg.dovecot.org/dovecot-2.1/rev/10994a5b64e1 > I have installed version 2.1.3 and this patch doesn't work as expected. I can't use Shared/*. Only Shared/test at domain.pl/* works. -- Pozdrawiam ?ukasz Che?chowski Informatyk Autoryzowany Dealer Ford Euro-Car Sp. z o.o. tel. 58 667 327 mailto: l.chelchowski at eurocar.pl -------------- next part -------------- A non-text attachment was scrubbed... Name: stopka2.gif Type: image/gif Size: 17309 bytes Desc: not available URL: From andrei at lctax.ro Mon Apr 23 19:17:35 2012 From: andrei at lctax.ro (Michescu Andrei) Date: Mon, 23 Apr 2012 12:17:35 -0400 Subject: [Dovecot] v2.1.5 released In-Reply-To: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <26fed0264da5829b3376056abd7dd1fd.squirrel@web.miau.ca> > Oh, and looks like I got accepted as a speaker to LinuxTag 2012. The plan > is to talk about some new things in Dovecot and dsync replication (looks > like I have to hurry up with the dsync rewrite). > Hey Timo, Thank you very much for your hard work ;) Please let us know if there is anything that we can help you clarify / design for the dsync rewrite project. Best regards, Andrei From cor at xs4all.nl Mon Apr 23 21:34:39 2012 From: cor at xs4all.nl (Cor Bosman) Date: Mon, 23 Apr 2012 20:34:39 +0200 Subject: [Dovecot] quota not being calculated In-Reply-To: <96E3D11C-8F71-4726-816A-6C448554BC70@iki.fi> References: <62C648EC-684C-4E09-8217-227C87813290@xs4all.nl> <28F58C71-C049-470A-9BA2-4DC411ED8A62@iki.fi> <2D57E521-3DB3-41D9-B8AB-F280DE546061@iki.fi> <96E3D11C-8F71-4726-816A-6C448554BC70@iki.fi> Message-ID: > > http://hg.dovecot.org/dovecot-2.1/rev/4c8f79d1f9f1 should fix it with dict quota. Thank you, this fixed it with dict quota. Cor From dovecot at allycomm.com Mon Apr 23 22:56:13 2012 From: dovecot at allycomm.com (Jeff Kletsky) Date: Mon, 23 Apr 2012 12:56:13 -0700 Subject: [Dovecot] dsync on large mailbox "fails" Message-ID: <4F95B3DD.1070609@allycomm.com> I've been trying to convert an existing set of mbox mail to maildir using dsync, but it seems to fail when it gets to the large boxes. $ dsync -Dv mirror maildir:~/Maildir Everything seems to work reasonably well until the first of the large boxes hits: dsync(jeff): Warning: Maildir /home/jeff/Maildir/.cron.2010-07-10: Synchronization took 1210 seconds (102289 new msgs, 0 flag change attempts, 0 expunge attempts) dsync(jeff): Warning: Transaction log file /home/jeff/Maildir/.cron.2010-07-10/dovecot.index.log was locked for 1210 seconds After this, the doveadm process goes from ~75% of the CPU core utilization to 100% and stays there, even overnight. I've also tried $ dsync -Dv backup maildir:~/Maildir with similar results. While I did see some threads on dsync speed, they seemed to be minutes for a few GB of data, not hours. There is plenty of space on the drive, and, from what I understand, there is not a file-count limit on FreeBSD's ufs (at least not one that I'm anywhere near) Any suggestions? Thanks! Jeff [jeff at mail ~]$ du -h -d 0 ~/mail 3.9G /home/jeff/mail [jeff at mail ~]$ ls -lh /var/mail/jeff -rw------- 1 jeff jeff 554M Apr 23 11:27 /var/mail/jeff [jeff at mail ~]$ df -h Filesystem Size Used Avail Capacity Mounted on /dev/ada0s2g 38G 18G 16G 53% / (Note that the hostname and the location of the ssl certificate has been manually edited for this message in the following output) From dmesg.boot: Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-STABLE #0: Sat Apr 21 17:13:07 PDT 2012 root at my.host.name:/usr/obj/usr/src/sys/GENERIC i386 CPU: Intel(R) Atom(TM) CPU 330 @ 1.60GHz (1600.03-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x106c2 Family = 6 Model = 1c Stepping = 2 Features=0xbfe9fbff Features2=0x40e31d AMD Features=0x20100000 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 2147483648 (2048 MB) avail memory = 2077757440 (1981 MB) [jeff at mail ~]$ doveconf -n # 2.1.3: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 9.0-STABLE i386 auth_mechanisms = plain login disable_plaintext_auth = no first_valid_gid = 1000 first_valid_uid = 1000 listen = * mail_location = mbox:~/mail/:INBOX=/var/mail/%u mail_privileged_group = mail namespace { inbox = yes location = prefix = separator = / } passdb { args = session=yes dovecot driver = pam } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { mode = 0600 } user = root } service imap-login { inet_listener imap { port = 0 } process_limit = 200 } ssl = required ssl_cert = Hey all, has anyone ever tried turning on the trash plugin when the directory is already over quota? I see some messages being deleted, but it seems it just deletes enough to fit the new email, not enough to go below quota. Example: . getquotaroot "Spam" * QUOTAROOT "Spam" "User quota" "Spam Quota" * QUOTA "User quota" (STORAGE 591036 5242880) * QUOTA "Spam Quota" (STORAGE 21173 20480) I now move a 5K message into this folder, and I see: Apr 23 22:26:43 lmtp1 dovecot: imap(cor): expunge: box=Spam, uid=226337, msgid=<3YVKIIE4II6R0064BA7IM55C2Q4MD01VX12U1I at shedevils.com>, size=12984 Apr 23 22:26:43 lmtp1 dovecot: imap(cor): expunge: box=Spam, uid=226338, msgid=, size=5623 Apr 23 22:26:43 lmtp1 dovecot: imap(cor): save: box=Spam, uid=227759, msgid=<20120423191509.F3B584A355B at signal.groundspeak.com>, size=2694 And now: . getquotaroot "Spam" * QUOTAROOT "Spam" "User quota" "Spam Quota" * QUOTA "User quota" (STORAGE 591016 5242880) * QUOTA "Spam Quota" (STORAGE 21158 20480) . OK Getquotaroot completed. This seems like a possible bug. For one, it shouldnt be allowed to actually save a message into that folder, but it does. The email does end up in the Spam folder even though the folder is over quota. And of course it doesnt clear up enough space to have the new email fit, it just deleted enough to match the 5K. Without the trash plugin quota is being properly enforced. Everything works fine if you start out below the quota limit btw. Regards, Cor ps: this is 2.1.4, but with the recent trash and quota patches applied. From asai at globalchangemusic.org Tue Apr 24 00:50:36 2012 From: asai at globalchangemusic.org (Asai) Date: Mon, 23 Apr 2012 14:50:36 -0700 Subject: [Dovecot] v2.1.5 released In-Reply-To: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <4F95CEAC.10103@globalchangemusic.org> Let me also thank you, Timo, for your selfless excellence in developing Dovecot. --asai On 4/23/2012 6:49 AM, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz > http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz.sig > > A few announcements first: > > I'm now going to start being more strict about not answering Dovecot questions sent to me privately (especially support requests, but also bug reports, feature requests, etc). Often those questions could be answered by other people in the mailing list as well, so they just waste my time. Even if you're sure that I'm the only person who can possibly answer it, it may be something that others are interested of as well and replying privately denies them the possibility of reading about it. If you really do not want to send something publicly, you still have an option for private communication: just buy one of the Dovecot support packages. > > This is also the first Dovecot release that has run through automated release testing. I've been planning on doing that for last 5 years at least, and finally it's done. The testing includes running a lot of tests to verify that the IMAP server gives proper responses, stress testing doesn't break it, memory isn't leaked and several other things. The tests are run for many different kinds of configurations. > > Some bugs show up only for some specific combination of settings, but unfortunately because there's such a large number of them it's impossible to test every combination. So here's another advantage that Dovecot support customers have: they can have their specific configuration included in the automated tests. This release already fixes one bug I found only after testing a customer's config. > > Oh, and looks like I got accepted as a speaker to LinuxTag 2012. The plan is to talk about some new things in Dovecot and dsync replication (looks like I have to hurry up with the dsync rewrite). > > And finally, the most important changes since v2.1.4: > > * IMAP: When neither the session nor the mailbox has modseq tracking > enabled, return the mailbox as having NOMODSEQ in SELECT/EXAMINE > reply. Old versions in this situation always simply returned > HIGHESTMODSEQ as 1, which could have broken some clients. > > + dict file: Added optional fcntl/flock locking (default is dotlock) > + fts-solr: doveadm fts rescan now resets indexes, which allows > reindexing mails. (This isn't a full rescan implementation like > fts-lucene has.) > + doveadm expunge: Added -d parameter to delete mailbox if it's > empty after expunging. > - IMAP: Several fixes related to mailbox listing in some configs > - director: A lot of fixes and performance improvements > - v2.1.4 didn't work without a mail home directory set > - mbox: Deleting a mailbox didn't delete its index files. > - pop3c: TOP command was sent incorrectly > - trash plugin didn't work properly > - LMTP: Don't add a duplicate Return-Path: header when proxying. > - listescape: Don't unescape namespace prefixes. > From dovecot at allycomm.com Tue Apr 24 06:38:16 2012 From: dovecot at allycomm.com (Jeff Kletsky) Date: Mon, 23 Apr 2012 20:38:16 -0700 Subject: [Dovecot] dsync on large mailbox "fails" In-Reply-To: <4F95B3DD.1070609@allycomm.com> References: <4F95B3DD.1070609@allycomm.com> Message-ID: <4F962028.3060501@allycomm.com> On 4/23/12 12:56 PM, Jeff Kletsky wrote: > I've been trying to convert an existing set of mbox mail to maildir > using dsync, but it seems to fail when it gets to the large boxes. > > $ dsync -Dv mirror maildir:~/Maildir > > Everything seems to work reasonably well until the first of the large > boxes hits: > > dsync(jeff): Warning: Maildir /home/jeff/Maildir/.cron.2010-07-10: > Synchronization took 1210 seconds (102289 new msgs, 0 flag change > attempts, 0 expunge attempts) > dsync(jeff): Warning: Transaction log file > /home/jeff/Maildir/.cron.2010-07-10/dovecot.index.log was locked for > 1210 seconds > > After this, the doveadm process goes from ~75% of the CPU core > utilization to 100% and stays there, even overnight. > I believe that some of problem may be that there were a handful of boxes named with "." in them. These were not "flagged" in the conversion process in any way. http://wiki2.dovecot.org/Migration/MailFormat indicates that you can't have a "." in the name of a Maildir box (I dutifully stopped reading that page at "If you can successfully use dsync, you can skip the rest of this page.") In my case, it may be a contributing factor that there were boxes that ended up "stacking" inside each other: * cron * cron.2010-07-10 * cron.2010-07-10.remaining Still not blisteringly fast, but dsync(jeff): Warning: Maildir /home/jeff/Maildir/.cron_2010-07-10: Synchronization took 686 seconds (102289 new msgs, 0 flag change attempts, 0 expunge attempts) dsync(jeff): Warning: Transaction log file /home/jeff/Maildir/.cron_2010-07-10/dovecot.index.log was locked for 686 seconds is certainly better. I'm still seeing 100% CPU utilization, with the tail of ls -Altr in ~/Maildir showing drwx------ 5 jeff jeff 512 Apr 23 18:19 .XDA drwx------ 5 jeff jeff 512 Apr 23 18:19 .Spam drwx------ 5 jeff jeff 512 Apr 23 18:42 .Health drwx------ 5 jeff jeff 512 Apr 23 18:42 .cron_2010-07-10_remaining drwx------ 5 jeff jeff 512 Apr 23 18:58 .cron_2010-07-10 It appears from looking in the .cron_2010-07-10 directory that indexing is the action underway [jeff at mail ~/Maildir]$ ls -Altr .cron_2010-07-10 total 27 drwx------ 2 jeff jeff 512 Apr 23 18:15 new -rw------- 1 jeff jeff 0 Apr 23 18:15 maildirfolder drwx------ 2 jeff jeff 5819392 Apr 23 18:42 tmp drwx------ 2 jeff jeff 5237248 Apr 23 18:42 cur -rw------- 1 jeff jeff 4122971 Apr 23 18:42 dovecot-uidlist -rw------- 1 jeff jeff 36 Apr 23 18:42 dovecot-keywords -rw------- 1 jeff jeff 1650764 Apr 23 18:42 dovecot.index.log.2 -rw------- 1 jeff jeff 1637008 Apr 23 18:42 dovecot.index -rw------- 1 jeff jeff 56124 Apr 23 19:04 dovecot.index.log -rw------- 1 jeff jeff 9442304 Apr 23 19:04 dovecot.index.cache -rw------- 1 jeff jeff 24 Apr 23 19:04 dovecot-uidlist.lock (time passes) It is now 20:35 and it has not advanced. I now no longer believe the problem is the mailbox naming, but the indexing. PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 25934 jeff 1 103 0 135M 102M CPU2 2 128:47 100.00% doveadm [jeff at mail ~/Maildir]$ ls -Altr .cron_2010-07-10 total 28 drwx------ 2 jeff jeff 512 Apr 23 18:15 new -rw------- 1 jeff jeff 0 Apr 23 18:15 maildirfolder drwx------ 2 jeff jeff 5819392 Apr 23 18:42 tmp drwx------ 2 jeff jeff 5237248 Apr 23 18:42 cur -rw------- 1 jeff jeff 4122971 Apr 23 18:42 dovecot-uidlist -rw------- 1 jeff jeff 36 Apr 23 18:42 dovecot-keywords -rw------- 1 jeff jeff 1650764 Apr 23 18:42 dovecot.index.log.2 -rw------- 1 jeff jeff 1637008 Apr 23 20:26 dovecot.index -rw------- 1 jeff jeff 289120 Apr 23 20:36 dovecot.index.log -rw------- 1 jeff jeff 9442304 Apr 23 20:36 dovecot.index.cache -rw------- 1 jeff jeff 24 Apr 23 20:36 dovecot-uidlist.lock Any suggestions as to how to proceed? Jeff From robert at schetterer.org Tue Apr 24 09:56:26 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 24 Apr 2012 08:56:26 +0200 Subject: [Dovecot] migration dove 2.0 2.1 shared namespace acl broken / shared/%%n/ works / solved by remove listescape In-Reply-To: References: <4F8D586D.3010706@schetterer.org> <4F8D80D3.2050505@schetterer.org> <4F8EFD77.90103@schetterer.org> <4F90250E.3000200@schetterer.org> <181B5335-5A28-4815-88F0-1836B9C6718D@iki.fi> <4F903DC1.5070501@schetterer.org> <4F905B2C.8040204@schetterer.org> <4F907529.2020403@schetterer.org> Message-ID: <4F964E9A.2050401@schetterer.org> Am 23.04.2012 13:12, schrieb Timo Sirainen: > On 19.4.2012, at 23.27, Robert Schetterer wrote: > >>> prefix = shared/%%u/ simply dont works >>> >>> i have reverted back to exact namespace config >>> of the 2.0.20 server, same result, so something broken in 2.1 >>> >>> >> Hi Timo >> problem solved by removing listescape plugin >> the acl paths did / instead of dot in domainname >> >> any chance to get listescape fixed ? > > Ah, that makes it easy to fix: http://hg.dovecot.org/dovecot-2.1/rev/63af3274fb6f > Hi Timo, for small tests problem seems to be fixed, thx youre great -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Tue Apr 24 10:50:06 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 24 Apr 2012 09:50:06 +0200 Subject: [Dovecot] doveadm delete folders ? / unsubscribe ? In-Reply-To: <4F906EE8.1030900@schetterer.org> References: <4F900066.9030808@schetterer.org> <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> <4F906EE8.1030900@schetterer.org> Message-ID: <4F965B2E.2010103@schetterer.org> Am 19.04.2012 22:00, schrieb Robert Schetterer: > Am 19.04.2012 17:16, schrieb Timo Sirainen: >> On 19.4.2012, at 15.09, Robert Schetterer wrote: >> >>> Hi , >>> is there a feature like >>> doveadm expunge savedbefore ... >>> for imap folder delete ? >> >> Here: >> >> http://hg.dovecot.org/dovecot-2.1/rev/2d8bafd11569 >> http://hg.dovecot.org/dovecot-2.1/rev/782570f644f7 >> > > thx Timo ! > Hi Timo doveadm expunge -d -u *@domain.example mailbox exampelfolder\* savedbefore 10d works like charm, but the folders do not get unsubscribed any chance to have i.e another add parameter ,or do it auto cause empty and deleted folders should not stay subscribed in my eyes -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From lists at wiesinger.com Tue Apr 24 11:59:20 2012 From: lists at wiesinger.com (Gerhard Wiesinger) Date: Tue, 24 Apr 2012 10:59:20 +0200 (CEST) Subject: [Dovecot] v2.1.5 released In-Reply-To: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: Hello Timo, Dovecot 2.1.5 seems to be broken. After upgrade from 2.1.3 + patch http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e I only see Inbox and Trash folder. Reverting => OK. Same config as before and disucssed per private E-mail and on the list. Guessing this comes again from: mail_full_filesystem_access=yes Ciao, Gerhard -- http://www.wiesinger.com/ On Mon, 23 Apr 2012, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz > http://dovecot.org/releases/2.1/dovecot-2.1.5.tar.gz.sig > > A few announcements first: > > I'm now going to start being more strict about not answering Dovecot questions sent to me privately (especially support requests, but also bug reports, feature requests, etc). Often those questions could be answered by other people in the mailing list as well, so they just waste my time. Even if you're sure that I'm the only person who can possibly answer it, it may be something that others are interested of as well and replying privately denies them the possibility of reading about it. If you really do not want to send something publicly, you still have an option for private communication: just buy one of the Dovecot support packages. > > This is also the first Dovecot release that has run through automated release testing. I've been planning on doing that for last 5 years at least, and finally it's done. The testing includes running a lot of tests to verify that the IMAP server gives proper responses, stress testing doesn't break it, memory isn't leaked and several other things. The tests are run for many different kinds of configurations. > > Some bugs show up only for some specific combination of settings, but unfortunately because there's such a large number of them it's impossible to test every combination. So here's another advantage that Dovecot support customers have: they can have their specific configuration included in the automated tests. This release already fixes one bug I found only after testing a customer's config. > > Oh, and looks like I got accepted as a speaker to LinuxTag 2012. The plan is to talk about some new things in Dovecot and dsync replication (looks like I have to hurry up with the dsync rewrite). > > And finally, the most important changes since v2.1.4: > > * IMAP: When neither the session nor the mailbox has modseq tracking > enabled, return the mailbox as having NOMODSEQ in SELECT/EXAMINE > reply. Old versions in this situation always simply returned > HIGHESTMODSEQ as 1, which could have broken some clients. > > + dict file: Added optional fcntl/flock locking (default is dotlock) > + fts-solr: doveadm fts rescan now resets indexes, which allows > reindexing mails. (This isn't a full rescan implementation like > fts-lucene has.) > + doveadm expunge: Added -d parameter to delete mailbox if it's > empty after expunging. > - IMAP: Several fixes related to mailbox listing in some configs > - director: A lot of fixes and performance improvements > - v2.1.4 didn't work without a mail home directory set > - mbox: Deleting a mailbox didn't delete its index files. > - pop3c: TOP command was sent incorrectly > - trash plugin didn't work properly > - LMTP: Don't add a duplicate Return-Path: header when proxying. > - listescape: Don't unescape namespace prefixes. > > > From alessio at skye.it Tue Apr 24 12:15:08 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 24 Apr 2012 11:15:08 +0200 Subject: [Dovecot] v2.1.5 released In-Reply-To: References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <4F966F1C.2080204@skye.it> Il 24/04/2012 10:59, Gerhard Wiesinger ha scritto: > Hello Timo, > > Dovecot 2.1.5 seems to be broken. After upgrade from > 2.1.3 + patch http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e > I only see Inbox and Trash folder. > > Reverting => OK. > > Same config as before and disucssed per private E-mail and on the list. > Guessing this comes again from: > mail_full_filesystem_access=yes > > Ciao, > Gerhard Hi Gerhard, for us 2.1.5 works fine. We are using Maildir as mail storage type. Ciao -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From joseba.torre at ehu.es Tue Apr 24 12:25:38 2012 From: joseba.torre at ehu.es (Joseba Torre) Date: Tue, 24 Apr 2012 11:25:38 +0200 Subject: [Dovecot] Error compiling pigeonhole 0.3.0 with dovecot-2.1.5 in openbsd 5.0 Message-ID: <4F967192.2060701@ehu.es> Hi, I've just tried to compile dovecot & pigeonhole in my openbsd box, and got this: gcc -DHAVE_CONFIG_H -I. -I../.. -I/home/seajob/build/dovecot-2.1.5 -I/home/seajob/build/dovecot-2.1.5/src/lib -I/home/seajob/build/dovecot-2.1.5/src/lib-dict -I/home/seajob/build/dovecot-2.1.5/src/lib-dns -I/home/seajob/build/dovecot-2.1.5/src/lib-mail -I/home/seajob/build/dovecot-2.1.5/src/lib-imap -I/home/seajob/build/dovecot-2.1.5/src/lib-fs -I/home/seajob/build/dovecot-2.1.5/src/lib-charset -I/home/seajob/build/dovecot-2.1.5/src/lib-index -I/home/seajob/build/dovecot-2.1.5/src/lib-storage -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index/raw -I../.. -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/include -MT managesieve-quote.o -MD -MP -MF .deps/managesieve-quote.Tpo -c -o managesieve-quote.o managesieve-quote.c mv -f .deps/managesieve-quote.Tpo .deps/managesieve-quote.Po gcc -DHAVE_CONFIG_H -I. -I../.. -I/home/seajob/build/dovecot-2.1.5 -I/home/seajob/build/dovecot-2.1.5/src/lib -I/home/seajob/build/dovecot-2.1.5/src/lib-dict -I/home/seajob/build/dovecot-2.1.5/src/lib-dns -I/home/seajob/build/dovecot-2.1.5/src/lib-mail -I/home/seajob/build/dovecot-2.1.5/src/lib-imap -I/home/seajob/build/dovecot-2.1.5/src/lib-fs -I/home/seajob/build/dovecot-2.1.5/src/lib-charset -I/home/seajob/build/dovecot-2.1.5/src/lib-index -I/home/seajob/build/dovecot-2.1.5/src/lib-storage -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index/raw -I../.. -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/include -MT managesieve-parser.o -MD -MP -MF .deps/managesieve-parser.Tpo -c -o managesieve-parser.o managesieve-parser.c managesieve-parser.c: In function 'quoted_string_istream_read': managesieve-parser.c:639: error: 'EPROTO' undeclared (first use in this function) managesieve-parser.c:639: error: (Each undeclared identifier is reported only once managesieve-parser.c:639: error: for each function it appears in.) *** Error code 1 Stop in /home/seajob/build/dovecot-2.1-pigeonhole-0.3.0/src/lib-managesieve (line 92 of /usr/share/mk/sys.mk). *** Error code 1 Stop in /home/seajob/build/dovecot-2.1-pigeonhole-0.3.0/src (line 289 of Makefile). *** Error code 1 Stop in /home/seajob/build/dovecot-2.1-pigeonhole-0.3.0 (line 474 of Makefile). *** Error code 1 Stop in /home/seajob/build/dovecot-2.1-pigeonhole-0.3.0 (line 353 of Makefile). From piotr.maier at gmail.com Tue Apr 24 12:51:43 2012 From: piotr.maier at gmail.com (Piotr Maier) Date: Tue, 24 Apr 2012 11:51:43 +0200 Subject: [Dovecot] Dovecot Virtual Users with Users Domain Mapping Message-ID: I have successfully compiled, configured and ran Dovecot with virtual users feature. Here's part of my /etc/dovecot.conf configuration file: mail_location = maildir:~/Maildir auth default { mechanisms = plain login userdb passwd-file { args = /home/%d/etc/passwd } passdb passwd-file { args = /home/%d/etc/shadow } socket listen { master { path = /var/run/dovecot/auth-worker mode = 0600 } } } I faced one issue I can't resolve myself. Is there anyway to create users' domains mapping and provide username in mail_location? Example of usersdomains file: domain.com:user1 domain2.org:user3 Examples: 1. currently I have /home/domain.com/user/Maildir 2. I'd like to have /home/SYSTEM_USER/domain.com/user/Maildir Can I achieve this somehow? From tss at iki.fi Tue Apr 24 13:26:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 24 Apr 2012 13:26:32 +0300 Subject: [Dovecot] dsync on large mailbox "fails" In-Reply-To: <4F962028.3060501@allycomm.com> References: <4F95B3DD.1070609@allycomm.com> <4F962028.3060501@allycomm.com> Message-ID: <4EE7C62E-928C-499D-BE4E-9944D6013198@iki.fi> On 24.4.2012, at 6.38, Jeff Kletsky wrote: > 25934 jeff 1 103 0 135M 102M CPU2 2 128:47 100.00% doveadm Looks like it could be in an infinite loop. To find out where: 1) strace for a few seconds: strace -tt -p 25934 2>log 2) get gdb backtrace a couple of times (2-5 should be enough, preferably slightly different ones): gdb /usr/local/bin/doveadm 25934 bt full cont bt full cont ..etc.. From markus at mpetri.org Tue Apr 24 14:00:05 2012 From: markus at mpetri.org (Markus Petri) Date: Tue, 24 Apr 2012 13:00:05 +0200 Subject: [Dovecot] Permission problems when trying to create new folders with ACL plugin Message-ID: <20120424130005.0db3d425@legolas.home.ceotex.de> Hi, I have the a problem with shared folders and the ACL plugin. The user "markus" has full access to "Shared/shared-user/Data": 3 GETACL "Shared/shared-user/Data" * ACL "Shared/shared-user/Data" "markus" akxeilprwtscd 3 OK Getacl completed. but creating a new subfolder under "Data" fails: 4 CREATE "Shared/shared-user/Data/New1" 4 NO [NOPERM] Permission denied This used to work with the same configuration and Dovecot 2.0. Cheers, Markus dovecot -n output: # 2.1.5: /opt/dovecot-2.1/etc/dovecot/dovecot.conf # OS: Linux 3.2.0-2-amd64 x86_64 Debian wheezy/sid auth_mechanisms = plain login disable_plaintext_auth = no listen = 192.168.56.11 mail_location = maildir:~/Maildir mail_plugins = acl managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = prefix = separator = / subscriptions = yes type = private } namespace { inbox = no list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/index/shared/%%u prefix = Shared/%%u/ separator = / subscriptions = no type = shared } passdb { args = /opt/dovecot-2.1/etc/dovecot/passwd driver = passwd-file } plugin { acl = vfile acl_anyone = allow acl_shared_dict = file:/var/lib/vdovecot/shared-mailboxes.db } protocols = imap service auth { unix_listener auth-userdb { mode = 0600 user = vdovecot } } ssl = no userdb { args = /opt/dovecot-2.1/etc/dovecot/passwd driver = passwd-file } verbose_proctitle = yes protocol imap { mail_plugins = acl imap_acl } From stephan at rename-it.nl Tue Apr 24 14:13:54 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 24 Apr 2012 13:13:54 +0200 Subject: [Dovecot] Error compiling pigeonhole 0.3.0 with dovecot-2.1.5 in openbsd 5.0 In-Reply-To: <4F967192.2060701@ehu.es> References: <4F967192.2060701@ehu.es> Message-ID: <4F968AF2.4090807@rename-it.nl> Op 4/24/2012 11:25 AM, Joseba Torre schreef: > Hi, > > I've just tried to compile dovecot & pigeonhole in my openbsd box, and > got this: > > gcc -DHAVE_CONFIG_H -I. -I../.. -I/home/seajob/build/dovecot-2.1.5 > -I/home/seajob/build/dovecot-2.1.5/src/lib > -I/home/seajob/build/dovecot-2.1.5/src/lib-dict > -I/home/seajob/build/dovecot-2.1.5/src/lib-dns > -I/home/seajob/build/dovecot-2.1.5/src/lib-mail > -I/home/seajob/build/dovecot-2.1.5/src/lib-imap > -I/home/seajob/build/dovecot-2.1.5/src/lib-fs > -I/home/seajob/build/dovecot-2.1.5/src/lib-charset > -I/home/seajob/build/dovecot-2.1.5/src/lib-index > -I/home/seajob/build/dovecot-2.1.5/src/lib-storage > -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index > -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index/raw -I../.. > -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes > -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 > -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 > -I/usr/include -MT managesieve-quote.o -MD -MP -MF > .deps/managesieve-quote.Tpo -c -o managesieve-quote.o managesieve-quote.c > > mv -f .deps/managesieve-quote.Tpo .deps/managesieve-quote.Po > gcc -DHAVE_CONFIG_H -I. -I../.. -I/home/seajob/build/dovecot-2.1.5 > -I/home/seajob/build/dovecot-2.1.5/src/lib > -I/home/seajob/build/dovecot-2.1.5/src/lib-dict > -I/home/seajob/build/dovecot-2.1.5/src/lib-dns > -I/home/seajob/build/dovecot-2.1.5/src/lib-mail > -I/home/seajob/build/dovecot-2.1.5/src/lib-imap > -I/home/seajob/build/dovecot-2.1.5/src/lib-fs > -I/home/seajob/build/dovecot-2.1.5/src/lib-charset > -I/home/seajob/build/dovecot-2.1.5/src/lib-index > -I/home/seajob/build/dovecot-2.1.5/src/lib-storage > -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index > -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index/raw -I../.. > -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes > -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 > -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 > -I/usr/include -MT managesieve-parser.o -MD -MP -MF > .deps/managesieve-parser.Tpo -c -o managesieve-parser.o > managesieve-parser.c > managesieve-parser.c: In function 'quoted_string_istream_read': > managesieve-parser.c:639: error: 'EPROTO' undeclared (first use in > this function) > managesieve-parser.c:639: error: (Each undeclared identifier is > reported only once > managesieve-parser.c:639: error: for each function it appears in.) Looks like BSD doesn't know EPROTO (although it is POSIX). Dovecot uses EIO elsewhere for such stream errors, so I adopted that for ManageSieve as well. This should fix it: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/ef5045159e6b Regards, Stephan. From ah-news-1204 at freenet.de Tue Apr 24 17:09:58 2012 From: ah-news-1204 at freenet.de (Andreas Helmcke) Date: Tue, 24 Apr 2012 16:09:58 +0200 Subject: [Dovecot] acls not copied when creating subfolder of private INBOX Message-ID: <4F96B436.9090402@freenet.de> Using brand new dovecot 2.1.5 I still have a problem with the inheritance of acls. Setting: - maildir with private INBOX and public folders. - users maildir directory (/home/mail/user/ahelmcke) contains dovecot-acl file - public folders root directory (/home/mail/Fax) contains dovecot-acl file When creating a subfolder of the public folder the dovecot-acl files gets copied to the subfolders directory as expected. When creating a subfolder of the users INBOX the dovecot-acl files does /not/ get copied to the subfolders directory. Debug output: doveadm -Dv mailbox create -u ahelmcke Fax/Buhhhh doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib01_acl_plugin.so doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib10_quota_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_lookup (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_zlib_plugin.so: undefined symbol: i_stream_create_deflate (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_list_backend (this is usually intentional, so just ignore this message) doveadm(ahelmcke): Debug: auth input: ahelmcke quota_rule=*:storage=2000M doveadm(ahelmcke): Debug: Added userdb setting: plugin/quota_rule=*:storage=2000M doveadm(ahelmcke): Debug: Effective uid=494, gid=491, home=/home/mail/user/ahelmcke doveadm(ahelmcke): Debug: Quota root: name=User quota backend=maildir args= doveadm(ahelmcke): Debug: Quota rule: root=User quota mailbox=* bytes=2097152000 messages=0 doveadm(ahelmcke): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke doveadm(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=/home/mail/user/ahelmcke, alt= doveadm(ahelmcke): Debug: acl: initializing backend with data: vfile doveadm(ahelmcke): Debug: acl: acl username = ahelmcke doveadm(ahelmcke): Debug: acl: owner = 1 doveadm(ahelmcke): Debug: acl vfile: Global ACL directory: (none) doveadm(ahelmcke): Debug: Namespace dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ doveadm(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke/public, index=, control=, inbox=, alt= doveadm(ahelmcke): Debug: acl: initializing backend with data: vfile doveadm(ahelmcke): Debug: acl: acl username = ahelmcke doveadm(ahelmcke): Debug: acl: owner = 1 doveadm(ahelmcke): Debug: acl vfile: Global ACL directory: (none) doveadm(ahelmcke): Debug: Namespace fax: type=public, prefix=Fax/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no location=maildir:/home/mail/Fax:INDEX=/home/mail/user/ahelmcke/SEEN.Fax doveadm(ahelmcke): Debug: maildir++: root=/home/mail/Fax, index=/home/mail/user/ahelmcke/SEEN.Fax, control=, inbox=, alt= doveadm(ahelmcke): Debug: acl: initializing backend with data: vfile doveadm(ahelmcke): Debug: acl: acl username = ahelmcke doveadm(ahelmcke): Debug: acl: owner = 0 doveadm(ahelmcke): Debug: acl vfile: Global ACL directory: (none) doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/dovecot-acl doveadm(ahelmcke): Debug: Namespace Fax/: Using permissions from /home/mail/Fax: mode=0700 gid=-1 doveadm(ahelmcke): Debug: Namespace INBOX/: Using permissions from /home/mail/user/ahelmcke: mode=0700 gid=-1 doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/user/ahelmcke/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/user/ahelmcke/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/user/ahelmcke/dovecot-acl doveadm(ahelmcke): Debug: Namespace Fax/: /home/mail/Fax/.Buhhhh doesn't exist yet, using default permissions doveadm(ahelmcke): Debug: Namespace Fax/: Using permissions from /home/mail/Fax: mode=0700 gid=-1 doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: file /home/mail/Fax/.Buhhhh/dovecot-acl not found doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/.Buhhhh/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/user/ahelmcke/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/.Buhhhh/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/.Buhhhh/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/user/ahelmcke/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/.Buhhhh/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/Fax/.Buhhhh/dovecot-acl doveadm(ahelmcke): Debug: acl vfile: reading file /home/mail/user/ahelmcke/dovecot-acl doveadm -Dv mailbox create -u ahelmcke INBOX/Buhhhh doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib01_acl_plugin.so doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib10_quota_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_lookup (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_zlib_plugin.so: undefined symbol: i_stream_create_deflate (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_list_backend (this is usually intentional, so just ignore this message) doveadm(ahelmcke): Debug: auth input: ahelmcke quota_rule=*:storage=2000M doveadm(ahelmcke): Debug: Added userdb setting: plugin/quota_rule=*:storage=2000M doveadm(ahelmcke): Debug: Effective uid=494, gid=491, home=/home/mail/user/ahelmcke doveadm(ahelmcke): Debug: Quota root: name=User quota backend=maildir args= doveadm(ahelmcke): Debug: Quota rule: root=User quota mailbox=* bytes=2097152000 messages=0 doveadm(ahelmcke): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke doveadm(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=/home/mail/user/ahelmcke, alt= doveadm(ahelmcke): Debug: acl: initializing backend with data: vfile doveadm(ahelmcke): Debug: acl: acl username = ahelmcke doveadm(ahelmcke): Debug: acl: owner = 1 doveadm(ahelmcke): Debug: acl vfile: Global ACL directory: (none) doveadm(ahelmcke): Debug: Namespace dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ doveadm(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke/public, index=, control=, inbox=, alt= doveadm(ahelmcke): Debug: acl: initializing backend with data: vfile doveadm(ahelmcke): Debug: acl: acl username = ahelmcke doveadm(ahelmcke): Debug: acl: owner = 1 doveadm(ahelmcke): Debug: acl vfile: Global ACL directory: (none) doveadm(ahelmcke): Debug: Namespace fax: type=public, prefix=Fax/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no location=maildir:/home/mail/Fax:INDEX=/home/mail/user/ahelmcke/SEEN.Fax doveadm(ahelmcke): Debug: maildir++: root=/home/mail/Fax, index=/home/mail/user/ahelmcke/SEEN.Fax, control=, inbox=, alt= doveadm(ahelmcke): Debug: acl: initializing backend with data: vfile doveadm(ahelmcke): Debug: acl: acl username = ahelmcke doveadm(ahelmcke): Debug: acl: owner = 0 doveadm(ahelmcke): Debug: acl vfile: Global ACL directory: (none) doveadm(ahelmcke): Debug: Namespace INBOX/: /home/mail/user/ahelmcke/.Buhhhh doesn't exist yet, using default permissions doveadm(ahelmcke): Debug: Namespace INBOX/: Using permissions from /home/mail/user/ahelmcke: mode=0700 gid=-1 doveconf -n # 2.1.5: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) ext4 auth_master_user_separator = * auth_verbose = yes first_valid_uid = 400 mail_debug = yes mail_gid = vmail mail_home = /home/mail/user/%u mail_location = maildir:/home/mail/user/%u mail_plugins = " quota acl" mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl namespace dummy { alias_for = INBOX/ hidden = yes list = no location = maildir:/home/mail/user/%u/public/ prefix = separator = / type = private } namespace fax { location = maildir:/home/mail/Fax:INDEX=/home/mail/user/%u/SEEN.Fax prefix = Fax/ separator = / subscriptions = no type = public } namespace inbox { inbox = yes location = prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile acl_shared_dict = file:/home/mail/shared-mailboxes quota = maildir:User quota quota_rule = *:storage=3G sieve = /home/mail/user/%u/sieve.active sieve_dir = /home/mail/user/%u/sieve.scripts } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = * } inet_listener imaps { address = * } } service managesieve-login { inet_listener sieve { address = * port = 4190 } } ssl = required ssl_cert = References: <4F96B436.9090402@freenet.de> Message-ID: <4F96BE96.9080505@schetterer.org> Am 24.04.2012 16:09, schrieb Andreas Helmcke: > Setting: > - maildir with private INBOX and public folders. > - users maildir directory (/home/mail/user/ahelmcke) contains dovecot-acl file > - public folders root directory (/home/mail/Fax) contains dovecot-acl file > > When creating a subfolder of the public folder the dovecot-acl files gets copied to the subfolders directory as expected. > When creating a subfolder of the users INBOX the dovecot-acl files does /not/ get copied to the subfolders directory. i am not sure ,but but i think that works like designed http://wiki2.dovecot.org/ACL .. ACL Inheritance Every time you create a new mailbox, it gets its ACLs from the parent mailbox. If you're creating a root-level mailbox, it uses the namespace's default ACLs. There is no actual inheritance, however: If you modify parent's ACLs, the child's ACLs stay the same. There is currently no support for ACL inheritance. Namespace's default ACLs are read from "dovecot-acl" file in the namespace's mail root directory (e.g. /var/public/Maildir). Note that currently these default ACLs are used only when creating new mailboxes, they aren't used for mailboxes without ACLs. .... -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From ah-news-1204 at freenet.de Tue Apr 24 18:17:39 2012 From: ah-news-1204 at freenet.de (Andreas Helmcke) Date: Tue, 24 Apr 2012 17:17:39 +0200 Subject: [Dovecot] acls not copied when creating subfolder of private INBOX In-Reply-To: <4F96BE96.9080505@schetterer.org> References: <4F96B436.9090402@freenet.de> <4F96BE96.9080505@schetterer.org> Message-ID: <4F96C413.7080709@freenet.de> Am 24.04.2012 16:54, schrieb Robert Schetterer: > Am 24.04.2012 16:09, schrieb Andreas Helmcke: >> When creating a subfolder of the public folder the dovecot-acl files gets copied to the subfolders directory as expected. >> When creating a subfolder of the users INBOX the dovecot-acl files does /not/ get copied to the subfolders directory. > > i am not sure ,but but i think that works like designed > > http://wiki2.dovecot.org/ACL > .. > ACL Inheritance > .... I do read this as: when creating a new mailbox; acls are copied from parent. This it what it does for public folders but /not/ for the private ones. So in my opinion at least it is not working as documented. From robert at schetterer.org Tue Apr 24 18:31:08 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 24 Apr 2012 17:31:08 +0200 Subject: [Dovecot] acls not copied when creating subfolder of private INBOX In-Reply-To: <4F96C413.7080709@freenet.de> References: <4F96B436.9090402@freenet.de> <4F96BE96.9080505@schetterer.org> <4F96C413.7080709@freenet.de> Message-ID: <4F96C73C.5010202@schetterer.org> Am 24.04.2012 17:17, schrieb Andreas Helmcke: > Am 24.04.2012 16:54, schrieb Robert Schetterer: >> Am 24.04.2012 16:09, schrieb Andreas Helmcke: >>> When creating a subfolder of the public folder the dovecot-acl files gets copied to the subfolders directory as expected. >>> When creating a subfolder of the users INBOX the dovecot-acl files does /not/ get copied to the subfolders directory. >> >> i am not sure ,but but i think that works like designed >> >> http://wiki2.dovecot.org/ACL >> .. >> ACL Inheritance >> .... > > I do read this as: when creating a new mailbox; acls are copied from parent. > This it what it does for public folders but /not/ for the private ones. So in my opinion at least it is not working as > documented. i think its missunderstandable written this is what counts i think http://wiki2.dovecot.org/ACL ... "There is currently no support for ACL inheritance" .. public folders are special cases.... it makes sense to have inheritance as default acl but this may not be a good idea as default at users folders read the list archive about this stuff, i cant remember all but Timo wrote some about it, at last ,develop of that feature may be still be not finshed -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From l.rame at griensu.com Tue Apr 24 19:07:12 2012 From: l.rame at griensu.com (Leonardo =?iso-8859-1?Q?M=2E_Ram=E9?=) Date: Tue, 24 Apr 2012 13:07:12 -0300 Subject: [Dovecot] Reading Mutt mboxes from Thunderbird Message-ID: <20120424160712.GB5048@leonardo-laptop> Hi, I'm a user of Mutt, and before moving to Thunderbird I'm trying to share mboxes between them. To do this, I installed Dovecot to create an IMAP server in my local machine, to serve mutt mboxes to be read by Thunderbird. So far, I can read my inbox from /var/mail/%u, but in mutt, I have many folders in ~/mail, like 2010-mails, 2011-mails, mailing-list-x, mailing-list-y, and so on. How can I configure Dovecot to read them?. In /etc/dovecot/conf.d/10-mail.conf I have this: mail_location = mbox:~/Mail:INBOX=/var/mail/%u I'm using Ubuntu 11.10, and Dovecot 2.0.13. Regards, -- Leonardo M. Ram? Medical IT - Griensu S.A. Av. Col?n 636 - Piso 8 Of. A X5000EPT -- C?rdoba Tel.: +54(351)4246924 +54(351)4247788 +54(351)4247979 int. 19 Cel.: +54(351)156629292 From interfasys at gmail.com Tue Apr 24 19:31:38 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Tue, 24 Apr 2012 17:31:38 +0100 Subject: [Dovecot] sievec segfaults Message-ID: <4F96D56A.60907@gmail.com> Hello, sievec segfaults when it reads a file. I've tried with both the standard and gold linker. I've tried with various sieve files and they all make it crash. I've compiled pigeonhole with -ggdb, but I'm not getting much info Program received signal SIGSEGV, Segmentation fault. 0x0000000800b8e587 in strlen () from /lib/libc.so.7 (gdb) bt full #0 0x0000000800b8e587 in strlen () from /lib/libc.so.7 No symbol table info available. #1 0x0000000800a12e8c in str_append () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #2 0x000000080077f02d in sieve_binary_save () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #3 0x0000000000605bf0 in environ () No symbol table info available. #4 0x0000000000000000 in ?? () No symbol table info available. #5 0x00007fffffffeb6e in ?? () No symbol table info available. #6 0x0000000000000001 in ?? () No symbol table info available. #7 0x0000000000000000 in ?? () No symbol table info available. #8 0x000000080060878d in ?? () from /libexec/ld-elf.so.1 No symbol table info available. #9 0x0000000000605cd9 in ?? () No symbol table info available. #10 0x0000000801000ea8 in ?? () No symbol table info available. #11 0x0000000000005008 in ?? () No symbol table info available. #12 0x0000000000000000 in ?? () No symbol table info available. #13 0x0000000801099040 in ?? () No symbol table info available. #14 0x0000000000000000 in ?? () No symbol table info available. #15 0x0000000000000180 in ?? () No symbol table info available. #16 0x0000000000605bf0 in environ () No symbol table info available. #17 0x0000000000000000 in ?? () No symbol table info available. #18 0x00007fffffffeb6e in ?? () No symbol table info available. #19 0x0000000000000001 in ?? () No symbol table info available. #20 0x0000000000000000 in ?? () No symbol table info available. #21 0x0000000000000000 in ?? () No symbol table info available. #22 0x0000000000402f93 in main () No symbol table info available. Cheers, Olivier From dovecot at allycomm.com Tue Apr 24 20:05:07 2012 From: dovecot at allycomm.com (Jeff Kletsky) Date: Tue, 24 Apr 2012 10:05:07 -0700 Subject: [Dovecot] dsync on large mailbox "fails" In-Reply-To: <4EE7C62E-928C-499D-BE4E-9944D6013198@iki.fi> References: <4F95B3DD.1070609@allycomm.com> <4F962028.3060501@allycomm.com> <4EE7C62E-928C-499D-BE4E-9944D6013198@iki.fi> Message-ID: <4F96DD43.3070700@allycomm.com> On 4/24/12 3:26 AM, Timo Sirainen wrote: > On 24.4.2012, at 6.38, Jeff Kletsky wrote: > >> 25934 jeff 1 103 0 135M 102M CPU2 2 128:47 100.00% doveadm > Looks like it could be in an infinite loop. To find out where: > > 1) strace for a few seconds: > > strace -tt -p 25934 2>log > > 2) get gdb backtrace a couple of times (2-5 should be enough, preferably slightly different ones): > > gdb /usr/local/bin/doveadm 25934 > bt full > > cont > > bt full > > cont > > ..etc.. Rather than clutter everyone's inbox, even if we know they have a great IMAP server... http://wildside.wagsky.com/dovecot/2012-04-24/54097/doveadm.gdb.log.bz2 http://wildside.wagsky.com/dovecot/2012-04-24/54097/doveadm.truss.log.bz2 Thanks for looking into this! Jeff From tss at iki.fi Tue Apr 24 20:08:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 24 Apr 2012 20:08:38 +0300 Subject: [Dovecot] dsync on large mailbox "fails" In-Reply-To: <4F96DD43.3070700@allycomm.com> References: <4F95B3DD.1070609@allycomm.com> <4F962028.3060501@allycomm.com> <4EE7C62E-928C-499D-BE4E-9944D6013198@iki.fi> <4F96DD43.3070700@allycomm.com> Message-ID: <78F42D61-1402-408B-953B-F85254479761@iki.fi> On 24.4.2012, at 20.05, Jeff Kletsky wrote: > On 4/24/12 3:26 AM, Timo Sirainen wrote: >> On 24.4.2012, at 6.38, Jeff Kletsky wrote: >> >>> 25934 jeff 1 103 0 135M 102M CPU2 2 128:47 100.00% doveadm >> Looks like it could be in an infinite loop. To find out where: > http://wildside.wagsky.com/dovecot/2012-04-24/54097/doveadm.truss.log.bz2 That's a lot of getdirentries() calls in there, repeating all the time. So I don't think it's an infinite loop, just a lot of slowness repeating the same work over and over again. See if it helps if you add: maildir_very_dirty_syncs = yes From stephan at rename-it.nl Tue Apr 24 20:10:21 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 24 Apr 2012 19:10:21 +0200 Subject: [Dovecot] sievec segfaults In-Reply-To: <4F96D56A.60907@gmail.com> References: <4F96D56A.60907@gmail.com> Message-ID: <4F96DE7D.6030807@rename-it.nl> Op 4/24/2012 6:31 PM, interfaSys s?rl schreef: > Hello, > sievec segfaults when it reads a file. > I've tried with both the standard and gold linker. > I've tried with various sieve files and they all make it crash. What version? Regards, Stephan. From interfasys at gmail.com Tue Apr 24 20:14:13 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Tue, 24 Apr 2012 18:14:13 +0100 Subject: [Dovecot] sievec segfaults In-Reply-To: <4F96DE7D.6030807@rename-it.nl> References: <4F96D56A.60907@gmail.com> <4F96DE7D.6030807@rename-it.nl> Message-ID: <4F96DF65.90909@gmail.com> I pulled the latest code from the dovecot-2.1-pigeonhole repository. Cheers, Olivier On 24/04/2012 18:10, Stephan Bosch wrote: > Op 4/24/2012 6:31 PM, interfaSys s?rl schreef: >> Hello, >> sievec segfaults when it reads a file. >> I've tried with both the standard and gold linker. >> I've tried with various sieve files and they all make it crash. > > What version? > > Regards, > > Stephan. From interfasys at gmail.com Tue Apr 24 20:27:51 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Tue, 24 Apr 2012 18:27:51 +0100 Subject: [Dovecot] sievec segfaults In-Reply-To: <4F96DE7D.6030807@rename-it.nl> References: <4F96D56A.60907@gmail.com> <4F96DE7D.6030807@rename-it.nl> Message-ID: <4F96E297.50300@gmail.com> I've recompiled Dovecot with ggdb and here is a more meaningful trace. Program received signal SIGSEGV, Segmentation fault. 0x000000080067796b in str_append () from /usr/lib/dovecot/libdovecot.so.0 (gdb) bt full #0 0x000000080067796b in str_append () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #1 0x000000080047b672 in sieve_binary_save () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #2 0x0000000800461b04 in sieve_save_as () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #3 0x00000000004032e6 in main (argc=2, argv=0x7fffffffe948) at sievec.c:147 svinst = (struct sieve_instance *) 0x800c90040 st = {st_dev = 1288178348, st_ino = 6479, st_mode = 33261, st_nlink = 1, st_uid = 12, st_gid = 6, st_rdev = 4294967295, st_atimespec = { tv_sec = 1335288301, tv_nsec = 418632262}, st_mtimespec = {tv_sec = 1285859319, tv_nsec = 0}, st_ctimespec = {tv_sec = 1297096638, tv_nsec = 658928714}, st_size = 1197, st_blocks = 4, st_blksize = 4096, st_flags = 0, st_gen = 8013395, st_lspare = 0, st_birthtimespec = { tv_sec = 1297096638, tv_nsec = 658928714}} sbin = (struct sieve_binary *) 0x800c99040 dump = false scriptfile = (const __unknown__ *) 0x7fffffffec56 "/etc/test.sieve" outfile = (const __unknown__ *) 0x0 exit_status = 0 c = -1 Cheers, Olivier On 24/04/2012 18:10, Stephan Bosch wrote: > Op 4/24/2012 6:31 PM, interfaSys s?rl schreef: >> Hello, >> sievec segfaults when it reads a file. >> I've tried with both the standard and gold linker. >> I've tried with various sieve files and they all make it crash. > > What version? > > Regards, > > Stephan. From stephan at rename-it.nl Tue Apr 24 20:34:39 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 24 Apr 2012 19:34:39 +0200 Subject: [Dovecot] sievec segfaults In-Reply-To: <4F96DF65.90909@gmail.com> References: <4F96D56A.60907@gmail.com> <4F96DE7D.6030807@rename-it.nl> <4F96DF65.90909@gmail.com> Message-ID: <4F96E42F.4090206@rename-it.nl> Op 4/24/2012 7:14 PM, interfaSys s?rl schreef: > I pulled the latest code from the dovecot-2.1-pigeonhole repository. This is caused by the recent addition of dict database support for storing Sieve scripts. This should fix it: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/93bfa75d9ab9 Regards, Stephan. From almarzuki2001 at hotmail.com Tue Apr 24 20:41:00 2012 From: almarzuki2001 at hotmail.com (Hadi Salem) Date: Tue, 24 Apr 2012 20:41:00 +0300 Subject: [Dovecot] dovecot LDA error Message-ID: Hi, My dovecot version is , # 2.0.20: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-164.el5 x86_64 CentOS release 5.4 (Final) auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_verbose_passwords = plain debug_log_path = /var/log/dovecot.log disable_plaintext_auth = no hostname = host1.bigmama.com info_log_path = /var/log/dovecot.log log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:~/Maildir passdb { args = scheme=PLAIN username_format=%u /etc/dovecot/users driver = passwd-file } postmaster_address = root at host1.bigmama.com service auth { unix_listener auth-userdb { group = mail mode = 0660 user = mail } } ssl = no userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol lda { info_log_path = /var/log/dovecot-lda.log log_path = /var/log/dovecot-lda-errors.log } Im getting error from dovecot LDA logs log_path = /var/log/dovecot-lda-errors.log Apr 24 20:25:46 lda(jamal at example.com): Fatal: setgid(5000(vmail) from userdb lo okup) failed with euid=8(mail), gid=12(mail), egid=12(mail): Operation not permi tted (This binary should probably be called with process group set to 5000(vmail ) instead of 12(mail)) My usersdb config jamal at example.com:{PLAIN}123:5000:5000::/var/vmail/example.com/jamal/ user 5000 system user as vmail Permissions for the /var/vmail directory as follow drwxr-xr-x 3 vmail vmail 4096 Apr 24 20:18 /var/vmail/ drwx------ 3 vmail vmail 4096 Apr 24 20:18 /var/vmail/example.com/ drwx------ 3 vmail vmail 4096 Apr 24 20:18 /var/vmail/example.com/jamal/ drwx------ 5 vmail vmail 4096 Apr 24 20:20 /var/vmail/example.com/jamal/Maildir/ From interfasys at gmail.com Tue Apr 24 20:42:29 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Tue, 24 Apr 2012 18:42:29 +0100 Subject: [Dovecot] sievec segfaults In-Reply-To: <4F96E42F.4090206@rename-it.nl> References: <4F96D56A.60907@gmail.com> <4F96DE7D.6030807@rename-it.nl> <4F96DF65.90909@gmail.com> <4F96E42F.4090206@rename-it.nl> Message-ID: <4F96E605.6080506@gmail.com> It worked! :) Thank you. Olivier On 24/04/2012 18:34, Stephan Bosch wrote: > Op 4/24/2012 7:14 PM, interfaSys s?rl schreef: >> I pulled the latest code from the dovecot-2.1-pigeonhole repository. > > This is caused by the recent addition of dict database support for > storing Sieve scripts. > > This should fix it: > > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/93bfa75d9ab9 > > Regards, > > Stephan. From tss at iki.fi Tue Apr 24 21:10:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 24 Apr 2012 21:10:17 +0300 Subject: [Dovecot] trash plugin bug? In-Reply-To: <932BA6EA-2884-4847-B66A-E056C8068B28@xs4all.nl> References: <932BA6EA-2884-4847-B66A-E056C8068B28@xs4all.nl> Message-ID: <69567A90-450A-457A-BC38-C0DE235156D9@iki.fi> On 23.4.2012, at 23.40, Cor Bosman wrote: > Hey all, has anyone ever tried turning on the trash plugin when the directory is already over quota? I see some messages being deleted, but it seems it just deletes enough to fit the new email, not enough to go below quota. Well, normally you shouldn't be over quota I guess.. :) Anyway, http://hg.dovecot.org/dovecot-2.1/rev/ec8564741aa8 http://hg.dovecot.org/dovecot-2.1/rev/dd3798681283 From tss at iki.fi Tue Apr 24 21:13:23 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 24 Apr 2012 21:13:23 +0300 Subject: [Dovecot] doveadm delete folders ? / unsubscribe ? In-Reply-To: <4F965B2E.2010103@schetterer.org> References: <4F900066.9030808@schetterer.org> <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> <4F906EE8.1030900@schetterer.org> <4F965B2E.2010103@schetterer.org> Message-ID: <283E4240-57E8-4BF2-8DCA-6DCC78C1D5A8@iki.fi> On 24.4.2012, at 10.50, Robert Schetterer wrote: > doveadm expunge -d -u *@domain.example mailbox exampelfolder\* > savedbefore 10d > > works like charm, but the folders do not get unsubscribed > any chance to have i.e another add parameter ,or do it auto > cause empty and deleted folders should not stay subscribed in my eyes Yeah, I guess unsubscribing is always wanted: http://hg.dovecot.org/dovecot-2.1/rev/b1653b7bd165 From cor at xs4all.nl Tue Apr 24 21:23:54 2012 From: cor at xs4all.nl (Cor Bosman) Date: Tue, 24 Apr 2012 20:23:54 +0200 Subject: [Dovecot] trash plugin bug? In-Reply-To: <69567A90-450A-457A-BC38-C0DE235156D9@iki.fi> References: <932BA6EA-2884-4847-B66A-E056C8068B28@xs4all.nl> <69567A90-450A-457A-BC38-C0DE235156D9@iki.fi> Message-ID: <6B17E4EF-17E9-4BD1-95DE-93CC0E0316AD@xs4all.nl> > >> Hey all, has anyone ever tried turning on the trash plugin when the directory is already over quota? I see some messages being deleted, but it seems it just deletes enough to fit the new email, not enough to go below quota. > > Well, normally you shouldn't be over quota I guess.. :) Anyway, :) It's hard to initialize new quota rules on a large userbase without some users falling outside the expected parameters :) Sucks when you have lots of users, you always always hit edge cases. Thanks! Cor From daniel.parthey at informatik.tu-chemnitz.de Wed Apr 25 05:13:55 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 25 Apr 2012 04:13:55 +0200 Subject: [Dovecot] How to remove duplicate GUID messages from storage? In-Reply-To: <20120421202928.GA12998@daniel.localdomain> References: <20120421202928.GA12998@daniel.localdomain> Message-ID: <20120425021355.GA10493@daniel.localdomain> Daniel Parthey schrieb: > I tried to import messages like this: > doveadm import -u username at example.org mdbox:/mail/dovecot/example.org/username/mail mbox:/root/mail.txt all > > This seemed to have imported the same messages in the mail storage, > as they were already there. And now I have got a lot of duplicates. > > Then I deleted the mbox:/root/mail mail stuff from "mailboxes" folder > and the storage remained, with duplicate GUIDs. > > The following command > doveadm -v force-resync -u username at example.org INBOX > outputs lots of messages like this one, with different GUID > doveadm(username at example.org): Error: mdbox /mail/dovecot/example.org/username/mail/storage: Duplicate GUID 0b8a032d66a0924fb42c0000de5f8128 in m.55:45484041 and m.14:52173045 > > The messages at m.55:45484041 and m.14:52173045 have the same content, > since they were seemingly imported from the mail store itself, > they are in the storage twice now and require twice as much disk > space as before. > > How can I manually remove these identical, duplicate messages > from the storage to save space? Dovecot does not do it > automatically. Should I edit the mdbox storage files directly using vim in order to remove the duplicate messages which I imported by accident, or is there any dovecot mdbox "repair toolkit" with removes duplicate messages? I attached the dovecot version and config. Regards, Daniel # doveconf -n # 2.0.20: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-40-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_verbose = yes auth_verbose_passwords = sha1 deliver_log_format = mailbox: deliver: msgid=%m from=%f: %$ dict { quota = mysql:/etc/dovecot/conf.d/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = Mailbox login_log_format = mailbox: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "mailbox: mail: %s(%u): " mail_plugins = quota mail_privileged_group = vmail mail_uid = vmail managesieve_implementation_string = Sieve managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 50 M mmap_disable = yes passdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve service dict { unix_listener dict { group = vmail mode = 0660 } } service imap-login { inet_listener imap { port = 19143 } } service lmtp { inet_listener lmtp { address = * port = 19024 } } service managesieve-login { inet_listener sieve { port = 19200 } } service pop3-login { inet_listener pop3 { port = 19110 } } service quota-warning { executable = script /usr/local/bin/quota-warning unix_listener quota-warning { user = vmail } user = dovecot } ssl = no userdb { driver = prefetch } userdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_plugins = quota imap_quota } protocol lmtp { mail_plugins = quota sieve } From robert at schetterer.org Wed Apr 25 09:13:02 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 08:13:02 +0200 Subject: [Dovecot] doveadm delete folders ? / unsubscribe ? In-Reply-To: <283E4240-57E8-4BF2-8DCA-6DCC78C1D5A8@iki.fi> References: <4F900066.9030808@schetterer.org> <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> <4F906EE8.1030900@schetterer.org> <4F965B2E.2010103@schetterer.org> <283E4240-57E8-4BF2-8DCA-6DCC78C1D5A8@iki.fi> Message-ID: <4F9795EE.4030604@schetterer.org> Am 24.04.2012 20:13, schrieb Timo Sirainen: > On 24.4.2012, at 10.50, Robert Schetterer wrote: > >> doveadm expunge -d -u *@domain.example mailbox exampelfolder\* >> savedbefore 10d >> >> works like charm, but the folders do not get unsubscribed >> any chance to have i.e another add parameter ,or do it auto >> cause empty and deleted folders should not stay subscribed in my eyes > > Yeah, I guess unsubscribing is always wanted: > > http://hg.dovecot.org/dovecot-2.1/rev/b1653b7bd165 > Hi Timo, i will try after update and report, jep unsubscribing should be the uni wanted behave thx again ! -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From weston.kris at ymail.com Wed Apr 25 09:17:18 2012 From: weston.kris at ymail.com (Kris) Date: Wed, 25 Apr 2012 07:17:18 +0100 Subject: [Dovecot] dovecot %u variable problem.... Message-ID: <4F9796EE.5070907@ymail.com> Hi guys n gals, im on an ubuntu 11 VPS on linode (dovecot version 2.013). ive spent a lot of time and googling trying to get postfix, dovecot, ssl, saslauthd and mysql to all play nice together and im quite close now but theres something going on i cant quite explain and id really appreciate some help. I have been going for ten hours on this today and it seems in direct conflict with what dovecot themselves say... the problem is a variable in dovecot (%u) which is normally used as a full username in sql query strings (i.e. user at domain) and is simply dropping the domain for some reason... From dovecot docs (http://wiki2.dovecot.org/Variables): The variables that work everywhere are: %u user full username (e.g. user at domain) so i have postfix and stuff set up ok to send at least and i know this because ive tested it by hard-coding the string in the sql query myself with a successful send... but, anyway, the problem with the '%u' variable is it is doing stuff like this: (from mail.log and mysql.log) localhost dovecot: auth: Debug: sql(admin at microhard.com,xx.xx.xx.xx): SELECT home,uid,gid FROM users WHERE id = 'admin at microhard.com' this was derived from the following query in /etc/dovecot/dovecot-sql.conf.ext user_query = SELECT home,uid,gid FROM users WHERE id = '%u' so no problem there.... but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user at domain): localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' Query SELECT id as user, crypt as password FROM users WHERE id= 'admin' which is derived from the following query also in /etc/dovecot/dovecot-sql.conf.ext password_query = SELECT id as user, crypt as password FROM users WHERE id= '%u' and to further confuse me, this 'unchangeable' variable sometimes decides to right itself without any obvious reason... localhost dovecot: auth: Debug: sql(admin at microhard.com,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin at microhard.com' im wondering if its something to do with my client dropping the domain, or whether dovecot has an error, or some other thing ive overlooked... my dovecot.conf looks a lot like this: # 2.0.13: dovecot.conf # OS: Linux 3.0.18-x86_64-linode24 x86_64 Ubuntu 11.10 ext3 first_valid_uid = 5000 last_valid_uid = 5000 login_greeting = I likes my ducks n geese I do. mail_location = maildir:/var/spool/mail/virtual/%u #enables logging all failed authentication attempts. auth_verbose=yes #enables all authentication debug logging (also enables auth_verbose). Passwords are logged as . auth_debug=yes #does everything that auth_debug=yes does, but it also removes password hiding. auth_debug_passwords=yes #enables all kinds of mail related debug logging, such as showing where Dovecot is looking for mails. mail_debug=yes #enables logging SSL errors and warnings. Even without this setting if connection is closed because of an SSL error, the error is logged as the disconnection reason (v1.1+). verbose_ssl=yes passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/etc/dovecot/acls #quota = maildir:storage=10240:messages=1000 trash = /etc/dovecot/trash.conf } protocols = " imap" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = dovecot mode = 0660 user = dovecot } user = dovecot } ssl_ca = /etc/pki/dovecot/certs/ca-bundle.crt ssl_cert = References: <4F9796EE.5070907@ymail.com> Message-ID: <4F979E5E.5000409@schetterer.org> Am 25.04.2012 08:17, schrieb Kris: > this was derived from the following query in > /etc/dovecot/dovecot-sql.conf.ext > user_query = SELECT home,uid,gid FROM users WHERE id = '%u' i use like this user_query = SELECT concat('/usr/local/virtual/', maildir) AS home, \ concat('*:bytes=', mailbox.quota) AS quota_rule, \ #when saving to Trash mailbox the user gets additional 50MB "Trash:storage=+50240" AS quota_rule2, \ #when saving to Sent mailbox the user gets additional 50MB "Sent:storage=+50240" AS quota_rule3, \ #when saving to Drafts mailbox the user gets additional 50MB "Drafts:storage=+50240" AS quota_rule4, \ #when saving to Templates mailbox the user gets additional 50MB "Templates:storage=+50240" AS quota_rule5, \ #when saving to Junk mailbox the user gets additional 50MB "Junk:storage=+50240" AS quota_rule6, \ #when saving to Archives mailbox the user gets additional 50MB "Archives:storage=+50240" AS quota_rule7, \ ########### concat('maildir:/usr/local/virtual/', maildir) AS mail, \ CASE '%s' WHEN 'pop3' THEN NULL ELSE 'yes' END AS 'namespace/inbox/inbox', \ CASE '%s' WHEN 'pop3' THEN 'yes' ELSE NULL END AS 'namespace/virtual/inbox', \ 1001 AS uid, 1001 AS gid FROM mailbox WHERE username = '%u' AND active = '1' > > so no problem there.... > > but then this happens when i attempt to send mail and this means the > user gets rejected because dovecot cant find the user id in the db > without the correct string (i.e. user at domain): > localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id > as user, crypt as password FROM users WHERE id= 'admin' > Query SELECT id as user, crypt as password FROM users WHERE id= 'admin' > which is derived from the following query also in > /etc/dovecot/dovecot-sql.conf.ext > password_query = SELECT id as user, crypt as password FROM users WHERE > id= '%u' > > and to further confuse me, this 'unchangeable' variable sometimes > decides to right itself without any obvious reason... > localhost dovecot: auth: Debug: sql(admin at microhard.com,xx.xx.xx.xx): > query: SELECT id as user, crypt as password FROM users WHERE id= > 'admin at microhard.com' > > im wondering if its something to do with my client dropping the domain, > or whether dovecot has an error, or some other thing ive overlooked... password_query = SELECT username as user, password, \ 1001 as userdb_uid, \ 1001 as userdb_gid, \ "/usr/local/virtual/%d/%u/" AS userdb_home, \ "maildir:/usr/local/virtual/%d/%u/" AS userdb_mail \ FROM mailbox WHERE username = '%u' AND active = '1' AND (imap_allowed = '1' or '%Ls' = 'pop3') this should not fit to you ( postfixadmin full domain layout with virtual plugin and quota additions ,clear text passwords etc), but perhaps gives you ideas by compare, what to fix at your side, however upgrade to dove latest 2.0.20 -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From dg at dguhl.org Wed Apr 25 11:26:15 2012 From: dg at dguhl.org (Dennis Guhl) Date: Wed, 25 Apr 2012 10:26:15 +0200 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F9796EE.5070907@ymail.com> References: <4F9796EE.5070907@ymail.com> Message-ID: <20120425082615.GA14759@laptop-dg.leere.eu> On Wed, Apr 25, 2012 at 07:17:18AM +0100, Kris wrote: [..] > the problem is a variable in dovecot (%u) which is normally used as > a full username in sql query strings (i.e. user at domain) and is > simply dropping the domain for some reason... [..] > and to further confuse me, this 'unchangeable' variable sometimes > decides to right itself without any obvious reason... > localhost dovecot: auth: Debug: > sql(admin at microhard.com,xx.xx.xx.xx): query: SELECT id as user, > crypt as password FROM users WHERE id= 'admin at microhard.com' > > im wondering if its something to do with my client dropping the > domain, or whether dovecot has an error, or some other thing ive > overlooked... Indeed, it seems some of your users try to authenticate with a username without the '@domain' part. You can either try to make your users comply or you can use auth_default_realm in your config: # Default realm/domain to use if none was specified. This is # used for both SASL realms and appending @domain to username # in plaintext logins. # #auth_default_realm = Dennis [..] From cor at xs4all.nl Wed Apr 25 11:27:18 2012 From: cor at xs4all.nl (Cor Bosman) Date: Wed, 25 Apr 2012 10:27:18 +0200 Subject: [Dovecot] trash plugin bug? In-Reply-To: <69567A90-450A-457A-BC38-C0DE235156D9@iki.fi> References: <932BA6EA-2884-4847-B66A-E056C8068B28@xs4all.nl> <69567A90-450A-457A-BC38-C0DE235156D9@iki.fi> Message-ID: <0D69989C-00E4-48CC-AF08-B58C52EA3A12@xs4all.nl> > > Well, normally you shouldn't be over quota I guess.. :) Anyway, > > http://hg.dovecot.org/dovecot-2.1/rev/ec8564741aa8 > http://hg.dovecot.org/dovecot-2.1/rev/dd3798681283 This indeed fixed the problem. Thank you, Cor From weston.kris at ymail.com Wed Apr 25 11:50:20 2012 From: weston.kris at ymail.com (Kris Weston) Date: Wed, 25 Apr 2012 09:50:20 +0100 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <20120425082615.GA14759@laptop-dg.leere.eu> References: <4F9796EE.5070907@ymail.com> <20120425082615.GA14759@laptop-dg.leere.eu> Message-ID: <4F97BACC.4080902@ymail.com> thanks for your help dennis... > Indeed, it seems some of your users try to authenticate with a > username without the '@domain' part. > > You can either try to make your users comply or you can use > auth_default_realm in your config: in this case the users are me :) same username every time i connect, but each time i see a load of db lookups that are different but using this same u variable... > # Default realm/domain to use if none was specified. This is > # used for both SASL realms and appending @domain to username > # in plaintext logins. > # > #auth_default_realm = > ahhh, ok. only i have multiple domains, so how does that werk ? just put em all in ? cheerz Kris From dg at dguhl.org Wed Apr 25 11:58:05 2012 From: dg at dguhl.org (Dennis Guhl) Date: Wed, 25 Apr 2012 10:58:05 +0200 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F97BACC.4080902@ymail.com> References: <4F9796EE.5070907@ymail.com> <20120425082615.GA14759@laptop-dg.leere.eu> <4F97BACC.4080902@ymail.com> Message-ID: <20120425085805.GB14759@laptop-dg.leere.eu> On Wed, Apr 25, 2012 at 09:50:20AM +0100, Kris Weston wrote: > > thanks for your help dennis... > >Indeed, it seems some of your users try to authenticate with a > >username without the '@domain' part. > > > >You can either try to make your users comply or you can use > >auth_default_realm in your config: > > in this case the users are me :) > same username every time i connect, but each time i see a load of db > lookups that are different but using this same u variable... If it's no bug, I doubt it randomly changes the value for %u while the input is the same. So it seems the problem lies somewhere else. And if it is no layer 8 problem it must be the client ;) > > # Default realm/domain to use if none was specified. This is > > # used for both SASL realms and appending @domain to username > > # in plaintext logins. > > # > > #auth_default_realm = > > > > ahhh, ok. only i have multiple domains, so how does that werk ? > just put em all in ? IIRC no, you can only name one default. Dennis From tss at iki.fi Wed Apr 25 13:45:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 13:45:08 +0300 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F979E5E.5000409@schetterer.org> References: <4F9796EE.5070907@ymail.com> <4F979E5E.5000409@schetterer.org> Message-ID: <629E1A55-910F-48B9-8986-C87A34852F24@iki.fi> On 25.4.2012, at 9.49, Robert Schetterer wrote: > #when saving to Trash mailbox the user gets additional 50MB > "Trash:storage=+50240" AS quota_rule2, \ > #when saving to Sent mailbox the user gets additional 50MB > "Sent:storage=+50240" AS quota_rule3, \ > #when saving to Drafts mailbox the user gets additional 50MB > "Drafts:storage=+50240" AS quota_rule4, \ > #when saving to Templates mailbox the user gets additional 50MB > "Templates:storage=+50240" AS quota_rule5, \ > #when saving to Junk mailbox the user gets additional 50MB > "Junk:storage=+50240" AS quota_rule6, \ > #when saving to Archives mailbox the user gets additional 50MB > "Archives:storage=+50240" AS quota_rule7, \ > ########### BTW. These are all static values and don't really need to be in SQL query: plugin { quota_rule2 = Trash:... quota_rule3 = ... ... } > 1001 as userdb_uid, \ > 1001 as userdb_gid, \ > "/usr/local/virtual/%d/%u/" AS userdb_home, \ > "maildir:/usr/local/virtual/%d/%u/" AS userdb_mail \ Same for these. Also slightly confusing that your passdb and userdb lookups return mail setting differently. From tss at iki.fi Wed Apr 25 13:48:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 13:48:45 +0300 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F9796EE.5070907@ymail.com> References: <4F9796EE.5070907@ymail.com> Message-ID: <5B92F036-DDBE-42AB-94A1-DBF21F11DE14@iki.fi> On 25.4.2012, at 9.17, Kris wrote: > but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user at domain): > localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' What exactly is doing this lookup? SMTP AUTH? Maybe the client's SMTP authentication is configured without the @domain part in username? From tss at iki.fi Wed Apr 25 13:52:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 13:52:26 +0300 Subject: [Dovecot] How to remove duplicate GUID messages from storage? In-Reply-To: <20120421202928.GA12998@daniel.localdomain> References: <20120421202928.GA12998@daniel.localdomain> Message-ID: On 21.4.2012, at 23.29, Daniel Parthey wrote: > The following command > doveadm -v force-resync -u username at example.org INBOX > outputs lots of messages like this one, with different GUID > doveadm(username at example.org): Error: mdbox /mail/dovecot/example.org/username/mail/storage: Duplicate GUID 0b8a032d66a0924fb42c0000de5f8128 in m.55:45484041 and m.14:52173045 I think this is a force-resync bug and it shouldn't really complain about duplicates. Although I'm not entirely sure why with you it's complaining about them at all. I have anyway in TODO to look into this.. > How can I manually remove these identical, duplicate messages > from the storage to save space? Dovecot does not do it > automatically. Perhaps force-resync + purge should do that, but currently it doesn't. Also it would be nice if doveadm import didn't add duplicates in the first place. This is also something for which I have vague plans, because it would help dsync as well. From robert at schetterer.org Wed Apr 25 14:07:56 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 13:07:56 +0200 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <629E1A55-910F-48B9-8986-C87A34852F24@iki.fi> References: <4F9796EE.5070907@ymail.com> <4F979E5E.5000409@schetterer.org> <629E1A55-910F-48B9-8986-C87A34852F24@iki.fi> Message-ID: <4F97DB0C.4040901@schetterer.org> Am 25.04.2012 12:45, schrieb Timo Sirainen: > On 25.4.2012, at 9.49, Robert Schetterer wrote: > >> #when saving to Trash mailbox the user gets additional 50MB >> "Trash:storage=+50240" AS quota_rule2, \ >> #when saving to Sent mailbox the user gets additional 50MB >> "Sent:storage=+50240" AS quota_rule3, \ >> #when saving to Drafts mailbox the user gets additional 50MB >> "Drafts:storage=+50240" AS quota_rule4, \ >> #when saving to Templates mailbox the user gets additional 50MB >> "Templates:storage=+50240" AS quota_rule5, \ >> #when saving to Junk mailbox the user gets additional 50MB >> "Junk:storage=+50240" AS quota_rule6, \ >> #when saving to Archives mailbox the user gets additional 50MB >> "Archives:storage=+50240" AS quota_rule7, \ >> ########### > > BTW. These are all static values and don't really need to be in SQL query: > > plugin { > quota_rule2 = Trash:... > quota_rule3 = ... > ... > } > >> 1001 as userdb_uid, \ >> 1001 as userdb_gid, \ >> "/usr/local/virtual/%d/%u/" AS userdb_home, \ >> "maildir:/usr/local/virtual/%d/%u/" AS userdb_mail \ > > Same for these. Also slightly confusing that your passdb and userdb lookups return mail setting differently. thx timo, i will look at this, thats all historic but ever worked currently i am still fighting with some upgrade issues to 2.1 just now some sieve experimental stuff failed , i will post about it -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Apr 25 16:06:29 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 15:06:29 +0200 Subject: [Dovecot] sieve vacation redirect problem Message-ID: <4F97F6D5.2080104@schetterer.org> Hi , since upgrade sieve vacation, redirect stopped working getting postfix error while sending end of data -- message may be sent more than once i played a lot about sieve , lda submission host conf entries etc but all failed , i am not seeing any heavy bug in the logs -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Apr 25 16:29:47 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 15:29:47 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F97F6D5.2080104@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> Message-ID: <4F97FC4B.6060803@schetterer.org> Am 25.04.2012 15:06, schrieb Robert Schetterer: > Hi , since upgrade > sieve vacation, redirect stopped working getting > > postfix error > while sending end of data -- message may be sent more than once > > i played a lot about sieve , lda submission host conf entries etc but > all failed , i am not seeing any heavy bug in the logs > that not full truth i am seeing master: service(lmtp): child 31952 killed with signal 11 (core not dumped) -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Wed Apr 25 16:46:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 16:46:31 +0300 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F97FC4B.6060803@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> Message-ID: <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> On 25.4.2012, at 16.29, Robert Schetterer wrote: > master: service(lmtp): child 31952 killed with signal 11 (core not dumped) Can you get a gdb backtrace? Might be a bit difficult to get a core dump from lmtp, but you should be able to attach gdb into it: service lmtp { process_min_avail = 1 } gdb -p `pidof lmtp` cont bt full From stephan at rename-it.nl Wed Apr 25 16:51:29 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 25 Apr 2012 15:51:29 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F97FC4B.6060803@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> Message-ID: <4F980161.9080007@rename-it.nl> Op 4/25/2012 3:29 PM, Robert Schetterer schreef: > Am 25.04.2012 15:06, schrieb Robert Schetterer: >> Hi , since upgrade >> sieve vacation, redirect stopped working getting >> >> postfix error >> while sending end of data -- message may be sent more than once >> >> i played a lot about sieve , lda submission host conf entries etc but >> all failed , i am not seeing any heavy bug in the logs >> > that not full truth i am seeing > > master: service(lmtp): child 31952 killed with signal 11 (core not dumped) Please try to obtain a gdb backtrace: http://dovecot.org/bugreport.html And provide dovecot -n output Regards, Stephan. From robert at schetterer.org Wed Apr 25 17:03:49 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 16:03:49 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> Message-ID: <4F980445.4090203@schetterer.org> Am 25.04.2012 15:46, schrieb Timo Sirainen: > bt full Program received signal SIGSEGV, Segmentation fault. smtp_client_open (set=0x252b0b0, destination=0x254e410 "ncitester2 at onmotive.de", return_path=0x24e4148 "root at mail02.mailspooler.com", file_r=0x7fff2040a748) at smtp-client.c:146 146 smtp-client.c: Datei oder Verzeichnis nicht gefunden. in smtp-client.c (gdb) Undefined command: " path = 0x180000

fd = #1 0x00007fd4e58b57a5 in act_redirect_send (action=, aenv=0x254e2c0, tr_context=, keep=) at cmd-redirect.c:339 size = msgctx = crlf_input = smtp_handle = 0x24e4148 data = ret = sender = 0x24e4148 "root at mail02.mailspooler.com" recipient = 0x24e4168 "robert at schetterer.name" input = f = hide_headers = {0x7fd4e58d2ee7 "Return-Path", 0x7fd4e58d6d3c "X-Sieve", 0x7fd4e58d7929 "X-Sieve-Redirected-From"} #2 act_redirect_commit (action=, aenv=0x254e2c0, tr_context=, keep=) at cmd-redirect.c:402 ctx = 0x254e408 mail = 0x252ba90 senv = 0x7fff2040a8c0 dupeid = #3 0x00007fd4e58a7eeb in sieve_result_execute (result=, keep=) at sieve-result.c:1156 impl_keep = true rsef = 0x0 implicit_keep = true success = true commit_ok = rac = 0x254e428 first_action = #4 0x00007fd4e58b8418 in sieve_multiscript_execute (mscript=0x254e3d8, sbin=0x25516c0, ehandler=0x254c100, flags=, final=true) at sieve.c:601 No locals. #5 sieve_multiscript_run (mscript=0x254e3d8, sbin=0x25516c0, ehandler=0x254c100, flags=, final=true) at sieve.c:630 No locals. #6 0x00007fd4e5af47e6 in lda_sieve_multiscript_execute (mdctx=, storage_r=) at lda-sieve-plugin.c:514 sbin = 0x25516c0 rtflags = SIEVE_RUNTIME_FLAG_NOGLOBAL cpflags = SIEVE_COMPILE_FLAG_NOGLOBAL mscript = 0x254e3d8 svinst = 0x2546e80 last_script = 0x254b8e0 ret = scripts = ehandler = 0x254c100 i = 1 more = true error = SIEVE_ERROR_NONE #7 lda_sieve_deliver_mail (mdctx=, storage_r=) at lda-sieve-plugin.c:794 scripts = i = _data_stack_cur_id = 3 srctx = {svinst = 0x2546e80, mdctx = 0x7fff2040aac0, scripts = 0x24e3340, script_count = 2, user_script = 0x254b8e0, main_script = 0x254b8e0, msgdata = 0x7fff2040a990, scriptenv = 0x7fff2040a8c0, user_ehandler = 0x254c100, master_ehandler = 0x254b7d0, userlog = 0x24e3658 "/usr/local/virtual/schetterer.name/robert at schetterer.name/dovecot.sieve.log"} svenv = {hostname = 0x250aba0 "mail02.mailspooler.com", base_dir = 0x2536a78 "/var/run/dovecot", username = 0x25354c8 "robert at schetterer.name", home_dir = 0x2537c40 "/usr/local/virtual/schetterer.name/robert at schetterer.name/", flags = SIEVE_FLAG_HOME_RELATIVE} svinst = 0x2546e80 msgdata = {mail = 0x252ba90, return_path = 0x24f0c68 "root at mail02.mailspooler.com", orig_envelope_to = 0x24f12c0 "robert at schetterer.name", final_envelope_to = 0x24f12c0 "robert at schetterer.name", auth_user = 0x25354c8 "robert at schetterer.name", id = 0x252c410 "<20120425140007.D3556840227 at mail02.mailspooler.com>"} scriptenv = {action_log_format = 0x250ac28 "msgid=%m: %$", user = 0x2535410, default_mailbox = 0x4077ac "INBOX", postmaster_address = 0x250ab78 "hostmaster at mail02.mailspooler.com", mailbox_autocreate = false, mailbox_autosubscribe = false, script_context = 0x7fff2040aac0, smtp_open = 0x7fd4e5af4180 , smtp_close = 0x7fd4e5af4160 , duplicate_check = 0x7fd4e5af4110 , duplicate_mark = 0x7fd4e5af4140 , reject_mail = 0x7fd4e5af4100 , exec_status = 0x7fff2040aa00, trace_stream = 0x0, trace_config = {level = SIEVE_TRLVL_NONE, flags = 0}} estatus = {message_saved = false, message_forwarded = false, tried_default_save = false, keep_original = false, last_storage = 0x0} master_ehandler = 0x254b7d0 user_location = default_location = sieve_before = sieve_after = script_sequence = {arr = {buffer = 0x24e3308, element_size = 8}, v = 0x24e3308, v_modifiable = 0x24e3308} debug = error = SIEVE_ERROR_NONE ret = #8 0x00007fd4e888a445 in mail_deliver (ctx=0x7fff2040aac0, storage_r=0x7fff2040ab50) at mail-deliver.c:389 ret = #9 0x0000000000405a33 in client_deliver (client=0x24f00d0) at commands.c:538 ns = storage = 0x0 sets = mail_error = MAIL_ERROR_NONE dctx = {pool = 0x252aa90, set = 0x250ab28, session = 0x252aab0, dup_ctx = 0x2546d90, session_id = 0x24f0c50 "ma3NNmcDmE/bDAAAAfUEdQ", src_mail = 0x252ba90, src_envelope_sender = 0x24f0c68 "root at mail02.mailspooler.com", dest_user = 0x2535410, dest_addr = 0x24f12c0 "robert at schetterer.name", final_dest_addr = 0x24f12c0 "robert at schetterer.name", dest_mailbox_name = 0x4077ac "INBOX", dest_mail = 0x0, var_expand_table = 0x0, tried_default_save = false, saved_mail = false, save_dest_mail = false, mailbox_full = false, dsn = false} input = error = #10 client_deliver_next (client=0x24f00d0) at commands.c:577 rcpts = 0x24f0cc0 #11 client_input_data_write_local (client=0x24f00d0) at commands.c:671 src_mail = 0x252ba90 first_uid = 4294967295 session = 0x252aab0 old_uid = 0 #12 client_input_data_write (client=0x24f00d0) at commands.c:779 input = 0x2513530 #13 client_input_data_handle (client=0x24f00d0) at commands.c:873 data = size = 861 ---Type to continue, or q to quit--- -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Apr 25 17:10:25 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 16:10:25 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F980161.9080007@rename-it.nl> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4F980161.9080007@rename-it.nl> Message-ID: <4F9805D1.8050101@schetterer.org> Am 25.04.2012 15:51, schrieb Stephan Bosch: > Op 4/25/2012 3:29 PM, Robert Schetterer schreef: >> Am 25.04.2012 15:06, schrieb Robert Schetterer: >>> Hi , since upgrade >>> sieve vacation, redirect stopped working getting >>> >>> postfix error >>> while sending end of data -- message may be sent more than once >>> >>> i played a lot about sieve , lda submission host conf entries etc but >>> all failed , i am not seeing any heavy bug in the logs >>> >> that not full truth i am seeing >> >> master: service(lmtp): child 31952 killed with signal 11 (core not >> dumped) > > Please try to obtain a gdb backtrace: > > http://dovecot.org/bugreport.html > > And provide dovecot -n output > > Regards, > > Stephan. did test a few sieve related stuff cause the working 2.0.20 didnt worked but now my ideas are running out # 2.1.5 (d2b01dcce79f): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-37-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 50 M auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes auth_verbose_passwords = plain auth_worker_max_count = 300 default_process_limit = 250 dict { acl = mysql:/etc/dovecot/dovecot-dict-acl-sql.conf.ext expire = mysql:/etc/dovecot/dovecot-dict-expire-sql.conf.ext quotadict = mysql:/etc/dovecot/dovecot-dict-quota-sql.conf.ext } disable_plaintext_auth = no first_valid_gid = 1001 first_valid_uid = 1001 hostname = mail02.mailspooler.com last_valid_gid = 1001 last_valid_uid = 1001 listen = * login_greeting = imap, pop ready mail_access_groups = vmail mail_debug = yes mail_fsync = always mail_gid = 1001 mail_location = maildir:~/ mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = " zlib expire virtual fts fts_squat mail_log autocreate notify acl listescape" mail_privileged_group = vmail mail_uid = 1001 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags notify mmap_disable = yes namespace { inbox = no list = children location = maildir:/usr/local/virtual/%%d/%%u:INDEX=~/shared/%%d/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { hidden = no list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } mailbox name { special_use = \Drafts \Junk \Sent \Trash } prefix = separator = / subscriptions = yes } namespace real { hidden = yes list = no location = prefix = RealMails/ separator = / } namespace virtual { hidden = yes list = no location = virtual:/etc/dovecot/virtual:LAYOUT=maildir++:INDEX=~/virtual prefix = virtual/ separator = / subscriptions = no } passdb { args = /etc/dovecot/dovecot-sql-master.conf.ext driver = sql master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl autocreate = Trash autocreate2 = Junk autocreate3 = Sent autocreate4 = Drafts autocreate5 = Templates autocreate6 = Hostmaster-Backup autocreate7 = archiv-backup-mailspooler autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Drafts autosubscribe5 = Templates expire = Trash expire2 = Trash/* expire3 = Junk expire4 = Junk/* expire5 = Hostmaster-Backup expire6 = Hostmaster-Backup/* expire7 = archiv-backup-mailspooler/* expire_dict = proxy::expire fts = squat home = /usr/local/virtual/%d/%u mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size mail_log_group_events = yes quota = dict:::proxy::quotadict quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /usr/local/virtual/%d/%u/dovecot.sieve sieve_before = /etc/dovecot/sieve/global.sieve sieve_dir = /usr/local/virtual/%d/%u/sieve sieve_extensions = +fileinto +reject +envelope +encoded-character +vacation +subaddress +relational +regex +imap4flags +copy +include +variables +body +enotify +environment +mailbox +date +ihave +imapflags +notify sieve_global_dir = /etc/dovecot/sieve/ sieve_global_path = /etc/dovecot/sieve/global.sieve zlib_save = gz zlib_save_level = 6 } postmaster_address = hostmaster at mail02.mailspooler.com protocols = imap pop3 lmtp sieve service anvil { client_limit = 1000 } service auth-worker { user = root } service auth { client_limit = 1000 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service dict { extra_groups = vmail unix_listener dict { group = vmail mode = 0660 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 8 service_count = 1 vsz_limit = 256 M } service imap-postlogin { executable = script-login /usr/local/bin/postlogin_imap.sh } service imap { executable = imap imap-postlogin process_limit = 1024 vsz_limit = 256 M } service lmtp { process_min_avail = 1 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } vsz_limit = 256 M } service managesieve-login { inet_listener { address = 127.0.0.1 212.52.224.210 212.52.224.206 port = 2000 } vsz_limit = 256 M } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } process_min_avail = 8 } service pop3-postlogin { executable = script-login /usr/local/bin/postlogin_pop3.sh } service pop3 { executable = pop3 pop3-postlogin process_limit = 1024 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { mode = 0666 user = vmail } user = vmail } ssl_ca = References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> Message-ID: <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> On 25.4.2012, at 17.03, Robert Schetterer wrote: > Program received signal SIGSEGV, Segmentation fault. > smtp_client_open (set=0x252b0b0, destination=0x254e410 > "ncitester2 at onmotive.de", return_path=0x24e4148 > "root at mail02.mailspooler.com", file_r=0x7fff2040a748) at smtp-client.c:146 > 146 smtp-client.c: Datei oder Verzeichnis nicht gefunden. > in smtp-client.c If you do the same and get here again, what does it say with: p *set From robert at schetterer.org Wed Apr 25 17:16:28 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 16:16:28 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> Message-ID: <4F98073C.3080406@schetterer.org> Am 25.04.2012 16:11, schrieb Timo Sirainen: > p *set hope doing it right (gdb) p *set $1 = {postmaster_address = 0xaa33d0 "P6\252", hostname = 0x0, submission_host = 0x0, sendmail_path = 0x7fbaf6e41f50 "H\211\\$\360H\211l$\370H\203\354\030\200=\207A%", rejection_subject = 0x7fbaf6e42b90 "H\211\\$\350L\211d$\370H\211\373H\211l$\360H\203\354(\200=?5%", rejection_reason = 0x7fbaf6e41ed0 "H\211\\$\360H\211l$\370H\203\354(\200=\aB%", deliver_log_format = 0x1
, recipient_delimiter = 0xa9c910 "\003", lda_original_recipient_header = 0xabd490 "\001", quota_full_tempfail = 16, lda_mailbox_autocreate = 208, lda_mailbox_autosubscribe = 171} -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From warden at geneseo.edu Wed Apr 25 17:20:41 2012 From: warden at geneseo.edu (David Warden) Date: Wed, 25 Apr 2012 10:20:41 -0400 Subject: [Dovecot] stats + fts squat plugins, indexer-worker error on message indexing Message-ID: On RHEL 6.2 + dovecot 2.1.5 with the stats and fts (squat) plugins enabled and each user having their own uid on the system I get the following error in the logs when the indexer-worker process indexes some messages: Apr 25 09:56:19 wardentest3 dovecot: imap-login: Login: user=warden, method=PLAIN, rip=137.238.60.164, lip=137.238.2.240, lport=1993, mpid=19464, encryption=TLS Apr 25 09:56:26 wardentest3 dovecot: service=indexer-worker user=warden rip= Indexed 12 messages in INBOX Apr 25 09:56:26 wardentest3 dovecot: service=indexer-worker user=warden rip= Error: open(/proc/self/io) failed: Permission denied Apr 25 09:56:29 wardentest3 dovecot: service=imap user=warden rip=137.238.60.164 Disconnected: Logged out bytes(in/out)=46/780 What I did to generate the above was remove the squat indexes for my Inbox then log in via telnet and search for a string in my Inbox, which forced an index of the messages in my Inbox. Based on http://wiki2.dovecot.org/Services#indexer-worker it seems like this is because the indexer-worker process is created as root but at the time of indexing the indexer-worker drops privs to the mail user. In /proc//, io is mode 0400 and owned by root, so it makes sense that the mail user would be unable to read that file. The strange thing is it looks like the stats are still collected (unless I am reading the output of the stats dump incorrectly): [root at wardentest3 mailtest]# doveadm stats dump -s /var/run/dovecot/mailtest/stats command user=warden cmd args session user last_update user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits select inbox 27fb87218302984f084c0000ae899e04 warden 1335362182.950250 0.000000 0.000000 172 0 46 0 0 12288 13 7883 4 381 0 0 0 0 0 search text "warden" 27fb87218302984f084c0000ae899e04 warden 1335362186.588023 0.001000 0.000000 32 0 4 1 0 0 4 108 1 43 0 0 0 0 0 logout 27fb87218302984f084c0000ae899e04 warden 1335362189.161790 0.000000 0.000000 1 0 0 3 0 0 1 108 3 157 0 0 0 0 0 This obviously isn't a show-stopper but if this is causing stats to be incorrect it would be great if this could be fixed. -David Warden My doveconf -n: # 2.1.5: /etc/dovecot/mailtest/dovecot.conf doveconf: Warning: service auth { client_limit=4096 } is lower than required under max. load (12288) doveconf: Warning: service anvil { client_limit=4096 } is lower than required under max. load (12291) # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.2 (Santiago) nfs auth_cache_negative_ttl = 0 auth_cache_size = 16 M auth_gssapi_hostname = $ALL auth_krb5_keytab = /etc/dovecot/mailtest.combined.keytab auth_master_user_separator = * auth_mechanisms = plain login gssapi auth_username_format = %Ln base_dir = /var/run/dovecot/mailtest/ default_client_limit = 4096 default_process_limit = 4096 deliver_log_format = msgid="%m" subject="%s" from="%f" size=%p result="%$" first_valid_uid = 0 hostname = mailtest.geneseo.edu instance_name = mailtest lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_log_format_elements = user=%u method=%m rip=%r lip=%l lport=%a mpid=%e encryption=%c mail_fsync = always mail_location = maildir:/Mail/mailhome/%Ln/mailtest/Maildir:CONTROL=/Mail/mailhome/%Ln/mailtest/.dovecot-control:INDEX=/Mail/mailhome/%Ln/mailtest/.dovecot-index mail_log_prefix = "service=%s user=%u rip=%r " mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = zlib quota mail_log notify fts fts_squat stats maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl mmap_disable = yes namespace default { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = no special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = . type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes } passdb { args = cache_key=%u dovecot driver = pam } plugin { fts = squat mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_fields = uid box msgid size from subject flags mail_log_group_events = yes quota = maildir:User quota quota_exceeded_message = Quota exceeded (mailbox for user is full). Please see http://go.geneseo.edu/emailoverquota for help deleting messages while over quota. quota_rule = *:storage=200M quota_rule2 = Trash:storage=+50M sieve = /Mail/mailhome/%Ln/mailtest/.filter.sieve sieve_dir = /Mail/mailhome/%Ln/mailtest/.sievedir sieve_max_redirects = 25 stats_memory_limit = 32 M stats_refresh = 30 secs stats_track_cmds = yes } postmaster_address = postmaster at geneseo.edu protocols = imap sieve lmtp quota_full_tempfail = yes service auth { unix_listener auth-exim { group = exim mode = 0660 } } service imap-login { inet_listener imap { port = 1143 } inet_listener imaps { port = 1993 } service_count = 0 vsz_limit = 256 M } service imap { process_limit = 4096 } service lmtp { inet_listener lmtp { port = 124 } } service managesieve-login { inet_listener sieve { port = 14190 } inet_listener sieve_deprecated { port = 12000 } } service pop3 { process_limit = 4096 } service stats { fifo_listener stats-mail { mode = 0666 } } ssl_ca = References: <4F95B3DD.1070609@allycomm.com> <4F962028.3060501@allycomm.com> <4EE7C62E-928C-499D-BE4E-9944D6013198@iki.fi> <4F96DD43.3070700@allycomm.com> <78F42D61-1402-408B-953B-F85254479761@iki.fi> Message-ID: <4F98091D.5070902@allycomm.com> On 04/24/2012 10:08 AM, Timo Sirainen wrote: > >>>> 25934 jeff 1 103 0 135M 102M CPU2 2 128:47 100.00% doveadm >>> Looks like it could be in an infinite loop. To find out where: >>> > That's a lot of getdirentries() calls in there, repeating all the time. So I don't think it's an infinite loop, just a lot of slowness repeating the same work over and over again. See if it helps if you add: > > maildir_very_dirty_syncs = yes I hope I understood correctly when I tried $ dsync -Dv -o maildir_very_dirty_syncs=yes backup maildir:~/Maildir into fresh directories (moving the previous Maildir aside). I didn't see any significant change (and it did not complete in several hours) on the usual UFS directory, and have also tried ZFS to see if it might be a filesystem issue. That has not completed either after over nine hours. From what I can see, I am guessing that it is "hanging" in index creation. Below, "mail" is the source directory, and "Maildir" is the destination of the sync. This is about nine hours of run time. [jeff at mail ~]$ ls -Al mail/.imap/cron_2010-07-10 total 9 -rw------- 1 jeff jeff 4910320 Feb 22 2011 dovecot.index -rw------- 1 jeff jeff 3600384 Apr 25 01:18 dovecot.index.cache -rw------- 1 jeff jeff 56 Feb 22 2011 dovecot.index.log [jeff at mail ~]$ ls -Al Maildir/.cron_2010-07-10 total 26 drwx------ 2 jeff jeff 102291 Apr 25 01:17 cur -rw------- 1 jeff jeff 36 Apr 25 01:18 dovecot-keywords -rw------- 1 jeff jeff 4122971 Apr 25 01:17 dovecot-uidlist -rw------- 1 jeff jeff 24 Apr 25 07:19 dovecot-uidlist.lock -rw------- 1 jeff jeff 1637008 Apr 25 07:03 dovecot.index -rw------- 1 jeff jeff 9442304 Apr 25 07:19 dovecot.index.cache -rw------- 1 jeff jeff 87984 Apr 25 07:19 dovecot.index.log -rw------- 1 jeff jeff 1048600 Apr 25 07:03 dovecot.index.log.2 -rw------- 1 jeff jeff 0 Apr 24 22:12 maildirfolder drwx------ 2 jeff jeff 2 Apr 24 22:12 new drwx------ 2 jeff jeff 2 Apr 25 01:17 tmp From tss at iki.fi Wed Apr 25 17:30:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 17:30:27 +0300 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F98073C.3080406@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> Message-ID: On 25.4.2012, at 17.16, Robert Schetterer wrote: > Am 25.04.2012 16:11, schrieb Timo Sirainen: >> p *set > > hope doing it right > > (gdb) p *set > $1 = {postmaster_address = 0xaa33d0 "P6\252", hostname = 0x0, > submission_host = 0x0, sendmail_path = 0x7fbaf6e41f50 > "H\211\\$\360H\211l$\370H\203\354\030\200=\207A%", > rejection_subject = 0x7fbaf6e42b90 > "H\211\\$\350L\211d$\370H\211\373H\211l$\360H\203\354(\200=?5%", > rejection_reason = 0x7fbaf6e41ed0 That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? From robert at schetterer.org Wed Apr 25 17:38:56 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 16:38:56 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> Message-ID: <4F980C80.2030908@schetterer.org> Am 25.04.2012 16:30, schrieb Timo Sirainen: > On 25.4.2012, at 17.16, Robert Schetterer wrote: > >> Am 25.04.2012 16:11, schrieb Timo Sirainen: >>> p *set >> >> hope doing it right >> >> (gdb) p *set >> $1 = {postmaster_address = 0xaa33d0 "P6\252", hostname = 0x0, >> submission_host = 0x0, sendmail_path = 0x7fbaf6e41f50 >> "H\211\\$\360H\211l$\370H\203\354\030\200=\207A%", >> rejection_subject = 0x7fbaf6e42b90 >> "H\211\\$\350L\211d$\370H\211\373H\211l$\360H\203\354(\200=?5%", >> rejection_reason = 0x7fbaf6e41ed0 > > That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? > its from latest http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ recompiled the debain way , never had bugged -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Wed Apr 25 17:43:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 17:43:06 +0300 Subject: [Dovecot] stats + fts squat plugins, indexer-worker error on message indexing In-Reply-To: References: Message-ID: <8F39E28F-3079-4EB1-BCAD-0A5C78DA05DC@iki.fi> On 25.4.2012, at 17.20, David Warden wrote: > Apr 25 09:56:26 wardentest3 dovecot: service=indexer-worker user=warden rip= Error: open(/proc/self/io) failed: Permission .. > Based on http://wiki2.dovecot.org/Services#indexer-worker it seems like this is because the indexer-worker process is created as root but at the time of indexing the indexer-worker drops privs to the mail user. In /proc//, io is mode 0400 and owned by root, so it makes sense that the mail user would be unable to read that file. Does the attached patch fix it? -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 2639 bytes Desc: not available URL: -------------- next part -------------- > The strange thing is it looks like the stats are still collected (unless I am reading the output of the stats dump incorrectly): > > [root at wardentest3 mailtest]# doveadm stats dump -s /var/run/dovecot/mailtest/stats command user=warden > cmd args session user last_update user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits > select inbox 27fb87218302984f084c0000ae899e04 warden 1335362182.950250 0.000000 0.000000 172 0 46 0 0 12288 13 7883 4 381 0 0 0 0 0 > search text "warden" 27fb87218302984f084c0000ae899e04 warden 1335362186.588023 0.001000 0.000000 32 0 4 1 0 0 4 108 1 43 0 0 0 0 0 > logout 27fb87218302984f084c0000ae899e04 warden 1335362189.161790 0.000000 0.000000 1 0 0 3 0 0 1 108 3 157 0 0 0 0 0 These are IMAP command statistics. The problem was only with indexer-worker. From xchris89x at googlemail.com Wed Apr 25 17:47:16 2012 From: xchris89x at googlemail.com (Chris) Date: Wed, 25 Apr 2012 16:47:16 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F980C80.2030908@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> <4F980C80.2030908@schetterer.org> Message-ID: 2012/4/25 Robert Schetterer : >> That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? >> > > its from latest > http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ > recompiled the debain way , never had bugged You are using packages compiled against debian sources under ubuntu??? -- Chris From robert at schetterer.org Wed Apr 25 17:50:16 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 16:50:16 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> <4F980C80.2030908@schetterer.org> Message-ID: <4F980F28.90205@schetterer.org> Am 25.04.2012 16:47, schrieb Chris: > 2012/4/25 Robert Schetterer : >>> That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? >>> >> >> its from latest >> http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ >> recompiled the debain way , never had bugged > > You are using packages compiled against debian sources under ubuntu??? > > -- > Chris jep that works since years -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From almarzuki2001 at hotmail.com Wed Apr 25 17:52:39 2012 From: almarzuki2001 at hotmail.com (Hadi Salem) Date: Wed, 25 Apr 2012 17:52:39 +0300 Subject: [Dovecot] dovecot-lda error Message-ID: Hi, im getting the fallowing error. from dovecot-lda log Apr 25 17:30:41 lda(jamal at example.com): Fatal: setgid(5003(vmail) from userdb lookup) failed with euid=8(mail), gid=12(mail), egid=12(mail): Operation not permitted (This binary should probably be called with process group set to 5003(vmail) instead of 12(mail)) this error occur if im using different uid and gid part from mail uid.. jamal at example.com:{PLAIN}123:5003:5003::/var/vmail/example.com/jamal/::userdb_mail=maildir:~/Maildir but if i use mail uid it work fine.. jamal at example.com:{PLAIN}123:8:12::/var/vmail/example.com/jamal/::userdb_mail=maildir:~/Maildir i tried using multiple uids as mention in wiki2.dovecot.org/LDA bye setting up setuid-root for dovecot-lda but im getting on-there error from sendmail :- Apr 25 17:30:41 host1 sendmail[31024]: q3PEUen2031022: to=, delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120359, relay=dovecot, dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/local/libexec/dovecot/dovecot-lda) exited with EX_TEMPFAIL im using sendmail. [root at host1 conf.d]# doveconf -n # 2.0.20: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-164.el5 x86_64 CentOS release 5.4 (Final) auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_verbose_passwords = plain debug_log_path = /var/log/dovecot.log disable_plaintext_auth = no first_valid_gid = 12 first_valid_uid = 8 hostname = host1.bigmama.com info_log_path = /var/log/dovecot.log last_valid_gid = 50000 last_valid_uid = 50000 log_path = /var/log/dovecot.log mail_debug = yes mail_gid = mail mail_location = maildir:~/Maildir mail_uid = mail passdb { args = scheme=PLAIN username_format=%u /etc/dovecot/users driver = passwd-file } postmaster_address = root at host1.bigmama.com service auth { unix_listener auth-userdb { group = mail mode = 0666 user = mail } } ssl = no userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol lda { info_log_path = /var/log/dovecot-lda.log log_path = /var/log/dovecot-lda-errors.log } From xchris89x at googlemail.com Wed Apr 25 17:54:03 2012 From: xchris89x at googlemail.com (Chris) Date: Wed, 25 Apr 2012 16:54:03 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: <4F980F28.90205@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> <4F980C80.2030908@schetterer.org> <4F980F28.90205@schetterer.org> Message-ID: 2012/4/25 Robert Schetterer : > Am 25.04.2012 16:47, schrieb Chris: >> 2012/4/25 Robert Schetterer : >>>> That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? >>>> >>> >>> its from latest >>> http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ >>> recompiled the debain way , never had bugged >> >> You are using packages compiled against debian sources under ubuntu??? >> >> -- >> Chris > > jep that works since years Thats really a very bad idea. Please recompile it under ubuntu 10.04. -- Chris From robert at schetterer.org Wed Apr 25 17:59:21 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 16:59:21 +0200 Subject: [Dovecot] sieve vacation redirect problem In-Reply-To: References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> <4F980C80.2030908@schetterer.org> <4F980F28.90205@schetterer.org> Message-ID: <4F981149.4070102@schetterer.org> Am 25.04.2012 16:54, schrieb Chris: > 2012/4/25 Robert Schetterer : >> Am 25.04.2012 16:47, schrieb Chris: >>> 2012/4/25 Robert Schetterer : >>>>> That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? >>>>> >>>> >>>> its from latest >>>> http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ >>>> recompiled the debain way , never had bugged >>> >>> You are using packages compiled against debian sources under ubuntu??? >>> >>> -- >>> Chris >> >> jep that works since years > > Thats really a very bad idea. Please recompile it under ubuntu 10.04. you miss understand i have it compiled under ubuntu 10.04 > > -- > Chris -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From joseba.torre at ehu.es Wed Apr 25 18:27:44 2012 From: joseba.torre at ehu.es (Joseba Torre) Date: Wed, 25 Apr 2012 17:27:44 +0200 Subject: [Dovecot] Error compiling pigeonhole 0.3.0 with dovecot-2.1.5 in openbsd 5.0 In-Reply-To: <4F968AF2.4090807@rename-it.nl> References: <4F967192.2060701@ehu.es> <4F968AF2.4090807@rename-it.nl> Message-ID: <4F9817F0.7030908@ehu.es> > > Looks like BSD doesn't know EPROTO (although it is POSIX). Dovecot uses > EIO elsewhere for such stream errors, so I adopted that for ManageSieve > as well. This should fix it: > > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/ef5045159e6b > > Regards, Perfect, thank you From robert at schetterer.org Wed Apr 25 18:48:20 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 17:48:20 +0200 Subject: [Dovecot] sieve vacation redirect problem / solved In-Reply-To: References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> Message-ID: <4F981CC4.3010600@schetterer.org> Am 25.04.2012 16:30, schrieb Timo Sirainen: > On 25.4.2012, at 17.16, Robert Schetterer wrote: > >> Am 25.04.2012 16:11, schrieb Timo Sirainen: >>> p *set >> >> hope doing it right >> >> (gdb) p *set >> $1 = {postmaster_address = 0xaa33d0 "P6\252", hostname = 0x0, >> submission_host = 0x0, sendmail_path = 0x7fbaf6e41f50 >> "H\211\\$\360H\211l$\370H\203\354\030\200=\207A%", >> rejection_subject = 0x7fbaf6e42b90 >> "H\211\\$\350L\211d$\370H\211\373H\211l$\360H\203\354(\200=?5%", >> rejection_reason = 0x7fbaf6e41ed0 > > That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? > jep it was broken , no idea why serveral new downloads give same result using http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz with recompile debian way ( same scripts etc ) to recent downloaded dovecot sources http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ works now no idea whats going on , but works now -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From weston.kris at ymail.com Wed Apr 25 18:52:12 2012 From: weston.kris at ymail.com (Kris Weston) Date: Wed, 25 Apr 2012 16:52:12 +0100 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <5B92F036-DDBE-42AB-94A1-DBF21F11DE14@iki.fi> References: <4F9796EE.5070907@ymail.com> <5B92F036-DDBE-42AB-94A1-DBF21F11DE14@iki.fi> Message-ID: <4F981DAC.8000106@ymail.com> On 25/04/12 11:48, Timo Sirainen wrote: > On 25.4.2012, at 9.17, Kris wrote: > >> but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user at domain): >> localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' > What exactly is doing this lookup? SMTP AUTH? Maybe the client's SMTP authentication is configured without the @domain part in username? > saslauthd via a query in dovecot-sql.conf.ext? after these suggestions im pretty sure nobody here has a clue about this, and this is not a known problem, i cant find anyone on the internet with it and the query is quite clearly coming up wrong and the unchangeable variable is quite clearly changing. the client can not be sending out different usernames every second surely. i put the username in myself in thunderbird, double, triple checked, made other accounts etc, the username changes by the second, ive watched the sql queries go in and they change for no obvious reason. From tss at iki.fi Wed Apr 25 18:55:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 18:55:40 +0300 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F981DAC.8000106@ymail.com> References: <4F9796EE.5070907@ymail.com> <5B92F036-DDBE-42AB-94A1-DBF21F11DE14@iki.fi> <4F981DAC.8000106@ymail.com> Message-ID: <8B6B7B68-278A-44C5-9013-AA14CCA991CD@iki.fi> On 25.4.2012, at 18.52, Kris Weston wrote: > On 25/04/12 11:48, Timo Sirainen wrote: >> On 25.4.2012, at 9.17, Kris wrote: >> >>> but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user at domain): >>> localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' >> What exactly is doing this lookup? SMTP AUTH? Maybe the client's SMTP authentication is configured without the @domain part in username? >> > saslauthd via a query in dovecot-sql.conf.ext? There is no saslauthd in Dovecot. > after these suggestions im pretty sure nobody here has a clue about this, and this is not a known problem, i cant find anyone on the internet with it and the query is quite clearly coming up wrong and the unchangeable variable is quite clearly changing. To me it looks like the client isn't sending the domain. > the client can not be sending out different usernames every second surely. i put the username in myself in thunderbird, double, triple checked, made other accounts etc, the username changes by the second, ive watched the sql queries go in and they change for no obvious reason. Show the full logs for a failing session, not just this one line. Even better would be full logs with auth_debug_passwords=yes, but in that case you'd better be using an unimportant password (some of the base64 encoded strings contain the password). From postmaster at mathiesen.info Wed Apr 25 20:56:18 2012 From: postmaster at mathiesen.info (postmaster) Date: Wed, 25 Apr 2012 19:56:18 +0200 Subject: [Dovecot] 2.1.5 problem with mail_location Message-ID: <4F983AC2.5010007@mathiesen.info> I've been advised to set : mail_location = maildir:~/ as the home directory is found through a sql request. this has been working perfectly up to and including 2.1.4 from 2.1.5 however this setting gives cause to a problem : instead of looking in the mail home eg /var/mail/domain/user\@domain/ dovecot is looking for mail in /var/mail/domain/user\@domain/~ the '~' directory is created upon the first request for mail and all future mail transactions in dovecot is done relative to this folder. is this behavior a bug or a feature ??? -- Bjarne D Mathiesen K?benhavn N ; Danmark ; Europa ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ denne besked er skrevet i et totalt M$-frit milj? MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3 From robert at schetterer.org Wed Apr 25 21:02:28 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 20:02:28 +0200 Subject: [Dovecot] sieve vacation redirect problem / solved / restested with lates hg pigeonhole broken In-Reply-To: <4F981CC4.3010600@schetterer.org> References: <4F97F6D5.2080104@schetterer.org> <4F97FC4B.6060803@schetterer.org> <4AE8582F-3287-4E5B-99E9-15F8334F10C8@iki.fi> <4F980445.4090203@schetterer.org> <954D12A8-7D2D-440D-862A-6457A01F5388@iki.fi> <4F98073C.3080406@schetterer.org> <4F981CC4.3010600@schetterer.org> Message-ID: <4F983C34.4030605@schetterer.org> Am 25.04.2012 17:48, schrieb Robert Schetterer: > Am 25.04.2012 16:30, schrieb Timo Sirainen: >> On 25.4.2012, at 17.16, Robert Schetterer wrote: >> >>> Am 25.04.2012 16:11, schrieb Timo Sirainen: >>>> p *set >>> >>> hope doing it right >>> >>> (gdb) p *set >>> $1 = {postmaster_address = 0xaa33d0 "P6\252", hostname = 0x0, >>> submission_host = 0x0, sendmail_path = 0x7fbaf6e41f50 >>> "H\211\\$\360H\211l$\370H\203\354\030\200=\207A%", >>> rejection_subject = 0x7fbaf6e42b90 >>> "H\211\\$\350L\211d$\370H\211\373H\211l$\360H\203\354(\200=?5%", >>> rejection_reason = 0x7fbaf6e41ed0 >> >> That's totally broken.. You sure you recompiled Pigeonhole correctly against the same Dovecot version? >> > > jep it was broken , no idea why serveral new downloads give same result > using > http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz > with recompile debian way ( same scripts etc ) to recent downloaded > dovecot sources > http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ > > works now > > no idea whats going on , but works now > just for info, i retested again with hg code latest hg clone http://hg.rename-it.nl/dovecot-2.1-pigeonhole/ compile runs ok... dpkg-buildpackage -rfakeroot -b ( on ubuntu 64 lucid latest patchlevel from today ) but at dpkg install it gives warning /usr/lib/dovecot/managesieve: symbol lookup error: /usr/lib/dovecot/managesieve: undefined symbol: sieve_file_script doveconf: Error: managesieve-login: dump-capability process returned 127 dont know if there is the bug reason however my test showes up described failures with latest hg code then sieve vacation redirect got broken it works if i use http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz for compile thx for your help and code anywhere, hope this results may help for debug -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Apr 25 21:16:58 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Apr 2012 20:16:58 +0200 Subject: [Dovecot] doveadm delete folders ? / unsubscribe ? In-Reply-To: <4F9795EE.4030604@schetterer.org> References: <4F900066.9030808@schetterer.org> <5178CE73-87CF-4889-8ED5-452EBBF5082B@iki.fi> <4F906EE8.1030900@schetterer.org> <4F965B2E.2010103@schetterer.org> <283E4240-57E8-4BF2-8DCA-6DCC78C1D5A8@iki.fi> <4F9795EE.4030604@schetterer.org> Message-ID: <4F983F9A.5040900@schetterer.org> Am 25.04.2012 08:13, schrieb Robert Schetterer: > Am 24.04.2012 20:13, schrieb Timo Sirainen: >> On 24.4.2012, at 10.50, Robert Schetterer wrote: >> >>> doveadm expunge -d -u *@domain.example mailbox exampelfolder\* >>> savedbefore 10d >>> >>> works like charm, but the folders do not get unsubscribed >>> any chance to have i.e another add parameter ,or do it auto >>> cause empty and deleted folders should not stay subscribed in my eyes >> >> Yeah, I guess unsubscribing is always wanted: >> >> http://hg.dovecot.org/dovecot-2.1/rev/b1653b7bd165 >> > > Hi Timo, i will try after update and report, jep > unsubscribing should be the uni wanted behave > thx again ! tested works nice thx Timo ! -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From weston.kris at ymail.com Wed Apr 25 21:28:42 2012 From: weston.kris at ymail.com (Kris Weston) Date: Wed, 25 Apr 2012 19:28:42 +0100 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <8B6B7B68-278A-44C5-9013-AA14CCA991CD@iki.fi> References: <4F9796EE.5070907@ymail.com> <5B92F036-DDBE-42AB-94A1-DBF21F11DE14@iki.fi> <4F981DAC.8000106@ymail.com> <8B6B7B68-278A-44C5-9013-AA14CCA991CD@iki.fi> Message-ID: <4F98425A.6020908@ymail.com> >>>> but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user at domain): >>>> localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' >>> What exactly is doing this lookup? SMTP AUTH? Maybe the client's SMTP authentication is configured without the @domain part in username? >>> >> saslauthd via a query in dovecot-sql.conf.ext? > There is no saslauthd in Dovecot. thats correct, its not in dovecot, its installed on the same machine. but regardless, its salsauthd. >> after these suggestions im pretty sure nobody here has a clue about this, and this is not a known problem, i cant find anyone on the internet with it and the query is quite clearly coming up wrong and the unchangeable variable is quite clearly changing. > To me it looks like the client isn't sending the domain. so the client is changing its domain every coupla seconds for what reason, surely the thunderbird devs would have found this error? and even if thunderbird is sending it without the domain, the server should deal with thunderbird correctly, the user should not have to configure her thunderbird to work with dovecot rather than the other way round ? i dont believe this is the problem at all. >> the client can not be sending out different usernames every second surely. i put the username in myself in thunderbird, double, triple checked, made other accounts etc, the username changes by the second, ive watched the sql queries go in and they change for no obvious reason. > Show the full logs for a failing session, not just this one line. Even better would be full logs with auth_debug_passwords=yes, but in that case you'd better be using an unimportant password (some of the base64 encoded strings contain the password). thanks for trying, but forget it, its clear to me from answers ive received that nobody has a clue about this problem. guess im on me own. cheers bye! From warden at geneseo.edu Wed Apr 25 21:30:06 2012 From: warden at geneseo.edu (David Warden) Date: Wed, 25 Apr 2012 14:30:06 -0400 Subject: [Dovecot] stats + fts squat plugins, indexer-worker error on message indexing In-Reply-To: <8F39E28F-3079-4EB1-BCAD-0A5C78DA05DC@iki.fi> References: <8F39E28F-3079-4EB1-BCAD-0A5C78DA05DC@iki.fi> Message-ID: <86EED9D8-8445-49B4-9D72-C77883F926F9@geneseo.edu> On Apr 25, 2012, at 10:43 AM, Timo Sirainen wrote: > On 25.4.2012, at 17.20, David Warden wrote: > >> Apr 25 09:56:26 wardentest3 dovecot: service=indexer-worker user=warden rip= Error: open(/proc/self/io) failed: Permission > .. >> Based on http://wiki2.dovecot.org/Services#indexer-worker it seems like this is because the indexer-worker process is created as root but at the time of indexing the indexer-worker drops privs to the mail user. In /proc//, io is mode 0400 and owned by root, so it makes sense that the mail user would be unable to read that file. > > Does the attached patch fix it? > > The error is still logged, but is that intentional on your part? It looks like the patch logs the failure to open() when the error returned is anything other than ENOENT and in my case the error is EACCESS. Other than that it looks like it correctly disables accessing that file. >> The strange thing is it looks like the stats are still collected (unless I am reading the output of the stats dump incorrectly): >> >> [root at wardentest3 mailtest]# doveadm stats dump -s /var/run/dovecot/mailtest/stats command user=warden >> cmd args session user last_update user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits >> select inbox 27fb87218302984f084c0000ae899e04 warden 1335362182.950250 0.000000 0.000000 172 0 46 0 0 12288 13 7883 4 381 0 0 0 0 0 >> search text "warden" 27fb87218302984f084c0000ae899e04 warden 1335362186.588023 0.001000 0.000000 32 0 4 1 0 0 4 108 1 43 0 0 0 0 0 >> logout 27fb87218302984f084c0000ae899e04 warden 1335362189.161790 0.000000 0.000000 1 0 0 3 0 0 1 108 3 157 0 0 0 0 0 > > These are IMAP command statistics. The problem was only with indexer-worker. From tss at iki.fi Wed Apr 25 21:55:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 21:55:12 +0300 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F98425A.6020908@ymail.com> References: <4F9796EE.5070907@ymail.com> <5B92F036-DDBE-42AB-94A1-DBF21F11DE14@iki.fi> <4F981DAC.8000106@ymail.com> <8B6B7B68-278A-44C5-9013-AA14CCA991CD@iki.fi> <4F98425A.6020908@ymail.com> Message-ID: On 25.4.2012, at 21.28, Kris Weston wrote: >>>>> but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user at domain): >>>>> localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' >>>> What exactly is doing this lookup? SMTP AUTH? Maybe the client's SMTP authentication is configured without the @domain part in username? >>>> >>> saslauthd via a query in dovecot-sql.conf.ext? >> There is no saslauthd in Dovecot. > > thats correct, its not in dovecot, its installed on the same machine. > but regardless, its salsauthd. saslauthd doesn't authenticate via Dovecot. It might authenticate via IMAP though. > thanks for trying, but forget it, its clear to me from answers ive received that nobody has a clue about this problem. guess im on me own. cheers bye! The reason why nobody has a clue is because you haven't provided the full logs. We can only guess what the problem is. With full logs there would be no guessing necessary. From tss at iki.fi Wed Apr 25 22:09:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 22:09:09 +0300 Subject: [Dovecot] stats + fts squat plugins, indexer-worker error on message indexing In-Reply-To: <86EED9D8-8445-49B4-9D72-C77883F926F9@geneseo.edu> References: <8F39E28F-3079-4EB1-BCAD-0A5C78DA05DC@iki.fi> <86EED9D8-8445-49B4-9D72-C77883F926F9@geneseo.edu> Message-ID: On 25.4.2012, at 21.30, David Warden wrote: >>> Apr 25 09:56:26 wardentest3 dovecot: service=indexer-worker user=warden rip= Error: open(/proc/self/io) failed: Permission >> >> Does the attached patch fix it? > > The error is still logged, but is that intentional on your part? It looks like the patch logs the failure to open() when the error returned is anything other than ENOENT and in my case the error is EACCESS. Other than that it looks like it correctly disables accessing that file. The patch intended to open the /proc/self/io immediately at startup while still running as root. Anyway, now that I'm testing it again myself, I see that it's supposed to work even without the patch, because /proc/self/io's permissions change after seteuid(). I noticed similar errors happening somewhat randomly in another CentOS 6.2 system. Changing the kernel fixed it. (Downgrading at least worked, possibly upgrading might also.) From tss at iki.fi Wed Apr 25 23:46:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Apr 2012 23:46:08 +0300 Subject: [Dovecot] 2.1.5 problem with mail_location In-Reply-To: <4F983AC2.5010007@mathiesen.info> References: <4F983AC2.5010007@mathiesen.info> Message-ID: <1B191C98-90A2-40B6-ACDF-2420E9A5161B@iki.fi> On 25.4.2012, at 20.56, postmaster wrote: > I've been advised to set : > mail_location = maildir:~/ > as the home directory is found through a sql request. > > this has been working perfectly up to and including 2.1.4 > > from 2.1.5 however this setting gives cause to a problem : > instead of looking in the mail home eg /var/mail/domain/user\@domain/ dovecot is looking for mail in /var/mail/domain/user\@domain/~ > > the '~' directory is created upon the first request for mail and all future mail transactions in dovecot is done relative to this folder. Yeah, it's a bug. Fixed and added to tests so it won't break in future: http://hg.dovecot.org/dovecot-2.1/rev/249c1c89d9d3 From tss at iki.fi Thu Apr 26 00:16:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:16:45 +0300 Subject: [Dovecot] v2.1.5 released In-Reply-To: References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: On 24.4.2012, at 11.59, Gerhard Wiesinger wrote: > Hello Timo, > > Dovecot 2.1.5 seems to be broken. After upgrade from > 2.1.3 + patch http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e > I only see Inbox and Trash folder. > > Reverting => OK. > > Same config as before and disucssed per private E-mail and on the list. Guessing this comes again from: > mail_full_filesystem_access=yes I guess the problem was this: http://hg.dovecot.org/dovecot-2.1/rev/249c1c89d9d3 From Ralf.Hildebrandt at charite.de Thu Apr 26 00:26:10 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 25 Apr 2012 23:26:10 +0200 Subject: [Dovecot] v2.1.5 released In-Reply-To: References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <20120425212610.GA5653@charite.de> * Timo Sirainen : > On 24.4.2012, at 11.59, Gerhard Wiesinger wrote: > > > Hello Timo, > > > > Dovecot 2.1.5 seems to be broken. After upgrade from > > 2.1.3 + patch http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e > > I only see Inbox and Trash folder. > > > > Reverting => OK. > > > > Same config as before and disucssed per private E-mail and on the list. Guessing this comes again from: > > mail_full_filesystem_access=yes > > I guess the problem was this: http://hg.dovecot.org/dovecot-2.1/rev/249c1c89d9d3 2.1.5 doesn't seem to build with pigeonhole-sieve (or it's pigeonhole-sieve itself?): /bin/bash ../../libtool --tag=CC --mode=link gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -export-dynamic -g -Wl,--as-needed -o sievec sievec.o ../../src/lib-sieve/libdovecot-sieve.la ../../src/lib-sieve-tool/libsieve-tool.la /usr/src/dovecot-2.1/dovecot-2.1/src/lib-storage/libdovecot-storage.la -lssl -lcrypto /usr/src/dovecot-2.1/dovecot-2.1/src/lib-lda/libdovecot-lda.la /usr/src/dovecot-2.1/dovecot-2.1/src/lib-dovecot/libdovecot.la -export-dynamic -ldl -lrt libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -g -Wl,--as-needed -o .libs/sievec sievec.o -Wl,--export-dynamic ../../src/lib-sieve/.libs/libdovecot-sieve.so ../../src/lib-sieve-tool/.libs/libsieve-tool.a /usr/src/dovecot-2.1/dovecot-2.1/src/lib-storage/.libs/libdovecot-storage.so -lssl -lcrypto /usr/src/dovecot-2.1/dovecot-2.1/src/lib-lda/.libs/libdovecot-lda.so /usr/src/dovecot-2.1/dovecot-2.1/src/lib-dovecot/.libs/libdovecot.so -ldl -lrt -Wl,-rpath -Wl,/usr/local/dovecot-2.1/lib/dovecot ../../src/lib-sieve/.libs/libdovecot-sieve.so: error: undefined reference to 'sieve_file_script' ../../src/lib-sieve/.libs/libdovecot-sieve.so: error: undefined reference to 'sieve_dict_script' sievec.c:116: error: undefined reference to 'sieve_scriptfile_has_extension' collect2: ld returned 1 exit status make[3]: *** [sievec] Error 1 make[3]: Leaving directory /usr/src/dovecot-2.1/dovecot-2.1-pigeonhole/src/sieve-tools' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory /usr/src/dovecot-2.1/dovecot-2.1-pigeonhole/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory /usr/src/dovecot-2.1/dovecot-2.1-pigeonhole' make: *** [all] Error 2 Making install in . -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Thu Apr 26 00:27:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:27:25 +0300 Subject: [Dovecot] Permission problems when trying to create new folders with ACL plugin In-Reply-To: <20120424130005.0db3d425@legolas.home.ceotex.de> References: <20120424130005.0db3d425@legolas.home.ceotex.de> Message-ID: <7D3AF25B-FAE0-4187-B53C-DBB477A4AE79@iki.fi> On 24.4.2012, at 14.00, Markus Petri wrote: > I have the a problem with shared folders and the ACL plugin. The user > "markus" has full access to "Shared/shared-user/Data": > > 3 GETACL "Shared/shared-user/Data" > * ACL "Shared/shared-user/Data" "markus" akxeilprwtscd > 3 OK Getacl completed. > > but creating a new subfolder under "Data" fails: > > 4 CREATE "Shared/shared-user/Data/New1" > 4 NO [NOPERM] Permission denied Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e8b80e0767ac From tss at iki.fi Thu Apr 26 00:27:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:27:45 +0300 Subject: [Dovecot] acls not copied when creating subfolder of private INBOX In-Reply-To: <4F96B436.9090402@freenet.de> References: <4F96B436.9090402@freenet.de> Message-ID: <06DE7611-7783-4E5A-B58A-1448115802F5@iki.fi> On 24.4.2012, at 17.09, Andreas Helmcke wrote: > Using brand new dovecot 2.1.5 I still have a problem with the inheritance of acls. Maybe http://hg.dovecot.org/dovecot-2.1/rev/e8b80e0767ac fixes this as well? From tss at iki.fi Thu Apr 26 00:28:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:28:55 +0300 Subject: [Dovecot] Reading Mutt mboxes from Thunderbird In-Reply-To: <20120424160712.GB5048@leonardo-laptop> References: <20120424160712.GB5048@leonardo-laptop> Message-ID: On 24.4.2012, at 19.07, Leonardo M. Ram? wrote: > Hi, I'm a user of Mutt, and before moving to Thunderbird I'm trying to > share mboxes between them. To do this, I installed Dovecot to create an > IMAP server in my local machine, to serve mutt mboxes to be read by > Thunderbird. > > So far, I can read my inbox from /var/mail/%u, but in mutt, I have many > folders in ~/mail, like 2010-mails, 2011-mails, mailing-list-x, > mailing-list-y, and so on. How can I configure Dovecot to read them?. > > In /etc/dovecot/conf.d/10-mail.conf I have this: > > mail_location = mbox:~/Mail:INBOX=/var/mail/%u ~/mail vs. ~/Mail? From tss at iki.fi Thu Apr 26 00:29:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:29:51 +0300 Subject: [Dovecot] dovecot LDA error In-Reply-To: References: Message-ID: On 24.4.2012, at 20.41, Hadi Salem wrote: > Apr 24 20:25:46 lda(jamal at example.com): Fatal: > setgid(5000(vmail) from userdb lo > okup) failed with > euid=8(mail), gid=12(mail), egid=12(mail): Operation not permi > tted (This binary should probably be called with process group set to 5000(vmail > ) instead of 12(mail)) Configure your MTA to run dovecot-lda as vmail instead of as mail. From tss at iki.fi Thu Apr 26 00:31:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:31:29 +0300 Subject: [Dovecot] dovecot-lda error In-Reply-To: References: Message-ID: <1353D406-AEF1-46A1-ADB8-DFC159EC9946@iki.fi> On 25.4.2012, at 17.52, Hadi Salem wrote: > i tried using multiple uids as mention in wiki2.dovecot.org/LDA bye setting up setuid-root for dovecot-lda but im getting on-there error from sendmail :- > > Apr 25 17:30:41 host1 sendmail[31024]: q3PEUen2031022: to=, delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120359, relay=dovecot, dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/local/libexec/dovecot/dovecot-lda) exited with EX_TEMPFAIL What do the Dovecot logs say about this? > protocol lda { > info_log_path = /var/log/dovecot-lda.log > log_path = /var/log/dovecot-lda-errors.log > } If there is nothing in dovecot-lda-errors.log, make sure dovecot-lda has permissions to write there (e.g. temporarily make it world-writable). From tss at iki.fi Thu Apr 26 00:33:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:33:13 +0300 Subject: [Dovecot] v2.1.5 released In-Reply-To: <20120425212610.GA5653@charite.de> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> <20120425212610.GA5653@charite.de> Message-ID: On 26.4.2012, at 0.26, Ralf Hildebrandt wrote: > 2.1.5 doesn't seem to build with pigeonhole-sieve (or it's > pigeonhole-sieve itself?): Builds against Pigeonhole 0.3.0 tarball at least. From Ralf.Hildebrandt at charite.de Thu Apr 26 00:35:20 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 25 Apr 2012 23:35:20 +0200 Subject: [Dovecot] v2.1.5 released In-Reply-To: References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> <20120425212610.GA5653@charite.de> Message-ID: <20120425213520.GB5653@charite.de> * Timo Sirainen : > On 26.4.2012, at 0.26, Ralf Hildebrandt wrote: > > > 2.1.5 doesn't seem to build with pigeonhole-sieve (or it's > > pigeonhole-sieve itself?): > > Builds against Pigeonhole 0.3.0 tarball at least. Thought so. I'm rebuilding against that one while I write this :O -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Thu Apr 26 00:58:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 00:58:13 +0300 Subject: [Dovecot] v2.1.5 released In-Reply-To: <20120425212610.GA5653@charite.de> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> <20120425212610.GA5653@charite.de> Message-ID: <05BAF7A2-FFEC-4684-BEEC-B4B22F684114@iki.fi> On 26.4.2012, at 0.26, Ralf Hildebrandt wrote: > * Timo Sirainen : > 2.1.5 doesn't seem to build with pigeonhole-sieve (or it's > pigeonhole-sieve itself?): > > ../../src/lib-sieve/.libs/libdovecot-sieve.so: error: undefined reference to 'sieve_file_script' > ../../src/lib-sieve/.libs/libdovecot-sieve.so: error: undefined reference to 'sieve_dict_script' That's from hg, and you didn't run autogen.sh? ./configure --enable-maintainer-mode also helps avoiding this. From Ralf.Hildebrandt at charite.de Thu Apr 26 01:02:42 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Thu, 26 Apr 2012 00:02:42 +0200 Subject: [Dovecot] v2.1.5 released In-Reply-To: <05BAF7A2-FFEC-4684-BEEC-B4B22F684114@iki.fi> References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> <20120425212610.GA5653@charite.de> <05BAF7A2-FFEC-4684-BEEC-B4B22F684114@iki.fi> Message-ID: <20120425220242.GY5653@charite.de> * Timo Sirainen : > On 26.4.2012, at 0.26, Ralf Hildebrandt wrote: > > > * Timo Sirainen : > > 2.1.5 doesn't seem to build with pigeonhole-sieve (or it's > > pigeonhole-sieve itself?): > > > > ../../src/lib-sieve/.libs/libdovecot-sieve.so: error: undefined reference to 'sieve_file_script' > > ../../src/lib-sieve/.libs/libdovecot-sieve.so: error: undefined reference to 'sieve_dict_script' > > That's from hg, yes > and you didn't run autogen.sh? ./configure --enable-maintainer-mode > also helps avoiding this. Damn, forgot that in the pigeonhole dir (I have that for the dovecot sources, but not for the pigeonhole sources)... -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From daniel.parthey at informatik.tu-chemnitz.de Thu Apr 26 01:51:16 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 26 Apr 2012 00:51:16 +0200 Subject: [Dovecot] How to remove duplicate GUID messages from storage? In-Reply-To: References: <20120421202928.GA12998@daniel.localdomain> Message-ID: <20120425225116.GB10062@daniel.localdomain> Timo Sirainen schrieb: > On 21.4.2012, at 23.29, Daniel Parthey wrote: > > > The following command > > doveadm -v force-resync -u username at example.org INBOX > > outputs lots of messages like this one, with different GUID > > doveadm(username at example.org): Error: mdbox /mail/dovecot/example.org/username/mail/storage: Duplicate GUID 0b8a032d66a0924fb42c0000de5f8128 in m.55:45484041 and m.14:52173045 > > I think this is a force-resync bug and it shouldn't really complain about duplicates. > Although I'm not entirely sure why with you it's complaining about them at all. I have anyway in TODO to look into this.. Maybe this is because I "manually" deleted all the mailboxes directories (rm -rf) containing the duplicates, which one should better avoid with mdbox mailboxes... :( So there is no "meta information" anymore, just the duplicate messages in the storage. and dovecot would need to "guess" where these messages from the store belong. > > How can I manually remove these identical, duplicate messages > > from the storage to save space? Dovecot does not do it > > automatically. > > Perhaps force-resync + purge should do that, but currently it doesn't. I already tried both, and purge is even running as a nightly cronjob. > Also it would be nice if doveadm import didn't add duplicates in the first place. > This is also something for which I have vague plans, because it would help dsync as well. Thanks for looking into this. I would really appreciate this idea of "duplicate GUID prevention" while duplicate messages (with different GUIDs) should probably be allowed. Maybe someone wants to store the same message several times... Regards, Daniel From daniel.parthey at informatik.tu-chemnitz.de Thu Apr 26 01:56:04 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 26 Apr 2012 00:56:04 +0200 Subject: [Dovecot] Reading Mutt mboxes from Thunderbird In-Reply-To: References: <20120424160712.GB5048@leonardo-laptop> Message-ID: <20120425225604.GA10291@daniel.localdomain> Timo Sirainen schrieb: > On 24.4.2012, at 19.07, Leonardo M. Ram? wrote: > > > Hi, I'm a user of Mutt, and before moving to Thunderbird I'm trying to > > share mboxes between them. To do this, I installed Dovecot to create an > > IMAP server in my local machine, to serve mutt mboxes to be read by > > Thunderbird. > > > > So far, I can read my inbox from /var/mail/%u, but in mutt, I have many > > folders in ~/mail, like 2010-mails, 2011-mails, mailing-list-x, > > mailing-list-y, and so on. How can I configure Dovecot to read them?. > > > > In /etc/dovecot/conf.d/10-mail.conf I have this: > > > > mail_location = mbox:~/Mail:INBOX=/var/mail/%u > > ~/mail vs. ~/Mail? On 2012-04-25 04:26:27 +0200, Daniel Parthey wrote: > Are you sure, dovecot does not read them, or are you just not "seeing" > your folders in thunderbird? Please try to right click your INBOX and > "subscribe" to the folders. On serverside, the subscribed folders are > listed in $mail_location/subscriptions, so you might auto-generate > this file by listing your mutt mailboxes. > > Regards, > Daniel You are right!, I just had to subscribe to the folders. Regards, Leonardo From dovecot at allycomm.com Thu Apr 26 02:16:29 2012 From: dovecot at allycomm.com (Jeff Kletsky) Date: Wed, 25 Apr 2012 16:16:29 -0700 Subject: [Dovecot] 2.1.x changes and dsync -- Wiki suggestion Message-ID: <4F9885CD.2000800@allycomm.com> There are a couple things that show up on the Upgrading/2.1 wiki page that would be useful to have on pages describing dsync (Tools/Dsync, Migration/MailFormat, Tools, perhaps others): * dsync was merged into doveadm * You can't dsync 2.0.x to/from 2.1.x That second point is especially "interesting" to me as it seems that Apple MacOS X Server is still on 2.0.16, at least from what I see on opensource.apple.com. I was planning on migrating mail off the current server onto my friend's new MacMini, but I'll have to figure out how to "downgrade" the mail jail to a 2.0 version, it seems. Thanks for all the great work, Jeff From jeff at j-simmons.net Thu Apr 26 04:34:49 2012 From: jeff at j-simmons.net (Jeff Simmons) Date: Wed, 25 Apr 2012 18:34:49 -0700 Subject: [Dovecot] dsync: convert only mbox in /var/mail to maildir? In-Reply-To: <04D01B3D-CD58-4E0A-8C1A-36B01E4792E7@iki.fi> References: <201204201226.41710.jeff@j-simmons.net> <04D01B3D-CD58-4E0A-8C1A-36B01E4792E7@iki.fi> Message-ID: <201204251834.50001.jeff@j-simmons.net> On Friday, April 20, 2012 02:27:59 pm Timo Sirainen wrote: > On 20.4.2012, at 22.26, Jeff Simmons wrote: > > I'm migrating an old UW pop server to a new Dovecot server. The old > > server did not allow mail to be stored on the server, so only mbox files > > exist in /var/mail, with nothing in /home/$user. I'd like to copy the > > /var/mail directory over to the new server and then use dsync to convert > > the mbox files and put them in /home/$user/Maildir, but every > > permutation of the command line I've tried dies with a variation on > > can't find/write either /var/mail/mail or /home/$user/mail, seems it > > won't deal with mbox files without an associated mail directory. Can > > dsync be used for this kind of conversion, or should I look elsewhere > > for conversion utilities? > > Assuming in destination server the mbox is in /var/mail/$user and you want > the destination to ~/Maildir: > > in dovecot.conf set: > mail_location = maildir.~/Maildir > > rm -rf /tmp/emptydir > dsync mirror mbox:/tmp/emptydir:/var/mail/$user Actually: dsync mirror mbox:/tmp/emptydir:INBOX=/var/mail/$user :) And be sure that /var/mail/$user has the same ownership:group as the user (i.e. $user:$user NOT $user:mail). > be sure to delete the emptydir between users. -- Jeff Simmons jeff at j-simmons.net Simmons Consulting - Network Engineering, Administration, Security From me at junc.org Thu Apr 26 04:53:08 2012 From: me at junc.org (Benny Pedersen) Date: Thu, 26 Apr 2012 03:53:08 +0200 Subject: [Dovecot] dovecot %u variable problem.... In-Reply-To: <4F97BACC.4080902@ymail.com> References: <4F9796EE.5070907@ymail.com> <20120425082615.GA14759@laptop-dg.leere.eu> <4F97BACC.4080902@ymail.com> Message-ID: <9d5bd8f6f2e4efc1c576f4a0246e03fb@junc.org> Den 2012-04-25 10:50, Kris Weston skrev: >> # auth_default_realm = > ahhh, ok. only i have multiple domains, so how does that werk ? > just put em all in ? if you use @ in imap/pop3 logins then the default for this setting is fine, if logins is without @ then you need to set it to the hostname of the machine where users is on, it cant be multiple hostnames on 127.0.0.1 :=) use localhost.example.org if unsure saslauthtest -u user -p passwd -r example.org OK or saslauthtest -u user at example.org -p passwd OK From jtam.home at gmail.com Thu Apr 26 05:14:38 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 25 Apr 2012 19:14:38 -0700 (PDT) Subject: [Dovecot] dovecot LDA with sendmail In-Reply-To: References: Message-ID: Timo, in response to Hadi, wrote: >> Apr 24 20:25:46 lda(jamal at example.com): Fatal: >> setgid(5000(vmail) from userdb lo >> okup) failed with >> euid=8(mail), gid=12(mail), egid=12(mail): Operation not permi >> tted (This binary should probably be called with process group set to 5000(vmail >> ) instead of 12(mail)) > > Configure your MTA to run dovecot-lda as vmail instead of as mail. Yeah, what he said. Someone helpfully added documentation on how to do this at the end of Dovecot1's wiki (http://wiki.dovecot.org/LDA/Sendmail), which probably should be copied to Dovecot 2's wiki. So that's what I did. Joseph Tam From abruce at tumnus.co.nz Thu Apr 26 08:35:00 2012 From: abruce at tumnus.co.nz (Bruce, Andrew) Date: Thu, 26 Apr 2012 17:35:00 +1200 Subject: [Dovecot] LDAP Lookup not returning value in maxStorage In-Reply-To: <4F74B274.8000600@tumnus.co.nz> References: <20120327091425.73963576@jimbo> <20120328092534.5690fa40@jimbo> <4F74B274.8000600@tumnus.co.nz> Message-ID: On 30 March 2012 08:05, Andrew Bruce wrote: > > On 28/03/2012 19:25, Nikita Koshikov wrote: >> >> On Wed, 28 Mar 2012 09:39:37 +1300 >> Bruce, Andrew wrote: >> >>> On 28 March 2012 09:36, Bruce, Andrew ?wrote: >>>> >>>> On 27 March 2012 19:14, Nikita Koshikov ?wrote: >>>>> >>>>> On Tue, 27 Mar 2012 13:57:04 +1300 >>>>> Bruce, Andrew wrote: >>>>> >>>>> Hi there, >>>>> >>>>> We're setting up a Dovecot virtual email setup - we've got everything >>>>> working perfect with LDAP logins authenticating against AD and so >>>>> forth, but we're having issues with retrieving the maxStorage value >>>>> from AD (this is a pre-setup field in AD that we'd like to use to set >>>>> per user quotas). >>>>> >>>>> In our LDAP lookup, we have the maxStorage entry listed under >>>>> user_attrs for the quota (user_attrs = >>>>> maxStorage=quota_rule=*:storage=%$M), and in the debug logs, can see >>>>> it trying to get the entry, but it fails with: >>>>> Mar 27 13:19:27 auth: Debug: ldap(username at site,192.168.1.5): user >>>>> search: base=dc=site,dc=local scope=subtree >>>>> filter=(&(objectClass=person)(| (userPrincipalName=username at site) >>>>> (|(mail=username at site)(samAccountName=username at site)))) >>>>> fields=maxStorage >>>>> Mar 27 13:19:27 auth: Debug: ldap(username at site,192.168.1.5): no >>>>> fields returned by the server >>>>> >>>>> At this point, we then see the default quota applied. >>>>> >>>> Try to change your quota rule to be like: >>>> maxStorage=quota_rule=*:bytes=%$ >>>> ? ? ? ? ? ? ? ? ? ? ? ?^^^^^^^^^ >>>> And put the value in bytes to maxStorage - if I remember correct - this >>>> is integer field and no K\M\G values is valid here. >>>> >>>> PS We successfully using maxStorage field to obtain non-default quota >>>> from AD, dovecot version 2.0.x >>>>> >>>>> If we change the name of the field from maxStorage to instanceType we >>>>> see the value show up in the logs and passed through to the quota >>>>> system and applied successfully: >>>>> Mar 27 11:09:01 auth: Debug: ldap(username at site,192.168.1.5): user >>>>> search: base=dc=site,dc=local scope=subtree >>>>> filter=(&(objectClass=person)(| (userPrincipalName=username at site) >>>>> (|(mail=username at site)(samAccountName=username at site)))) >>>>> fields=instanceType >>>>> Mar 27 11:09:01 auth: Debug: ldap(username at site,192.168.1.5): result: >>>>> instanceType(quota_rule=*:storage=%$M)=*:storage=4M >>>>> Mar 27 11:09:01 auth: Debug: master out: USER ? 3901227009 >>>>> username at site ? ?quota_rule=*:storage=4M >>>>> >>>>> >>>>> Which seems a bit weird. >>>>> >>>>> If we use ldapsearch and pass it the same search string and look for >>>>> the field maxStorage, we clearly see the field and the value being >>>>> returned. ?The result looks the same if we also lookup instanceType. >>>>> >>>>> We're using Dovecot 2.0.9. >>>>> >>>>> Does anyone have any idea as to why we can't use this field? >>>>> >>>>> Thanks, >>>>> >>>>> Andrew >>> >>> Tried your suggestion Nikita, no joy unfortunately. ?It still looks >>> like the value never gets returned from the LDAP server to Dovecot. >>> It definitely has something in the field (equivalent of 10GB, but in >>> bytes as suggested) and I changed the user_attrs also, but still get >>> the same "no fields returned by the server" error message. >>> >>> Modifying the user_attrs to lookup from a different field >>> (instanceType) definitely works. >>> >>> What exact version are you using - perhaps it's a problem with our >>> copy of 2.0.9. >>> >>> Thanks, >>> >>> Andrew >> >> maybe you met restriction of ldap port >> 3268?(http://wiki2.dovecot.org/AuthDatabase/LDAP) >> > Dead on - it was a restriction of ldap port 3268 - as soon as we pointed > ldapsearch at the same port, we got the same result - some of the fields > were missing. ?It all makes perfect sense and I wish I noticed that earlier. > > Now need to work out why Dovecot can get the fields and username back from > ldap on port 389, but it can't do the auth through it like it could with > 3268. > > Thanks Nikita for your help. > > Andrew Just to update the solution for us in the end... We couldn't get auth to work on port 389, but in the end we managed to get the maxStorage field to be presented via the query on port 3268. What we needed to do was to ensure that it was added as an attribute to the Global Catalog on our Windows Servers. We could then get this entry back via our query. There's instructions on how to do this on the following page: http://technet.microsoft.com/en-us/library/cc737521(v=ws.10).aspx Thanks for the help. Regards, Andrew From lists at wiesinger.com Thu Apr 26 09:26:34 2012 From: lists at wiesinger.com (Gerhard Wiesinger) Date: Thu, 26 Apr 2012 08:26:34 +0200 Subject: [Dovecot] v2.1.5 released In-Reply-To: References: <2363C77C-B8C6-4421-BC41-32F19E49DEAB@iki.fi> Message-ID: <4F98EA9A.9080909@wiesinger.com> Hello Timo, Yes, fixed it. Thnx. I also had a coredump with 2.1.3 + http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e A02 LIST "" "*" imap(username: Panic: file mail-storage.c: line 628 (mailbox_alloc): assertion failed: (uni_utf8_str_is_valid(vname)) Also fixed with 2.1.5 + http://hg.dovecot.org/dovecot-2.1/rev/249c1c89d9d3 Ciao, Gerhard -- http://www.wiesinger.com/ On 25.04.2012 23:16, Timo Sirainen wrote: > On 24.4.2012, at 11.59, Gerhard Wiesinger wrote: > >> Hello Timo, >> >> Dovecot 2.1.5 seems to be broken. After upgrade from >> 2.1.3 + patch http://hg.dovecot.org/dovecot-2.1/rev/86e6dc46a80e >> I only see Inbox and Trash folder. >> >> Reverting => OK. >> >> Same config as before and disucssed per private E-mail and on the list. Guessing this comes again from: >> mail_full_filesystem_access=yes > I guess the problem was this: http://hg.dovecot.org/dovecot-2.1/rev/249c1c89d9d3 > > From markus at mpetri.org Thu Apr 26 09:34:44 2012 From: markus at mpetri.org (Markus Petri) Date: Thu, 26 Apr 2012 08:34:44 +0200 Subject: [Dovecot] Permission problems when trying to create new folders with ACL plugin In-Reply-To: <7D3AF25B-FAE0-4187-B53C-DBB477A4AE79@iki.fi> References: <20120424130005.0db3d425@legolas.home.ceotex.de> <7D3AF25B-FAE0-4187-B53C-DBB477A4AE79@iki.fi> Message-ID: <20120426083444.50be1185@legolas.home.ceotex.de> On Thu, 26 Apr 2012 00:27:25 +0300 Timo Sirainen wrote: > On 24.4.2012, at 14.00, Markus Petri wrote: > > > I have the a problem with shared folders and the ACL plugin. The > > user "markus" has full access to "Shared/shared-user/Data": > > > > 3 GETACL "Shared/shared-user/Data" > > * ACL "Shared/shared-user/Data" "markus" akxeilprwtscd > > 3 OK Getacl completed. > > > > but creating a new subfolder under "Data" fails: > > > > 4 CREATE "Shared/shared-user/Data/New1" > > 4 NO [NOPERM] Permission denied > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e8b80e0767ac > That fixed it. Thanks. From tomislav.mihalicek at gmail.com Thu Apr 26 13:37:32 2012 From: tomislav.mihalicek at gmail.com (Tomislav Mihalicek) Date: Thu, 26 Apr 2012 03:37:32 -0700 (PDT) Subject: [Dovecot] dovecot 2:2.1.5-0~auto+6 on debian squeeze Message-ID: <33752072.post@talk.nabble.com> After upgrade from latest 2.1.4 to 2.1.5 occasional coredumps on lmtp/lda daemon happens. It results in deffering mail in previous hop server and mail delay [1383350.925251] lmtp[22130]: segfault at 0 ip 00007f8829e2b4c5 sp 00007fff29fd37e0 error 4 in libdovecot-lda.so.0.0.0[7f8829e23000+b000] [1383351.038095] lmtp[22121]: segfault at 0 ip 00007f18863084c5 sp 00007fff65d87b20 error 4 in libdovecot-lda.so.0.0.0[7f1886300000+b000] [1383366.682285] lmtp[22101]: segfault at 0 ip 00007fe87a36a4c5 sp 00007fffb391b430 error 4 in libdovecot-lda.so.0.0.0[7fe87a362000+b000] [1383530.705974] lmtp[22126]: segfault at 0 ip 00007f0a7caa74c5 sp 00007ffff5bd0050 error 4 in libdovecot-lda.so.0.0.0[7f0a7ca9f000+b000] [1383530.844101] lmtp[17613]: segfault at 0 ip 00007f43aa1e84c5 sp 00007fff977ffd70 error 4 in libdovecot-lda.so.0.0.0[7f43aa1e0000+b000] [1383544.463542] lmtp[17576]: segfault at 0 ip 00007fbec6bc64c5 sp 00007fff3e347910 error 4 in libdovecot-lda.so.0.0.0[7fbec6bbe000+b000] -- View this message in context: http://old.nabble.com/dovecot-2%3A2.1.5-0%7Eauto%2B6-on-debian-squeeze-tp33752072p33752072.html Sent from the Dovecot mailing list archive at Nabble.com. From tomislav.mihalicek at gmail.com Thu Apr 26 13:39:22 2012 From: tomislav.mihalicek at gmail.com (Tomislav Mihalicek) Date: Thu, 26 Apr 2012 03:39:22 -0700 (PDT) Subject: [Dovecot] segfaults with dovecot 2:2.1.5-0~auto+6 on debian squeeze Message-ID: <33752072.post@talk.nabble.com> After upgrade from latest 2.1.4 to 2.1.5 occasional coredumps on lmtp/lda daemon happens. It results in deffering mail in previous hop server and mail delay [1383350.925251] lmtp[22130]: segfault at 0 ip 00007f8829e2b4c5 sp 00007fff29fd37e0 error 4 in libdovecot-lda.so.0.0.0[7f8829e23000+b000] [1383351.038095] lmtp[22121]: segfault at 0 ip 00007f18863084c5 sp 00007fff65d87b20 error 4 in libdovecot-lda.so.0.0.0[7f1886300000+b000] [1383366.682285] lmtp[22101]: segfault at 0 ip 00007fe87a36a4c5 sp 00007fffb391b430 error 4 in libdovecot-lda.so.0.0.0[7fe87a362000+b000] [1383530.705974] lmtp[22126]: segfault at 0 ip 00007f0a7caa74c5 sp 00007ffff5bd0050 error 4 in libdovecot-lda.so.0.0.0[7f0a7ca9f000+b000] [1383530.844101] lmtp[17613]: segfault at 0 ip 00007f43aa1e84c5 sp 00007fff977ffd70 error 4 in libdovecot-lda.so.0.0.0[7f43aa1e0000+b000] [1383544.463542] lmtp[17576]: segfault at 0 ip 00007fbec6bc64c5 sp 00007fff3e347910 error 4 in libdovecot-lda.so.0.0.0[7fbec6bbe000+b000] -- View this message in context: http://old.nabble.com/segfaults-with-dovecot-2%3A2.1.5-0%7Eauto%2B6-on-debian-squeeze-tp33752072p33752072.html Sent from the Dovecot mailing list archive at Nabble.com. From interfasys at gmail.com Thu Apr 26 17:23:25 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 26 Apr 2012 15:23:25 +0100 Subject: [Dovecot] LDA crashes on delivery (using sieve) Message-ID: <4F995A5D.7070708@gmail.com> Hello, The current version of LDA can crash (seg fault 11) under certain circumstances. I don't know why it happens only with some accounts, but here is a backtrace. The user has a sieve filter in place : (gdb) bt full #0 0x0000000800443434 in smtp_client_open () from /usr/lib/dovecot/libdovecot-lda.so.0 No symbol table info available. #1 0x00000008009943ab in lda_sieve_smtp_open.2406.2210 () from /usr/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #2 0x0000000800e80532 in sieve_smtp_open () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #3 0x0000000800e6157d in act_redirect_send.23018 () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #4 0x0000000800e7e175 in act_redirect_commit.23025 () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #5 0x0000000800e5f2bd in sieve_result_execute () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #6 0x0000000800e5ee7b in sieve_multiscript_execute.25305 () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #7 0x0000000800e496f8 in sieve_multiscript_run () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #8 0x0000000800994652 in lda_sieve_multiscript_execute.2399.2297 () from /usr/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #9 0x00000008009942c7 in lda_sieve_deliver_mail.2402.2206 () from /usr/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #10 0x00000008004435d8 in mail_deliver () from /usr/lib/dovecot/libdovecot-lda.so.0 No symbol table info available. #11 0x00000000004029f7 in main (argc=7, argv=0x7fffffffe830) at main.c:434 set_roots = {0x404520, 0x0} ctx = {pool = 0x800a0d820, set = 0x800a3cc98, session = 0x800a0d840, dup_ctx = 0x800abf0c0, session_id = 0x0, src_mail = 0x800a34840, src_envelope_sender = 0x800a0d858 "mailtest at example.com", dest_user = 0x800a89040, dest_addr = 0x7fffffffebfd "test at example.com", final_dest_addr = 0x7fffffffebfd "test at example.com", dest_mailbox_name = 0x40356b "INBOX", dest_mail = 0x0, var_expand_table = 0x0, tried_default_save = false, saved_mail = false, save_dest_mail = false, mailbox_full = false, dsn = false} service_flags = 1027 user = (const __unknown__ *) 0x7fffffffebcc "test at example.com" errstr = (const __unknown__ *) 0x0 path = (const __unknown__ *) 0x0 storage_service = (struct mail_storage_service_ctx *) 0x800a30840 service_user = (struct mail_storage_service_user *) 0x800a3c0a0 service_input = {module = 0x403567 "lda", service = 0x403567 "lda", username = 0x7fffffffebcc "test at example.com", session_id = 0x0, local_ip = {family = 0, u = {ip6 = {__u6_addr = {__u6_addr8 = '\0' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__u6_addr = {__u6_addr8 = '\0' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0} storage = (struct mail_storage *) 0x0 user_source = (const __unknown__ *) 0x4034c0 "" destaddr_source = (const __unknown__ *) 0x403576 "-a parameter" process_euid = 0 stderr_rejection = false ret = 1 c = -1 error = MAIL_ERROR_NONE Cheers, Olivier From interfasys at gmail.com Thu Apr 26 18:32:23 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 26 Apr 2012 16:32:23 +0100 Subject: [Dovecot] LDA crashes on delivery (using sieve) Message-ID: <4F996A87.90701@gmail.com> Hello, The current version of LDA can crash (seg fault 11) under certain circumstances. I don't know why it happens only with some accounts, but here is a backtrace. The user has a sieve filter in place : (gdb) bt full #0 0x0000000800443434 in smtp_client_open () from /usr/lib/dovecot/libdovecot-lda.so.0 No symbol table info available. #1 0x00000008009943ab in lda_sieve_smtp_open.2406.2210 () from /usr/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #2 0x0000000800e80532 in sieve_smtp_open () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #3 0x0000000800e6157d in act_redirect_send.23018 () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #4 0x0000000800e7e175 in act_redirect_commit.23025 () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #5 0x0000000800e5f2bd in sieve_result_execute () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #6 0x0000000800e5ee7b in sieve_multiscript_execute.25305 () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #7 0x0000000800e496f8 in sieve_multiscript_run () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #8 0x0000000800994652 in lda_sieve_multiscript_execute.2399.2297 () from /usr/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #9 0x00000008009942c7 in lda_sieve_deliver_mail.2402.2206 () from /usr/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #10 0x00000008004435d8 in mail_deliver () from /usr/lib/dovecot/libdovecot-lda.so.0 No symbol table info available. #11 0x00000000004029f7 in main (argc=7, argv=0x7fffffffe830) at main.c:434 set_roots = {0x404520, 0x0} ctx = {pool = 0x800a0d820, set = 0x800a3cc98, session = 0x800a0d840, dup_ctx = 0x800abf0c0, session_id = 0x0, src_mail = 0x800a34840, src_envelope_sender = 0x800a0d858 "mailtest at example.com", dest_user = 0x800a89040, dest_addr = 0x7fffffffebfd "test at example.com", final_dest_addr = 0x7fffffffebfd "test at example.com", dest_mailbox_name = 0x40356b "INBOX", dest_mail = 0x0, var_expand_table = 0x0, tried_default_save = false, saved_mail = false, save_dest_mail = false, mailbox_full = false, dsn = false} service_flags = 1027 user = (const __unknown__ *) 0x7fffffffebcc "test at example.com" errstr = (const __unknown__ *) 0x0 path = (const __unknown__ *) 0x0 storage_service = (struct mail_storage_service_ctx *) 0x800a30840 service_user = (struct mail_storage_service_user *) 0x800a3c0a0 service_input = {module = 0x403567 "lda", service = 0x403567 "lda", username = 0x7fffffffebcc "test at example.com", session_id = 0x0, local_ip = {family = 0, u = {ip6 = {__u6_addr = {__u6_addr8 = '\0' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__u6_addr = {__u6_addr8 = '\0' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0} storage = (struct mail_storage *) 0x0 user_source = (const __unknown__ *) 0x4034c0 "" destaddr_source = (const __unknown__ *) 0x403576 "-a parameter" process_euid = 0 stderr_rejection = false ret = 1 c = -1 error = MAIL_ERROR_NONE Cheers, Olivier From tss at iki.fi Thu Apr 26 18:35:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Apr 2012 18:35:17 +0300 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <4F996A87.90701@gmail.com> References: <4F996A87.90701@gmail.com> Message-ID: <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> On 26.4.2012, at 18.32, interfaSys s?rl wrote: > The current version of LDA can crash (seg fault 11) under certain > circumstances. "The current" means what exactly? (Stephan's autobuilt .debs?) From feltrin at gmail.com Thu Apr 26 18:40:30 2012 From: feltrin at gmail.com (Jean Michel) Date: Thu, 26 Apr 2012 12:40:30 -0300 Subject: [Dovecot] Troubles with /.dovecot.lda-dupe on dovecot 2.1.5 Message-ID: Seems that is something with vacation message, Apr 26 12:20:02 server dovecot: lmtp(26937, user at domain): Error: file_dotlock_create(/storage/domain/users/user/.dovecot.lda-dupes) failed: Resource temporarily unavailable Apr 26 12:20:02 server dovecot: lmtp(26937, user at domain): Fatal: master: service(lmtp): child 26937 killed with signal 11 (core dumps disabled) And its created a .dovecot.lda-dupes.lock Any idea ? -- ------------------------------ Jean Michel Feltrin From interfasys at gmail.com Thu Apr 26 18:47:08 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 26 Apr 2012 16:47:08 +0100 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> References: <4F996A87.90701@gmail.com> <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> Message-ID: <4F996DFC.8070607@gmail.com> Latest revision from these repositories: dovecot-2.1 dovecot-2.1-pigeonhole On 26/04/2012 16:35, Timo Sirainen wrote: > On 26.4.2012, at 18.32, interfaSys s?rl wrote: > >> The current version of LDA can crash (seg fault 11) under certain >> circumstances. > > > "The current" means what exactly? (Stephan's autobuilt .debs?) > > > From goetz.reinicke at filmakademie.de Thu Apr 26 18:48:36 2012 From: goetz.reinicke at filmakademie.de (=?ISO-8859-15?Q?G=F6tz_Reinicke?=) Date: Thu, 26 Apr 2012 17:48:36 +0200 Subject: [Dovecot] OT but E-Mail related. save a copy off all mail as eml Message-ID: <4F996E54.3020706@filmakademie.de> Hi, I'm sorry to bug you with an not directly dovecot related question, but this is for me right now the place with the best mailsystem admins I know. I'm asked if there is a way to save a copy of all mail we send trough our sendmail server (Red Hat EL 5.x) in and out as an .eml file for further processing by a DMS system. The files should be saved on some network share. Any suggestion are welcome and thanks. Best regards G?tz -- G?tz Reinicke IT-Koordinator Tel. +49 7141 969 82 420 Fax +49 7141 969 55 420 E-Mail goetz.reinicke at filmakademie.de Filmakademie Baden-W?rttemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: J?rgen Walter MdL Staatssekret?r im Ministerium f?r Wissenschaft, Forschung und Kunst Baden-W?rttemberg Gesch?ftsf?hrer: Prof. Thomas Schadt -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5161 bytes Desc: S/MIME Kryptografische Unterschrift URL: From robert at schetterer.org Thu Apr 26 19:02:36 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 26 Apr 2012 18:02:36 +0200 Subject: [Dovecot] Troubles with /.dovecot.lda-dupe on dovecot 2.1.5 In-Reply-To: References: Message-ID: <4F99719C.40500@schetterer.org> Am 26.04.2012 17:40, schrieb Jean Michel: > Seems that is something with vacation message, > > Apr 26 12:20:02 server dovecot: lmtp(26937, user at domain): Error: > file_dotlock_create(/storage/domain/users/user/.dovecot.lda-dupes) failed: > Resource temporarily unavailable > Apr 26 12:20:02 server dovecot: lmtp(26937, user at domain): Fatal: master: > service(lmtp): child 26937 killed with signal 11 (core dumps disabled) > And its created a .dovecot.lda-dupes.lock > > Any idea ? jep thats from sieve ,see my my mails from yesterday -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From interfasys at gmail.com Thu Apr 26 19:03:31 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 26 Apr 2012 17:03:31 +0100 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> References: <4F996A87.90701@gmail.com> <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> Message-ID: <4F9971D3.4020107@gmail.com> And it's on FreeBSD 8.2, compiled from source, using GCC 4.6 and the gold linker. On 26/04/2012 16:35, Timo Sirainen wrote: > On 26.4.2012, at 18.32, interfaSys s?rl wrote: > >> The current version of LDA can crash (seg fault 11) under certain >> circumstances. > > > "The current" means what exactly? (Stephan's autobuilt .debs?) > > > From robert at schetterer.org Thu Apr 26 19:04:58 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 26 Apr 2012 18:04:58 +0200 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <4F996DFC.8070607@gmail.com> References: <4F996A87.90701@gmail.com> <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> <4F996DFC.8070607@gmail.com> Message-ID: <4F99722A.7080300@schetterer.org> Am 26.04.2012 17:47, schrieb interfaSys s?rl: > Latest revision from these repositories: > dovecot-2.1 > dovecot-2.1-pigeonhole dovecot-2.1-pigeonhole from hg ,is your problem use i.e http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz to recompile , this works > > > On 26/04/2012 16:35, Timo Sirainen wrote: >> On 26.4.2012, at 18.32, interfaSys s?rl wrote: >> >>> The current version of LDA can crash (seg fault 11) under certain >>> circumstances. >> >> >> "The current" means what exactly? (Stephan's autobuilt .debs?) >> >> >> -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Thu Apr 26 19:09:01 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 26 Apr 2012 18:09:01 +0200 Subject: [Dovecot] OT but E-Mail related. save a copy off all mail as eml In-Reply-To: <4F996E54.3020706@filmakademie.de> References: <4F996E54.3020706@filmakademie.de> Message-ID: <4F99731D.3010405@schetterer.org> Am 26.04.2012 17:48, schrieb G?tz Reinicke: > Hi, > > I'm sorry to bug you with an not directly dovecot related question, but > this is for me right now the place with the best mailsystem admins I know. > > I'm asked if there is a way to save a copy of all mail we send trough > our sendmail server (Red Hat EL 5.x) in and out as an .eml file for > further processing by a DMS system. The files should be saved on some > network share. you need archive solution, in postfix its done like i.e.this recipient_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre sender_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre then use i.e procmail for filter , but i wouldnt recommend .eml i may be not easy to build uniq file names from i.e mail subject storing in maildir should be good enough ( each mail is a file ) > > Any suggestion are welcome and thanks. Best regards G?tz -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Thu Apr 26 19:10:59 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 26 Apr 2012 18:10:59 +0200 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <4F9971D3.4020107@gmail.com> References: <4F996A87.90701@gmail.com> <3669E6C2-B9CA-4F47-BD74-9EFB8D8A612F@iki.fi> <4F9971D3.4020107@gmail.com> Message-ID: <4F997393.5050306@schetterer.org> Am 26.04.2012 18:03, schrieb interfaSys s?rl: > And it's on FreeBSD 8.2, compiled from source, using GCC 4.6 and the > gold linker. > > On 26/04/2012 16:35, Timo Sirainen wrote: >> On 26.4.2012, at 18.32, interfaSys s?rl wrote: >> >>> The current version of LDA can crash (seg fault 11) under certain >>> circumstances. >> >> >> "The current" means what exactly? (Stephan's autobuilt .debs?) >> >> >> oh ,may be more complicated with bsd i think i saw some late patches to compile -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From interfasys at gmail.com Thu Apr 26 22:18:10 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 26 Apr 2012 20:18:10 +0100 Subject: [Dovecot] LDA crashes on delivery (using sieve) Message-ID: <4F999F72.1060603@gmail.com> I can confirm that the issues are gone when using pigeonhole-0.3.0 I thought the sievec crashes might re-appear, but this isn't the case. Thank you Robert. Olivier > Am 26.04.2012 17:47, schrieb interfaSys s?rl: > >/ Latest revision from these repositories: > />/ dovecot-2.1 > />/ dovecot-2.1-pigeonhole > / > dovecot-2.1-pigeonhole from hg ,is your problem > use i.e > > http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz > to recompile , this works > > >/ > />/ > />/ On 26/04/2012 16:35, Timo Sirainen wrote: > />>/ On 26.4.2012, at 18.32, interfaSys s?rl wrote: > />>/ > />>>/ The current version of LDA can crash (seg fault 11) under certain > />>>/ circumstances. > />>/ > />>/ > />>/ "The current" means what exactly? (Stephan's autobuilt .debs?) > />>/ > />>/ > />>/ > / > > -- > Best Regards > > MfG Robert Schetterer > > Germany/Munich/Bavaria From stephan at rename-it.nl Fri Apr 27 03:28:04 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 27 Apr 2012 02:28:04 +0200 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <4F996A87.90701@gmail.com> References: <4F996A87.90701@gmail.com> Message-ID: <4F99E814.3050509@rename-it.nl> On 4/26/2012 5:32 PM, interfaSys s?rl wrote: > Hello, > > The current version of LDA can crash (seg fault 11) under certain > circumstances. > > I don't know why it happens only with some accounts, but here is a > backtrace. The user has a sieve filter in place : Fixed: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/f146790b5db3 Regards, Stephan. From nick.urbanik at optusnet.com.au Fri Apr 27 09:34:31 2012 From: nick.urbanik at optusnet.com.au (Nick Urbanik) Date: Fri, 27 Apr 2012 16:34:31 +1000 Subject: [Dovecot] 2.1.5: Panic: file mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: (mailbox_list_is_valid_pattern(_list, name)) Message-ID: <20120427063431.GA12180@nick.optusnet.com.au> Dear Folks, We had this assertion failure on Dovecot 2.1.1 on a particular mailbox. I upgraded to 2.1.5, and we have the same assertion failure, on the same mailbox: Apr 27 16:02:03 imap(nu-imaptest00258 at example.com): Panic: file mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: (mailbox_list_is_valid_pattern(_list, name)) Apr 27 16:02:03 imap(nu-imaptest00258 at example.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x4246a) [0x7f310c28146a] -> /usr/lib64/dovecot/libdovecot.so.0(+0x424b6) +[0x7f310c2814b6] -> /usr/lib64/dovecot/libdovecot.so.0(+0x1950a) [0x7f310c25850a] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x889b5) [0x7f310c5539b5] -> +/usr/lib64/dovecot/lib10_quota_plugin.so(+0x8a30) [0x7f3109ccea30] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0x8cdf) [0x7f3109ccecdf] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0x93c8) +[0x7f3109ccf3c8] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0x9436) [0x7f3109ccf436] -> /usr/lib64/dovecot/lib10_quota_plugin.so(quota_get_resource+0x72) [0x7f3109cca792] -> +/usr/lib64/dovecot/lib10_quota_plugin.so(quota_test_alloc+0x1bd) [0x7f3109ccaaed] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0xa870) [0x7f3109cd0870] -> +/usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_save_begin+0x4d) [0x7f310c544b9d] -> dovecot/imap [nu-imaptest00258 at example.com 127.0.0.1 APPEND]() [0x408ac4] -> dovecot/imap +[nu-imaptest00258 at example.com 127.0.0.1 APPEND](cmd_append+0x139) [0x408d69] -> dovecot/imap [nu-imaptest00258 at example.com 127.0.0.1 APPEND](command_exec+0x3d) [0x410aed] -> dovecot/imap +[nu-imaptest00258 at example.com 127.0.0.1 APPEND]() [0x40fa2e] -> dovecot/imap [nu-imaptest00258 at example.com 127.0.0.1 APPEND]() [0x40fb1a] -> dovecot/imap [nu-imaptest00258 at example.com +127.0.0.1 APPEND](client_handle_input+0x135) [0x40fd45] -> dovecot/imap [nu-imaptest00258 at example.com 127.0.0.1 APPEND](client_input+0x5f) [0x41066f] -> +/usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f310c28d9d6] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) [0x7f310c28ea5f] -> +/usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f310c28d978] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f310c279c73] -> dovecot/imap [nu-imaptest00258 at example.com +127.0.0.1 APPEND](main+0x29d) [0x418b3d] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f310bebccdd] -> dovecot/imap [nu-imaptest00258 at example.com 127.0.0.1 APPEND]() [0x408449] Apr 27 16:02:03 imaptest01 perdition.imap4[15893]: Closing session: 211.29.132.209:42922->211.29.132.245:143 authorisation_id=NONE authentication_id="nu-imaptest00258" received=67 sent=53 Apr 27 16:02:03 imap(nu-imaptest00258 at example.com): Fatal: master: service(imap): child 15894 killed with signal 6 (core not dumped) Any suggestions welcome. I guess enabling core dumps is a good start, I'll do that, but has anyone seen this before? -- Nick Urbanik http://nicku.org 808-71011 nick.urbanik at optusnet.com.au GPG: 7FFA CDC7 5A77 0558 DC7A 790A 16DF EC5B BB9D 2C24 ID: BB9D2C24 I disclaim, therefore I am. From robert at schetterer.org Fri Apr 27 09:50:24 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 27 Apr 2012 08:50:24 +0200 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <4F99E814.3050509@rename-it.nl> References: <4F996A87.90701@gmail.com> <4F99E814.3050509@rename-it.nl> Message-ID: <4F9A41B0.8040302@schetterer.org> Am 27.04.2012 02:28, schrieb Stephan Bosch: > On 4/26/2012 5:32 PM, interfaSys s?rl wrote: >> Hello, >> >> The current version of LDA can crash (seg fault 11) under certain >> circumstances. >> >> I don't know why it happens only with some accounts, but here is a >> backtrace. The user has a sieve filter in place : > > Fixed: > > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/f146790b5db3 > > Regards, > > Stephan. Hi Stephan, i can confirm by small tests using deb source latest from http://hg.rename-it.nl your patch fixed the lmtp crash with sieve and redirect/vacation Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From fumiyas at osstech.jp Fri Apr 27 10:20:28 2012 From: fumiyas at osstech.jp (SATOH Fumiyasu) Date: Fri, 27 Apr 2012 16:20:28 +0900 Subject: [Dovecot] [PATCH] Dovecot 2.1 compiling fix for AIX Message-ID: <87vcklfxoj.wl%fumiyas@osstech.jp> Hi, I'm trying to build Dovecot 2.1.5 on AIX 6.1 and got the following build error: $ ./configure ... $ make ... In file included from stats-connection.c:9: stats-plugin.h:14: error: field 'user_cpu' has incomplete type stats-plugin.h:14: error: field 'sys_cpu' has incomplete type ... The attached patch fixes this problem. -- -- Name: SATOH Fumiyasu (fumiyas @ osstech co jp) -- Business Home: http://www.OSSTech.co.jp/ -- Personal Home: https://github.com/fumiyas/ -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-2.1-stats-plugin-build-on-AIX.patch Type: application/octet-stream Size: 821 bytes Desc: not available URL: From fumiyas at osstech.jp Fri Apr 27 10:49:51 2012 From: fumiyas at osstech.jp (SATOH Fumiyasu) Date: Fri, 27 Apr 2012 16:49:51 +0900 Subject: [Dovecot] [PATCH] Dovecot 2.1 compiling fix for AIX In-Reply-To: <87vcklfxoj.wl%fumiyas@osstech.jp> References: <87vcklfxoj.wl%fumiyas@osstech.jp> Message-ID: <87sjfpfwbk.wl%fumiyas@osstech.jp> Revised patch. Sorry. At Fri, 27 Apr 2012 16:20:28 +0900, SATOH Fumiyasu wrote: > I'm trying to build Dovecot 2.1.5 on AIX 6.1 and got the following > build error: > > $ ./configure > ... > $ make > ... > In file included from stats-connection.c:9: > stats-plugin.h:14: error: field 'user_cpu' has incomplete type > stats-plugin.h:14: error: field 'sys_cpu' has incomplete type > ... > > The attached patch fixes this problem. -- -- Name: SATOH Fumiyasu (fumiyas @ osstech co jp) -- Business Home: http://www.OSSTech.co.jp/ -- Personal Home: https://github.com/fumiyas/ -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-2.1-include-sys-time.h-to-build-on-AIX.patch Type: application/octet-stream Size: 1230 bytes Desc: not available URL: From interfasys at gmail.com Fri Apr 27 20:11:14 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Fri, 27 Apr 2012 18:11:14 +0100 Subject: [Dovecot] LDA crashes on delivery (using sieve) In-Reply-To: <4F99E814.3050509@rename-it.nl> References: <4F996A87.90701@gmail.com> <4F99E814.3050509@rename-it.nl> Message-ID: <4F9AD332.4080803@gmail.com> Cheers Stephan! :) On 27/04/2012 01:28, Stephan Bosch wrote: > On 4/26/2012 5:32 PM, interfaSys s?rl wrote: >> Hello, >> >> The current version of LDA can crash (seg fault 11) under certain >> circumstances. >> >> I don't know why it happens only with some accounts, but here is a >> backtrace. The user has a sieve filter in place : > > Fixed: > > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/f146790b5db3 > > Regards, > > Stephan. From almarzuki2001 at hotmail.com Sat Apr 28 03:45:08 2012 From: almarzuki2001 at hotmail.com (Hadi Salem) Date: Sat, 28 Apr 2012 03:45:08 +0300 Subject: [Dovecot] dovecot LDA error In-Reply-To: References: , Message-ID: Hi Timo Sirainen, It?s working with me fine. With no problem. After I configured my MTA (sendmail) to run dovecot-lda as root with chmod 4750 /local/user/libexce/dovecot/dovecot-lda. And create group called secmail. I had to change dovecot-lda group to secmail. And then added mail to secmail group. It works with me. Before when I changed dovecot-lda group direct to mail doesn?t work. But it should to work.!!! I think that was the error. Later after creating separate group and adding mail to it, it?s working fine.. Any idea here? why doesn?t work if I changed dovecot-lda direct to mail group instead of creating new group and adding mail to it???. Hadi- > From: tss at iki.fi > Date: Thu, 26 Apr 2012 00:29:51 +0300 > To: almarzuki2001 at hotmail.com > CC: dovecot at dovecot.org > Subject: Re: [Dovecot] dovecot LDA error > > On 24.4.2012, at 20.41, Hadi Salem wrote: > > > Apr 24 20:25:46 lda(jamal at example.com): Fatal: > > setgid(5000(vmail) from userdb lo > > okup) failed with > > euid=8(mail), gid=12(mail), egid=12(mail): Operation not permi > > tted (This binary should probably be called with process group set to 5000(vmail > > ) instead of 12(mail)) > > Configure your MTA to run dovecot-lda as vmail instead of as mail. > From daniel.parthey at informatik.tu-chemnitz.de Sat Apr 28 14:00:23 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 28 Apr 2012 13:00:23 +0200 Subject: [Dovecot] lmtp proxy timeout while waiting for reply to DATA reply Message-ID: <20120428110023.GA9236@daniel.localdomain> Hi, we are experiencing similar sporadic data timeout issues with dovecot 2.0.20 as in http://dovecot.org/pipermail/dovecot/2011-June/059807.html at least once a week. Some mails get temporarily deferred in the postfix queue since dovecot director lmtp refuses them and the mails are delivered at a later time. The problem applies to mails delivered to single users as well as distribution lists where the mails gets delivered to several mailboxes by postfix. The postfix log shows the following output: postfix.log.7.gz:Apr 21 18:48:46 10.129.3.233 postfix/lmtp[15868]: 88062C08E: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=35, delays=5.4/0.01/0/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) The dovecot.log (mailbox and director log there via syslog) shows the following output: Apr 21 18:46:53 10.129.3.233 dovecot: lmtp(15869): Debug: none: root=, index=, control=, inbox=, alt= Apr 21 18:46:53 10.129.3.233 dovecot: lmtp(15869): Connect from 127.0.0.1 Apr 21 18:46:53 10.129.3.233 dovecot: lmtp(15869): Debug: auth input: user=username at example.org proxy port=19024 host=10.129.3.190 proxy_refresh=450 Apr 21 18:46:53 10.129.3.233 dovecot: lmtp(15869): Disconnect from 127.0.0.1: Client quit (in reset) Apr 21 18:47:29 10.129.3.233 dovecot: lmtp(15869): Debug: none: root=, index=, control=, inbox=, alt= Apr 21 18:47:29 10.129.3.233 dovecot: lmtp(15869): Connect from 127.0.0.1 Apr 21 18:47:29 10.129.3.233 dovecot: lmtp(15869): Debug: auth input: user=username at example.org proxy port=19024 host=10.129.3.190 proxy_refresh=450 Apr 21 18:47:29 10.129.3.233 dovecot: lmtp(15869): Disconnect from 127.0.0.1: Client quit (in reset) Apr 21 18:48:15 10.129.3.233 dovecot: lmtp(15869): Debug: none: root=, index=, control=, inbox=, alt= Apr 21 18:48:15 10.129.3.233 dovecot: lmtp(15869): Connect from 127.0.0.1 Apr 21 18:48:15 10.129.3.233 dovecot: lmtp(15869): Debug: auth input: user=username at example.org proxy port=19024 host=10.129.3.190 proxy_refresh=450 Apr 21 18:48:46 10.129.3.233 dovecot: lmtp(15869): Disconnect from 127.0.0.1: Client quit (in reset) One cannot identify the lmtp log origin from the log_format prefixes, but it seems that the "lmtp($PID)" log originated from the director, since proxy port 19024 is mentioned in the logs, which is our dovecot mailbox backend. Config for dovecot (mailbox) and director (proxy) is attached. mail_debug is yes for both dovecot and director. How can we find out more information? Shall I apply the following patches from http://dovecot.org/pipermail/dovecot/2011-June/059987.html or are they already included with version 2.0.20? Regards, Daniel -------------- next part -------------- # 2.0.20: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-40-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_cache_ttl = 1 mins auth_verbose = yes auth_verbose_passwords = sha1 deliver_log_format = mailbox: deliver: msgid=%m from=%f: %$ dict { quota = mysql:/etc/dovecot/conf.d/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = Mailbox login_log_format = mailbox: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "mailbox: mail: %s(%u): " mail_plugins = quota mail_privileged_group = vmail mail_uid = vmail managesieve_implementation_string = Sieve managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 50 M mmap_disable = yes passdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve service dict { unix_listener dict { group = vmail mode = 0660 } } service imap-login { inet_listener imap { port = 19143 } } service lmtp { inet_listener lmtp { address = * port = 19024 } } service managesieve-login { inet_listener sieve { port = 19200 } } service pop3-login { inet_listener pop3 { port = 19110 } } service quota-warning { executable = script /usr/local/bin/quota-warning unix_listener quota-warning { user = vmail } user = dovecot } ssl = no userdb { driver = prefetch } userdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_plugins = quota imap_quota } protocol lmtp { mail_plugins = quota sieve } -------------- next part -------------- # 2.0.20: /etc/dovecot-director/dovecot-director.conf # OS: Linux 2.6.32-40-server x86_64 Ubuntu 10.04.4 LTS auth_verbose = yes auth_verbose_passwords = sha1 base_dir = /var/run/dovecot-director deliver_log_format = director: deliver: msgid=%m from=%f: %$ director_mail_servers = 10.129.3.193 10.129.3.192 10.129.3.191 10.129.3.190 director_servers = 10.129.3.193 10.129.3.192 10.129.3.191 10.129.3.190 lmtp_proxy = yes login_greeting = Mail Balancer login_log_format = director: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "director: mail: %s(%u): " mail_privileged_group = vmail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes passdb { args = proxy=y nopassword=y user=%n at dovecotmail.%d driver = static } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director inet_listener imap { port = 20143 } inet_listener imaps { port = 20993 ssl = yes } } service lmtp { inet_listener lmtp { address = * port = 20024 } } service managesieve-login { inet_listener sieve { port = 20200 } } service pop3-login { executable = pop3-login director inet_listener pop3 { port = 20110 } inet_listener pop3s { port = 20995 ssl = yes } } ssl_cert = -- View this message in context: http://old.nabble.com/ERR-Attentication-error-dovecot-tp33763251p33763251.html Sent from the Dovecot mailing list archive at Nabble.com. From tss at iki.fi Sat Apr 28 17:53:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Apr 2012 17:53:12 +0300 Subject: [Dovecot] Session IDs Message-ID: <3AFABAEC-9FB7-449D-9473-64714C2038A7@iki.fi> v2.1.2 added support for "session IDs", which is a 9 year unique identifier for that specific IMAP/POP3 connection. Initially I thought it would only be useful for tracking connections going through Dovecot proxies to backends (the session ID is forwarded), but then I thought it could be useful for everyone if auth process also logs the session ID. So I thought for v2.1.6 I'd change the defaults: a) Failed login: Apr 28 17:37:23 auth: Debug: client in: AUTH 1 PLAIN service=imap secured session=XtG7KL6+jQB/AAAB lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=42125 resp= Apr 28 17:37:23 auth: Debug: passwd-file(foo,127.0.0.1,): lookup: user=foo file=/usr/local/etc/passwd.imap Apr 28 17:37:23 auth: Info: passwd-file(foo,127.0.0.1,): unknown user Apr 28 17:37:25 auth: Debug: client out: FAIL 1 user=foo Apr 28 17:37:50 imap-login: Info: Aborted login (auth failed, 1 attempts in 27 secs): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, session= b) Successful login: Apr 28 17:38:27 auth: Debug: client in: AUTH 2 PLAIN service=imap secured session=rNuHLL6+jgB/AAAB lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=42126 resp= Apr 28 17:38:27 auth: Debug: passwd-file(tss,127.0.0.1,): lookup: user=tss file=/usr/local/etc/passwd.imap Apr 28 17:38:27 auth: Debug: client out: OK 2 user=tss Apr 28 17:38:27 auth: Debug: master in: REQUEST 1375993857 14321 2 0f2f78deedcb7e38a278cf79819360c5 Apr 28 17:38:27 auth: Debug: passwd-file(tss,127.0.0.1,): lookup: user=tss file=/usr/local/etc/passwd.imap Apr 28 17:38:27 auth: Debug: master out: USER 1375993857 tss uid=1000 gid=1000 home=/home/tss Apr 28 17:38:27 imap-login: Info: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=14338, secured, session= .. Apr 28 17:47:15 imap(tss,): Info: Disconnected: Logged out in=8 out=369 The last line requires a modified mail_log_prefix, I wasn't planning on changing its default. I'm mainly wondering: 1) Does this change break someone's auth log line parsing? 2) Does the auth prefix look prettier with or without <> around session id? :) 3) Anything else I should do about this now that I'm changing it anyway? From dieterknopf at googlemail.com Sat Apr 28 19:07:08 2012 From: dieterknopf at googlemail.com (Dieter Knopf) Date: Sat, 28 Apr 2012 18:07:08 +0200 Subject: [Dovecot] Refiltering mails without loosing flags In-Reply-To: <4EE5E8FF.8080306@rename-it.nl> References: <4EE5E8FF.8080306@rename-it.nl> Message-ID: 2011/12/12 Stephan Bosch : > Since last week, the new sieve-filter command line tool, which is intended > for just such a task, is part of the normal compile output for Pigeonhole > 0.3.x for Dovecot v2.1. Unfortunately, it is not released yet, so I guess > you're stuck with your old getmail-based solution for now. > > The tool is still experimental and I must say that I have not tested the > preservation of flags for some time and many changes were performed since > then. I will verify that later today. Also, the interaction with the new > editheader extension needs attention. Is there any update about that? Would be really nice to have an nice and easy way like with procmail. Just via command line not via getmail/imap/.... Thanks From dieterknopf at googlemail.com Sat Apr 28 19:09:10 2012 From: dieterknopf at googlemail.com (Dieter Knopf) Date: Sat, 28 Apr 2012 18:09:10 +0200 Subject: [Dovecot] SETANNOTATION for Dovecot Message-ID: Hello, i'm using kmail2 and for Folder Settings the SETANNOATION command is needed. Is there any way to add this command to Dovecot? I already found a dovecot-metadata-plugin but there is no Package available for Debian. Thanks Dieter From daniel.parthey at informatik.tu-chemnitz.de Sat Apr 28 20:16:54 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 28 Apr 2012 19:16:54 +0200 Subject: [Dovecot] Session IDs In-Reply-To: <3AFABAEC-9FB7-449D-9473-64714C2038A7@iki.fi> References: <3AFABAEC-9FB7-449D-9473-64714C2038A7@iki.fi> Message-ID: <20120428171654.GA19481@daniel.localdomain> Timo Sirainen wrote: > v2.1.2 added support for "session IDs", which is a 9 year unique identifier > for that specific IMAP/POP3 connection. Initially I thought it would only be > useful for tracking connections going through Dovecot proxies to backends > (the session ID is forwarded) Sounds like an interesting debugging feature. > Apr 28 17:38:27 imap-login: Info: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=14338, secured, session= > The last line requires a modified mail_log_prefix, I wasn't planning on changing its default. What does your mail_log_prefix look like? Can you document the needed variable at http://wiki2.dovecot.org/Variables please? > 1) Does this change break someone's auth log line parsing? We don't do any parsing, just sending logs to a central syslog server for debugging purposes. > 2) Does the auth prefix look prettier with or without <> around session id? :) It should be separated using a fixed name like session=... for easy parsing. Regards, Daniel From jeep at rahul.net Sat Apr 28 23:54:45 2012 From: jeep at rahul.net (Jeff Lacki) Date: Sat, 28 Apr 2012 13:54:45 -0700 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query Message-ID: <20120428205446.0C08D1298B0@aqua.rahul.net> Hello- Sorry if this is a noob question, but I cant seem to get my password_query to work with dovecot + mysql. Im using 'Password verification by SQL server' at: http://wiki2.dovecot.org/AuthDatabase/SQL trying to modify it to work with my encrypted passwords in the DB. Im using the following which isnt working: password_query = SELECT NULL AS password, \ 'Y' as nopassword, userid AS user \ FROM users WHERE userid='%u' AND AES_DECRYPT(password, 'mykey')=password 1. Is it even possible to do this via 'password_query'? 2. If so, what am I doing wrong? Thanks, Jeff /mf/home/jeep/shell/.signature From gedalya at gedalya.net Sun Apr 29 00:16:59 2012 From: gedalya at gedalya.net (Gedalya) Date: Sat, 28 Apr 2012 17:16:59 -0400 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <20120428205446.0C08D1298B0@aqua.rahul.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> Message-ID: <4F9C5E4B.5080704@gedalya.net> On 04/28/2012 04:54 PM, Jeff Lacki wrote: > Hello- > > Sorry if this is a noob question, but I cant seem to get my password_query to > work with dovecot + mysql. Im using 'Password verification by SQL server' at: > > http://wiki2.dovecot.org/AuthDatabase/SQL > > trying to modify it to work with my encrypted passwords in the DB. > > Im using the following which isnt working: > > password_query = SELECT NULL AS password, \ > 'Y' as nopassword, userid AS user \ > FROM users WHERE userid='%u' AND AES_DECRYPT(password, 'mykey')=password > > 1. Is it even possible to do this via 'password_query'? > > 2. If so, what am I doing wrong? > > Thanks, > Jeff > > /mf/home/jeep/shell/.signature The wiki says: "The password is in %w variable" Maybe you meant password_query = SELECT NULL AS password, \ 'Y' as nopassword, userid AS user \ FROM users WHERE userid='%u' AND AES_DECRYPT(password, 'mykey')='%w' From jeep at rahul.net Sun Apr 29 00:25:36 2012 From: jeep at rahul.net (Jeff Lacki) Date: Sat, 28 Apr 2012 14:25:36 -0700 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <4F9C5E4B.5080704@gedalya.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <4F9C5E4B.5080704@gedalya.net> Message-ID: <20120428212537.20EDF12954C@aqua.rahul.net> > > > > /mf/home/jeep/shell/.signature > The wiki says: "The password is in %w variable" > Maybe you meant > > password_query = SELECT NULL AS password, \ > 'Y' as nopassword, userid AS user \ > FROM users WHERE userid='%u' AND AES_DECRYPT(password, 'mykey')='%w' > > Thanks, but when I do that, I get an empty value for '%w', not sure why? Apr 28 14:23:48 mydomain dovecot: auth-worker(13349): Debug: sql(jeff,127.0.0.1): query: SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users WHERE userid='jeff' AND AES_DECRYPT(password, 'mykey')='' /mf/home/jeep/shell/.signature From daniel.parthey at informatik.tu-chemnitz.de Sun Apr 29 01:11:29 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 29 Apr 2012 00:11:29 +0200 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <20120428205446.0C08D1298B0@aqua.rahul.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> Message-ID: <20120428221129.GA24819@daniel.localdomain> Hi Jeff, Jeff Lacki wrote: > Sorry if this is a noob question, but I cant seem to get my password_query to > work with dovecot + mysql. Im using 'Password verification by SQL server' at: > > http://wiki2.dovecot.org/AuthDatabase/SQL > > trying to modify it to work with my encrypted passwords in the DB. > > Im using the following which isn't working: > > password_query = SELECT NULL AS password, \ > 'Y' as nopassword, userid AS user \ > FROM users WHERE userid='%u' AND AES_DECRYPT(password, 'mykey')=password > > 1. Is it even possible to do this via 'password_query'? Please provide your dovecot version and output of the following command: doveconf -n and the complete external sql query files without passwords. You might alsolet the SQL server compare the encrypted password in the database with the encrypted string: password_query = SELECT NULL AS password, \ 'Y' as nopassword, userid AS user \ FROM users WHERE userid='%u' AND AES_ENCRYPT('%w','mykey')=password Regards, Daniel From gedalya at gedalya.net Sun Apr 29 01:25:18 2012 From: gedalya at gedalya.net (Gedalya) Date: Sat, 28 Apr 2012 18:25:18 -0400 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <20120428212537.20EDF12954C@aqua.rahul.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <4F9C5E4B.5080704@gedalya.net> <20120428212537.20EDF12954C@aqua.rahul.net> Message-ID: <4F9C6E4E.6030608@gedalya.net> On 04/28/2012 05:25 PM, Jeff Lacki wrote: >>> /mf/home/jeep/shell/.signature >> The wiki says: "The password is in %w variable" >> Maybe you meant >> >> password_query = SELECT NULL AS password, \ >> 'Y' as nopassword, userid AS user \ >> FROM users WHERE userid='%u' AND AES_DECRYPT(password, 'mykey')='%w' >> >> > Thanks, but when I do that, I get an empty value for '%w', not sure why? > > Apr 28 14:23:48 mydomain dovecot: auth-worker(13349): Debug: sql(jeff,127.0.0.1): query: SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users WHERE userid='jeff' AND AES_DECRYPT(password, 'mykey')='' > > > /mf/home/jeep/shell/.signature Are you using a plaintext authentication mechanism? From jeep at rahul.net Sun Apr 29 01:28:43 2012 From: jeep at rahul.net (Jeff Lacki) Date: Sat, 28 Apr 2012 15:28:43 -0700 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <20120428221129.GA24819@daniel.localdomain> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <20120428221129.GA24819@daniel.localdomain> Message-ID: <20120428222843.C0F44129904@aqua.rahul.net> > > > > 1. Is it even possible to do this via 'password_query'? > > Please provide your dovecot version and output of the following command: > doveconf -n > and the complete external sql query files without passwords. > > You might alsolet the SQL server compare the encrypted > password in the database with the encrypted string: > > password_query = SELECT NULL AS password, \ > 'Y' as nopassword, userid AS user \ > FROM users WHERE userid='%u' AND AES_ENCRYPT('%w','mykey')=password > > Regards, > Daniel Thank you Daniel. I downloaded and compiled 2.1.5 yesterday. The problem seems to be that '%w' evaulates to an empty string: Debug: sql(jeff,127.0.0.1): query: SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users WHERE userid='jeff' AND AES_DECRYPT('', 'key')=password I also just noticed that version 2.0.15 in my output below is coming from somewhere? I did try setting things up under 2.0.15 initially last week, but wanted to be up to date so downloaded the latest yesterday. I never did get it all working under 2.0.15 either btw. dovecot -n -c /opt/dovecot/etc/dovecot/dovecot.conf # 2.0.15: /opt/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.35.14-106.fc14.x86_64 x86_64 Fedora release 14 (Laughlin) ext4 auth_debug = yes auth_debug_passwords = yes auth_mechanisms = cram-md5 auth_verbose = yes auth_verbose_passwords = plain default_client_limit = 225 first_valid_uid = 1000 listen = * lock_method = flock mail_location = mbox:/var/mail/%d/%1n/%n:INDEX=/var/indexes/%d/%1n/%n mail_privileged_group = mail mbox_lock_timeout = 1 mins mbox_write_locks = fcntl namespace { inbox = yes location = prefix = separator = . type = private } passdb { args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } protocols = imap service auth { inet_listener { port = 12345 } unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } user = $default_internal_user } service imap-login { inet_listener imap { port = 143 } service_count = 1 } ssl_cert = References: <20120428205446.0C08D1298B0@aqua.rahul.net> <20120428221129.GA24819@daniel.localdomain> <20120428222843.C0F44129904@aqua.rahul.net> Message-ID: <4F9C7160.6030904@gedalya.net> On 04/28/2012 06:28 PM, Jeff Lacki wrote: >>> 1. Is it even possible to do this via 'password_query'? >> Please provide your dovecot version and output of the following command: >> doveconf -n >> and the complete external sql query files without passwords. >> >> You might alsolet the SQL server compare the encrypted >> password in the database with the encrypted string: >> >> password_query = SELECT NULL AS password, \ >> 'Y' as nopassword, userid AS user \ >> FROM users WHERE userid='%u' AND AES_ENCRYPT('%w','mykey')=password >> >> Regards, >> Daniel > Thank you Daniel. I downloaded and compiled 2.1.5 yesterday. > The problem seems to be that '%w' evaulates to an empty string: > > Debug: sql(jeff,127.0.0.1): query: SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users WHERE userid='jeff' AND AES_DECRYPT('', 'key')=password > > I also just noticed that version 2.0.15 in my output below is coming from > somewhere? I did try setting things up under 2.0.15 initially last week, > but wanted to be up to date so downloaded the latest yesterday. I never did > get it all working under 2.0.15 either btw. > > dovecot -n -c /opt/dovecot/etc/dovecot/dovecot.conf > # 2.0.15: /opt/dovecot/etc/dovecot/dovecot.conf > # OS: Linux 2.6.35.14-106.fc14.x86_64 x86_64 Fedora release 14 (Laughlin) ext4 > auth_debug = yes > auth_debug_passwords = yes > auth_mechanisms = cram-md5 > auth_verbose = yes > auth_verbose_passwords = plain > default_client_limit = 225 > first_valid_uid = 1000 > listen = * > lock_method = flock > mail_location = mbox:/var/mail/%d/%1n/%n:INDEX=/var/indexes/%d/%1n/%n > mail_privileged_group = mail > mbox_lock_timeout = 1 mins > mbox_write_locks = fcntl > namespace { > inbox = yes > location = > prefix = > separator = . > type = private > } > passdb { > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > driver = sql > } > protocols = imap > service auth { > inet_listener { > port = 12345 > } > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0666 > user = postfix > } > user = $default_internal_user > } > service imap-login { > inet_listener imap { > port = 143 > } > service_count = 1 > } > ssl_cert = ssl_key = userdb { > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > driver = sql > } > userdb { > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > driver = sql > } > userdb { > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > driver = sql > } > protocol imap { > imap_idle_notify_interval = 1 mins > imap_max_line_length = 64 k > mail_max_userip_connections = 5 > } > > > /mf/home/jeep/shell/.signature Yeap, you seem to only allow cram-md5. In this case, you client isn't transmitting the actual password that the user is typing, so dovecot simply doesn't have the password you want it to put in %w. It rather has a digest of it. The only way to use a non-plaintext auth mechanism is to provide dovecot the correct password from the database in plaintext. http://wiki2.dovecot.org/Authentication/Mechanisms From jeep at rahul.net Sun Apr 29 02:02:42 2012 From: jeep at rahul.net (Jeff Lacki) Date: Sat, 28 Apr 2012 16:02:42 -0700 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <4F9C7160.6030904@gedalya.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <20120428221129.GA24819@daniel.localdomain> <20120428222843.C0F44129904@aqua.rahul.net> <4F9C7160.6030904@gedalya.net> Message-ID: <20120428230243.BCF50129947@aqua.rahul.net> Gedalya wrote: > On 04/28/2012 06:28 PM, Jeff Lacki wrote: > >>> 1. Is it even possible to do this via 'password_query'? > >> Please provide your dovecot version and output of the following command: > >> doveconf -n > >> and the complete external sql query files without passwords. > >> > >> You might alsolet the SQL server compare the encrypted > >> password in the database with the encrypted string: > >> > >> password_query = SELECT NULL AS password, \ > >> 'Y' as nopassword, userid AS user \ > >> FROM users WHERE userid='%u' AND AES_ENCRYPT('%w','mykey')=password > >> > >> Regards, > >> Daniel > > Thank you Daniel. I downloaded and compiled 2.1.5 yesterday. > > The problem seems to be that '%w' evaulates to an empty string: > > > > Debug: sql(jeff,127.0.0.1): query: SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM users WHERE userid='jeff' AND AES_DECRYPT('', 'key')=password > > > > I also just noticed that version 2.0.15 in my output below is coming from > > somewhere? I did try setting things up under 2.0.15 initially last week, > > but wanted to be up to date so downloaded the latest yesterday. I never did > > get it all working under 2.0.15 either btw. > > > > dovecot -n -c /opt/dovecot/etc/dovecot/dovecot.conf > > # 2.0.15: /opt/dovecot/etc/dovecot/dovecot.conf > > # OS: Linux 2.6.35.14-106.fc14.x86_64 x86_64 Fedora release 14 (Laughlin) ext4 > > auth_debug = yes > > auth_debug_passwords = yes > > auth_mechanisms = cram-md5 > > auth_verbose = yes > > auth_verbose_passwords = plain > > default_client_limit = 225 > > first_valid_uid = 1000 > > listen = * > > lock_method = flock > > mail_location = mbox:/var/mail/%d/%1n/%n:INDEX=/var/indexes/%d/%1n/%n > > mail_privileged_group = mail > > mbox_lock_timeout = 1 mins > > mbox_write_locks = fcntl > > namespace { > > inbox = yes > > location = > > prefix = > > separator = . > > type = private > > } > > passdb { > > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > > driver = sql > > } > > protocols = imap > > service auth { > > inet_listener { > > port = 12345 > > } > > unix_listener /var/spool/postfix/private/auth { > > group = postfix > > mode = 0666 > > user = postfix > > } > > user = $default_internal_user > > } > > service imap-login { > > inet_listener imap { > > port = 143 > > } > > service_count = 1 > > } > > ssl_cert = > ssl_key = > userdb { > > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > > driver = sql > > } > > userdb { > > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > > driver = sql > > } > > userdb { > > args = /opt/dovecot/etc/dovecot/conf.d/dovecot-sql.conf.ext > > driver = sql > > } > > protocol imap { > > imap_idle_notify_interval = 1 mins > > imap_max_line_length = 64 k > > mail_max_userip_connections = 5 > > } > > > > > > /mf/home/jeep/shell/.signature > > Yeap, you seem to only allow cram-md5. In this case, you client isn't > transmitting the actual password that the user is typing, so dovecot > simply doesn't have the password you want it to put in %w. It rather has > a digest of it. > The only way to use a non-plaintext auth mechanism is to provide dovecot > the correct password from the database in plaintext. > > http://wiki2.dovecot.org/Authentication/Mechanisms Ok thank you both for your help. I see why I didnt get anything for %w now. After reading the docs the past few days Im a little unsure of what setup I should use then for security purposes (primarily potential sniffing). Obviously using DIGEST-MD5 (per the docs) is better than CRAM-MD5, but is that possible to use here if I am trying to match passwords out of my mysql DB? I certainly do not want to use plaintext (and quite honestly Im not sure what 'login' really means either, but thats another issue). My desire is: - Only virtual users - All users info stored in mysql - login is via squirrelmail ATM, and later imap/iphone or other. Security is my #1 focus right now. Can someone explain the best solution? Or is the best solution to just get an SSL cert and use plaintext? (which is actually my future plan). Thanks! Jeff /mf/home/jeep/shell/.signature From gedalya at gedalya.net Sun Apr 29 02:31:07 2012 From: gedalya at gedalya.net (Gedalya) Date: Sat, 28 Apr 2012 19:31:07 -0400 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <20120428230243.BCF50129947@aqua.rahul.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <20120428221129.GA24819@daniel.localdomain> <20120428222843.C0F44129904@aqua.rahul.net> <4F9C7160.6030904@gedalya.net> <20120428230243.BCF50129947@aqua.rahul.net> Message-ID: <4F9C7DBB.2000604@gedalya.net> On 04/28/2012 07:02 PM, Jeff Lacki wrote: > Security is my #1 focus right now. > > Can someone explain the best solution? Or is the best solution to just get > an SSL cert and use plaintext? (which is actually my future plan). You absolutely must use SSL if you want security. A non-plaintext authentication mechanism only obfuscates the password itself during the login stage. The IMAP session itself (email content) needs to be secured and that can be more important than the email password (people emailing to each other passwords to more interesting things). Getting your certificate signed by a recognized CA helps your clients to verify that the server they are talking to is the server they want to be talking to. It doesn't make the encryption any stronger. If your clients are willing to click "I know what I'm doing, I trust this certificate", then you have the same results. You can try to get a free certificate here - http://www.startssl.com/ - their certificates are trusted by Mozilla and Microsoft products but not by RIM (blackberry) or java. Anyway, given your current setup: you're not using SSL, you want to AES-encrypt your passwords in mysql (you don't trust your database server) and keep your encryption key in the dovecot configuration (you do trust your dovecot server), you can just do: password_query = SELECT AES_DECRYPT(password, 'mykey') AS password, \ userid AS user \ FROM users WHERE userid='%u' This would allow you to use a digest-based authentication mechanism. However, you still have the liability of having your users' passwords in a reversibly encrypted format, with the key available nearby. Once you get SSL set up, it would be better to store the passwords in a salted hash format such as SSHA, and use plaintext auth (over SSL, of course). From jeep at rahul.net Sun Apr 29 05:29:49 2012 From: jeep at rahul.net (Jeff Lacki) Date: Sat, 28 Apr 2012 19:29:49 -0700 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <4F9C7DBB.2000604@gedalya.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <20120428221129.GA24819@daniel.localdomain> <20120428222843.C0F44129904@aqua.rahul.net> <4F9C7160.6030904@gedalya.net> <20120428230243.BCF50129947@aqua.rahul.net> <4F9C7DBB.2000604@gedalya.net> Message-ID: <20120429022949.9559D129606@aqua.rahul.net> > salted hash format such as SSHA, and use plaintext auth (over SSL, of > course). Thank you so much for your in depth reponse Gedalya, I appreciate it! So now that I changed things up and my conf is: auth_mechanisms = plain login disable_plaintext_auth = no I get the following, which I dont get, because now it seems to be telling me that my decrypted AES password needs to be MD5-CRYPT or PLAIN-MD5? Debug: sql(jeff,127.0.0.1): query: SELECT AES_DECRYPT(password, 'mykey') AS password, userid AS user FROM users WHERE userid='jeff' AND AES_DECRYPT(password, 'mykey')='mypass' Error: sql(jeff,127.0.0.1): Invalid password 'mypass' in passdb: Not a valid MD5-CRYPT or PLAIN-MD5 password Debug: sql(jeff,127.0.0.1): MD5(mypass) != 'mypass', try PLAIN scheme instead Apr 28 19:18:56 mydomain dovecot: auth: Debug: client out: FAIL#0111#011user=jeff This is even more confusing as my mechanisms say plain, not any type of MD5? Thanks again! /mf/home/jeep/shell/.signature From gedalya at gedalya.net Sun Apr 29 05:53:20 2012 From: gedalya at gedalya.net (Gedalya) Date: Sat, 28 Apr 2012 22:53:20 -0400 Subject: [Dovecot] help with AES_DECRYPT and password lookup - mysql password_query In-Reply-To: <20120429022949.9559D129606@aqua.rahul.net> References: <20120428205446.0C08D1298B0@aqua.rahul.net> <20120428221129.GA24819@daniel.localdomain> <20120428222843.C0F44129904@aqua.rahul.net> <4F9C7160.6030904@gedalya.net> <20120428230243.BCF50129947@aqua.rahul.net> <4F9C7DBB.2000604@gedalya.net> <20120429022949.9559D129606@aqua.rahul.net> Message-ID: <4F9CAD20.9020706@gedalya.net> On 4/28/2012 10:29 PM, Jeff Lacki wrote: > Error: sql(jeff,127.0.0.1): Invalid password 'mypass' in passdb: Not a valid MD5-CRYPT or PLAIN-MD5 password > Debug: sql(jeff,127.0.0.1): MD5(mypass) != 'mypass', try PLAIN scheme instead In dovecot-sql.conf.ext, set: default_pass_scheme = PLAIN Dovecot needs to be told what format the stored password is in. From ml at well-adjusted.de Sun Apr 29 14:04:51 2012 From: ml at well-adjusted.de (Jochen Spieker) Date: Sun, 29 Apr 2012 13:04:51 +0200 Subject: [Dovecot] Sieve doesn't find user scripts Message-ID: <20120429110451.GF12090@wasteland.homelinux.net> Hi, I want to use Sieve filtering with my Dovecot 1.2 installation on Debian squeeze. I have a virtual domain setup using Portgresql. ManageSieve works fine so far, I can edit and activate/deactive scripts (using Thunderbird + Plugin) and they show up in the filesystem where I expect them to be, see below. The problem is that LDA doesn't find the script. From /var/log/dovecot-deliver.log: | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Loading modules from directory: /usr/lib/dovecot/modules/lda | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: uid=105 | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: gid=109 | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Quota root: name=storage=512000 backend=maildir args= | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: maildir: data=/srv/mail/well-adjusted.de/jrspieker | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: maildir++: root=/srv/mail/well-adjusted.de/jrspieker, index=, control=, inbox= | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: sieve: user has no valid personal script | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: sieve: no scripts to execute: reverting to default delivery. AFAICS, the sieve module is loaded and the home directory is correct. The content of this directory looks like this: | $ ls -ltRa /srv/mail/home/well-adjusted.de/jrspieker/ | /srv/mail/home/well-adjusted.de/jrspieker/: | total 12 | drwx------ 3 vmail vmail 4096 Apr 29 12:22 . | lrwxrwxrwx 1 vmail vmail 16 Apr 29 12:22 .dovecot.sieve -> sieve/test.sieve | drwx------ 3 vmail vmail 4096 Apr 28 22:31 sieve | drwx------ 3 vmail vmail 4096 Apr 27 20:55 .. | | /srv/mail/home/well-adjusted.de/jrspieker/sieve: | total 16 | drwx------ 3 vmail vmail 4096 Apr 29 12:22 .. | drwx------ 3 vmail vmail 4096 Apr 28 22:31 . | drwx------ 2 vmail vmail 4096 Apr 28 22:31 tmp | -rw------- 1 vmail vmail 338 Apr 28 22:31 test.sieve | ? dovecot -n shows this for the plugin section: | plugin: | quota: maildir:storage=512000 | sieve_dir: ~/sieve/ | sieve: ~/.dovecot.sieve And finally, the content of the script: | require["fileinto"]; | | if header :is "subject" "test" { | fileinto "Test"; | } else { | keep; | } What am I missing here? Thanks, Jochen. -- Fashion is more important to me than war, famine, disease or art. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From giles at coochey.net Sun Apr 29 14:47:04 2012 From: giles at coochey.net (Giles Coochey) Date: Sun, 29 Apr 2012 12:47:04 +0100 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <20120429110451.GF12090@wasteland.homelinux.net> References: <20120429110451.GF12090@wasteland.homelinux.net> Message-ID: <4F9D2A38.1010400@coochey.net> On 29/04/2012 12:04, Jochen Spieker wrote: > Hi, > > I want to use Sieve filtering with my Dovecot 1.2 installation on Debian > squeeze. I have a virtual domain setup using Portgresql. > > ManageSieve works fine so far, I can edit and activate/deactive scripts (using > Thunderbird + Plugin) and they show up in the filesystem where I expect them to > be, see below. > > The problem is that LDA doesn't find the script. From > /var/log/dovecot-deliver.log: > > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Loading modules from directory: /usr/lib/dovecot/modules/lda > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: uid=105 > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: gid=109 > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Quota root: name=storage=512000 backend=maildir args= > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: maildir: data=/srv/mail/well-adjusted.de/jrspieker > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: maildir++: root=/srv/mail/well-adjusted.de/jrspieker, index=, control=, inbox= > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: sieve: user has no valid personal script > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: sieve: no scripts to execute: reverting to default delivery. > > AFAICS, the sieve module is loaded and the home directory is correct. The > content of this directory looks like this: > > | $ ls -ltRa /srv/mail/home/well-adjusted.de/jrspieker/ > | /srv/mail/home/well-adjusted.de/jrspieker/: > | total 12 > | drwx------ 3 vmail vmail 4096 Apr 29 12:22 . > | lrwxrwxrwx 1 vmail vmail 16 Apr 29 12:22 .dovecot.sieve -> sieve/test.sieve > | drwx------ 3 vmail vmail 4096 Apr 28 22:31 sieve > | drwx------ 3 vmail vmail 4096 Apr 27 20:55 .. > | > | /srv/mail/home/well-adjusted.de/jrspieker/sieve: > | total 16 > | drwx------ 3 vmail vmail 4096 Apr 29 12:22 .. > | drwx------ 3 vmail vmail 4096 Apr 28 22:31 . > | drwx------ 2 vmail vmail 4096 Apr 28 22:31 tmp > | -rw------- 1 vmail vmail 338 Apr 28 22:31 test.sieve > | ? > What user is deliver running as? -- Best Regards, Giles Coochey, CCNA Security, CCNA NetSecSpec Ltd giles.coochey at netsecspec.co.uk Tel: +44 (0) 7983 877 438 Live Messenger: giles at coochey.net http://www.netsecspec.co.uk http://www.coochey.net -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4928 bytes Desc: S/MIME Cryptographic Signature URL: From ml at well-adjusted.de Sun Apr 29 16:15:08 2012 From: ml at well-adjusted.de (Jochen Spieker) Date: Sun, 29 Apr 2012 15:15:08 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <4F9D2A38.1010400@coochey.net> References: <20120429110451.GF12090@wasteland.homelinux.net> <4F9D2A38.1010400@coochey.net> Message-ID: <20120429131508.GG12090@wasteland.homelinux.net> Giles Coochey: > On 29/04/2012 12:04, Jochen Spieker wrote: >> >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Loading modules from directory: /usr/lib/dovecot/modules/lda >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: uid=105 >>| 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: gid=109 These IDs translate to user/group vmail: | $ grep 105 /etc/passwd | vmail:x:105:109::/home/vmail:/bin/false | | $ grep 109 /etc/group | vmail:x:109: > What user is deliver running as? Should be vmail: /etc/postfix/master.cf: | dovecot unix - n n - 20 pipe | flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} All mailboxes and home directories belong to this user and group. J. -- If nightclub doormen recognised me I would be more fulfilled. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From ml at well-adjusted.de Sun Apr 29 16:44:12 2012 From: ml at well-adjusted.de (Jochen Spieker) Date: Sun, 29 Apr 2012 15:44:12 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <20120429110451.GF12090@wasteland.homelinux.net> References: <20120429110451.GF12090@wasteland.homelinux.net> Message-ID: <20120429134411.GH12090@wasteland.homelinux.net> Jochen Spieker: > > The problem is that LDA doesn't find the script. Ok, using a global script works: deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: auth input: uid=105 deliver(jrspieker at well-adjusted.de): Info: auth input: gid=109 deliver(jrspieker at well-adjusted.de): Info: Quota root: name=storage=512000 backend=maildir args= deliver(jrspieker at well-adjusted.de): Info: maildir: data=/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: maildir++: root=/srv/mail/well-adjusted.de/jrspieker, index=, control=, inbox=/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: sieve: using sieve path for user's script: /srv/mail/home/global/sieve/global.sieve deliver(jrspieker at well-adjusted.de): Info: sieve: opening script /srv/mail/home/global/sieve/global.sieve deliver(jrspieker at well-adjusted.de): Info: sieve: executing compiled script /srv/mail/home/global/sieve/global.sieve deliver(jrspieker at well-adjusted.de): Info: Namespace : Using permissions from /srv/mail/well-adjusted.de/jrspieker: mode=0755 gid=-1 deliver(jrspieker at well-adjusted.de): Info: sieve: msgid=<20120429132328.D581B1E9 at manowar.wasteland.homelinux.net>: stored mail into mailbox 'INBOX' I suspect LDA doesn't look for the script where I expect it to look. Again, dovecot -n show this (now including the global settings): plugin: quota: maildir:storage=512000 sieve_dir: ~/sieve/ sieve: ~/.dovecot.sieve sieve_global_path: /srv/mail/home/global/sieve/global.sieve sieve_global_dir: /srv/mail/home/global/sieve/ I would expect ~ to translate to the userdb_home from the log entry above (/srv/mail/home/well-adjusted.de/jrspieker). ManageSieve's behaviour confirms that expectation. The wiki says "A relative path (or just a filename) will be interpreted to point under the user's home directory."? This works for ManageSieve just like the ~-notation, but LDA still doesn't find the script. J. ? http://wiki.dovecot.org/LDA/Sieve/Dovecot#Per-user_Sieve_script_location -- When I get home from the supermarket I don't know what to do with all the plastic. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From ml at well-adjusted.de Sun Apr 29 16:59:46 2012 From: ml at well-adjusted.de (Jochen Spieker) Date: Sun, 29 Apr 2012 15:59:46 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <20120429134411.GH12090@wasteland.homelinux.net> References: <20120429110451.GF12090@wasteland.homelinux.net> <20120429134411.GH12090@wasteland.homelinux.net> Message-ID: <20120429135945.GI12090@wasteland.homelinux.net> Someone asked for the complete dovecot -n: # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.34-xen-r4 i686 Debian 6.0.4 rootfs log_timestamp: %Y-%m-%d %H:%M:%S protocols: pop3s imaps managesieve ssl_cert_file: /etc/ssl/certs/mail.well-adjusted.de.cert ssl_key_file: /etc/ssl/private/mail.well-adjusted.de.key verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login login_processes_count: 1 login_max_processes_count: 32 first_valid_uid: 1 mail_privileged_group: vmail mail_location: maildir:/srv/mail/%Ld/%Ln fsync_disable: yes mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve lda: fsync_disable: no postmaster_address: postmaster at wasteland.xen-host.de log_path: /var/log/dovecot-deliver.log info_log_path: /var/log/dovecot-deliver.log mail_plugins: quota sieve auth_socket_path: /var/run/dovecot/auth-master mail_debug: yes auth default: mechanisms: plain login user: postgres username_chars: +*abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ username_format: %Lu passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: prefetch userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix master: path: /var/run/dovecot/auth-master mode: 432 user: vmail group: vmail plugin: quota: maildir:storage=512000 sieve_dir: sieve/ sieve: .dovecot.sieve sieve_global_path: /srv/mail/home/global/sieve/global.sieve sieve_global_dir: /srv/mail/home/global/sieve/ J. -- Tony Blair is a hypnotised self-seeking scarecrow just like all the rest. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From daniel.parthey at informatik.tu-chemnitz.de Mon Apr 30 01:49:28 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Mon, 30 Apr 2012 00:49:28 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <20120429135945.GI12090@wasteland.homelinux.net> References: <20120429110451.GF12090@wasteland.homelinux.net> <20120429134411.GH12090@wasteland.homelinux.net> <20120429135945.GI12090@wasteland.homelinux.net> Message-ID: <20120429224927.GA9001@daniel.localdomain> Hi Jochen, Jochen Spieker wrote: > mail_location: maildir:/srv/mail/%Ld/%Ln > plugin: > sieve_dir: sieve/ > sieve: .dovecot.sieve Sieve requires a mail_home directory. The sieve_dir is searched below mail_home, not mail_location where the actual mailboxes are stored. http://wiki.dovecot.org/VirtualUsers/Home Home vs. mail directory Regards, Daniel From snabb at epipe.com Sun Apr 29 21:09:43 2012 From: snabb at epipe.com (Janne Snabb) Date: Sun, 29 Apr 2012 18:09:43 +0000 (UTC) Subject: [Dovecot] setacl fails - does not find dovecot-acl file In-Reply-To: <1320442758.21919.222.camel@hurina> References: <4EA2C213.7030204@mike2k.de> <1320436059.21919.152.camel@hurina> <4EB44B1C.90505@mike2k.de> <1320442758.21919.222.camel@hurina> Message-ID: On Fri, 4 Nov 2011, Timo Sirainen wrote: > On Fri, 2011-11-04 at 21:29 +0100, Michael Stilkerich wrote: > > > Nov 4 16:29:03 keira dovecot: imap(isa): Error: fcntl(unlock) locking > > failed for file /home/dovecot/isa/dovecot.index.log: No such file or > > directory > > Nov 4 16:29:03 keira dovecot: imap(isa): Error: fstat() failed with > > file /home/dovecot/isa/dovecot.index.log: No such file or directory > > These simply shouldn't happen. I'd say it's a kernel bug. You're running > a default Ubuntu kernel? I wonder if other Ubuntu users have this > problem. I am seeing this same problem on Ubuntu 11.10 and 12.04 with stock kernels. The problem is clearly AppArmor related. The imap process seems to be using the "usr.sbin.dovecot" profile which prevents access to these files. There is a separate profile "usr.lib.dovecot.imap" but it seems that it does not get applied to the imap process for some odd reason. This is especially strange beacuse both profiles are enabled in "complain" and not in "enforce" mode, thus they should not enforce any of the rules. I am simultaneously getting messages similar to the following in my audit log: type=AVC msg=audit(1335712674.515:655016): apparmor="ALLOWED" operation="getattr" parent=10922 profile="/usr/sbin/dovecot//null-107//null-10b//null-118" name="/home/foobar/Maildir/.foobar/dovecot.index.log" pid=10937 comm="imap" requested_mask="r" denied_mask="r" fsuid=1000 ouid=1000 After disabling the "usr.sbin.dovecot" profile everything seems fine. Other dovecot related AppArmor profiles do not seem to cause problems. This looks like an issue in AppArmor to me... -- Janne Snabb / EPIPE Communications snabb at epipe.com - http://epipe.com/ From sdavies at sdc.com.au Mon Apr 30 04:08:37 2012 From: sdavies at sdc.com.au (Stephen Davies) Date: Mon, 30 Apr 2012 10:38:37 +0930 Subject: [Dovecot] Log messages Message-ID: <201204301038.37694.sdavies@sdc.com.au> Hello. I have been on leave for several weeks and have managed to lose all emails received since December due to a server crash. I asked about log synch error back in March and have (repeatedly) deleted all .imap files but the errors continue. # 2.1.1: /usr/etc/dovecot/dovecot.conf # OS: Linux 2.6.33.5-desktop-2mnb i686 Mandriva Linux 2010.2 mail_access_groups = mail mail_location = mbox:~/Mail:INBOX=/var/spool/mail/%u managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve sieve ssl_cert = References: <20120429110451.GF12090@wasteland.homelinux.net> <20120429134411.GH12090@wasteland.homelinux.net> <20120429135945.GI12090@wasteland.homelinux.net> <20120429224927.GA9001@daniel.localdomain> Message-ID: <20120430053946.GK12090@wasteland.homelinux.net> Daniel Parthey: > Jochen Spieker wrote: > >> mail_location: maildir:/srv/mail/%Ld/%Ln >> plugin: >> sieve_dir: sieve/ >> sieve: .dovecot.sieve > > Sieve requires a mail_home directory. Is this different from userdb_home? Both mail and home directory are returned from my user/password_queries and LDA reports exactly what I expect: deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker ManageSieve puts scripts under the directory reported as userdb_home: $ ls -ltra /srv/mail/home/well-adjusted.de/jrspieker total 12 drwx------ 3 vmail vmail 4096 Apr 27 20:55 .. drwx------ 3 vmail vmail 4096 Apr 28 22:31 sieve lrwxrwxrwx 1 vmail vmail 16 Apr 29 15:00 .dovecot.sieve -> sieve/test.sieve drwx------ 3 vmail vmail 4096 Apr 29 15:00 . J. -- I feel yawning hollowness whilst talking to people at parties. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From oni-neko at gmx.net Mon Apr 30 11:59:32 2012 From: oni-neko at gmx.net (oni-neko at gmx.net) Date: Mon, 30 Apr 2012 10:59:32 +0200 Subject: [Dovecot] restoring mails and directories Message-ID: <20120430085932.267600@gmx.net> Good day! is there an easy way to restore e.g. deleted mails and/or mail directories? let me specify: I'm running dovecot 1.2.9 on a current ubuntu lts. It is set to Maildir and I backup the folder with the users maildata nightly via rsync. now, when someone deletes a mail that they still need, and I copy it back from the backup, it will not be visible to mail clients. When I change the mailfile by a letter or so, it shows. that's easy enough, if not exactly awesome ;-) But, if said person deletes a whole folder of mails, this way to make restored mails show up becomes rather unreasonable. surely there is a better way to do this? I'd be glad for any and all hints! =) best regards silvana -- Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de From reuben-dovecot at reub.net Mon Apr 30 13:43:10 2012 From: reuben-dovecot at reub.net (Reuben Farrelly) Date: Mon, 30 Apr 2012 20:43:10 +1000 Subject: [Dovecot] Dovecot dsync mail replication issues Message-ID: <4F9E6CBE.4070609@reub.net> Hi, I'm trying to set up some dsync based replication between two hosts on my network. The current topology is a single server running Postfix with a single dovecot installation with a Maildir per user (only 4 users including myself). No NFS, just local system users on ext4. I am only using system users, ie no virtual users. I am using dovecot deliver to deliver mails into the Maildir's. This - being a very standard Maildir installation - all works just fine. I'm running dovecot-2.1.5 (release versions) x86_64 with Gentoo on both hosts and using key based ssh to transport the data using the root login. What I would like to do is extend the design so as to replicate the Maildirs across a second machine and in the future be able to connect via IMAP into either machine to sync mail. Two way dsync would be rather cool because I could then deliver via SMTP to either system and have changes automagically propagate - but initially just simple replication would be a good start. I was hoping that dsync would allow me to do this, but I've run into quite a number of problems while getting this to work. http://dovecot.org/list/dovecot/2012-March/064243.html ...was very useful and I've based my config on that. Initially I've tried to sync up the user Maildirs, and this has more or less worked: doveadm sync -u lyn remote:root at dustbowl.reub.net This succeeds without error on the initial sync. However if I try to run the re-sync again (such as a use case of if the sync of another Maildir takes 4 hours so I want to resync up the earlier ones again) I end up with a mysteriously named INBOX folder in both the source and destination Maildirs: drwx------ 5 lyn lyn 4096 Apr 30 19:32 .INBOX_7a86a62d465a974fb92f00003b258734 It has the basic structure of a Maildir but is empty in terms of mails: drwx------ 2 lyn lyn 4096 Apr 30 19:32 cur -rw------- 1 lyn lyn 220 Apr 30 19:32 dovecot.index.log -rw------- 1 lyn lyn 51 Apr 30 19:32 dovecot-uidlist -rw------- 1 lyn lyn 0 Apr 30 19:32 maildirfolder drwx------ 2 lyn lyn 4096 Apr 30 19:32 new drwx------ 2 lyn lyn 4096 Apr 30 19:32 tmp First question: why is this random named directory being created in the origin Maildir? Shouldn't the replication be more or less read-only in the origin Maildir? Second question: If I re-attempt a doveadm sync a second time I get this error: tornado Maildir # doveadm sync -u lyn remote:root at dustbowl.reub.net dsync-local(lyn): Error: Can't rename mailbox INBOX_7a86a62d465a974fb92f00003b258734 to INBOX: Target mailbox already exists dsync-local(lyn): Error: Can't rename mailbox INBOX to INBOX_eb15f30ea563be4b703200002bd68bb1: Renaming INBOX isn't supported. tornado Maildir # It's not clear if the second attempt has failed or succeeded, and it's a bit odd that it errors out on a directory that the dovecot sync process itself has created. Third question: Upon starting Dovecot lots of ugliness is logged in the mail log: Apr 30 19:44:59 tornado dovecot: master: Dovecot v2.1.5 starting up (core dumps disabled) Apr 30 19:44:59 tornado dovecot: doveadm(mozsync): Error: user mozsync: Initialization failed: Namespace '': mkdir(/var/www/xxx/server-full/Maildir) failed: Permission denied (euid=1016(mozsync) egid=1016() missing +w perm: /var/www/xxx/server-full, dir owned by 0:0 mode=0755) Apr 30 19:44:59 tornado dovecot: doveadm(mozsync): Error: sync: User init failed Apr 30 19:44:59 tornado dovecot: doveadm(mozsync): Warning: I/O leak: 0x414190 (line 102, fd 16) Apr 30 19:44:59 tornado dovecot: dsync-local(cisco): Error: remote: doveadm(cisco): Fatal: User doesn't exist Apr 30 19:44:59 tornado dovecot: dsync-local(cisco): Error: read() from worker server failed: EOF Users mozsync and cisco are not valid mail users and it's not appropriate that Dovecot tries to create a Maildir for either of them. The users are system unprivileged users only, and do not ever send or receive mail. And - I/O leak? ;) Also, user cisco is local to one box only, and does not exist (nor does it need to) on the remote host. So any complaints about this user are invalid and dovecot needs to ignore replication for this user anyway. Fourth question, upon starting dovecot again, mail.log then spews several hundred of these messages: Apr 30 19:45:06 tornado dovecot: dsync-local(reuben): Error: msg-get failed: box=Trash uid=114863 guid=1335382569.M98089P29952.tornado,S=6479,W=6625 Before aborting entirely with: Apr 30 19:45:09 tornado dovecot: imap: Server shutting down. in=328 out=2042 It seems to me that a a few of those problems logged could be solved by being able to specify which system users to synchronise, rather than Dovecot making a blind assumption that all users actually have valid Maildirs that need to be created, and all need to be sync'd between two hosts. Subsequent delivery based sync'ing fails silently (pending more investigation) but I'd like to try and fix the above listed problems first before worrying about the later stages of the sync not working. Has anyone else run into these problems or found better ways of making this type of setup all work? Thanks, Reuben From stephan at rename-it.nl Mon Apr 30 16:06:07 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 30 Apr 2012 15:06:07 +0200 Subject: [Dovecot] Refiltering mails without loosing flags In-Reply-To: References: <4EE5E8FF.8080306@rename-it.nl> Message-ID: <4F9E8E3F.1010004@rename-it.nl> On 4/28/2012 6:07 PM, Dieter Knopf wrote: > 2011/12/12 Stephan Bosch: > >> Since last week, the new sieve-filter command line tool, which is intended >> for just such a task, is part of the normal compile output for Pigeonhole >> 0.3.x for Dovecot v2.1. Unfortunately, it is not released yet, so I guess >> you're stuck with your old getmail-based solution for now. >> >> The tool is still experimental and I must say that I have not tested the >> preservation of flags for some time and many changes were performed since >> then. I will verify that later today. Also, the interaction with the new >> editheader extension needs attention. > Is there any update about that? Would be really nice to have an nice > and easy way > like with procmail. Just via command line not via getmail/imap/.... I didn't receive much feedback so far, so you may very well encounter some bugs. But I did test flag preservation just now and that just works. Pigeonhole v0.3.0 is released and contains all you need. Just experiment with it a bit before you start using it for real mail. Regards, Stephan. From thierry at odry.net Mon Apr 30 17:07:33 2012 From: thierry at odry.net (Thierry de Montaudry) Date: Mon, 30 Apr 2012 16:07:33 +0200 Subject: [Dovecot] restoring mails and directories In-Reply-To: <20120430085932.267600@gmx.net> References: <20120430085932.267600@gmx.net> Message-ID: <98F4E6D0-A89B-408A-ACC1-7BC15FAC5231@odry.net> On 30 Apr 2012, at 10:59, oni-neko at gmx.net wrote: > Good day! > > is there an easy way to restore e.g. deleted mails and/or mail directories? > let me specify: I'm running dovecot 1.2.9 on a current ubuntu lts. It is set to Maildir and I backup the folder with the users maildata nightly via rsync. > > now, when someone deletes a mail that they still need, and I copy it back from the backup, it will not be visible to mail clients. When I change the mailfile by a letter or so, it shows. that's easy enough, if not exactly awesome ;-) > > But, if said person deletes a whole folder of mails, this way to make restored mails show up becomes rather unreasonable. > > surely there is a better way to do this? I'd be glad for any and all hints! =) > > best regards > silvana > -- > Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir > belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de Hi, Not sure about the availability on version 1.2.9, but the following command should do the trick: doveadm force-resync -u Regards, Thierry From stephan at rename-it.nl Mon Apr 30 17:12:03 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 30 Apr 2012 16:12:03 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <20120429110451.GF12090@wasteland.homelinux.net> References: <20120429110451.GF12090@wasteland.homelinux.net> Message-ID: <4F9E9DB3.1000707@rename-it.nl> On 4/29/2012 1:04 PM, Jochen Spieker wrote: > Hi, > > I want to use Sieve filtering with my Dovecot 1.2 installation on Debian > squeeze. I have a virtual domain setup using Portgresql. > > ManageSieve works fine so far, I can edit and activate/deactive scripts (using > Thunderbird + Plugin) and they show up in the filesystem where I expect them to > be, see below. > > The problem is that LDA doesn't find the script. From > /var/log/dovecot-deliver.log: > > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Loading modules from directory: /usr/lib/dovecot/modules/lda > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: uid=105 > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: auth input: gid=109 > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: Quota root: name=storage=512000 backend=maildir args= > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: maildir: data=/srv/mail/well-adjusted.de/jrspieker > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: maildir++: root=/srv/mail/well-adjusted.de/jrspieker, index=, control=, inbox= > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: sieve: user has no valid personal script > | 2012-04-29 12:17:48 deliver(jrspieker at well-adjusted.de): Info: sieve: no scripts to execute: reverting to default delivery. > > AFAICS, the sieve module is loaded and the home directory is correct. The > content of this directory looks like this: Could you enable mail_debug in your configuration? This will provide more detail on why it is not finding a script. Regards, Stephan. From ml at well-adjusted.de Mon Apr 30 17:43:58 2012 From: ml at well-adjusted.de (Jochen Spieker) Date: Mon, 30 Apr 2012 16:43:58 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <4F9E9DB3.1000707@rename-it.nl> References: <20120429110451.GF12090@wasteland.homelinux.net> <4F9E9DB3.1000707@rename-it.nl> Message-ID: <20120430144358.GL12090@wasteland.homelinux.net> Stephan Bosch: >> >> AFAICS, the sieve module is loaded and the home directory is correct. The >> content of this directory looks like this: > > Could you enable mail_debug in your configuration? This will provide > more detail on why it is not finding a script. With globally enabled mail_debug and auth_debug I get this in mail.log: postfix/qmgr[30005]: E306113C1E1: from=, size=1987, nrcpt=1 (queue active) dovecot: auth(default): master in: USER 1 jrspieker at well-adjusted.de service=deliver dovecot: auth(default): prefetch(jrspieker at well-adjusted.de): passdb didn't return userdb entries, trying the next userdb dovecot: auth(default): sql(jrspieker at well-adjusted.de): SELECT u.storage || ':' || d.basedir || u.userid as mail, '/srv/mail/home/well-adjusted.de/jrspieker' as userdb_home, 105 as uid, 109 as gid FROM mailuser u, domain d WHERE u.userid = 'jrspieker' AND u.domain = 'well-adjusted.de' AND u.domain = d.name dovecot: auth(default): master out: USER 1 jrspieker at well-adjusted.de mail=maildir:/srv/mail/well-adjusted.de/jrspieker userdb_home=/srv/mail/home/well-adjusted.de/jrspieker uid=105 gid=109 postfix/smtpd[5421]: disconnect from e177130134.adsl.alicedsl.de[85.177.130.134] postfix/pipe[5428]: E306113C1E1: to=, relay=dovecot, delay=1.1, delays=0.6/0/0/0.5, dsn=2.0.0, status=sent (delivered via dovecot service) postfix/qmgr[30005]: E306113C1E1: removed and this in dovecot-deliver.log: deliver(jrspieker at well-adjusted.de): Info: Loading modules from directory: /usr/lib/dovecot/modules/lda deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so deliver(jrspieker at well-adjusted.de): Info: Module loaded: /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so deliver(jrspieker at well-adjusted.de): Info: auth input: mail=maildir:/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: auth input: uid=105 deliver(jrspieker at well-adjusted.de): Info: auth input: gid=109 deliver(jrspieker at well-adjusted.de): Info: Quota root: name=storage=512000 backend=maildir args= deliver(jrspieker at well-adjusted.de): Info: maildir: data=/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: maildir++: root=/srv/mail/well-adjusted.de/jrspieker, index=, control=, inbox=/srv/mail/well-adjusted.de/jrspieker deliver(jrspieker at well-adjusted.de): Info: sieve: user has no valid personal script deliver(jrspieker at well-adjusted.de): Info: sieve: no scripts to execute: reverting to default delivery. deliver(jrspieker at well-adjusted.de): Info: Namespace : Using permissions from /srv/mail/well-adjusted.de/jrspieker: mode=0755 gid=-1 deliver(jrspieker at well-adjusted.de): Info: msgid=<20120430142818.354B31FE at manowar.wasteland.homelinux.net>: saved mail to INBOX Doesn't look different than before, I had already enabled mail_debug in the "protocol lda" section. J. -- I am not scared of death but terrified of people in Tommy Hilfiger sweatshirts. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From tss at iki.fi Mon Apr 30 18:18:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 30 Apr 2012 18:18:03 +0300 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <20120430144358.GL12090@wasteland.homelinux.net> References: <20120429110451.GF12090@wasteland.homelinux.net> <4F9E9DB3.1000707@rename-it.nl> <20120430144358.GL12090@wasteland.homelinux.net> Message-ID: <1B683452-BBC7-4D97-ACCF-CC1A6568F1A3@iki.fi> On 30.4.2012, at 17.43, Jochen Spieker wrote: > deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker It should say "home" here, not "userdb_home". From ml at well-adjusted.de Mon Apr 30 18:40:52 2012 From: ml at well-adjusted.de (Jochen Spieker) Date: Mon, 30 Apr 2012 17:40:52 +0200 Subject: [Dovecot] Sieve doesn't find user scripts In-Reply-To: <1B683452-BBC7-4D97-ACCF-CC1A6568F1A3@iki.fi> References: <20120429110451.GF12090@wasteland.homelinux.net> <4F9E9DB3.1000707@rename-it.nl> <20120430144358.GL12090@wasteland.homelinux.net> <1B683452-BBC7-4D97-ACCF-CC1A6568F1A3@iki.fi> Message-ID: <20120430154052.GM12090@wasteland.homelinux.net> Timo Sirainen: > On 30.4.2012, at 17.43, Jochen Spieker wrote: > >> deliver(jrspieker at well-adjusted.de): Info: auth input: userdb_home=/srv/mail/home/well-adjusted.de/jrspieker > > It should say "home" here, not "userdb_home". Thanks, that's it! dovecot-deliver.log now reads: deliver(jrspieker at well-adjusted.de): Info: auth input: home=/srv/mail/home/well-adjusted.de/jrspieker ? deliver(jrspieker at well-adjusted.de): Info: sieve: using sieve path for user's script: /srv/mail/home/well-adjusted.de/jrspieker/.dovecot.sieve deliver(jrspieker at well-adjusted.de): Info: sieve: opening script /srv/mail/home/well-adjusted.de/jrspieker/.dovecot.sieve deliver(jrspieker at well-adjusted.de): Info: sieve: executing compiled script /srv/mail/home/well-adjusted.de/jrspieker/.dovecot.sieve Now I can finally start to migrate my .procmailrc. J. -- Driving behind lorries carrying hazardous chemicals makes me wish for a simpler life. [Agree] [Disagree] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From andrei at lctax.ro Mon Apr 30 19:25:43 2012 From: andrei at lctax.ro (Michescu Andrei) Date: Mon, 30 Apr 2012 12:25:43 -0400 Subject: [Dovecot] Dovecot dsync mail replication issues In-Reply-To: <4F9E6CBE.4070609@reub.net> References: <4F9E6CBE.4070609@reub.net> Message-ID: Hello Reuben, I'm having a very similar setup. The 2 main differences: all my users are virtual and the "2nd server" is on a different continent (high latency sync). Unfortunately the dsync is not working for the moment. Timo is in the process of redesigning it. So once it is release will know about it. > > drwx------ 5 lyn lyn 4096 Apr 30 19:32 > .INBOX_7a86a62d465a974fb92f00003b258734 > > First question: why is this random named directory being created in the > origin Maildir? Shouldn't the replication be more or less read-only in > the origin Maildir? - the number it is not random, but rather it is the GUID of the folder on the other server. To get rid of this annoying problem you need to clean your source of all these newly created folders, rsync your folders in between the 2 machines, run dsync again (this time it will not mess up with your folder structure) > Second question: If I re-attempt a doveadm sync a second time I get > this error: > > tornado Maildir # doveadm sync -u lyn remote:root at dustbowl.reub.net > dsync-local(lyn): Error: Can't rename mailbox > INBOX_7a86a62d465a974fb92f00003b258734 to INBOX: Target mailbox already > exists > dsync-local(lyn): Error: Can't rename mailbox INBOX to > INBOX_eb15f30ea563be4b703200002bd68bb1: Renaming INBOX isn't supported. > tornado Maildir # > > It's not clear if the second attempt has failed or succeeded, and it's a > bit odd that it errors out on a directory that the dovecot sync process > itself has created. > do the fix at Q1 and you will not run into this... it is not a permission problem but rather a meta-info problem. The setup will run fine as long as you only update 1 server and the other one is backup. The current release does not handle well the master-master model (you'll endup with emails like the folders above: duplicated, with GUID appended to them etc etc)... Wish Timo good luck and inspiration! Best regards, Andrei From robert at schetterer.org Mon Apr 30 21:55:11 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 30 Apr 2012 20:55:11 +0200 Subject: [Dovecot] question dovecot Inheritance global acl vs userfolder acl Message-ID: <4F9EE00F.5050308@schetterer.org> Hi Timo my tests resulted in inheritance is given if a userfolder has set some acl to its new created subfolder , which is nice if some userfolder has its acl from global acl there is no inheritance to its new created subfolders, that subfolders will always created with full owner rights i am not really sure if its a good idea to have inheritance from global acl and if its hackabel what is your idea to this ? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From dmiller at amfes.com Mon Apr 30 22:42:00 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 30 Apr 2012 12:42:00 -0700 Subject: [Dovecot] Single Instance Storage management Message-ID: Was there a doveadm command that checks the SIS files and removes any not referenced? Will the "doveadm purge -A" catch that? -- Daniel From dmiller at amfes.com Mon Apr 30 22:43:22 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 30 Apr 2012 12:43:22 -0700 Subject: [Dovecot] mdbox packing Message-ID: Is there a way to manually force an mdbox storage to be rebuilt into new files? Particularly files of the maximum size? -- Daniel From dmiller at amfes.com Mon Apr 30 23:16:01 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 30 Apr 2012 13:16:01 -0700 Subject: [Dovecot] Dovecot/doveadm crash Message-ID: Having a problem with a mailbox. I've been trying to rebuild - but doveadm force-resync crashes. This is mdbox with sis. doveadm purge -u dmiller at amfes.com doveadm(dmiller at amfes.com): Panic: file istream.c: line 466 (i_stream_grow_buffer): assertion failed: (stream->max_buffer_size > 0) doveadm(dmiller at amfes.com): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x4167a) [0x7fb73465467a] -> /usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7fb734654762] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fb73462c2a0] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4949b) [0x7fb73465c49b] -> /usr/local/lib/dovecot/libdovecot.so.0(i_stream_get_buffer_space+0x82) [0x7fb73465c522] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4b1c8) [0x7fb73465e1c8] -> /usr/local/lib/dovecot/libdovecot.so.0(i_stream_read+0x69) [0x7fb73465bb69] -> /usr/local/lib/dovecot/libdovecot.so.0(i_stream_read_next_line+0x1d) [0x7fb73465c04d] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mdbox_purge+0x921) [0x7fb7348d7d71] -> doveadm() [0x40e1e6] -> doveadm() [0x40e018] -> doveadm() [0x40ea83] -> doveadm(doveadm_mail_try_run+0x141) [0x40eef1] -> doveadm(main+0x3c1) [0x415911] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xff) [0x7fb73429deff] -> doveadm() [0x40de39] -- Daniel From marcin at mejor.pl Mon Apr 30 23:48:50 2012 From: marcin at mejor.pl (=?ISO-8859-2?Q?Marcin_Miros=B3aw?=) Date: Mon, 30 Apr 2012 22:48:50 +0200 Subject: [Dovecot] mdbox packing In-Reply-To: References: Message-ID: <4F9EFAB2.8040303@mejor.pl> W dniu 2012-04-30 21:43, Daniel L. Miller pisze: > Is there a way to manually force an mdbox storage to be rebuilt into new > files? Particularly files of the maximum size? > Hello, probably doveadm purge should do it. Regards, Marcin