[Dovecot] Best way from Mbox to Maildir using 2.17?
Can someone advise the best way to convert mailboxes from Mbox to Maildir for Dovecot 2.17 on Ubuntu? Thanks.
On 02/11/2013 20:25, Regan Yelcich wrote:
Can someone advise the best way to convert mailboxes from Mbox to Maildir for Dovecot 2.17 on Ubuntu? Thanks.
mb2md.pl
On 2/11/2013, at 11:40 pm, Noel Butler <noel.butler@ausics.net> wrote:
On 02/11/2013 20:25, Regan Yelcich wrote:
Can someone advise the best way to convert mailboxes from Mbox to Maildir for Dovecot 2.17 on Ubuntu? Thanks.
mb2md.pl
I don't need to do anything specific for Dovecot? It'll see the new Maildir account and automatically create the indexes etc?
On 04/11/2013 14:33, Regan Yelcich wrote:
On 2/11/2013, at 11:40 pm, Noel Butler <noel.butler@ausics.net> wrote:
On 02/11/2013 20:25, Regan Yelcich wrote:
Can someone advise the best way to convert mailboxes from Mbox to Maildir for Dovecot 2.17 on Ubuntu? Thanks.
mb2md.pl
I don't need to do anything specific for Dovecot? It'll see the new Maildir account and automatically create the indexes etc?
IIRC it shows you how to use it, you need to indicate where the new maildir is, if you have only a few, do them all manually, if you have many, write a quick bash or perl script to do them, and dovecot will create the indexes when they login.
You will need to tell dovecot to look for the new location though, if your using system users as I suspect you are, then maildir:~/Maildir should do it *but* dont forget to make sure your MTA knows to use maildir as well, I've not worked with system users for a decade, but I think in postfix home_mailbox = Maildir/ will do it, with sendmail its much more tricky and your best sticking with mbox, if exim, NFI - dont use it :)
Thanks for the tips Noel I'll give that a go. I'm moving just a small number of mailboxes over from system users on one server to a new server using a virtual user setup = dovecot + postfix & postfix admin using MySQL. Thanks!
On 4/11/2013, at 10:29 pm, Noel Butler <noel.butler@ausics.net> wrote:
On 04/11/2013 14:33, Regan Yelcich wrote:
On 2/11/2013, at 11:40 pm, Noel Butler <noel.butler@ausics.net> wrote:
On 02/11/2013 20:25, Regan Yelcich wrote: Can someone advise the best way to convert mailboxes from Mbox to Maildir for Dovecot 2.17 on Ubuntu? Thanks. mb2md.pl http://batleth.sapienti-sat.org/projects/mb2md/ I don't need to do anything specific for Dovecot? It'll see the new Maildir account and automatically create the indexes etc?
IIRC it shows you how to use it, you need to indicate where the new maildir is, if you have only a few, do them all manually, if you have many, write a quick bash or perl script to do them, and dovecot will create the indexes when they login.
You will need to tell dovecot to look for the new location though, if your using system users as I suspect you are, then maildir:~/Maildir should do it *but* dont forget to make sure your MTA knows to use maildir as well, I've not worked with system users for a decade, but I think in postfix home_mailbox = Maildir/ will do it, with sendmail its much more tricky and your best sticking with mbox, if exim, NFI - dont use it :)
On 11/04/2013 02:29 AM, Noel Butler wrote:
On 04/11/2013 14:33, Regan Yelcich wrote:
On 2/11/2013, at 11:40 pm, Noel Butler <noel.butler@ausics.net> wrote:
On 02/11/2013 20:25, Regan Yelcich wrote:
Can someone advise the best way to convert mailboxes from Mbox to Maildir for Dovecot 2.17 on Ubuntu? Thanks.
mb2md.pl
The program has at least 2 bugs in it:
. If the body has paragraph break (i.e., '\n') followed by the RFC822 keyword 'From', the original message will loose the last half of the message and a phantom message will be created.
Change from my notes:
# if ( /^From /
# -to-
# if ( /^From .*? \d\d:\d\d:\d\d \d\d\d\d/. I never could figure out where the second bug was. This one created some messages with blank subject lines.
Another annoyance was prefixing the newly created mail folders with a '.'
Our site is small (300 users) with 15-20 years of e-mail history. A conservative estimate of the conversion was something like 30 million messages.
Did it work? Yes. I wish I had more time for testing, though. In the end, we fixed the problems on the back side.
I don't need to do anything specific for Dovecot? It'll see the new Maildir account and automatically create the indexes etc?
IIRC it shows you how to use it, you need to indicate where the new maildir is, if you have only a few, do them all manually, if you have many, write a quick bash or perl script to do them, and dovecot will create the indexes when they login.
You will need to tell dovecot to look for the new location though, if your using system users as I suspect you are, then maildir:~/Maildir should do it *but* dont forget to make sure your MTA knows to use maildir as well, I've not worked with system users for a decade, but I think in postfix home_mailbox = Maildir/ will do it, with sendmail its much more tricky and your best sticking with mbox, if exim, NFI - dont use it :)
--
Mark
--0-Mark Moore UCAR/NCAR/CGD mmoore@ucar.edu 1850 Table Mesa Drive (W) 303 497-1338 Boulder, CO 80305 (F) 303 497-1324
On 2013-11-04 10:16 AM, Mark Moore <mmoore@ucar.edu> wrote:
Another annoyance was prefixing the newly created mail folders with a '.'
this is standard for maildir...
http://www.courier-mta.org/maildir.html
--
Best regards,
*/Charles/*
On 05/11/2013 01:16, Mark Moore wrote:
mb2md.pl
The program has at least 2 bugs in it:
. If the body has paragraph break (i.e., '\n') followed by the RFC822 keyword 'From', the original message will loose the last half of the message and a phantom message will be created.
Change from my notes:
if ( /^From /
-to-
if ( /^From .*? \d\d:\d\d:\d\d \d\d\d\d/
. I never could figure out where the second bug was. This one created some messages with blank subject lines.
Never noticed this, but, it was a very very long time ago I last used it, and since most users over here have always been pop3, probably never had many to convert in the first place when I did use it, so risk was so low. I think it was around the time we merged, and had to combine sendmail/dovecot and qmail/vpopmail/dovecot systems into just a postfix/dovecot solution, either we fluked it, or any affecteds didnt bother to report it.
Another annoyance was prefixing the newly created mail folders with a '.'
As Charles has already mentioned, thats how Maildir works
On 11/04/2013 06:28 PM, Noel Butler wrote:
Another annoyance was prefixing the newly created mail folders with a '.'
As Charles has already mentioned, thats how Maildir works
I'd modify this to 'the way Maildir wants it'....not necessarily the way it _must_ work. I suspected the preference, but, again, didn't have enough time to investigate.
(Some) users requested that their folders be visible, which is what they were familiar with under mbox. BASH to the rescue and a short while later....POOF!
Also, Thunderbird is quite happy creating folders without the preceding '.', which seems to work without incident with the Maildir format.
...and in case it's not quite obvious by this point, the intricacies of e-mail storage are not my long suite. I bow to the great authorities. :-)
--
Mark
--0-Mark Moore UCAR/NCAR/CGD mmoore@ucar.edu 1850 Table Mesa Drive (W) 303 497-1338 Boulder, CO 80305 (F) 303 497-1324
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Tue, 5 Nov 2013, Noel Butler wrote:
On 05/11/2013 01:16, Mark Moore wrote:
mb2md.pl
The program has at least 2 bugs in it:
. If the body has paragraph break (i.e., '\n') followed by the RFC822 keyword 'From', the original message will loose the last half of the message and a phantom message will be created.
Change from my notes:
if ( /^From /
-to-
if ( /^From .*? \d\d:\d\d:\d\d \d\d\d\d/
. I never could figure out where the second bug was. This one created some messages with blank subject lines.
You have a badly formatted mbox file, if there is such distinction neccessary:
http://manpages.ubuntu.com/manpages/precise/man5/mbox.5.html
" In order to avoid misinterpretation of lines in message bodies which begin with the four characters "From", followed by a space character, the mail delivery agent must quote any occurrence of "From " at the start of a body line. "
also look at the example of the From-line with date, your date is not RFC822. However, the date-format varies. Therefore, the start token of a (next) message in mbox are the five characters "From " at the start of a line.
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux)
iQEVAwUBUniZ9l3r2wJMiz2NAQKN5wf+NkbKObZHgFX0AqPmFJFkY7eZb4KKPBMa LuRTbk+nm7DBwtNs0kbsh7WQJLxcEc+Kdt7O7Oe1+vnE3dKMsJDEvD7iIlItKVS/ 5pZW0a4LZKEdefh5RCXXZIJbKp4isvo3LKXMFEJhMY3evoEnQAo9P1if8B+weUz9 v4DxcoasDS53JgZFUBmb2ZCxpHFvthtQdTTyJA/8RvkEioUu+iLmS18HMx1pjCZ8 G/U7O28ru2Ver8nXn5L0k9Q0GKyzZTfKg4m/YiUXe98qOc299X5D1PfHFGpJafy5 AMVFXdb9XHtXa5D9re7Zu+LTIlHxg291UgUm6oLgRETyrg1av9sZ+Q== =cCgO -----END PGP SIGNATURE-----
At Tue, 5 Nov 2013 08:10:46 +0100 (CET), Steffen Kaiser wrote:
The program has at least 2 bugs in it:
. If the body has paragraph break (i.e., '\n') followed by the RFC822 keyword 'From', the original message will loose the last half of the message and a phantom message will be created.
Change from my notes:
if ( /^From /
-to-
if ( /^From .*? \d\d:\d\d:\d\d \d\d\d\d/
. I never could figure out where the second bug was. This one created some messages with blank subject lines.
You have a badly formatted mbox file, if there is such distinction neccessary:
No. There are some variants of mbox format. See https://en.wikipedia.org/wiki/Mbox#Family
http://manpages.ubuntu.com/manpages/precise/man5/mbox.5.html
" In order to avoid misinterpretation of lines in message bodies which begin with the four characters "From", followed by a space character, the mail delivery agent must quote any occurrence of "From " at the start of a body line. "
-- -- Name: SATOH Fumiyasu @ OSS Technology Corp. (fumiyas @ osstech co jp) -- Business Home: http://www.OSSTech.co.jp/ -- GitHub Home: https://GitHub.com/fumiyas/ -- PGP Fingerprint: BBE1 A1C9 525A 292E 6729 CDEC ADC2 9DCA 5E1C CBCA
On 05/11/2013 17:30, SATOH Fumiyasu wrote:
At Tue, 5 Nov 2013 08:10:46 +0100 (CET), Steffen Kaiser wrote:
The program has at least 2 bugs in it:
. If the body has paragraph break (i.e., '\n') followed by the RFC822 keyword 'From', the original message will loose the last half of the message and a phantom message will be created.
Change from my notes:
if ( /^From /
-to-
if ( /^From .*? \d\d:\d\d:\d\d \d\d\d\d/
. I never could figure out where the second bug was. This one created some messages with blank subject lines.
You have a badly formatted mbox file, if there is such distinction neccessary:
No. There are some variants of mbox format. See https://en.wikipedia.org/wiki/Mbox#Family
RFC 4155
o Each message in the mbox database MUST be immediately preceded by a single separator line, which MUST conform to the following syntax:
The exact character sequence of "From";
a single Space character (0x20);
the email address of the message sender (as obtained from the
message envelope or other authoritative source), conformant
with the "addr-spec" syntax from RFC 2822;http://manpages.ubuntu.com/manpages/precise/man5/mbox.5.html
" In order to avoid misinterpretation of lines in message bodies which begin with the four characters "From", followed by a space
character, the mail delivery agent must quote any occurrence of "From " at the start of a body line. "
On Mon, 2013-11-04 at 19:29 +1000, Noel Butler wrote: [...]
think in postfix home_mailbox = Maildir/ will do it, with sendmail its much more tricky and your best sticking with mbox, if exim, NFI - dont
Or - strategically - you use dovecot's LDA which should know where to throw the mails in.
For sendmail, read http://platyna.platinum.linux.pl/blog/?p=3:
Basically add somewhere in the middle of the /etc/mail/sendmail.mc
(where the other FEATURE() lines are)
---- snip ----
FEATURE(local_procmail, /usr/libexec/dovecot/deliver', deliver -d $u', SPhnu9')dnl MODIFY_MAILER_FLAGS(LOCAL', `-f')dnl
---- snip ----
and add the end, we do not need the procmail-mailer (as above uses/modifies the local mailer)
---- snip ----
dnl MAILER(procmail)dnl
---- snip ----
Caveat emptor: It's Fedora/RedHat-world over here and installation from RPMs. So check the paths;-)
Bernd-- Bernd Petrovitsch Email : bernd@petrovitsch.priv.at LUGA : http://www.luga.at
On 05/11/2013 19:44, Bernd Petrovitsch wrote:
On Mon, 2013-11-04 at 19:29 +1000, Noel Butler wrote: [...]
think in postfix home_mailbox = Maildir/ will do it, with sendmail its much more tricky and your best sticking with mbox, if exim, NFI - dont
Or - strategically - you use dovecot's LDA which should know where to throw the mails in.
but using system users, you wouldnt use dovecot's LDA :)
On 05/11/2013 11:04, Noel Butler wrote:
On 05/11/2013 19:44, Bernd Petrovitsch wrote:
On Mon, 2013-11-04 at 19:29 +1000, Noel Butler wrote: [...]
think in postfix home_mailbox = Maildir/ will do it, with sendmail its much more tricky and your best sticking with mbox, if exim, NFI - dont
Or - strategically - you use dovecot's LDA which should know where to throw the mails in.
but using system users, you wouldnt use dovecot's LDA :)
Why not?
Cheers, Daniele
On 05/11/2013 20:11, Daniele Nicolodi wrote:
On 05/11/2013 11:04, Noel Butler wrote:
but using system users, you wouldnt use dovecot's LDA :)
Why not?
pure overkill, your MTA already knows where it goes, it doesnt need to do any special lookups, would you use postfix virtual, to deliver local user? no, of course you wouldnt :)
On 05/11/2013 12:24, Noel Butler wrote:
On 05/11/2013 20:11, Daniele Nicolodi wrote:
On 05/11/2013 11:04, Noel Butler wrote:
but using system users, you wouldnt use dovecot's LDA :)
Why not?
pure overkill, your MTA already knows where it goes, it doesnt need to do any special lookups, would you use postfix virtual, to deliver local user? no, of course you wouldnt :)
Using dovecot-lda has nothing to do with postfix virtual users, it is the only way I know to use sieve filtering and have messages indexed at delivery.
Cheers, Daniele
On 05/11/2013 22:04, Daniele Nicolodi wrote:
On 05/11/2013 12:24, Noel Butler wrote:
On 05/11/2013 20:11, Daniele Nicolodi wrote:
On 05/11/2013 11:04, Noel Butler wrote:
but using system users, you wouldnt use dovecot's LDA :)
Why not?
pure overkill, your MTA already knows where it goes, it doesnt need to do any special lookups, would you use postfix virtual, to deliver local user? no, of course you wouldnt :)
Using dovecot-lda has nothing to do with postfix virtual users, it is the only way I know to use sieve filtering and have messages indexed at delivery.
Cheers, Daniele
again, overkill, system users means users have full access to system account and can write procmail rules, if you dont allow that access, then you dont trust them, so you should be using virtual users.
On 05/11/2013 23:39, Noel Butler wrote:
On 05/11/2013 22:04, Daniele Nicolodi wrote:
On 05/11/2013 12:24, Noel Butler wrote:
On 05/11/2013 20:11, Daniele Nicolodi wrote:
On 05/11/2013 11:04, Noel Butler wrote:
but using system users, you wouldnt use dovecot's LDA :)
Why not?
pure overkill, your MTA already knows where it goes, it doesnt need to do any special lookups, would you use postfix virtual, to deliver local user? no, of course you wouldnt :)
Using dovecot-lda has nothing to do with postfix virtual users, it is the only way I know to use sieve filtering and have messages indexed at delivery.
Cheers, Daniele
again, overkill, system users means users have full access to system account and can write procmail rules, if you dont allow that access, then you dont trust them, so you should be using virtual users.
You may enjoy writing procmail rules, most people does not :) Anyhow dovecot indexing at delivery time is the real advantage, and I really don't see any difference in calling dovecot-lda instead of procmail.
Cheers, Daniele
Noel Butler skrev den 2013-11-05 12:24:
On 05/11/2013 20:11, Daniele Nicolodi wrote:
On 05/11/2013 11:04, Noel Butler wrote:
pure overkill, your MTA already knows where it goes, it doesnt need to do any special lookups, would you use postfix virtual, to deliver local user? no, of course you wouldnt :)
one day postfix will as exim support sieve, just wait :)
On 06/11/2013 05:08, Benny Pedersen wrote:
Noel Butler skrev den 2013-11-05 12:24:
On 05/11/2013 20:11, Daniele Nicolodi wrote:
On 05/11/2013 11:04, Noel Butler wrote:
pure overkill, your MTA already knows where it goes, it doesnt need to do any special lookups, would you use postfix virtual, to deliver local user? no, of course you wouldnt :)
one day postfix will as exim support sieve, just wait :)
why would I wait, we use postifx and only in virtual users, very zippy, very resource nice, makes us very happy, but maybe you were meaning something esle, as I've just woken up so off to have some coffee, It might click later on :)
participants (9)
-
Benny Pedersen
-
Bernd Petrovitsch
-
Charles Marcus
-
Daniele Nicolodi
-
Mark Moore
-
Noel Butler
-
Regan Yelcich
-
SATOH Fumiyasu
-
Steffen Kaiser