On 24.11.2010, at 18.59, Brian Kroth wrote:

...

...

Hi, I'm running version 1.2.15 (so no doveadm) You could build Dovecot v2.0 and only use doveadm binary from it.

Does it just issue the command via IMAP? No direct filesystem operations?

It's all direct filesystem operations, no IMAP. But v1.2.15 can read v2.0's index files just fine.

...

Looks about ok. The main thing I'm worried is what happens if user creates mailboxes containing " or ' or ` characters.

Yeah, that was mostly me being lazy in my wanting to deal with escaping, so I just ignored them.

In what I originally wrote, I think it just won't touch them.

Or is the issue that the find command might remove them and then the indexes don't get fixed up? I suppose I could just make sure that the find ignores those dirs, but I thought (from other maillist reading) that the next time their client SELECTs the folder it'll fix it up anyways.

I was more thinking what happens if the user creates a mailbox called rm -rf / or something.. Also if there are " or \ characters I think the LIST output will use literals and your parsing will break more or less badly.

I suppose another spin on this would be for me to script the preauth imap client to figure out which mailboxes have messages marked for deletion of such and such an age and then try to use EXPUNGE to wipe just them out. I'm not sure off hand if that's possible.

That would be a bit difficult at least to do via IMAP..

Brian Kroth <bpkroth@gmail.com> 2010-11-24 13:28:

Timo Sirainen <tss@iki.fi> 2010-11-24 19:04:

...

On 24.11.2010, at 18.59, Brian Kroth wrote:

...

...

...

Hi, I'm running version 1.2.15 (so no doveadm) You could build Dovecot v2.0 and only use doveadm binary from it.

Does it just issue the command via IMAP? No direct filesystem operations?

It's all direct filesystem operations, no IMAP. But v1.2.15 can read v2.0's index files just fine.

...

...

Looks about ok. The main thing I'm worried is what happens if user creates mailboxes containing " or ' or ` characters.

Yeah, that was mostly me being lazy in my wanting to deal with escaping, so I just ignored them.

In what I originally wrote, I think it just won't touch them.

Or is the issue that the find command might remove them and then the indexes don't get fixed up? I suppose I could just make sure that the find ignores those dirs, but I thought (from other maillist reading) that the next time their client SELECTs the folder it'll fix it up anyways.

I was more thinking what happens if the user creates a mailbox called rm -rf / or something.. Also if there are " or \ characters I think the LIST output will use literals and your parsing will break more or less badly.

That's certainly true. I guess I was just hoping to skip over those mailboxes with unpleasant characters for the moment :}

More likely I'll rewrite this more carefully in Perl.

...

...

I suppose another spin on this would be for me to script the preauth imap client to figure out which mailboxes have messages marked for deletion of such and such an age and then try to use EXPUNGE to wipe just them out. I'm not sure off hand if that's possible.

That would be a bit difficult at least to do via IMAP..

So I'm finding. I guess I was thinking I could find the messages in a SELECTed mailbox via some parsing of either

• UID SEARCH DELETED X-SINCE $N_days_ago (where X-SINCE search X-SAVEDATE instead of INTERNALDATE), or
• UID FETCH 1:* (INTERNALDATE X-SAVEDATE FLAGS) as I've seen bantered about, or
• combine the two and SEARCH DELETED, then UID FETCH $initial_uid_list (X-SAVEDATE FLAGS) to refine the list.

Then use the (U?)IDs I get back from that to do

• UID EXPUNGE $uid_list

Of course I've only started researching that avenue, so maybe that's not so reasonable.

I'm starting to see why so much effort has been expended on this front.

Thanks, Brian

So, I redid this in Perl to only use IMAP rather than any sudo or find calls. In theory then one doesn't need to worry about the indexes being out of sync. I still skipped over the "strange characters" mailboxes for the moment. I'm wondering what you think of this second rendition?

The only thing I'm not quite sure about is if there's some sort of race between clients accessing/altering UIDs that may or may not get reused. But I think this one is at least clear of the problem you mentioned earlier.

In theory one would call it from cron like so:

30 0 * * * root /opt/cron/dovecot-maintenance.sh | logger -i -p mail.info -t dovecot-maintenance

Which loops over all the relevant users calling dovecot-maintenance.pl on them in turn. Could probably even fork off some small number to run in parallel.

Thanks, Brian

Timo Sirainen <tss@iki.fi> 2010-11-26 15:15:

On Fri, 2010-11-26 at 09:03 -0600, Brian Kroth wrote:

...

So, I redid this in Perl to only use IMAP rather than any sudo or find calls. In theory then one doesn't need to worry about the indexes being out of sync. I still skipped over the "strange characters" mailboxes for the moment. I'm wondering what you think of this second rendition?

Using some IMAP parser would be much more robust than doing it via regexps. Then you wouldn't have to worry about strange characters either.

*sigh*

Should have started there. It appears there's a nice module that even has an example for talking to Dovecot via preauth: http://search.cpan.org/~jettero/Net-IMAP-Simple-1.2018/Simple.pod#PREAUTH

Thanks again, Brian

...

if ($line =~ qr/^\* (LIST|LSUB) \((\\Has(No)?Children)?\) "\/" "(.+)"\s*$/) {

Just ignore the flags in the middle, there might be others: \([^)]*\)

...

if ($line =~ /^\* [0-9]+ FETCH \(UID ([0-9]+) X-SAVEDATE "([0-9]{1,2}-[A-Z][a-z][a-z]-[0-9]{4} [0-9][0-9]:[0-9][0-9]:[0-9][0-9] [0-9+-]+)" FLAGS \(([^)]+)\) ENVELOPE \((.*)\)\)\s*$/) {

ENVELOPE reply might not be in one line either. For example see what happens if subject has " character.

If the list of UIDs is really large, sending a command might fail because the command line is too long (imap_max_line_length setting).