[Dovecot] [ rc28 ] dict{} seems to be ignored

newer
[Dovecot] mbox sync/lock issue,...

older
[Dovecot] Mailbox isn't a valid...

Emiliano Gabrielli (aka AlberT)

27 Mar 2007 27 Mar '07

3:59 p.m.

Hi all, I'm new to the list, so ... hello everybody :-)

Let's go to the point... I'm implementing quota over imap support in my mail server, so since a number of days I'm tring to upgrade to the new 1.x version (debian sarge) ...

After having a number of troubles using the debian backport packege I am now using the -rc28 package that I have built by myself...

I installed libmysqlclient14 and compiled dovecot with the following configure options:

   ./configure --with-ssl=openssl \
                --with-mysql \
                --disable-ipv6 \
                --with-cyrus-sasl2 \
                --with-gssapi \
                --with-deliver \
                --host=$(DEB_HOST_GNU_TYPE) \
                --build=$(DEB_BUILD_GNU_TYPE) \
                --prefix=/usr \
                --sysconfdir=/etc/dovecot \
                --libexecdir=\$${prefix}/lib \
                --localstatedir=/var \
                --mandir=\$${prefix}/share/man \
                --infodir=\$${prefix}/share/info \
                --with-moduledir=\$${prefix}/lib/dovecot/modules

as taken from the sarge-backports debian package and after having removed pgsql, ldap and ipv6 support, as I don't need them...

The imap over quota seem to work perfectly, since I can see quota info in IMAP clients (and via telnet) the only thing I can't make work is the dict: storage for quota info!!

It seems the dict{ .. } section in the dovecot.conf file is *totally* ignored:

dict { quotadict = mysq:/etc/dovecot/dovecot-dict-quota.conf }

As you can see I have a typo in the driver name, but no error nor error message is given ...

I followed the wiki instruction about Quota/Dict and I have created the necessary table ... that is not filled of course ..

Anybody can tell me if may be a compilation problem, a configuration one or a bug in the current dovecot release ??

Thanks in advance

<?php echo ' Emiliano Gabrielli (aka AlberT) ',"\n", ' GrUSP founder - ZCE ',"\n", ' AlberT_at_SuperAlberT_it - www.SuperAlberT.it ',"\n", ' IRC: #php,#AES azzurra.com ',"\n",'ICQ: 158591185'; ?>

Show replies by date

Timo Sirainen

27 Mar 27 Mar

10:21 p.m.

On 27.3.2007, at 16.59, Emiliano Gabrielli (aka AlberT) wrote:

...

It seems the dict{ .. } section in the dovecot.conf file is
*totally* ignored:

It is, unless you tell quota code to use it, like:

quota = dict:storage=10240:messages=1000 proxy::quotadict

If you still can't get it to work, set mail_debug=yes and see if
there's anything interesting in the logs.

Emiliano Gabrielli (aka AlberT)

28 Mar 28 Mar

10:38 a.m.

On Tuesday 27 March 2007 22:21, Timo Sirainen wrote:

...

It is, unless you tell quota code to use it, like:

quota = dict:storage=10240:messages=1000 proxy::quotadict

I have already done it ... in the plugin{ .. } section and it does nothing

...

If you still can't get it to work, set mail_debug=yes and see if there's anything interesting in the logs.

already done .. nothing and nothing ... I can see the login SQL queries .. I can see the quota fetched from the userdb_quota field ... but nothing about the quota update on the mysql storage ... dovecot updatets the maildir quota on the filesystem ...

any idea ?

thanks

-- <?php echo ' Emiliano Gabrielli (aka AlberT) ',"\n", ' GrUSP founder - ZCE ',"\n", ' AlberT_at_SuperAlberT_it - www.SuperAlberT.it ',"\n", ' IRC: #php,#AES azzurra.com ',"\n",'ICQ: 158591185'; ?>

Timo Sirainen

6:56 p.m.

On 28.3.2007, at 11.38, Emiliano Gabrielli (aka AlberT) wrote:

...

Show the whole dovecot -n output

Emiliano Gabrielli (aka AlberT)

29 Mar 29 Mar

7:15 p.m.

On Wednesday 28 March 2007 18:56, Timo Sirainen wrote:

...

it's attached ...

tomorrow i'll compile and try the rc29 ...

<?php echo ' Emiliano Gabrielli (aka AlberT) ',"\n", ' GrUSP founder - ZCE ',"\n", ' AlberT_at_SuperAlberT_it - www.SuperAlberT.it ',"\n", ' IRC: #php,#AES azzurra.com ',"\n",'ICQ: 158591185'; ?>

Emiliano Gabrielli (aka AlberT)

30 Mar 30 Mar

6:52 p.m.

On Thursday 29 March 2007 19:15, Emiliano Gabrielli (aka AlberT) wrote:

...

it's attached ...

tomorrow i'll compile and try the rc29 ...

just tried rc29 .. nothing changed for me ..

Any suggestion ?

-- <?php echo ' Emiliano Gabrielli (aka AlberT) ',"\n", ' GrUSP founder - ZCE ',"\n", ' AlberT_at_SuperAlberT_it - www.SuperAlberT.it ',"\n", ' IRC: #php,#AES azzurra.com ',"\n",'ICQ: 158591185'; ?>

Timo Sirainen

1 Apr 1 Apr

8:51 a.m.

On 28.3.2007, at 11.38, Emiliano Gabrielli (aka AlberT) wrote:

...

If it updates maildir quota, it sees quota=maildir setting somewhere.
So since it's not in dovecot.conf, apparently it gets that from
userdb's user_query?

Emiliano Gabrielli (aka AlberT)

2 Apr 2 Apr

10:35 a.m.

On Sunday 01 April 2007 08:51, Timo Sirainen wrote:

...

yes it gets it from the userdb_quota field ... and infact the quota are honoured for the users ... only the "quota storage" seems to be ignored...

the total quota a user has assigned to is retrieved from the "quota" field in the "mailbox" table (where user's info are stored, if he is active, and so on) ...

As saied in the wiki I'm tring to "store" the actual quota used by the user in *another* SQL table ("quota") ... but it seems to be ignored ...

-- <?php echo ' Emiliano Gabrielli (aka AlberT) ',"\n", ' GrUSP founder - ZCE ',"\n", ' AlberT_at_SuperAlberT_it - www.SuperAlberT.it ',"\n", ' IRC: #php,#AES azzurra.com ',"\n",'ICQ: 158591185'; ?>

Timo Sirainen

12:30 p.m.

On Mon, 2007-04-02 at 10:35 +0200, Emiliano Gabrielli (aka AlberT) wrote:

...

Sorry, but your reply is way too vague for me to even guess what the problem could be. Show me the exact settings you have. Show dovecot-sql.conf, your SQL tables and contents of one user's row. Maybe even set mail_debug=yes and show what the logs say. Then I can point exactly where the problem is without having to guess.

Emiliano Gabrielli (aka AlberT)

1:13 p.m.

On Monday 02 April 2007 12:30, Timo Sirainen wrote:

...

uh?.. but I have attached also the output of dovecot -n ... some msgs ago in the thread as you asked ..

ok, I'll try to give you more infos:

mail_debug=yes does *not* show any info about the "quota update" process, but shows the correct quota for the user given by the userdb_quota:

dovecot: Apr 02 13:01:11 Info: auth(default): client in: AUTH 1 PLAIN service=IMAP lip=10.0.0.101 rip=10.0.0.102 resp=<hidden> dovecot: Apr 02 13:01:11 Info: auth-worker(default): sql(_test_@XXXXXXX.it,10.0.0.102): query: SELECT username as user, password, '/var/vmail/domains/XXXXXXX.it/_test_' as userdb_home, 'maildir:/var/vmail/domains/XXXXXXX.it/_test_' as userdb_mail, 10000 AS userdb_uid, 10000 AS userdb_gid, CONCAT('maildir:storage=', CEIL(quota/1000)) AS userdb_quota FROM mailbox WHERE username = '_test_@XXXXXXX.it' dovecot: Apr 02 13:01:11 Info: auth(default): client out: OK 1 user=_test_@XXXXXXX.it dovecot: Apr 02 13:01:11 Info: auth(default): master in: REQUEST 17 17470 1 dovecot: Apr 02 13:01:11 Info: auth-worker(default): sql(_test_@XXXXXXX.it,10.0.0.102): SELECT 10000 AS uid, 10000 AS gid, '/var/vmail/domains/XXXXXXX.it/_test_' as home, 'maildir:/var/vmail/domains/XXXXXXX.it/_test_' as mail, CONCAT('maildir:storage=', CEIL(quota/1000)) AS quota FROM mailbox WHERE username = '_test_@XXXXXXX.it' AND active = '1' dovecot: Apr 02 13:01:11 Info: auth(default): master out: USER 17 _test_@XXXXXXX.it uid=10000 gid=10000 home=/var/vmail/domains/XXXXXXX.it/_test_ mail=maildir:/var/vmail/domains/XXXXXXX.it/_test_ quota=maildir:storage=10240 dovecot: Apr 02 13:01:11 Info: IMAP(_test_@XXXXXXX.it): Loading modules from directory: /usr/lib/dovecot/modules/imap dovecot: Apr 02 13:01:11 Info: IMAP(_test_@XXXXXXX.it): Module loaded: /usr/lib/dovecot/modules/imap/lib10_quota_plugin.so dovecot: Apr 02 13:01:11 Info: IMAP(_test_@XXXXXXX.it): Module loaded: /usr/lib/dovecot/modules/imap/lib11_imap_quota_plugin.so dovecot: Apr 02 13:01:11 Info: IMAP(_test_@XXXXXXX.it): Effective uid=10000, gid=10000, home=/var/vmail/domains/XXXXXXX.it/_test_ dovecot: Apr 02 13:01:11 Info: IMAP(_test_@XXXXXXX.it): maildir: data=/var/vmail/domains/XXXXXXX.it/_test_ dovecot: Apr 02 13:01:11 Info: IMAP(_test_@XXXXXXX.it): maildir: root=/var/vmail/domains/XXXXXXX.it/_test_, index=/var/vmail/domains/XXXXXXX.it/_test_, control=, inbox= dovecot: Apr 02 13:01:11 Info: imap-login: Login: user=<_test_@XXXXXXX.it>, method=PLAIN, rip=10.0.0.102, lip=10.0.0.101 dovecot: Apr 02 13:01:11 Info: auth(default): client in: AUTH 1 PLAIN service=IMAP lip=10.0.0.101 rip=10.0.0.102 resp=<hidden> dovecot: Apr 02 13:01:11 Info: auth-worker(default): sql(_test_@XXXXXXX.it,10.0.0.102): query: SELECT username as user, password, '/var/vmail/domains/XXXXXXX.it/_test_' as userdb_home, 'maildir:/var/vmail/domains/XXXXXXX.it/_test_' as userdb_mail, 10000 AS userdb_uid, 10000 AS userdb_gid, CONCAT('maildir:storage=', CEIL(quota/1000)) AS userdb_quota FROM mailbox WHERE username = '_test_@XXXXXXX.it' dovecot: Apr 02 13:01:11 Info: auth(default): client out: OK 1 user=_test_@XXXXXXX.it dovecot: Apr 02 13:01:11 Info: auth(default): master in: REQUEST 18 17512 1 dovecot: Apr 02 13:01:11 Info: auth-worker(default): sql(_test_@XXXXXXX.it,10.0.0.102): SELECT 10000 AS uid, 10000 AS gid, '/var/vmail/domains/XXXXXXX.it/_test_' as home, 'maildir:/var/vmail/domains/XXXXXXX.it/_test_' as mail, CONCAT('maildir:storage=', CEIL(quota/1000)) AS quota FROM mailbox WHERE username = '_test_@XXXXXXX.it' AND active = '1' dovecot: Apr 02 13:01:11 Info: auth(default): master out: USER 18 _test_@XXXXXXX.it uid=10000 gid=10000 home=/var/vmail/domains/XXXXXXX.it/_test_ mail=maildir:/var/vmail/domains/XXXXXXX.it/_test_ quota=maildir:storage=10240 dovecot: Apr 02 13:01:11 Info: imap-login: Login: user=<_test_@XXXXXXX.it>, method=PLAIN, rip=10.0.0.102, lip=10.0.0.101 dovecot: Apr 02 13:01:11 Info: IMAP(_test_@XXXXXXX.it): Disconnected dovecot: Apr 02 13:01:11 Info: IMAP(_test_@XXXXXXX.it): Loading modules from directory: /usr/lib/dovecot/modules/imap dovecot: Apr 02 13:01:11 Info: IMAP(_test_@XXXXXXX.it): Module loaded: /usr/lib/dovecot/modules/imap/lib10_quota_plugin.so dovecot: Apr 02 13:01:11 Info: IMAP(_test_@XXXXXXX.it): Module loaded: /usr/lib/dovecot/modules/imap/lib11_imap_quota_plugin.so dovecot: Apr 02 13:01:11 Info: IMAP(_test_@XXXXXXX.it): Effective uid=10000, gid=10000, home=/var/vmail/domains/XXXXXXX.it/_test_ dovecot: Apr 02 13:01:11 Info: IMAP(_test_@XXXXXXX.it): maildir: data=/var/vmail/domains/XXXXXXX.it/_test_ dovecot: Apr 02 13:01:11 Info: IMAP(_test_@XXXXXXX.it): maildir: root=/var/vmail/domains/XXXXXXX.it/_test_, index=/var/vmail/domains/XXXXXXX.it/_test_, control=, inbox= dovecot: Apr 02 13:01:11 Info: IMAP(_test_@XXXXXXX.it): Disconnected: Logged out

my conf is in the attached output of dovecont -n
the user's info are stored in the "mailbox" table: CREATE TABLE mailbox ( id bigint(20) unsigned NOT NULL auto_increment, username varchar(255) character set latin1 NOT NULL default '', password varchar(255) character set latin1 NOT NULL default '', name varchar(255) character set latin1 NOT NULL default '', maildir varchar(255) character set latin1 NOT NULL default '', quota int(10) NOT NULL default '10240000', domain varchar(255) character set latin1 NOT NULL default '', created datetime NOT NULL default '0000-00-00 00:00:00', modified datetime NOT NULL default '0000-00-00 00:00:00', active tinyint(1) NOT NULL default '1', bypass_virus_checks char(1) character set latin1 default 'N', bypass_spam_checks char(1) character set latin1 default 'N', PRIMARY KEY (username), UNIQUE KEY id (id)
ENGINE=MyISAM DEFAULT CHARSET=utf8 COMMENT='Postfix Admin - Virtual Mailboxes' AUTO_INCREMENT=46 ; INSERT INTO mailbox VALUES (3, '_test_@XXXXXXXXXX.it', '$1$89fgsfdg$1KVXXv8EBWMr8AVOVt7D0.', 'User Name', 'XXXXXXX.it/_test_/', 51200000, 'XXXXXXX.it', '2006-05-05 18:22:46', '2007-03-26 19:12:22', 1, 'N', 'N');
the table I'd like to use for quota's info storage is: CREATE TABLE quota ( username varchar(255) NOT NULL default '', path varchar(245) NOT NULL default '', current bigint(20) default NULL, PRIMARY KEY (username,path)
ENGINE=MyISAM DEFAULT CHARSET=latin1;

I hope now I gave enough infos :-))

thanks in advance, and sorry for previous messagesif they was poor of info ... really ;-(

<?php echo ' Emiliano Gabrielli (aka AlberT) ',"\n", ' GrUSP founder - ZCE ',"\n", ' AlberT_at_SuperAlberT_it - www.SuperAlberT.it ',"\n", ' IRC: #php,#AES azzurra.com ',"\n",'ICQ: 158591185'; ?>

Timo Sirainen

2:58 p.m.

On 2.4.2007, at 14.13, Emiliano Gabrielli (aka AlberT) wrote:

...

dovecot -n unfortunately doesn't show dovecot-sql.conf's contents.
Maybe in Dovecot v2.0 it'll do that. :)

...

CONCAT('maildir:storage=', CEIL(quota/1000)) AS userdb_quota FROM
mailbox WHERE username = '_test_@XXXXXXX.it'

There is the "maildir". It should say "dict" there instead if you
want it to use dict. So change it from user_query.

Emiliano Gabrielli (aka AlberT)

4:14 p.m.

On Monday 02 April 2007 14:58, Timo Sirainen wrote:

...

:-) sorry

...

dovecot: Apr 02 16:11:36 Error: IMAP(_test_@XXXXXXX.it): dict quota: URI missing from parameters: dict:storage=10240

maybe I have totally misundestood how this plugin works ... ;-(

this is the lines in my dovecot.cof: dict { quotadict = mysql:/etc/dovecot/dovecot-dict-quota.conf } plugin { quota = dict:storage=20480:messages=1000 proxy::quotadict }

dovecot-dict-quota.conf has only the mysql connection, table and field settings, no connection problem appears in the logs...

and here are the SQL queries in the dovecot-sql.conf:

user_query = SELECT 10000 AS uid, 10000 AS gid, '/var/vmail/domains/%d/%n' as home, 'maildir:/var/vmail/domains/%d/%n' as mail, CONCAT('maildir:storage=', CEIL(quota/1000)) AS quota FROM mailbox WHERE username = '%u' AND active = '1'

password_query = SELECT username as user, password, '/var/vmail/domains/%d/%n' as userdb_home, 'maildir:/var/vmail/domains/%d/%n' as userdb_mail, 10000 AS userdb_uid, 10000 AS userdb_gid, CONCAT('maildir:storage=', CEIL(quota/1000)) AS userdb_quota FROM mailbox WHERE username = '%u'

These are 1 per line, I wrapped it only for readability ...

thank again

-- <?php echo ' Emiliano Gabrielli (aka AlberT) ',"\n", ' GrUSP founder - ZCE ',"\n", ' AlberT_at_SuperAlberT_it - www.SuperAlberT.it ',"\n", ' IRC: #php,#AES azzurra.com ',"\n",'ICQ: 158591185'; ?>

Timo Sirainen

4:37 p.m.

On Mon, 2007-04-02 at 16:14 +0200, Emiliano Gabrielli (aka AlberT) wrote:

...

I think the misunderstandment is that plugin { quota } is completely ignored, because it's overridden by your user_query statement.

...

So here should be:

CONCAT('dict:storage=2', ceil(quota/1000), ' proxy::quotadict') AS quota

(or quota /1024?)

...

If you want to use the userdb_* here you should also enable prefetch userdb, otherwise they're ignored.

Emiliano Gabrielli (aka AlberT)

5:01 p.m.

On Monday 02 April 2007 16:37, Timo Sirainen wrote:

...

If you want to use the userdb_* here you should also enable prefetch userdb, otherwise they're ignored.

yes, I'm using the preftch :-) ..

I have a meeting now, so I'll try your changes ASAP, if it is the issue I suggest to make the wiki just a little more "misunderstanding people" aware :-PP

thank you so much for your help

<?php echo ' Emiliano Gabrielli (aka AlberT) ',"\n", ' GrUSP founder - ZCE ',"\n", ' AlberT_at_SuperAlberT_it - www.SuperAlberT.it ',"\n", ' IRC: #php,#AES azzurra.com ',"\n",'ICQ: 158591185'; ?>

Timo Sirainen

3 Apr 3 Apr

6:56 a.m.

BTW. I just added "Inaccuracy problems" to http://wiki.dovecot.org/Quota/Dict. Please read that before actually using dict quota.

Emiliano Gabrielli (aka AlberT)

1:20 p.m.

On Monday 02 April 2007 16:37, Timo Sirainen wrote:

...

So here should be:

CONCAT('dict:storage=2', ceil(quota/1000), ' proxy::quotadict') AS quota

ok, this is working some way (without the "2" in the storage= ... right?) ..

but I have still some things not clear... only some users have their quota updated on the DB ...

both pop3 and imap have the "quota" in their mail_plugin parameter, imap has imap_quota too ...

BTW, in the wiki the quota table suggested structure uses 2 varchar(255) fields, with a PK on ... but the maximum allowed lenght for an index il only 500 .. at least on my version of mysql ....

I suggest to correct in 255 for the user and 245 for the path :-)

-- <?php echo ' Emiliano Gabrielli (aka AlberT) ',"\n", ' GrUSP founder - ZCE ',"\n", ' AlberT_at_SuperAlberT_it - www.SuperAlberT.it ',"\n", ' IRC: #php,#AES azzurra.com ',"\n",'ICQ: 158591185'; ?>

Timo Sirainen

6 Apr 6 Apr

9:46 a.m.

On 3.4.2007, at 14.20, Emiliano Gabrielli (aka AlberT) wrote:

...

Right.

...

Well, I don't know about this. Do they never get it? Setting
mail_debug=yes should show in logs for those users if Dovecot sees
all the quota settings.

...

Changed path to 100. It's never more than 14 characters currently.

Emiliano Gabrielli (aka AlberT)

10:35 a.m.

On Friday 06 April 2007 09:46, Timo Sirainen wrote:

...

I decided not to use the DB storage for now, so I'll investigate better in the future :-) thank for the help

...

Ok, very well ... another issue with this is that if the table is utf-8 then the maximum index size is 1000, but every char takes 4 .. so even with 255 and 14 it should overflow this limit...

but user has to be 255 as it can be an email address (and, if I remember an email address can be 255 chars) ... maybe the quota table should never be utf-8 ... as I suppose, can you confirm ??

have a nice day

<?php echo ' Emiliano Gabrielli (aka AlberT) ',"\n", ' GrUSP founder - ZCE ',"\n", ' AlberT_at_SuperAlberT_it - www.SuperAlberT.it ',"\n", ' IRC: #php,#AES azzurra.com ',"\n",'ICQ: 158591185'; ?>

Timo Sirainen

10:59 a.m.

On Fri, 2007-04-06 at 10:35 +0200, Emiliano Gabrielli (aka AlberT) wrote:

...

It's actually larger. RFC 2821 says local-part can be 64 chars and domain 255 chars. Anyway they are your own usernames, so you can use whatever limits you want.

...

. maybe the quota table should never be utf-8 ... as I suppose, can you confirm ??

I guess it depends on if you want to have UTF-8 usernames. But I've really no idea about MySQL.

6619

Age (days ago)

6629

Last active (days ago)

List overview

18 comments

2 participants

participants (2)

Emiliano Gabrielli (aka AlberT)
Timo Sirainen