March 2015 - dovecot - dovecot.org

Dovecot Full Text Search results in SolrException: undefined field text [SERIOUS]
by Kevin Laurie 05 Mar '15

05 Mar '15

Hello, My dovecot constantly runs into this error. I want to fix this one last time, I am tired of troubleshooting so please someone give me a lasting and proper solution for this error. I think its a problem with the dovecot-solr module. Please tell me how do I find the root of this problem with Dovecot. There is a problem with the body search text field. It always fails(with no result), other searches work(ie. search date, subject etc, ) The field-text I believe is missing. Please help. Desperate here! 2/25/2015, 11:32:30 PM ERROR SolrCore org.apache.solr.common. SolrException: undefined field text org.apache.solr.common.SolrException: undefined field text at org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1269) at org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:434) at org.apache.lucene.analysis.DelegatingAnalyzerWrapper$DelegatingReuseStrategy.getReusableComponents(DelegatingAnalyzerWrapper.java:74) at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:175) at org.apache.lucene.util.QueryBuilder.createFieldQuery(QueryBuilder.java:207) at org.apache.solr.parser.SolrQueryParserBase.newFieldQuery(SolrQueryParserBase.java:374) at org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:742) at org.apache.solr.parser.SolrQueryParserBase.handleBareTokenQuery(SolrQueryParserBase.java:541) at org.apache.solr.parser.QueryParser.Term(QueryParser.java:299) at org.apache.solr.parser.QueryParser.Clause(QueryParser.java:185) at org.apache.solr.parser.QueryParser.Query(QueryParser.java:107) at org.apache.solr.parser.QueryParser.TopLevelQuery(QueryParser.java:96) at org.apache.solr.parser.SolrQueryParserBase.parse(SolrQueryParserBase.java:151) at org.apache.solr.search.LuceneQParser.parse(LuceneQParser.java:50) at org.apache.solr.search.QParser.getQuery(QParser.java:141) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:148) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:197) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1967) at org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:64) at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1739) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)

3 3

Re: Re: IP drop list
by Joseph Tam 05 Mar '15

05 Mar '15

Nick Edwards writes: > I thought Timo once said dovecot had tarpitting, its useless if it is > there, and if it is, it needs user configurable timings, or maybe its > one of those things thats been in the gunna happen list > for a long time, like other stuff If I remember correctly, I think this was the "auth_failure_delay" feature. However, these delays are only inserted into the same session -- no IP tracking is done so a BFD attacking host can just keep opening up new connections. Dave McGuire writes: >>>> Please add this support to iptables instead of Dovecot. It's a >>>> waste of effort to code it into every application that listens on >>>> the network. >>> >>> <head explodes> >>> >>> Would you care to integrate it into IOS on my Cisco as well? >>> >>> There are things connected to the Internet that aren't PCs running >>> Linux, you know. It may be hard to accept, but that's the way it is. >>> >> I assume your dovecot runs on some kind of *nix > > Of course. I run it under Solaris. Oddly enough, if you run some versions of Solaris, it uses IPFilter as the native firewall, and it *does* have userland hooks so that you can make pass/block decisions based on userland executable. Not well documented though (see auth rules). You would need a firewall rule like auth in proto tcp from any to any port = 143 flags S/SA keep state then write a program that does ioctl(ipauth) calls to inspect and accept/reject packets; not for the faint of heart. However, you can use whatever weird and wonderful methods you want to determine IMAP/POP/SMTP network access policy, including DNSRBL or parsing a text file. This thread seems to be spinning into non-dovecot subjects, and I'm not helping, so I'll stop. Joseph Tam <jtam.home(a)gmail.com>

1 0

Re: IP drop list
by Joseph Tam 05 Mar '15

05 Mar '15

Dave McGuire writes: >> http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/AllowNets >> >> then setup fail2ban to manage extrafields > > Now that's a very interesting idea, thank you! I will investigate this. If you don't expect yor firewall to handle 45K+ IPs, I'm not how you expect dovecot will handle a comma separated string with 45K+ entries any better. If you want to turn your global backlist into a per-user whitelist, that would be perfectly doable though. Joseph Tam <jtam.home(a)gmail.com>

14 31

Rebuilding SIS attachment links from log
by Daniel Miller 04 Mar '15

04 Mar '15

This seems simple enough...I'm just not script wizard. If someone can throw together a starting point I can test and tweak it from there. It seems to me: 1. Read /var/mail/mail.err or specified logfile 2. For each "failed: read(/var/mail/attachments/aa/bb/attachmentHash-userHash" line, a. Confirm /var/mail/attachments/aa/bb/hashes/attachmentHash exists i. If attachmentHash is missing display such for possible backup searching. b. create link attachmentHash-userHash to hashes/attachmentHash 3. Continue to end of file Can this be done via "pure" BASH? Need sed/awk as well? -- Daniel

3 3

[OT] Extdata / Extprograms Plugins on CentOS 7?
by E.B. 04 Mar '15

04 Mar '15

Hello, I want to install Dovecot Pigeonhole and use the Extdata and Extprograms plugins on CentOS 7. I prefer to install software via yum, and a reasonably new version of Dovecot is available in the CentOS repo. But according to the dovecot documentation, these plugins need to be compiled, so I don't know if yum is out of the question? * Does anyone know if either of these plugins built into the Pigeonhole that is available in the CentOS 7 extras repo? OR if there is a version of Pigeonhole available in a different 3rd party repo with what I want? (like EPEL, DAG, etc) * Is it possible to use these plugins with Pigeonhole installed from yum? Does the compilation requirement mean that I have to remove not only Pigeonhole, but also Dovecot itself from my system and build everything by hand? :-( THANK YOU!!!!!!

1 0

New FREAK SSL Attack CVE-2015-0204
by Adrian Minta 04 Mar '15

04 Mar '15

Hello, about the CVE-2015-0204, in apache the following config seems to disable this vulnerability: SSLProtocol All -SSLv2 -SSLv3 SSLCipherSuite HIGH:MEDIUM:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4 Is something similar possible with dovecot ? If yes, what are the implications with old mail clients ? -- Best regards, Adrian Minta

2 4

Re: Rebuilding SIS attachment links from log
by Daniel Miller 04 Mar '15

04 Mar '15

Well, with no guarantees or promises whatsoever...here's my first attempt. I'm certain someone else can come up with a much more robust solution but it's a starting point. Since I typically see a batch of the error messages during an FTS update, my current usage is: 1. doveadm fts rescan -u user-to-fix 2. Perform a fts search to do a full mailbox scan 3. Check the mail log to see the date/time of the errors (I usually have a window open with tail -f) 4. Something like "grep 'Mar 3 20:17' mail.log > mail.err" gives me a starting point execute dovesisfix and see if it helps. -- Daniel On 3/3/2015 1:08 PM, Daniel Miller wrote: > This seems simple enough...I'm just not script wizard. If someone can > throw together a starting point I can test and tweak it from there. > It seems to me: > > 1. Read /var/mail/mail.err or specified logfile > 2. For each "failed: > read(/var/mail/attachments/aa/bb/attachmentHash-userHash" line, > a. Confirm /var/mail/attachments/aa/bb/hashes/attachmentHash exists > i. If attachmentHash is missing display such for possible > backup searching. > b. create link attachmentHash-userHash to hashes/attachmentHash > 3. Continue to end of file > > Can this be done via "pure" BASH? Need sed/awk as well? >

1 0

ot: maildir has overdrawn his diskspace quota
by voytek＠sbt.net.au 04 Mar '15

04 Mar '15

I have Postfix/Dovecot with virtual domains, same setup unaltered since server was set up quite a while ago last month, added a new virtual domain, 'just like before'. but, today noticed this in the queue/log 'overdrawn his diskspace quota'[1]: user's Maildir cur has like 48,762,696 bytes (lot less than other users) tried some searches, found similar probs, but, no solution as yet any ideas ? after physically removing some emails from this user's 'cur', I re-queued 'stuck' message, it was then accepted # quota -g vmail # quota -u vmail # [1] # grep 719855FC42 /var/log/maillog Mar 3 15:20:58 emu postfix/smtpd[18535]: 719855FC42: client=mail-pa0-f51.google.com[209.85.220.51] Mar 3 15:20:58 emu postfix/cleanup[18631]: 719855FC42: message-id=<54f536ad.21da440a.6572.3c07(a)mx.google.com> Mar 3 15:21:01 emu postfix/qmgr[1906]: 719855FC42: from=<dsome1(a)gmail.com>, size=5487190, nrcpt=1 (queue active) Mar 3 15:21:01 emu postfix/virtual[20248]: 719855FC42: to=<anna(a)tld.com>, relay=virtual, delay=3.5, delays=3.4/0/0/0.08, dsn=4.2.2, status=deferred (maildir delivery failed: Sorry, the user's maildir has overdrawn his diskspace quota, please try again later.) Mar 3 15:27:53 emu postfix/qmgr[1906]: 719855FC42: from=<dsome1(a)gmail.com>, size=5487190, nrcpt=1 (queue active) Mar 3 15:27:53 emu postfix/virtual[23352]: 719855FC42: to=<anna(a)tld.com>, relay=virtual, delay=415, delays=415/0.01/0/0.07, dsn=4.2.2, status=deferred (maildir delivery failed: Sorry, the user's maildir has overdrawn his diskspace quota, please try again later.)

1 0

Re: IP drop list
by Joseph Tam 03 Mar '15

03 Mar '15

Dave McGuire writes: >>>> then setup fail2ban to manage extrafields >>> >>> Now that's a very interesting idea, thank you! I will investigate this. >> >> If you don't expect yor firewall to handle 45K+ IPs, I'm not how you >> expect dovecot will handle a comma separated string with 45K+ entries >> any better. > > My firewall can handle that without breaking a sweat. I just haven't > found a way (that I'm comfortable with) to automatically inject rules > into it from a machine on the network. > > Doing it via a DNSBL is an elegant solution to the problem, IMO. I'm agnostic as far as which method you want to use. All I'm saying is that using dovecot's allow_net facility is as difficult, if not more so, than letting your firewall handle it. Joseph Tam <jtam.home(a)gmail.com>

4 3

Connect failed to database
by Dan LaSota 03 Mar '15

03 Mar '15

I have dovecot version 2.2.10 dovecot -n output below I am seeing connection errors being written to my dovecot error log: Mar 1 19:51:15 mail dovecot: auth-worker(2224): Error: mysql(localhost): Connect failed to database (servermail): Access denied for user 'usermail'@'localhost' (using password: YES) - waiting for 5 seconds before retry My connection script located at /etc/dovecot/dovecot-sql.conf.ext is like (password edited): driver = mysql connect = host=localhost dbname=servermail user='usermail' password='MY_SUPER_SECRET_PASSWORD' default_pass_scheme = SHA512-CRYPT password_query = SELECT email as user, password FROM virtual_users WHERE email='%u'; I have verified that I can gain access to the SQL database with # mysql -u usermail -p I can make select statements on the 'servermail' database and all of its tables. I've searched for similar errors from users, but most of the questions are unanswered, or answered incorrectly. I did follow one thread's suggest of setting the MYSQL password for the 'usermail' with OLD_PASSWORD instead of PASSWORD. I did that. I could still login from the shell using mysql -u usermail -p But dovecot still wrote the same error. (I did a flush privileges, and restarted mysql, and dovecot) I then set the password in SQL back using PASSWORD. (flush'd priveleges and restarte mysql and dovecot). Still Errors. Looking for leads. Thanks, Dan LaSota Instructional Designer, UAF eLearning (907) 451-4067 dan.lasota(a)alaska.edu http://elearning.uaf.edu dovecot -n # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-123.20.1.el7.x86_64 x86_64 CentOS Linux release 7.0.1406 (Core) xfs auth_debug = yes auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = plain mail_debug = yes mail_location = maildir:/var/mail/vhosts/%d/%n mail_privileged_group = mail mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocols = imap lmtp service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = dovecot } service imap-login { inet_listener imap { port = 0 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl = required ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n driver = static } verbose_ssl = yes

4 9