[dovecot/core] edc654: lib-fts: Fix fts_tokenizer_delete_trailing_partial...

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/dovecot/core
  Commit: edc654a35c3368dfb529ba784aee41dff6f45149
      https://github.com/dovecot/core/commit/edc654a35c3368dfb529ba784aee41dff6f45149
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-11-22 (Tue, 22 Nov 2016)

  Changed paths:
    M src/lib-fts/test-fts-tokenizer.c

  Log Message:
  -----------
  lib-fts: Fix fts_tokenizer_delete_trailing_partial_char() unit test


  Commit: b1b0b2b543dc1a10015272fc970ad7534f84e0c5
      https://github.com/dovecot/core/commit/b1b0b2b543dc1a10015272fc970ad7534f84e0c5
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-11-22 (Tue, 22 Nov 2016)

  Changed paths:
    M src/lib-fts/fts-tokenizer-address.c
    M src/lib-fts/test-fts-tokenizer.c

  Log Message:
  -----------
  lib-fts: Make sure address tokenizer can't return empty tokens.

This happened when address was a token that first looked like it could be
a valid address, but then got truncated due to reaching maxlen, followed
by truncating the UTF8-sequence and finally all the rest of the '-' or
'.' chars that were valid at the beginning of the address are stripped
away by fts_tokenizer_delete_trailing_invalid_char(), leaving nothing left.

Fixes:
Panic: file fts-tokenizer.c: line 206 (fts_tokenizer_next): assertion failed: (ret <= 0 || (*token_r)[0] != '\0')


Compare: https://github.com/dovecot/core/compare/9644b7914445...b1b0b2b543dc