[dovecot/core] ad6c57: lib: strfuncs - string match length and prefix che...

[GitHub]

  Branch: refs/heads/master-2.3
  Home:   https://github.com/dovecot/core
  Commit: ad6c57fafd2b92ee2f14bf9d167497b5ca428b52
      https://github.com/dovecot/core/commit/ad6c57fafd2b92ee2f14bf9d167497b5ca428b52
  Author: Phil Carmody <phil at dovecot.fi>
  Date:   2018-06-08 (Fri, 08 Jun 2018)

  Changed paths:
    M src/lib/strfuncs.c
    M src/lib/strfuncs.h
    M src/lib/test-strfuncs.c

  Log Message:
  -----------
  lib: strfuncs - string match length and prefix checking helpers

strncmp(input, "literal", 7) is an idiom used everywhere, but leaves
room for human error in calculating the length.

strncmp(input, "literal", strlen("literal")) is an idiom also used
everywhere, but is both verbose and might be inefficient on some
legacy or ultralightweight compilers.

The old techniques are presumed to be optimal code-wise, but are
verbose (and, containing redundancy, they leave room for human error),
so make the macro fall back onto this operation, simply avoiding the
redundancy/verbosity.

The macro expansion does not multiply evaluate any of its parameters,
so should be safe even in the strangest of situations.

Signed-off-by: Phil Carmody <phil at dovecot.fi>


  Commit: 9be765076fd7ac8593b7f759d1ec7dcf96654a2a
      https://github.com/dovecot/core/commit/9be765076fd7ac8593b7f759d1ec7dcf96654a2a
  Author: Phil Carmody <phil at dovecot.fi>
  Date:   2018-06-08 (Fri, 08 Jun 2018)

  Changed paths:
    M src/auth/auth-client-connection.c
    M src/auth/auth-master-connection.c
    M src/auth/auth-request.c
    M src/auth/auth-worker-server.c
    M src/auth/db-ldap.c
    M src/auth/db-oauth2.c
    M src/auth/main.c
    M src/auth/mech-oauth2.c
    M src/auth/mech-otp.c
    M src/auth/passdb-blocking.c
    M src/auth/passdb-bsdauth.c
    M src/auth/passdb-checkpassword.c
    M src/auth/passdb-lua.c
    M src/auth/passdb-pam.c
    M src/auth/passdb-vpopmail.c
    M src/auth/password-scheme-crypt.c
    M src/auth/password-scheme.c
    M src/auth/userdb-blocking.c
    M src/auth/userdb-checkpassword.c
    M src/auth/userdb-lua.c
    M src/auth/userdb-passwd-file.c
    M src/auth/userdb-vpopmail.c
    M src/config/config-connection.c
    M src/config/old-set-parser.c
    M src/config/sysinfo-get.c
    M src/director/login-connection.c
    M src/dns/dns-client.c
    M src/doveadm/client-connection-tcp.c
    M src/doveadm/doveadm-auth-server.c
    M src/doveadm/doveadm-auth.c
    M src/doveadm/doveadm-dsync.c
    M src/doveadm/doveadm-dump-dbox.c
    M src/doveadm/doveadm-mail-fetch.c
    M src/doveadm/doveadm-mail-mailbox-metadata.c
    M src/doveadm/doveadm-mail-server.c
    M src/doveadm/doveadm-oldstats.c
    M src/doveadm/doveadm-proxy.c
    M src/doveadm/doveadm-util.c
    M src/doveadm/doveadm-zlib.c
    M src/doveadm/server-connection.c
    M src/imap-hibernate/imap-client.c
    M src/imap-login/imap-proxy.c
    M src/imap-urlauth/imap-urlauth.c
    M src/imap/cmd-notify.c
    M src/imap/imap-fetch-body.c
    M src/imap/imap-sync.c
    M src/lib-auth/auth-client-request.c
    M src/lib-auth/auth-master.c
    M src/lib-auth/auth-server-connection.c
    M src/lib-dcrypt/dcrypt-openssl.c
    M src/lib-dict-backend/dict-sql-settings.c
    M src/lib-dict/dict-client.c
    M src/lib-dict/dict-memcached-ascii.c
    M src/lib-dict/dict-memcached.c
    M src/lib-dict/dict-redis.c
    M src/lib-dns/dns-lookup.c
    M src/lib-fs/fs-posix.c
    M src/lib-imap-storage/imap-msgpart.c
    M src/lib-imap/imap-base-subject.c
    M src/lib-index/mail-index.c
    M src/lib-mail/test-message-header-encode.c
    M src/lib-master/master-login-auth.c
    M src/lib-master/master-login.c
    M src/lib-master/master-service-settings-cache.c
    M src/lib-master/master-service-settings.c
    M src/lib-master/master-service.c
    M src/lib-program-client/program-client.c
    M src/lib-settings/settings-parser.c
    M src/lib-smtp/test-smtp-client-errors.c
    M src/lib-sql/driver-pgsql.c
    M src/lib-storage/index/imapc/imapc-mail-fetch.c
    M src/lib-storage/index/index-status.c
    M src/lib-storage/index/index-storage.c
    M src/lib-storage/index/maildir/maildir-sync-index.c
    M src/lib-storage/index/mbox/mbox-save.c
    M src/lib-storage/index/pop3c/pop3c-mail.c
    M src/lib-storage/index/pop3c/pop3c-storage.c
    M src/lib-storage/list/mailbox-list-fs-flags.c
    M src/lib-storage/list/mailbox-list-maildir-iter.c
    M src/lib-storage/mail-namespace.c
    M src/lib-storage/mail-search-register-imap.c
    M src/lib-storage/mail-storage-hooks.c
    M src/lib-storage/mail-storage-service.c
    M src/lib-storage/mail-storage-settings.c
    M src/lib-storage/mail-storage.c
    M src/lib-storage/mailbox-list.c
    M src/lib-storage/test-mail-storage.c
    M src/lib/iostream-rawlog.c
    M src/lib/module-dir.c
    M src/lib/test-str-sanitize.c
    M src/lib/unlink-directory.c
    M src/log/log-connection.c
    M src/login-common/client-common-auth.c
    M src/login-common/sasl-server.c
    M src/master/main.c
    M src/old-stats/client-export.c
    M src/old-stats/mail-session.c
    M src/plugins/acl/acl-backend-vfile.c
    M src/plugins/acl/acl-backend.c
    M src/plugins/fs-compress/fs-compress.c
    M src/plugins/fts-lucene/fts-lucene-plugin.c
    M src/plugins/fts-solr/fts-solr-plugin.c
    M src/plugins/fts-squat/fts-backend-squat.c
    M src/plugins/fts-squat/squat-test.c
    M src/plugins/fts/fts-build-mail.c
    M src/plugins/fts/fts-indexer.c
    M src/plugins/old-stats/mail-stats-fill.c
    M src/plugins/quota/quota-status.c
    M src/plugins/quota/quota-util.c
    M src/plugins/quota/quota.c
    M src/pop3-login/pop3-proxy.c
    M src/util/script.c

  Log Message:
  -----------
  global - migrate strncmp literals to str_begins

Simplify a bunch of strncmp(,,number) calls.

git ls-files \*.[ch] | xargs perl -p -i -e 's/strncmp\((.*?), ?(\".*?\"), ?(\d+)\) == 0/str_begins($1, $2)/g'
git ls-files \*.[ch] | xargs perl -p -i -e 's/strncmp\((.*?), ?(\".*?\"), ?(\d+)\) != 0/!str_begins($1, $2)/g'

I ran a longer script to verify that all of the string literals and
the length matched. They didn't:

$ git grep strncmp | perl -ne 'print if(m/strncmp\([^,]*,\s*"(.*?)",\s*(\d+)/ and ($s=$1,$t=$2,$s=~s/\\[tn]/#/g,length($s)) != $t)'
src/auth/db-oauth2.c:               if (strncmp(field, "oauth2:", 8) == 0 &&

With the new functions, that kind of typo is impossible.

Signed-off-by: Phil Carmody <phil at dovecot.fi>


  Commit: d026e2d2e010ae804a6984e0c28824e5824f9945
      https://github.com/dovecot/core/commit/d026e2d2e010ae804a6984e0c28824e5824f9945
  Author: Phil Carmody <phil at dovecot.fi>
  Date:   2018-06-08 (Fri, 08 Jun 2018)

  Changed paths:
    M src/doveadm/doveadm-dict.c
    M src/imap-hibernate/imap-client.c
    M src/imap-login/imap-proxy.c
    M src/lib-dict-backend/dict-cdb.c
    M src/lib-dict-extra/dict-fs.c
    M src/lib-dict/dict-memcached-ascii.c
    M src/lib-dict/dict-memcached.c
    M src/lib-dict/dict-redis.c
    M src/lib-dict/dict.c
    M src/lib-fs/fs-metawrap.c
    M src/lib-fs/fs-posix.c
    M src/lib-imap-storage/imap-metadata.c
    M src/lib-storage/index/dbox-multi/mdbox-purge.c
    M src/lib-storage/index/dbox-single/sdbox-sync-rebuild.c
    M src/lib-storage/index/index-search-mime.c
    M src/lib-storage/index/index-storage.c
    M src/lib-storage/index/mbox/mbox-storage.c
    M src/lib-storage/list/mailbox-list-delete.c
    M src/lib-storage/list/mailbox-list-fs.c
    M src/lib-storage/mail-storage-service.c
    M src/lib-storage/mailbox-attribute.c
    M src/lib-storage/mailbox-attribute.h
    M src/lib-storage/mailbox-list.c
    M src/lib/mempool-alloconly.c
    M src/lib/test-strfuncs.c
    M src/master/main.c
    M src/plugins/acl/acl-api.c
    M src/plugins/acl/acl-attributes.c
    M src/plugins/imap-acl/imap-acl-plugin.c

  Log Message:
  -----------
  global - migrate from strncmp to str_begins

Simplify a bunch of verbose strncmp(,,strlen()) calls.

--- 8< --- strncmp.cocci ---
@@
expression e1, e2;
@@

- strncmp(e1, e2, strlen(e2)) == 0
+ str_begins(e1, e2)

@@
expression e1, e2;
@@

- strncmp(e1, e2, strlen(e2)) != 0
+ !str_begins(e1, e2)

@@
expression e1, e2;
@@

- strncmp(e1, e2, strlen(e1)) == 0
+ str_begins(e2, e1)

@@
expression e1, e2;
@@

- strncmp(e1, e2, strlen(e1)) != 0
+ !str_begins(e2, e1)

--- 8< ---------------------

Signed-off-by: Phil Carmody <phil at dovecot.fi>


  Commit: 1367401f9fa34707050bd40f0154f11588c3d27e
      https://github.com/dovecot/core/commit/1367401f9fa34707050bd40f0154f11588c3d27e
  Author: Phil Carmody <phil at dovecot.fi>
  Date:   2018-06-08 (Fri, 08 Jun 2018)

  Changed paths:
    M src/auth/auth-cache.c
    M src/config/old-set-parser.c
    M src/doveadm/doveadm-cmd.c
    M src/doveadm/doveadm-mail.c
    M src/doveadm/dsync/dsync-brain-mailbox-tree.c
    M src/imap/cmd-notify.c
    M src/imap/cmd-rename.c
    M src/imap/imap-notify.c
    M src/lib-storage/index/imapc/imapc-list.c
    M src/lib-storage/index/imapc/imapc-storage.c
    M src/lib-storage/index/index-search-mime.c
    M src/lib-storage/index/shared/shared-list.c
    M src/lib-storage/list/mailbox-list-fs-iter.c
    M src/lib-storage/list/mailbox-list-index-backend.c
    M src/lib-storage/list/mailbox-list-iter.c
    M src/lib-storage/mailbox-list.c

  Log Message:
  -----------
  global - migrate more complicated strncmp expressions to str_begins

Simplify a bunch of verbose var=strlen(); strncmp(,,var) calls.
Fortunately, all of these examples use the length variable, so no
"unused value" warnings occur.

--- 8< --- strncmp2.cocci ---
@@
expression e1, e2;
identifier i2;
@@

  i2 = strlen(e2)
  ...
- strncmp(e1, e2, i2) == 0
+ str_begins(e1, e2)

@@
expression e1, e2;
identifier i2;
@@

  i2 = strlen(e2)
  ...
- strncmp(e1, e2, i2) != 0
+ !str_begins(e1, e2)

@@
expression e1, e2;
identifier i1;
@@

  i1 = strlen(e1)
  ...
- strncmp(e1, e2, i1) == 0
+ str_begins(e2, e1)

@@
expression e1, e2;
identifier i1;
@@

  i1 = strlen(e1)
  ...
- strncmp(e1, e2, i1) != 0
+ !str_begins(e2, e1)

--- 8< ---------------------

Signed-off-by: Phil Carmody <phil at dovecot.fi>


  Commit: 7cf910e89a8334cf939caa81e3b35511fc0e21ae
      https://github.com/dovecot/core/commit/7cf910e89a8334cf939caa81e3b35511fc0e21ae
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-08 (Fri, 08 Jun 2018)

  Changed paths:
    M src/lib-master/master-login-auth.c

  Log Message:
  -----------
  lib-master: Add how long request took for "Internal auth failure" errors


  Commit: f37d91677c5aa12cb648ace8f02c2dc1a8731720
      https://github.com/dovecot/core/commit/f37d91677c5aa12cb648ace8f02c2dc1a8731720
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-08 (Fri, 08 Jun 2018)

  Changed paths:
    M src/lib-master/master-auth.c

  Log Message:
  -----------
  lib-master: If connect() to backend UNIX socket is retried, log a warning

For example if imap-login process needs to retry before it successfully
connects to imap process's socket, a warning is logged. This warning is
important because it means that the imap-login process may have been
sleeping up to 0.5 seconds and causing all the other connections to hang
during it.

It would be better to make this retrying asynchronous, but before spending
time on doing that, lets see if this warning is ever even being logged.


  Commit: 54c4210100c22f12498fe187fe968a8251186da5
      https://github.com/dovecot/core/commit/54c4210100c22f12498fe187fe968a8251186da5
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-08 (Fri, 08 Jun 2018)

  Changed paths:
    M src/lib-master/master-login.c
    M src/lib-master/master-login.h

  Log Message:
  -----------
  lib-master: Keep linked list of master_login_clients per connection

This allows improving logging on connection errors.


  Commit: 082f6f24453542d93cf95b3c0895179e07895304
      https://github.com/dovecot/core/commit/082f6f24453542d93cf95b3c0895179e07895304
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-08 (Fri, 08 Jun 2018)

  Changed paths:
    M src/lib-master/master-login.c
    M src/lib-master/master-login.h

  Log Message:
  -----------
  lib-master: Improve error logging when master_login_connection gets disconnected


  Commit: e2250ad44c674212d1395dbac6fcf6e2566671f6
      https://github.com/dovecot/core/commit/e2250ad44c674212d1395dbac6fcf6e2566671f6
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-08 (Fri, 08 Jun 2018)

  Changed paths:
    M src/lib-master/master-login.c
    M src/lib-master/master-login.h

  Log Message:
  -----------
  lib-master: Improve error logging for post-login script failures


  Commit: 5da2d47ca42cd6402b90f9f83d55daafbfc195d4
      https://github.com/dovecot/core/commit/5da2d47ca42cd6402b90f9f83d55daafbfc195d4
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-08 (Fri, 08 Jun 2018)

  Changed paths:
    M src/lib-master/master-auth.c

  Log Message:
  -----------
  lib-master: Improve error logging for master_auth_connection failures


  Commit: 207dcaf6978b6d48ea12390d92e1efaa115f2ac6
      https://github.com/dovecot/core/commit/207dcaf6978b6d48ea12390d92e1efaa115f2ac6
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-08 (Fri, 08 Jun 2018)

  Changed paths:
    M src/lib-master/master-login-auth.c

  Log Message:
  -----------
  lib-master: Use more exact timestamp in master_login_auth_request failures


  Commit: 9d00608a6ea0c8773edf15e0518d04931bd36ff3
      https://github.com/dovecot/core/commit/9d00608a6ea0c8773edf15e0518d04931bd36ff3
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-08 (Fri, 08 Jun 2018)

  Changed paths:
    M src/lib-master/master-login-auth.c

  Log Message:
  -----------
  lib-master: Add request time also for internal failure error messages


  Commit: e8cddebde141663852b9ca5638dbdc965ea08ca1
      https://github.com/dovecot/core/commit/e8cddebde141663852b9ca5638dbdc965ea08ca1
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-08 (Fri, 08 Jun 2018)

  Changed paths:
    M src/lib-master/master-login-auth.c

  Log Message:
  -----------
  lib-master: Add auth connect & handshake times to master_login_auth_request errors


  Commit: 4c7f044d8ad8581a91d8b9c45eb2a938be280351
      https://github.com/dovecot/core/commit/4c7f044d8ad8581a91d8b9c45eb2a938be280351
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-08 (Fri, 08 Jun 2018)

  Changed paths:
    M src/auth/auth-master-connection.c
    M src/auth/auth-master-connection.h
    M src/auth/auth-request-handler.c

  Log Message:
  -----------
  auth: Improve auth-master connections' error logging

Include connect and handshake times in the error message.


  Commit: 0db446f1363a519b65754c5c277ae4482dec1124
      https://github.com/dovecot/core/commit/0db446f1363a519b65754c5c277ae4482dec1124
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-08 (Fri, 08 Jun 2018)

  Changed paths:
    M src/lib-master/master-login.c

  Log Message:
  -----------
  lib-master: postlogin: Don't unreference already closed login-connection

If the login-connection was already closed, this caused too many
unreferences.

Fixes:
Panic: file master-login.c: line 544 (master_login_conn_unref): assertion failed: (conn->clients == NULL)


Compare: https://github.com/dovecot/core/compare/d7b27d1c7643...0db446f1363a
      **NOTE:** This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/

      Functionality will be removed from GitHub.com on January 31st, 2019.