I wrote a small shell utility patch2testlist that might be useful for fellow MySQL
developers. It reads a diff and outputs the list of tests touched in this diff to run
in a format suitable for mysql-test-run.pl consumption. Furthermore, when provided
with a path to the source tree of the diff, it handles included files.
There are two ways to invoke it.
Quick-and-dirty mode that does not handle included files:
$ ./mtr `git diff | patch2testlist` ...
Thorough mode that considers included files, if the source tree path is given:
$ ./mtr `git diff | patch2testlist ../..` ...
What does it do? Let's consider an example:
$ git diff | diffstat mysql-test/extra/rpl_tests/rpl_replica_start_after_clone.inc | 2 mysql-test/include/keyring_tests/binlog/rpl_encryption_master_key_rotation_at_startup.inc | 5 - mysql-test/include/keyring_tests/mats/rpl_encryption.inc | 2 mysql-test/include/keyring_tests/mats/rpl_encryption_master_key_generation_recovery.inc | 2 mysql-test/suite/auth_sec/include/acl_tables_row_locking_test.inc | 4 mysql-test/suite/binlog/t/binlog_restart_server_with_exhausted_index_value.test | 1 mysql-test/suite/component_keyring_file/inc/rpl_setup_component.inc | 1 mysql-test/suite/innodb/t/log_8_0_11_case1.test | 1 mysql-test/suite/rocksdb/r/sys_tables.result | 2 mysql-test/suite/rocksdb/r/sys_tables_acl_tables_row_locking.result | 384 +++++++++++++++++++--------------------------------------------------------------- mysql-test/suite/rocksdb/r/sys_tables_is_statistics_mysql.result | 4 mysql-test/suite/rocksdb/r/sys_tables_mysqlcheck.result | 8 - mysql-test/suite/rpl/t/rpl_cloned_slave_relay_log_info.test | 4 mysql-test/suite/rpl/t/rpl_encryption.test | 3 mysql-test/suite/rpl/t/rpl_encryption_master_key_generation_recovery.test | 3 mysql-test/suite/rpl/t/rpl_encryption_master_key_rotation_at_startup.test | 5 - mysql-test/suite/rpl/t/rpl_gtid_innodb_sys_header.test | 2 mysql-test/suite/rpl_gtid/t/rpl_gtid_xa_commit_failure_before_gtid_externalization.test | 1 mysql-test/suite/rpl_gtid/t/rpl_gtid_xa_commit_one_phase_failure_before_prepare_in_engines.test | 1 mysql-test/suite/rpl_gtid/t/rpl_gtid_xa_prepare_failure_before_prepare_in_engines.test | 1 mysql-test/suite/rpl_gtid/t/rpl_gtid_xa_rollback_failure_before_gtid_externalization.test | 1 mysql-test/suite/rpl_nogtid/t/rpl_assign_gtids_to_anonymous_transactions_clone.test | 4 mysql-test/suite/rpl_nogtid/t/rpl_gtid_mode.test | 5 - mysql-test/suite/rpl_nogtid/t/rpl_nogtid_encryption_read.test | 3 mysql-test/suite/test_services/t/test_host_application_signal_plugin.test | 3 mysql-test/t/basedir.test | 5 - mysql-test/t/mysqld_daemon.test | 3 mysql-test/t/mysqld_safe.test | 27 ++--- mysql-test/t/restart_server.test | 3 mysql-test/t/restart_server_no_acl.test | 3 ... $ git diff | patch2testlist binlog.binlog_restart_server_with_exhausted_index_value innodb.log_8_0_11_case1 main.basedir main.mysqld_daemon main.mysqld_safe main.restart_server main.restart_server_no_acl rocksdb.sys_tables rocksdb.sys_tables_acl_tables_row_locking rocksdb.sys_tables_is_statistics_mysql rocksdb.sys_tables_mysqlcheck rpl.rpl_cloned_slave_relay_log_info rpl.rpl_encryption rpl.rpl_encryption_master_key_generation_recovery rpl.rpl_encryption_master_key_rotation_at_startup rpl.rpl_gtid_innodb_sys_header rpl_gtid.rpl_gtid_xa_commit_failure_before_gtid_externalization rpl_gtid.rpl_gtid_xa_commit_one_phase_failure_before_prepare_in_engines rpl_gtid.rpl_gtid_xa_prepare_failure_before_prepare_in_engines rpl_gtid.rpl_gtid_xa_rollback_failure_before_gtid_externalization rpl_nogtid.rpl_assign_gtids_to_anonymous_transactions_clone rpl_nogtid.rpl_gtid_mode rpl_nogtid.rpl_nogtid_encryption_read test_services.test_host_application_signal_plugin
The quick-and-dirty mode above does not require a hundred line script, a ten-line one
will do. But notice that several of the changed files in the diffstat output are
test include files (i.e. rpl_replica_start_after_clone.inc). Ideally we'd want to
run any tests that include (directly and indirectly) such files, and the ten-line
script does not handle this case.
That's what the other ninety lines of the script do. If the optional source tree path
argument is given, then it greps for any included files under mysql-test/, then
greps for newly-found files and so on until it finds no more:
$ git diff | patch2testlist ../.. auth_sec.acl_tables_row_locking binlog.binlog_restart_server_with_exhausted_index_value component_keyring_file.rpl_binlog_cache_encryption component_keyring_file.rpl_binlog_cache_temp_file_encryption component_keyring_file.rpl_default_table_encryption component_keyring_file.rpl_encryption component_keyring_file.rpl_encryption_master_key_generation_recovery component_keyring_file.rpl_encryption_master_key_rotation_at_startup innodb.log_8_0_11_case1 main.basedir main.mysqld_daemon main.mysqld_safe main.restart_server main.restart_server_no_acl rocksdb.sys_tables rocksdb.sys_tables_acl_tables_row_locking rocksdb.sys_tables_is_statistics_mysql rocksdb.sys_tables_mysqlcheck rpl.rpl_cloned_slave_relay_log_info rpl.rpl_encryption rpl.rpl_encryption_master_key_generation_recovery rpl.rpl_encryption_master_key_rotation_at_startup rpl.rpl_gtid_innodb_sys_header rpl.rpl_slave_start_after_clone rpl_gtid.rpl_gtid_only_start_replica_after_clone rpl_gtid.rpl_gtid_xa_commit_failure_before_gtid_externalization rpl_gtid.rpl_gtid_xa_commit_one_phase_failure_before_prepare_in_engines rpl_gtid.rpl_gtid_xa_prepare_failure_before_prepare_in_engines rpl_gtid.rpl_gtid_xa_rollback_failure_before_gtid_externalization rpl_nogtid.rpl_assign_gtids_to_anonymous_transactions_clone rpl_nogtid.rpl_gtid_mode rpl_nogtid.rpl_nogtid_encryption_read test_services.test_host_application_signal_plugin
As you can see the list is now significantly longer, indicating a more thorough test
run coverage of the diff. All this extra grepping takes about 90 seconds on my
machine, if some popular include files are touched. I have no idea whether that's
with hot or cold FS cache. I also don't know whether replacing grep with rg would
it make it faster.
To minimize the false positives in included file search, grep considers the lines
that don't start with the MTR language comment character #, and are like
...source...basename-of-included-file. This allows false positives in indented
comments and inside string literals (that one should be rare) and it cannot tell
apart files with the same name in different directories. In theory it also allows
false negatives if an include file is referenced using a string variable to store its
name. Any suggestions for better
regexps are welcome.
It goes without saying that it is best applied on test-only patches. If you touch the source code, then you should be looking at whole MTR runs, or, if possible, MTR runs of selected suites. But if you are indeed working on a test-only patch, this script reduces the required test time effectively.
Should be portable but currently tested on macOS only. Feedback is welcome!
No comments:
Post a Comment