Tuesday, July 09, 2024

A MySQL 9.0.0 branch with patches

Recently I made public a branch of MySQL 8.4.0 with patches. Now that newer upstream versions have been released, I have rebased the branch on 9.0.0: patched-mysql-9.0.0.

I did not add any new patches, and dropped the LLVM compilation fix patch. Thus the current patch set improves AddressSanitizer support and includes several fixes/improvements for the clone plugin to support a second transactional storage engine:

  1. Fix for Bug 115120: Provide memory debugging macro implementations for AddressSanitizer.
  2. Fix for Bug 109922: SEs do not coordinate clone rollbacks on instance startup.
  3. Fix for Bug 109920: SEs cannot add to clone data size estimates.
  4. Fix for Bug 109919: Clone plugin does not support arbitrary length O_DIRECT files.
  5. Fix for Bug 107715: Multiple SE clone not calling clone_end after mixed clone_begin failure/success.
  6. Fix for Bug 106750: Some clone tests needlessly marked as Linux-only.
  7. Fix for Bug 106737: Wrong SE called to ack clone application if >1 SE.
No comments:
Labels:

Thursday, July 04, 2024

Building and testing MySQL 8.0.38 / 8.4.1 / 9.0.0 on macOS

Oracle has just released MySQL 8.0.38/8.4.1/9.0.0, so let's see how the Valgrind testing of the previous set of releases is going: 

[ 65%] innodb.bug33788578_rec_IV_set            w2  [ fail ]  Found warnings/errors in error log file!

It only managed two thirds of a run (a --big-test one) in a few weeks, which means that I either need a faster machine, or that Valgrind testing does not take even remotely reasonable amount of time. Despite the run being incomplete, it resulted in couple of bug reports because every single test gives an error: #115229: pwrite(buf) points to uninitialised byte(s) in os_fusionio_get_sector_size(). I also noticed that one test always times out: #114998: Test rpl_gtid.rpl_gtid_parallel times out under Valgrind. I need to rethink Valgrind testing going forward, maybe by trying dropping --big-test first.

Back to the new releases. A nice surprise is that the they build with XCode 15.3 even though the associated bug (#113123: Compilation fails with LLVM 17 and XCode 15.3) is still open. However, an LLVM 18 build fails in the bundled zlib, and this time I refrained from reporting a bug. GCC 11–14 also fail, but they are officially unsupported for macOS, so I won't be reporting that, unless I encounter issues on Linux.

Fixed and no longer reproducing bugs:

New bugs:

Little to no changes

Finally, I did not retest #113113: Build failure with Homebrew LLVM 14-17 on macOS, where I keep using the workaround of setting CMAKE_AR to Homebrew ar.

To sum up, 5 bugs fixed or no longer reproducing, 7 new, 11 with no changes, 1 not tested, and 2 new Valgrind ones. I don't like this trend and I miss the middle-8.0 releases that were fully clean under macOS.

2 comments:
Labels:

Friday, June 14, 2024

Implementing a Table-Returning MySQL Function

Edit: links to Percona SEQUENCE_TABLE implementation added to the end

MySQL has many native functions and it's relatively straightforward to write new ones. While the official internals manual has a chapter on adding native functions, some aspects aren't immediately obvious, such as the possible return types for functions. The linked docs suggest that functions can return a double, an integer, or a string, but doesn't mention that they can also return JSON documents (i.e. JSON_ARRAY) and relational tables (i.e. JSON_TABLE). I have filed a bug for this omission, meanwhile this post will be my attempt at documenting the implementation of table-returning functions.

We will implement a table-returning function called LAST_INSERT_IDS, We will focus on the server infrastructure to support that, not on the function's purpose or the actual data it returns. Let's do the simplest possible thing: return a table with one column and one row.

Exploring the Codebase

Grepping the source tree for JSON_TABLE points to Table_function class declared in sql/table_function.h. Source code history then points to this commit, which in turn mentions Worklog #8867 (Add JSON table functions). Luckily for us it's a relatively old worklog, thus available publicly! From the worklog we can learn that any result tables will start as in-memory temporary tables which will spill to disk as needed. Good. The worklog also touches upon a lot of stuff elsewhere in the server so that the query optimizer will know how to deal with table-returning functions. We won't need to deal with that while implementing a new function, but it may provide debugging starters should anything go wrong.

Function class

To start writing code, we have to derive a new concrete class from Table_function and let the compiler tell us what methods must be implemented: 

class Table_function_last_insert_ids final : public Table_function {
 public:
  bool init() override;
  bool fill_result_table() override;
  const char *func_name() const override;
  bool print(const THD *thd, String *str,
             enum_query_type query_type) const override;
  bool walk(Item_processor processor, enum_walk walk, uchar *arg) override;

 private:
  List<Create_field> *get_field_list() override;
  bool do_init_args() override;
};

But let's not implement them just yet (stub them out with assert(0) bodies to learn what gets called when), and let's look into instantiating the class object first. The regular functions get registered with MySQL in func_array variable in sql/item_create.cc, and the surrounding code knows how to create their objects. But JSON_TABLE is not there! Grepping shows that it is wired directly to the parser, probably because its arguments are non-trivial. Our function is much simpler in that regard, can we get away with putting it there instead of patching parser? It turns out that no, we cannot, because the simpler functions derive from Item_func class, which derives from Item, which is what this infrastructure expects. And our table-returning function derives from Table_function, which has no further ancestors.

Patching the parser

OK, off to the parser we go. To find a starting point, let's check what JSON_TABLE does. Grepping for Table_function_json in the parser (the worklog is not too specific in this area), we get a match for PT_table_factor_function class in sql/parse_tree_nodes.cc. Then we grep for that class in the parser grammar and we get a hint why the table-returning functions are harder to implement than other kinds of functions: 

table_function:
        JSON_TABLE_SYM '(' expr ',' text_literal columns_clause ')'
        opt_table_alias
        {
          // Alias isn't optional, follow derived's behavior
          if ($8 == NULL_CSTR)
          {
            my_message(ER_TF_MUST_HAVE_ALIAS,
                       ER_THD(YYTHD, ER_TF_MUST_HAVE_ALIAS), MYF(0));
            MYSQL_YYABORT;
          }

          $$= NEW_PTN PT_table_factor_function($3, $5, $6, to_lex_string($8));
        }
      ;

We can see that the parser knows how to work with exactly one table-returning function that is named JSON_TABLE, thus we have to patch the parser. Let's keep everything under the non-terminal symbol table_function: 

table_function:
          json_table_function
        | last_insert_ids_function
        ;

last_insert_ids_function:
          LAST_INSERT_IDS_SYM '(' ')'
          opt_table_alias
          {
            // Alias isn't optional, follow derived's behavior
            if ($4 == NULL_CSTR)
            {
                my_message(ER_TF_MUST_HAVE_ALIAS,
                           ER_THD(YYTHD, ER_TF_MUST_HAVE_ALIAS), MYF(0));
                MYSQL_YYABORT;
            }

            $$= NEW_PTN PT_last_insert_ids_function(to_lex_string($4));
          }
        ;

json_table_function:
        // As before

Unfortunately the opt_table_alias block must be duplicated. But, no increase in the parser shift/reduce conflicts! There is more not particularly interesting stuff to do in the lexer and the parser:

  • Add the function name to the lexer, {SYM("LAST_INSERT_IDS", LAST_INSERT_IDS_SYM)}, to sql/lex.h.
  • Declare this lexer symbol as a token for the parser. I did not make it a keyword because JSON_TABLE wasn't one neither: %token LAST_INSERT_IDS_SYM 10024.
  • Finally, declare last_insert_ids_function symbol to be of table_reference type, same as table_function was before.

Parse Tree Node class

That's enough for the lexer and the parser, next we need to implement PT_last_insert_ids_function class: 

class PT_last_insert_ids_function : public PT_table_reference {
 public:
  PT_last_insert_ids_function(const LEX_STRING &table_alias)
      : m_table_alias{table_alias} {}

 private:
  const LEX_STRING m_table_alias;
};

Without a PT_last_insert_ids_function::contextualize method, which would actually create the Table_function_last_insert_ids object, the above implementation fails rather non-obviously: 

mysqltest: At line 24: Query 'SELECT * FROM LAST_INSERT_IDS() as ids' failed.
ERROR 1096 (HY000): No tables used

Let's add that method by copying, pasting, and deleting the non-applicable bits of PT_table_factor_function::contextualize. The job of this method will be to create the function object and to assign a name and a table for the query optimizer for the result. 

bool PT_last_insert_ids_function::contextualize(Parse_context *pc) {
  if (super::contextualize(pc)) return true;
  auto *const fn = new (pc->mem_root) Table_function_last_insert_ids{};
  if (unlikely(fn == nullptr)) return true;

  LEX_CSTRING alias;
  alias.length = strlen(fn->func_name());
  alias.str = sql_strmake(fn->func_name(), alias.length);
  if (unlikely(alias.str == nullptr)) return true;

  auto *const ti = new (pc->mem_root) Table_ident(alias, fn);
  if (ti == nullptr) return true;

  m_table_ref = pc->select->add_table_to_list(pc->thd, ti, m_table_alias.str, 0,
                                              TL_READ, MDL_SHARED_READ);
  if (m_table_ref == nullptr || pc->select->add_joined_table(m_table_ref))
    return true;

  return false;
}

Filling out the function class implementation

This implementation forces us to define the first method in Table_function_last_insert_ids, a very simple one: 

const char *func_name() const override { return "last_insert_ids"; }

Let's run it! 

SELECT * FROM LAST_INSERT_IDS() as ids;
...
Assertion failed: (0), function init, file table_function.h, line 440.

…which points to the stub Table_function_last_insert_ids::init(). Let's say we have no meaningful initialization at this point, so remove the assert from the stub: 

Assertion failed: (false), function get_field_list, file table_function.h, line 464.

This is get_field_list method, and we have to implement it to describe the schema of the table we will be returning: 

List<Create_field> *Table_function_last_insert_ids::get_field_list() {
  assert(fields.is_empty());

  auto *const field = new Create_field;
  field->init_for_tmp_table(MYSQL_TYPE_LONGLONG, MAX_BIGINT_WIDTH,
                            DECIMAL_NOT_SPECIFIED, /* is_nullable */ false,
                            /* is_unsigned */ true, 0, "insert_id");
  fields.push_back(field);

  return &fields;
}

This implementation assumes it is not a simple getter and thus is called exactly once. If the assumption will shown to be incorrect by the assert, I'll move the code to init and will reduce this one to a simple getter. Also, something will have to free the memory we are allocating for the field, that's for later.

Run again… 

Assertion failed: (false), function do_init_args, file table_function.h, line 467.

For JSON_TABLE, this method is documented to "check whether given default values can be saved to fields." Let's assume that we have no meaningful actions here, remove the assert, return success, run again: 

Assertion failed: (0), function fill_result_table, file table_function.h, line 442.

Finally we get to the second serious method that should provide the payload for the result table: 

bool Table_function_last_insert_ids::fill_result_table() {
  assert(!fields.is_empty());
  assert(!table->materialized);
  assert(table->s->fields == 1);

  empty_table();

  auto *const field = get_field(0);
  assert(field->field_index() == 0);
  field->store(current_thd->first_successful_insert_id_in_prev_stmt);
  field->set_notnull();
  write_row();

  return false;
}

And… 

SELECT * FROM LAST_INSERT_IDS() as ids;
insert_id
0

Success!

Implementing the remaining stubs

We still have two stub methods with asserts in them: print and walk. Let's try figuring out under what conditions they get called, again, by looking at JSON_TABLE implementation.

Let's start with print. It gets called by sql_lex.cc Table_ref::print. There are references to optimizer trace and something else around that. Let's see if that "something else" refers to the good old EXPLAIN: 

EXPLAIN SELECT * FROM LAST_INSERT_IDS() as ids;
Assertion failed: (false), function print, file table_function.h, line 450.
7   mysqld                              0x000000010385ffb8 Table_function_last_insert_ids::walk(bool (Item::*)(unsigned char*), enum_walk, unsigned char*) + 0

Interesting! We tried to find what calls print, but we found what calls walk instead. Since our function is atomic with regard to the query optimizer and has no internal structure, there is nothing to walk, let's remove the assert: 

EXPLAIN SELECT * FROM LAST_INSERT_IDS() as ids;
Assertion failed: (false), function print, file table_function.h, line 450.

Great, so the same EXPLAIN statement exercises both walk and print. Let's add a simple implementation for the latter: 

bool print(const THD *, String *str, enum_query_type) const override {
  return !str->append(STRING_WITH_LEN("last_insert_ids()"));
}

And now the EXPLAIN! 

EXPLAIN SELECT * FROM LAST_INSERT_IDS() as ids;
id	select_type	table	partitions	type	possible_keys	key	key_len	ref	rows	filtered	Extra
1	SIMPLE	ids	NULL	ALL	NULL	NULL	NULL	NULL	2	100.00	Table function: last_insert_ids; Using temporary
Warnings:
Note	1003	/* select#1 */ select `ids`.`insert_id` AS `insert_id` from last_insert_ids() `ids`

Memory management

One last thing is that Field object which was allocated in Table_function_last_insert_ids::get_field_list and never freed. Let's see if the standard tooling catches this error. AddressSanitizer under Linux includes LeakSanitizer, let's try it: 

==31384==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1440 byte(s) in 6 object(s) allocated from:
    #0 0xab9c0d1a99ac in operator new(unsigned long) (/home/laurynas/vilniusdb/last_insert_ids/_build-debug-llvm-14-san/runtime_output_directory/mysqld+0xb0599ac) (BuildId: 611e155cd8dc43455bd5d5e0ddbdfe53ae205fa4)
    #1 0xab9c0e6bb8bc in Table_function_last_insert_ids::get_field_list() /home/laurynas/vilniusdb/last_insert_ids/sql/table_function.cc:778:23
    #2 0xab9c0e6b5514 in Table_function::create_result_table(THD*, unsigned long long, char const*) /home/laurynas/vilniusdb/last_insert_ids/sql/table_function.cc:64:46
    #3 0xab9c0e0024fc in Table_ref::setup_table_function(THD*) /home/laurynas/vilniusdb/last_insert_ids/sql/sql_derived.cc:933:23
    #4 0xab9c0e388bfc in Query_block::resolve_placeholder_tables(THD*, bool) /home/laurynas/vilniusdb/last_insert_ids/sql/sql_resolver.cc:1310:15
    #5 0xab9c0e3854a4 in Query_block::prepare(THD*, mem_root_deque<Item*>*) /home/laurynas/vilniusdb/last_insert_ids/sql/sql_resolver.cc:247:7
    #6 0xab9c0e3e3da4 in Sql_cmd_select::prepare_inner(THD*) /home/laurynas/vilniusdb/last_insert_ids/sql/sql_select.cc:484:17
    #7 0xab9c0e3e2d74 in Sql_cmd_dml::prepare(THD*) /home/laurynas/vilniusdb/last_insert_ids/sql/sql_select.cc:399:11
    #8 0xab9c0e3e4360 in Sql_cmd_dml::execute(THD*) /home/laurynas/vilniusdb/last_insert_ids/sql/sql_select.cc:539:9
    #9 0xab9c0e263934 in mysql_execute_command(THD*, bool, unsigned long long*) /home/laurynas/vilniusdb/last_insert_ids/sql/sql_parse.cc:5489:29
    #10 0xab9c0e25c8c8 in dispatch_sql_command(THD*, Parser_state*, unsigned long long*) /home/laurynas/vilniusdb/last_insert_ids/sql/sql_parse.cc:6276:21
    #11 0xab9c0e253894 in dispatch_command(THD*, COM_DATA const*, enum_server_command) /home/laurynas/vilniusdb/last_insert_ids/sql/sql_parse.cc:2565:7
    #12 0xab9c0e259124 in do_command(THD*) /home/laurynas/vilniusdb/last_insert_ids/sql/sql_parse.cc:1745:18
    #13 0xab9c0e7f6f10 in handle_connection(void*) /home/laurynas/vilniusdb/last_insert_ids/sql/conn_handler/connection_handler_per_thread.cc:307:13
    #14 0xab9c12076ee8 in pfs_spawn_thread(void*) /home/laurynas/vilniusdb/last_insert_ids/storage/perfschema/pfs.cc:3022:3
    #15 0xfbb124cc5978 in start_thread nptl/./nptl/pthread_create.c:447:8
    #16 0xfbb124d2ba48 in thread_start misc/../sysdeps/unix/sysv/linux/aarch64/clone3.S:76

Yep, a leak. Let's try freeing that memory in the destructor: 

Table_function_last_insert_ids::~Table_function_last_insert_ids() {
  assert(fields.elements == 1);
  delete fields.head();
}

Run again, no leak or any other Sanitizer error.

Summary checklist

So, to summarize, the checklist for adding a new table-returning function to MySQL:

  • Add the function name as a lexer symbol in sql/lex.h
  • Add the function name symbol as a token in the parser (sql/sql_yacc.yy)
  • In the parser, rename table_function to json_table_function and add new table_function with json_table_function and new function alternatives. All these symbols must be typed as table_reference
  • Create a new parse tree node class, for example:
class PT_last_insert_ids_function : public PT_table_reference {
  using super = PT_table_reference;

 public:
  PT_last_insert_ids_function(const LEX_STRING &table_alias)
      : m_table_alias{table_alias} {}

  bool contextualize(Parse_context *pc) override;

 private:
  const LEX_STRING m_table_alias;
};

bool PT_last_insert_ids_function::contextualize(Parse_context *pc) {
  if (super::contextualize(pc)) return true;
  auto *const fn = new (pc->mem_root) Table_function_last_insert_ids{};
  if (unlikely(fn == nullptr)) return true;

  LEX_CSTRING alias;
  alias.length = strlen(fn->func_name());
  alias.str = sql_strmake(fn->func_name(), alias.length);
  if (unlikely(alias.str == nullptr)) return true;

  auto *const ti = new (pc->mem_root) Table_ident(alias, fn);
  if (ti == nullptr) return true;

  m_table_ref = pc->select->add_table_to_list(pc->thd, ti, m_table_alias.str, 0,
                                              TL_READ, MDL_SHARED_READ);
  if (m_table_ref == nullptr || pc->select->add_joined_table(m_table_ref))
    return true;

  return false;
}
  • Create a new table function class:
class Table_function_last_insert_ids final : public Table_function {
 public:
  ~Table_function_last_insert_ids() override;

  bool init() override { return false; }

  bool fill_result_table() override;

  const char *func_name() const override { return "last_insert_ids"; }

  bool print(const THD *, String *str, enum_query_type) const override {
    return !str->append(STRING_WITH_LEN("last_insert_ids()"));
  }

  bool walk(Item_processor, enum_walk, uchar *) override {
    return false;
  }

 private:
  List<Create_field> *get_field_list() override;

  bool do_init_args() override { return false; }

  List<Create_field> fields;
};

Table_function_last_insert_ids::~Table_function_last_insert_ids() {
  assert(fields.elements == 1);
  delete fields.head();
}

List<Create_field> *Table_function_last_insert_ids::get_field_list() {
  assert(fields.is_empty());

  auto *const field = new Create_field;
  field->init_for_tmp_table(MYSQL_TYPE_LONGLONG, MAX_BIGINT_WIDTH,
                            DECIMAL_NOT_SPECIFIED, /* is_nullable */ false,
                            /* is_unsigned */ true, 0, "insert_id");
  fields.push_back(field);

  return &fields;
}

bool Table_function_last_insert_ids::fill_result_table() {
  assert(!fields.is_empty());
  assert(!table->materialized);
  assert(table->s->fields == 1);

  empty_table();

  auto *const field = get_field(0);
  assert(field->field_index() == 0);
  field->store(current_thd->first_successful_insert_id_in_prev_stmt);
  field->set_notnull();
  write_row();

  return false;
}

Percona SEQUENCE_TABLE

After I have posted this, Percona's Yura Soroking pointed out on LinkedIn their implementation of SEQUENCE_TABLE function: blog post, the initial implementation source code. This implementation follows similar framework as the toy implementation above.
No comments:
Labels:

Tuesday, May 28, 2024

A MySQL 8.4.0 branch with patches

While writing the previous post, I noticed that I didn't have a central location for the patches that I submitted to Oracle. Some were in local branches, some were .patch files lying around. So now I pushed a tree that has all those patches applied in a single place: patched-mysql-8.4.0, and I even added a README. This tree hopefully will make it easier to rebase on future Oracle releases. I had maintained similar branches before while pushing some of the Percona patches to Oracle around early 8.0 times.

As for the patches themselves, the majority of them add various missing features for the clone plugin to be able to support more than one transactional storage engine. There is also a compilation fix and a slightly improved AddressSanitizer support patch.

All MySQL 8.4.0 users are advised to migrate ASAP! /s

No comments:
Labels:

Wednesday, May 08, 2024

Building and testing MySQL 8.0.37 & 8.4.0 on macOS

The first MySQL LTS release, 8.4.0, is out, together with 8.0.37! Which means it's time for me to build and test them in my main work environment, continuing the series (8.3.0/8.0.36, 8.2.0/8.0.35).

The first surprise is that both these releases do not build with the current XCode (15.3 at the time of writing), because the LLVM 17 compilation failure I previously reported (#113123: Compilation fails with LLVM 17 and XCode 15.3) is not fixed yet and started affected Apple toolchain too. I am using Homebrew-packaged LLVM 16 for all the builds of these versions. I didn't do any LLVM 18 nor GCC testing neither.

For the good news, I no longer get a build-breaking warning in NDB, even though its bug (#113662) is still open. Finally, for the build-related no-news, nothing has changed with regard to system vs bundled libraries: both versions continue to build with all the system libraries, except for zlib.

On to the testsuite. It seems that 8.4.0 release build testsuite takes some 20% less time than the 8.0.37 one. It must be the payoff from the deprecated feature removal removing their tests too, although I did not try to confirm that.

Fixed bugs and no longer reproducing test failures on open bugs:

New test failures:

I wanted to call the following the list of bugs with no changes, but I can't. While some of the bugs indeed have no changes, there is also a small but definite creep there: a test that only failed in one configuration before now fails in two. A second sibling test started failing with same symptoms. That's definitely not "no changes", but not new independent bugs neither. It's worrying.

I did not re-test two bugs:

Thus, the numbers are: 4 new bugs reported, 5 fixed or no longer reproducing, 10 bugs with no changes or with a bit of creep, 2 non-tested.

But wait, there's more! I have also sent some patches to Oracle by OCA! I am only counting the recent patches I developed at VilniusDB and not the ones I submitted at Percona. So how many have they applied since 8.3.0?

So, zero. Of course two out of four patches are for clone with a 2nd transactional storage engine, thus the excuse of Oracle MySQL not having such engine sounds plausible.

That's about it. I am still running Valgrind tests, and they should complete any week now, unless I have to kill them to take their machine for something else. I will not be updating this post or writing a new one unless their results are very, very unexpected. Otherwise here's to the new MySQL releases and hopefully I will continue the series in three months.

No comments:
Labels:

Tuesday, January 30, 2024

Introducing patch2testlist for MySQL development

I wrote a small shell utility patch2testlist that might be useful for fellow MySQL developers. It reads a diff and outputs the list of tests touched in this diff to run in a format suitable for mysql-test-run.pl consumption. Furthermore, when provided with a path to the source tree of the diff, it handles included files.

There are two ways to invoke it.

  1. Quick-and-dirty mode that does not handle included files: 

    $ ./mtr `git diff | patch2testlist` ...

  2. Thorough mode that considers included files, if the source tree path is given: 

    $ ./mtr `git diff | patch2testlist ../..` ...

What does it do? Let's consider an example: 

$ git diff | diffstat
 mysql-test/extra/rpl_tests/rpl_replica_start_after_clone.inc                                    |    2 
 mysql-test/include/keyring_tests/binlog/rpl_encryption_master_key_rotation_at_startup.inc       |    5 -
 mysql-test/include/keyring_tests/mats/rpl_encryption.inc                                        |    2 
 mysql-test/include/keyring_tests/mats/rpl_encryption_master_key_generation_recovery.inc         |    2 
 mysql-test/suite/auth_sec/include/acl_tables_row_locking_test.inc                               |    4 
 mysql-test/suite/binlog/t/binlog_restart_server_with_exhausted_index_value.test                 |    1 
 mysql-test/suite/component_keyring_file/inc/rpl_setup_component.inc                             |    1 
 mysql-test/suite/innodb/t/log_8_0_11_case1.test                                                 |    1 
 mysql-test/suite/rocksdb/r/sys_tables.result                                                    |    2 
 mysql-test/suite/rocksdb/r/sys_tables_acl_tables_row_locking.result                             |  384 +++++++++++++++++++---------------------------------------------------------------
 mysql-test/suite/rocksdb/r/sys_tables_is_statistics_mysql.result                                |    4 
 mysql-test/suite/rocksdb/r/sys_tables_mysqlcheck.result                                         |    8 -
 mysql-test/suite/rpl/t/rpl_cloned_slave_relay_log_info.test                                     |    4 
 mysql-test/suite/rpl/t/rpl_encryption.test                                                      |    3 
 mysql-test/suite/rpl/t/rpl_encryption_master_key_generation_recovery.test                       |    3 
 mysql-test/suite/rpl/t/rpl_encryption_master_key_rotation_at_startup.test                       |    5 -
 mysql-test/suite/rpl/t/rpl_gtid_innodb_sys_header.test                                          |    2 
 mysql-test/suite/rpl_gtid/t/rpl_gtid_xa_commit_failure_before_gtid_externalization.test         |    1 
 mysql-test/suite/rpl_gtid/t/rpl_gtid_xa_commit_one_phase_failure_before_prepare_in_engines.test |    1 
 mysql-test/suite/rpl_gtid/t/rpl_gtid_xa_prepare_failure_before_prepare_in_engines.test          |    1 
 mysql-test/suite/rpl_gtid/t/rpl_gtid_xa_rollback_failure_before_gtid_externalization.test       |    1 
 mysql-test/suite/rpl_nogtid/t/rpl_assign_gtids_to_anonymous_transactions_clone.test             |    4 
 mysql-test/suite/rpl_nogtid/t/rpl_gtid_mode.test                                                |    5 -
 mysql-test/suite/rpl_nogtid/t/rpl_nogtid_encryption_read.test                                   |    3 
 mysql-test/suite/test_services/t/test_host_application_signal_plugin.test                       |    3 
 mysql-test/t/basedir.test                                                                       |    5 -
 mysql-test/t/mysqld_daemon.test                                                                 |    3 
 mysql-test/t/mysqld_safe.test                                                                   |   27 ++---
 mysql-test/t/restart_server.test                                                                |    3 
 mysql-test/t/restart_server_no_acl.test                                                         |    3 
...
$ git diff | patch2testlist
binlog.binlog_restart_server_with_exhausted_index_value innodb.log_8_0_11_case1 main.basedir
main.mysqld_daemon main.mysqld_safe main.restart_server main.restart_server_no_acl
rocksdb.sys_tables rocksdb.sys_tables_acl_tables_row_locking
rocksdb.sys_tables_is_statistics_mysql rocksdb.sys_tables_mysqlcheck
rpl.rpl_cloned_slave_relay_log_info rpl.rpl_encryption
rpl.rpl_encryption_master_key_generation_recovery
rpl.rpl_encryption_master_key_rotation_at_startup rpl.rpl_gtid_innodb_sys_header
rpl_gtid.rpl_gtid_xa_commit_failure_before_gtid_externalization
rpl_gtid.rpl_gtid_xa_commit_one_phase_failure_before_prepare_in_engines
rpl_gtid.rpl_gtid_xa_prepare_failure_before_prepare_in_engines
rpl_gtid.rpl_gtid_xa_rollback_failure_before_gtid_externalization
rpl_nogtid.rpl_assign_gtids_to_anonymous_transactions_clone rpl_nogtid.rpl_gtid_mode
rpl_nogtid.rpl_nogtid_encryption_read test_services.test_host_application_signal_plugin

The quick-and-dirty mode above does not require a hundred line script, a ten-line one will do. But notice that several of the changed files in the diffstat output are test include files (i.e. rpl_replica_start_after_clone.inc). Ideally we'd want to run any tests that include (directly and indirectly) such files, and the ten-line script does not handle this case.

That's what the other ninety lines of the script do. If the optional source tree path argument is given, then it greps for any included files under mysql-test/, then greps for newly-found files and so on until it finds no more: 

$ git diff | patch2testlist ../..
auth_sec.acl_tables_row_locking binlog.binlog_restart_server_with_exhausted_index_value
component_keyring_file.rpl_binlog_cache_encryption
component_keyring_file.rpl_binlog_cache_temp_file_encryption
component_keyring_file.rpl_default_table_encryption component_keyring_file.rpl_encryption
component_keyring_file.rpl_encryption_master_key_generation_recovery
component_keyring_file.rpl_encryption_master_key_rotation_at_startup innodb.log_8_0_11_case1
main.basedir main.mysqld_daemon main.mysqld_safe main.restart_server main.restart_server_no_acl
rocksdb.sys_tables rocksdb.sys_tables_acl_tables_row_locking rocksdb.sys_tables_is_statistics_mysql
rocksdb.sys_tables_mysqlcheck rpl.rpl_cloned_slave_relay_log_info rpl.rpl_encryption
rpl.rpl_encryption_master_key_generation_recovery rpl.rpl_encryption_master_key_rotation_at_startup
rpl.rpl_gtid_innodb_sys_header rpl.rpl_slave_start_after_clone
rpl_gtid.rpl_gtid_only_start_replica_after_clone
rpl_gtid.rpl_gtid_xa_commit_failure_before_gtid_externalization rpl_gtid.rpl_gtid_xa_commit_one_phase_failure_before_prepare_in_engines
rpl_gtid.rpl_gtid_xa_prepare_failure_before_prepare_in_engines
rpl_gtid.rpl_gtid_xa_rollback_failure_before_gtid_externalization
rpl_nogtid.rpl_assign_gtids_to_anonymous_transactions_clone rpl_nogtid.rpl_gtid_mode
rpl_nogtid.rpl_nogtid_encryption_read test_services.test_host_application_signal_plugin

As you can see the list is now significantly longer, indicating a more thorough test run coverage of the diff. All this extra grepping takes about 90 seconds on my machine, if some popular include files are touched. I have no idea whether that's with hot or cold FS cache. I also don't know whether replacing grep with rg would it make it faster.

To minimize the false positives in included file search, grep considers the lines that don't start with the MTR language comment character #, and are like ...source...basename-of-included-file. This allows false positives in indented comments and inside string literals (that one should be rare) and it cannot tell apart files with the same name in different directories. In theory it also allows false negatives if an include file is referenced using a string variable to store its name. Any suggestions for better regexps are welcome.

It goes without saying that it is best applied on test-only patches. If you touch the source code, then you should be looking at whole MTR runs, or, if possible, MTR runs of selected suites. But if you are indeed working on a test-only patch, this script reduces the required test time effectively.

Should be portable but currently tested on macOS only. Feedback is welcome!

No comments:
Labels:

Wednesday, January 24, 2024

Building and testing MySQL 8.0.36 and 8.3.0 on macOS

The previous releases (8.0.35 and 8.2.0) resulted in me reporting fifteen bugs. Let's find out whether 8.0.36 and 8.3.0 will fare better on an M1 Mac.

Let's start with the build. Boost goes away as an external dependency in 8.3.0, removing the need to specify Boost-related CMake options, good. The server continues to build successfully with -DWITH_SYSTEM_LIBS=ON but now started requiring -DWITH_ZLIB=bundled, because 8.3.0 made the system libraries option govern zlib too, and the one in XCode is one patch level version too old. The Homebrew-installed version is ignored.

8.0.36 Release configuration builds with a single potentially-fatal warning: bug #113662 (NDB compilation error on macOS Release build). Finding this made me look, why is NDB built at all, if I did not add -DWITH_NDB=ON? This resulted in bug #113661 (NDB storage engine built ignoring -DWITH_NDB=OFF (which is OFF by default too)).

The most serious build-related issue I saw previously was incorrect query results if compiled with LLVM 15 and newer, reported as bug #113049 (MTR test json.array_index fails with a result difference) and bug #113046 (MTR tests for SELECT fail with ICP, MRR, possibly other flags). This issue has been fixed, although the bugs are still open (thus no release notes entries neither). As Tor Didriksen explained, they are open due to still remaining issues with recent MSVC compilers. But, LLVM works fine for me now and that's great.

The previous releases also required -ffp-contract=off compilation flag workaround to take care of some failures: bug #113047 (MTR test main.derived_limit fails with small cost differences), bug #113048 (MTR test gis.gis_bugs_crashes fails with a result difference). This has been mostly addressed, except that #113047 is fixed in 8.0.36 and 8.4.0 but not 8.3.0, so that failure still remains if the workaround is dropped.

The previous releases could not be compiled with LLVM 17, and no changes occurred here, bug #113123 (Compilation fails with LLVM 17) still applies.

Moving on to tests in Release, Debug, and Debug+ASan+UBsan configurations. Looking better than the last time, this is what I had to report:

So, to sum up, 10 bugs reported, 4 bugs confirmed fixed, 5 bugs (#113123, #113260, #113189, #113190, #113258) have no changes, and 1 bug (#113023) I did not test.

All in all, this looks OK. While no perfect clean testsuite results I was used to in some older releases, no miscompilation-like bugs neither, and that's fine.

No comments:
Labels:

Thursday, January 11, 2024

MySQL clone plugin internals and MyRocks clone design

I just realized that about MySQL and MyRocks clone I never actually published anything more serious than a single-emoji post on Facebook, a link to an Oracle umbrella bug, and this tweet.

So, let's talk about clone. MySQL has a clone plugin which can be used to copy new instances from existing ones, and it's also integrated into group replication for the same purpose. In Oracle releases this plugin copies only InnoDB tables, making the feature unsuitable for MyRocks instances.

Now MyRocks is the first storage engine, besides InnoDB, to get the clone support, and it works on mixed MyRocks/InnoDB instances while ensuring that the cloned instances are consistent across engines too. The code is in Meta's branch, I don't believe there are any user docs (but MyRocks support is so seamless that Oracle docs suffice! Only half-joking here), but there are IMHO extensive internals docs at Meta's wiki: MyRocks Clone Plugin. 

Their scope is broader than the title might suggest. Not only the MyRocks clone design is discussed, but there is also a clone background section, which discusses how clone works internally and fully applies to the Oracle branches too.

Like with all things 3rd party storage engines, it is rare to develop a feature without having to patch the server (or in this case server, clone, & InnoDB plugins). The details of this patch are also discussed in the Wiki, and also in the aforementioned Oracle umbrella bug.

Last but not least, with background and patches elsewhere out of the way, the Wiki has the design of MyRocks clone proper.

I hope the feature will reach MyRocks downstreams one day. Since MariaDB currently has no clone plugin, that leaves Percona Server. Maybe these docs will help with the porting, and also for advanced end-user troubleshooting. Clone away!

No comments:
Labels:

Thursday, December 14, 2023

MySQL 8.0.35 and 8.2.0 are out, here are my 15 compilation/test bug reports

I'm only a month and a half late to the party. That's, unfortunately, because I tried to build it and run its tests, on macOS, of all things. First the good news: it builds, and does so with the maximum set of 3rd party libraries possible.

Next I tried running the testsuite. I am used to clean test results in Oracle releases, under good conditions at least (not too heavy a load on the system, not too high a --parallel setting), with only occasional issues. This time I saw dozens of failures under debug, debug+sanitizers, release configurations, and tried to convert them to bug reports, best-effort.

First I identified a Homebrew-packaged Perl incompatibility with a test script: https://bugs.mysql.com/bug.php?id=113023.

Then I had a couple of test output differences where the difference was in floating point values: https://bugs.mysql.com/bug.php?id=113047 (MTR test main.derived_limit fails with small cost differences) and https://bugs.mysql.com/bug.php?id=113048 (MTR test gis.gis_bugs_crashes fails with a result difference). I am not a floating point programming expert, but somewhat luckily I remembered that there is a GCC option -ffp-contract=off, and that MySQL CMake script checks whether to add it. On a hunch that maybe the CMake test is incomplete (it is Linux-only and I was on macOS) I tried adding it as a workaround and it worked!

The next set of bugs was nastier. A bunch of query optimizer tests were failing with incorrect query results (https://bugs.mysql.com/bug.php?id=113046), and so did a JSON array test (https://bugs.mysql.com/bug.php?id=113049). To find the triggering conditions I tried different compilers, and, found that the tests pass if compiled with LLVM 14 and fail with LLVM 15, 16, 17, and XCode 15. I had no idea whether this is a compiler bug, MySQL undefined behavior, or something else, but Tor Didriksen posted on #113049 that "Recent versions of Clang have changed their implementation of std::sort(), and our own 'varlen_sort()' function returns wrong results.", one less mystery then.

Checking those different compiler versions was not trivial, because Homebrew-packaged LLVM 14 to 17 fail to build MySQL: https://bugs.mysql.com/bug.php?id=113113. Something about some incompatibility between system ar and LLVM ranlib utilities, with a workaround to use the ar coming from LLVM, i.e. -DCMAKE_AR=/opt/homebrew/opt/llvm@16/bin/llvm-ar. My build script is at 700 lines now, and that's already with some parts factored out.

On the top of the previous bug, LLVM 17, being new, had its regular and expected share of new warnings/errors: https://bugs.mysql.com/bug.php?id=113123.

Back from the build-with-different-compilers detour, there were still some test failures unaccounted: a debug assertion in group replication (https://bugs.mysql.com/bug.php?id=113257), all the TLS 1.3-using tests failing (https://bugs.mysql.com/bug.php?id=113258), spam in the replicating server error log (https://bugs.mysql.com/bug.php?id=113260).

At this point I stopped processing MTR tests, as I had already logged many bugs, and it became harder to avoid duplicates, so thought I could look at the unit tests. Here I'll just give a list of partial findings:

That's why it took me ~six weeks (and fifteen bug reports) to celebrate the new MySQL releases. That's halfway to the next expected release date on the quarterly schedule, and I hope I will be able to write a much shorter blog post much sooner after that release, as usual!


2 comments:
Labels:

Monday, October 23, 2023

Strong typing: comparing Rust newtype to C++

Rust source code often makes heavy use of the newtype idiom, where a new type is created for an underlying primitive type to differentiate it from other uses of the same underlying type. The term "newtype" comes from Haskell where it's not an idiom, but a keyword, thus a built-in language feature.

I was wondering why I never heard of newtype as a C++ developer, because C++ is obviously a strongly-typed language, where the same issue exists. There are different ways to approach it and this is my attempt to get the thoughts on the topic in order, there will be no earth-shattering insights.

Rust: newtype

Suppose you are developing a database and have transaction IDs and log sequence numbers. Both are u64 but are not interoperable in any way. So in Rust, a natural implementation would be to apply newtype idiom twice: 

pub struct TransactionId(u64);

impl TransactionId {
    fn new(id: u64) -> Self {
        Self(id)
    }

    fn get(&self) -> u64 {
      self.0
    }
    ...
}

impl fmt::Display for TransactionId { ... }

pub struct LogSequenceNumber(u64);

impl LogSequenceNumber { ... }
...

Let's enumerate the options in C++.

C++: do nothing

Do nothing, and use std::uint64_t for both types. No compiler protection, no documentation at the type name, thus the most bug-prone option. Obviously there is nothing to stop us from using this same option in Rust too.

C++: use type aliases

Introduce type aliases: 

// Can also be done with typedef, but let's stick to modern C++:
using transaction_id = std::uint64_t;
using log_sequence_number = std::uint64_t;

This expresses the intent, documents things whenever the type name appears, and is not too verbose. The downside is that it does not introduce new types, only aliases for existing ones, meaning that transaction IDs assign to LSNs and back freely.

C++: introduce new types

Introduce new types. Like in Rust, differently-named structs with identical fields can be used. 

struct transaction_id {
  std::uint64_t val;
}

struct log_sequence_id {
  std::uint64_t val;
}

Now type safety is increased and the type mix-up is prevented by the compiler. But so are most operations with type variables, requiring writing extra code to have the desired functionality, compared to the first two options. Writing this extra code will be more verbose than the same in Rust because the latter has support for traits, which can have default implementations. 

struct log_sequence_id {
  ...
  // explicit is important, we don't want to make the incompatible types
  // implicitly-covertable again inadvertently
  explicit log_sequence_id(std::uint64_t v) : val{v} {}

  log_sequence_id& operator += (std::size_t log_delta) {
    val += log_delta;
    return *this;
  }
  ...
}

Naturally, limiting available operations is advantageous too, in both languages. For example, it makes no sense to add two transaction IDs together.

Since this is C++, meaning that we have the template-hammer, making all the problems look like template-nails for better or worse, we could try avoiding spelling out structs every time: 

// Written this way only to show a point. The actual implementation would be more
// complex to be able to handle move-only types and wrap large objects efficiently.
template<typename T, typename Tag>
class newtype {
 public:
  explicit newtype(T v) : val{v} {}
  void set(T v) { val = v; }
  T get() const { return val; }
 private:
  T val;
};

struct log_sequence_id_tag{};
using log_sequence_id = newtype<std::uint64_t, log_sequence_id_tag>;

struct transaction_id_tag{};
using transaction_id = newtype<std::uint64_t, transaction_id_tag>;

Now introducing a newtype is reduced to two lines of code. Again, C++ developers do not usually discuss newtype but they do discuss strongly-typed using and typedefs, which is the same thing, called differently.

In most cases we are wrapping a single value of a primitive or string type. Those wrapped values are then operated using free functions or methods of some other classes. Thus, in this setting, this is a great option and we are done. But suppose we want to add some methods to the newly-introduced type instead of using free functions. The newtype template will not allow this, not unless we introduce inheritance: 

using log_sequence_id_base = newtype<std::uint64_t, log_sequence_id_tag>;

class log_sequence_id : public log_sequence_id_base {
   ...
};

At which point the use of the newtype template becomes questionable and the code simplifies by folding the value into the class: 

class log_sequence_id {
 public:
  ...
 private:
  std::uint64_t value;
};

Here we are back to creating a new type manually, just like before, without templates. This seems to be different from Rust, where a single-field struct will clearly show its newtype origins in the declaration, regardless of how much functionality it acquired later on.

So, there you have it. Both languages are strongly typed and have means to introduce new distinct types built on the existing ones, with Rust calling this newtype, and developers having a choice in C++ between type aliases, which don't actually increase type safety, to succinct templates and verbose types with some trade-offs.

No comments:
Labels: ,

Monday, September 25, 2023

Implementing durability in a MySQL storage engine

update 2023-09-28: edited for non-durable SE commits under group commit, and fixed the trx->flush_log_later discussion.

update 2023-09-27: Binlog group commits asks the storage engines to commit non-durably, will edit the post even more.

update 2023-09-26: trx->flush_log_later is actually used. Will edit the post.

Let's review how a MySQL storage engine should implement transaction durability by flushing / syncing WAL writes to disk. For performance reasons (group 2PC), let's also review when it specifically should not sync writes to disk. The reference durability implementation is, of course, InnoDB.

The main storage engine entry point is handlerton::commit. Since in general the storage engines participate in two-phase commit protocol with the binary log, there is also handlerton::prepare, and handlerton::flush_logs participates too. Let's ignore rollbacks, savepoints, explicit XA transactions, read only transactions, transactions on temporary tables only, crash recovery, and transaction coordinators other than the binlog.

Background: Group Commit

It was implemented (WL#5223) in its current form in MySQL 5.6, and its internals are described in this Mats Kindahl's blog post. I will not repeat everything here (and I'm sure I'd miss a lot of details), but for durability discussion, from the storage engine side, the group commit looks as follows:

  • prepare(t1) with reduced durability;
  • prepare(t2) with reduced durability;
  • …
  • prepare(tn) with reduced durability;
  • flush_logs(), making all the prepares above durable;
  • commit(t1) with reduced durability;
  • commit(t2) with reduced durability;
  • …
  • commit(tn) with reduced durability.

A surprise here is that the commits are performed with reduced durability too. How do reduced-durability commits implement full durability for the committed transactions, then? Turns out, the design of binlog group commit is only the commit of binlog itself is durable, and for the storage engines, prepares are made durable in batches and that's it. If their commits are lost, binlog crash recovery will roll forward the prepared transactions.

This design is counterintuitive if one thinks that innodb-flush-log-at-trx-commit=1, as documented, makes InnoDB commits durable in this setup, which it does not, and it is possible to see binlog crash recovery in action. Davi Arnaut reported this as bug #75519 in 2015, and IMHO few users are aware of this behavior.

Anyway, back to the implementation. Apparently the server developers did not want to change the prepare/commit handlerton interface, so the server durability request (full or reduced) is not passed in as an argument, but must be queried by thd_get_durability_property returning an enum with two possible values HA_REGULAR_DURABILITY and HA_IGNORE_DURABILITY.

Later, in 8.0, this durability property was reused to implement correct & performant commit order on multithreaded replicas, when binlog is disabled (WL#7846).

InnoDB: handlerton::commit

Implemented by innobase_commit.

Comes last in the group commit, but let's review it first. In other setups it might be the only entry point.

Wherever I say "write [to the disk] and sync|flush", the mental model is that of a buffered write with a separate flush/sync afterwards. If O_SYNC or O_DSYNC is used to write the log instead, then the write and the sync are a single operation.

Let's ignore non-default innobase_commit_concurrency setups.

First the code sets trx->flush_log_later and then goes through the call stack innobase_commit -> innobase_commit_low -> trx_commit_for_mysql -> trx_commit -> trx_commit_low. The last one calls trx_write_serialisation_history, which makes the necessary commit writes to a mini-transaction, then trx_commit_low commits the mini-transaction by creating the redo log records. Nothing is done for durability yet at this point. Finally trx_commit_low calls trx_commit_in_memory, which sees that trx->flush_log_later is set and sets trx->must_flush_log_later. (if trx_commit is called from other API than SE commit, then flush_log_later will not be set and the durability will be ensured in this function).

At this point the callstack returns all the way back to innobase_commit, which calls trx_complete_for_mysql, which now checks trx->must_flush_log_later (set), durability request (reduced), and whether this is a DDL transaction. If it is not, then nothing is done, and InnoDB reports the commit as successful. If it is a DDL transaction, then log is flushed ignoring the reduced durability request and innodb_flush_log_at_trx_commit setting..

The above mentioned that DDL transactions are flushed more than regular ones, regardless of innodb_flush_log_at_trx_commit setting. This is a deliberate design decision, which has to do with the data dictionary, I believe. To understand why, consider the relevant parts of server startup sequence:

  1. InnoDB comes up, and performs its own recovery from its redo log.
  2. Server data dictionary is initialized.
  3. Binlog crash recovery runs.
If any DD transactions are trapped in prepared state by the time of the data dictionary initialization, they will be invisible, while their disk changes (e.g. a tablespace renamed on disk) will be present on disk. This inconsistency is likely to be fatal for the DD, and binlog crash recovery runs too late to recover from that.

InnoDB: handlerton::prepare

Implemented by innobase_xa_prepare.

It calls trx_prepare_for_mysql -> trx_prepare -> trx_prepare_low, which updates the undo log state for the transaction a mini-transaction, committing which makes the top-level transaction prepared. Then trx_prepare calls trx_flush_logs, which will either do nothing or write and flush the redo log up to the mini-transaction's commit LSN, depending on the server durability request.

InnoDB: handlerton::flush_logs

Implemented by innobase_flush_logs.

It has a bool argument telling whether it was invoked as a part of binlog group commit, which is the interesting case here, ignoring the other option of it being invoked by FLUSH LOGS SQL statement. It writes the redo log buffer to disk and flushes it according it to innodb_flush_log_at_trx_commit value.

Bugs reported while writing this:

Bugs found while writting this:

Bugs that made me write this:

No comments:
Labels:

Friday, August 25, 2023

MySQL Build Times: Use Ninja

I had noticed Ninja as one of the possible CMake generators long time ago, but never paid attention to it, as I could not imagine it being better than Make so much that it'd be worth switching. Then, when I posted my MySQL -ftime-trace results, I got a comment on LinkedIn that Ninja visualizes build time nicely. I tried that, and it did, and I went back to Make builds.

A few months later, I am looking at Vittorio Romeo's "Improving Compilation Times" presentation slides (download them, do not read inline on GitHub; there is also the talk video itself), and the very first low-hanging fruit advice is "use Ninja".

OK, so let's actually try, say, Debug build on Facebook MySQL 8.0.28:

  • make -j13: 4m43s
  • ninja: 4m17s

A 10% improvement with roughly zero effort is nice. There are other niceties too: you don't have to figure out the right make parallelism argument for -j, as Ninja handles that automatically, and the terminal is not spammed with the build log of all the source files that have been built uneventfully. Only compiler warnings and any irregular build output is there.

To use it, add -G Ninja to CMake invocation, and then use ninja instead of make to build. I have patched my scripts.

No comments:
Labels: ,

Tuesday, May 09, 2023

Tips for making MySQL builds & tests faster

Recently, Mark discovered that a part of FB MySQL sources were recompiled three times in a single build. That has been fixed, and I also played with clang -ftime-trace to see where the build time goes. I believe there is more to this topic, so I wanted to organize my thoughts on the subject.

In software development, the shorter the change-build-test cycle is, the higher the developer productivity. Yet MySQL is not making it easy to have short "build" and "test" steps in this cycle. A reasonably powerful Intel Core i9 laptop used to take 20-30 minutes for a clean debug build without the unit tests. The MTR testsuite takes hours, and it is possible to make it run for days if you wish.

What can be done about this? Here's what's working for me, focusing on 8.0 trees. There is no silver bullet here, and some of the suggestions might be even somewhat effort-intensive to set up. Luckily, most suggestions are independent and optional.

Source trees and build artifacts

TL;DR: build incrementally as much as possible. Disk space is cheap, while your time is expensive. A common "antipattern" (quotes because there is nothing wrong with such workflow otherwise) is to have a single local git clone with a single build directory. Changing a branch with git checkout forces a clean build. Changing the build type (i.e. you built Debug previously, now need RelWithDebInfo, or Debug + AddressSanitizer enabled) forces a clean build. Don't force clean builds; keep all the previously-built artifacts around as much as possible:

  • Have one build dir for every build type (e.g. build/debug, build/release, build/debug-asan). Never delete a build dir unless forced to.
  • Never use git checkout in the local clone directory, use git worktrees for everything. Only delete a worktree (and its build dirs) once that branch is merged.
  • You are free to store the build dirs wherever you want, but in order to keep track which build dir belongs to which source tree, for me, the simplest option was to keep them below the source tree. Oracle MySQL has build/ in its .gitignore, so that's a good prefix dir for them.

One objection to incremental builds is that they might somehow result in differences in build artifacts compared to the same clean builds, and that would be bad. In practice, sometimes something does break an incremental build with a build error, forcing a clean rebuild–an occasional checkpoint if you will. I have never encountered a silent divergence that was somehow detrimental, and your CI/CD farm will build your PRs cleanly anyway.

Now if you follow the worktree advice, you are likely to have quite a few of them. Some of them will be your personal feature branches, while others will be shared feature branches, and main/master/8.0/5.7 trunk branches where others commit and push. Set up a cron job to pull the later and build overnight. Pros: ready builds for your work in the morning. Cons: you left your work last night with a working build, and someone pushed a commit that broke the build for you, which is what you find in the morning. IMHO the pros outweigh the cons.

Some MySQL source trees, like the Meta one, are incompatible with git worktrees. Luckily there is a not too-complicated workaround of adding the following CMake options: -DMYSQL_GITHASH=0 -DMYSQL_GITDATE=2100-02-29 -DROCKSDB_GITHASH=0 -DROCKSDB_GITDATE=2100-02-29. Maybe one day it will be fixed properly.

A thing that did not work well for me is ccache. While easy to set up, at least twice I had to waste a lot of time on apparent source-binary mismatch only to figure out that ccache is substituting an incorrect binary object file. That was enough for me to drop it, and I could never measure its benefit anyway.

Build options

According to the MySQL docs, there are over 160 CMake options. Some of them can affect the build times for better.

Use system libraries as much possible

You are in the business of developing MySQL, not its bundled 3rd party libraries. If you are lucky, you are also not in the business of developing MySQL integration with any of them. So, ignore the bundled libraries as much as possible and use your system ones: -DWITH_SYSTEM_LIBS=ON, after installing all the dependencies (which I won't list here). Unfortunately, that's only a theory, and in practice there's a difference between theory and practice. Let's take macOS, for example, to see which of the bundled still have to be used:

  • 8.0.33: -DWITH_RAPIDJSON=bundled
  • 8.0.32-29: system libs only, yay!
  • 8.0.28-27: -DWITH_RAPIDJSON=bundled -DWITH_LZ4=bundled -DWITH_FIDO=bundled
  • 8.0.26: -DWITH_RAPIDJSON=bundled -DWITH_LZ4=bundled

That's not too bad, and, given that we are stuck with every release for at least three months (much longer than that if using, say, the Meta tree), it's worth figuring out.

Skip the unit tests but be careful

-DWITH_UNIT_TESTS=OFF is by far the single most time-saving CMake option. Usually it is also not as bad as it may sound for development because the MTR tests are still there, and depending on what you are working on, the MTR tests might cover your testing needs completely. The biggest risk there is updating some internal API in a not particularly interesting way and forgetting to update its users in the unit tests. I am still figuring out the best way here to have my cake and eat it too.

Older versions: skip the functionality you don't need

This is quickly becoming an obsolete tip, but including it for completeness. Group replication used to be an optional build part, and the X plugin still is (-DWITH_MYSQLX=OFF). Unfortunately, disabling the X plugin breaks quite a few unrelated-to-X MTR tests, so I don't do that anymore.

Testing

Use libeatmydata

Install Stewart Smith's libeatmydata and always use it, except if building with sanitizers on Linux. It is packaged for Ubuntu, macOS Homebrew, and likely elsewhere. It cuts about 25% of MTR testsuite runtime by silently substituting all the fsync and related calls with no-ops for the tested processes. It is transparent, invisible, not getting in your way etc.–a pure win. Its invocation in the context of MTR tests is a handful to type, so I am using a shell script helper: 


UNAME_OUT="$(uname -s)"
if [ "$UNAME_OUT" = "Darwin" ]; then
    if [ "$(arch)" = "arm64" ]; then
        BREW="/opt/homebrew/opt"
    else
        BREW="/usr/local/opt"
    fi
    EMD_LIBDIR="$BREW/libeatmydata/lib"
    unset BREW
    export MTR_EMD=(
        "–mysqld-env=DYLD_LIBRARY_PATH=$EMD_LIBDIR"
        "–mysqld-env=DYLD_FORCE_FLAT_NAMESPACE=1"
        "–mysqld-env=DYLD_INSERT_LIBRARIES=$EMD_LIBDIR/libeatmydata.dylib")
    unset EMD_LIBDIR
else
    export MTR_EMD=(
        "–mysqld-env=LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libeatmydata.so")
fi

mtr_emd() {
    ./mtr "${MTR_EMD[@]}" "$@"
}

MTR also provides the --mem option which tries to use a non-persistent filesystem for running tests. In theory this should speed it up even more than libeatmydata, but I could never get it work reliably.

Use --parallel, find its best value for your machine & your tests.

For instance, for single-server tests on an Apple M1 Max (8 performance and 2 efficiency cores) the fastest I found was --parallel=15. Replication tests complicate this by spawning three servers per test instead of one.

Don't test what you don't need

Now we are deep in Captain Obvious territory, but still. Developing a plugin, say clone? Great, most of the time --suite=clone is enough. Sure, even with plugins the separation is not perfect, and there are dependencies, for example group replication uses clone too, but it's a start. You are less lucky if you work on InnoDB, or something in i.e. THD or Handler that is a dependency of everything.

Hardware

Throwing hardware at the build time problem is a great option if you have the resources. IMHO, there are two main routes to consider: the offline-portable one & the connected-powerful one.

If you need a laptop and want the option of working offline, then Apple Silicon is second to none. Replacing an Intel Core i9 laptop with a M1 Max one made MySQL builds five times faster. Five actual times! The downside is that macOS is not exactly a datacenter server OS, and so if you are the only one on the team on macOS, guess who just became the macOS port maintainer? There is an option of running Linux on Apple Silicon, which I heard virtualizes at near-bare metal speed, but I haven't explored it yet. Even then you become the ARM port maintainer, which is not that bad considering Graviton.

If a desktop is OK, then there are options, although I haven't tried this myself. Apple should still be in the running, and you could get an AMD CPU with up to 64 cores, which should make a short work of even MySQL build.

If a network is OK, then Sunny recommends using icecream to distribute compilation. I haven't tried this either.

Conclusion

I wrote down everything I knew about making MySQL build and test faster. Have I missed anything? Please comment.

No comments:
Labels:

Thursday, April 27, 2023

MySQL, clang -ftime-trace, & ClangBuildAnalyzer

TL;DR: install ClangBuildAnalyzer, compile MySQL with CC='clang -ftime-trace' CXX='clang++ -ftime-trace', run the analyzer, enjoy nice reports telling you why it's so slow.

Inspired by Mark's question on Twitter, I decided to try out clang -ftime-trace on MySQL source tree. This clang flag was added by Aras Pranckevičius (sure I'll call out a Lithuanian author by name) to LLVM 9.0, and it outputs per-source-file JSONs in Chrome Tracing format explaining where did the compilation time go. Now, instead of stating that for this file parsing took X seconds, template instantiation Y minutes, and register allocation Z microseconds it provides actionable information with object granularity, i.e. it tells you which exactly templates were expensive to instantiate etc.

To get those reports, -ftime-trace needs to be added to compilation flags. Don't add it to MySQL CMake CMAKE_CXX_FLAGS etc, because you'll have to get the rest of -O2 -DNDEBUG etc. right. Instead, pretend that this flag is a part of compiler invocation itself, and set CMAKE_CXX_COMPILER='clang++ -ftime-trace', and likewise for CMAKE_C_COMPILER. Or, use CC and CXX environment variables as I did in the TL;DR section.

A clean MySQL 8.0.33 results in a few thousands of those JSONs. Each one could be loaded in Chrome by navigating to chrome://tracing and loading a file. Individually. This blog post suggests zipping them all up, enabling Chrome to load that zip instead. But the post also warns that it takes a long time to load, and I can confirm Chrome crashing after some 40 minutes of work.

Luckily for us Aras wrote a project-level analyzer tool: ClangBuildAnalyzer. It handles those thousands of JSONs just fine, and very quickly, and for a regular 8.0.33 Release configuration build outputs the following: 


**** Time summary:
Compilation (6522 times):
  Parsing (frontend):         4669.4 s
  Codegen & opts (backend):   3139.1 s

**** Files that took longest to parse (compiler frontend):
 16499 ms: ./unittest/gunit/xplugin/xpl/CMakeFiles/xpl_test_src.dir/mock/mock.cc.o
 12057 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/session_execute_t.cc.o
 12011 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/session_connect_t.cc.o
 11907 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/session_options_t.cc.o
 11533 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/session_negotiation_t.cc.o
 11454 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/protocol_send_recv_t.cc.o
 11085 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/session_capability_t.cc.o
 10930 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/auth_chaining_t.cc.o
 10201 ms: ./unittest/gunit/xplugin/xpl/CMakeFiles/xpl_test_src.dir/timeouts_t.cc.o
 10195 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/session_general_t.cc.o

**** Files that took longest to codegen (compiler backend):
 48974 ms: ./unittest/gunit/innodb/CMakeFiles/merge_innodb_tests-t.dir/ut0new-t.cc.o
 31598 ms: ./unittest/gunit/xplugin/xpl/CMakeFiles/xpl_test_src.dir/mock/mock.cc.o
 30849 ms: ./sql/CMakeFiles/sql_gis.dir/gis/intersection_functor.cc.o
 30007 ms: ./sql/CMakeFiles/sql_gis.dir/gis/difference_functor.cc.o
 29611 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/auth_chaining_t.cc.o
 27047 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/session_connect_t.cc.o
 26776 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/session_negotiation_t.cc.o
 26253 ms: ./unittest/gunit/CMakeFiles/merge_large_tests-t.dir/hypergraph_optimizer-t.cc.o
 25877 ms: ./sql/CMakeFiles/sql_gis.dir/gis/symdifference_functor.cc.o
 25600 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/session_options_t.cc.o

**** Templates that took longest to instantiate:
109225 ms: std::__function::__func<(lambda at /Users/laurynas/vilniusdb/mysql-8... (7110 times, avg 15 ms)
107390 ms: std::__function::__func<(lambda at /Users/laurynas/vilniusdb/mysql-8... (7110 times, avg 15 ms)
 93389 ms: std::function<void ()>::function<(lambda at /Users/laurynas/vilniusd... (2370 times, avg 39 ms)
 92573 ms: std::__function::__value_func<void ()>::__value_func<(lambda at /Use... (2370 times, avg 39 ms)
 92539 ms: std::function<void ()>::function<(lambda at /Users/laurynas/vilniusd... (2370 times, avg 39 ms)
 91689 ms: std::__function::__value_func<void ()>::__value_func<(lambda at /Use... (2370 times, avg 38 ms)
 91551 ms: std::__function::__value_func<void ()>::__value_func<(lambda at /Use... (2370 times, avg 38 ms)
 90546 ms: std::__function::__value_func<void ()>::__value_func<(lambda at /Use... (2370 times, avg 38 ms)
 64823 ms: std::__function::__alloc_func<(lambda at /Users/laurynas/vilniusdb/m... (7110 times, avg 9 ms)
 63866 ms: std::__function::__alloc_func<(lambda at /Users/laurynas/vilniusdb/m... (7110 times, avg 8 ms)
 39867 ms: std::__function::__func<(lambda at /Users/laurynas/vilniusdb/mysql-8... (4740 times, avg 8 ms)
 39246 ms: std::__function::__func<(lambda at /Users/laurynas/vilniusdb/mysql-8... (4740 times, avg 8 ms)
 22360 ms: std::unique_ptr<std::unordered_multimap<const MDL_key *, MDL_ticket_... (1016 times, avg 22 ms)
 22285 ms: std::unique_ptr<std::unordered_multimap<const MDL_key *, MDL_ticket_... (1018 times, avg 21 ms)
 22138 ms: std::default_delete<std::unordered_multimap<const MDL_key *, MDL_tic... (1018 times, avg 21 ms)
 17698 ms: std::copy_n<const char16_t *, unsigned long, char16_t *> (3325 times, avg 5 ms)
 17666 ms: testing::internal::ValueArray<bool, bool>::operator ParamGenerator<b... (525 times, avg 33 ms)
 17432 ms: std::copy<const char16_t *, char16_t *> (3326 times, avg 5 ms)
 17359 ms: std::copy_n<const wchar_t *, unsigned long, wchar_t *> (3326 times, avg 5 ms)
 17057 ms: std::copy<const wchar_t *, wchar_t *> (3326 times, avg 5 ms)
 16950 ms: net::basic_waitable_timer<std::chrono::steady_clock>::cancel (150 times, avg 113 ms)
 16946 ms: net::io_context::cancel<net::basic_waitable_timer<std::chrono::stead... (150 times, avg 112 ms)
 16813 ms: std::copy_n<const char *, unsigned long, char *> (3314 times, avg 5 ms)
 16622 ms: std::copy_n<const char32_t *, unsigned long, char32_t *> (3326 times, avg 4 ms)
 16448 ms: std::__copy<const char16_t *, const char16_t *, char16_t *, 0> (3326 times, avg 4 ms)
 16373 ms: std::copy<const char *, char *> (3326 times, avg 4 ms)
 16342 ms: std::copy<const char32_t *, char32_t *> (3326 times, avg 4 ms)
 16110 ms: net::basic_waitable_timer<std::chrono::steady_clock>::~basic_waitabl... (139 times, avg 115 ms)
 16047 ms: std::__copy<const wchar_t *, const wchar_t *, wchar_t *, 0> (3326 times, avg 4 ms)
 15781 ms: std::unordered_multimap<const MDL_key *, MDL_ticket_store::MDL_ticke... (1018 times, avg 15 ms)

**** Template sets that took longest to instantiate:
333347 ms: std::function<$>::function<$> (8367 times, avg 39 ms)
330713 ms: std::__function::__value_func<$>::__value_func<$> (8367 times, avg 39 ms)
270430 ms: std::__function::__func<$>::__func (8367 times, avg 32 ms)
236822 ms: testing::internal::FunctionMocker<$>::Invoke (2370 times, avg 99 ms)
229859 ms: std::__function::__alloc_func<$>::__alloc_func (25098 times, avg 9 ms)
227397 ms: testing::internal::FunctionMocker<$>::InvokeWith (2370 times, avg 95 ms)
174340 ms: std::forward_as_tuple<$> (34235 times, avg 5 ms)
163483 ms: std::unique_ptr<$> (75303 times, avg 2 ms)
145187 ms: std::tuple<$> (45300 times, avg 3 ms)
141977 ms: std::__function::__func<$>::__clone (16733 times, avg 8 ms)
 98501 ms: std::__hash_table<$> (11200 times, avg 8 ms)
 94028 ms: std::allocator_traits<$> (31254 times, avg 3 ms)
 93971 ms: std::__compressed_pair<$>::__compressed_pair<$> (35840 times, avg 2 ms)
 91566 ms: std::unordered_map<$> (8606 times, avg 10 ms)
 90396 ms: std::decay<$> (20916 times, avg 4 ms)
 82788 ms: std::copy<$> (16492 times, avg 5 ms)
 80875 ms: std::map<$> (14763 times, avg 5 ms)
 77163 ms: std::__copy<$> (16493 times, avg 4 ms)
 73875 ms: std::vector<$>::push_back (6296 times, avg 11 ms)
 71983 ms: std::copy_n<$> (13989 times, avg 5 ms)
 71340 ms: stdx::expected<$> (9078 times, avg 7 ms)
 70331 ms: std::__tree<$> (19917 times, avg 3 ms)
 68211 ms: std::vector<$> (32556 times, avg 2 ms)
 67929 ms: std::vector<$>::__push_back_slow_path<$> (6078 times, avg 11 ms)
 62076 ms: std::vector<$>::__swap_out_circular_buffer (8254 times, avg 7 ms)
 57906 ms: std::__decay<$> (12966 times, avg 4 ms)
 54976 ms: std::__compressed_pair<$> (17033 times, avg 3 ms)
 53465 ms: std::unordered_map<$>::unordered_map (4697 times, avg 11 ms)
 53099 ms: testing::internal::MatcherBase<$>::MatcherBase<$> (5591 times, avg 9 ms)
 52731 ms: testing::internal::MatcherBase<$>::Init<$> (5591 times, avg 9 ms)

**** Functions that took longest to compile:
  2560 ms: MYSQLparse(THD*, Parse_tree_root**) (/Users/laurynas/vilniusdb/mysql-8.0.33/_build-release-time-report/sql/sql_yacc.cc)
  1602 ms: _GLOBAL__sub_I_gis_union_t.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/gis_union-t.cc)
  1577 ms: _GLOBAL__sub_I_gis_difference_t.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/gis_difference-t.cc)
  1396 ms: _GLOBAL__sub_I_gis_intersection_t.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/gis_intersection-t.cc)
  1280 ms: _GLOBAL__sub_I_gis_symdifference_t.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/gis_symdifference-t.cc)
   877 ms: __cxx_global_var_init.334 (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/router/tests/test_keyring_frontend.cc)
   839 ms: json_binary_unittest::JsonBinaryTest_BasicTest_Test::TestBody() (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/json_binary-t.cc)
   823 ms: _GLOBAL__sub_I_sys_vars.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/sys_vars.cc)
   821 ms: __cxx_global_var_init.143 (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/router/tests/test_keyring_frontend.cc)
   809 ms: ShareConnectionTinyPoolOneServerTest_not_sharable_Test::TestBody() (/Users/laurynas/vilniusdb/mysql-8.0.33/router/tests/integration/test_routing_sharing_constrained_pools.cc)
   732 ms: testing::internal::FlatTupleBase<testing::internal::FlatTuple<Target... (/Users/laurynas/vilniusdb/mysql-8.0.33/router/tests/component/test_bootstrap_clusterset.cc)
   719 ms: AccountReuseCreateComboTestP::gen_testcases() (/Users/laurynas/vilniusdb/mysql-8.0.33/router/tests/component/test_bootstrap_account.cc)
   689 ms: KeyringFrontendTest_ensure_Test::TestBody() (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/router/tests/test_keyring_frontend.cc)
   624 ms: duk__js_execute_bytecode_inner (/Users/laurynas/vilniusdb/mysql-8.0.33/extra/duktape/duktape-2.7.0/src/duktape.c)
   609 ms: _GLOBAL__sub_I_test_classic_protocol_message.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/mysql_protocol/tests/test_classic_protocol_message.cc)
   591 ms: MySQLRouter::prepare_command_options() (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/router/src/router_app.cc)
   579 ms: json_dom_unittest::JsonDomTest_BasicTest_Test::TestBody() (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/json_dom-t.cc)
   578 ms: SetObjectMembers(std::__1::unique_ptr<Json_object, std::__1::default... (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/join_optimizer/explain_access_path.cc)
   538 ms: operations_unittest::KeyringCommonOperations_test_OperationsTestWith... (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/components/keyring_common/operations-t.cc)
   531 ms: strnxfrm_unittest::StrmxfrmHashTest_HashStability_Test::TestBody() (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/strings_strnxfrm-t.cc)
   511 ms: spec_adder(rapidjson::GenericDocument<rapidjson::UTF8<char>, rapidjs... (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/rest_routing/src/rest_routing_plugin.cc)
   499 ms: mysys_my_time::MysysMyTime_StrToDatetime_Test::TestBody() (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/mysys_my_time-t.cc)
   476 ms: my_strnncoll_uca_900(CHARSET_INFO const*, unsigned char const*, unsi... (/Users/laurynas/vilniusdb/mysql-8.0.33/strings/ctype-uca.cc)
   467 ms: dd_properties_unittest::PropertiesTest_ValidSetGetIntBool_Test::Test... (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/dd_properties-t.cc)
   450 ms: testing::internal::FlatTupleBase<testing::internal::FlatTuple<Target... (/Users/laurynas/vilniusdb/mysql-8.0.33/router/tests/component/test_bootstrap_clusterset.cc)
   444 ms: KeyringManager_init_with_key_file_Test::TestBody() (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/harness/tests/test_keyring_manager.cc)
   427 ms: _GLOBAL__sub_I_hypergraph_optimizer_t.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/hypergraph_optimizer-t.cc)
   425 ms: _GLOBAL__sub_I_admin_cmd_arguments_object_t.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/xplugin/xpl/admin_cmd_arguments_object_t.cc)
   416 ms: __cxx_global_var_init.49 (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/http/tests/test_passwd.cc)
   378 ms: _GLOBAL__sub_I_expr_generator_parametric_t.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/xplugin/xpl/expr_generator_parametric_t.cc)

**** Function sets that took longest to compile / optimize:
 39400 ms: testing::internal::FunctionMocker<$>::InvokeWith(std::__1::tuple<$>&&) (1961 times, avg 20 ms)
 21051 ms: testing::internal::TypedExpectation<$>::ExplainMatchResultTo(std::__... (1993 times, avg 10 ms)
 19210 ms: testing::internal::ParameterizedTestSuiteInfo<$>::RegisterTests() (406 times, avg 47 ms)
 18661 ms: void testing::internal::TuplePrefix<$>::ExplainMatchFailuresTo<$>(st... (2627 times, avg 7 ms)
 17837 ms: testing::internal::TypeParameterizedTest<$>::Register(char const*, t... (1286 times, avg 13 ms)
 14176 ms: testing::internal::TypedExpectation<$>::GetCurrentAction(testing::in... (1993 times, avg 7 ms)
 12635 ms: bool boost::geometry::detail::partition::partition_two_ranges<$>::ap... (562 times, avg 22 ms)
 11895 ms: testing::internal::FunctionMocker<$>::PerformDefaultAction(std::__1:... (1961 times, avg 6 ms)
  6589 ms: testing::internal::FunctionMocker<$>::PerformAction(void const*, std... (1961 times, avg 3 ms)
  5890 ms: testing::internal::FunctionMocker<$>::DescribeDefaultActionTo(std::_... (1993 times, avg 2 ms)
  5854 ms: testing::internal::FunctionMocker<$>::PrintTriedExpectationsLocked(s... (1993 times, avg 2 ms)
  5767 ms: testing::internal::FunctionMocker<$>::UntypedFindMatchingExpectation... (1993 times, avg 2 ms)
  5176 ms: std::__1::ostreambuf_iterator<$> std::__1::__pad_and_output<$>(std::... (1053 times, avg 4 ms)
  4801 ms: testing::internal::OnCallSpec<$>::GetAction() const (1961 times, avg 2 ms)
  4660 ms: void std::__1::__introsort<$>(boost::geometry::detail::overlay::turn... (46 times, avg 101 ms)
  4271 ms: testing::internal::TypedExpectation<$>::GetActionForArguments(testin... (1993 times, avg 2 ms)
  3850 ms: std::__1::basic_ostream<$>& std::__1::__put_character_sequence<$>(st... (1053 times, avg 3 ms)
  3807 ms: testing::internal::SuiteApiResolver<$>::GetSetUpCaseOrSuite(char con... (2157 times, avg 1 ms)
  3796 ms: testing::internal::MockSpec<$>::InternalExpectedAt(char const*, int,... (359 times, avg 10 ms)
  3777 ms: std::__1::__function::__func<$>::target(std::type_info const&) const (1456 times, avg 2 ms)
  3254 ms: testing::internal::ParameterizedTestSuiteInfo<$>* testing::internal:... (406 times, avg 8 ms)
  3079 ms: std::__1::__function::__func<$>::__clone() const (906 times, avg 3 ms)
  2966 ms: testing::internal::TestFactoryImpl<$>::CreateTest() (968 times, avg 3 ms)
  2809 ms: classic_protocol::Codec<$>::decode(net::const_buffer const&, std::__... (120 times, avg 23 ms)
  2744 ms: testing::internal::SuiteApiResolver<$>::GetTearDownCaseOrSuite(char ... (2157 times, avg 1 ms)
  2600 ms: testing::internal::MatcherBase<$>::DescribeTo(std::__1::basic_ostrea... (1491 times, avg 1 ms)
  2496 ms: testing::internal::FunctionMocker<$>::ClearDefaultActionsLocked() (846 times, avg 2 ms)
  2447 ms: bool boost::geometry::detail::partition::partition_two_ranges<$>::ap... (104 times, avg 23 ms)
  2442 ms: testing::internal::MatcherBase<$>::~MatcherBase() (588 times, avg 4 ms)
  2394 ms: testing::Matcher<$>::~Matcher() (889 times, avg 2 ms)

*** Expensive headers:
209141 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/sql/sql_class.h (included 658 times, avg 317 ms), included via:
  replicated_columns_view.cc.o replicated_columns_view.h column_filter_factory.h column_filter_inbound_func_indexes.h column_filter.h  (877 ms)
  mysql_connection_attributes_iterator_imp.cc.o  (868 ms)
  mysql_query_attributes_imp.cc.o  (827 ms)
  mysql_thd_attributes_imp.cc.o  (809 ms)
  trx0i_s.cc.o  (806 ms)
  hold_transactions.cc.o hold_transactions.h  (800 ms)
  ...

181774 ms: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/__functional/boyer_moore_searcher.h (included 3328 times, avg 54 ms), included via:
  classic_stmt_reset_forwarder.h forwarding_processor.h processor.h basic_protocol_splicer.h functional  (310 ms)
  sql_authorization.h functional  (244 ms)
  functional  (239 ms)
  keycache.h string_view functional  (231 ms)
  geometry.hpp geometry.hpp radian_access.hpp cast.hpp converter.hpp converter_policies.hpp functional  (231 ms)
  json_dom.h functional  (229 ms)
  ...

165255 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/extra/googletest/googletest-release-1.12.0/googletest/include/gtest/gtest.h (included 525 times, avg 314 ms), included via:
  cell_calculator-t.cc.o  (757 ms)
  strings_valid_check-t.cc.o  (717 ms)
  varlen_sort-t.cc.o  (717 ms)
  reference_cache-t.cc.o  (701 ms)
  allocator-t.cc.o  (692 ms)
  val_int_compare-t.cc.o  (691 ms)
  ...

155001 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/include/m_string.h (included 2102 times, avg 73 ms), included via:
  mf_path.cc.o  (598 ms)
  mf_loadpath.cc.o  (558 ms)
  NdbTCP.cpp.o ndb_global.h  (557 ms)
  mf_tempdir.cc.o  (531 ms)
  my_symlink.cc.o  (476 ms)
  mf_same.cc.o my_sys.h  (474 ms)
  ...

125881 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/include/lex_string.h (included 1271 times, avg 99 ms), included via:
  sql_hints.yy.cc.o  (390 ms)
  dd_trigger.cc.o dd_trigger.h  (386 ms)
  rpl_async_conn_failover_table_operations.cc.o log_builtins.h log.h  (382 ms)
  xpl_log.cc.o xpl_log.h log_builtins.h log.h  (364 ms)
  sql_authentication.cc.o sql_authentication.h  (360 ms)
  mysql_audit_print_service_double_data_source_imp.cc.o events.h  (336 ms)
  ...

121427 ms: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/__memory/shared_ptr.h (included 3369 times, avg 36 ms), included via:
  vector __split_buffer memory  (134 ms)
  fstream __locale memory  (122 ms)
  fstream __locale memory  (120 ms)
  gtest.h memory  (115 ms)
  ndb_global.h m_string.h algorithm memory  (113 ms)
  geometry.hpp geometry.hpp radian_access.hpp cast.hpp converter.hpp converter_policies.hpp functional boyer_moore_searcher.h  (110 ms)
  ...

117425 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/sql/field.h (included 693 times, avg 169 ms), included via:
  table_access_service.cc.o  (790 ms)
  field.cc.o  (733 ms)
  row.cc.o  (692 ms)
  i_s.cc.o  (692 ms)
  lob0update.cc.o  (618 ms)
  sql_select.cc.o sql_select.h  (585 ms)
  ...

107213 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/extra/googletest/googletest-release-1.12.0/googlemock/include/gmock/gmock.h (included 271 times, avg 395 ms), included via:
  socket_acceptor_task_t.cc.o  (907 ms)
  timeouts_t.cc.o  (829 ms)
  gmock-all.cc.o  (813 ms)
  sasl_plain_auth_t.cc.o  (787 ms)
  sha256_cache_t.cc.o  (783 ms)
  gcs_xcom_control_interface-t.cc.o gcs_base_test.h  (782 ms)
  ...

107076 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/sql/table.h (included 845 times, avg 126 ms), included via:
  dd_routine.cc.o dd_routine.h  (718 ms)
  global.cc.o global.h  (700 ms)
  table.cc.o  (664 ms)
  rpl_sys_key_access.cc.o rpl_sys_key_access.h  (619 ms)
  zlob0update.cc.o  (575 ms)
  pfs_instr_class.cc.o  (396 ms)
  ...

102310 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/sql/handler.h (included 923 times, avg 110 ms), included via:
  page_track_service.cc.o  (717 ms)
  ha_example.cc.o ha_example.h  (668 ms)
  ha_archive.cc.o ha_archive.h  (631 ms)
  ha_mock.cc.o ha_mock.h  (625 ms)
  ha_perfschema.cc.o ha_perfschema.h  (624 ms)
  ha_blackhole.cc.o ha_blackhole.h  (598 ms)
  ...

  done in 1.4s.

Nice, isn'it? Now what can we find in there?

  • "Files that took longest to parse (compiler frontend)": all tests
  • "Files that took longest to codegen (compiler backend)": tests again, with a bit of GIS code (no surprise there because Boost is used there)
  • "Templates that took longest to instantiate": lots of truncated output of std::function (cannot tell where used because truncation and too lazy to rerun the analyzer with the longer output option)
  • "Template sets that took longest to instantiate": std::function again, Google Test, and regular C++ standard library things.
  • "Functions that took longest to compile": MYSQLparse (no surprise there), GIS code (again no surprise), tests (by now no surprise neither)
  • "Function sets that took longest to compile / optimize:": tests tests tests with a bit of GIS
  • "Expensive headers": sql_class.h (I knew it!), C++ library (why so much of shared_ptr BTW?), and tests again.

Let's not compile the tests, then

MySQL CMake allows setting -DWITH_UNIT_TESTS=OFF, which presumably should get rid of most things in the above. Of course, not compiling the unit tests has the inconvenience of not being able to run them after a build, and breaking the build for Mark. But let's experiment.

Rerunning the above with -DWITH_UNIT_TESTS=OFF cut the wall compilation time almost in half, which is unfortunately invalidated by Chrome slowly making its way to a crash in the first third of the full compilation, and the report now looks very different: 


**** Time summary:
Compilation (4955 times):
  Parsing (frontend):         2912.1 s
  Codegen & opts (backend):   1280.0 s

**** Files that took longest to parse (compiler frontend):
  7134 ms: ./sql/CMakeFiles/sql_gis.dir/gis/union_functor.cc.o
  6966 ms: ./router/src/router/src/CMakeFiles/router_frontend_lib.dir/router_app.cc.o
  6820 ms: ./sql/CMakeFiles/sql_gis.dir/gis/symdifference_functor.cc.o
  6527 ms: ./sql/CMakeFiles/sql_gis.dir/gis/difference_functor.cc.o
  6454 ms: ./sql/CMakeFiles/sql_gis.dir/gis/intersection_functor.cc.o
  5857 ms: ./sql/CMakeFiles/sql_gis.dir/gis/srs/wkt_parser.cc.o
  5748 ms: ./sql/CMakeFiles/sql_gis.dir/gis/symdifference_functor.cc.o
  5577 ms: ./sql/CMakeFiles/sql_main.dir/auth/sql_authorization.cc.o
  5543 ms: ./sql/CMakeFiles/sql_gis.dir/gis/within.cc.o
  5132 ms: ./sql/CMakeFiles/sql_gis.dir/item_geofunc.cc.o

**** Files that took longest to codegen (compiler backend):
 32200 ms: ./sql/CMakeFiles/sql_gis.dir/gis/intersection_functor.cc.o
 30934 ms: ./sql/CMakeFiles/sql_gis.dir/gis/difference_functor.cc.o
 26649 ms: ./sql/CMakeFiles/sql_gis.dir/gis/symdifference_functor.cc.o
 26390 ms: ./sql/CMakeFiles/sql_gis.dir/gis/touches.cc.o
 23451 ms: ./sql/CMakeFiles/sql_gis.dir/gis/crosses.cc.o
 21500 ms: ./sql/CMakeFiles/sql_gis.dir/gis/within.cc.o
 19349 ms: ./sql/CMakeFiles/sql_gis.dir/gis/distance_functor.cc.o
 19187 ms: ./sql/CMakeFiles/sql_gis.dir/gis/union_functor.cc.o
 18280 ms: ./sql/CMakeFiles/sql_gis.dir/gis/overlaps.cc.o
 16962 ms: ./sql/CMakeFiles/sql_gis.dir/gis/buffer.cc.o

**** Templates that took longest to instantiate:
 20773 ms: std::unique_ptr<std::unordered_multimap<const MDL_key *, MDL_ticket_... (914 times, avg 22 ms)
 20707 ms: std::unique_ptr<std::unordered_multimap<const MDL_key *, MDL_ticket_... (915 times, avg 22 ms)
 20550 ms: std::default_delete<std::unordered_multimap<const MDL_key *, MDL_tic... (915 times, avg 22 ms)
 14289 ms: std::unordered_multimap<const MDL_key *, MDL_ticket_store::MDL_ticke... (915 times, avg 15 ms)
 12743 ms: std::copy_n<const wchar_t *, unsigned long, wchar_t *> (2547 times, avg 5 ms)
 12637 ms: std::copy_n<const char *, unsigned long, char *> (2542 times, avg 4 ms)
 12551 ms: std::copy<const wchar_t *, wchar_t *> (2547 times, avg 4 ms)
 12346 ms: std::copy_n<const char16_t *, unsigned long, char16_t *> (2546 times, avg 4 ms)
 12307 ms: std::copy<const char *, char *> (2547 times, avg 4 ms)
 12145 ms: std::copy<const char16_t *, char16_t *> (2547 times, avg 4 ms)
 11944 ms: std::__copy<const wchar_t *, const wchar_t *, wchar_t *, 0> (2547 times, avg 4 ms)
 11894 ms: std::copy_n<const char32_t *, unsigned long, char32_t *> (2547 times, avg 4 ms)
 11674 ms: std::copy<const char32_t *, char32_t *> (2547 times, avg 4 ms)
 11667 ms: std::__copy<const char *, const char *, char *, 0> (2547 times, avg 4 ms)
 11526 ms: std::__copy<const char16_t *, const char16_t *, char16_t *, 0> (2547 times, avg 4 ms)
 10919 ms: std::__copy<const char32_t *, const char32_t *, char32_t *, 0> (2547 times, avg 4 ms)
 10219 ms: collation_unordered_map<std::string, std::unique_ptr<user_var_entry,... (600 times, avg 17 ms)
  9859 ms: std::basic_string<char>::basic_string (2726 times, avg 3 ms)
  9834 ms: std::vector<unsigned char *>::push_back (784 times, avg 12 ms)
  9728 ms: malloc_unordered_map<char **, std::unique_ptr<char, My_free_deleter>... (601 times, avg 16 ms)
  9355 ms: std::vector<unsigned char *>::__push_back_slow_path<unsigned char *c... (784 times, avg 11 ms)
  9318 ms: std::unordered_map<std::string, unsigned long> (846 times, avg 11 ms)
  8973 ms: std::unordered_map<std::string, std::unique_ptr<user_var_entry, void... (600 times, avg 14 ms)
  8932 ms: std::basic_string<wchar_t>::basic_string (2611 times, avg 3 ms)
  8793 ms: std::__scalar_hash<std::_PairT, 2>::operator() (2580 times, avg 3 ms)
  8642 ms: net::basic_waitable_timer<std::chrono::steady_clock>::cancel (81 times, avg 106 ms)
  8639 ms: net::io_context::cancel<net::basic_waitable_timer<std::chrono::stead... (81 times, avg 106 ms)
  8568 ms: collation_unordered_map<std::string, std::unique_ptr<Table_ref, My_f... (600 times, avg 14 ms)
  8505 ms: std::basic_string<char16_t>::basic_string (2609 times, avg 3 ms)
  8493 ms: std::basic_string<char32_t>::basic_string (2618 times, avg 3 ms)

**** Template sets that took longest to instantiate:
 91311 ms: std::unique_ptr<$> (44993 times, avg 2 ms)
 81831 ms: std::__hash_table<$> (9702 times, avg 8 ms)
 74391 ms: std::unordered_map<$> (7352 times, avg 10 ms)
 64459 ms: std::allocator_traits<$> (22807 times, avg 2 ms)
 62238 ms: std::map<$> (11577 times, avg 5 ms)
 59739 ms: std::function<$>::function<$> (1549 times, avg 38 ms)
 59342 ms: std::__function::__value_func<$>::__value_func<$> (1549 times, avg 38 ms)
 54833 ms: std::copy<$> (11586 times, avg 4 ms)
 54215 ms: std::__tree<$> (15582 times, avg 3 ms)
 51692 ms: std::__copy<$> (11585 times, avg 4 ms)
 50051 ms: std::copy_n<$> (10264 times, avg 4 ms)
 48536 ms: std::decay<$> (11965 times, avg 4 ms)
 48319 ms: std::__function::__func<$>::__func (1549 times, avg 31 ms)
 48146 ms: std::vector<$>::push_back (3923 times, avg 12 ms)
 45687 ms: std::unordered_map<$>::unordered_map (3885 times, avg 11 ms)
 45017 ms: std::vector<$>::__push_back_slow_path<$> (3822 times, avg 11 ms)
 44368 ms: std::vector<$> (21423 times, avg 2 ms)
 41048 ms: std::vector<$>::__swap_out_circular_buffer (5253 times, avg 7 ms)
 40485 ms: std::__function::__alloc_func<$>::__alloc_func (4647 times, avg 8 ms)
 38700 ms: std::basic_string<$>::basic_string (11728 times, avg 3 ms)
 36048 ms: std::unique_ptr<$>::reset (4008 times, avg 8 ms)
 35840 ms: std::pair<$> (16452 times, avg 2 ms)
 35779 ms: std::__hash_table<$>::__hash_table (4198 times, avg 8 ms)
 35340 ms: std::unique_ptr<$>::~unique_ptr (3870 times, avg 9 ms)
 34007 ms: std::__compressed_pair<$> (10756 times, avg 3 ms)
 33242 ms: std::__uninitialized_allocator_move_if_noexcept<$> (5252 times, avg 6 ms)
 33225 ms: std::basic_string<$> (11487 times, avg 2 ms)
 32465 ms: std::forward_as_tuple<$> (6724 times, avg 4 ms)
 31334 ms: malloc_unordered_map<$> (3118 times, avg 10 ms)
 30751 ms: std::__decay<$> (7475 times, avg 4 ms)

**** Functions that took longest to compile:
  2514 ms: MYSQLparse(THD*, Parse_tree_root**) (/Users/laurynas/vilniusdb/mysql-8.0.33/_build-release-time-report-no-unit-tests/sql/sql_yacc.cc)
   819 ms: _GLOBAL__sub_I_sys_vars.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/sys_vars.cc)
   592 ms: duk__js_execute_bytecode_inner (/Users/laurynas/vilniusdb/mysql-8.0.33/extra/duktape/duktape-2.7.0/src/duktape.c)
   582 ms: SetObjectMembers(std::__1::unique_ptr<Json_object, std::__1::default... (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/join_optimizer/explain_access_path.cc)
   510 ms: MySQLRouter::prepare_command_options() (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/router/src/router_app.cc)
   456 ms: my_strnncoll_uca_900(CHARSET_INFO const*, unsigned char const*, unsi... (/Users/laurynas/vilniusdb/mysql-8.0.33/strings/ctype-uca.cc)
   429 ms: spec_adder(rapidjson::GenericDocument<rapidjson::UTF8<char>, rapidjs... (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/rest_routing/src/rest_routing_plugin.cc)
   384 ms: init_handlers(mysql_harness::PluginFuncEnv*, mysql_harness::LoaderCo... (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/harness/src/logging/logger_plugin.cc)
   351 ms: rapidjson::internal::Schema<rapidjson::GenericSchemaDocument<rapidjs... (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/harness/src/dynamic_state.cc)
   304 ms: my_strnxfrm_uca_900(CHARSET_INFO const*, unsigned char*, unsigned lo... (/Users/laurynas/vilniusdb/mysql-8.0.33/strings/ctype-uca.cc)
   276 ms: spec_adder(rapidjson::GenericDocument<rapidjson::UTF8<char>, rapidjs... (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/rest_metadata_cache/src/rest_metadata_cache_plugin.cc)
   271 ms: my_hash_sort_uca_900(CHARSET_INFO const*, unsigned char const*, unsi... (/Users/laurynas/vilniusdb/mysql-8.0.33/strings/ctype-uca.cc)
   250 ms: test_sql(void*) (/Users/laurynas/vilniusdb/mysql-8.0.33/plugin/test_service_sql_api/test_sql_lock.cc)
   247 ms: CmdArgHandler::process(std::__1::vector<std::__1::basic_string<char,... (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/harness/src/arg_handler.cc)
   242 ms: open_table_def(THD*, TABLE_SHARE*, dd::Table const&) (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/dd_table_share.cc)
   237 ms: trx_undo_report_row_operation(unsigned long, unsigned long, que_thr_... (/Users/laurynas/vilniusdb/mysql-8.0.33/storage/innobase/trx/trx0rec.cc)
   230 ms: test_wl4435_3() (/Users/laurynas/vilniusdb/mysql-8.0.33/testclients/mysql_client_test.cc)
   212 ms: void std::__1::__introsort<std::__1::_ClassicAlgPolicy, boost::geome... (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/gis/overlaps.cc)
   211 ms: dd::tables::Tables::Tables() (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/dd/impl/tables/tables.cc)
   210 ms: spec_adder(rapidjson::GenericDocument<rapidjson::UTF8<char>, rapidjs... (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/rest_connection_pool/src/rest_connection_pool_plugin.cc)
   209 ms: TlsServerContext::default_ciphers() (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/harness/src/tls_server_context.cc)
   209 ms: main (/Users/laurynas/vilniusdb/mysql-8.0.33/client/mysqltest.cc)
   207 ms: build_gcs_parameters(Gcs_interface_parameters&) (/Users/laurynas/vilniusdb/mysql-8.0.33/plugin/group_replication/src/plugin.cc)
   205 ms: init_server_components() (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/mysqld.cc)
   202 ms: void std::__1::__introsort<std::__1::_ClassicAlgPolicy, boost::geome... (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/gis/overlaps.cc)
   200 ms: ConnectJoins(int, int, int, QEP_TAB*, THD*, CallingContext, std::__1... (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/sql_executor.cc)
   194 ms: lock_wait_timeout_thread() (/Users/laurynas/vilniusdb/mysql-8.0.33/storage/innobase/lock/lock0wait.cc)
   193 ms: void std::__1::__introsort<std::__1::_ClassicAlgPolicy, boost::geome... (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/gis/crosses.cc)
   187 ms: (anonymous namespace)::CostingReceiver::FoundSingleNode(int) (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/join_optimizer/join_optimizer.cc)
   186 ms: handle_slave_io (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/rpl_replica.cc)

**** Function sets that took longest to compile / optimize:
 13251 ms: bool boost::geometry::detail::partition::partition_two_ranges<$>::ap... (562 times, avg 23 ms)
  4962 ms: void std::__1::__introsort<$>(boost::geometry::detail::overlay::turn... (46 times, avg 107 ms)
  2395 ms: bool boost::geometry::detail::partition::partition_two_ranges<$>::ap... (104 times, avg 23 ms)
  2264 ms: std::__1::ostreambuf_iterator<$> std::__1::__pad_and_output<$>(std::... (536 times, avg 4 ms)
  1943 ms: void std::__1::__introsort<$>(boost::geometry::detail::relate::linea... (26 times, avg 74 ms)
  1717 ms: std::__1::basic_ostream<$>& std::__1::__put_character_sequence<$>(st... (536 times, avg 3 ms)
  1657 ms: std::__1::basic_stringbuf<$>::str() const (180 times, avg 9 ms)
  1298 ms: unsigned int std::__1::__sort3<$>(boost::geometry::detail::overlay::... (46 times, avg 28 ms)
  1248 ms: bool boost::geometry::partition<$>::apply<$>(boost::geometry::sectio... (281 times, avg 4 ms)
  1224 ms: unsigned int std::__1::__sort5<$>(boost::geometry::detail::overlay::... (46 times, avg 26 ms)
  1220 ms: void std::__1::__tree_balance_after_insert<$>(std::__1::__tree_node_... (317 times, avg 3 ms)
  1211 ms: std::__1::deque<$>::__add_back_capacity() (94 times, avg 12 ms)
  1049 ms: bool boost::geometry::detail::overlay::get_turn_info_for_endpoint<$>... (52 times, avg 20 ms)
   991 ms: bool boost::geometry::detail::partition::partition_one_range<$>::app... (104 times, avg 9 ms)
   968 ms: spec_adder(rapidjson::GenericDocument<$>&) (4 times, avg 242 ms)
   968 ms: std::__1::__tree_node_base<$>*& std::__1::__tree<$>::__find_equal<$>... (253 times, avg 3 ms)
   956 ms: bool boost::geometry::detail::partition::partition_two_ranges<$>::ap... (64 times, avg 14 ms)
   939 ms: (anonymous namespace)::Function_factory<$>::create_func(THD*, MYSQL_... (270 times, avg 3 ms)
   938 ms: std::__1::basic_stringbuf<$>::overflow(int) (180 times, avg 5 ms)
   922 ms: classic_protocol::Codec<$>::decode(net::const_buffer const&, std::__... (70 times, avg 13 ms)
   884 ms: bool boost::geometry::detail::partition::partition_two_ranges<$>::ap... (40 times, avg 22 ms)
   866 ms: boost::geometry::policies::relate::segments_intersection_policy<$>::... (30 times, avg 28 ms)
   864 ms: std::__1::basic_string<$>::basic_string[abi:v15006]<$>(char const*) (351 times, avg 2 ms)
   837 ms: rapidjson::internal::Schema<$>::Schema(rapidjson::GenericSchemaDocum... (5 times, avg 167 ms)
   814 ms: bool std::__1::__insertion_sort_incomplete<$>(boost::geometry::detai... (46 times, avg 17 ms)
   813 ms: void std::__1::__introsort<$>(boost::geometry::detail::relate::linea... (14 times, avg 58 ms)
   810 ms: std::__1::__tree<$>::destroy(std::__1::__tree_node<$>*) (443 times, avg 1 ms)
   808 ms: unsigned int std::__1::__sort4<$>(boost::geometry::detail::overlay::... (46 times, avg 17 ms)
   795 ms: void std::__1::__tree_remove<$>(std::__1::__tree_node_base<$>*, std:... (90 times, avg 8 ms)
   788 ms: std::__1::vector<$>::~vector[abi:v15006]() (418 times, avg 1 ms)

*** Expensive headers:
181934 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/sql/sql_class.h (included 587 times, avg 309 ms), included via:
  mysql_thd_attributes_imp.cc.o  (936 ms)
  replicated_columns_view.cc.o replicated_columns_view.h column_filter_factory.h column_filter_inbound_func_indexes.h column_filter.h  (910 ms)
  mysql_connection_attributes_iterator_imp.cc.o  (838 ms)
  ndb_create_helper.cc.o  (812 ms)
  sql_class.cc.o  (810 ms)
  replicated_columns_view_with_gipk_on_source.cc.o replicated_columns_view_with_gipk_on_source.h replicated_columns_view.h column_filter_factory.h column_filter_inbound_func_indexes.h column_filter.h  (771 ms)
  ...

140903 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/include/m_string.h (included 1793 times, avg 78 ms), included via:
  my_print_defaults.cc.o  (428 ms)
  my_strtoll10.cc.o  (412 ms)
  file.cc.o buf0checksum.h buf0types.h os0event.h univ.i  (406 ms)
  NdbThread.cpp.o ndb_global.h  (389 ms)
  AccLock.cpp.o AccLock.hpp SignalData.hpp ndb_global.h  (386 ms)
  NdbReceiver.cpp.o API.hpp ndb_global.h  (382 ms)
  ...

129393 ms: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/__functional/boyer_moore_searcher.h (included 2549 times, avg 50 ms), included via:
  bgc_ticket_manager.h atomic_bgc_ticket_guard.h functional  (231 ms)
  acl_table_user.h functional  (228 ms)
  config_parser.h functional  (217 ms)
  dynamic_privilege_table.h functional  (217 ms)
  loader_config.h config_parser.h functional  (210 ms)
  random_generator.h random discrete_distribution.h numeric functional  (208 ms)
  ...

114343 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/include/lex_string.h (included 1195 times, avg 95 ms), included via:
  sql_authentication.cc.o sql_authentication.h  (329 ms)
  audit_api_connection_service_imp.cc.o sql_audit.h  (308 ms)
  dd_trigger.cc.o dd_trigger.h  (305 ms)
  sql_hints.yy.cc.o  (299 ms)
  sql_user_table.cc.o sql_user_table.h sql_system_table_check.h log_builtins.h log.h  (297 ms)
  show_query_builder.cc.o show_query_builder.h  (292 ms)
  ...

105206 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/sql/field.h (included 634 times, avg 165 ms), included via:
  table_access_service.cc.o  (733 ms)
  rpl_sys_table_access.cc.o rpl_sys_table_access.h  (611 ms)
  sql_select.cc.o sql_select.h  (597 ms)
  field.cc.o  (597 ms)
  row.cc.o  (581 ms)
  lob0update.cc.o  (577 ms)
  ...

95348 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/sql/table.h (included 779 times, avg 122 ms), included via:
  dd_routine.cc.o dd_routine.h  (633 ms)
  global.cc.o global.h  (582 ms)
  table.cc.o  (551 ms)
  zlob0update.cc.o  (493 ms)
  rpl_sys_key_access.cc.o rpl_sys_key_access.h  (488 ms)
  table_replication_group_member_actions.cc.o rpl_sys_key_access.h  (369 ms)
  ...

94011 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/storage/innobase/include/univ.i (included 209 times, avg 449 ms), included via:
  mach0data.cc.o mach0data.h mtr0types.h sync0rw.h  (710 ms)
  buf0block_hint.cc.o buf0block_hint.h buf0types.h os0event.h  (705 ms)
  btr0sea.cc.o btr0sea.h  (700 ms)
  srv0tmp.cc.o srv0tmp.h srv0srv.h buf0checksum.h buf0types.h os0event.h  (684 ms)
  file.cc.o buf0checksum.h buf0types.h os0event.h  (681 ms)
  btr0btr.cc.o btr0btr.h btr0types.h  (674 ms)
  ...

93077 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/sql/log_event.h (included 241 times, avg 386 ms), included via:
  global.cc.o global.h context.h  (889 ms)
  log_event.cc.o  (879 ms)
  context.cc.o context.h  (862 ms)
  recovery.cc.o recovery.h global.h context.h  (811 ms)
  iterators.cc.o iterators.h binlog_reader.h  (775 ms)
  binlog_istream.cc.o  (775 ms)
  ...

91706 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/sql/handler.h (included 842 times, avg 108 ms), included via:
  ha_example.cc.o ha_example.h  (756 ms)
  ha_mock.cc.o ha_mock.h  (685 ms)
  page_track_service.cc.o  (649 ms)
  ha_tina.cc.o ha_tina.h  (614 ms)
  ha_archive.cc.o ha_archive.h  (580 ms)
  api0misc.cc.o api0misc.h  (575 ms)
  ...

87795 ms: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/__memory/shared_ptr.h (included 2580 times, avg 34 ms), included via:
  AccLock.hpp SignalData.hpp ndb_global.h m_string.h algorithm memory  (102 ms)
  dynamic_privilege_table.h functional boyer_moore_searcher.h  (98 ms)
  signing_key.h string memory  (97 ms)
  ndb_global.h m_string.h algorithm memory  (96 ms)
  dict0dict.h set __node_handle memory  (93 ms)
  my_byteorder.h template_utils.h algorithm memory  (93 ms)
  ...

  done in 0.8s.

What changed?

  • "Files that took longest to parse (compiler frontend)": all GIS
  • "Files that took longest to codegen (compiler backend)": all GIS
  • "Templates that took longest to instantiate": all those std::function gone, so must have been tests. Something about MDL remaining.
  • "Template sets that took longest to instantiate": std::unique_ptr. Oh well.
  • "Functions that took longest to compile": MYSQLparse (no surprise there), system variables handling, 3rd party code, and variety of other things
  • "Function sets that took longest to compile / optimize:": Boost/GIS, the standard library.
  • "Expensive headers": sql_class.h, and the C++ standard library headers mostly gone together with shared_ptr (so it was tests using it).

Interestingly (and usefully) this provides a different kind of insight than what Mark discovered for Meta branch: it would not discover RocksDB being compiled three times. It would be great if the MySQL developers looked into this tool, which seems very easy to use, and reducing compilation times. That sql_class.h file is way overdue for splitting!

No comments:
Labels:
Subscribe to: Posts (Atom)