MySQL, clang -ftime-trace, & ClangBuildAnalyzer

TL;DR: install ClangBuildAnalyzer, compile MySQL with CC='clang -ftime-trace' CXX='clang++ -ftime-trace', run the analyzer, enjoy nice reports telling you why it's so slow.

Inspired by Mark's question on Twitter, I decided to try out clang -ftime-trace on MySQL source tree. This clang flag was added by Aras Pranckevičius (sure I'll call out a Lithuanian author by name) to LLVM 9.0, and it outputs per-source-file JSONs in Chrome Tracing format explaining where did the compilation time go. Now, instead of stating that for this file parsing took X seconds, template instantiation Y minutes, and register allocation Z microseconds it provides actionable information with object granularity, i.e. it tells you which exactly templates were expensive to instantiate etc.

To get those reports, -ftime-trace needs to be added to compilation flags. Don't add it to MySQL CMake CMAKE_CXX_FLAGS etc, because you'll have to get the rest of -O2 -DNDEBUG etc. right. Instead, pretend that this flag is a part of compiler invocation itself, and set CMAKE_CXX_COMPILER='clang++ -ftime-trace', and likewise for CMAKE_C_COMPILER. Or, use CC and CXX environment variables as I did in the TL;DR section.

A clean MySQL 8.0.33 results in a few thousands of those JSONs. Each one could be loaded in Chrome by navigating to chrome://tracing and loading a file. Individually. This blog post suggests zipping them all up, enabling Chrome to load that zip instead. But the post also warns that it takes a long time to load, and I can confirm Chrome crashing after some 40 minutes of work.

Luckily for us Aras wrote a project-level analyzer tool: ClangBuildAnalyzer. It handles those thousands of JSONs just fine, and very quickly, and for a regular 8.0.33 Release configuration build outputs the following:

**** Time summary:
Compilation (6522 times):
  Parsing (frontend):         4669.4 s
  Codegen & opts (backend):   3139.1 s

**** Files that took longest to parse (compiler frontend):
 16499 ms: ./unittest/gunit/xplugin/xpl/CMakeFiles/xpl_test_src.dir/mock/mock.cc.o
 12057 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/session_execute_t.cc.o
 12011 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/session_connect_t.cc.o
 11907 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/session_options_t.cc.o
 11533 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/session_negotiation_t.cc.o
 11454 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/protocol_send_recv_t.cc.o
 11085 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/session_capability_t.cc.o
 10930 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/auth_chaining_t.cc.o
 10201 ms: ./unittest/gunit/xplugin/xpl/CMakeFiles/xpl_test_src.dir/timeouts_t.cc.o
 10195 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/session_general_t.cc.o

**** Files that took longest to codegen (compiler backend):
 48974 ms: ./unittest/gunit/innodb/CMakeFiles/merge_innodb_tests-t.dir/ut0new-t.cc.o
 31598 ms: ./unittest/gunit/xplugin/xpl/CMakeFiles/xpl_test_src.dir/mock/mock.cc.o
 30849 ms: ./sql/CMakeFiles/sql_gis.dir/gis/intersection_functor.cc.o
 30007 ms: ./sql/CMakeFiles/sql_gis.dir/gis/difference_functor.cc.o
 29611 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/auth_chaining_t.cc.o
 27047 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/session_connect_t.cc.o
 26776 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/session_negotiation_t.cc.o
 26253 ms: ./unittest/gunit/CMakeFiles/merge_large_tests-t.dir/hypergraph_optimizer-t.cc.o
 25877 ms: ./sql/CMakeFiles/sql_gis.dir/gis/symdifference_functor.cc.o
 25600 ms: ./unittest/gunit/xplugin/xcl/CMakeFiles/xclient_unit_tests.dir/session_options_t.cc.o

**** Templates that took longest to instantiate:
109225 ms: std::__function::__func<(lambda at /Users/laurynas/vilniusdb/mysql-8... (7110 times, avg 15 ms)
107390 ms: std::__function::__func<(lambda at /Users/laurynas/vilniusdb/mysql-8... (7110 times, avg 15 ms)
 93389 ms: std::function<void ()>::function<(lambda at /Users/laurynas/vilniusd... (2370 times, avg 39 ms)
 92573 ms: std::__function::__value_func<void ()>::__value_func<(lambda at /Use... (2370 times, avg 39 ms)
 92539 ms: std::function<void ()>::function<(lambda at /Users/laurynas/vilniusd... (2370 times, avg 39 ms)
 91689 ms: std::__function::__value_func<void ()>::__value_func<(lambda at /Use... (2370 times, avg 38 ms)
 91551 ms: std::__function::__value_func<void ()>::__value_func<(lambda at /Use... (2370 times, avg 38 ms)
 90546 ms: std::__function::__value_func<void ()>::__value_func<(lambda at /Use... (2370 times, avg 38 ms)
 64823 ms: std::__function::__alloc_func<(lambda at /Users/laurynas/vilniusdb/m... (7110 times, avg 9 ms)
 63866 ms: std::__function::__alloc_func<(lambda at /Users/laurynas/vilniusdb/m... (7110 times, avg 8 ms)
 39867 ms: std::__function::__func<(lambda at /Users/laurynas/vilniusdb/mysql-8... (4740 times, avg 8 ms)
 39246 ms: std::__function::__func<(lambda at /Users/laurynas/vilniusdb/mysql-8... (4740 times, avg 8 ms)
 22360 ms: std::unique_ptr<std::unordered_multimap<const MDL_key *, MDL_ticket_... (1016 times, avg 22 ms)
 22285 ms: std::unique_ptr<std::unordered_multimap<const MDL_key *, MDL_ticket_... (1018 times, avg 21 ms)
 22138 ms: std::default_delete<std::unordered_multimap<const MDL_key *, MDL_tic... (1018 times, avg 21 ms)
 17698 ms: std::copy_n<const char16_t *, unsigned long, char16_t *> (3325 times, avg 5 ms)
 17666 ms: testing::internal::ValueArray<bool, bool>::operator ParamGenerator<b... (525 times, avg 33 ms)
 17432 ms: std::copy<const char16_t *, char16_t *> (3326 times, avg 5 ms)
 17359 ms: std::copy_n<const wchar_t *, unsigned long, wchar_t *> (3326 times, avg 5 ms)
 17057 ms: std::copy<const wchar_t *, wchar_t *> (3326 times, avg 5 ms)
 16950 ms: net::basic_waitable_timer<std::chrono::steady_clock>::cancel (150 times, avg 113 ms)
 16946 ms: net::io_context::cancel<net::basic_waitable_timer<std::chrono::stead... (150 times, avg 112 ms)
 16813 ms: std::copy_n<const char *, unsigned long, char *> (3314 times, avg 5 ms)
 16622 ms: std::copy_n<const char32_t *, unsigned long, char32_t *> (3326 times, avg 4 ms)
 16448 ms: std::__copy<const char16_t *, const char16_t *, char16_t *, 0> (3326 times, avg 4 ms)
 16373 ms: std::copy<const char *, char *> (3326 times, avg 4 ms)
 16342 ms: std::copy<const char32_t *, char32_t *> (3326 times, avg 4 ms)
 16110 ms: net::basic_waitable_timer<std::chrono::steady_clock>::~basic_waitabl... (139 times, avg 115 ms)
 16047 ms: std::__copy<const wchar_t *, const wchar_t *, wchar_t *, 0> (3326 times, avg 4 ms)
 15781 ms: std::unordered_multimap<const MDL_key *, MDL_ticket_store::MDL_ticke... (1018 times, avg 15 ms)

**** Template sets that took longest to instantiate:
333347 ms: std::function<$>::function<$> (8367 times, avg 39 ms)
330713 ms: std::__function::__value_func<$>::__value_func<$> (8367 times, avg 39 ms)
270430 ms: std::__function::__func<$>::__func (8367 times, avg 32 ms)
236822 ms: testing::internal::FunctionMocker<$>::Invoke (2370 times, avg 99 ms)
229859 ms: std::__function::__alloc_func<$>::__alloc_func (25098 times, avg 9 ms)
227397 ms: testing::internal::FunctionMocker<$>::InvokeWith (2370 times, avg 95 ms)
174340 ms: std::forward_as_tuple<$> (34235 times, avg 5 ms)
163483 ms: std::unique_ptr<$> (75303 times, avg 2 ms)
145187 ms: std::tuple<$> (45300 times, avg 3 ms)
141977 ms: std::__function::__func<$>::__clone (16733 times, avg 8 ms)
 98501 ms: std::__hash_table<$> (11200 times, avg 8 ms)
 94028 ms: std::allocator_traits<$> (31254 times, avg 3 ms)
 93971 ms: std::__compressed_pair<$>::__compressed_pair<$> (35840 times, avg 2 ms)
 91566 ms: std::unordered_map<$> (8606 times, avg 10 ms)
 90396 ms: std::decay<$> (20916 times, avg 4 ms)
 82788 ms: std::copy<$> (16492 times, avg 5 ms)
 80875 ms: std::map<$> (14763 times, avg 5 ms)
 77163 ms: std::__copy<$> (16493 times, avg 4 ms)
 73875 ms: std::vector<$>::push_back (6296 times, avg 11 ms)
 71983 ms: std::copy_n<$> (13989 times, avg 5 ms)
 71340 ms: stdx::expected<$> (9078 times, avg 7 ms)
 70331 ms: std::__tree<$> (19917 times, avg 3 ms)
 68211 ms: std::vector<$> (32556 times, avg 2 ms)
 67929 ms: std::vector<$>::__push_back_slow_path<$> (6078 times, avg 11 ms)
 62076 ms: std::vector<$>::__swap_out_circular_buffer (8254 times, avg 7 ms)
 57906 ms: std::__decay<$> (12966 times, avg 4 ms)
 54976 ms: std::__compressed_pair<$> (17033 times, avg 3 ms)
 53465 ms: std::unordered_map<$>::unordered_map (4697 times, avg 11 ms)
 53099 ms: testing::internal::MatcherBase<$>::MatcherBase<$> (5591 times, avg 9 ms)
 52731 ms: testing::internal::MatcherBase<$>::Init<$> (5591 times, avg 9 ms)

**** Functions that took longest to compile:
  2560 ms: MYSQLparse(THD*, Parse_tree_root**) (/Users/laurynas/vilniusdb/mysql-8.0.33/_build-release-time-report/sql/sql_yacc.cc)
  1602 ms: _GLOBAL__sub_I_gis_union_t.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/gis_union-t.cc)
  1577 ms: _GLOBAL__sub_I_gis_difference_t.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/gis_difference-t.cc)
  1396 ms: _GLOBAL__sub_I_gis_intersection_t.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/gis_intersection-t.cc)
  1280 ms: _GLOBAL__sub_I_gis_symdifference_t.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/gis_symdifference-t.cc)
   877 ms: __cxx_global_var_init.334 (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/router/tests/test_keyring_frontend.cc)
   839 ms: json_binary_unittest::JsonBinaryTest_BasicTest_Test::TestBody() (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/json_binary-t.cc)
   823 ms: _GLOBAL__sub_I_sys_vars.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/sys_vars.cc)
   821 ms: __cxx_global_var_init.143 (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/router/tests/test_keyring_frontend.cc)
   809 ms: ShareConnectionTinyPoolOneServerTest_not_sharable_Test::TestBody() (/Users/laurynas/vilniusdb/mysql-8.0.33/router/tests/integration/test_routing_sharing_constrained_pools.cc)
   732 ms: testing::internal::FlatTupleBase<testing::internal::FlatTuple<Target... (/Users/laurynas/vilniusdb/mysql-8.0.33/router/tests/component/test_bootstrap_clusterset.cc)
   719 ms: AccountReuseCreateComboTestP::gen_testcases() (/Users/laurynas/vilniusdb/mysql-8.0.33/router/tests/component/test_bootstrap_account.cc)
   689 ms: KeyringFrontendTest_ensure_Test::TestBody() (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/router/tests/test_keyring_frontend.cc)
   624 ms: duk__js_execute_bytecode_inner (/Users/laurynas/vilniusdb/mysql-8.0.33/extra/duktape/duktape-2.7.0/src/duktape.c)
   609 ms: _GLOBAL__sub_I_test_classic_protocol_message.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/mysql_protocol/tests/test_classic_protocol_message.cc)
   591 ms: MySQLRouter::prepare_command_options() (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/router/src/router_app.cc)
   579 ms: json_dom_unittest::JsonDomTest_BasicTest_Test::TestBody() (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/json_dom-t.cc)
   578 ms: SetObjectMembers(std::__1::unique_ptr<Json_object, std::__1::default... (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/join_optimizer/explain_access_path.cc)
   538 ms: operations_unittest::KeyringCommonOperations_test_OperationsTestWith... (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/components/keyring_common/operations-t.cc)
   531 ms: strnxfrm_unittest::StrmxfrmHashTest_HashStability_Test::TestBody() (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/strings_strnxfrm-t.cc)
   511 ms: spec_adder(rapidjson::GenericDocument<rapidjson::UTF8<char>, rapidjs... (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/rest_routing/src/rest_routing_plugin.cc)
   499 ms: mysys_my_time::MysysMyTime_StrToDatetime_Test::TestBody() (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/mysys_my_time-t.cc)
   476 ms: my_strnncoll_uca_900(CHARSET_INFO const*, unsigned char const*, unsi... (/Users/laurynas/vilniusdb/mysql-8.0.33/strings/ctype-uca.cc)
   467 ms: dd_properties_unittest::PropertiesTest_ValidSetGetIntBool_Test::Test... (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/dd_properties-t.cc)
   450 ms: testing::internal::FlatTupleBase<testing::internal::FlatTuple<Target... (/Users/laurynas/vilniusdb/mysql-8.0.33/router/tests/component/test_bootstrap_clusterset.cc)
   444 ms: KeyringManager_init_with_key_file_Test::TestBody() (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/harness/tests/test_keyring_manager.cc)
   427 ms: _GLOBAL__sub_I_hypergraph_optimizer_t.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/hypergraph_optimizer-t.cc)
   425 ms: _GLOBAL__sub_I_admin_cmd_arguments_object_t.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/xplugin/xpl/admin_cmd_arguments_object_t.cc)
   416 ms: __cxx_global_var_init.49 (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/http/tests/test_passwd.cc)
   378 ms: _GLOBAL__sub_I_expr_generator_parametric_t.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/unittest/gunit/xplugin/xpl/expr_generator_parametric_t.cc)

**** Function sets that took longest to compile / optimize:
 39400 ms: testing::internal::FunctionMocker<$>::InvokeWith(std::__1::tuple<$>&&) (1961 times, avg 20 ms)
 21051 ms: testing::internal::TypedExpectation<$>::ExplainMatchResultTo(std::__... (1993 times, avg 10 ms)
 19210 ms: testing::internal::ParameterizedTestSuiteInfo<$>::RegisterTests() (406 times, avg 47 ms)
 18661 ms: void testing::internal::TuplePrefix<$>::ExplainMatchFailuresTo<$>(st... (2627 times, avg 7 ms)
 17837 ms: testing::internal::TypeParameterizedTest<$>::Register(char const*, t... (1286 times, avg 13 ms)
 14176 ms: testing::internal::TypedExpectation<$>::GetCurrentAction(testing::in... (1993 times, avg 7 ms)
 12635 ms: bool boost::geometry::detail::partition::partition_two_ranges<$>::ap... (562 times, avg 22 ms)
 11895 ms: testing::internal::FunctionMocker<$>::PerformDefaultAction(std::__1:... (1961 times, avg 6 ms)
  6589 ms: testing::internal::FunctionMocker<$>::PerformAction(void const*, std... (1961 times, avg 3 ms)
  5890 ms: testing::internal::FunctionMocker<$>::DescribeDefaultActionTo(std::_... (1993 times, avg 2 ms)
  5854 ms: testing::internal::FunctionMocker<$>::PrintTriedExpectationsLocked(s... (1993 times, avg 2 ms)
  5767 ms: testing::internal::FunctionMocker<$>::UntypedFindMatchingExpectation... (1993 times, avg 2 ms)
  5176 ms: std::__1::ostreambuf_iterator<$> std::__1::__pad_and_output<$>(std::... (1053 times, avg 4 ms)
  4801 ms: testing::internal::OnCallSpec<$>::GetAction() const (1961 times, avg 2 ms)
  4660 ms: void std::__1::__introsort<$>(boost::geometry::detail::overlay::turn... (46 times, avg 101 ms)
  4271 ms: testing::internal::TypedExpectation<$>::GetActionForArguments(testin... (1993 times, avg 2 ms)
  3850 ms: std::__1::basic_ostream<$>& std::__1::__put_character_sequence<$>(st... (1053 times, avg 3 ms)
  3807 ms: testing::internal::SuiteApiResolver<$>::GetSetUpCaseOrSuite(char con... (2157 times, avg 1 ms)
  3796 ms: testing::internal::MockSpec<$>::InternalExpectedAt(char const*, int,... (359 times, avg 10 ms)
  3777 ms: std::__1::__function::__func<$>::target(std::type_info const&) const (1456 times, avg 2 ms)
  3254 ms: testing::internal::ParameterizedTestSuiteInfo<$>* testing::internal:... (406 times, avg 8 ms)
  3079 ms: std::__1::__function::__func<$>::__clone() const (906 times, avg 3 ms)
  2966 ms: testing::internal::TestFactoryImpl<$>::CreateTest() (968 times, avg 3 ms)
  2809 ms: classic_protocol::Codec<$>::decode(net::const_buffer const&, std::__... (120 times, avg 23 ms)
  2744 ms: testing::internal::SuiteApiResolver<$>::GetTearDownCaseOrSuite(char ... (2157 times, avg 1 ms)
  2600 ms: testing::internal::MatcherBase<$>::DescribeTo(std::__1::basic_ostrea... (1491 times, avg 1 ms)
  2496 ms: testing::internal::FunctionMocker<$>::ClearDefaultActionsLocked() (846 times, avg 2 ms)
  2447 ms: bool boost::geometry::detail::partition::partition_two_ranges<$>::ap... (104 times, avg 23 ms)
  2442 ms: testing::internal::MatcherBase<$>::~MatcherBase() (588 times, avg 4 ms)
  2394 ms: testing::Matcher<$>::~Matcher() (889 times, avg 2 ms)

*** Expensive headers:
209141 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/sql/sql_class.h (included 658 times, avg 317 ms), included via:
  replicated_columns_view.cc.o replicated_columns_view.h column_filter_factory.h column_filter_inbound_func_indexes.h column_filter.h  (877 ms)
  mysql_connection_attributes_iterator_imp.cc.o  (868 ms)
  mysql_query_attributes_imp.cc.o  (827 ms)
  mysql_thd_attributes_imp.cc.o  (809 ms)
  trx0i_s.cc.o  (806 ms)
  hold_transactions.cc.o hold_transactions.h  (800 ms)
  ...

181774 ms: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/__functional/boyer_moore_searcher.h (included 3328 times, avg 54 ms), included via:
  classic_stmt_reset_forwarder.h forwarding_processor.h processor.h basic_protocol_splicer.h functional  (310 ms)
  sql_authorization.h functional  (244 ms)
  functional  (239 ms)
  keycache.h string_view functional  (231 ms)
  geometry.hpp geometry.hpp radian_access.hpp cast.hpp converter.hpp converter_policies.hpp functional  (231 ms)
  json_dom.h functional  (229 ms)
  ...

165255 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/extra/googletest/googletest-release-1.12.0/googletest/include/gtest/gtest.h (included 525 times, avg 314 ms), included via:
  cell_calculator-t.cc.o  (757 ms)
  strings_valid_check-t.cc.o  (717 ms)
  varlen_sort-t.cc.o  (717 ms)
  reference_cache-t.cc.o  (701 ms)
  allocator-t.cc.o  (692 ms)
  val_int_compare-t.cc.o  (691 ms)
  ...

155001 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/include/m_string.h (included 2102 times, avg 73 ms), included via:
  mf_path.cc.o  (598 ms)
  mf_loadpath.cc.o  (558 ms)
  NdbTCP.cpp.o ndb_global.h  (557 ms)
  mf_tempdir.cc.o  (531 ms)
  my_symlink.cc.o  (476 ms)
  mf_same.cc.o my_sys.h  (474 ms)
  ...

125881 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/include/lex_string.h (included 1271 times, avg 99 ms), included via:
  sql_hints.yy.cc.o  (390 ms)
  dd_trigger.cc.o dd_trigger.h  (386 ms)
  rpl_async_conn_failover_table_operations.cc.o log_builtins.h log.h  (382 ms)
  xpl_log.cc.o xpl_log.h log_builtins.h log.h  (364 ms)
  sql_authentication.cc.o sql_authentication.h  (360 ms)
  mysql_audit_print_service_double_data_source_imp.cc.o events.h  (336 ms)
  ...

121427 ms: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/__memory/shared_ptr.h (included 3369 times, avg 36 ms), included via:
  vector __split_buffer memory  (134 ms)
  fstream __locale memory  (122 ms)
  fstream __locale memory  (120 ms)
  gtest.h memory  (115 ms)
  ndb_global.h m_string.h algorithm memory  (113 ms)
  geometry.hpp geometry.hpp radian_access.hpp cast.hpp converter.hpp converter_policies.hpp functional boyer_moore_searcher.h  (110 ms)
  ...

117425 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/sql/field.h (included 693 times, avg 169 ms), included via:
  table_access_service.cc.o  (790 ms)
  field.cc.o  (733 ms)
  row.cc.o  (692 ms)
  i_s.cc.o  (692 ms)
  lob0update.cc.o  (618 ms)
  sql_select.cc.o sql_select.h  (585 ms)
  ...

107213 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/extra/googletest/googletest-release-1.12.0/googlemock/include/gmock/gmock.h (included 271 times, avg 395 ms), included via:
  socket_acceptor_task_t.cc.o  (907 ms)
  timeouts_t.cc.o  (829 ms)
  gmock-all.cc.o  (813 ms)
  sasl_plain_auth_t.cc.o  (787 ms)
  sha256_cache_t.cc.o  (783 ms)
  gcs_xcom_control_interface-t.cc.o gcs_base_test.h  (782 ms)
  ...

107076 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/sql/table.h (included 845 times, avg 126 ms), included via:
  dd_routine.cc.o dd_routine.h  (718 ms)
  global.cc.o global.h  (700 ms)
  table.cc.o  (664 ms)
  rpl_sys_key_access.cc.o rpl_sys_key_access.h  (619 ms)
  zlob0update.cc.o  (575 ms)
  pfs_instr_class.cc.o  (396 ms)
  ...

102310 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/sql/handler.h (included 923 times, avg 110 ms), included via:
  page_track_service.cc.o  (717 ms)
  ha_example.cc.o ha_example.h  (668 ms)
  ha_archive.cc.o ha_archive.h  (631 ms)
  ha_mock.cc.o ha_mock.h  (625 ms)
  ha_perfschema.cc.o ha_perfschema.h  (624 ms)
  ha_blackhole.cc.o ha_blackhole.h  (598 ms)
  ...

  done in 1.4s.

Nice, isn'it? Now what can we find in there?

• "Files that took longest to parse (compiler frontend)": all tests
• "Files that took longest to codegen (compiler backend)": tests again, with a bit of GIS code (no surprise there because Boost is used there)
• "Templates that took longest to instantiate": lots of truncated output of std::function (cannot tell where used because truncation and too lazy to rerun the analyzer with the longer output option)
• "Template sets that took longest to instantiate": std::function again, Google Test, and regular C++ standard library things.
• "Functions that took longest to compile": MYSQLparse (no surprise there), GIS code (again no surprise), tests (by now no surprise neither)
• "Function sets that took longest to compile / optimize:": tests tests tests with a bit of GIS
• "Expensive headers": sql_class.h (I knew it!), C++ library (why so much of shared_ptr BTW?), and tests again.

Let's not compile the tests, then

MySQL CMake allows setting -DWITH_UNIT_TESTS=OFF, which presumably should get rid of most things in the above. Of course, not compiling the unit tests has the inconvenience of not being able to run them after a build, and breaking the build for Mark. But let's experiment.

Rerunning the above with -DWITH_UNIT_TESTS=OFF cut the wall compilation time almost in half, which is unfortunately invalidated by Chrome slowly making its way to a crash in the first third of the full compilation, and the report now looks very different:

**** Time summary:
Compilation (4955 times):
  Parsing (frontend):         2912.1 s
  Codegen & opts (backend):   1280.0 s

**** Files that took longest to parse (compiler frontend):
  7134 ms: ./sql/CMakeFiles/sql_gis.dir/gis/union_functor.cc.o
  6966 ms: ./router/src/router/src/CMakeFiles/router_frontend_lib.dir/router_app.cc.o
  6820 ms: ./sql/CMakeFiles/sql_gis.dir/gis/symdifference_functor.cc.o
  6527 ms: ./sql/CMakeFiles/sql_gis.dir/gis/difference_functor.cc.o
  6454 ms: ./sql/CMakeFiles/sql_gis.dir/gis/intersection_functor.cc.o
  5857 ms: ./sql/CMakeFiles/sql_gis.dir/gis/srs/wkt_parser.cc.o
  5748 ms: ./sql/CMakeFiles/sql_gis.dir/gis/symdifference_functor.cc.o
  5577 ms: ./sql/CMakeFiles/sql_main.dir/auth/sql_authorization.cc.o
  5543 ms: ./sql/CMakeFiles/sql_gis.dir/gis/within.cc.o
  5132 ms: ./sql/CMakeFiles/sql_gis.dir/item_geofunc.cc.o

**** Files that took longest to codegen (compiler backend):
 32200 ms: ./sql/CMakeFiles/sql_gis.dir/gis/intersection_functor.cc.o
 30934 ms: ./sql/CMakeFiles/sql_gis.dir/gis/difference_functor.cc.o
 26649 ms: ./sql/CMakeFiles/sql_gis.dir/gis/symdifference_functor.cc.o
 26390 ms: ./sql/CMakeFiles/sql_gis.dir/gis/touches.cc.o
 23451 ms: ./sql/CMakeFiles/sql_gis.dir/gis/crosses.cc.o
 21500 ms: ./sql/CMakeFiles/sql_gis.dir/gis/within.cc.o
 19349 ms: ./sql/CMakeFiles/sql_gis.dir/gis/distance_functor.cc.o
 19187 ms: ./sql/CMakeFiles/sql_gis.dir/gis/union_functor.cc.o
 18280 ms: ./sql/CMakeFiles/sql_gis.dir/gis/overlaps.cc.o
 16962 ms: ./sql/CMakeFiles/sql_gis.dir/gis/buffer.cc.o

**** Templates that took longest to instantiate:
 20773 ms: std::unique_ptr<std::unordered_multimap<const MDL_key *, MDL_ticket_... (914 times, avg 22 ms)
 20707 ms: std::unique_ptr<std::unordered_multimap<const MDL_key *, MDL_ticket_... (915 times, avg 22 ms)
 20550 ms: std::default_delete<std::unordered_multimap<const MDL_key *, MDL_tic... (915 times, avg 22 ms)
 14289 ms: std::unordered_multimap<const MDL_key *, MDL_ticket_store::MDL_ticke... (915 times, avg 15 ms)
 12743 ms: std::copy_n<const wchar_t *, unsigned long, wchar_t *> (2547 times, avg 5 ms)
 12637 ms: std::copy_n<const char *, unsigned long, char *> (2542 times, avg 4 ms)
 12551 ms: std::copy<const wchar_t *, wchar_t *> (2547 times, avg 4 ms)
 12346 ms: std::copy_n<const char16_t *, unsigned long, char16_t *> (2546 times, avg 4 ms)
 12307 ms: std::copy<const char *, char *> (2547 times, avg 4 ms)
 12145 ms: std::copy<const char16_t *, char16_t *> (2547 times, avg 4 ms)
 11944 ms: std::__copy<const wchar_t *, const wchar_t *, wchar_t *, 0> (2547 times, avg 4 ms)
 11894 ms: std::copy_n<const char32_t *, unsigned long, char32_t *> (2547 times, avg 4 ms)
 11674 ms: std::copy<const char32_t *, char32_t *> (2547 times, avg 4 ms)
 11667 ms: std::__copy<const char *, const char *, char *, 0> (2547 times, avg 4 ms)
 11526 ms: std::__copy<const char16_t *, const char16_t *, char16_t *, 0> (2547 times, avg 4 ms)
 10919 ms: std::__copy<const char32_t *, const char32_t *, char32_t *, 0> (2547 times, avg 4 ms)
 10219 ms: collation_unordered_map<std::string, std::unique_ptr<user_var_entry,... (600 times, avg 17 ms)
  9859 ms: std::basic_string<char>::basic_string (2726 times, avg 3 ms)
  9834 ms: std::vector<unsigned char *>::push_back (784 times, avg 12 ms)
  9728 ms: malloc_unordered_map<char **, std::unique_ptr<char, My_free_deleter>... (601 times, avg 16 ms)
  9355 ms: std::vector<unsigned char *>::__push_back_slow_path<unsigned char *c... (784 times, avg 11 ms)
  9318 ms: std::unordered_map<std::string, unsigned long> (846 times, avg 11 ms)
  8973 ms: std::unordered_map<std::string, std::unique_ptr<user_var_entry, void... (600 times, avg 14 ms)
  8932 ms: std::basic_string<wchar_t>::basic_string (2611 times, avg 3 ms)
  8793 ms: std::__scalar_hash<std::_PairT, 2>::operator() (2580 times, avg 3 ms)
  8642 ms: net::basic_waitable_timer<std::chrono::steady_clock>::cancel (81 times, avg 106 ms)
  8639 ms: net::io_context::cancel<net::basic_waitable_timer<std::chrono::stead... (81 times, avg 106 ms)
  8568 ms: collation_unordered_map<std::string, std::unique_ptr<Table_ref, My_f... (600 times, avg 14 ms)
  8505 ms: std::basic_string<char16_t>::basic_string (2609 times, avg 3 ms)
  8493 ms: std::basic_string<char32_t>::basic_string (2618 times, avg 3 ms)

**** Template sets that took longest to instantiate:
 91311 ms: std::unique_ptr<$> (44993 times, avg 2 ms)
 81831 ms: std::__hash_table<$> (9702 times, avg 8 ms)
 74391 ms: std::unordered_map<$> (7352 times, avg 10 ms)
 64459 ms: std::allocator_traits<$> (22807 times, avg 2 ms)
 62238 ms: std::map<$> (11577 times, avg 5 ms)
 59739 ms: std::function<$>::function<$> (1549 times, avg 38 ms)
 59342 ms: std::__function::__value_func<$>::__value_func<$> (1549 times, avg 38 ms)
 54833 ms: std::copy<$> (11586 times, avg 4 ms)
 54215 ms: std::__tree<$> (15582 times, avg 3 ms)
 51692 ms: std::__copy<$> (11585 times, avg 4 ms)
 50051 ms: std::copy_n<$> (10264 times, avg 4 ms)
 48536 ms: std::decay<$> (11965 times, avg 4 ms)
 48319 ms: std::__function::__func<$>::__func (1549 times, avg 31 ms)
 48146 ms: std::vector<$>::push_back (3923 times, avg 12 ms)
 45687 ms: std::unordered_map<$>::unordered_map (3885 times, avg 11 ms)
 45017 ms: std::vector<$>::__push_back_slow_path<$> (3822 times, avg 11 ms)
 44368 ms: std::vector<$> (21423 times, avg 2 ms)
 41048 ms: std::vector<$>::__swap_out_circular_buffer (5253 times, avg 7 ms)
 40485 ms: std::__function::__alloc_func<$>::__alloc_func (4647 times, avg 8 ms)
 38700 ms: std::basic_string<$>::basic_string (11728 times, avg 3 ms)
 36048 ms: std::unique_ptr<$>::reset (4008 times, avg 8 ms)
 35840 ms: std::pair<$> (16452 times, avg 2 ms)
 35779 ms: std::__hash_table<$>::__hash_table (4198 times, avg 8 ms)
 35340 ms: std::unique_ptr<$>::~unique_ptr (3870 times, avg 9 ms)
 34007 ms: std::__compressed_pair<$> (10756 times, avg 3 ms)
 33242 ms: std::__uninitialized_allocator_move_if_noexcept<$> (5252 times, avg 6 ms)
 33225 ms: std::basic_string<$> (11487 times, avg 2 ms)
 32465 ms: std::forward_as_tuple<$> (6724 times, avg 4 ms)
 31334 ms: malloc_unordered_map<$> (3118 times, avg 10 ms)
 30751 ms: std::__decay<$> (7475 times, avg 4 ms)

**** Functions that took longest to compile:
  2514 ms: MYSQLparse(THD*, Parse_tree_root**) (/Users/laurynas/vilniusdb/mysql-8.0.33/_build-release-time-report-no-unit-tests/sql/sql_yacc.cc)
   819 ms: _GLOBAL__sub_I_sys_vars.cc (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/sys_vars.cc)
   592 ms: duk__js_execute_bytecode_inner (/Users/laurynas/vilniusdb/mysql-8.0.33/extra/duktape/duktape-2.7.0/src/duktape.c)
   582 ms: SetObjectMembers(std::__1::unique_ptr<Json_object, std::__1::default... (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/join_optimizer/explain_access_path.cc)
   510 ms: MySQLRouter::prepare_command_options() (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/router/src/router_app.cc)
   456 ms: my_strnncoll_uca_900(CHARSET_INFO const*, unsigned char const*, unsi... (/Users/laurynas/vilniusdb/mysql-8.0.33/strings/ctype-uca.cc)
   429 ms: spec_adder(rapidjson::GenericDocument<rapidjson::UTF8<char>, rapidjs... (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/rest_routing/src/rest_routing_plugin.cc)
   384 ms: init_handlers(mysql_harness::PluginFuncEnv*, mysql_harness::LoaderCo... (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/harness/src/logging/logger_plugin.cc)
   351 ms: rapidjson::internal::Schema<rapidjson::GenericSchemaDocument<rapidjs... (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/harness/src/dynamic_state.cc)
   304 ms: my_strnxfrm_uca_900(CHARSET_INFO const*, unsigned char*, unsigned lo... (/Users/laurynas/vilniusdb/mysql-8.0.33/strings/ctype-uca.cc)
   276 ms: spec_adder(rapidjson::GenericDocument<rapidjson::UTF8<char>, rapidjs... (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/rest_metadata_cache/src/rest_metadata_cache_plugin.cc)
   271 ms: my_hash_sort_uca_900(CHARSET_INFO const*, unsigned char const*, unsi... (/Users/laurynas/vilniusdb/mysql-8.0.33/strings/ctype-uca.cc)
   250 ms: test_sql(void*) (/Users/laurynas/vilniusdb/mysql-8.0.33/plugin/test_service_sql_api/test_sql_lock.cc)
   247 ms: CmdArgHandler::process(std::__1::vector<std::__1::basic_string<char,... (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/harness/src/arg_handler.cc)
   242 ms: open_table_def(THD*, TABLE_SHARE*, dd::Table const&) (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/dd_table_share.cc)
   237 ms: trx_undo_report_row_operation(unsigned long, unsigned long, que_thr_... (/Users/laurynas/vilniusdb/mysql-8.0.33/storage/innobase/trx/trx0rec.cc)
   230 ms: test_wl4435_3() (/Users/laurynas/vilniusdb/mysql-8.0.33/testclients/mysql_client_test.cc)
   212 ms: void std::__1::__introsort<std::__1::_ClassicAlgPolicy, boost::geome... (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/gis/overlaps.cc)
   211 ms: dd::tables::Tables::Tables() (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/dd/impl/tables/tables.cc)
   210 ms: spec_adder(rapidjson::GenericDocument<rapidjson::UTF8<char>, rapidjs... (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/rest_connection_pool/src/rest_connection_pool_plugin.cc)
   209 ms: TlsServerContext::default_ciphers() (/Users/laurynas/vilniusdb/mysql-8.0.33/router/src/harness/src/tls_server_context.cc)
   209 ms: main (/Users/laurynas/vilniusdb/mysql-8.0.33/client/mysqltest.cc)
   207 ms: build_gcs_parameters(Gcs_interface_parameters&) (/Users/laurynas/vilniusdb/mysql-8.0.33/plugin/group_replication/src/plugin.cc)
   205 ms: init_server_components() (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/mysqld.cc)
   202 ms: void std::__1::__introsort<std::__1::_ClassicAlgPolicy, boost::geome... (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/gis/overlaps.cc)
   200 ms: ConnectJoins(int, int, int, QEP_TAB*, THD*, CallingContext, std::__1... (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/sql_executor.cc)
   194 ms: lock_wait_timeout_thread() (/Users/laurynas/vilniusdb/mysql-8.0.33/storage/innobase/lock/lock0wait.cc)
   193 ms: void std::__1::__introsort<std::__1::_ClassicAlgPolicy, boost::geome... (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/gis/crosses.cc)
   187 ms: (anonymous namespace)::CostingReceiver::FoundSingleNode(int) (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/join_optimizer/join_optimizer.cc)
   186 ms: handle_slave_io (/Users/laurynas/vilniusdb/mysql-8.0.33/sql/rpl_replica.cc)

**** Function sets that took longest to compile / optimize:
 13251 ms: bool boost::geometry::detail::partition::partition_two_ranges<$>::ap... (562 times, avg 23 ms)
  4962 ms: void std::__1::__introsort<$>(boost::geometry::detail::overlay::turn... (46 times, avg 107 ms)
  2395 ms: bool boost::geometry::detail::partition::partition_two_ranges<$>::ap... (104 times, avg 23 ms)
  2264 ms: std::__1::ostreambuf_iterator<$> std::__1::__pad_and_output<$>(std::... (536 times, avg 4 ms)
  1943 ms: void std::__1::__introsort<$>(boost::geometry::detail::relate::linea... (26 times, avg 74 ms)
  1717 ms: std::__1::basic_ostream<$>& std::__1::__put_character_sequence<$>(st... (536 times, avg 3 ms)
  1657 ms: std::__1::basic_stringbuf<$>::str() const (180 times, avg 9 ms)
  1298 ms: unsigned int std::__1::__sort3<$>(boost::geometry::detail::overlay::... (46 times, avg 28 ms)
  1248 ms: bool boost::geometry::partition<$>::apply<$>(boost::geometry::sectio... (281 times, avg 4 ms)
  1224 ms: unsigned int std::__1::__sort5<$>(boost::geometry::detail::overlay::... (46 times, avg 26 ms)
  1220 ms: void std::__1::__tree_balance_after_insert<$>(std::__1::__tree_node_... (317 times, avg 3 ms)
  1211 ms: std::__1::deque<$>::__add_back_capacity() (94 times, avg 12 ms)
  1049 ms: bool boost::geometry::detail::overlay::get_turn_info_for_endpoint<$>... (52 times, avg 20 ms)
   991 ms: bool boost::geometry::detail::partition::partition_one_range<$>::app... (104 times, avg 9 ms)
   968 ms: spec_adder(rapidjson::GenericDocument<$>&) (4 times, avg 242 ms)
   968 ms: std::__1::__tree_node_base<$>*& std::__1::__tree<$>::__find_equal<$>... (253 times, avg 3 ms)
   956 ms: bool boost::geometry::detail::partition::partition_two_ranges<$>::ap... (64 times, avg 14 ms)
   939 ms: (anonymous namespace)::Function_factory<$>::create_func(THD*, MYSQL_... (270 times, avg 3 ms)
   938 ms: std::__1::basic_stringbuf<$>::overflow(int) (180 times, avg 5 ms)
   922 ms: classic_protocol::Codec<$>::decode(net::const_buffer const&, std::__... (70 times, avg 13 ms)
   884 ms: bool boost::geometry::detail::partition::partition_two_ranges<$>::ap... (40 times, avg 22 ms)
   866 ms: boost::geometry::policies::relate::segments_intersection_policy<$>::... (30 times, avg 28 ms)
   864 ms: std::__1::basic_string<$>::basic_string[abi:v15006]<$>(char const*) (351 times, avg 2 ms)
   837 ms: rapidjson::internal::Schema<$>::Schema(rapidjson::GenericSchemaDocum... (5 times, avg 167 ms)
   814 ms: bool std::__1::__insertion_sort_incomplete<$>(boost::geometry::detai... (46 times, avg 17 ms)
   813 ms: void std::__1::__introsort<$>(boost::geometry::detail::relate::linea... (14 times, avg 58 ms)
   810 ms: std::__1::__tree<$>::destroy(std::__1::__tree_node<$>*) (443 times, avg 1 ms)
   808 ms: unsigned int std::__1::__sort4<$>(boost::geometry::detail::overlay::... (46 times, avg 17 ms)
   795 ms: void std::__1::__tree_remove<$>(std::__1::__tree_node_base<$>*, std:... (90 times, avg 8 ms)
   788 ms: std::__1::vector<$>::~vector[abi:v15006]() (418 times, avg 1 ms)

*** Expensive headers:
181934 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/sql/sql_class.h (included 587 times, avg 309 ms), included via:
  mysql_thd_attributes_imp.cc.o  (936 ms)
  replicated_columns_view.cc.o replicated_columns_view.h column_filter_factory.h column_filter_inbound_func_indexes.h column_filter.h  (910 ms)
  mysql_connection_attributes_iterator_imp.cc.o  (838 ms)
  ndb_create_helper.cc.o  (812 ms)
  sql_class.cc.o  (810 ms)
  replicated_columns_view_with_gipk_on_source.cc.o replicated_columns_view_with_gipk_on_source.h replicated_columns_view.h column_filter_factory.h column_filter_inbound_func_indexes.h column_filter.h  (771 ms)
  ...

140903 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/include/m_string.h (included 1793 times, avg 78 ms), included via:
  my_print_defaults.cc.o  (428 ms)
  my_strtoll10.cc.o  (412 ms)
  file.cc.o buf0checksum.h buf0types.h os0event.h univ.i  (406 ms)
  NdbThread.cpp.o ndb_global.h  (389 ms)
  AccLock.cpp.o AccLock.hpp SignalData.hpp ndb_global.h  (386 ms)
  NdbReceiver.cpp.o API.hpp ndb_global.h  (382 ms)
  ...

129393 ms: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/__functional/boyer_moore_searcher.h (included 2549 times, avg 50 ms), included via:
  bgc_ticket_manager.h atomic_bgc_ticket_guard.h functional  (231 ms)
  acl_table_user.h functional  (228 ms)
  config_parser.h functional  (217 ms)
  dynamic_privilege_table.h functional  (217 ms)
  loader_config.h config_parser.h functional  (210 ms)
  random_generator.h random discrete_distribution.h numeric functional  (208 ms)
  ...

114343 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/include/lex_string.h (included 1195 times, avg 95 ms), included via:
  sql_authentication.cc.o sql_authentication.h  (329 ms)
  audit_api_connection_service_imp.cc.o sql_audit.h  (308 ms)
  dd_trigger.cc.o dd_trigger.h  (305 ms)
  sql_hints.yy.cc.o  (299 ms)
  sql_user_table.cc.o sql_user_table.h sql_system_table_check.h log_builtins.h log.h  (297 ms)
  show_query_builder.cc.o show_query_builder.h  (292 ms)
  ...

105206 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/sql/field.h (included 634 times, avg 165 ms), included via:
  table_access_service.cc.o  (733 ms)
  rpl_sys_table_access.cc.o rpl_sys_table_access.h  (611 ms)
  sql_select.cc.o sql_select.h  (597 ms)
  field.cc.o  (597 ms)
  row.cc.o  (581 ms)
  lob0update.cc.o  (577 ms)
  ...

95348 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/sql/table.h (included 779 times, avg 122 ms), included via:
  dd_routine.cc.o dd_routine.h  (633 ms)
  global.cc.o global.h  (582 ms)
  table.cc.o  (551 ms)
  zlob0update.cc.o  (493 ms)
  rpl_sys_key_access.cc.o rpl_sys_key_access.h  (488 ms)
  table_replication_group_member_actions.cc.o rpl_sys_key_access.h  (369 ms)
  ...

94011 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/storage/innobase/include/univ.i (included 209 times, avg 449 ms), included via:
  mach0data.cc.o mach0data.h mtr0types.h sync0rw.h  (710 ms)
  buf0block_hint.cc.o buf0block_hint.h buf0types.h os0event.h  (705 ms)
  btr0sea.cc.o btr0sea.h  (700 ms)
  srv0tmp.cc.o srv0tmp.h srv0srv.h buf0checksum.h buf0types.h os0event.h  (684 ms)
  file.cc.o buf0checksum.h buf0types.h os0event.h  (681 ms)
  btr0btr.cc.o btr0btr.h btr0types.h  (674 ms)
  ...

93077 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/sql/log_event.h (included 241 times, avg 386 ms), included via:
  global.cc.o global.h context.h  (889 ms)
  log_event.cc.o  (879 ms)
  context.cc.o context.h  (862 ms)
  recovery.cc.o recovery.h global.h context.h  (811 ms)
  iterators.cc.o iterators.h binlog_reader.h  (775 ms)
  binlog_istream.cc.o  (775 ms)
  ...

91706 ms: /Users/laurynas/vilniusdb/mysql-8.0.33/sql/handler.h (included 842 times, avg 108 ms), included via:
  ha_example.cc.o ha_example.h  (756 ms)
  ha_mock.cc.o ha_mock.h  (685 ms)
  page_track_service.cc.o  (649 ms)
  ha_tina.cc.o ha_tina.h  (614 ms)
  ha_archive.cc.o ha_archive.h  (580 ms)
  api0misc.cc.o api0misc.h  (575 ms)
  ...

87795 ms: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/__memory/shared_ptr.h (included 2580 times, avg 34 ms), included via:
  AccLock.hpp SignalData.hpp ndb_global.h m_string.h algorithm memory  (102 ms)
  dynamic_privilege_table.h functional boyer_moore_searcher.h  (98 ms)
  signing_key.h string memory  (97 ms)
  ndb_global.h m_string.h algorithm memory  (96 ms)
  dict0dict.h set __node_handle memory  (93 ms)
  my_byteorder.h template_utils.h algorithm memory  (93 ms)
  ...

  done in 0.8s.

What changed?

• "Files that took longest to parse (compiler frontend)": all GIS
• "Files that took longest to codegen (compiler backend)": all GIS
• "Templates that took longest to instantiate": all those std::function gone, so must have been tests. Something about MDL remaining.
• "Template sets that took longest to instantiate": std::unique_ptr. Oh well.
• "Functions that took longest to compile": MYSQLparse (no surprise there), system variables handling, 3rd party code, and variety of other things
• "Function sets that took longest to compile / optimize:": Boost/GIS, the standard library.
• "Expensive headers": sql_class.h, and the C++ standard library headers mostly gone together with shared_ptr (so it was tests using it).

Interestingly (and usefully) this provides a different kind of insight than what Mark discovered for Meta branch: it would not discover RocksDB being compiled three times. It would be great if the MySQL developers looked into this tool, which seems very easy to use, and reducing compilation times. That sql_class.h file is way overdue for splitting!