Thursday, January 11, 2024

MySQL clone plugin internals and MyRocks clone design

I just realized that about MySQL and MyRocks clone I never actually published anything more serious than a single-emoji post on Facebook, a link to an Oracle umbrella bug, and this tweet.

So, let's talk about clone. MySQL has a clone plugin which can be used to copy new instances from existing ones, and it's also integrated into group replication for the same purpose. In Oracle releases this plugin copies only InnoDB tables, making the feature unsuitable for MyRocks instances.

Now MyRocks is the first storage engine, besides InnoDB, to get the clone support, and it works on mixed MyRocks/InnoDB instances while ensuring that the cloned instances are consistent across engines too. The code is in Meta's branch, I don't believe there are any user docs (but MyRocks support is so seamless that Oracle docs suffice! Only half-joking here), but there are IMHO extensive internals docs at Meta's wiki: MyRocks Clone Plugin

Their scope is broader than the title might suggest. Not only the MyRocks clone design is discussed, but there is also a clone background section, which discusses how clone works internally and fully applies to the Oracle branches too.

Like with all things 3rd party storage engines, it is rare to develop a feature without having to patch the server (or in this case server, clone, & InnoDB plugins). The details of this patch are also discussed in the Wiki, and also in the aforementioned Oracle umbrella bug.

Last but not least, with background and patches elsewhere out of the way, the Wiki has the design of MyRocks clone proper.

I hope the feature will reach MyRocks downstreams one day. Since MariaDB currently has no clone plugin, that leaves Percona Server. Maybe these docs will help with the porting, and also for advanced end-user troubleshooting. Clone away!

No comments: