How to Deploy MariaDB Cluster 10.5 for High Availability

ClusterControl 1.8.1 includes support for MariaDB Cluster 10.5. MariaDB 10.5 is equipped with.

• More Granular Privileges
• InnoDB Performance Improvements
• Full GTID Support for Galera Cluster
• More Metadata for Replication and Binary Logs
• More SQL syntax statements are introduced (RETURNING statement to INSERT, EXCEPT ALL and INTERSECT ALL, …)
• Performance Schema Updates to Match MySQL 5.7
• The S3 Storage Engine

You can check further on our previous blog What’s New in MariaDB Server 10.5? to know more details about the release. With the specific updates for MariaDB Cluster, in version 10.5, the key features don’t differ much from MariaDB Cluster 10.4, but the list below are some of the important changes in this version.

• GTID consistency 
• Cluster inconsistency/error voting 
• non-blocking DDL operations (available only on enterprise version)
• Black box (available only on enterprise version) 
• Upgraded its Galera wsrep Library for which 26.4.6 is the latest version

XA Transaction Support is expected on this release ( do not be confused as XA Transactions are supported by MariaDB Server but not on Galera Cluster) but due to some reasons it's delayed and shall be expected on the next MariaDB Cluster 10.6 release.

MariaDB Cluster For High Availability

The MariaDB Cluster is basically a Galera Cluster that uses the MariaDB implementation as the database layer to interface with Innodb or XtraDB engine. MariaDB Galera Cluster is a virtually synchronous multi-master cluster for MariaDB. It is available on Linux only, and only supports the XtraDB/InnoDB storage engines (although there is experimental support for MyISAM - see the wsrep_replicate_myisam system variable). When Galera Cluster is in use, database reads and writes can be directed to any node. Any individual node can be lost without interruption in operations and without using complex failover procedures.

With Galera's nature adapted within MariaDB Cluster, it's basically a high availability solution with synchronous replication, failover, and resynchronization. It brings the benefits of no data loss, no slave lag, read and write scalability, and does high availability on different data centers.

Deploying MariaDB Cluster 10.5

MariaDB provides a very straightforward and easy setup for installing your MariaDB Cluster 10.5. The manual process can be tedious but with automated scripts provided by MariaDB, repositories can be setup in accordance to your target database version, OS type, and OS version.

For this exercise, I have the following 3-node Galera Cluster setup with the following IP addresses: 192.168.40.210, 192.168.40.220, 192.168.40.230.

Setup Your Repository

As mentioned earlier, MariaDB has a script named mariadb_repo_setup. It's very straightforward and easy to use. You can specify the target version of your database, the OS type, and the version of your OS. 

For example, I am installing using CentOS 8,

curl -LsS https://downloads.mariadb.com/MariaDB/mariadb_repo_setup |  sudo bash -s -- --mariadb-server-version="mariadb-10.5" --os-type=rhel --os-version=8

or installing it in Ubuntu Focal Fossa,

curl -LsS https://downloads.mariadb.com/MariaDB/mariadb_repo_setup |  sudo bash -s -- --mariadb-server-version="mariadb-10.5" --os-type=ubuntu --os-version=focal

Take note that when using mariadb_repo_setup script, it requires the package apt-transport-https as a dependency. So install this package first before you can take advantage of the mariadb_repo_setup script. 

apt update

apt install apt-transport-https

Now, run the command to your three nodes in accordance to their OS and of course the MariaDB version has to be 10.5.

Setup MySQL Configuration

Configuration file depends on your server resources, type of server environment, and assigned IP address. For this blog, you can use this production ready MariaDB Cluster/PXC configuration setup which we used to deploy in our Percona XtraDB Cluster/MariaDB Cluster databases using ClusterControl. Notable variables that you need or subject to change are the following:

• innodb_buffer_pool_size - set the buffer pool from 70% - 80% available RAM of your server
• wsrep_provider - Path of the Galera compiled library. For RHEL/CentOS, the path shall be  /usr/lib64/galera-4/libgalera_smm.so. Whereas Debian/Ubuntu is in /usr/lib/galera/libgalera_smm.so.
• wsrep_node_address - This is the node IP address
• wsrep_sst_method - either you can change it but we recommend you use mariabackup. Possible values you can choose are rsync, mysqldump, xtrabackup, xtrabackup-v2, mariabackup.
• wsrep_cluster_name - The name of your MariaDB Cluster. It has to be identical to all your nodes in a single MariaDB Cluster.
• wsrep_cluster_address - This contains the addresses of your nodes within the cluster. It has to be a valid IP, hostname, or FQDN.
• wsrep_node_name - The name of your node. The name can be used in wsrep_sst_donor as a preferred donor. Note that multiple nodes in a cluster can have the same name.

For performing SST, the user and password for the following sections [mysqldump], [xtrabackup], and [mysqld] can change if you want. For this exercise, let's keep it simple and you can just let the values as is.

Now, copy the configuration file and place it to /etc/my.cnf. Do this to all of your three Galera nodes.

Installing Required Packages

Install the packages for all the three Galera nodes. Follow the command below based on your target OS environment.

For RHEL/CentOS,

sudo yum install MariaDB-server MariaDB-client galera-4 MariaDB-backup

For Debian/Ubuntu,

sudo apt update

sudo apt-get install mariadb-server galera-4 mariadb-client libmariadb3 mariadb-backup mariadb-common

Once installation is done, stop the MariaDB process and initialize the cluster as a single node.This shall bootstrap your Galera Cluster. At this stage, I'm running it on node 192.168.40.210,

$ /usr/bin/galera_new_cluster

Create SST/IST User

Create the backup user which shall be used for SST or IST. Only run the following SQL statements below on the first node that you have initiated the cluster. At this stage, I have executed it in node 192.168.40.210.

CREATE USER backupuser@localhost IDENTIFIED BY 'backuppassword';

GRANT PROCESS, RELOAD, LOCK TABLES, REPLICATION CLIENT ON *.* TO 'backupuser'@'localhost';

At this point, we're already setup and ready to connect the remaining nodes.

Start The MariaDB Server

Now we have already setup the first node. We are ready to connect the remaining nodes. Simply, just start the mariadb service. Just run the command below,

systemctl start mariadb

Run the command for the remaining nodes. Do it one at a time.

At this point, all nodes are in synced and we can verify with the following SQL statement as follows,

Adding Steroids For Your MariaDB Cluster 10.5 for HA

Likely, in production setup, deploying the MariaDB Cluster 10.5 for HA might not suffice your needs. Adding more steroids such as installing HAProxy together with Keepalived for your redundancy, will bring more high availability for your database environment.

Setting up your HAProxy and then Keepalived adds more hassle to obtain the desired topology and environment. This can be achieved with automation using ClusterControl. Using ClusterControl allows you to deploy MariaDB Cluster 10.5 and add more solutions such as ProxySQL, MaxScale, or garbd for load balancing. While you can add Keepalived to your cluster to add redundancy and auto failover solutions in case of disaster happens.

You can download ClusterControl, which is free and allows you to deploy these HA solutions for free and manage your database giving you a 30-day subscription license to enjoy the benefits of advanced and enterprise features.

Deploying MariaDB 10.5 Cluster with ClusterControl

Once you have installed ClusterControl, click the icon from the upper right corner and you'll see the deployment Wizard just like below.

You can setup easily. Just follow the series of steps based on the flow of the UI,

Deploy HAProxy For Load Balancing Management

At this point, I would assume you have your MariaDB Cluster 10.5 all setup. Now, lets deploy the HAProxy,

Alternatively, you can go to Manage → Load Balancer → HAProxy.

Then select or type the address where the HAProxy to be installed and select your Galera nodes that have to be monitored by the HAProxy. See example below,

Add at least two deployments of HAProxy to add more availability such that whenever one of your HAProxy goes down, your application will route over to the other node that is still available or online. This is very important especially when you are handling database or system upgrades, aside from catastrophic or disaster events.

Deploy Keepalived

The same process you can use with how you deploy HAProxy with Clustercontrol. Below is a perfect example on you are going to deploy Keepalived,

If you noticed, I have two HAProxy instances for which I am going to install my Keepalived which shall be present in every node where the HAProxy is running.

Finalizing your MariaDB Cluster 10.5 With High Availability

Now that we have already setup, you have environment which shall look like this,

Conclusion

This setup gives you the benefits of achieving high availability with more 9's so to speak. HAProxy gives your MariaDB Cluster 10.5 more load balancing capability with its read and write separation. Then, Keepalived assures that in case one of your HAProxy dies, Keepalived assures that it will fail over to the next one available. Your application shall only connect to the virtual IP address (which follows the VRRP) and no extra configuration or setup to be done in case of such HAProxy instance failure occurs.

Of course, this setup is not perfect when you speak of high availability. You can replace HAProxy with ProxySQL to add more flexibility and add more read/write separation by just playing only in one port. Achieving a perfect setup for HA is hard to achieve and comes with drawbacks as well. But this setup, it gives you more high presence from your application and database interoperability. What matters most is that a lower downtime to a possible no downtime has to be achieved.