How To Learn Real Application Clusters (RAC) Administration

This article provides some ideas about how someone new to Real Application Clusters (RAC) should approach trying to learn about it. These are just ideas, not a definitive guide.

• Before You Begin
• Reading
• Virtualization
• Automatic Storage Manager (ASM)
• Basic RAC Installation
• Test Scenarios

Before You Begin

Your primary focus should be on being a good DBA before you look at RAC. Despite what you may believe, many of the issues you face in RAC are the same as those of single instance databases. If you are a weak DBA before embarking on the process of learning RAC, you are not going to have a fun time.

The rest of the article assumes you are a competent single-instance DBA.

Reading

Before, during and after, you should be reading as much as possible about RAC. The following manuals are a must!

• Database 2 Day + Real Application Clusters Guide
• Automatic Storage Management Administrator's Guide
• Clusterware Administration and Deployment Guide
• Real Application Clusters Administration and Deployment Guide

The more you've read about the theory, the easier the practical will be. Ultimately, you need both.

Virtualization

Get to grips with basic virtualization using VirtualBox. It's free and you'll probably need it if you are planning on doing RAC installations at home. You will need to be comfortable with adding additional disks and network interfaces, because RAC installations will require multiples of both. In order to do this you will have to be familiar with how the OS handles multiple disks and network adapters.

Assuming you are using Linux, it would make sense to become more familiar with basic Linux system administration before moving on. The combination of virtualization and Linux system administration will help a lot when doing RAC installations and diagnosing issues. You can read about basic Linux administration here.

Automatic Storage Manager (ASM)

Make sure you have a decent grasp of Automatic Storage Manager (ASM) before you move on. ASM is mandatory for RAC on Standard Edition (SE). For Enterprise Edition (EE) you can use other forms of shared storage that don't require ASM, but the likelihood is you will use ASM with RAC on EE too. Most of the important ASM skills can be learnt on single instance systems. Some things to try out are listed below.

• Create a VM and install the Oracle software.
• Add several virtual hard drives to a VM.
• Configure them to be ASM disks.
• Practice adding and removing disks within ASM.
• Define failure groups. Most of the time you will use hardware RAID for your LUNs, so failure groups will not be necessary, but it's a good idea to understand the ASM redundancy model anyway.
• Practice backup/recovery using ASM.
• Practice disaster recovery using ASM.
• Practice Data Guard between databases using ASM.

Many of these topics are described in the ASM Oracle documentation here.

Basic RAC Installation

You are probably ready at this point to do some basic RAC installations. You can find examples of RAC installations on VirtualBox here. If you've done the groundwork mentioned previously, it should go really well. If you've rushed to this point, it will probably be a painful process.

Don't assume one successful installation means you are ready to move on. Try installing different versions of RAC on different operating system versions. You will hit problems, but treat these as learning experiences.

Test Scenarios

Once you've got a basic RAC installation done, you will want to start testing RAC functionality. Here are some things, in no particular order, you should probably try out.

• Clean shutdown of a node. Check what happens in the remaining nodes log files.
• Power off a node. Check what happens in the remaining nodes log files.
• Turn off the network adapter for the interconnect on one node. Check what happens in the remaining nodes log files.
• Kill a RAC database instance and check the results.
• Kill an ASM instance and check the results.
• Check session failover during the above failures scenarios.
• Check loss and recovery of OCR and voting disks.
• Remove a node from the cluster.
• Add a node to the cluster.
• Practice patches/upgrades of the installation. You should try rolling and non-rolling patches.
• Practice backup/recovery of a RAC database. Block media recovery. Datafile and tablespace recovery. Point in Time Recovery (PITR).
• Practice DR of a single RAC instance. What happens if you lose a single server from the cluster and have to rebuild it from backups or from scratch?
• Practice DR of a whole RAC database. What happens if you lose everything. Can you recover the whole cluster from backups?
• Set up Data Guard between you RAC database and a standby RAC. This might be difficult depending on the kit you have available. Running two RAC systems on one PC can be hard work. :)
• Run a heavy load against the RAC database and see how the work gets spread across the nodes. Use Swingbench to supply the load.
• Direct load to specific nodes using node-specific services. Many beginners forget that RAC is not great at coping with multiple nodes fighting over the same blocks repeatedly. You will often need to partition the workloads to different nodes using node-specific services to prevent interconnect-thrashing.

Many of these scenarios are covered in the RAC documentation linked below.

For more information see:

• Database 2 Day + Real Application Clusters Guide
• Automatic Storage Management Administrator's Guide
• Clusterware Administration and Deployment Guide
• Real Application Clusters Administration and Deployment Guide
• Learning to be an Oracle Database Administrator (DBA)
• Swingbench

Hope this helps. Regards Tim.

Created: 2016-07-28 Updated: 2016-08-01