Oracle Technologies Blog

By ASKM

11gR2 – RAC Shared Storage Preparation(ASM) – Part1

Posted by Srikrishna Murthy Annam on March 18, 2010

RAC storage Options

  1. ASM Storage
  2. OCFS (Release 1 or 2)
  3. NFS ( NAS or SAN )
  4. Raw Devices
  5. Third party cluster filesystem such as GPFS or Veritas

Preparing ASM storate for clusterware

  • Partition the Shared Disks
  • Installing and Configuring ASMLib
  • Using ASMLib to Mark the Shared Disks as Candidate Disks

For this installation we will be using ASM for Clusterware and Database storage on top of SAN technology. The following Table shows the storage layout for this implementation:

Block Device     ASMlib Name     Size     Comments
/dev/sda     OCR_VOTE01     1 GB     ASM Diskgroup for OCR and Voting Disks
/dev/sdb     OCR_VOTE02     1 GB     ASM Diskgroup for OCR and Voting Disks
/dev/sdc     OCR_VOTE03     1 GB     ASM Diskgroup for OCR and Voting Disks
/dev/sdd     ASM_DATA01     2 GB     ASM Data Diskgroup
/dev/sde     ASM_DATA02     2 GB     ASM Data Diskgroup
/dev/sdf     ASM_DATA03     2 GB     ASM Data Diskgroup
/dev/sdg     ASM_DATA04     2 GB     ASM Data Diskgroup
/dev/sdh     ASM_DATA05     2 GB     ASM Flash Recovery Area Diskgroup
/dev/sdi     ASM_DATA06     2 GB     ASM Flash Recovery Area Diskgroup
/dev/sdj     ASM_DATA07     2 GB     ASM Flash Recovery Area Diskgroup
/dev/sdk     ASM_DATA08     2 GB     ASM Flash Recovery Area Diskgroup

I.Partition the Shared Disks
As root user on Node1, run the following command

#fdisk /dev/sda
Command (m for help): u
Changing display/entry units to sectors

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First sector (61-1048575, default 61): 2048
Last sector or +size or +sizeM or +sizeK (2048-1048575, default 1048575):
Using default value 1048575

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Now Load the updated block device partition tables by running the following on ALL servers participating in the cluster:

#/sbin/partprobe

NOTE : Repeat above two steps for all the disks and create partitions as per the table mentioned above.

II.Installing and Configuring ASMLib :

The ASMLib is highly recommended for those systems that will be using ASM for shared storage within the cluster due to the performance and managability benefits that it provides. Perform the following steps to install and configure ASMLib on the cluster nodes:

NOTE: ASMLib automatically provides LUN persistence, so when using ASMLib there is no need to manually configure LUN persistence for the ASM devices on the system.

1. Download the following packages from the ASMLib OTN page, if you are a Enterprise Linux customer you can obtain the software through the Unbreakable Linux network.

NOTE: The ASMLib kernel driver MUST match the kernel revision number, the kernel revision number of your system can be identified by running the “uname -r” command. Also, be sure to download the set of RPMs which pertain to your platform architecture, in our case this is x86_64.

oracleasm-support-2.1.3-1.el5x86_64.rpm
oracleasmlib-2.0.4-1.el5.x86_64.rpm
oracleasm-2.6.18-92.1.17.0.2.el5-2.0.5-1.el5.x86_64.rpm

2. Install the RPMs by running the following as the root user:

# rpm -ivh oracleasm-support-2.1.3-1.el5x86_64.rpm \
oracleasmlib-2.0.4-1.el5.x86_64.rpm \
oracleasm-2.6.18-92.1.17.0.2.el5-2.0.5-1.el5.x86_64.rpm

3. Configure ASMLib by running the following as the root user:

NOTE: If using user and group separation for the installation (as shown in this guide), the ASMLib driver interface owner is grid and the group to own the driver interface is asmdba (oracle and grid are both members of this group). These groups were created in section 2.1. If a more simplistic installation using only the Oracle user is performed, the owner will be oracle and the group owner will be dba.

#/etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library driver. The following questions will determine whether the driver is loaded on boot and what permissions it will have. The current values will be shown in brackets (‘[]’). Hitting <ENTER> without typing an answer will keep that current value. Ctrl-C will abort.

Default user to own the driver interface []: grid
Default group to own the driver interface []: asmdba
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver: [ OK ]
Scanning the system for Oracle ASMLib disks: [ OK ]

4. Repeat steps 2 – 4 on ALL cluster nodes.

III.Using ASMLib to Mark the Shared Disks as Candidate Disks
To create ASM disks using ASMLib:
1. As the root user, use oracleasm to create ASM disks using the following syntax:

# /usr/sbin/oracleasm createdisk disk_name device_partition_name

In this command, disk_name is the name you choose for the ASM disk. The name you choose must contain only ASCII capital letters, numbers, or underscores, and the disk name must start with a letter, for example, DISK1 or VOL1, or RAC_FILE1. The name of the disk partition to mark as an ASM disk is the device_partition_name. For example:

# /usr/sbin/oracleasm createdisk OCR_VOTE01 /dev/sda1
# /usr/sbin/oracleasm createdisk OCR_VOTE02 /dev/sdb1
# /usr/sbin/oracleasm createdisk OCR_VOTE03 /dev/sdc1
# /usr/sbin/oracleasm createdisk ASMDATA01 /dev/sdd1
# /usr/sbin/oracleasm createdisk ASMDATA02 /dev/sde1
# /usr/sbin/oracleasm createdisk ASMDATA03 /dev/sdf1
# /usr/sbin/oracleasm createdisk ASMDATA04 /dev/sdg1
# /usr/sbin/oracleasm createdisk ASMDATA05 /dev/sdh1
# /usr/sbin/oracleasm createdisk ASMDATA06 /dev/sdi1
# /usr/sbin/oracleasm createdisk ASMDATA07 /dev/sdj1
# /usr/sbin/oracleasm createdisk ASMDATA08 /dev/sdk1

If you need to unmark a disk that was used in a createdisk command, you can use the following syntax as the root user:

# /usr/sbin/oracleasm deletedisk disk_name

2. Repeat step 1 for each disk that will be used by Oracle ASM.

3. After you have created all the ASM disks for your cluster, use the listdisks command to verify their availability:

# /usr/sbin/oracleasm listdisks
OCR_VOTE01
OCR_VOTE02
OCR_VOTE03
ASMDATA01
ASMDATA02
ASMDATA03
ASMDATA04
ASMDATA05
ASMDATA06
ASMDATA07
ASMDATA08

4. On all the other nodes in the cluster, use the scandisks command as the root user to pickup the newly created ASM disks. You do not need to create the ASM disks on each node, only on one node in the cluster.

# /usr/sbin/oracleasm scandisks
Scanning system for ASM disks [ OK ]

5. After scanning for ASM disks, display the available ASM disks on each node to verify their availability:

# /usr/sbin/oracleasm listdisks
OCR_VOTE01
OCR_VOTE02
OCR_VOTE03
ASMDATA01
ASMDATA02
ASMDATA03
ASMDATA04
ASMDATA05
ASMDATA06
ASMDATA07
ASMDATA08

ASMLib installation
Configuring ASM disks

About these ads

12 Responses to “11gR2 – RAC Shared Storage Preparation(ASM) – Part1”

  1. abc said

    Hi i have confusion in choosing the rpms to be installed.
    Can you pls help me to choose the rpms for RHEL5

    Thanks

  2. learnwithme11g said

    All ASMLib installations require the oracleasmlib and oracleasm-support packages appropriate for their machine. The driver packages are named after the kernel they support. Run the ” uname -r” command on your machine to determine your kernel version. The corresponding package has the name oracleasm-.

  3. Anish said

    Srikrishna,

    Great post. Just one question is there a reason the videos are private on youtube? I wanted to double check my own ASM config steps with your own.

  4. learnwithme11g said

    Actually No. I prepared these demos on live env and i dont want these details public just for that cause. When i have these demos prepared on my own setup, i wil replace them. You can also see the other demos which are public and they are prepared on my environments. Hope you can understand the confidentiality.

    Thanks
    SRI

  5. jdbaig said

    Hi,

    It’s really been a good experience reading your blog you ! However i m unable to watch your
    Videos tutorials ! what should i do in order to see it ?

    Also could you kindly post some materials on Oracle apps r12 Workflow management & Cloning in EBS r12
    And i ‘ll be really obliged if u could email some Interview questions & ANs of Oracle EBS 11i/R12.

    Thanks & regards
    Jdbaig

  6. jdbaig said

    Hello,

    It’s really been a good experience reading your blog you ! However i m unable to watch your
    Videos tutorials ! what should i do in order to see it ?

    Also could you kindly post some materials on Oracle apps r12 Workflow management & Cloning in EBS r12
    And i ‘ll be really obliged if u could email some Interview questions & ANs of Oracle EBS 11i/R12.

    Thanks & regards
    Jdbaig

  7. Please let us view the videos. that would be of great help. You can hide the some sections..

  8. learnwithme11g said

    Hi Rajiv, Please let me know which video you are interested to watch ?

  9. ramesh said

    not able to watch the videos .

    Regards
    Ramesh

  10. Hi,

    I have Ubuntu 12.04 installed on my both the nodes where I want to create ASM + RAC. But Im not able to find ASMLibs for Ubuntu 12.04 and its kernel revision number. Im doing this in my test Lab and I dont have Unbreakable Linux Network access.

    So, what ASMLibs should I use ? Can we use Ubuntu for ASM? or I have to use other OS only?
    Please let me know your inputs…

    Thanks,
    Santosh

  11. learnwithme11g said

    Hi Santosh, There are no direct downloads of ASMLibs for Ubuntu. You have to recompile the source code from https://oss.oracle.com. If you feel it difficult, you can have UDEV SCSI Rules Configuration for ASM (/etc/udev/rules.d).
    Thanks
    SRI

  12. Raghavendra said

    Hi Krishna

    Can you please enable me to watch videos which are posted here. So i can clearly understand the things.

    Thanks for sharing info.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: