Oracle Technologies Blog

By ASKM

11gR2 – RAC Shared Storage Preparation on Window 2003

Posted by Srikrishna Murthy Annam on March 23, 2010

This article describes how to prepare the shared storage for Oracle RAC

  1. Shared Disk Layout
  2. Enable Automounting of disks on Windows 2003
  3. Clean the Shared Disks – Optional Step
  4. Create Logical partitions inside Extended partitions
  5. Drive Letters
  6. View Disks

1. Shared Disk Layout
It is assumed that the two nodes have local disk primarily for the operating system and the local Oracle Homes. Labelled C: The Oracle Clusterware software also resides on the local disks on each node. The 2 nodes must also share some central disks. This disk must not have cache enabled at the node level. i.e. if the HBA drivers support caching of reads/writes it should be disabled. If the SAN support cacheing that is visible to all nodes then this can be enabled.
At least 2 empty partitions are required for the Oracle Clusterware install using external mirroring/redundancy

  • One for the Oracle Cluster Registry (OCR).
  • One for the vote device

If the disk system used to provide the shared physical disk does not provide some form of RAID protection then Oracle recommends that you use the built in mirroring/redundancy capabilities provided by Oracle Clusterware for the required Clusterware RAW partitions

  • Two for the Oracle Cluster Registry (OCR).
  • Three for the vote devices

All these partitions should be a minimum of 256MB each. Because of space used by extended and logical partitions it is recommended that these LUNs be a minimum of 270MB. If you choose to use Oracle mirroring you should ensure that the partitions created for the mirrors are on separate physical devices.

The first disk (Disk 0) is the local C: drive used for the software install. The Storage Administrator has provisioned 14 LUNs from the shared disk array (Disk1 – Disk14)

  • Disk1 – Disk5 are used for Oracle Clusterware
  • Disk6 – Disk11 are used by ASM for the +DATA diskgroup
  • Disk12- Disk14 are used by ASM for the +FLASH diskgroup

In this document we will use the diskpart command line tool to manage these LUNs. You must create logical partitions inside extended partitions for the disks to be used by Oracle Clusterware and Oracle ASM. There must be no drive letters assigned to any of the Disks1 – Disk14 on any node. It is possible to use diskmgmt.msc instead of diskpart (as used in the following sections) to create these partitions.
2. Enable Automount
You must enable automounting of disks for them to be visible to Oracle Clusterware. On Each node logged in as someone with Administrator privileges then Click START->RUN and type diskpart

C:\>diskpart

Microsoft DiskPart version 5.2.3790.3959
Copyright (C) 1999-2001 Microsoft Corporation.
On computer: WINNODE1

DISKPART>AUTOMOUNT ENABLE

Repeat the above command on all nodes in the cluster
3. Clean the Shared Disks – Optional Step
You may want to clean your shared disks before starting the install. Cleaning will remove data from any previous failed install. But see a later Appendix for coping with failed installs. On Node1 from within diskpart you should clean each of the disks. WARNING this will destroy all of the data on the disk. Do not select the disk containing the operating system or you will have to reinstall the OS
Cleaning the disk ‘scrubs’ every block on the disk. This may take some time to complete.

DISKPART> list disk

Disk ###  Status      Size     Free     Dyn  Gpt
——–  ———-  ——-  ——-  —  —
Disk 0    Online      8189 MB  8033 KB
Disk 1    Online       298 MB   298 MB
Disk 2    Online       298 MB   298 MB
Disk 3    Online       298 MB   298 MB
Disk 4    Online       298 MB   298 MB
Disk 5    Online       298 MB   298 MB
Disk 6    Online      1279 MB  1279 MB
Disk 7    Online      1279 MB  1279 MB
Disk 8    Online      1279 MB  1279 MB
Disk 9    Online      1279 MB  1279 MB
Disk 10   Online      1279 MB  1279 MB
Disk 11   Online      1279 MB  1279 MB
Disk 12   Online      1279 MB  1279 MB
Disk 13   Online      1279 MB  1279 MB
Disk 14   Online      1279 MB  1279 MB

Now you should clean disks 1 – 14 (Not disk0 as this the local C: drive)

DISKPART>select disk 1
Disk 1 is now the selected disk.
DISKPART> clean all

DISKPART> select disk 2
Disk 2 is now the selected disk.
DISKPART> clean all

DISKPART> select disk 3
Disk 3 is now the selected disk.
DISKPART> clean all

DISKPART> select disk 4
Disk 4 is now the selected disk.
DISKPART> clean all

DISKPART> select disk 5
Disk 5 is now the selected disk.
DISKPART> clean all

DISKPART> select disk 6
Disk 6 is now the selected disk.
DISKPART> clean all

DISKPART> select disk 7
Disk 7 is now the selected disk.
DISKPART> clean all

DISKPART> select disk 8
Disk 8 is now the selected disk.
DISKPART> clean all

DISKPART> select disk 9
Disk 9 is now the selected disk.
DISKPART> clean all

DISKPART> select disk 10
Disk 10 is now the selected disk.
DISKPART> clean all

DISKPART> select disk 11
Disk 11 is now the selected disk.
DISKPART> clean all

DISKPART> select disk 12
Disk 12 is now the selected disk.
DISKPART> clean all

DISKPART> select disk 13
Disk 13 is now the selected disk.
DISKPART> clean all

DISKPART> select disk 14
Disk 14 is now the selected disk.
DISKPART> clean all

4. Create Logical partitions inside Extended partitions
Assuming the disks you are going to use are completely empty you must create an extended partition and then inside that partition a logical partition. In the following example, for Oracle Clusterware, I have dedicated LUNS for each device. This is not strictly necessary, you can combine an OCR & vote device onto one physical LUN, creating 2 partitions inside the extended partition although it would be foolish for example to place the OCR and OCRMIRROR devices on the same LUN.

DISKPART> select disk 1
Disk 1 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.

DISKPART> select disk 2
Disk 2 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.

DISKPART> select disk 3
Disk 3 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition..

DISKPART> select disk 4
Disk 4 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.

DISKPART> select disk 5
Disk 5 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.

DISKPART> select disk 6
Disk 6 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.

DISKPART> select disk 7
Disk 7 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.

DISKPART> select disk 8
Disk 8 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.

DISKPART> select disk 9
Disk 9 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.

DISKPART> select disk 10
Disk 10 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.

DISKPART> select disk 11
Disk 11 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.

DISKPART> select disk 12
Disk 12 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.

DISKPART> select disk 13
Disk 13 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.

DISKPART> select disk 14
Disk 14 is now the selected disk.
DISKPART> create part ext
DiskPart succeeded in creating the specified partition.
DISKPART> create part log
DiskPart succeeded in creating the specified partition.

4.1. View Created partitions

DISKPART> list disk

Disk ###  Status      Size     Free     Dyn  Gpt
——–  ———-  ——-  ——-  —  —
Disk 0    Online      8189 MB  8033 KB
Disk 1    Online       290 MB      0 B
Disk 2    Online       290 MB      0 B
Disk 3    Online       290 MB      0 B
Disk 4    Online       290 MB      0 B
Disk 5    Online       290 MB      0 B
Disk 6    Online      1271 MB      0 B
Disk 7    Online      1271 MB      0 B
Disk 8    Online      1271 MB      0 B
Disk 9    Online      1271 MB      0 B
Disk 10   Online      1271 MB      0 B
Disk 11   Online      1271 MB      0 B
Disk 12   Online      1271 MB      0 B
Disk 13   Online      1271 MB      0 B
* Disk 14   Online      1271 MB      0 B

5. List Drive Letters
Diskpart should not add drive letters to the partitions on the local node. The partitions on the other node may have drive letters assigned. You must remove them. On earlier versions of Windows 2003 a reboot of the ‘other’ node will be required for the new partitions to become visible. Windows 2003 SP2 does not suffer from this issue.

Using diskpart on Node2

DISKPART> list volume

Volume ###  Ltr  Label        Fs     Type        Size     Status     Info
———-  —  ———–  —–  ———-  ——-  ———  ——–
Volume 0     H                RAW    Partition    290 MB  Healthy
Volume 1     J                RAW    Partition   1271 MB  Healthy
Volume 2     O                RAW    Partition   1271 MB  Healthy
Volume 3     M                RAW    Partition   1271 MB  Healthy
Volume 4     D                       DVD-ROM         0 B  Healthy
Volume 5     L                RAW    Partition   1271 MB  Healthy
Volume 6     K                RAW    Partition   1271 MB  Healthy
Volume 7     C                NTFS   Partition   8182 MB  Healthy    System
Volume 8     E                RAW    Partition    290 MB  Healthy
Volume 9     N                RAW    Partition   1271 MB  Healthy
Volume 10    G                RAW    Partition    290 MB  Healthy
Volume 11    F                RAW    Partition    290 MB  Healthy
Volume 12    I                RAW    Partition    290 MB  Healthy
Volume 13    P                RAW    Partition   1271 MB  Healthy
Volume 14    Q                RAW    Partition   1271 MB  Healthy
Volume 15    R                RAW    Partition   1271 MB  Healthy

Notice that the volumes are listed in a completely different order compared to the disk list.
5.1. Remove Drive Letters
You need to remove the drive letters H J O M L K E N G F I P Q R These relate to volumes 0 1 2 3 5 6 8 9 10 11 12 13 14 15 Do NOT remove drive letters C & D which, in this case, are local disks (and CD-ROM drive) These relate to volumes 7 & 4 (In this example)

DISKPART> select volume 0
Volume 0 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.

DISKPART> select volume 1
Volume 1 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.

DISKPART> select volume 2
Volume 2 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.

DISKPART> select volume 3
Volume 3 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.

DISKPART> select volume 5
Volume 5 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.

DISKPART> select volume 6
Volume 6 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.

DISKPART> select volume 8
Volume 8 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.

DISKPART> select volume 9
Volume 9 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.

DISKPART> select volume 10
Volume 10 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.

DISKPART> select volume 11
Volume 11 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.

DISKPART> select volume 12
Volume 12 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.

DISKPART> select volume 13
Volume 13 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.

DISKPART> select volume 14
Volume 14 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.

DISKPART> select volume 15
Volume 15 is the selected volume.
DISKPART> remov
DiskPart successfully removed the drive letter or mount point.

5.2. List volumes on Second node
You should check that none of the RAW partitions have drive letters assigned

DISKPART> list vol

Volume ###  Ltr  Label        Fs     Type        Size     Status     Info
———-  —  ———–  —–  ———-  ——-  ———  ——–
Volume 0                      RAW    Partition    290 MB  Healthy
Volume 1                      RAW    Partition   1271 MB  Healthy
Volume 2                      RAW    Partition   1271 MB  Healthy
Volume 3                      RAW    Partition   1271 MB  Healthy
Volume 4     D                       DVD-ROM         0 B  Healthy
Volume 5                      RAW    Partition   1271 MB  Healthy
Volume 6                      RAW    Partition   1271 MB  Healthy
Volume 7     C                NTFS   Partition   8182 MB  Healthy    System
Volume 8                      RAW    Partition    290 MB  Healthy
Volume 9                      RAW    Partition   1271 MB  Healthy
Volume 10                     RAW    Partition    290 MB  Healthy
Volume 11                     RAW    Partition    290 MB  Healthy
Volume 12                     RAW    Partition    290 MB  Healthy
Volume 13                     RAW    Partition   1271 MB  Healthy
Volume 14                     RAW    Partition   1271 MB  Healthy
* Volume 15                     RAW    Partition   1271 MB  Healthy

You can now exit diskpart on all nodes

The disks should look like the above on all nodes in the cluster.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: