oraclecrRAC配置手册

ORACLE 12C R2 Real Application Cluster Installation Guide

朱海清

StarTimes Software Technology Co., Ltd

ASM磁盘空间最低要求

12C R2相比前一版本，OCR的磁盘占用需求有了明显增长。

为了方便操作，设置如下：

External: 1个卷x40G

Normal: 3个卷x30G

Hight: 5个卷x25G

Flex: 3个卷x30G

OCR+VOLTING+MGMT存储通常放到一个磁盘组，且选择Normal的冗余方式，也即最少3块asm磁盘80G空间。

操作系统安装

操作系统安装时把“Server with GUI“和”Compatibility Libraries”勾上，其他都不用选择。

版本采用CentOS 7、RHEL 7或者Oracle Linux 7

安装oracle预安装包

yum install -y oracle-rdbms-server-12cR1-preinstall

创建用户和组

oracle用户和dba、oinstall组已经在上一步创建完毕。

rac所有节点的oracle用户和grid用户的uid和gid必须一致，所以创建的时候最好制定uid和gid。groupadd --gid 54323 asmdba

groupadd --gid 54324 asmoper

groupadd --gid 54325 asmadmin

groupadd --gid 54326 oper

groupadd --gid 54327 backupdba

groupadd --gid 54328 dgdba

groupadd --gid 54329 kmdba

usermod --uid 54321 --gid oinstall --groups dba,oper,asmdba,asmoper,backupdba,dgdba,kmdba oracle useradd --uid 54322 --gid oinstall --groups dba,asmadmin,asmdba,asmoper grid

安装目录

mkdir -p /u01/app/ -p /u01/app/grid

mkdir -p /u01/app/oracle

chown -R grid:oinstall /u01

chown oracle:oinstall /u01/app/oracle

chmod -R 775 /u01/

用户环境变量

grid环境变量

cat <>/home/grid/.bash_profile

ORACLE_SID=+ASM1

ORACLE_HOME=/u01/ ORACLE_SID CLASSPATH ORACLE_HOME LD_LIBRARY_PATH PATH

EOF

在节点2，ORACLE_SID=+ASM2

oracle环境变量

cat <>/home/oracle/.bash_profile

ORACLE_SID=starboss1

ORACLE_HOME=/u01/app/oracle/product/ ORACLE_SID ORACLE_HOME ORACLE_HOSTNAME PATH LD_LIBRARY_PATH CLASSPATH EOF

在节点2，ORACLE_SID=starboss2，ORACLE_HOSTNAME=rac02

修改

# vi /etc/systemd/

RemoveIPC=no

# systemctl daemon-reload

# systemctl restart systemcd-logind

加载模块

echo "session required " >> /etc/login 禁用selinux

setenforce 0

vi /etc/sysconfig/selinux

禁用防火墙

# systemctl stop firewalld && systemctl disable firewalld 修改ulimit

cat <> /etc/security/

oracle soft nproc 16384

oracle hard nproc 16384

oracle soft nofile 1024

oracle hard nofile 65536

oracle soft stack 10240

oracle hard stack 32768

grid soft nproc 16384

grid hard nproc 16384

grid soft nofile 1024

grid hard nofile 65536

grid soft stack 10240

grid hard stack 32768

EOF

创建自定义的ulimit

cat <> /etc/

if [ $USER = "oracle" ]; then

if [ $SHELL = "/bin/ksh" ]; then ulimit -u 16384

ulimit -n 65536

else

ulimit -u 16384 -n 65536

fi

fi

if [ $USER = "grid" ]; then

if [ $SHELL = "/bin/ksh" ]; then

ulimit -u 16384

ulimit -n 65536

else

ulimit -u 16384 -n 65536

fi

fi

EOF

修改共享内存分区大小

将如下参数添加到/etc/fstab，具体大小数值根据实际情况调整，因为这个数值和物理内存以及MEMORY_TARGET有关。echo “shm /dev/shm tmpfs size=12g 0 0” >> /etc/fstab

修改后，只需重新对shm进行挂载即可：

mount -o remount /dev/shm

多路径

# yum install device-mapper-multipath

# cp /usr/share/doc/ /etc/

获取scsi id

# /usr/lib/udev/scsi_id --whitelisted --replace-whitespace –-device=/dev/sda # vi /etc/

multipaths {

multipath {

wwid 36000d000006

alias vol01

}

multipath {

wwid 36000d000005

alias vol02

}

}

# systemctl start

# multipath -ll

配置磁盘

ASMlib方式

安装ASMLib

Oracle Linux 7

yum install -y kmod-oracleasm

CentOS 7

磁盘配置

12C R2中对磁盘组空间要求比12C R1更大。

[root@rac01 ~]# /etc/oracleasm configure -i

Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library

driver. The following questions will determine whether the driver is

loaded on boot and what permissions it will have. The current values

will be shown in brackets ('[]'). Hitting without typing an

answer will keep that current value. Ctrl-C will abort.

Default user to own the driver interface []: grid

Default group to own the driver interface []: asmadmin

Start Oracle ASM library driver on boot (y/n) [n]: y

Scan for Oracle ASM disks on boot (y/n) [y]: y

Writing Oracle ASM library driver configuration: done

Initializing the Oracle ASMLib driver: [ OK ]

Scanning the system for Oracle ASMLib disks: [ OK ]

[root@rac01 ~]# reboot

用fdisk在共享磁盘上创建主分区：

[root@rac01 ~]# fdisk /dev/sdd

Welcome to fdisk (util-linux will remain in memory only, until you decide to write them. Be careful before using the write command.

Device does not contain a recognized partition table

Building a new DOS disklabel with disk identifier 0x86f899a0.

Command (m for help): n

Partition type:

p primary (0 primary, 0 extended, 4 free)

e extended

Select (default p): p

Partition number (1-4, default 1):

First sector (9, default 2048):

Using default value 2048

Last sector, +sectors or +size{K,M,G} (9, default ):

Using default value

Partition 1 of type Linux and of size GiB is set

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

Syncing disks.

在集群的任意节点创建asm磁盘：

[root@rac01 ~]# /etc/oracleasm createdisk OCR01 /dev/sdd1

Marking disk "OCR01" as an ASM disk: [ OK ] [root@rac01 ~]# /etc/oracleasm createdisk OCR02 /dev/sde1

Marking disk "OCR02" as an ASM disk: [ OK ] [root@rac01 ~]# /etc/oracleasm createdisk OCR03 /dev/sdf1

Marking disk "OCR03" as an ASM disk: [ OK ] [root@rac01 ~]# /etc/oracleasm createdisk DATA01 /dev/sdb1

Marking disk "DATA01" as an ASM disk: [ OK ] [root@rac01 ~]# /etc/oracleasm createdisk DATA02 /dev/sdc1

Marking disk "DATA02" as an ASM disk: [ OK ]分别两个节点执行：

[root@rac01 ~]# /etc/oracleasm scandisks

[root@rac01 ~]# /etc/oracleasm listdisks

注：

如果需要清空磁盘，重新部署asm，需要使用dd命令，如：

dd if=/dev/zero of=/dev/sdb1 bs=8192 count=128000

UDEV方式

参考：

确认在所有RAC节点上已经安装了必要的UDEV包

[root@rh2 ~]# rpm -qa|grep udev

6/Oracle Linux 6/RHEL 6

1.通过scsi_id获取设备的块设备的唯一标识名，假设系统上已有LUN sdc-sdp for i in c d e f g h i j k l m n o p ;

do

echo "sd$i" "`scsi_id -g -u -s /block/sd$i` ";

done

sdc 1IET_00010001

sdd 1IET_00010002

sde 1IET_00010003

sdf 1IET_00010004

以上列出于块设备名对应的唯一标识名

2.创建必要的UDEV配置文件，

首先切换到配置文件目录

[root@rh2 ~]# cd /etc/udev/

定义必要的规则配置文件

[root@rh2 ]# touch

[root@rh2 ]# cat

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_00010001", NAME="ocr1", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_00010002", NAME="ocr2", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_00010003", NAME="asm-disk1", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="1IET_00010004", NAME="asm-disk2", OWNER="grid", GROUP="asmadmin", MODE="0660"

Result 为/sbin/scsi_id -g -u -s %p的输出，按顺序填入刚才获取的唯一标识名即可

OWNER一般为grid，GROUP为asmadmin，MODE即为磁盘读写权限，采用0660即可

NAME为UDEV映射后的设备名，

建议为OCR和VOTE DISK创建独立的DISKGROUP，为了容易区分将该DISKGROUP专用的设备命名为ocr1..ocrn的形式

其余磁盘可以根据其实际用途或磁盘组名来命名

3.将该规则文件拷贝到其他节点上

[root@rh2 ]# scp Other_node:/etc/udev/

4.在所有节点上启动udev服务，或者重启服务器即可

[root@rh2 ]# /sbin/udevcontrol reload_rules

[root@rh2 ]# /sbin/start_udev

Starting udev: [ OK ] 5.检查设备是否到位

[root@rh2 ]# cd /dev

[root@rh2 dev]# ls -l ocr*

brw-rw---- 1 grid asmadmin 8, 32 Jul 10 17:31 ocr1

brw-rw---- 1 grid asmadmin 8, 48 Jul 10 17:31 ocr2

[root@rh2 dev]# ls -l asm-disk*

brw-rw---- 1 grid asmadmin 8, 64 Jul 10 17:31 asm-disk1

brw-rw---- 1 grid asmadmin 8, 80 Jul 10 17:31 asm-disk2

brw-rw---- 1 grid asmadmin 8, 96 Jul 10 17:31 asm-disk3

brw-rw---- 1 grid asmadmin 8, 112 Jul 10 17:31 asm-disk4

CentOS 7/Oracle Linux 7/RHEL 7

获取块设备id

# /usr/lib/udev/scsi_id -g -u -d /dev/sdb1

14f504e46494c45526a422d2d4b436a65

# /usr/lib/udev/scsi_id -g -u -d /dev/sdc1 14f504e46494c5a7a414d2d62494b6f2d5a6f6a42 # /usr/lib/udev/scsi_id -g -u -d /dev/sdd1 14f504e46494c324e626c2d4770654c2d6b443064 # /usr/lib/udev/scsi_id -g -u -d /dev/sde1 14f504e46494c6e7547552d2d6135576a

# /usr/lib/udev/scsi_id -g -u -d /dev/sdf1 14f504e46494c63526f742d534a75392d36374f69创建参数文件

touch /etc/

options=-g

# cat /etc/udev/

KERNEL=="sd1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",

RESULT=="14f504e46494c45526a422d2d4b436a65", SYMLINK+="asm-disk1", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",

RESULT=="14f504e46494c5a7a414d2d62494b6f2d5a6f6a42", SYMLINK+="asm-disk2", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",

RESULT=="14f504e46494c324e626c2d4770654c2d6b443064", SYMLINK+="asm-disk3", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",

RESULT=="14f504e46494c6e7547552d2d6135576a", SYMLINK+="asm-disk4", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",

RESULT=="14f504e46494c63526f742d534a75392d36374f69", SYMLINK+="asm-disk5", OWNER="grid", GROUP="asmadmin", MODE="0660"

加载并刷新块设备分区表

# /sbin/partprobe /dev/sdb1

# /sbin/partprobe /dev/sdc1

# /sbin/partprobe /dev/sdd1

# /sbin/partprobe /dev/sde1

# /sbin/partprobe /dev/sdf1

udev测试

# /sbin/udevadm test /block/sdb/sdb1 # /sbin/udevadm test /block/sdc/sdc1 # /sbin/udevadm test /block/sdd/sdd1 # /sbin/udevadm test /block/sde/sde1 # /sbin/udevadm test /block/sdf/sdf1启动服务

# /sbin/udevadm control --reload-rules 检查连接生成

[root@udev ~]# ls -l /dev/asm-disk*

lrwxrwxrwx 1 root root 4 Aug 22 13:19 /dev/asm-disk1 -> sdb1 lrwxrwxrwx 1 root root 4 Aug 22 13:19 /dev/asm-disk2 -> sdc1 lrwxrwxrwx 1 root root 4 Aug 22 13:19 /dev/asm-disk3 -> sdd1 lrwxrwxrwx 1 root root 4 Aug 22 13:19 /dev/asm-disk4 -> sde1 lrwxrwxrwx 1 root root 4 Aug 22 13:19 /dev/asm-disk5 -> sdf1

禁用ntp

/sbin/service ntpd stop

chkconfig ntpd off

mv /etc/ /etc/ /var/run/

停止avahi-daemon服务

systemctl stop avahi-dnsconfd

systemctl stop avahi-daemon

systemctl disable avahi-dnsconfd

systemctl disable avahi-daemon

IP配置

如果不安装DNS服务，通过hosts文件来解析，则只能配置一个SCAN IP，只能连接rac某一个节点，无法实现负载均衡。#public，接业务交换机，bond

rac01

rac02

#private，直连心跳，bond

rac01-priv

rac02-priv

#virtual

rac01-vip