RHEL环境下ORACLE 10gR2 RAC安装报告

Oracle10g RAC安装手册2008-01-09 11:26第一部分 Oracle10g RAC数据库简介oracle10g RAC结构下图显示了 Oracle RAC 10g 配置的主要组件。

集群中的节点通常是单独的服务器（主机），本文中，两个节点即是两台s4800服务器。

根据集群规模的大小，可以扩展到更多节点。

硬件在硬件级别上，RAC 集群中的各节点共享三种功能：1.对共享磁盘存储的访问2.与专用网络的连接3.对公共网络的访问。

共享磁盘存储Oracle RAC 依赖于一个共享磁盘体系结构。

数据库文件、联机重做日志和数据库的控制文件必须都能为集群中的每个节点所访问。

共享磁盘还存储 Oracle Cluster Registry 和 Voting Disk（稍后讨论）。

配置共享存储有多种方法，包括直接连接磁盘（通常是使用铜缆或光纤的 SCSI）、存储区域网 (SAN) 和网络连接存储 (NAS)。

专用网络每个集群节点通过专用高速网络连接到所有其他节点，这种专用高速网络也称为集群互联或高速互联 (HSI)。

Oracle 的 Cache Fusion 技术使用这种网络将每个主机的物理内存 (RAM) 有效地组合成一个高速缓存。

Oracle Cache Fusion 通过在专用网络上传输某个 Oracle 实例高速缓存中存储的数据允许其他任何实例访问这些数据。

它还通过在集群节点中传输锁定和其他同步信息保持数据完整性和高速缓存一致性。

专用网络通常是用千兆以太网构建的，但是对于高容量的环境，很多厂商提供了专门为 Oracle RAC 设计的低延迟、高带宽的专有解决方案。

Linux 还提供一种将多个物理 NIC 绑定为一个虚拟 NIC 的方法（此处不涉及）来增加带宽和提高可用性。

公共网络为维持高可用性，为每个集群节点分配了一个虚拟 IP 地址 (VIP)。

如果主机发生故障，则可以将故障节点的 IP 地址重新分配给一个可用节点，从而允许应用程序通过相同的 IP 地址继续访问数据库。

/archives/164vmware workstation 7安装oracle 10gr2 rac (5) 升级clusterware到10.2.0.4ON2011年10月17日·L E A V E A C O M M E N T本文描述vmware workstation 7安装oracle 10gr2 rac中的升级clusterware到10.2.0.4部分一.准备工作解压和改变属主unzip p6810189_10204_Linux-x86-64.zipchown -R oracle:oinstall Disk1/二.oracle用户登录执行安装[oracle@rac1 ~]$ crs_stop -allAttempting to stop `ora.rac1.gsd` on member `rac1`Attempting to stop `ora.rac2.gsd` on member `rac2`Attempting to stop `ora.rac2.ons` on member `rac2`Attempting to stop `ora.rac1.ons` on member `rac1`Stop of `ora.rac1.gsd` on member `rac1` succeeded. Stop of `ora.rac2.gsd` on member `rac2` succeeded. Stop of `ora.rac1.ons` on member `rac1` succeeded. Stop of `ora.rac2.ons` on member `rac2` succeeded. Attempting to stop `ora.rac1.vip` on member `rac1` Attempting to stop `ora.rac2.vip` on member `rac2` Stop of `ora.rac1.vip` on member `rac1` succeeded. Stop of `ora.rac2.vip` on member `rac2` succeeded.[oracle@rac1 ~]$ crs_stat -tName Type Target State Host ————————————————————ora.rac1.gsd application OFFLINE OFFLINEora.rac1.ons application OFFLINE OFFLINEora.rac1.vip application OFFLINE OFFLINEora.rac2.gsd application OFFLINE OFFLINEora.rac2.ons application OFFLINE OFFLINEora.rac2.vip application OFFLINE OFFLINE [oracle@rac1 ~]$cd /tmp/Disk1/./runInstaller1.欢迎界面2.升级CRS_HOME3.升级模式4.环境检测5.INSTALL呗6.继续跑脚本吧/oracle/crs/bin/crsctl stop crs & /oracle/crs/install/root102.sh跑完脚本clusterware就升级成功了下接vmware workstation 7安装oracle 10gr2 rac (6) 升级database到10.2.0.4。

Oracle 11G R2 RAC实施过程图文版（本文仅限于11G R2 on redhatlinux 6.x系列）内容目录1 实施准备 (2)1.1 调研与方案制定 (2)1.2 存储划分方案（参考） (3)1.3 介质 (4)2 设置系统 (4)2.1 系统检查 (4)2.1.1 cpu信息 (4)2.1.1 确认内存大小 (5)2.1.2 检查文件系统 (5)2.1.3 检查网络设置 (6)2.2 设置操作系统 (7)2.2.1 主机正名 (7)2.2.2 配置本地主机名解析hosts文件 (7)2.2.3 修改内核参数 (7)2.2.4 关闭防火墙服务 (8)2.2.5 关闭selinux (8)2.2.6 配置时钟同步 (9)2.3 设置软件安装/运行用户 (9)2.3.1 创建用户 (9)2.3.2 设置初始密码 (10)2.3.3 设置用户进程资源配额 (10)2.4 设置存储 (10)2.4.1 多路径软件配置 (10)2.4.2 udev绑定参考 (11)3 安装软件 (12)3.1 准备安装介质 (12)3.1.1 准备目录和权限（所有节点） (12)3.1.2 上传zip包(推荐ftp方式) (12)3.1.3 使用grid用户解包 (13)3.2 安装集群软件 (13)3.2.1 运行安装程序runInstaller (13)3.2.2 设置grid用户环境变量 (37)3.2.3 验证资源和服务状态，确定成功安装 (37)3.2.4 验证缺省监听 (38)3.2.5 查看scan_listener (39)3.3 安装数据库软件 (39)3.3.1 运行交互式安装程序runInstaller (39)3.3.2 为oracle用户的添加软件环境变量 (47)4 创建数据库 (47)4.1 创建asm diskgroup (47)4.1.1 以grid用户运行asmca工具 (47)4.2 创建数据库zbwshx（在1节点操作） (49)5 应用累计补丁更新PSU (69)5.1 升级补丁工具opatch (69)5.2 补丁1节点软件 (69)5.3 补丁2节点软件 (71)5.4 升级数据库（1节点操作） (71)6 附加脚本（可选） (72)6.1 调整数据库一些缺省设置 (72)6.2 管理集群服务 (73)7 初始化业务环境（举例） (73)1 实施准备1.1 调研与方案制定1.2 存储划分方案（参考）存储要1T1T分配，超过2T会有问题1.3 介质2 设置系统2.1 系统检查2.1.1 cpu信息1 查看物理cpu数shell(root)>cat /proc/cpuinfo |grep physical\ id|sort|uniq2 查看单个cpu coresshell(root)>cat /proc/cpuinfo |grepcpu\ cores|uniq3 查看单个是否超线程shell(root)>cat /proc/cpuinfo |grepcpu\ cores|uniq4 合计cpu coresshell(root)>cat /proc/cpuinfo | grep name | cut -f2 -d: | uniq -c shell(root)>mpstat 12.1.1 确认内存大小物理内存是256Gswap取值算法为物理内存的一倍，当物理内存超过64G，建议>=物理内存,这里32G符合最小2.1.2 检查文件系统/var和/tmp空间至少2G，建议10G/dev/shm共享空间设为物理内存/2 2.1.3 检查网络设置检查1节点public网卡设置检查2节点public网卡设置检查1节点private网卡地址（第一组）检查2节点private网卡设置(第一组)检查1节点 private网卡（第二组）检查2节点private网卡（第二组）2.2 设置操作系统2.1.1 主机正名1节点（全部小写最好了）2节点注意：重新登陆确认改变生效2.1.2 配置本地主机名解析hosts文件shell(root)>vi /etc/hosts#public address192.168.9.101 zbwshx01192.168.9.102 zbwshx02#public vip address192.168.9.103 zbwshx01-vip192.168.9.104 zbwshx02-vip192.168.9.105 zbwshx-cluster-scan（建议3个，用DNS解析）#private group #110.10.11.11 zbwshx01-priv110.10.11.12 zbwshx02-priv1#private group #210.10.12.11 zbwshx01-priv210.10.12.12 zbwshx02-priv2#NTP服务器192.168.5.11 ntpserver2.1.3 修改内核参数1.编辑sysctl.confshell(root)>vi /etc/sysctl.conf#ORACLE SETTINGfs.aio-max-nr = 1048576fs.file-max = 6815744#{shmall}单位pages,命令"echo $((物理内存/$(getconf PAGESIZE)))"（计算：物理内存/4096）kernel.shmall = 67108864#{shmmax}，单位bytes,为物理内存一半,echo $((256*1024*1024*1024/2))kernel.shmmax = 137438953472（物理内存/2）kernel.shmmni = 4096kernel.sem = 250 32000 100 128net.ipv4.ip_local_port_range = 9000 65500net.core.rmem_default = 262144net.core.rmem_max = 4194304net.core.wmem_default = 262144net.core.wmem_max = 1048586net.ipv4.ipfrag_high_thresh=524288net.ipv4.ipfrag_low_thresh=393216net.ipv4.tcp_rmem=4096 524288 16777216net.ipv4.tcp_wmem=4096 524288 16777216net.ipv4.tcp_timestamps=0net.ipv4.tcp_sack=0net.ipv4.tcp_window_scaling=1net.core.optmem_max=524287dev_max_backlog=2500sunrpc.tcp_slot_table_entries=128sunrpc.udp_slot_table_entries=128net.ipv4.tcp_mem=16384 16384 16384#多心跳网卡时需要多播技术net.ipv4.conf.default.rp_filter = 0#多播设备eth2,eth4net.ipv4.conf.eth2.rp_filter = 0net.ipv4.conf.eth4.rp_filter = 02.生效shell(root)>modprobesunrpcshell(root)>sysctl -p2.1.4 关闭防火墙服务shell(root)>chkconfig --level 2345 iptables offshell(root)>chkconfig --level 2345 ip6tables offshell(root)>service iptables stopshell(root)>service ip6tables stop2.1.5 关闭selinuxshell(root)>setenforce 0shell(root)>vi /etc/selinux/configSELINUX=disabled2.1.6 配置时钟同步∙配置NTP服务器<非数据库内容，略>∙配置ntp客户端shell(root)>vi /etc/ntp.conf注释掉其他restrict和server条目，增加如下serverntpserverrestrictntpserver mask 255.255.255.255nomodifynotrapnoquery ∙修改ntpd配置文件shell(root)>vi /etc/sysconfig/ntpdSYNC_HWCLOCK=yesOPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid"∙重启ntpd进程shell(root)>chkconfig --level 2345 ntpd onshell(root)>service ntpd start∙确认握手成功shell(root)>ntpq -p(初次同步需要一定时间）∙检查状态shell(root)>ntpstat2.2 设置软件安装/运行用户2.2.1 创建用户shell(root)>groupadd -g 10501 oinstallshell(root)>groupadd -g 10502 asmadminshell(root)>groupadd -g 10503 asmdbashell(root)>groupadd -g 10504 asmopershell(root)>groupadd -g 10505 dbashell(root)>groupadd -g 10506 opershell(root)>useradd -m -u 10501 -g oinstall -G asmadmin,asmdba,asmoper,dba -d /home/grid -s /bin/bash -c "Grid Infrastructure Owner" gridshell(root)>useradd -m -u 10502 -g oinstall -G dba,oper,asmdba -d /home/oracle -s /bin/bash -c "Oracle Software Owner" oracle2.2.2 设置初始密码shell(root)>password gridshell(root)>password oracle2.2.3 设置用户进程资源配额shell(root)>vi /etc/security/limits.conf#ORACLE SETTINGroot soft nproc 2047root hard nproc 16384root soft nofile 1024root hard nofile 65536root soft stack 10240root hard stack 32768grid soft nproc 2047grid hard nproc 16384grid soft nofile 1024grid hard nofile 65536grid soft stack 10240grid hard stack 32768oracle soft nproc 2047oracle hard nproc 16384oracle soft nofile 1024oracle hard nofile 65536oracle soft stack 10240oracle hard stack 327682.3 设置存储要求：✓同一个lun/vdisk在每个节点上设备名一致✓根据方案,通过scsi_id确认每一个lun/vdisk从存储上划分正确的✓共享模式要求在lun/vdisk在每个节点能同时读写✓RAC安装，lun/vdisk在操作系统上的设备名的属主是grid,组主是asmadmin，权限6602.3.1 多路径软件配置<非数据库内容,略>redhatlinux自带多路径软件multipath的参考文档官方文档:https:///documentation/zh-CN/Red_Hat_Enterprise_Linux/6/html/DM_Multipath/index.html百度搜索文档:/i6241962348353946114/绑定用户组案例：/2014/01/multipath实现设备用户组设置.html2.3.2 udev绑定参考注意：∙如果多路径软件已经实现了设备名绑定、属（组）主设置、权限设置，则不需要再配置udev∙udev不具有多路径failover能力∙本例为scsi磁盘∙∙1.shell(root)>fdisk -l|grep Disk\ /dev/sd根据系统人员告知,设备sda是本地操作系统宿主硬盘，不能使用,剩下的是共享磁盘2.查看scsi_idshell(root)>scsi_id -gvu /dev/sdN3.新建或编辑udev的rule文件shell(root)>vi /etc/udev/rules.d/99-oracle.rules（必须是99）KERNEL=="sd*", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk",ENV{ID_SERIAL}=="36006016010402600126dd2e653ebe511", NAME+="crs_01", OWNER="grid", GROUP="asmadmin", MODE="0660"KERNEL=="sd*", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk",ENV{ID_SERIAL}=="360060160104026004eb5b3db53ebe511", NAME+="crs_02", OWNER="grid", GROUP="asmadmin", MODE="0660"KERNEL=="sd*", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk",ENV{ID_SERIAL}=="360060160104026006edcb7d153ebe511", NAME+="crs_03", OWNER="grid", GROUP="asmadmin", MODE="0660"KERNEL=="sd*", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk",ENV{ID_SERIAL}=="36006016010402600f0b413ee53ebe511", NAME+="fra_data_01", OWNER="grid", GROUP="asmadmin", MODE="0660"KERNEL=="sd*", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk",ENV{ID_SERIAL}=="36006016010402600162f6d1254ebe511", NAME+="zbwshx_data_01", OWNER="grid", GROUP="asmadmin", MODE="0660"4.启动udevshell(root)>udevadm control reload-rulesshell(root)>start_udev3 安装软件3.1 准备安装介质(介质上传到/u01/media/oracle/linux_x64目录)3.1.1 准备目录和权限（所有节点）shell(root)>mkdir -p /u01/media/oracle/linux_x64shell(root)>chown -R grid:oinstall /u01shell(root)>chmod -R g+w /u013.1.2 上传zip包(推荐ftp方式)✓用grid用户上传✓上传grid安装包,database安装包，补丁包，补丁工具✓上传后介质属主grid,组主oinstall3.1.3 使用grid用户解包shell(grid)>cd /u01/media/oracle/linux_x64shell(grid)>unzip p1*******_112040_Linux-x86-64_1of7.zipshell(grid)>unzip p1*******_112040_Linux-x86-64_2of7.zipshell(grid)>unzip p1*******_112040_Linux-x86-64_3of7.zip3.2 安装集群软件3.2.1 运行安装程序runInstallershell(grid)>cd gridshell(grid)>./runInstaller提示：为了把x-windows程序的界面投射到windows终端上，需要执行2个步骤1.在windows上启动本地X11Forward程序，推荐x-manager中的xpassive(端口默认6000,注意firewall)2.在服务器上设定shell环境变量：export DISPLAY=YourIP:0.03.执行xclock测试在DHCP+DNS的环境，需要激活GNS，这里关闭本方案只有一个SCAN IP，使用/etc/hosts解析（也就是说需要将SCAN IP写入到/etc/hosts 下）建立主机之间ssh互信，这样可以免密码互相登陆，oracle安装过程中会把1节点的软件自动复制到2节点。

项目流程如下：一、RAC安装1、实验平台：Oracle 10gR2 RAC + RHEL 5.4 +VMware-server-1.0.92、安装步骤：（1）系统需求：操作系统：红帽5.0或5.4内存：800M交换空间：2048M硬盘：10G网卡：2(public IP + Virtual IP,priv IP)主机节点：2(node1,node2)存储设备：1（DB files）（2）网络规划：主机名类型 IP地址node1 公共 142.105.100.111node2 公共 142.105.100.112node1-vip 虚拟 142.105.100.211node2-vip 虚拟 142.105.100.212node1-priv 专用 192.168.100.111node2-priv 专用 192.168.100.112（3）安装流程>安装虚拟机（vmware server 1.0.9）>安装操作系统(RHEL 5.4)>配置操作系统>复制虚拟机>配置双机信任关系>安装clusterware软件>安装database软件>配置监听listener>建立数据库实例>客户端TAF(transparent application failover)配置（透明切换）（4）安装前准备及OS安装配置，准备好安装包，包括：VMware-server-1.0.9-156507.i386.rpm(虚拟机包),rhel-server-5.4-i386-dvd.iso(红帽镜像),Oracle10201_clusterware_linux32.zip(集群软件),oracleasm-support-2.1.3-1.el5.i386.rpm(ASM包),oracleasm-2.6.18-164.el5-2.0.5-1.el5.i686.rpm(ASM包),oracleasmlib-2.0.4-1.el5.i386.rpm(ASM包),compat-gcc-7.3-2.96.128.i386.rpm(集群软件包),compat-libstdc++-7.3-2.96.128.i386.rpm(集群软件包),compat-libstdc++-devel-7.3-2.96.128.i386.rpm(集群软件包),compat-gcc-c++-7.3-2.96.128.i386.rpm(集群软件包),oracle10g01_10.2.0.1.0_database_linux32.zip(数据库包)RAC1 VotingdiskOCRdiskvmware sharedisk ASMdisk1(两个ASM,一个用于用户数据,一个闪回数据)ASMdisk2RAC21）安装vmware－1.0.9版虚拟机rpm -ivh VMware-server-1.0.9-156507.i386.rpmRAC1 (新建－>选择桥结－>LSI Logic－>SCSI－>RAC1.vmdk(10G)－>Votingdisk.vmdk(0.1G)(scsi1:1)(Independent)－>OCRdisk.vmdk(0.1G)(scsi1:2)(Independent)－>ASMdisk1(2G)(scsi1:3)(Independent)－>ASMdisk2(1G)(scsi1:4)(Independent))->ethernet(加一块网卡(桥结))RAC2（同一，也可待建完后复制）2)安装redhat Linux 5.4 操作系统关键点：分区（swap,sda1）包（开发全选，基本系统全选，服务器FTP）防火墙(禁用)，SELINUX（禁用）,RAC1.vmx.[root@node1 ~]# uname -aLinux node1 2.6.18-164.el5 #1 SMP Tue Aug 18 15:51:54 EDT 2009 i686 i686 i386 GNU/Linux[root@node1 ~]# tail -7 /etc/hosts127.0.0.1 localhost.localdomain localhost142.105.100.111 node1142.105.100.112 node2142.105.100.211 node1-vip142.105.100.212 node2-vip192.168.100.111 node1-priv192.168.100.112 node2-priv[root@node1 ~]# tail -7 /etc/sysctl.confkernel.sem = 250 32000 100 128fs.file-max = 65536net.ipv4.ip_local_port_range = 1024 65000net.core.rmem_default = 1048576net.core.rmem_max = 1048576net.core.wmem_default = 262144net.core.wmem_max = 262144[root@node1 ~]# sysctl -pnet.ipv4.ip_forward = 0net.ipv4.conf.default.rp_filter = 1net.ipv4.conf.default.accept_source_route = 0kernel.sysrq = 0kernel.core_uses_pid = 1net.ipv4.tcp_syncookies = 1kernel.msgmnb = 65536kernel.msgmax = 65536kernel.shmmax = 4294967295kernel.shmall = 268435456kernel.sem = 250 32000 100 128fs.file-max = 65536net.ipv4.ip_local_port_range = 1024 65000net.core.rmem_default = 1048576net.core.rmem_max = 1048576net.core.wmem_default = 262144net.core.wmem_max = 262144[root@node1 ~]#sysctl -p[root@node1 ~]# tail -1 /etc/rc.localmodprobe hangcheck-timer hangcheck-tick=30 hangcheck_margin=20[root@node1 ~]# vi /etc/redhat-release[root@node1 ~]#> /etc/resolv.conf[root@node1 ~]#groupadd oinstall[root@node1 ~]#groupadd dba[root@node1 ~]#useradd oracle[root@node1 ~]#passwd oracle[root@node1 ~]# tail -7 /home/oracle/.bash_profileexport PATHexport ORACLE_BASE=/u01/app/oracleexport ORACLE_HOME=$ORACLE_BASE/product/10.2.0/db_1export ORACLE_SID=RACDB1export PATH=$ORACLE_HOME/bin:$PATHexport LD_LIBRARY_PATH=$ORACLE_HOME/bin:$PATHexport CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib [root@node1 ~]#mkdir -p /u01/app/oracle[root@node1 ~]#chown -R oracle:oinstall /u01[root@node1 ~]#chmod -R 775 /u01[root@node1 ~]# fdisk -lDisk /dev/sda: 10.7 GB, 10737418240 bytes255 heads, 63 sectors/track, 1305 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesDevice Boot Start End Blocks Id System/dev/sda1 * 1 1241 9968301 83 Linux/dev/sda2 1242 1305 514080 82 Linux swap / SolarisDisk /dev/sdb: 107 MB, 107374080 bytes64 heads, 32 sectors/track, 102 cylindersUnits = cylinders of 2048 * 512 = 1048576 bytesDevice Boot Start End Blocks Id System/dev/sdb1 1 102 104432 83 LinuxDisk /dev/sdc: 107 MB, 107374080 bytes64 heads, 32 sectors/track, 102 cylindersUnits = cylinders of 2048 * 512 = 1048576 bytesDevice Boot Start End Blocks Id System/dev/sdc1 1 102 104432 83 LinuxDisk /dev/sdd: 2147 MB, 2147483648 bytes255 heads, 63 sectors/track, 261 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesDevice Boot Start End Blocks Id System/dev/sdd1 1 261 2096451 83 LinuxDisk /dev/sde: 1073 MB, 1073741824 bytes255 heads, 63 sectors/track, 130 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesDevice Boot Start End Blocks Id System/dev/sde1 1 130 1044193+ 83 Linux[root@node1 ~]#rpm -ivh oracleasm-support-2.1.3-1.el5.i386.rpm[root@node1 ~]#rpm -ivh oracleasm-2.6.18-164.el5-2.0.5-1.el5.i686.rpm[root@node1 ~]#rpm -ivh oracleasmlib-2.0.4-1.el5.i386.rpm[root@node1 ~]#/etc/init.d/oracleasm configureDefault user to own the driver interface []:oracleDefault group to own the driver interface []:dbaStart oracle ASM library driver interdace on boot （y/n）[n]:yScan for oracle ASM disks on boot (y/n)[ｙ]:ｙ[root@node1 ~]#/etc/init.d/oracleasm createdisk VOL1 /dev/sdd1[root@node1 ~]#/etc/init.d/oracleasm createdisk VOL2 /dev/sde1[root@node1 ~]#/etc/init.d/oracleasm listdisks[root@node1 ~]# tail -5 /etc/udev/rules.d/60-raw.rulesACTION=="add", KERNEL=="sdb1", RUN+="/bin/raw /dev/raw/raw1 %N"ACTION=="add", ENV{MAJOR}=="8", ENV{MINOR}=="17", RUN+="/bin/raw /dev/raw/raw1 %M %m" ACTION=="add", KERNEL=="sdc1", RUN+="/bin/raw /dev/raw/raw2 %N"ACTION=="add", ENV{MAJOR}=="8", ENV{MINOR}=="33", RUN+="/bin/raw /dev/raw/raw2 %M %m" KERNEL=="raw[1-2]",OWNER="oracle",GROUP="oinstall",MODE="640"[root@node1 ~]#start_udev[root@node1 ~]#ls -l /dev/raw[root@node1 ~]#rpm -ivh /oradisk/compat-gcc-7.3-2.96.128.i386.rpm[root@node1 ~]#rpm -ivh /oradisk/compat-libstdc++-7.3-2.96.128.i386.rpm —force[root@node1 ~]#rpm -ivh /oradisk/compat-libstdc++-devel-7.3-2.96.128.i386.rpm[root@node1 ~]#rpm -ivh /oradisk/compat-gcc-c++-7.3-2.96.128.i386.rpmNode2[root@node1 ~]#mkdir /vmware/RAC2[root@node1 ~]#Cp /vmware/RAC1/* /vmware/RAC1/ -rf[root@node1 ~]#Vim rac2.vmx(rac1改为rac2)打开vmware下RAC2，（create）[root@node1 ~]#Hostname node2[root@node2 ~]#Cat /etc/sysconf/network[root@node2 ~]#Cd /etc/sysconfig/network-scripts/[root@node2 network-scripts]#Mv ifcfg-eth0.bak ifcfg-eth0[root@node2 network-scripts]#Mv ifcfg-eth1.bak ifcfg-eth1[root@node2 network-scripts]#Ifconfig eth0 | grep Hwaddr[root@node2 network-scripts]#Ifconfig eth1 | grep Hwaddr[root@node2 network-scripts]#Vim ifconfig eth0(改硬件地址)[root@node2 network-scripts]#Vim ifconfig eth1(改硬件地址)[root@node2 network-scripts]#Service network restart[root@node2 network-scripts]#vim /home/oracle/.bash_profile(改ORACLE_SID为RACDB2)[root@node2 ~]#ping node1[root@node2 ~]#ping node2[root@node2 ~]#ping node1-priv[root@node2 ~]#ping node1-priv[root@node2 ~]#/etc/init.d/oracleasm scandisks[root@node2 ~]#/etc/init.d/oracleasm listdisks[root@node2 ~]#service xinetd start[root@node2 ~]#chkconfig time-stream on[root@node2 ~]#exit[root@node2 ~]#crontab -e*** * rdate -s 142.105.100.112[root@node1 ~]#su - oracle[oracle@node1 ~]#mkdir .ssh[oracle@node1 ~]#chmod 700 .ssh[oracle@node1 ~]#ssh-keygen -t rsa[oracle@node1 ~]#ssh-keygen -t dsa[root@node2 ~]#su - oracle[oracle@node2 ~]#mkdir .ssh[oracle@node2 ~]#chmod 700 .ssh[oracle@node2 ~]#ssh-keygen -t rsa[oracle@node2 ~]#ssh-keygen -t dsa[oracle@node1 ~]#cat /home/oracle/.ssh/id_rsa.pub >>/home/oracle/.ssh/authorized_keys[oracle@node1 ~]#cat /home/oracle/.ssh/id_dsa.pub >>/home/oracle/.ssh/authorized_keys[oracle@node1 ~]#ssh node2 cat /home/oracle/.ssh/id_rsa.pub >> /home/oracle/.ssh/authorized_keys [oracle@node1 ~]#ssh node2 cat /home/oracle/.ssh/id_dsa.pub >> /home/oracle/.ssh/authorized_keys [oracle@node1 ~]#scp /home/oracle/.ssh/authorized_keys node2:/home/oracle/.ssh/authorized_keys [oracle@node1 ~]#ssh node1 date[oracle@node1 ~]#ssh node2 date[oracle@node1 ~]#ssh node1-priv date[oracle@node1 ~]#ssh node1-priv date[oracle@node1 ~]#ssh node2[oracle@node2 ~]#ssh node1 date[oracle@node2 ~]#ssh node2 date[oracle@node2 ~]#ssh node1-priv date[oracle@node2 ~]#ssh node2-priv date安装Oracle 10gR2 clusterware:[oracle@node1 ~]#cd /oradisk[oracle@node1 ~]#unzip Oracle10201_clusterware_linux32.zip[oracle@node1 ~]#chown -R oracle:oinstall /oradisk[oracle@node1 ~]#chmod -R 775 /oradisk[oracle@node1 ~]#cd /oradisk/clusterware[oracle@node1 clusterware]#/oradisk/clusterware/cluvfy/runcluvfy.sh stage -pre crsinst -n node1,node2-verbose[oracle@node1 ~]#su - root[root@node1 ~]#cd /oradisk/clusterware/rootpre[root@node1 ~]#exit[oracle@node1 ~]#cd /oradisk/clusterware[oracle@node1 clusterware]#export LANG=en_US.UTF-8[oracle@node1 clusterware]#./runInstaller -ignoreSysPreReqs关键点：1)在specify cluster configuration中加node2 node2-priv node2-vip2)自specify network interface usage中验证interdace Type选项是否是eth0 private;eth1 public3)在specify oracle cluster registry(OCR) location中选择Extermal redundancyn并加/dev/raw/raw24)在specify Voting Disk location中选择Extermal redundancyn并加/dev/raw/raw15)运行完显示完成并出现选项OK和2个脚本(不是第一次装，也可能只有1个脚本)，先别点，接着作下面的：[oracle@node1 ~]#/u01/app/oracle/oraInventory/orainstRoot.sh[oracle@node1 ~]#ssh node2[oracle@node2 ~]#/u01/app/oracle/oraInventory/orainstRoot.sh[oracle@node1 ~]#cd /u01/app/oracle/product/10.2.0/crs_1/bin[oracle@node1 ~]#vim vipca#Remove this workaround when the bug 3937317 is fixedarch=`uname -m`if [ "$arch" = "i686" -o "$arch" = "ia64" ]thenLD_ASSUME_KERNEL=2.4.19（改为当前系统内核版本（如：2.6.18））export LD_ASSUME_KERNELfiunset LD_ASSUME_KERNEL（加上这行）#End workaround[oracle@node1 ~]#vim srvctl#Remove this workaround when the bug 3937317 is fixedLD_ASSUME_KERNEL=2.4.19（改为当前系统内核版本（如：2.6.18））export LD_ASSUME_KERNELunset LD_ASSUME_KERNEL（加上这行）# Run ops control utility[oracle@node1 ~]#vim racgvipset it to 0 for checkIf() to return success if default gateway is not found,# otherwise set it to 1FAIL_WHEN_DEFAULTGW_NOT_FOUND=0(将1改为0)# hard code default gateway here if neededDEFAULTGW=142.105.100.1（加上网关）# renice the process[oracle@node1 ~]#vim vipca[oracle@node1 ~]#vim srvctl 同样改node2[oracle@node1 ~]#vim racgvip[root@node1 ~]#/u01/app/oracle/product/10.2.0/crs_1/root.sh[root@node2 ~]#/u01/app/oracle/product/10.2.0/crs_1/root.sh[root@node2 ~]#cd /u01/app/oracle/product/10.2.0/crs_1/bin[root@node2 bin]#export LANG=en_US.UTF-8[root@node2 bin]#./vipca[root@node2 bin]#./oifcfg iflist[root@node2 bin]#./oifcfg setif -global eth0/142.105.100.0:public[root@node2 bin]#./oifcfg setif -global eth1/192.168.100.0:cluster_interconnect [root@node2 bin]#./oifcfg getif[root@node2 bin]#./vipca关键点：IP Alias Name中加：node1-vip和node2-vip完成后（Exit）点击OK6)[root@node1 ~]#cd /u01/app/oracle/product/10.2.0/crs_1/bin[root@node1 ~]#./crs_stat -tName Type Target State Host------------------------------------------------------------ora.node1.gsd application ONLINE ONLINE node1ora.node1.ons application ONLINE ONLINE node1ora.node1.vip application ONLINE ONLINE node1ora.node2.gsd application ONLINE ONLINE node2ora.node2.ons application ONLINE ONLINE node2ora.node2.vip application ONLINE ONLINE node2[root@node1 ~]#tail -3 /etc/inittab (显示启集群软件3行命令)[root@node2 ~]#cd /u01/app/oracle/product/10.2.0/crs_1/bin[root@node2 ~]#./crs_stat -t[root@node1 ~]#./crs_stop ora.node1.vip[root@node1 ~]#./crs_stat ora.node1.vip卸载（clusterware）集群软件方法1：1)脚本自动卸载（所有节点上分别进行）$$ORACLE_HOME/install/rootdelete.sh$$ORACLE_HOME/install/rootdeinstall.sh方法1：2)手工卸载（所有节点上分别进行）#srvctl stop nodeapps -n $NODENAME#rm -rf /etc/oracle/*#rm -rf /etc/init.d/init.*#rm -rf /etc/rc?.d/*.crs#rm -rf /etc/inittab.crs#cp /etc/inittab.orig /etc/inittab#ps -ef | grep crs;kill $PID#ps -ef | grep css;kill $PID#ps -ef | grep evm;kill $PID#rm -rf /tmp/.oracle/*#rm -rf /var/tmp/.oracle/*#rm -rf /u01/app/oracle/*#dd if=/dev/zero of=/dev/sdb1 bs=1M count=100#dd if=/dev/zero of=/dev/sdc1 bs=1M count=100#删除分区并重新建立#shutdown -r now安装Oracle 10gR2 database[root@node1 ~]#cd /oradisk[root@node1 ~]#unzip oracle10g01_10.2.0.1.0_database_linux32.zip[root@node1 ~]#chown -R oracle:oinstall /oradisk[root@node1 ~]#chmod -R 775 /oradisk[oracle@node1 ~]$ cat /home/oracle/.bash_profile | grep productexport ORACLE_HOME=$ORACLE_BASE/product/10.2.0/db_1[oracle@node1 ~]$[oracle@node1 ~]$ ssh node2 cat /home/oracle/.bash_profile | grep product export ORACLE_HOME=$ORACLE_BASE/product/10.2.0/db_1[oracle@node1 ~]$ cd /oradisk/database/[oracle@node1 database]$ export LANG=en_US.UTF-8[root@node1 ~]#./runInstaller -ignoreSysPreReqs关键点：1)Specify Home Details:oraDb10g_home1;/u01/app/oracle/product/10.2.0/db_12)node1,node2节点全选上[root@node1 ~]#/u01/app/oracle/product/10.2.0/db_1/root.sh[root@node2 ~]#/u01/app/oracle/product/10.2.0/db_1/root.sh配置netca[oracle@node2 ~]#export LANG=en_US.UTF-8[oracle@node2 ~]#netca关键点：1)选cluster configuration2)node1,node2全选3)选listener configuration4)add5)选TCP[oracle@node1 ~]#/u01/app/oracle/product/10.2.0/crs_1/bin/crs_stat -t[oracle@node2 ~]#/u01/app/oracle/product/10.2.0/crs_1/bin/crs_stat -t创建ASM 实例配置dbca创建数据库[oracle@node1 bin]#export LANG=en_US.UTF-8[oracle@node1 bin]#dbca关键点：1)create a Datebase2)node1,node2全选3)custon Database4)RACDB5)configure the Database with EnterPrise Manager;use Database control for Database Management6)选ASM7)create initialization parameter file:{ORACLE_BASE}/admin/+ASM/prfile/init.ora8)create new:(RAC_VOLGRP1,Extemal,show candidates,VOL1)(FLASH_RECOVERY_AREA,Extemal,show candidates,VOL2)9)use oracle-managed files:+RAC_VOLGRP110)Specify Flash recover Area:+FLASH_RECOVERY_AREA 100011)create Database[oracle@node1 bin]#./crs_stat -t[oracle@node2 bin]#./crs_stat -t[oracle@node1 bin]#export ORACLE_SID=+ASM1[oracle@node1 bin]#sqlplus / as sysdbaSQL>select instance_name from v$instance;INSTANCE_NAME+ASM1[oracle@node1 bin]#export ORACLE_SID=RACDB1[oracle@node1 bin]#sqlplus / as sysdbaSQL>select instance_name from v$instance;INSTANCE_NAMERACDB1[oracle@node2 bin]#export ORACLE_SID=+ASM2[oracle@node2 bin]#sqlplus / as sysdbaSQL>select instance_name from v$instance;INSTANCE_NAME+ASM2[oracle@node2 bin]#export ORACLE_SID=RACDB2[oracle@node2 bin]#sqlplus / as sysdbaSQL>select instance_name from v$instance;INSTANCE_NAME+RACDB2检查RAC状态[oracle@node1 bin]#./crs_stat -t[oracle@node2 bin]#./crs_stat -tName Type Target State Host------------------------------------------------------------ora....B1.inst application ONLINE OFFLINEora....B2.inst application ONLINE OFFLINEora.RACDB.db application ONLINE UNKNOWN node2ora....SM1.asm application ONLINE UNKNOWN node1ora....E1.lsnr application ONLINE UNKNOWN node1ora.node1.gsd application ONLINE UNKNOWN node1ora.node1.ons application ONLINE UNKNOWN node1ora.node1.vip application ONLINE ONLINE node1ora....SM2.asm application ONLINE UNKNOWN node2 ora....E2.lsnr application ONLINE UNKNOWN node2ora.node2.gsd application ONLINE UNKNOWN node2 ora.node2.ons application ONLINE UNKNOWN node2 ora.node2.vip application ONLINE ONLINE node2[root@node1 bin]#./crs_stop -f ora.RACDB.db[root@node1 bin]#./crs_stop -f ora.node1.ASM1.asm[root@node1 bin]#./crs_stop -f ora.node2.ASM2.asm[root@node1 bin]#./crs_stop -f ora.node1.LISTENER_NODE1.lsnr [root@node1 bin]#./crs_stop -f ora.node2.LISTENER_NODE2.lsnr [root@node1 bin]#./crs_stop -f ora.node1.gsd[root@node1 bin]#./crs_stop -f ora.node1.ons[root@node1 bin]#./crs_stop -f ora.node2.gsd[root@node1 bin]#./crs_stop -f ora.node2.ons[root@node1 bin]#./crs_stop -f ora.node1.vip[root@node1 bin]#./crs_stop -f ora.node2.vip[root@node1 bin]#./crs_stat -tName Type Target State Host------------------------------------------------------------ora....B1.inst application ONLINE OFFLINEora....B2.inst application ONLINE OFFLINEora.RACDB.db application ONLINE OFFLINEora....SM1.asm application ONLINE OFFLINEora....E1.lsnr application ONLINE OFFLINEora.node1.gsd application ONLINE OFFLINEora.node1.ons application ONLINE OFFLINEora.node1.vip application ONLINE OFFLINEora....SM2.asm application ONLINE OFFLINEora....E2.lsnr application ONLINE OFFLINEora.node2.gsd application ONLINE OFFLINEora.node2.ons application ONLINE OFFLINEora.node2.vip application ONLINE OFFLINE[oracle@node1 bin]#./crs_start -all[oracle@node1 bin]#./crs_stat -tName Type Target State Host------------------------------------------------------------ora....B1.inst application ONLINE ONLINE node1 ora....B2.inst application ONLINE ONLINE node2ora.RACDB.db application ONLINE ONLINE node1 ora....SM1.asm application ONLINE ONLINE node1 ora....E1.lsnr application ONLINE ONLINE node1ora.node1.gsd application ONLINE ONLINE node1 ora.node1.ons application ONLINE ONLINE node1 ora.node1.vip application ONLINE ONLINE node1 ora....SM2.asm application ONLINE ONLINE node2 ora....E2.lsnr application ONLINE ONLINE node2 ora.node2.gsd application ONLINE ONLINE node2 ora.node2.ons application ONLINE ONLINE node2 ora.node2.vip application ONLINE ONLINE node2 RAC启停及警告文件：> 关闭node1$>cd /u01/app/oracle/product/10.2.0/crs_1/binnode1$>./crs_stat -tnode1$>lsnrctl stopnode2$>lsnrctl stopnode1$>./srvctl status database -d RACDBnode1$>./srvctl stop database -d RACDB -o immediatenode1$>su - rootnode1$>cd /u01/app/oracle/product/10.2.0/crs_1/binnode1$>./crsctl check crsnode1$>./crsctl stop crsnode2$>su - rootnode2$>cd /u01/app/oracle/product/10.2.0/crs_1/binnode2$>./crsctl check crsnode2$>./crsctl stop crs> 启动node1$>su - rootnode1$>cd /u01/app/oracle/product/10.2.0/crs_1/binnode1$>./crsctl start crsnode1$>./crsctl check crsnode2$>su - rootnode2$>cd /u01/app/oracle/product/10.2.0/crs_1/binnode2$>./crsctl start crsnode2$>./crsctl check crsnode1$>cd /u01/app/oracle/product/10.2.0/crs_1/binnode1$>./srvctl start database -d RACDBnode1$>./srvctl status database -d RACDBnode1$>lsnrctl startnode2$>lsnrctl startnode1$>./crs_stat -t> 启动后检查[oracle@node1 bin]#pwd/u01/app/oracle/product/10.2.0/crs_1/bin[oracle@node1 bin]#./crsctl check crsCSS appears healthyCRS appears healthyEVM appears healthy[oracle@node1 bin]#./stvctl status datanase -d RACDB[oracle@node1 bin]#./crs_stat -tcrs警告文件：［oracle@rac1 ~］$tail -f /u01/app/oracle/product/10.2.0/crs_1/log/rac1/alertrac1.log ［oracle@rac2 ~］$tail -f /u01/app/oracle/product/10.2.0/crs_1/log/rac2/alertrac2.log ASM警告文件：［oracle@rac1 ~］$tail -f /u01/app/oracle/admin/+ASM/bdump/alert_+ASM1.log［oracle@rac2 ~］$tail -f /u01/app/oracle/admin/+ASM/bdump/alert_+ASM2.log数据库警告文件：［oracle@rac1 ~］$tail -f /u01/app/oracle/admin/RACDB/bdump/alert_RACDB1.log ［oracle@rac2 ~］$tail -f /u01/app/oracle/admin/RACDB/bdump/alert_RACDB2.log客户端TAF（Transparent applacation failover）配置a.服务器确认listener.oranode1$ cat listener.oraSID_LIST_LISTENER_NODE1 =(SID_LIST =(SID_DESC =(SID_NAME = PLSExtProc)(ORACLE_HOME=/u01/app/oracle/product/10.2.0/db_1)(PROGRAM = extproc)))LISTENER_NODE1 =(DESCRIPTION_LIST =(DESCRIPTION =(ADDRESS=(PROTOCOL = TCP)(HOST = NODE1-VIP)(PORT=1521)(IP=FIRST)) (ADDRESS=(PROTOCOL = TCP)(HOST =142.105.100.111)(PORT=1521)(IP=FIRST)) ))node1$ lsnctl status LISTENER_NODE1node2$ cat listener.oraSID_LIST_LISTENER_NODE2 =(SID_LIST =(SID_DESC =(SID_NAME = PLSExtProc)(ORACLE_HOME=/u01/app/oracle/product/10.2.0/db_1)(PROGRAM = extproc))LISTENER_NODE2 =(DESCRIPTION_LIST =(DESCRIPTION =(ADDRESS=(PROTOCOL = TCP)(HOST = NODE1-VIP)(PORT=1521)(IP=FIRST)) (ADDRESS=(PROTOCOL = TCP)(HOST =142.105.100.112)(PORT=1521)(IP=FIRST))))node1$ lsnctl status LISTENER_NODE2FOR RAC BUG ORA-12545 IN ORACLE 10.2(SEE NORE:364855.1):(连不上node1转而连node2) node1$ sqlplus / as sysdbaSQL>show parameter instance_name;RACDB1SQL>alter system set local_listener='LISTENERS_RACDB' sid='*'; (node1/2都行)node2$ sqlplus / as sysdbaSQL>show parameter instance_name;RACDB2SQL>alter system set local_listener='LISTENERS_RACDB' sid='*';$cat tnsnames.ora……LISTENERS_RACDB =(ADDRESS_LIST =(ADDRESS = (PROTOCOL = TCP)(HOST = node1-vip)(PORT = 1521))(ADDRESS = (PROTOCOL = TCP)(HOST = node2-vip)(PORT = 1521)))b. 客户端确认tnsnames.oraRACDB =(DESCRIPTION =(DESCRIPTION =(ADDRESS=(PROTOCOL = TCP)(HOST =142.105.100.211)(PORT=1521))(ADDRESS=(PROTOCOL = TCP)(HOST =142.105.100.212)(PORT=1521))(连接不上就连node2) )(CONNECT_DATE =(SERVICE_NAME = RACDB(此名用于连接数据库))(FAILOVER_MODE =(TYPE = SELECT)(METHOD = BASIC))))SELECT(2个vip选1个，连上1个就不连另一个)有问题临时连TypePRECONNECT（连1同时连2）有问题时不需要临时连BASIC(查询到一半中断时自动连另一个继续查询(接着的))MethodSESSION(查询到一半中断时自动连另一个重新查询(重新开始))客户端负载均衡（Load balance）配置RACDB =(DESCRIPTION =(DESCRIPTION =(ADDRESS=(PROTOCOL = TCP)(HOST =142.105.100.211)(PORT=1521)) (ADDRESS=(PROTOCOL = TCP)(HOST =142.105.100.212)(PORT=1521)) (LOAD_BALANCE = YES)(负载均衡，2个vip随机连))(CONNECT_DATE =(SERVICE_NAME = RACDB)(FAILOVER_MODE =(TYPE = SELECT)(METHOD = BASIC))))。