集中备份系统应急方案
- 格式:doc
- 大小:322.50 KB
- 文档页数:14
集中备份系统应急方案 4/25/2022 集中备份系统应急方案
Page1 of 14 4/25/2022 目录 1. 前言 ......................................................................................................................... 1 2. 数据库日志备份应急措施 ..................................................................................... 1 2.1. Oracle数据库的应急处理方法 ···················································· 1 2.2. Informix数据库的应急处理方法 ················································· 2
3. 集中备份系统故障应急措施 ................................................................................. 3 3.1. 故障分类 ··············································································· 3 3.2. 带库硬件故障应急步骤及恢复策略 ·············································· 4 3.2.1. 带库单点故障 ··························································································· 4 3.2.2. 驱动器故障 ······························································································· 6 3.2.3. 驱动器卡带故障 ······················································································· 8 3.3. 备份软件故障应急步骤及恢复策略 ·············································· 8 3.3.1. 备份客户端软件故障 ··············································································· 8 3.3.2. 备份服务器备份服务启不来 ··································································· 9 3.4. 备份服务器硬件故障 ······························································· 10 3.4.1. 备份服务器硬件故障应急步骤及恢复策略 ········································· 10 3.5. 光纤交换机故障应急步骤及恢复策略 ·········································· 10 3.5.1. 光纤交换机个别端口故障 ····································································· 10 3.5.2. 光纤线有问题 ························································································· 11 3.6. 网络故障应急步骤及恢复策略 ··················································· 11 3.7. 主机HBA故障 ······································································ 11 集中备份系统应急方案
Page1 of 14 4/25/2022 1. 前言 集中备份系统由IBM 3584带库/STK L1400、SYMANTEC|netbackup备份软件、SYMANTEC|netbackup备份客户端、光纤交换机,IBM备份服务器等组成。其中由于3584带库及L1400带库属机械设备,存在出现机械故障的风险点,还有其他组件也存在出现故障的可能性;现针对集中备份系统可能出现的故障点,为后期备份系统的进一步完善,特制定本应急方案。
说明:以下操作步骤如果没有特殊说明都是由备份管理员操作.
2. 数据库日志备份应急措施 针对Oracle ,Informix数据库在线备份,都需要打开归档功能做日志备份.如果备份系统出现问题,不能进行日志备份的话,需采取如下应急措施:
2.1. Oracle数据库的应急处理方法 目前集中备份系统使用ORACLE数据库的业务为ODSB,且归档日志与ODSB项目组确认后是可以删除的,清理日志的机制已下发分行。如分行有特色系统使用ORACLE数据库且已经加入集中备份系统: 1、保留归档日志 方法一. 可更改归档日志生成路径,但确保更改后的文件系统足够大。 命令举例:
方法二. 当归档日志文件系统使用率达60%,可将归档日志拷贝、转移到其他文件系统(本机或异机)。 2、不保留归档日志 集中备份系统应急方案 Page 2 of 14 4/25/2022 2
数据库管理员可使用rman命令删除归档日志(不要直接使用rm命令删除),命令举例:
如果希望归档日志自动清理,请参照ODSB归档日志自动清理机制。 2.2. Informix数据库的应急处理方法 参数配置前提 针对重要系统informix数据库的逻辑日志采用onbar进行备份时,目前的LTAPEDEV通常设置为一个文件,不能是null。 故障现象 当日志无法备份时,从数据库的角度,出现大量未备份的日志,(U------),最终导致日志被用满,数据库的日志出现错误 故障处理步骤 1、停止带库备份软件进程(如果临时应急时间很短,可以考虑不停止) 2、检查系统中是否有挂起的onbar进程,如果存在,kill杀掉该进程(如果该进程僵死,存在杀不掉的可能) 3、查看目前onconfig中的LTAPE配置参数
察看当前的LTAPEDEV所配置的文件是否存在 咨询具体负责人,是否要保留日志 不保留日志:
LTAPEDEV /home/db/informix/log.bak # Log tape device path LTAPEBLK 32 # Log tape block size (Kbytes) LTAPESIZE 102400 # Max amount of data to put on log tape (Kbytes) 集中备份系统应急方案 Page 3 of 14 4/25/2022 3
1、将/dev/null链接到具体的文件上,以上图为例,ln –s /dev/null /home/db/informix/log.bak 2、修改onconfig参数ALARMPROGRAM所指向的文件,即逻辑日志备份脚本,将BACKUP_CMD="onbar -b -l"改为BACKUP_CMD="ontape -a" 3、再次确认所有onbar进程已经被kill掉 4、通过onstat –l监控目前日志的变化情况 注:此过程无需重启informix数据库。 保留逻辑日志: 1、可将逻辑日志备份至本地磁带机或文件系统,如果备份至文件系统需保证文件系统空间(逻辑日志落地空间)充足; 2、在有空间的文件系统创建一个文件(权限:660;informix:informix),并将文件链接到LTAPEDEV所指向的文件,或将LATPEDEV指向磁带机设备; 3、修改onconfig参数ALARMPROGRAM所指向的文件,即逻辑日志备份脚本,将BACKUP_CMD="onbar -b -l"改为BACKUP_CMD="",再直接运行ontape -c,这个命令的作用是是连续不断地备份数据库逻辑日志,只需要运行一次,一直挂界面上运行,或则放在后台运行; 4、再次确认所有onbar进程已经被kill掉,并使用onstat –l查看日志状态; 5、对已经备份出来的日志进行妥善保管,以应对紧急情况下的日志回滚,并加强对文件系统空间使用率监控。 注:此过程无需重启informix数据库。
3. 集中备份系统故障应急措施 3.1. 故障分类 (1)带库硬件故障(A.机械臂,B.控制面板,C.IO Station,D.MCP卡(负责磁带机通讯),E.ACC卡(控制机械臂的电路板)等单点故障、驱动器故障)
(2)NBU备份软件故障(备份客户端软件故障、备份服务器备份进程异常) (3)备份服务器故障(备份服务器硬件故障) (4)光纤交换机故障(光纤交换机个别端口故障、光纤线有问题) (5)网络故障(网络不通) (6)主机HBA故障