AIX常见问题

AIX⽂件系统var空间100%的问题⼀、问题说明/var/spool/mqueue⽬录下出现了多个df打头的⽂件，导致/var空间最终100%EBANK_P570_MAIN/var/spool/mqueue#ls -ltotal 8732656-rw-r--r-- 1root system 571 Aug 20 2012 Qfq7JGw5K21643126-rw-r--r-- 1root system 569 Aug 23 2012 Qfq7MGvxch987418-rw-r--r-- 1root system 572 Jun 25 02:00 Qfs5OHxF6d1339552-rw-r--r-- 1root system 570 Jul 04 01:59 Qfs63HwefO1359972-rw-r--r-- 1root system 773 Jul 04 02:02 Qfs63I1DfN1790554-rw-r--r-- 1root system 570 Jul 06 02:00 Qfs65Hx5Vq1315640-rw-r--r-- 1root system 773 Jul 06 02:02 Qfs65I16Vp1483544-rw------- 1root system 760 Jul 06 02:39 Qfs65IaKdR1397376-rw-r--r-- 1root system 573 Jul 07 00:58 Qfs66GvJMf1332120-rw-r--r-- 1root system 0 Aug 23 2012 dfq7JGw5K21643126-rw-r--r-- 1root system 0 Aug 23 2012 dfq7MGvxch987418-rw-r--r-- 1root system 1153011528 Jul 06 02:00 dfs65Hx5Vq1315640-rw-r--r-- 1root system 1153012885 Jul 06 02:02 dfs65I16Vp1483544-rw------- 1root system 1153012885 Jul 06 02:39 dfs65IaKdR1397376-rw-r--r-- 1root system 1011801454 Jul 07 00:58 dfs66GvJMf1332120⼆、原因分析：1、查询资料：/var/spool/mqueue⽬录包含与队列中的每条消息关联的临时⽂件。

AIX操作系统卷组故障维护AIX操作系统卷组故障维护故障描述：4.20日早晨，发现日报没有正常发送，登录数据库备机查看原因，查看系统的log命令：errpt |more没有发现什么异常，不过发现有如下错误：F3931284 0410055009 I H ent2 ETHERNET NETWORK RECOVERY MODEF3931284 0410055009 I H ent0 ETHERNET NETWORK RECOVERY MODE 173C787F 0410053709 I S topsvcs Possible malfunction on local adapter173C787F 0410053709 I S topsvcs Possible malfunction on local adapterEC0BCCD4 0410053709 T H ent2 ETHERNET DOWNEC0BCCD4 0410053709 T H ent0 ETHERNET DOWN这个时间正好是同事更换以太网交换机的时间查看数据库同步脚本log：# sh /home/oracle/sh/rmanres.sh[YOU HAVE NEW MAIL]0516-040 lqueryvg: Unable to read the specified physical volumedescriptor area.0516-932 /usr/sbin/syncvg: Unable to synchronize volume group backvg.[YOU HAVE NEW MAIL]restoring datafile 00058 to /u01/oracle/product/9.2.0/oradata/orcl/yy33.dbf restoring datafile 00059 to/u01/oracle/product/9.2.0/oradata/orcl/yy34.dbfreleased channel: ch1RMAN-00571:========================================= =============== ===RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ======== =======RMAN-00571:========================================= =============== ===RMAN-03002: failure of restore command at 04/20/2009 12:06:25ORA-19501: read error on file "/u03/orabackup/rman/orcl_db_684391660_523_1", blo ckno 8192001 (blocksize=8192)ORA-27063: skgfospo: number of bytes read/written is incorrectIBM AIX RISC System/6000 Error: 12: Not enough spaceAdditional information: -1Additional information: 1048576ORA-19501: read error on file "/u03/orabackup/rman/orcl_db_684391660_523_1", blo ckno 8191873 (blocksize=8192)ORA-27063: skgfospo: number of bytes read/written is incorrectRecovery Manager complete.[YOU HAVE NEW MAIL]SQL*Plus: Release 9.2.0.1.0 - Production on Mon Apr 20 12:06:26 2009Copyright (c) 1982, 2002, Oracle Corporation. All rightsreserved.SP2-0640: Not connectedSP2-0640: Not connectedERROR:ORA-12500: TNS:listener failed to start a dedicated server processSP2-0640: Not connectedSP2-0640: Not connected系统日志：# ps -ef |moreUID PID PPID C STIME TTY TIME CMDroot 1 0 0 Dec 16 - 0:55 /etc/initroot 61572 78170 0 Dec 16 - 359:56 dtgreetroot 69798 1 0 Dec 16 - 0:00 /usr/lib/errdemonroot 73882 1 0 Dec 16 - 71:56 /usr/sbin/syncd 60root 90242 1 0 Dec 16 - 0:00 /usr/dt/bin/dtlogin -daemon root 102438 344388 0 13:18:46 pts/7 0:00 -kshroot 118898 102438 0 13:19:03 pts/7 0:00 ps -efroot 127086 1 0 Dec 16 - 0:00 /usr/ccs/bin/shlap64root 143514 106918 0 Dec 16 - 0:00 /usr/sbin/rsct/bin/IBM.ERrmdroot 155816 106918 0 Dec 16 - 2:24 /usr/sbin/rsct/bin/IBM.CSMAgentRMd root 159976 106918 0 Dec 16 - 3:08 /usr/sbin/rsct/bin/rmcd -a IBM.LPCom mands -r root 164070 352610 0 Dec 16 - 37:11 /usr/sbin/rsct/bin/hats_nimdaemon 168160 106918 0 Dec 16 - 0:00 /usr/sbin/rpc.statd -d 0 -t 50oracle 180262 1 0 Dec 16 - 0:02 ora_reco_rmandbroot 184400 106918 0 Dec 16 - 1:01 /usr/sbin/gsclvmdoracle 205000 1 0 11:26:43 - 0:00 ora_pmon_orclroot 233570 106918 0 Dec 16 - 7:56 /usr/sbin/rsct/bin/IBM.HostRMd oracle 237696 1 0 12:29:22 - 0:00 oracleorcl (LOCAL=NO)root 241712 352610 0 Dec 16 - 50:29 /usr/sbin/rsct/bin/hats_rs232_nim root 245830 106918 0 Dec 16 - 0:00 /usr/sbin/muxatmdroot 278610 352610 0 Dec 16 - 30:31 /usr/sbin/rsct/bin/hats_nimoracle 307362 1 0 Dec 16 - 0:06 ora_d000_rmandbroot 315394 106918 0 Dec 16 - 0:10 /usr/sbin/aixmibdroot 352384 106918 0 Dec 16 - 0:05 /usr/sbin/snmpmibdroot 372834 1 0 12:13:02 - 0:00 lsvg -ooracle 389264 1 0 11:26:43 - 0:00 ora_ckpt_orclroot 393248 1 0 12:11:24 - 0:00 lsvg -oroot 397368 1 0 12:21:43 - 0:00 lsvg -oroot 405556 1 0 12:15:51 - 0:00 lspvroot 417854 450810 0 12:06:28 - 0:00 lqueryvg -g 00c64e4b00004c0000000 11dbddadf95 -CXroot 426226 1 0 12:47:15 - 0:00 lsvg statvgoracle 434210 1 0 12:07:13 - 0:00 oracleorcl (LOCAL=NO)oracle 442388 1 0 11:26:43 - 0:00 ora_lgwr_orcloracle 446680 1 0 11:26:43 - 0:00 ora_dbw0_orclroot 450810 1 0 12:06:28 - 0:00 /usr/bin/ksh /usr/sbin/varyoffvg backvg root 61802 90242 0 Dec 16 - 8:20 /usr/lpp/X11/bin/X -D /usr/lib/X11//rgb -T -force :0 -auth /var/dt/A:0-ozyiiaroot 74076 106918 0 Dec 16 - 1:34 /usr/sbin/snmpdroot 78170 90242 0 Dec 16 - 0:00 dtlogin <:0> -daemonroot 86416 106918 0 Dec 16 - 0:02 /usr/sbin/syslogdroot 94582 106918 0 Dec 16 - 0:00 /usr/sbin/inetdroot 98768 106918 0 Dec 16 - 13:14 /usr/es/sbin/cluster/clcomd -droot 106918 1 0 Dec 16 - 0:00 /usr/sbin/srcmstrroot 115134 106918 0 Dec 16 - 0:00 /usr/sbin/portmaproot 119210 1 0 Dec 16 - 0:22 /usr/sbin/cronroot 131516 1 0 Dec 16 - 0:00 /usr/sbin/uprintfdroot 139680 1 0 Dec 16 lft0 0:00 /usr/sbin/getty /dev/consoleroot 143754 102438 0 13:19:03 pts/7 0:00 moreroot 151986 106918 0 Dec 16 - 0:00 /usr/sbin/rsct/bin/IBM.ServiceRMd root 156076 106918 0 Dec 16 - 0:00 /usr/sbin/rsct/bin/IBM.AuditRMd oracle 168230 1 0 11:26:43 - 0:00 ora_d000_orcloracle 172368 1 0 11:26:43 - 0:00 ora_arc0_orcloracle 287158 1 0 11:26:43 - 0:00 ora_smon_orcloracle 299364 1 0 11:26:43 - 0:00 ora_reco_orclroot 319924 1 0 11:51:24 - 0:00 lspv hdisk5root 332234 106918 0 Dec 16 - 5:53 hagsd grpsvcsoracle 336330 1 0 Dec 16 - 5:07 ora_dbw0_rmandbroot 344388 94582 0 13:18:45 - 0:00 telnetd -aroot 352610 106918 0 Dec 16 - 55:44 /usr/sbin/rsct/bin/hatsd -n 1 -o dead ManSwitchoracle 356856 1 0 Dec 16 - 11:53 ora_ckpt_rmandboracle 360852 1 0 Dec 16 - 5:24 ora_smon_rmandbroot 369086 106918 0 Dec 16 - 51:38 /usr/es/sbin/cluster/clstrmgrroot 389556 106918 0 Dec 16 - 11:02 /usr/es/sbin/cluster/clinfooracle 393484 1 0 Dec 16 - 4:17 ora_pmon_rmandboracle 418112 1 0 Dec 16 - 0:04 /home/oracle/product/9.2.0/bin/tnslsnr LI STENER -inherit root 422200 106918 0 Dec 16 - 0:08 haemd HACMP 1 Cluster SECNOS UPPORTroot 438682 106918 0 Dec 16 - 0:05 /usr/sbin/qdaemonroot 442776 106918 0 Dec 16 - 0:00 /usr/sbin/rpc.lockd -d 0 root 446934 106918 0 Dec 16 - 0:00 /usr/sbin/writesrvroot 451032 106918 0 Dec 16 - 0:00 /usr/sbin/biod 6root 471540 106918 0 Dec 16 - 0:21 sendmail: accepting connections oracle 479602 1 0 Dec 16 - 1:33 ora_lgwr_rmandb root 491900 106918 0 Dec 16 - 0:05 /usr/sbin/hostmibdoracle 495908 1 0 11:26:43 - 0:00 ora_arc1_orcl环境：两台小机，一个存储阵列，两台机器是hacmp的有三个卷组，dbvg, statvg, backvg主机卷组dbvg备机卷组：statvgbackvg两机都可以访问，用于备份的问题描述：现在备机只要是执行和卷组，pv相关的命令就挂在那，没有反应我通过进程信息，可以判断是卷组锁定了backvg，我执行过的操作，再备机上：chvg -u backvg ，已经3个小时了，还是没有结果，挂载那然后又在备机上执行exportvg backvg 又很长时间了，一个多小时，还是挂在那，请问如何解决这个问题，解锁backvg，我在主机varyonvg backvg时，提示：# varyonvg backvg0516-013 varyonvg: The volume group cannot be varied on because there are no good copies of the descriptor area.Command: failed stdout: yes stderr: noBefore command completion, additional instructions may appear below.0516-024 lqueryvg: Unable to open physical volume.Either PV was not configured or could not be opened. Run diagnostics.0516-024 lqueryvg: Unable to open physical volume.Either PV was not configured or could not be opened. Run diagnostics.0516-1140 importvg: Unable to read the volume group descriptor area on specified physical volume.问题产生的原因：因为backvg卷组是共享卷组(不是并发卷组),在每日的04:00-05:40这段时间是数据库用backvg备份,而在每次使用卷组的时候都要更改卷组的vgda,vgsa中的时间戳,而在这段时间里同事更换了交换机,导致两个小机的卷组的VGDA不一致从而会出现这个错误解决方法：首要目的:让备机释放掉对pv,卷组的管理进程,以达到我可以从新管理备机的卷组信息由于一些原因,我强行kill掉相关LVM命令,导致这些进程都被系统接管,根本无法再kil l掉,即使用kill -9,也是不可以我当时在想有两个方法可以解决此种情况1.有一些特殊的方法可以kill掉这些进程2.重新启动机器让其释放所有资源咨询了很多人,又google半天,也没有找到可以kill那些进程的方法最后决定重启机器因为我的环境是两台小机做了hacmp,为了避免出万一,决定23号凌晨去机房维护,出什么问题也好就近解决主要是担心网卡down了,远程连接不上当到了机房,就在外边的维护室(机房太冷了!!能不进去就不进去啊), 我的hacmp配置为有优先级的cascading模式，按优先级来接管资源。

Hello,Good day all.Could someone this group help me out in fixing my problem? When I triedto run my application it is giving me the following error...exec(): 0509-036 Cannot load program ./sssu_aix because of thefollowing errors:0509-130 Symbol resolution failed for sssu_aix because:0509-136 Symbol _Getctype__FPCc (number 101) is not exportedfromdependent module /usr/lib/libC.a(ansi_32.o).0509-136 Symbol _Getnumpunct__FPCc (number 105) is notexported fromdependent module /usr/lib/libC.a(ansi_32.o).0509-192 Examine .loader section symbols with the'dump -Tv' command.What could be the possible reason? My application runs on the machinewhere it was compiled. When I tried to execute it on the machine, thatdoes not have compiler installed it is giving me the above said error.Thanks in advancce.~hemIn article <1137404964.542841.124500@>, hem <hemachandra.reddy@> wrote:Quote:Do this on both machines:# lslpp -l xlC.rteI'm guessing you have a lower version of xlC.rte installed on the machine without a compiler installed.You do not need to install a compiler on the other machine; you just need to make sure xlC.rte is at least the same version as the machine you built the application on.Otherwise, you run into issues with potentially different APIs or symbols -- the usual 'backwards compatibility' credo: older -> newer ok; newer -> older not ok.-DanHi Foster,Thanks for your quick reference.I have a different xlC.rte at both the sides. Can I eliminate this dependency without having to install xlC.rte at the customer place. Isit possible to staically link this xlC.rte while linking?Regards,Hem"hem" <hemachandra.reddy@> writes:Quote:You can "solve" the problem by downgrading xlC.rte on your machine to the lowest level you want to support. You should also explicitlystate what minimum level of xlC.rte you *are* supporting.Quote:This is possible, but a) complicated to get right; b) not recommended;c) not supported (AFAIK).Cheers,--In order to understand recursion you must first understand recursion. Remove /-nsp/ for email.Hi Paul,Thanks for your reply. Now I want to uninstall the xlc.rte7.0 and to install xlc.rte6.0 in my machine. Could you help me where can I get the RPM to install it? Also install and uninstall process of xlC as I amquite new to AIX.Regards,hemxlC.rte7.0 is installed when I installvacpp.60.aix50.sep2005.ptf.tar.Z. Can I uninstall only xlC.rte7.0 and install xlC.rte6.0? Does xlC.rte6.0 has any dependency on vacpp?Can you point me the vacpp package having xlC.rte6.0? My machine is installed with AIX5.3.hem schrieb:Quote:vac and vacpp have dependencies on xlC.rte.xlC.rte is redistributable with your apps.The latest levels of VAC6 rely on the VAC7 C++ runtime for AIX5L Quote:intstall xlC.rte and xlC.rte.aix50 from the Sep 05 maintenance packageon you AIX 5.3 box.The version of runtime libs in production should always be equal or newer than on the development machine. Not less.--UliHi Uli,From/support/docview.wss?uid=swg24010550I read that the xlC.rt.aix50 Sep05 patch has xlC.rte.7.0.0.0 andxlC.rte.7.0.0.1 and not the xlC.rte6.0.I tried downloading most of the packages from the above URL. All are having xlC7.0.Could you help me out?Thanks,Reddy。

AIX LVM一．LVM基本概念1.PV物理卷，通常操作系统层面一块硬盘就是一个PV，PV必须加入VG后才能使用其存储空间每个PV头部都有一个保留区域，记录PVID等信息。

当将硬盘加入操作系统后，执行cfgmgr或者chedev –l hdiskx –a pv=yes会为此硬盘分配pvid。

chdev –l hdiskx –a pv=clear 会清除pvid信息，慎用。

硬盘有了PVID后，才能进行后续的LVM操作。

# lspv查看PVIDhdisk0 000a1ddc7f36a2f0 rootvg active2.VG卷组，由一个或者多个PV组成，VG的信息保存在VGSA和VGDA中。

AIX卷组有三种类型：Normal，Big，Scalable，区别如下：●LP逻辑分区，相当于HPUX的LE●PP物理分区，类似与HPUX的PE，大小为2的幂—-4、8…1024MB●LV逻辑卷，由LP组成，1个LP对应1-3个PP，AIX中常见的逻辑卷用途如下：●LVCB逻辑卷控制块，位于逻辑卷的开始位置，占用512个字节。

逻辑卷控制块包括的信息有：逻辑卷创建日期逻辑卷的镜像拷贝数和安装点。

（HPUX中逻辑卷不存在LVCB）可以通过getlvcb命令查看LVCB信息# getlvcb -AT lvtestAIX LVCBintrapolicy = mcopies = 2interpolicy = mlvid = 00c97cbe00004c000000012e3282db2e.1lvname = lvtestlabel = /testmachine id = 97CBE4C00number lps = 1relocatable = ystrict = ystripe width = 0stripe size in exponent = 0type = jfs2upperbound = 32fs = vfs=jfs2:log=INLINE:options=rw:account=falsetime created = Thu Feb 17 15:55:06 2011time modified = Thu Feb 17 15:59:41 2011●VGDA卷组描述区，包含整个卷组，包括逻辑卷，物理卷的信息，LP与PP的对应关系通常情况，1块硬盘组成的vg，每块硬盘保存两份VGDA；两块硬盘组成的vg，其中一块硬盘保存两份VGDA，另外一块保存一份VGDA；三块硬盘组成的卷组，每块硬盘保存一份VGDA。