heartbeat crm模塊如何使用
現狀: 兩台apache httpd伺服器,各自單獨的對外提供服務
需求:希望一號機作為主機,二號機作為備機,使用一個對外的IP地址對外提供服務
正常運行情況下,該IP在一號機上,當一號機宕機,網線斷網,或者httpd服務down掉時,
IP漂移到二號機上。
目前我使用heartbeat可以做到當一號機宕機、網線斷網時,IP能飄到二號機,但是沒法監控到
httpd這個應用的運行情況,網上有些文章介紹了heartbeat 的crm模塊,可我不會使用
不知道哪位用過這東西,能給點意見,謝謝
我注意到有些文章說要自己寫一個腳本,腳本能支持 start stop status 參數,能返回固定格式的字元串
小弟真是才疏學淺,不知道對於apache httpd 服務而言,這個腳本怎麼寫,哪位能給個例子嗎?
《解決方案》
目前配置了兩個節點
kf28-1 IP 192.1.101.128 固定地址 主節點
oradb2-1 IP 192.1.101.12 固定地址 備節點
虛擬IP 192.1.101.212
兩台機器上均安裝配置好了apache httpd
在沒有配置crm之前 , 兩個節點分別做 heartbeat start ,隨後兩個節點的httpd服務均能被拉起來,然後192.1.101.212 能被分配到 kf28-1節點上
在kf28-1 上做heartbeat stop之後, 該節點上的httpd能被down掉,隨後192.1.101.212 漂移至oradb2-1上
隨後再在kf28-1 上做heartbeat start之後, 該節點上的httpd能被啟動,隨後192.1.101.212能夠漂移回到kf28-1上
在oradb2-1上做heartbeat stop之後, 該節點上的httpd能被down掉。
如上所述,表現一切正常。
接著我希望heartbeat能監視httpd的運行情況,希望在某節點的httpd服務down掉之後,另一節點能接管192.1.101.212這個ip
按照網上的資料描述,我自己寫了一個符合lsb規範的腳本myhttpd,並參考網上的crm教程做了相關配置,現在發現出了問題,現象描述如下:
1、kf28-1 上啟動heartbeat ,能在一段時候后拉起httpd服務,192.1.101.212這個ip也能分配到這個節點上,
在kf28-1上使用crm_mon命令查看,結果如下:
Refresh in 10s...
============
Last updated: Tue Aug 21 16:50:18 2012
Current DC: kf28-1 (b275747d-b787-43c1-b05e-15a84603ebbf)
2 Nodes configured.
1 Resources configured.
============
Node: oradb2-1 (34bd366d-3b86-4cb1-9bb1-b901f0e4e08b): OFFLINE
Node: kf28-1 (b275747d-b787-43c1-b05e-15a84603ebbf): online
Resource Group: group_1
IPaddr_192_1_101_212 (heartbeat::ocf:IPaddr): Started kf28-1
myhttpd_2 (lsb:myhttpd): Started kf28-1
2、在oradb2-1 上啟動heartbeat ,無法拉起httpd應用,使用crm_mon查看,結果如下:
Not connected:Refresh in 3s...
下面我將兩個節點的相關配置文件列示如下
ha.cf (該文件在兩個節點基本相同):
# ha.cf
debugfile /var/log/ha-debug
logfile /var/log/ha-log
keepalive 2
deadtime 30
warntime 10
initdead 120
udpport 694
ucast eth0 192.1.101.12 (oradb2-1節點上這一行寫的是 ucast eth0 192.1.101.128)
auto_failback on
crm yes
node kf28-1
node oradb2-1
authkeys (該文件在兩個節點相同):
auth 1
1 crc
#2 sha1 HI!
#3 md5 Hello!
haresources(該文件在兩個節點上相同):
kf28-1 192.1.101.212 myhttpd
myhttpd是我寫的一個腳本,放在/etc/init.d中 ,內容如下(兩個節點均放置了這個腳本):
#!/bin/sh
ARGV="$@"
#echo $ARGV
#echo "------------"
Start="start"
#echo $Start
Stop="stop"
#echo $Stop
Status="status"
if [ "$ARGV" == "$Start" ];then
echo " starting httpd ..............."
/users/ems/apache/bin/apachectl start
elif [ "$ARGV" = "$Stop" ];then
echo " stopping httpd ..............."
/users/ems/apache/bin/apachectl stop
elif [ "$ARGV" = "$Status" ];then
http=`ps -A |grep httpd |wc -l`
two=2
let "http=http-two"
if [ $http -gt 0 ];then
echo "OK."
else
echo "stopped"
fi
else
echo " can not accept the input parameter......"
fi
該腳本中對status的判斷中減去了一個2,是因為在使用status選項啟動的時候,即使系統沒有啟動httpd服務,腳本也報有2個,不知道怎麼回事,所以我就強行減去了個2,這樣就符合了lsb的規範,哪位知道是什麼原因,請不吝賜教。
生成的cib.xml文件分別如下:
kf28-1: 下面有好幾個類似的文件
cib.xml內容為:
<cib admin_epoch="0" epoch="4" num_updates="61" generated="true" have_quorum="true" ignore_dtd="false" num_peers="2" cib_feature_revision="1.3" cib-last-written="Tue Aug 21 17:06:18 2012" ccm_transition="1" dc_uuid="b275747d-b787-43c1-b05e-15a84603ebbf">
<configuration>
<crm_config>
<cluster_property_set id="cib-bootstrap-options">
<attributes>
<nvpair id="cib-bootstrap-options-symmetric-cluster" name="symmetric-cluster" value="true"/>
<nvpair id="cib-bootstrap-options-no_quorum-policy" name="no_quorum-policy" value="stop"/>
<nvpair id="cib-bootstrap-options-default-resource-stickiness" name="default-resource-stickiness" value="0"/>
<nvpair id="cib-bootstrap-options-default-resource-failure-stickiness" name="default-resource-failure-stickiness" value="0"/>
<nvpair id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" value="false"/>
<nvpair id="cib-bootstrap-options-stonith-action" name="stonith-action" value="reboot"/>
<nvpair id="cib-bootstrap-options-stop-orphan-resources" name="stop-orphan-resources" value="true"/>
<nvpair id="cib-bootstrap-options-stop-orphan-actions" name="stop-orphan-actions" value="true"/>
<nvpair id="cib-bootstrap-options-remove-after-stop" name="remove-after-stop" value="false"/>
<nvpair id="cib-bootstrap-options-short-resource-names" name="short-resource-names" value="true"/>
<nvpair id="cib-bootstrap-options-transition-idle-timeout" name="transition-idle-timeout" value="5min"/>
<nvpair id="cib-bootstrap-options-default-action-timeout" name="default-action-timeout" value="5s"/>
<nvpair id="cib-bootstrap-options-is-managed-default" name="is-managed-default" value="true"/>
</attributes>
</cluster_property_set>
</crm_config>
<nodes>
<node id="34bd366d-3b86-4cb1-9bb1-b901f0e4e08b" uname="oradb2-1" type="normal"/>
<node id="b275747d-b787-43c1-b05e-15a84603ebbf" uname="kf28-1" type="normal"/>
</nodes>
<resources>
<group id="group_1">
<primitive class="ocf" id="IPaddr_192_1_101_212" provider="heartbeat" type="IPaddr">
<operations>
<op id="IPaddr_192_1_101_212_mon" interval="5s" name="monitor" timeout="5s"/>
</operations>
<instance_attributes id="IPaddr_192_1_101_212_inst_attr">
<attributes>
<nvpair id="IPaddr_192_1_101_212_attr_0" name="ip" value="192.1.101.212"/>
</attributes>
</instance_attributes>
</primitive>
<primitive class="lsb" id="myhttpd_2" provider="heartbeat" type="myhttpd">
<operations>
<op id="myhttpd_2_mon" interval="120s" name="monitor" timeout="60s"/>
</operations>
</primitive>
</group>
</resources>
<constraints>
<rsc_location id="rsc_location_group_1" rsc="group_1">
<rule id="prefered_location_group_1" score="100">
<expression attribute="#uname" id="prefered_location_group_1_expr" operation="eq" value="kf28-1"/>
</rule>
</rsc_location>
</constraints>
</configuration>
</cib>
《解決方案》
cib.xml.last內容為:
<cib admin_epoch="0" epoch="4" num_updates="61" generated="true" have_quorum="true" ignore_dtd="false" num_peers="2" cib_feature_revision="1.3" cib-last-written="Tue Aug 21 17:06:15 2012" ccm_transition="1" dc_uuid="b275747d-b787-43c1-b05e-15a84603ebbf">
<configuration>
<crm_config>
<cluster_property_set id="cib-bootstrap-options">
<attributes>
<nvpair id="cib-bootstrap-options-symmetric-cluster" name="symmetric-cluster" value="true"/>
<nvpair id="cib-bootstrap-options-no_quorum-policy" name="no_quorum-policy" value="stop"/>
<nvpair id="cib-bootstrap-options-default-resource-stickiness" name="default-resource-stickiness" value="0"/>
<nvpair id="cib-bootstrap-options-default-resource-failure-stickiness" name="default-resource-failure-stickiness" value="0"/>
<nvpair id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" value="false"/>
<nvpair id="cib-bootstrap-options-stonith-action" name="stonith-action" value="reboot"/>
<nvpair id="cib-bootstrap-options-stop-orphan-resources" name="stop-orphan-resources" value="true"/>
<nvpair id="cib-bootstrap-options-stop-orphan-actions" name="stop-orphan-actions" value="true"/>
<nvpair id="cib-bootstrap-options-remove-after-stop" name="remove-after-stop" value="false"/>
<nvpair id="cib-bootstrap-options-short-resource-names" name="short-resource-names" value="true"/>
<nvpair id="cib-bootstrap-options-transition-idle-timeout" name="transition-idle-timeout" value="5min"/>
<nvpair id="cib-bootstrap-options-default-action-timeout" name="default-action-timeout" value="5s"/>
<nvpair id="cib-bootstrap-options-is-managed-default" name="is-managed-default" value="true"/>
</attributes>
</cluster_property_set>
</crm_config>
<nodes>
<node id="34bd366d-3b86-4cb1-9bb1-b901f0e4e08b" uname="oradb2-1" type="normal"/>
<node id="b275747d-b787-43c1-b05e-15a84603ebbf" uname="kf28-1" type="normal"/>
</nodes>
<resources>
<group id="group_1">
<primitive class="ocf" id="IPaddr_192_1_101_212" provider="heartbeat" type="IPaddr">
<operations>
<op id="IPaddr_192_1_101_212_mon" interval="5s" name="monitor" timeout="5s"/>
</operations>
<instance_attributes id="IPaddr_192_1_101_212_inst_attr">
<attributes>
<nvpair id="IPaddr_192_1_101_212_attr_0" name="ip" value="192.1.101.212"/>
</attributes>
</instance_attributes>
</primitive>
<primitive class="lsb" id="myhttpd_2" provider="heartbeat" type="myhttpd">
<operations>
<op id="myhttpd_2_mon" interval="120s" name="monitor" timeout="60s"/>
</operations>
</primitive>
</group>
</resources>
<constraints>
<rsc_location id="rsc_location_group_1" rsc="group_1">
<rule id="prefered_location_group_1" score="100">
<expression attribute="#uname" id="prefered_location_group_1_expr" operation="eq" value="kf28-1"/>
</rule>
</rsc_location>
</constraints>
</configuration>
</cib>
《解決方案》
cib.xml.sig內容為:
cef71bb53253e460c4fd704f70c80258
cib.xml.sig.last內容為:
fd885cb2a132302679fdc28d0af89dc3
oradb2-1中只有一個cib.xml文件,內容為:
?xml version="1.0" ?>
<cib admin_epoch="0" epoch="0" num_updates="0">
<configuration>
<crm_config>
<cluster_property_set id="cib-bootstrap-options">
<attributes>
<nvpair id="cib-bootstrap-options-symmetric-cluster" name="symmetric-cluster" value="true"/>
<nvpair id="cib-bootstrap-options-no_quorum-policy" name="no_quorum-policy" value="stop"/>
<nvpair id="cib-bootstrap-options-default-resource-stickiness" name="default-resource-stickiness" value="0"/>
<nvpair id="cib-bootstrap-options-default-resource-failure-stickiness" name="default-resource-failure-stickiness" value="0"/>
<nvpair id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" value="false"/>
<nvpair id="cib-bootstrap-options-stonith-action" name="stonith-action" value="reboot"/>
<nvpair id="cib-bootstrap-options-stop-orphan-resources" name="stop-orphan-resources" value="true"/>
<nvpair id="cib-bootstrap-options-stop-orphan-actions" name="stop-orphan-actions" value="true"/>
<nvpair id="cib-bootstrap-options-remove-after-stop" name="remove-after-stop" value="false"/>
<nvpair id="cib-bootstrap-options-short-resource-names" name="short-resource-names" value="true"/>
<nvpair id="cib-bootstrap-options-transition-idle-timeout" name="transition-idle-timeout" value="5min"/>
<nvpair id="cib-bootstrap-options-default-action-timeout" name="default-action-timeout" value="5s"/>
<nvpair id="cib-bootstrap-options-is-managed-default" name="is-managed-default" value="true"/>
</attributes>
</cluster_property_set>
</crm_config>
<nodes/>
<resources>
<group id="group_1">
<primitive class="ocf" id="IPaddr_192_1_101_212" provider="heartbeat" type="IPaddr">
<operations>
<op id="IPaddr_192_1_101_212_mon" interval="5s" name="monitor" timeout="5s"/>
</operations>
<instance_attributes id="IPaddr_192_1_101_212_inst_attr">
<attributes>
<nvpair id="IPaddr_192_1_101_212_attr_0" name="ip" value="192.1.101.212"/>
</attributes>
</instance_attributes>
</primitive>
<primitive class="lsb" id="myhttpd_2" provider="heartbeat" type="myhttpd">
<operations>
<op id="myhttpd_2_mon" interval="120s" name="monitor" timeout="60s"/>
</operations>
</primitive>
</group>
</resources>
<constraints>
<rsc_location id="rsc_location_group_1" rsc="group_1">
<rule id="prefered_location_group_1" score="100">
<expression attribute="#uname" id="prefered_location_group_1_expr" operation="eq" value="kf28-1"/>
</rule>
</rsc_location>
</constraints>
</configuration>
<status/>
</cib>
《解決方案》
oradb2-1的日誌文件/var/log/ha-debug內容如下(由於比較多,我只列出有代表性的幾段)
......................
cib: 2012/08/21_17:25:28 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:25:28 ERROR: init_start: CCM Activation failed 30 (max) times
cib: 2012/08/21_17:25:28 ERROR: init_start: Couldnt start all communication channels, exiting.
mgmtd: 2012/08/21_17:25:28 ERROR: cib_native_signon: No reply message - disconnected - 0
mgmtd: 2012/08/21_17:25:28 info: login to cib: 3, ret:-3
heartbeat: 2012/08/21_17:25:28 info: Exiting /usr/local/lib/heartbeat/cib process 9163 returned rc 0.
heartbeat: 2012/08/21_17:25:28 ERROR: Respawning client "/usr/local/lib/heartbeat/cib":
crmd: 2012/08/21_17:25:28 ERROR: cib_native_signon: No reply message - disconnected - 0
heartbeat: 2012/08/21_17:25:28 info: Starting child client "/usr/local/lib/heartbeat/cib" (503,501)
crmd: 2012/08/21_17:25:28 WARN: cib_native_signon: Connection to CIB failed: not connected
crmd: 2012/08/21_17:25:28 WARN: do_cib_control: Couldn't complete CIB registration 20 times... pause and retry
crmd: 2012/08/21_17:25:28 WARN: crm_fsa_trigger: FSA took 29220ms to complete
heartbeat: 2012/08/21_17:25:28 info: Starting "/usr/local/lib/heartbeat/cib" as uid 503 gid 501 (pid 9414)
cib: 2012/08/21_17:25:28 info: G_main_add_SignalHandler: Added signal handler for signal 15
cib: 2012/08/21_17:25:28 info: G_main_add_TriggerHandler: Added signal manual handler
cib: 2012/08/21_17:25:28 info: G_main_add_SignalHandler: Added signal handler for signal 17
cib: 2012/08/21_17:25:28 info: main: Retrieval of a per-action CIB: disabled
cib: 2012/08/21_17:25:28 info: cib_register_ha: Signing in with Heartbeat
cib: 2012/08/21_17:25:29 info: cib_register_ha: FSA Hostname: oradb2-1
cib: 2012/08/21_17:25:29 info: readCibXmlFile: Reading cluster configuration from: /usr/local/var/lib/heartbeat/crm/cib.xml
cib: 2012/08/21_17:25:29 WARN: validate_cib_digest: No on-disk digest present
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <cib admin_epoch="0" epoch="0" num_updates="0" generated="false" have_quorum="false">
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <configuration>
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <crm_config>
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <cluster_property_set id="cib-bootstrap-options">
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <attributes>
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-symmetric-cluster" name="symmetric-cluster" value="true"/>
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-no_quorum-policy" name="no_quorum-policy" value="stop"/>
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-default-resource-stickiness" name="default-resource-stickiness" value="0"/>
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-default-resource-failure-stickiness" name="default-resource-failure-stickiness" value="0"/>
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" value="false"/>
crmd: 2012/08/21_17:25:29 info: crm_timer_popped: Wait Timer (I_NULL) just popped!
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stonith-action" name="stonith-action" value="reboot"/>
crmd: 2012/08/21_17:25:29 WARN: cib_native_signon: Connection to CIB failed: connection failed
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stop-orphan-resources" name="stop-orphan-resources" value="true"/>
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stop-orphan-actions" name="stop-orphan-actions" value="true"/>
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-remove-after-stop" name="remove-after-stop" value="false"/>
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-short-resource-names" name="short-resource-names" value="true"/>
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-transition-idle-timeout" name="transition-idle-timeout" value="5min"/>
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-default-action-timeout" name="default-action-timeout" value="5s"/>
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-is-managed-default" name="is-managed-default" value="true"/>
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: </attributes>
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: </cluster_property_set>
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: </crm_config>
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <nodes/>
mgmtd: 2012/08/21_17:25:29 info: login to cib: 4, ret:-10
cib: 2012/08/21_17:25:29 info: log_data_element: readCibXmlFile: <resources>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: <group id="group_1">
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: <primitive class="ocf" id="IPaddr_192_1_101_212" provider="heartbeat" type="IPaddr">
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: <operations>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: <op id="IPaddr_192_1_101_212_mon" interval="5s" name="monitor" timeout="5s"/>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: </operations>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: <instance_attributes id="IPaddr_192_1_101_212_inst_attr">
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: <attributes>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: <nvpair id="IPaddr_192_1_101_212_attr_0" name="ip" value="192.1.101.212"/>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: </attributes>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: </instance_attributes>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: </primitive>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: <primitive class="lsb" id="myhttpd_2" provider="heartbeat" type="myhttpd">
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: <operations>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: <op id="myhttpd_2_mon" interval="120s" name="monitor" timeout="60s"/>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: </operations>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: </primitive>
crmd: 2012/08/21_17:25:30 WARN: cib_native_signon: Connection to CIB failed: connection failed
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: </group>
crmd: 2012/08/21_17:25:30 WARN: do_cib_control: Couldn't complete CIB registration 21 times... pause and retry
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: </resources>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: <constraints>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: <rsc_location id="rsc_location_group_1" rsc="group_1">
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: <rule id="prefered_location_group_1" score="100">
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: <expression attribute="#uname" id="prefered_location_group_1_expr" operation="eq" value="kf28-1"/>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: </rule>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: </rsc_location>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: </constraints>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: </configuration>
cib: 2012/08/21_17:25:30 info: log_data_element: readCibXmlFile: <status/>
mgmtd: 2012/08/21_17:25:31 info: login to cib failed
cib: 2012/08/21_17:25:31 info: log_data_element: readCibXmlFile: </cib>
mgmtd: 2012/08/21_17:25:31 ERROR: Can't initialize management library.Shutting down.(-1)
heartbeat: 2012/08/21_17:25:31 WARN: Exiting /usr/local/lib/heartbeat/mgmtd -v process 8963 returned rc 1.
heartbeat: 2012/08/21_17:25:31 ERROR: Respawning client "/usr/local/lib/heartbeat/mgmtd -v":
heartbeat: 2012/08/21_17:25:31 info: Starting child client "/usr/local/lib/heartbeat/mgmtd -v" (0,0)
heartbeat: 2012/08/21_17:25:31 info: Starting "/usr/local/lib/heartbeat/mgmtd -v" as uid 0 gid 0 (pid 9466)
cib: 2012/08/21_17:25:31 notice: readCibXmlFile: Enabling DTD validation on the existing (sane) configuration
mgmtd: 2012/08/21_17:25:31 info: G_main_add_SignalHandler: Added signal handler for signal 15
cib: 2012/08/21_17:25:31 info: startCib: CIB Initialization completed successfully
crmd: 2012/08/21_17:25:31 info: crm_timer_popped: Wait Timer (I_NULL) just popped!
mgmtd: 2012/08/21_17:25:31 debug: Enabling coredumps
cib: 2012/08/21_17:25:31 WARN: init_start: CCM Activation failed
mgmtd: 2012/08/21_17:25:31 info: G_main_add_SignalHandler: Added signal handler for signal 10
cib: 2012/08/21_17:25:31 WARN: init_start: CCM Connection failed 1 times (30 max)
mgmtd: 2012/08/21_17:25:31 info: G_main_add_SignalHandler: Added signal handler for signal 12
mgmtd: 2012/08/21_17:25:31 info: init_crm
cib: 2012/08/21_17:25:32 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:25:32 WARN: init_start: CCM Connection failed 2 times (30 max)
cib: 2012/08/21_17:25:33 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:25:33 WARN: init_start: CCM Connection failed 3 times (30 max)
cib: 2012/08/21_17:25:34 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:25:34 WARN: init_start: CCM Connection failed 4 times (30 max)
cib: 2012/08/21_17:25:35 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:25:35 WARN: init_start: CCM Connection failed 5 times (30 max)
cib: 2012/08/21_17:25:36 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:25:36 WARN: init_start: CCM Connection failed 6 times (30 max)
cib: 2012/08/21_17:25:37 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:25:37 WARN: init_start: CCM Connection failed 7 times (30 max)
cib: 2012/08/21_17:25:38 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:25:38 WARN: init_start: CCM Connection failed 8 times (30 max)
cib: 2012/08/21_17:25:39 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:25:39 WARN: init_start: CCM Connection failed 9 times (30 max)
cib: 2012/08/21_17:25:40 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:25:40 WARN: init_start: CCM Connection failed 10 times (30 max)
cib: 2012/08/21_17:25:41 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:25:41 WARN: init_start: CCM Connection failed 11 times (30 max)
cib: 2012/08/21_17:25:42 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:25:42 WARN: init_start: CCM Connection failed 12 times (30 max)
cib: 2012/08/21_17:25:43 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:25:43 WARN: init_start: CCM Connection failed 13 times (30 max)
cib: 2012/08/21_17:25:44 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:25:44 WARN: init_start: CCM Connection failed 14 times (30 max)
cib: 2012/08/21_17:25:45 WARN: init_start: CCM Activation failed
《解決方案》
cib: 2012/08/21_17:24:25 ERROR: init_start: Couldnt start all communication channels, exiting.
crmd: 2012/08/21_17:24:25 ERROR: cib_native_signon: No reply message - disconnected - 0
heartbeat: 2012/08/21_17:24:25 info: Exiting /usr/local/lib/heartbeat/cib process 8631 returned rc 0.
crmd: 2012/08/21_17:24:25 WARN: cib_native_signon: Connection to CIB failed: not connected
heartbeat: 2012/08/21_17:24:25 ERROR: Respawning client "/usr/local/lib/heartbeat/cib":
heartbeat: 2012/08/21_17:24:25 info: Starting child client "/usr/local/lib/heartbeat/cib" (503,501)
mgmtd: 2012/08/21_17:24:25 ERROR: cib_native_signon: No reply message - disconnected - 0
mgmtd: 2012/08/21_17:24:25 info: login to cib: 3, ret:-3
heartbeat: 2012/08/21_17:24:25 info: Starting "/usr/local/lib/heartbeat/cib" as uid 503 gid 501 (pid 8918)
cib: 2012/08/21_17:24:25 info: G_main_add_SignalHandler: Added signal handler for signal 15
cib: 2012/08/21_17:24:25 info: G_main_add_TriggerHandler: Added signal manual handler
cib: 2012/08/21_17:24:25 info: G_main_add_SignalHandler: Added signal handler for signal 17
cib: 2012/08/21_17:24:25 info: main: Retrieval of a per-action CIB: disabled
cib: 2012/08/21_17:24:25 info: cib_register_ha: Signing in with Heartbeat
cib: 2012/08/21_17:24:25 info: cib_register_ha: FSA Hostname: oradb2-1
cib: 2012/08/21_17:24:25 info: readCibXmlFile: Reading cluster configuration from: /usr/local/var/lib/heartbeat/crm/cib.xml
cib: 2012/08/21_17:24:26 WARN: validate_cib_digest: No on-disk digest present
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <cib admin_epoch="0" epoch="0" num_updates="0" generated="false" have_quorum="false">
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <configuration>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <crm_config>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <cluster_property_set id="cib-bootstrap-options">
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <attributes>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-symmetric-cluster" name="symmetric-cluster" value="true"/>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-no_quorum-policy" name="no_quorum-policy" value="stop"/>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-default-resource-stickiness" name="default-resource-stickiness" value="0"/>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-default-resource-failure-stickiness" name="default-resource-failure-stickiness" value="0"/>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" value="false"/>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stonith-action" name="stonith-action" value="reboot"/>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stop-orphan-resources" name="stop-orphan-resources" value="true"/>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stop-orphan-actions" name="stop-orphan-actions" value="true"/>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-remove-after-stop" name="remove-after-stop" value="false"/>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-short-resource-names" name="short-resource-names" value="true"/>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-transition-idle-timeout" name="transition-idle-timeout" value="5min"/>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-default-action-timeout" name="default-action-timeout" value="5s"/>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-is-managed-default" name="is-managed-default" value="true"/>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: </attributes>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: </cluster_property_set>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: </crm_config>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <nodes/>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <resources>
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <group id="group_1">
cib: 2012/08/21_17:24:26 info: log_data_element: readCibXmlFile: <primitive class="ocf" id="IPaddr_192_1_101_212" provider="heartbeat" type="IPaddr">
crmd: 2012/08/21_17:24:26 WARN: cib_native_signon: Connection to CIB failed: connection failed
mgmtd: 2012/08/21_17:24:26 info: login to cib: 4, ret:-10
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: <operations>
crmd: 2012/08/21_17:24:27 WARN: do_cib_control: Couldn't complete CIB registration 17 times... pause and retry
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: <op id="IPaddr_192_1_101_212_mon" interval="5s" name="monitor" timeout="5s"/>
crmd: 2012/08/21_17:24:27 ERROR: crm_fsa_trigger: FSA took 30120ms to complete
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: </operations>
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: <instance_attributes id="IPaddr_192_1_101_212_inst_attr">
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: <attributes>
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: <nvpair id="IPaddr_192_1_101_212_attr_0" name="ip" value="192.1.101.212"/>
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: </attributes>
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: </instance_attributes>
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: </primitive>
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: <primitive class="lsb" id="myhttpd_2" provider="heartbeat" type="myhttpd">
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: <operations>
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: <op id="myhttpd_2_mon" interval="120s" name="monitor" timeout="60s"/>
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: </operations>
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: </primitive>
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: </group>
crmd: 2012/08/21_17:24:27 info: crm_timer_popped: Wait Timer (I_NULL) just popped!
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: </resources>
crmd: 2012/08/21_17:24:27 WARN: cib_native_signon: Connection to CIB failed: connection failed
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: <constraints>
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: <rsc_location id="rsc_location_group_1" rsc="group_1">
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: <rule id="prefered_location_group_1" score="100">
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: <expression attribute="#uname" id="prefered_location_group_1_expr" operation="eq" value="kf28-1"/>
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: </rule>
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: </rsc_location>
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: </constraints>
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: </configuration>
cib: 2012/08/21_17:24:27 info: log_data_element: readCibXmlFile: <status/>
cib: 2012/08/21_17:24:28 info: log_data_element: readCibXmlFile: </cib>
mgmtd: 2012/08/21_17:24:28 info: login to cib failed
cib: 2012/08/21_17:24:28 notice: readCibXmlFile: Enabling DTD validation on the existing (sane) configuration
mgmtd: 2012/08/21_17:24:28 ERROR: Can't initialize management library.Shutting down.(-1)
heartbeat: 2012/08/21_17:24:28 WARN: Exiting /usr/local/lib/heartbeat/mgmtd -v process 8386 returned rc 1.
heartbeat: 2012/08/21_17:24:28 ERROR: Respawning client "/usr/local/lib/heartbeat/mgmtd -v":
heartbeat: 2012/08/21_17:24:28 info: Starting child client "/usr/local/lib/heartbeat/mgmtd -v" (0,0)
heartbeat: 2012/08/21_17:24:28 info: Starting "/usr/local/lib/heartbeat/mgmtd -v" as uid 0 gid 0 (pid 8963)
cib: 2012/08/21_17:24:28 info: startCib: CIB Initialization completed successfully
mgmtd: 2012/08/21_17:24:28 info: G_main_add_SignalHandler: Added signal handler for signal 15
cib: 2012/08/21_17:24:28 WARN: init_start: CCM Activation failed
mgmtd: 2012/08/21_17:24:28 debug: Enabling coredumps
cib: 2012/08/21_17:24:28 WARN: init_start: CCM Connection failed 1 times (30 max)
mgmtd: 2012/08/21_17:24:28 info: G_main_add_SignalHandler: Added signal handler for signal 10
mgmtd: 2012/08/21_17:24:28 info: G_main_add_SignalHandler: Added signal handler for signal 12
mgmtd: 2012/08/21_17:24:28 info: init_crm
cib: 2012/08/21_17:24:29 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:24:29 WARN: init_start: CCM Connection failed 2 times (30 max)
《解決方案》
下面是一個完整的錯誤日誌:
heartbeat: 2012/08/21_17:34:10 WARN: File /usr/local/etc/ha.d/haresources exists.
heartbeat: 2012/08/21_17:34:10 WARN: This file is not used because crm is enabled
heartbeat: 2012/08/21_17:34:11 WARN: Logging daemon is disabled --enabling logging daemon is recommended
heartbeat: 2012/08/21_17:34:11 info: **************************
heartbeat: 2012/08/21_17:34:11 info: Configuration validated. Starting heartbeat 2.0.8
heartbeat: 2012/08/21_17:34:11 info: heartbeat: version 2.0.8
heartbeat: 2012/08/21_17:34:11 info: Heartbeat generation: 6
heartbeat: 2012/08/21_17:34:11 info: G_main_add_TriggerHandler: Added signal manual handler
heartbeat: 2012/08/21_17:34:11 info: G_main_add_TriggerHandler: Added signal manual handler
heartbeat: 2012/08/21_17:34:11 info: Removing /usr/local/var/run/heartbeat/rsctmp failed, recreating.
heartbeat: 2012/08/21_17:34:11 info: glib: ucast: write socket priority set to IPTOS_LOWDELAY on eth0
heartbeat: 2012/08/21_17:34:11 info: glib: ucast: bound send socket to device: eth0
heartbeat: 2012/08/21_17:34:11 info: glib: ucast: bound receive socket to device: eth0
heartbeat: 2012/08/21_17:34:11 info: glib: ucast: started on port 694 interface eth0 to 192.1.101.128
heartbeat: 2012/08/21_17:34:11 info: G_main_add_SignalHandler: Added signal handler for signal 17
heartbeat: 2012/08/21_17:34:11 info: Local status now set to: 'up'
heartbeat: 2012/08/21_17:34:12 info: Link kf28-1:eth0 up.
heartbeat: 2012/08/21_17:34:12 info: Status update for node kf28-1: status active
heartbeat: 2012/08/21_17:34:13 info: Comm_now_up(): updating status to active
heartbeat: 2012/08/21_17:34:13 info: Local status now set to: 'active'
heartbeat: 2012/08/21_17:34:13 info: Starting child client "/usr/local/lib/heartbeat/ccm" (503,501)
heartbeat: 2012/08/21_17:34:13 info: Starting child client "/usr/local/lib/heartbeat/cib" (503,501)
heartbeat: 2012/08/21_17:34:13 info: Starting child client "/usr/local/lib/heartbeat/lrmd -r" (0,0)
heartbeat: 2012/08/21_17:34:13 info: Starting child client "/usr/local/lib/heartbeat/stonithd" (0,0)
heartbeat: 2012/08/21_17:34:13 info: Starting "/usr/local/lib/heartbeat/cib" as uid 503 gid 501 (pid 13740)
heartbeat: 2012/08/21_17:34:13 info: Starting child client "/usr/local/lib/heartbeat/attrd" (503,501)
heartbeat: 2012/08/21_17:34:13 info: Starting child client "/usr/local/lib/heartbeat/crmd" (503,501)
heartbeat: 2012/08/21_17:34:13 info: Starting "/usr/local/lib/heartbeat/ccm" as uid 503 gid 501 (pid 13739)
heartbeat: 2012/08/21_17:34:13 info: Starting child client "/usr/local/lib/heartbeat/mgmtd -v" (0,0)
heartbeat: 2012/08/21_17:34:13 info: Starting "/usr/local/lib/heartbeat/stonithd" as uid 0 gid 0 (pid 13742)
heartbeat: 2012/08/21_17:34:13 info: Starting "/usr/local/lib/heartbeat/lrmd -r" as uid 0 gid 0 (pid 13741)
heartbeat: 2012/08/21_17:34:13 WARN: G_CH_dispatch_int: Dispatch function for read child took too long to execute: 340 ms (> 50 ms) (GSource: 0x8724728)
heartbeat: 2012/08/21_17:34:13 info: Starting "/usr/local/lib/heartbeat/attrd" as uid 503 gid 501 (pid 13743)
heartbeat: 2012/08/21_17:34:13 info: Starting "/usr/local/lib/heartbeat/crmd" as uid 503 gid 501 (pid 13744)
heartbeat: 2012/08/21_17:34:13 info: Starting "/usr/local/lib/heartbeat/mgmtd -v" as uid 0 gid 0 (pid 13745)
cib: 2012/08/21_17:34:13 info: G_main_add_SignalHandler: Added signal handler for signal 15
cib: 2012/08/21_17:34:13 info: G_main_add_TriggerHandler: Added signal manual handler
cib: 2012/08/21_17:34:13 info: G_main_add_SignalHandler: Added signal handler for signal 17
cib: 2012/08/21_17:34:13 info: main: Retrieval of a per-action CIB: disabled
cib: 2012/08/21_17:34:13 info: cib_register_ha: Signing in with Heartbeat
lrmd: 2012/08/21_17:34:13 info: G_main_add_SignalHandler: Added signal handler for signal 15
cib: 2012/08/21_17:34:13 info: cib_register_ha: FSA Hostname: oradb2-1
cib: 2012/08/21_17:34:13 info: readCibXmlFile: Reading cluster configuration from: /usr/local/var/lib/heartbeat/crm/cib.xml
cib: 2012/08/21_17:34:13 WARN: validate_cib_digest: No on-disk digest present
cib: 2012/08/21_17:34:13 info: log_data_element: readCibXmlFile: <cib admin_epoch="0" epoch="0" num_updates="0" generated="false" have_quorum="false">
cib: 2012/08/21_17:34:13 info: log_data_element: readCibXmlFile: <configuration>
cib: 2012/08/21_17:34:13 info: log_data_element: readCibXmlFile: <crm_config>
cib: 2012/08/21_17:34:13 info: log_data_element: readCibXmlFile: <cluster_property_set id="cib-bootstrap-options">
mgmtd: 2012/08/21_17:34:13 info: G_main_add_SignalHandler: Added signal handler for signal 15
attrd: 2012/08/21_17:34:13 info: G_main_add_SignalHandler: Added signal handler for signal 15
stonithd: 2012/08/21_17:34:13 info: G_main_add_SignalHandler: Added signal handler for signal 10
lrmd: 2012/08/21_17:34:13 info: G_main_add_SignalHandler: Added signal handler for signal 17
crmd: 2012/08/21_17:34:13 info: main: CRM Hg Version: 2d298bca0d0a:STABLE-2.0.8
cib: 2012/08/21_17:34:13 info: log_data_element: readCibXmlFile: <attributes>
mgmtd: 2012/08/21_17:34:13 debug: Enabling coredumps
ccm: 2012/08/21_17:34:13 info: Hostname: oradb2-1
attrd: 2012/08/21_17:34:13 info: register_with_ha: Hostname: oradb2-1
stonithd: 2012/08/21_17:34:13 info: G_main_add_SignalHandler: Added signal handler for signal 12
lrmd: 2012/08/21_17:34:13 info: G_main_add_SignalHandler: Added signal handler for signal 10
crmd: 2012/08/21_17:34:13 info: init_start: Starting crmd
cib: 2012/08/21_17:34:13 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-symmetric-cluster" name="symmetric-cluster" value="true"/>
mgmtd: 2012/08/21_17:34:13 info: G_main_add_SignalHandler: Added signal handler for signal 10
attrd: 2012/08/21_17:34:13 info: register_with_ha: UUID: 34bd366d-3b86-4cb1-9bb1-b901f0e4e08b
stonithd: 2012/08/21_17:34:13 info: Signing in with heartbeat.
lrmd: 2012/08/21_17:34:13 info: G_main_add_SignalHandler: Added signal handler for signal 12
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-no_quorum-policy" name="no_quorum-policy" value="stop"/>
crmd: 2012/08/21_17:34:14 info: G_main_add_SignalHandler: Added signal handler for signal 15
mgmtd: 2012/08/21_17:34:14 info: G_main_add_SignalHandler: Added signal handler for signal 12
attrd: 2012/08/21_17:34:14 WARN: cib_native_signon: Connection to CIB failed: connection failed
stonithd: 2012/08/21_17:34:14 notice: /usr/local/lib/heartbeat/stonithd start up successfully.
lrmd: 2012/08/21_17:34:14 info: Started.
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-default-resource-stickiness" name="default-resource-stickiness" value="0"/>
crmd: 2012/08/21_17:34:14 info: G_main_add_TriggerHandler: Added signal manual handler
ccm: 2012/08/21_17:34:14 info: G_main_add_SignalHandler: Added signal handler for signal 15
stonithd: 2012/08/21_17:34:14 info: G_main_add_SignalHandler: Added signal handler for signal 17
mgmtd: 2012/08/21_17:34:14 info: init_crm
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-default-resource-failure-stickiness" name="default-resource-failure-stickiness" value="0"/>
crmd: 2012/08/21_17:34:14 info: G_main_add_SignalHandler: Added signal handler for signal 17
ccm: 2012/08/21_17:34:14 ERROR: socket_wait_conn_new: unlink failure(/usr/local/var/run/heartbeat/ccm/ccm): Permission denied
mgmtd: 2012/08/21_17:34:14 info: login to cib: 0, ret:-10
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" value="false"/>
crmd: 2012/08/21_17:34:14 WARN: cib_native_signon: Connection to CIB failed: connection failed
ccm: 2012/08/21_17:34:14 ERROR: socket_wait_conn_new: trying to create in /usr/local/var/run/heartbeat/ccm/ccm bind:: Permission denied
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stonith-action" name="stonith-action" value="reboot"/>
ccm: 2012/08/21_17:34:14 ERROR: Can't create wait channel: Permission denied
heartbeat: 2012/08/21_17:34:14 WARN: Exiting /usr/local/lib/heartbeat/ccm process 13739 returned rc 1.
heartbeat: 2012/08/21_17:34:14 ERROR: Respawning client "/usr/local/lib/heartbeat/ccm":
heartbeat: 2012/08/21_17:34:14 info: Starting child client "/usr/local/lib/heartbeat/ccm" (503,501)
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stop-orphan-resources" name="stop-orphan-resources" value="true"/>
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stop-orphan-actions" name="stop-orphan-actions" value="true"/>
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-remove-after-stop" name="remove-after-stop" value="false"/>
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-short-resource-names" name="short-resource-names" value="true"/>
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-transition-idle-timeout" name="transition-idle-timeout" value="5min"/>
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-default-action-timeout" name="default-action-timeout" value="5s"/>
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-is-managed-default" name="is-managed-default" value="true"/>
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: </attributes>
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: </cluster_property_set>
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: </crm_config>
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: <nodes/>
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: <resources>
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: <group id="group_1">
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: <primitive class="ocf" id="IPaddr_192_1_101_212" provider="heartbeat" type="IPaddr">
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: <operations>
cib: 2012/08/21_17:34:14 info: log_data_element: readCibXmlFile: <op id="IPaddr_192_1_101_212_mon" interval="5s" name="monitor" timeout="5s"/>
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: </operations>
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: <instance_attributes id="IPaddr_192_1_101_212_inst_attr">
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: <attributes>
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: <nvpair id="IPaddr_192_1_101_212_attr_0" name="ip" value="192.1.101.212"/>
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: </attributes>
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: </instance_attributes>
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: </primitive>
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: <primitive class="lsb" id="myhttpd_2" provider="heartbeat" type="myhttpd">
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: <operations>
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: <op id="myhttpd_2_mon" interval="120s" name="monitor" timeout="60s"/>
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: </operations>
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: </primitive>
mgmtd: 2012/08/21_17:34:15 info: login to cib: 1, ret:-10
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: </group>
crmd: 2012/08/21_17:34:15 WARN: cib_native_signon: Connection to CIB failed: connection failed
heartbeat: 2012/08/21_17:34:15 info: Starting "/usr/local/lib/heartbeat/ccm" as uid 503 gid 501 (pid 13756)
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: </resources>
crmd: 2012/08/21_17:34:15 WARN: do_cib_control: Couldn't complete CIB registration 1 times... pause and retry
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: <constraints>
crmd: 2012/08/21_17:34:15 info: init_start: Starting crmd's mainloop
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: <rsc_location id="rsc_location_group_1" rsc="group_1">
ccm: 2012/08/21_17:34:15 info: Hostname: oradb2-1
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: <rule id="prefered_location_group_1" score="100">
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: <expression attribute="#uname" id="prefered_location_group_1_expr" operation="eq" value="kf28-1"/>
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: </rule>
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: </rsc_location>
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: </constraints>
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: </configuration>
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: <status/>
cib: 2012/08/21_17:34:15 info: log_data_element: readCibXmlFile: </cib>
cib: 2012/08/21_17:34:15 notice: readCibXmlFile: Enabling DTD validation on the existing (sane) configuration
cib: 2012/08/21_17:34:16 info: startCib: CIB Initialization completed successfully
cib: 2012/08/21_17:34:16 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:16 WARN: init_start: CCM Connection failed 1 times (30 max)
crmd: 2012/08/21_17:34:16 info: crm_timer_popped: Wait Timer (I_NULL) just popped!
ccm: 2012/08/21_17:34:16 info: G_main_add_SignalHandler: Added signal handler for signal 15
ccm: 2012/08/21_17:34:16 ERROR: socket_wait_conn_new: unlink failure(/usr/local/var/run/heartbeat/ccm/ccm): Permission denied
ccm: 2012/08/21_17:34:16 ERROR: socket_wait_conn_new: trying to create in /usr/local/var/run/heartbeat/ccm/ccm bind:: Permission denied
ccm: 2012/08/21_17:34:16 ERROR: Can't create wait channel: Permission denied
heartbeat: 2012/08/21_17:34:16 WARN: Exiting /usr/local/lib/heartbeat/ccm process 13756 returned rc 1.
heartbeat: 2012/08/21_17:34:16 ERROR: Respawning client "/usr/local/lib/heartbeat/ccm":
heartbeat: 2012/08/21_17:34:16 info: Starting child client "/usr/local/lib/heartbeat/ccm" (503,501)
cib: 2012/08/21_17:34:17 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:17 WARN: init_start: CCM Connection failed 2 times (30 max)
heartbeat: 2012/08/21_17:34:17 info: Starting "/usr/local/lib/heartbeat/ccm" as uid 503 gid 501 (pid 13803)
《解決方案》
續上:
cib: 2012/08/21_17:34:18 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:18 WARN: init_start: CCM Connection failed 3 times (30 max)
ccm: 2012/08/21_17:34:18 info: Hostname: oradb2-1
cib: 2012/08/21_17:34:19 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:19 WARN: init_start: CCM Connection failed 4 times (30 max)
cib: 2012/08/21_17:34:20 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:20 WARN: init_start: CCM Connection failed 5 times (30 max)
ccm: 2012/08/21_17:34:20 info: G_main_add_SignalHandler: Added signal handler for signal 15
ccm: 2012/08/21_17:34:20 ERROR: socket_wait_conn_new: unlink failure(/usr/local/var/run/heartbeat/ccm/ccm): Permission denied
ccm: 2012/08/21_17:34:20 ERROR: socket_wait_conn_new: trying to create in /usr/local/var/run/heartbeat/ccm/ccm bind:: Permission denied
ccm: 2012/08/21_17:34:20 ERROR: Can't create wait channel: Permission denied
heartbeat: 2012/08/21_17:34:20 WARN: Exiting /usr/local/lib/heartbeat/ccm process 13803 returned rc 1.
heartbeat: 2012/08/21_17:34:20 ERROR: Respawning client "/usr/local/lib/heartbeat/ccm":
heartbeat: 2012/08/21_17:34:20 info: Starting child client "/usr/local/lib/heartbeat/ccm" (503,501)
cib: 2012/08/21_17:34:21 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:21 WARN: init_start: CCM Connection failed 6 times (30 max)
heartbeat: 2012/08/21_17:34:21 info: Starting "/usr/local/lib/heartbeat/ccm" as uid 503 gid 501 (pid 13821)
ccm: 2012/08/21_17:34:21 info: Hostname: oradb2-1
cib: 2012/08/21_17:34:22 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:22 WARN: init_start: CCM Connection failed 7 times (30 max)
cib: 2012/08/21_17:34:23 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:23 WARN: init_start: CCM Connection failed 8 times (30 max)
cib: 2012/08/21_17:34:24 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:24 WARN: init_start: CCM Connection failed 9 times (30 max)
cib: 2012/08/21_17:34:25 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:25 WARN: init_start: CCM Connection failed 10 times (30 max)
cib: 2012/08/21_17:34:26 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:26 WARN: init_start: CCM Connection failed 11 times (30 max)
ccm: 2012/08/21_17:34:26 info: G_main_add_SignalHandler: Added signal handler for signal 15
ccm: 2012/08/21_17:34:26 ERROR: socket_wait_conn_new: unlink failure(/usr/local/var/run/heartbeat/ccm/ccm): Permission denied
ccm: 2012/08/21_17:34:26 ERROR: socket_wait_conn_new: trying to create in /usr/local/var/run/heartbeat/ccm/ccm bind:: Permission denied
ccm: 2012/08/21_17:34:26 ERROR: Can't create wait channel: Permission denied
heartbeat: 2012/08/21_17:34:26 WARN: Exiting /usr/local/lib/heartbeat/ccm process 13821 returned rc 1.
heartbeat: 2012/08/21_17:34:26 ERROR: Respawning client "/usr/local/lib/heartbeat/ccm":
heartbeat: 2012/08/21_17:34:26 info: Starting child client "/usr/local/lib/heartbeat/ccm" (503,501)
cib: 2012/08/21_17:34:27 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:27 WARN: init_start: CCM Connection failed 12 times (30 max)
heartbeat: 2012/08/21_17:34:27 info: Starting "/usr/local/lib/heartbeat/ccm" as uid 503 gid 501 (pid 13886)
cib: 2012/08/21_17:34:28 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:28 WARN: init_start: CCM Connection failed 13 times (30 max)
ccm: 2012/08/21_17:34:28 info: Hostname: oradb2-1
cib: 2012/08/21_17:34:29 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:29 WARN: init_start: CCM Connection failed 14 times (30 max)
ccm: 2012/08/21_17:34:29 info: G_main_add_SignalHandler: Added signal handler for signal 15
ccm: 2012/08/21_17:34:29 ERROR: socket_wait_conn_new: unlink failure(/usr/local/var/run/heartbeat/ccm/ccm): Permission denied
ccm: 2012/08/21_17:34:29 ERROR: socket_wait_conn_new: trying to create in /usr/local/var/run/heartbeat/ccm/ccm bind:: Permission denied
ccm: 2012/08/21_17:34:29 ERROR: Can't create wait channel: Permission denied
heartbeat: 2012/08/21_17:34:29 WARN: Exiting /usr/local/lib/heartbeat/ccm process 13886 returned rc 1.
heartbeat: 2012/08/21_17:34:29 ERROR: Respawning client "/usr/local/lib/heartbeat/ccm":
heartbeat: 2012/08/21_17:34:29 info: Starting child client "/usr/local/lib/heartbeat/ccm" (503,501)
cib: 2012/08/21_17:34:30 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:30 WARN: init_start: CCM Connection failed 15 times (30 max)
heartbeat: 2012/08/21_17:34:30 info: Starting "/usr/local/lib/heartbeat/ccm" as uid 503 gid 501 (pid 13896)
ccm: 2012/08/21_17:34:30 info: Hostname: oradb2-1
cib: 2012/08/21_17:34:31 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:31 WARN: init_start: CCM Connection failed 16 times (30 max)
cib: 2012/08/21_17:34:32 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:32 WARN: init_start: CCM Connection failed 17 times (30 max)
cib: 2012/08/21_17:34:33 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:33 WARN: init_start: CCM Connection failed 18 times (30 max)
cib: 2012/08/21_17:34:34 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:34 WARN: init_start: CCM Connection failed 19 times (30 max)
cib: 2012/08/21_17:34:35 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:35 WARN: init_start: CCM Connection failed 20 times (30 max)
ccm: 2012/08/21_17:34:35 info: G_main_add_SignalHandler: Added signal handler for signal 15
ccm: 2012/08/21_17:34:35 ERROR: socket_wait_conn_new: unlink failure(/usr/local/var/run/heartbeat/ccm/ccm): Permission denied
ccm: 2012/08/21_17:34:35 ERROR: socket_wait_conn_new: trying to create in /usr/local/var/run/heartbeat/ccm/ccm bind:: Permission denied
ccm: 2012/08/21_17:34:35 ERROR: Can't create wait channel: Permission denied
heartbeat: 2012/08/21_17:34:35 WARN: Exiting /usr/local/lib/heartbeat/ccm process 13896 returned rc 1.
heartbeat: 2012/08/21_17:34:35 ERROR: Respawning client "/usr/local/lib/heartbeat/ccm":
heartbeat: 2012/08/21_17:34:35 info: Starting child client "/usr/local/lib/heartbeat/ccm" (503,501)
cib: 2012/08/21_17:34:36 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:36 WARN: init_start: CCM Connection failed 21 times (30 max)
heartbeat: 2012/08/21_17:34:36 info: Starting "/usr/local/lib/heartbeat/ccm" as uid 503 gid 501 (pid 13959)
ccm: 2012/08/21_17:34:36 info: Hostname: oradb2-1
cib: 2012/08/21_17:34:37 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:37 WARN: init_start: CCM Connection failed 22 times (30 max)
cib: 2012/08/21_17:34:38 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:38 WARN: init_start: CCM Connection failed 23 times (30 max)
ccm: 2012/08/21_17:34:38 info: G_main_add_SignalHandler: Added signal handler for signal 15
ccm: 2012/08/21_17:34:38 ERROR: socket_wait_conn_new: unlink failure(/usr/local/var/run/heartbeat/ccm/ccm): Permission denied
ccm: 2012/08/21_17:34:38 ERROR: socket_wait_conn_new: trying to create in /usr/local/var/run/heartbeat/ccm/ccm bind:: Permission denied
ccm: 2012/08/21_17:34:38 ERROR: Can't create wait channel: Permission denied
heartbeat: 2012/08/21_17:34:38 WARN: Exiting /usr/local/lib/heartbeat/ccm process 13959 returned rc 1.
heartbeat: 2012/08/21_17:34:38 ERROR: Respawning client "/usr/local/lib/heartbeat/ccm":
heartbeat: 2012/08/21_17:34:38 info: Starting child client "/usr/local/lib/heartbeat/ccm" (503,501)
cib: 2012/08/21_17:34:39 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:39 WARN: init_start: CCM Connection failed 24 times (30 max)
heartbeat: 2012/08/21_17:34:39 info: Starting "/usr/local/lib/heartbeat/ccm" as uid 503 gid 501 (pid 13977)
ccm: 2012/08/21_17:34:39 info: Hostname: oradb2-1
cib: 2012/08/21_17:34:40 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:40 WARN: init_start: CCM Connection failed 25 times (30 max)
cib: 2012/08/21_17:34:41 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:41 WARN: init_start: CCM Connection failed 26 times (30 max)
cib: 2012/08/21_17:34:42 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:42 WARN: init_start: CCM Connection failed 27 times (30 max)
cib: 2012/08/21_17:34:43 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:43 WARN: init_start: CCM Connection failed 28 times (30 max)
cib: 2012/08/21_17:34:44 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:44 WARN: init_start: CCM Connection failed 29 times (30 max)
ccm: 2012/08/21_17:34:44 info: G_main_add_SignalHandler: Added signal handler for signal 15
ccm: 2012/08/21_17:34:44 ERROR: socket_wait_conn_new: unlink failure(/usr/local/var/run/heartbeat/ccm/ccm): Permission denied
ccm: 2012/08/21_17:34:44 ERROR: socket_wait_conn_new: trying to create in /usr/local/var/run/heartbeat/ccm/ccm bind:: Permission denied
ccm: 2012/08/21_17:34:44 ERROR: Can't create wait channel: Permission denied
heartbeat: 2012/08/21_17:34:44 WARN: Exiting /usr/local/lib/heartbeat/ccm process 13977 returned rc 1.
heartbeat: 2012/08/21_17:34:44 ERROR: Respawning client "/usr/local/lib/heartbeat/ccm":
heartbeat: 2012/08/21_17:34:44 info: Starting child client "/usr/local/lib/heartbeat/ccm" (503,501)
cib: 2012/08/21_17:34:45 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:45 ERROR: init_start: CCM Activation failed 30 (max) times
cib: 2012/08/21_17:34:45 ERROR: init_start: Couldnt start all communication channels, exiting.
attrd: 2012/08/21_17:34:45 ERROR: cib_native_signon: No reply message - disconnected - 0
heartbeat: 2012/08/21_17:34:45 info: Exiting /usr/local/lib/heartbeat/cib process 13740 returned rc 0.
attrd: 2012/08/21_17:34:45 WARN: cib_native_signon: Connection to CIB failed: not connected
heartbeat: 2012/08/21_17:34:45 ERROR: Respawning client "/usr/local/lib/heartbeat/cib":
heartbeat: 2012/08/21_17:34:45 info: Starting child client "/usr/local/lib/heartbeat/cib" (503,501)
mgmtd: 2012/08/21_17:34:45 ERROR: cib_native_signon: No reply message - disconnected - 0
mgmtd: 2012/08/21_17:34:45 info: login to cib: 2, ret:-3
crmd: 2012/08/21_17:34:45 ERROR: cib_native_signon: No reply message - disconnected - 0
crmd: 2012/08/21_17:34:45 WARN: cib_native_signon: Connection to CIB failed: not connected
heartbeat: 2012/08/21_17:34:45 info: Starting "/usr/local/lib/heartbeat/cib" as uid 503 gid 501 (pid 14008)
cib: 2012/08/21_17:34:45 info: G_main_add_SignalHandler: Added signal handler for signal 15
cib: 2012/08/21_17:34:45 info: G_main_add_TriggerHandler: Added signal manual handler
cib: 2012/08/21_17:34:45 info: G_main_add_SignalHandler: Added signal handler for signal 17
cib: 2012/08/21_17:34:45 info: main: Retrieval of a per-action CIB: disabled
cib: 2012/08/21_17:34:45 info: cib_register_ha: Signing in with Heartbeat
cib: 2012/08/21_17:34:45 info: cib_register_ha: FSA Hostname: oradb2-1
cib: 2012/08/21_17:34:45 info: readCibXmlFile: Reading cluster configuration from: /usr/local/var/lib/heartbeat/crm/cib.xml
cib: 2012/08/21_17:34:45 WARN: validate_cib_digest: No on-disk digest present
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: <cib admin_epoch="0" epoch="0" num_updates="0" generated="false" have_quorum="false">
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: <configuration>
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: <crm_config>
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: <cluster_property_set id="cib-bootstrap-options">
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: <attributes>
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-symmetric-cluster" name="symmetric-cluster" value="true"/>
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-no_quorum-policy" name="no_quorum-policy" value="stop"/>
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-default-resource-stickiness" name="default-resource-stickiness" value="0"/>
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-default-resource-failure-stickiness" name="default-resource-failure-stickiness" value="0"/>
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" value="false"/>
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stonith-action" name="stonith-action" value="reboot"/>
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stop-orphan-resources" name="stop-orphan-resources" value="true"/>
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stop-orphan-actions" name="stop-orphan-actions" value="true"/>
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-remove-after-stop" name="remove-after-stop" value="false"/>
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-short-resource-names" name="short-resource-names" value="true"/>
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-transition-idle-timeout" name="transition-idle-timeout" value="5min"/>
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-default-action-timeout" name="default-action-timeout" value="5s"/>
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-is-managed-default" name="is-managed-default" value="true"/>
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: </attributes>
heartbeat: 2012/08/21_17:34:45 info: Starting "/usr/local/lib/heartbeat/ccm" as uid 503 gid 501 (pid 14007)
cib: 2012/08/21_17:34:45 info: log_data_element: readCibXmlFile: </cluster_property_set>
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: </crm_config>
《解決方案》
再續上:
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: <nodes/>
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: <resources>
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: <group id="group_1">
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: <primitive class="ocf" id="IPaddr_192_1_101_212" provider="heartbeat" type="IPaddr">
mgmtd: 2012/08/21_17:34:46 info: login to cib: 3, ret:-10
crmd: 2012/08/21_17:34:46 WARN: cib_native_signon: Connection to CIB failed: connection failed
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: <operations>
crmd: 2012/08/21_17:34:46 WARN: do_cib_control: Couldn't complete CIB registration 2 times... pause and retry
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: <op id="IPaddr_192_1_101_212_mon" interval="5s" name="monitor" timeout="5s"/>
crmd: 2012/08/21_17:34:46 ERROR: crm_fsa_trigger: FSA took 30160ms to complete
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: </operations>
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: <instance_attributes id="IPaddr_192_1_101_212_inst_attr">
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: <attributes>
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: <nvpair id="IPaddr_192_1_101_212_attr_0" name="ip" value="192.1.101.212"/>
ccm: 2012/08/21_17:34:46 info: Hostname: oradb2-1
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: </attributes>
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: </instance_attributes>
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: </primitive>
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: <primitive class="lsb" id="myhttpd_2" provider="heartbeat" type="myhttpd">
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: <operations>
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: <op id="myhttpd_2_mon" interval="120s" name="monitor" timeout="60s"/>
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: </operations>
crmd: 2012/08/21_17:34:46 info: crm_timer_popped: Wait Timer (I_NULL) just popped!
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: </primitive>
crmd: 2012/08/21_17:34:46 WARN: cib_native_signon: Connection to CIB failed: connection failed
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: </group>
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: </resources>
cib: 2012/08/21_17:34:46 info: log_data_element: readCibXmlFile: <constraints>
cib: 2012/08/21_17:34:47 info: log_data_element: readCibXmlFile: <rsc_location id="rsc_location_group_1" rsc="group_1">
cib: 2012/08/21_17:34:47 info: log_data_element: readCibXmlFile: <rule id="prefered_location_group_1" score="100">
cib: 2012/08/21_17:34:47 info: log_data_element: readCibXmlFile: <expression attribute="#uname" id="prefered_location_group_1_expr" operation="eq" value="kf28-1"/>
cib: 2012/08/21_17:34:47 info: log_data_element: readCibXmlFile: </rule>
cib: 2012/08/21_17:34:47 info: log_data_element: readCibXmlFile: </rsc_location>
cib: 2012/08/21_17:34:47 info: log_data_element: readCibXmlFile: </constraints>
mgmtd: 2012/08/21_17:34:47 info: login to cib: 4, ret:-10
cib: 2012/08/21_17:34:47 info: log_data_element: readCibXmlFile: </configuration>
cib: 2012/08/21_17:34:47 info: log_data_element: readCibXmlFile: <status/>
cib: 2012/08/21_17:34:47 info: log_data_element: readCibXmlFile: </cib>
cib: 2012/08/21_17:34:47 notice: readCibXmlFile: Enabling DTD validation on the existing (sane) configuration
cib: 2012/08/21_17:34:47 info: startCib: CIB Initialization completed successfully
cib: 2012/08/21_17:34:47 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:47 WARN: init_start: CCM Connection failed 1 times (30 max)
ccm: 2012/08/21_17:34:47 info: G_main_add_SignalHandler: Added signal handler for signal 15
ccm: 2012/08/21_17:34:47 ERROR: socket_wait_conn_new: unlink failure(/usr/local/var/run/heartbeat/ccm/ccm): Permission denied
ccm: 2012/08/21_17:34:47 ERROR: socket_wait_conn_new: trying to create in /usr/local/var/run/heartbeat/ccm/ccm bind:: Permission denied
ccm: 2012/08/21_17:34:47 ERROR: Can't create wait channel: Permission denied
heartbeat: 2012/08/21_17:34:47 WARN: Exiting /usr/local/lib/heartbeat/ccm process 14007 returned rc 1.
heartbeat: 2012/08/21_17:34:47 ERROR: Respawning client "/usr/local/lib/heartbeat/ccm":
heartbeat: 2012/08/21_17:34:47 info: Starting child client "/usr/local/lib/heartbeat/ccm" (503,501)
mgmtd: 2012/08/21_17:34:48 info: login to cib failed
mgmtd: 2012/08/21_17:34:48 ERROR: Can't initialize management library.Shutting down.(-1)
heartbeat: 2012/08/21_17:34:48 WARN: Exiting /usr/local/lib/heartbeat/mgmtd -v process 13745 returned rc 1.
heartbeat: 2012/08/21_17:34:48 ERROR: Respawning client "/usr/local/lib/heartbeat/mgmtd -v":
heartbeat: 2012/08/21_17:34:48 info: Starting child client "/usr/local/lib/heartbeat/mgmtd -v" (0,0)
heartbeat: 2012/08/21_17:34:48 info: Starting "/usr/local/lib/heartbeat/mgmtd -v" as uid 0 gid 0 (pid 14063)
mgmtd: 2012/08/21_17:34:48 info: G_main_add_SignalHandler: Added signal handler for signal 15
mgmtd: 2012/08/21_17:34:48 debug: Enabling coredumps
mgmtd: 2012/08/21_17:34:48 info: G_main_add_SignalHandler: Added signal handler for signal 10
mgmtd: 2012/08/21_17:34:48 info: G_main_add_SignalHandler: Added signal handler for signal 12
mgmtd: 2012/08/21_17:34:48 info: init_crm
cib: 2012/08/21_17:34:48 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:48 WARN: init_start: CCM Connection failed 2 times (30 max)
heartbeat: 2012/08/21_17:34:48 info: Starting "/usr/local/lib/heartbeat/ccm" as uid 503 gid 501 (pid 14054)
ccm: 2012/08/21_17:34:48 info: Hostname: oradb2-1
cib: 2012/08/21_17:34:49 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:49 WARN: init_start: CCM Connection failed 3 times (30 max)
cib: 2012/08/21_17:34:50 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:50 WARN: init_start: CCM Connection failed 4 times (30 max)
cib: 2012/08/21_17:34:51 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:51 WARN: init_start: CCM Connection failed 5 times (30 max)
cib: 2012/08/21_17:34:52 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:52 WARN: init_start: CCM Connection failed 6 times (30 max)
cib: 2012/08/21_17:34:53 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:53 WARN: init_start: CCM Connection failed 7 times (30 max)
ccm: 2012/08/21_17:34:53 info: G_main_add_SignalHandler: Added signal handler for signal 15
ccm: 2012/08/21_17:34:53 ERROR: socket_wait_conn_new: unlink failure(/usr/local/var/run/heartbeat/ccm/ccm): Permission denied
ccm: 2012/08/21_17:34:53 ERROR: socket_wait_conn_new: trying to create in /usr/local/var/run/heartbeat/ccm/ccm bind:: Permission denied
ccm: 2012/08/21_17:34:53 ERROR: Can't create wait channel: Permission denied
heartbeat: 2012/08/21_17:34:53 WARN: Exiting /usr/local/lib/heartbeat/ccm process 14054 returned rc 1.
heartbeat: 2012/08/21_17:34:53 ERROR: Respawning client "/usr/local/lib/heartbeat/ccm":
heartbeat: 2012/08/21_17:34:53 info: Starting child client "/usr/local/lib/heartbeat/ccm" (503,501)
cib: 2012/08/21_17:34:54 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:54 WARN: init_start: CCM Connection failed 8 times (30 max)
heartbeat: 2012/08/21_17:34:54 info: Starting "/usr/local/lib/heartbeat/ccm" as uid 503 gid 501 (pid 14084)
ccm: 2012/08/21_17:34:54 info: Hostname: oradb2-1
cib: 2012/08/21_17:34:55 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:55 WARN: init_start: CCM Connection failed 9 times (30 max)
cib: 2012/08/21_17:34:56 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:56 WARN: init_start: CCM Connection failed 10 times (30 max)
cib: 2012/08/21_17:34:57 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:57 WARN: init_start: CCM Connection failed 11 times (30 max)
ccm: 2012/08/21_17:34:58 info: G_main_add_SignalHandler: Added signal handler for signal 15
ccm: 2012/08/21_17:34:58 ERROR: socket_wait_conn_new: unlink failure(/usr/local/var/run/heartbeat/ccm/ccm): Permission denied
ccm: 2012/08/21_17:34:58 ERROR: socket_wait_conn_new: trying to create in /usr/local/var/run/heartbeat/ccm/ccm bind:: Permission denied
ccm: 2012/08/21_17:34:58 ERROR: Can't create wait channel: Permission denied
heartbeat: 2012/08/21_17:34:58 WARN: Exiting /usr/local/lib/heartbeat/ccm process 14084 returned rc 1.
heartbeat: 2012/08/21_17:34:58 ERROR: Client /usr/local/lib/heartbeat/ccm "respawning too fast"
cib: 2012/08/21_17:34:58 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:58 WARN: init_start: CCM Connection failed 12 times (30 max)
cib: 2012/08/21_17:34:59 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:34:59 WARN: init_start: CCM Connection failed 13 times (30 max)
cib: 2012/08/21_17:35:00 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:00 WARN: init_start: CCM Connection failed 14 times (30 max)
cib: 2012/08/21_17:35:01 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:01 WARN: init_start: CCM Connection failed 15 times (30 max)
cib: 2012/08/21_17:35:02 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:02 WARN: init_start: CCM Connection failed 16 times (30 max)
cib: 2012/08/21_17:35:03 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:03 WARN: init_start: CCM Connection failed 17 times (30 max)
cib: 2012/08/21_17:35:04 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:04 WARN: init_start: CCM Connection failed 18 times (30 max)
cib: 2012/08/21_17:35:05 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:05 WARN: init_start: CCM Connection failed 19 times (30 max)
cib: 2012/08/21_17:35:06 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:06 WARN: init_start: CCM Connection failed 20 times (30 max)
cib: 2012/08/21_17:35:07 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:07 WARN: init_start: CCM Connection failed 21 times (30 max)
cib: 2012/08/21_17:35:08 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:08 WARN: init_start: CCM Connection failed 22 times (30 max)
cib: 2012/08/21_17:35:09 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:09 WARN: init_start: CCM Connection failed 23 times (30 max)
cib: 2012/08/21_17:35:10 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:10 WARN: init_start: CCM Connection failed 24 times (30 max)
cib: 2012/08/21_17:35:11 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:11 WARN: init_start: CCM Connection failed 25 times (30 max)
cib: 2012/08/21_17:35:12 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:12 WARN: init_start: CCM Connection failed 26 times (30 max)
cib: 2012/08/21_17:35:13 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:13 WARN: init_start: CCM Connection failed 27 times (30 max)
cib: 2012/08/21_17:35:14 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:14 WARN: init_start: CCM Connection failed 28 times (30 max)
cib: 2012/08/21_17:35:15 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:15 WARN: init_start: CCM Connection failed 29 times (30 max)
cib: 2012/08/21_17:35:16 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:16 ERROR: init_start: CCM Activation failed 30 (max) times
cib: 2012/08/21_17:35:16 ERROR: init_start: Couldnt start all communication channels, exiting.
mgmtd: 2012/08/21_17:35:16 ERROR: cib_native_signon: No reply message - disconnected - 0
crmd: 2012/08/21_17:35:16 ERROR: cib_native_signon: No reply message - disconnected - 0
attrd: 2012/08/21_17:35:16 ERROR: cib_native_signon: No reply message - disconnected - 0
heartbeat: 2012/08/21_17:35:16 info: Exiting /usr/local/lib/heartbeat/cib process 14008 returned rc 0.
crmd: 2012/08/21_17:35:16 WARN: cib_native_signon: Connection to CIB failed: not connected
mgmtd: 2012/08/21_17:35:16 info: login to cib: 0, ret:-3
heartbeat: 2012/08/21_17:35:16 ERROR: Respawning client "/usr/local/lib/heartbeat/cib":
attrd: 2012/08/21_17:35:16 WARN: cib_native_signon: Connection to CIB failed: not connected
crmd: 2012/08/21_17:35:16 WARN: do_cib_control: Couldn't complete CIB registration 3 times... pause and retry
heartbeat: 2012/08/21_17:35:16 info: Starting child client "/usr/local/lib/heartbeat/cib" (503,501)
crmd: 2012/08/21_17:35:16 WARN: crm_fsa_trigger: FSA took 29660ms to complete
heartbeat: 2012/08/21_17:35:16 info: Starting "/usr/local/lib/heartbeat/cib" as uid 503 gid 501 (pid 14291)
cib: 2012/08/21_17:35:16 info: G_main_add_SignalHandler: Added signal handler for signal 15
cib: 2012/08/21_17:35:16 info: G_main_add_TriggerHandler: Added signal manual handler
《解決方案》
再續上:
cib: 2012/08/21_17:35:16 info: G_main_add_SignalHandler: Added signal handler for signal 17
cib: 2012/08/21_17:35:16 info: main: Retrieval of a per-action CIB: disabled
cib: 2012/08/21_17:35:16 info: cib_register_ha: Signing in with Heartbeat
cib: 2012/08/21_17:35:16 info: cib_register_ha: FSA Hostname: oradb2-1
cib: 2012/08/21_17:35:16 info: readCibXmlFile: Reading cluster configuration from: /usr/local/var/lib/heartbeat/crm/cib.xml
cib: 2012/08/21_17:35:16 WARN: validate_cib_digest: No on-disk digest present
cib: 2012/08/21_17:35:16 info: log_data_element: readCibXmlFile: <cib admin_epoch="0" epoch="0" num_updates="0" generated="false" have_quorum="false">
cib: 2012/08/21_17:35:16 info: log_data_element: readCibXmlFile: <configuration>
cib: 2012/08/21_17:35:16 info: log_data_element: readCibXmlFile: <crm_config>
cib: 2012/08/21_17:35:16 info: log_data_element: readCibXmlFile: <cluster_property_set id="cib-bootstrap-options">
cib: 2012/08/21_17:35:16 info: log_data_element: readCibXmlFile: <attributes>
cib: 2012/08/21_17:35:16 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-symmetric-cluster" name="symmetric-cluster" value="true"/>
cib: 2012/08/21_17:35:16 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-no_quorum-policy" name="no_quorum-policy" value="stop"/>
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-default-resource-stickiness" name="default-resource-stickiness" value="0"/>
crmd: 2012/08/21_17:35:17 info: crm_timer_popped: Wait Timer (I_NULL) just popped!
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-default-resource-failure-stickiness" name="default-resource-failure-stickiness" value="0"/>
crmd: 2012/08/21_17:35:17 WARN: cib_native_signon: Connection to CIB failed: connection failed
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" value="false"/>
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stonith-action" name="stonith-action" value="reboot"/>
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stop-orphan-resources" name="stop-orphan-resources" value="true"/>
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-stop-orphan-actions" name="stop-orphan-actions" value="true"/>
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-remove-after-stop" name="remove-after-stop" value="false"/>
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-short-resource-names" name="short-resource-names" value="true"/>
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-transition-idle-timeout" name="transition-idle-timeout" value="5min"/>
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-default-action-timeout" name="default-action-timeout" value="5s"/>
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <nvpair id="cib-bootstrap-options-is-managed-default" name="is-managed-default" value="true"/>
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: </attributes>
mgmtd: 2012/08/21_17:35:17 info: login to cib: 1, ret:-10
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: </cluster_property_set>
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: </crm_config>
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <nodes/>
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <resources>
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <group id="group_1">
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <primitive class="ocf" id="IPaddr_192_1_101_212" provider="heartbeat" type="IPaddr">
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <operations>
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <op id="IPaddr_192_1_101_212_mon" interval="5s" name="monitor" timeout="5s"/>
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: </operations>
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <instance_attributes id="IPaddr_192_1_101_212_inst_attr">
cib: 2012/08/21_17:35:17 info: log_data_element: readCibXmlFile: <attributes>
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: <nvpair id="IPaddr_192_1_101_212_attr_0" name="ip" value="192.1.101.212"/>
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: </attributes>
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: </instance_attributes>
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: </primitive>
crmd: 2012/08/21_17:35:18 WARN: cib_native_signon: Connection to CIB failed: connection failed
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: <primitive class="lsb" id="myhttpd_2" provider="heartbeat" type="myhttpd">
crmd: 2012/08/21_17:35:18 WARN: do_cib_control: Couldn't complete CIB registration 4 times... pause and retry
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: <operations>
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: <op id="myhttpd_2_mon" interval="120s" name="monitor" timeout="60s"/>
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: </operations>
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: </primitive>
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: </group>
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: </resources>
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: <constraints>
mgmtd: 2012/08/21_17:35:18 info: login to cib: 2, ret:-10
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: <rsc_location id="rsc_location_group_1" rsc="group_1">
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: <rule id="prefered_location_group_1" score="100">
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: <expression attribute="#uname" id="prefered_location_group_1_expr" operation="eq" value="kf28-1"/>
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: </rule>
crmd: 2012/08/21_17:35:18 info: crm_timer_popped: Wait Timer (I_NULL) just popped!
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: </rsc_location>
crmd: 2012/08/21_17:35:18 WARN: cib_native_signon: Connection to CIB failed: connection failed
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: </constraints>
cib: 2012/08/21_17:35:18 info: log_data_element: readCibXmlFile: </configuration>
cib: 2012/08/21_17:35:19 info: log_data_element: readCibXmlFile: <status/>
cib: 2012/08/21_17:35:19 info: log_data_element: readCibXmlFile: </cib>
cib: 2012/08/21_17:35:19 notice: readCibXmlFile: Enabling DTD validation on the existing (sane) configuration
cib: 2012/08/21_17:35:19 info: startCib: CIB Initialization completed successfully
cib: 2012/08/21_17:35:19 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:19 WARN: init_start: CCM Connection failed 1 times (30 max)
cib: 2012/08/21_17:35:20 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:20 WARN: init_start: CCM Connection failed 2 times (30 max)
cib: 2012/08/21_17:35:21 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:21 WARN: init_start: CCM Connection failed 3 times (30 max)
cib: 2012/08/21_17:35:22 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:22 WARN: init_start: CCM Connection failed 4 times (30 max)
cib: 2012/08/21_17:35:23 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:23 WARN: init_start: CCM Connection failed 5 times (30 max)
cib: 2012/08/21_17:35:24 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:24 WARN: init_start: CCM Connection failed 6 times (30 max)
cib: 2012/08/21_17:35:25 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:25 WARN: init_start: CCM Connection failed 7 times (30 max)
cib: 2012/08/21_17:35:26 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:26 WARN: init_start: CCM Connection failed 8 times (30 max)
cib: 2012/08/21_17:35:27 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:27 WARN: init_start: CCM Connection failed 9 times (30 max)
cib: 2012/08/21_17:35:28 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:28 WARN: init_start: CCM Connection failed 10 times (30 max)
cib: 2012/08/21_17:35:29 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:29 WARN: init_start: CCM Connection failed 11 times (30 max)
cib: 2012/08/21_17:35:30 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:30 WARN: init_start: CCM Connection failed 12 times (30 max)
cib: 2012/08/21_17:35:31 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:31 WARN: init_start: CCM Connection failed 13 times (30 max)
cib: 2012/08/21_17:35:32 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:32 WARN: init_start: CCM Connection failed 14 times (30 max)
cib: 2012/08/21_17:35:33 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:33 WARN: init_start: CCM Connection failed 15 times (30 max)
cib: 2012/08/21_17:35:34 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:34 WARN: init_start: CCM Connection failed 16 times (30 max)
cib: 2012/08/21_17:35:35 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:35 WARN: init_start: CCM Connection failed 17 times (30 max)
cib: 2012/08/21_17:35:36 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:36 WARN: init_start: CCM Connection failed 18 times (30 max)
cib: 2012/08/21_17:35:37 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:37 WARN: init_start: CCM Connection failed 19 times (30 max)
cib: 2012/08/21_17:35:38 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:38 WARN: init_start: CCM Connection failed 20 times (30 max)
cib: 2012/08/21_17:35:39 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:39 WARN: init_start: CCM Connection failed 21 times (30 max)
cib: 2012/08/21_17:35:40 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:40 WARN: init_start: CCM Connection failed 22 times (30 max)
cib: 2012/08/21_17:35:41 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:41 WARN: init_start: CCM Connection failed 23 times (30 max)
cib: 2012/08/21_17:35:42 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:42 WARN: init_start: CCM Connection failed 24 times (30 max)
cib: 2012/08/21_17:35:43 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:43 WARN: init_start: CCM Connection failed 25 times (30 max)
cib: 2012/08/21_17:35:44 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:44 WARN: init_start: CCM Connection failed 26 times (30 max)
cib: 2012/08/21_17:35:45 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:45 WARN: init_start: CCM Connection failed 27 times (30 max)
cib: 2012/08/21_17:35:46 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:46 WARN: init_start: CCM Connection failed 28 times (30 max)
cib: 2012/08/21_17:35:47 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:47 WARN: init_start: CCM Connection failed 29 times (30 max)
cib: 2012/08/21_17:35:48 WARN: init_start: CCM Activation failed
cib: 2012/08/21_17:35:48 ERROR: init_start: CCM Activation failed 30 (max) times
cib: 2012/08/21_17:35:48 ERROR: init_start: Couldnt start all communication channels, exiting.
crmd: 2012/08/21_17:35:48 ERROR: cib_native_signon: No reply message - disconnected - 0
attrd: 2012/08/21_17:35:48 ERROR: cib_native_signon: No reply message - disconnected - 0
heartbeat: 2012/08/21_17:35:48 info: Exiting /usr/local/lib/heartbeat/cib process 14291 returned rc 0.
crmd: 2012/08/21_17:35:48 WARN: cib_native_signon: Connection to CIB failed: not connected
attrd: 2012/08/21_17:35:48 WARN: cib_native_signon: Connection to CIB failed: not connected
heartbeat: 2012/08/21_17:35:48 ERROR: Respawning client "/usr/local/lib/heartbeat/cib":
crmd: 2012/08/21_17:35:48 WARN: do_cib_control: Couldn't complete CIB registration 5 times... pause and retry
heartbeat: 2012/08/21_17:35:48 info: Starting child client "/usr/local/lib/heartbeat/cib" (503,501)
crmd: 2012/08/21_17:35:48 WARN: crm_fsa_trigger: FSA took 29370ms to complete
mgmtd: 2012/08/21_17:35:48 ERROR: cib_native_signon: No reply message - disconnected - 0
mgmtd: 2012/08/21_17:35:48 info: login to cib: 3, ret:-3
heartbeat: 2012/08/21_17:35:48 info: Starting "/usr/local/lib/heartbeat/cib" as uid 503 gid 501 (pid 14549)
cib: 2012/08/21_17:35:48 info: G_main_add_SignalHandler: Added signal handler for signal 15
cib: 2012/08/21_17:35:48 info: G_main_add_TriggerHandler: Added signal manual handler
cib: 2012/08/21_17:35:48 info: G_main_add_SignalHandler: Added signal handler for signal 17
cib: 2012/08/21_17:35:48 info: main: Retrieval of a per-action CIB: disabled
cib: 2012/08/21_17:35:48 info: cib_register_ha: Signing in with Heartbeat
cib: 2012/08/21_17:35:48 info: cib_register_ha: FSA Hostname: oradb2-1
cib: 2012/08/21_17:35:48 info: readCibXmlFile: Reading cluster configuration from: /usr/local/var/lib/heartbeat/crm/cib.xml
cib: 2012/08/21_17:35:48 WARN: validate_cib_digest: No on-disk digest present
cib: 2012/08/21_17:35:48 info: log_data_element: readCibXmlFile: <cib admin_epoch="0" epoch="0" num_updates="0" generated="false" have_quorum="false">
cib: 2012/08/21_17:35:48 info: log_data_element: readCibXmlFile: <configuration>