cman與openais

火星人 @ 2014-03-04 , reply:0


cman與openais

我在網上搜到有些文章在配置RHCS時,在openaic.conf里指定bindnetaddr參數為接私網的網卡IP網段。

但是在http://sources.redhat.com/cluster/wiki/HomePage?action=AttachFile&do=get&target=aiscman.pdf裡面看到的確實cman根本就不會使用openais.conf,它是從CCS內置的openais數據結構讀取參數值,並映射進openais.conf文件,因此,你可以在CCS里的<totem>部份添加條目並使用這個標誌,查閱openais.conf man手冊來獲得詳細的配置方法。
所以,有些人喜歡這樣子來設置心跳,適合一些人來熟悉新的想法和新的CCS特性。

原文如下:
openais.conf
cman does not use the openais.conf file at all. What it does do is load the values from CCS into
internal OpenAIS data structures that map onto the entries to openais.conf. So that's why you
can add an entry in a <totem> section in CCS and it will be used by totem. Refer to the
openais.conf(5) man page for the openaisspecific
configuration keys.
So, those people who like to tweak heartbeat settings and suchlike, have some new concepts and
new CCS keys to learn!

我e文差勁,沒弄懂這個openais.conf到底對設置心跳網路有沒有幫助?

歡迎跟帖討論
《解決方案》

紅帽的官方的文檔上是有的,如果沒有特殊的需要未必一定要使用rhcsChapter 6. Clustering and federation


Messaging ClustersA Messaging Cluster is a group of brokers that act as a single broker. Changes on any broker are replicated to all other brokers in the same Messaging Cluster, so if one broker fails, its clients can fail-over to another broker without loss of state. The brokers in a Messaging Cluster may run on the same host or on different hosts. Two brokers are in the same cluster if [*]They use the same OpenAIS mcastaddr, mcastport, and bindnetaddr, and
[*]They use the same cluster name.


Messaging Clusters are implemented using using OpenAIS, which provides a reliable multicast protocol, tools, and infrastructure for implementing replicated services. You must install and configure OpenAIS to use MRG broker groups. Once you have installed OpenAIS, configure MRG Messaging to run in a cluster as follows.
[*]Set the binding address for openais in /etc/ais/openais.conf. Use ifconfig to find the inet addr and the netmask for the interface you want:
# ifconfigeth0  Link encap:Ethernet  HWaddr 00:E0:81:76:B6:C6        inet addr:10.16.44.222  Bcast:10.16.47.255  Mask:255.255.248.0      inet6 addr: fe80::2e0:81ff:fe76:b6c6/64 Scope:Link      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1      RX packets:35914541 errors:6 dropped:0 overruns:0 frame:6      TX packets:6529841 errors:0 dropped:0 overruns:0 carrier:0      collisions:0 txqueuelen:1000       RX bytes:20294124383 (18.9 GiB)  TX bytes:12925473031 (12.0 GiB)      Interrupt:98 Base address:0x8000The binding address in /etc/ais/openais.conf should be the network address for the interface, which you can find by doing a bitwise AND of the inet addr (in this case, 10.16.44.222) and the network mask (in this case, 255.255.248.0). The result is 10.16.40.0. As a sanity check, you can use route and make sure the address you computed is associated with the interface:
$ /sbin/routeKernel IP routing tableDestination   Gateway       Genmask        Flags Metric Ref Use Iface20.0.10.0     *             255.255.255.0  U     0      0     0 eth1192.168.122.0 *             255.255.255.0  U     0      0     0 virbr010.16.40.0    *             255.255.248.0  U     0      0     0 eth0169.254.0.0   *             255.255.0.0    U     0      0     0 eth1default       10.16.47.254  0.0.0.0        UG    0      0     0 eth0To use eth0 as the interface for the cluster, find the setting for bindnetaddr in /etc/ais/openais.conf, and set it to 10.16.40.0:
bindnetaddr: 10.16.40.0[*]Make sure that the primary group for the user running qpidd is 「ais」. For instance, if you are running qpidd as a daemon, the user is named qpidd. You can make ais the primary group for qpidd as follows:
# usermod -g ais qpidd[*]Set the name of the cluster in qpidd.conf.
cluster_name="Mick"[*]Use qpid-tool to see the cluster.
# qpid-tool localhost:5672The cluster is one of the objects shown by the list command.
qpid: listManagement Object Types:ObjectType                         Active  Deleted====================================================com.redhat.rhm.store:journal       1       0com.redhat.rhm.store:store         1       0org.apache.qpid.broker:binding     5       0org.apache.qpid.broker:broker      1       0org.apache.qpid.broker:connection  1       0org.apache.qpid.broker:exchange    7       0org.apache.qpid.broker:queue       2       0org.apache.qpid.broker:session     1       0org.apache.qpid.broker:system      1       0org.apache.qpid.broker:vhost       1       0org.apache.qpid.cluster:cluster    1       0To see the properties of the cluster, use show cluster:
qpid: show clusterObject of type org.apache.qpid.cluster:cluster: (last sample time: 13:56:40)Type      Element       112==================================================================================================property  brokerRef     102property  clusterName   fooproperty  clusterID     da821ff9-2a88-4002-b976-f18680556290property  publishedURL  amqp:tcp:10.16.44.222:52265,tcp:20.0.10.15:52265,tcp:192.168.122.1:52265property  clusterSize   1property  status        ACTIVEproperty  members       amqp:tcp:10.16.44.222:52265,tcp:20.0.10.15:52265,tcp:192.168.122.1:52265
Messaging Clusters can be used together with Red Hat Clustering Services (RHCS) by starting brokers with the --cluster-cman option.
FederationFederation is used to provide geographical distribution of brokers. A number of individual brokers, or clusters of brokers, can be federated together. This allows client machines to see and interact with the federation as though it were a single broker. Federation can also be used where client machines need to remain on a local network, even though their messages have to be routed out.
http://wideopen.com/docs/en-US/Red_Hat_Enterprise_MRG/1.1/html/Messaging_Installation_Guide/images/Federation.png
Federation is used primarily for connecting disparate locations across a wide area network. Full connectivity across an enterprise can be achieved while keeping local message traffic isolated to a single location. Departmental brokers can be specified with individual policies that control inter-departmental message traffic flow.
Some applications can benefit from having a broker co-resident with the client. This is good for situations where the client produces data that must be delivered reliably but connectivity can not be guaranteed. In this case, a co-resident broker provides queueing and durability that is not available in the client on its own.
Federation bridges disjointed IP networks. Message brokers can be configured to allow message connectivity between networks where there is no IP connectivity. For example, an isolated, private IP network can have messaging connectivity to brokers in other outside IP networks.
Links and routesFederation is configured through a series of links and routes.
A link is a connection between two brokers that allows messages to be passed between them. A link is a transport level connection (using a protocol such as TCP, RDMA, or SSL) that is initiated by a broker and accepted by another broker. The broker that initiates the link is considered the client in the connection. The broker that receives that connection will not treat it any differently from any other client connection, other than annotating it as being for federation.
Routes are the paths that messages take from one broker to another, and can run along one or more links to the final destination. A route is associated with an AMQP session established over the link connection. A route controls the flow of messages across the link between brokers, and multiple routes can share the same link. Messages will flow over a single route in only one direction. For bi-directional connectivity a pair of routes must be created, one for each direction of message flow. Routes always consist of a session and a subscription for consuming messages. Depending on the configuration, a route can have a private queue on the source broker with a binding to an exchange on that broker.
《解決方案》

個人感覺RHCS就是一個大雜燴,這裡一點那裡一點,也在詫異心跳地址在那裡配置呢。

[ 本帖最後由 rainbow 於 2009-3-6 16:36 編輯 ]
《解決方案》

回復 #3 rainbow 的帖子

我在搜仲裁盤的時候看到如下描述:

RHEL2.1使用共享磁碟分區來協調集群狀態,IP網路作為備份
RHEL3使用IP網路作為主要協調集群狀態而共享磁碟分區作為備份
RHEL4和5結合了GFS(全局文件系統)使用IP網路作為協調機制,不再需要Quorum分區

個人對仲裁盤和心跳的理解:
1、rhel5根據現場環境,可以使用仲裁盤來作為備份手段防止腦裂現象發生。
2、心跳線是接在交換機或HUB上,採用默認的Multicast多播地址以廣播的形式在集群節點間進行通訊,一旦節點探測失效,探測次數達到臨界點,將被健康節點發送fence命令強行重啟,釋放資源,保證資源的完整性。
3、cluster.conf里的節點名稱,必須是運行hostname命令所得到的主機名,那麼ccs_tool來做群集節點間的通訊和信息交換是否是依賴於hosts文件里所設置的主機名和對應的IP呢?如果是,那麼心跳信息不是全部從public網路走了嘛?集群管理手冊里硬體部份所需要的private網路僅僅是為了接Power switch?

發帖子的目的就是想弄明白,RHCS5的心跳到底從哪個網路走,我的客戶2個網卡做bonding提供publice networking ,另2個網卡做bonding提供private networking,fence設備是用HP的ilo設備,集群也做成功了,就是不太明白這個private networking是否是必須的,quorum 機制在2節點的集群用的人多不多?
《解決方案》

原帖由 hbwangting 於 2009-3-6 11:20 發表 http://linux.chinaunix.net/bbs/images/common/back.gif
我在網上搜到有些文章在配置RHCS時,在openaic.conf里指定bindnetaddr參數為接私網的網卡IP網段。

但是在http://sources.redhat.com/cluster/wiki/HomePage?action=AttachFile&do=get&target=aiscman.pdf裡面 ...

對於這個問題,我就一句話:你看不到心跳,但不等於沒有心跳,所以你就放心吧。
《解決方案》

1.對;2.對;
3.cluster_node_name和hostname可以不一樣。private網路可以不要,心跳和服務可以跑一條線路上。
兩個節點用quorum disk沒有太大意義。
《解決方案》

回復 #6 jerrywjl 的帖子

在A description of networking in Red Hat Cluster (http://sources.redhat.com/cluster/wiki/HomePage?action=AttachFile&do=get&target=CSNetworking.pdf)一文中提到:
cman tries hard to match the local host name(s) to those mentioned in cluster.conf. Here's how it does
it:
1. It looks up $HOSTNAME in cluster.conf
2. If this fails it strips the domain name from $HOSTNAME and looks up that in cluster.conf
3. If this fails it looks in cluster.conf for a fully-qualified name whose short version matches the
short version of $HOSTNAME
4. If all this fails then it will search the interfaces list for an (ipv4 only) address that matches a
name in cluster.conf
cman will then bind to the address that it has matched.

翻譯過來的話,cman_tool確實是依靠$HOSTNAME命令得出的結果去尋找集群節點與之通訊,可以是帶域名後綴的全稱,或者去掉域名後綴
《解決方案》

那你試試如果將/etc/cluster/cluster.conf中的cluster_node_name指定成和你當前系統名稱不一樣的名字情況會怎樣?
比如你的系統名稱是host1和host2,而cluster_node_name是node1和node2,然後在/etc/hosts中指定node1和node2。
《解決方案》

回復 #8 jerrywjl 的帖子

hostname命令得出的主機名,和/etc/hosts文件里的主機名肯定是一樣的啦

暈死。。。
《解決方案》

RHCS搞的太複雜了。我懷疑是因為這個架構既要支持HA又要支持GFS導致結構複雜。



[火星人 via ] cman與openais已經有141次圍觀

http://www.coctec.com/docs/service/show-post-6055.html