Contents
As you can see below, the CIB[1] is broken primarily into 2 parts configuration and status. You should never need to touch the status portion, however you can read more about it here: ClusterInformationBase/UnderstandingStatus[2].
<cib>
<configuration>
<crm_config/>
<nodes/>
<resources/>
<constraints/>
</configuration>
<status/>
</cib>
Example crm_config section:
<crm_config>
<nvpair id="require_quorum" name="require_quorum" value="true"/>
<nvpair id="symmetric_cluster" name="symetric_cluster" value="true"/>
<nvpair id="suppress_cib_writes" name="suppress_cib_writes" value="true"/>
</crm_config>
The nodes section can be mostly ignored as it will be filled in automatically by the system. It is usually best to allow this to happen automatically as the most important value is the UUID of the node which can be easy to get wrong. Annotated DTD[4]
<node id="de937e3d-0309-4b5d-b85c-f96edc1ed8e3" uname="c001n01" type="member"/>
Native resources are the simplest type of resource in the cluster. They correspond to exactly one IP address, or one webserver, (or whatever). They can be used in the cluster as-is, but can also be aggregated to create other, more powerful, resource sets. Annotated DTD[5]
OCF[6] resource agents[7] take name=value parameters in their environment. Each RA contains a list of supported parameters and their defaults. In accordance with the OCF spec, they are passed to the RA as with a prefix of OCF_RESKEY_. So ip=192.168.9.1 becomes OCF_RESKEY_ip=192.168.9.1
<primitive id="WebServerIP" class="ocf" type="IPaddr" provider="heartbeat">
<instance_attributes>
<attributes>
<nvpair name="ip" value="127.0.0.26"/>
</attributes>
</instance_attributes>
</primitive>
This is the same resource using a Heartbeat RA. Note the name of the parameter. Since Heartbeat RAs only support positional parameters, the name value must be an integer specify its relative order (lower numbers first).
<primitive id="WebServerIP" class="heartbeat" type="IPaddr" provider="heartbeat">
<instance_attributes>
<attributes>
<nvpair name="1" value="127.0.0.26"/>
</attributes>
</instance_attributes>
</primitive>
LSB[8] scripts on the other hand take no arguments. However if you want to pass environment variables to them, you can use the same mechanism as for OCF scripts.
<primitive id="Named" class="lsb" type="named" provider="heartbeat"/>
Unlike a two node cluster, where a resource is either on one node or the other, larger clusters need a better way to specify where a resource should be running.
We do this using constraints. Annotated DTD[9]
There are 3 constraint categories:
In one way or another, all constraints have a score. A score adjusts the resource's preference for running on any node matching the condition. Resources are never placed on nodes that end up with a negative score.
Two special values of score exist: INFINITY and -INFINITY. Processing of these special values is as follows:
INFINITY +/- -INFINITY : ERROR INFINITY +/- int : INFINITY -INFINITY +/- int : -INFINITY
Some constraints are implicit when the advanced resource types are used. For example in a group resource, it implied that they must all run on the same node, and that they will be started in order of listing (and stopped in the reverse order).
The simplest type of constraint to configure is a colocational one. Annotated DTD[10] One needs only to provide the id of two resources and a score to indicate if they cannot or if they must run on the same node.
The score for a co-locational constraint can currently only be INFINITY or -INFINITY, this may change in the future.
Use INFINITY to indicate the resource must run on the same node, and -INFINITY to indicate they should NEVER run on the same node.
Run the Apache instance for the corporate webserver on the same machine as the IP address:
<rsc_colocation id="web_same" from="WebServerIP"
to="WebServerApache" score="INFINITY"/>
DON'T run the test environment on the same machine as the production server
<rsc_colocation id="production_testing_not_same" from="WebServerIP"
to="WebServerIP_testing" score="-INFINITY"/>
Locational constraints[11] use rules[12] and expressions[13] to match one or more cluster nodes and apply a score.
It is very important to point out how the symmetric_cluster[3] option affects the locational constraints you will need.
If symmetric_cluster is TRUE, then resources can run anywhere by default and you will need to either:
If symmetric_cluster is FALSE, then resources cannot run anywhere by default and you will need to either:
The examples below work regardless of the value for symmetric_cluster.
More on rules and constraints can be found on the page ClusterInformationBase/AdvancedRules[14]
If possible, run WebServerIP on the node with uname="c001n01"
<rsc_location id="run_WebServerIP" rsc="WebServerIP">
<rule id="pref_run_WebServerIP" score="100">
<expression attribute="#uname" operation="eq" value="c001n01"/>
</rule>
</rsc_location>
DON'T ever run WebServerIP on the node with id="de937e3d-0309-4b5d-b85c-f96edc1ed8e3"
<rsc_location id="dont_run_WebServerIP" rsc="WebServerIP">
<rule id="pref_dont_run_rsc_c001n01" score="-INFINITY">
<expression attribute="#id" operation="eq"
value="de937e3d-0309-4b5d-b85c-f96edc1ed8e3"/>
</rule>
</rsc_location>
We could have equally used <expression attribute="#uname" operation="eq" value="c001n01"/> to achieve the same result.
To specify that WebServerIP can only ever run on the node with uname="c001n01" and nowhere else, one would use the following:
<rsc_location id="run_WebServerIP" rsc="WebServerIP">
<rule id="pref_run_WebServerIP" score="INFINITY">
<expression attribute="#uname" operation="eq" value="c001n01"/>
</rule>
<rule id="pref_run_WebServerIP" score="-INFINITY">
<expression attribute="#uname" operation="ne" value="c001n01"/>
</rule>
</rsc_location>
The following is the CIB generated by the ClusterTestSuite[15] for an 8 node cluster.
General configuration properties:
There is a resource called DcIPaddr which can only run on the master CRMd node (otherwise known as the DesignatedCoordinator[16]).
For every node in the cluster it contains a resource named "rsc_" + node_name. Additionally there are constraints named "run_" + rsc_name which specify that if node_node is a part of the cluster, then "rsc_" + node_name should run there.
Also, there are constraints that specify rsc_c001n0(X) should start before rsc_c001n0(X+1).
<cib cib_feature_revision="1" num_updates="1" have_quorum="false" epoche="1">
<configuration>
<crm_config>
<nvpair id="transition_timeout" name="transition_timeout" value="120s"/>
<nvpair id="symmetric_cluster" name="symetric_cluster" value="true"/>
<nvpair id="require_quorum" name="require_quorum" value="true"/>
<nvpair id="no_quorum_policy" name="no_quorum_policy" value="stop"/>
<nvpair id="suppress_cib_writes" name="suppress_cib_writes" value="true"/>
</crm_config>
<nodes/>
<resources>
<primitive id="DcIPaddr" class="ocf" type="IPaddr" provider="heartbeat">
<operations>
<op id="1" name="monitor" interval="5s" timeout="3s"/>
</operations>
<instance_attributes>
<attributes>
<nvpair name="ip" value="127.0.0.10"/>
</attributes>
</instance_attributes>
</primitive>
<primitive id="rsc_c001n01" class="ocf" type="IPaddr" provider="heartbeat">
<operations>
<op id="1" name="monitor" interval="5s" timeout="3s"/>
</operations>
<instance_attributes>
<attributes>
<nvpair name="ip" value="127.0.0.21"/>
</attributes>
</instance_attributes>
</primitive>
<primitive id="rsc_c001n02" class="ocf" type="IPaddr" provider="heartbeat">
<operations>
<op id="1" name="monitor" interval="5s" timeout="3s"/>
</operations>
<instance_attributes>
<attributes>
<nvpair name="ip" value="127.0.0.22"/>
</attributes>
</instance_attributes>
</primitive>
<primitive id="rsc_c001n03" class="ocf" type="IPaddr" provider="heartbeat">
<operations>
<op id="1" name="monitor" interval="5s" timeout="3s"/>
</operations>
<instance_attributes>
<attributes>
<nvpair name="ip" value="127.0.0.23"/>
</attributes>
</instance_attributes>
</primitive>
<primitive id="rsc_c001n04" class="ocf" type="IPaddr" provider="heartbeat">
<operations>
<op id="1" name="monitor" interval="5s" timeout="3s"/>
</operations>
<instance_attributes>
<attributes>
<nvpair name="ip" value="127.0.0.24"/>
</attributes>
</instance_attributes>
</primitive>
<primitive id="rsc_c001n05" class="ocf" type="IPaddr" provider="heartbeat">
<operations>
<op id="1" name="monitor" interval="5s" timeout="3s"/>
</operations>
<instance_attributes>
<attributes>
<nvpair name="ip" value="127.0.0.25"/>
</attributes>
</instance_attributes>
</primitive>
<primitive id="rsc_c001n06" class="ocf" type="IPaddr" provider="heartbeat">
<operations>
<op id="1" name="monitor" interval="5s" timeout="3s"/>
</operations>
<instance_attributes>
<attributes>
<nvpair name="ip" value="127.0.0.26"/>
</attributes>
</instance_attributes>
</primitive>
<primitive id="rsc_c001n07" class="ocf" type="IPaddr" provider="heartbeat">
<operations>
<op id="1" name="monitor" interval="5s" timeout="3s"/>
</operations>
<instance_attributes>
<attributes>
<nvpair name="ip" value="127.0.0.27"/>
</attributes>
</instance_attributes>
</primitive>
<primitive id="rsc_c001n08" class="ocf" type="IPaddr" provider="heartbeat">
<operations>
<op id="1" name="monitor" interval="5s" timeout="3s"/>
</operations>
<instance_attributes>
<attributes>
<nvpair name="ip" value="127.0.0.28"/>
</attributes>
</instance_attributes>
</primitive>
</resources>
<constraints>
<rsc_location id="run_DcIPaddr" rsc="DcIPaddr">
<rule id="cant_run_DcIPaddr" score="-INFINITY" boolean_op="and">
<expression attribute="#is_dc" operation="eq" value="false"/>
</rule>
</rsc_location>
<rsc_location id="run_rsc_c001n01" rsc="rsc_c001n01">
<rule id="pref_run_rsc_c001n01" score="100" boolean_op="and">
<expression attribute="#uname" operation="eq" value="c001n01"/>
</rule>
</rsc_location>
<rsc_order id="order_rsc_c001n01_rsc_c001n02" from="rsc_c001n01" action="start"
type="before" to="rsc_c001n02"/>
<rsc_location id="run_rsc_c001n02" rsc="rsc_c001n02">
<rule id="pref_run_rsc_c001n02" score="100" boolean_op="and">
<expression attribute="#uname" operation="eq" value="c001n02"/>
</rule>
</rsc_location>
<rsc_order id="order_rsc_c001n02_rsc_c001n03" from="rsc_c001n02" action="start"
type="before" to="rsc_c001n03"/>
<rsc_location id="run_rsc_c001n03" rsc="rsc_c001n03">
<rule id="pref_run_rsc_c001n03" score="100" boolean_op="and">
<expression attribute="#uname" operation="eq" value="c001n03"/>
</rule>
</rsc_location>
<rsc_order id="order_rsc_c001n03_rsc_c001n04" from="rsc_c001n03" action="start"
type="before" to="rsc_c001n04"/>
<rsc_location id="run_rsc_c001n04" rsc="rsc_c001n04">
<rule id="pref_run_rsc_c001n04" score="100" boolean_op="and">
<expression attribute="#uname" operation="eq" value="c001n04"/>
</rule>
</rsc_location>
<rsc_order id="order_rsc_c001n04_rsc_c001n05" from="rsc_c001n04" action="start"
type="before" to="rsc_c001n05"/>
<rsc_location id="run_rsc_c001n05" rsc="rsc_c001n05">
<rule id="pref_run_rsc_c001n05" score="100" boolean_op="and">
<expression attribute="#uname" operation="eq" value="c001n05"/>
</rule>
</rsc_location>
<rsc_order id="order_rsc_c001n05_rsc_c001n06" from="rsc_c001n05" action="start"
type="before" to="rsc_c001n06"/>
<rsc_location id="run_rsc_c001n06" rsc="rsc_c001n06">
<rule id="pref_run_rsc_c001n06" score="100" boolean_op="and">
<expression attribute="#uname" operation="eq" value="c001n06"/>
</rule>
</rsc_location>
<rsc_order id="order_rsc_c001n06_rsc_c001n07" from="rsc_c001n06" action="start"
type="before" to="rsc_c001n07"/>
<rsc_location id="run_rsc_c001n07" rsc="rsc_c001n07">
<rule id="pref_run_rsc_c001n07" score="100" boolean_op="and">
<expression attribute="#uname" operation="eq" value="c001n07"/>
</rule>
</rsc_location>
<rsc_order id="order_rsc_c001n07_rsc_c001n08" from="rsc_c001n07" action="start"
type="before" to="rsc_c001n08"/>
<rsc_location id="run_rsc_c001n08" rsc="rsc_c001n08">
<rule id="pref_run_rsc_c001n08" score="100" boolean_op="and">
<expression attribute="#uname" operation="eq" value="c001n08"/>
</rule>
</rsc_location>
</constraints>
</configuration>
<status/>
</cib>
| [1] | http://www.linux-ha.org/CIB |
| [2] | http://www.linux-ha.org/ClusterInformationBase/UnderstandingStatus |
| [3] | http://www.linux-ha.org/ClusterResourceManager/DTD1.0/Annotated#crm_config |
| [4] | http://www.linux-ha.org/ClusterResourceManager/DTD1.0/Annotated#node |
| [5] | http://www.linux-ha.org/ClusterResourceManager/DTD1.0/Annotated#resource |
| [6] | http://www.linux-ha.org/OCF |
| [7] | http://www.linux-ha.org/ResourceAgent |
| [8] | http://www.linux-ha.org/LSB |
| [9] | http://www.linux-ha.org/ClusterResourceManager/DTD1.0/Annotated#constraint |
| [10] | http://www.linux-ha.org/ClusterResourceManager/DTD1.0/Annotated#rsc_colocation |
| [11] | http://www.linux-ha.org/ClusterResourceManager/DTD1.0/Annotated#rsc_location |
| [12] | http://www.linux-ha.org/ClusterResourceManager/DTD1.0/Annotated#rule |
| [13] | http://www.linux-ha.org/ClusterResourceManager/DTD1.0/Annotated#expression |
| [14] | http://www.linux-ha.org/ClusterInformationBase/AdvancedRules |
| [15] | http://www.linux-ha.org/ClusterTestSuite |
| [16] | http://www.linux-ha.org/DesignatedCoordinator |
This information provided courtesy of the Linux-HA project at http://linux-ha.org/