Print v2/Concepts/MultiState: from Linux HA

Contents

Introduction
Extra Options
Simple Example
What if?
FAQ
Writing a Clone Resource Agent
Advanced Example
Location Constraints for Masters
Location Constraints for Slaves
Co-locational and Ordering Constraints related to Masters
Co-locational and Ordering Constraints related to Slaves
Finding Masters in the Status Section

Related Pages: v2/Concepts[1] v2/Concepts/Clones[2] outdated documentation[3]

Introduction

Superset of a cloned resource.

Can be either anonymous or unique

Currently supports two states

The states mean whatever you want them to mean

Extra Options

master_max (default: 1)
- The maximum number of resource masters that should be run in the cluster.
master_node_max (default: 1)
- The maximum number of resource masters that should be run on any given node.

Simple Example

In the example below we have specified that the WebServerIP resource should be a ClusteredIP. It will be started on at most 3 nodes and at most there can be 2 clone active on any given node. Instance 1 will serve the first "bucket", instance 2 the second and so on.

<master_slave id="MyCorpWWW" ordered="false" interleave="false" notify="false">
  <instance_attributes>
    <attributes>
      <nvpair name="clone_max" value="3"/>
      <nvpair name="clone_node_max" value="2"/>
      <nvpair name="master_max" value="2"/>
      <nvpair name="master_node_max" value="1"/>
    </attributes>
  </instance_attributes>
  <primitive id="WebServerIP" class="ocf" type="IPaddr" provider="heartbeat">
    <instance_attributes>
      <attributes>
        <nvpair name="ip" value="127.0.0.26"/>
      </attributes>
    </instance_attributes>
  </primitive>
</master_slave>

What if?

If there are >=1 nodes available:
- Two nodes will be chosen and each node will run 1 master
If there are 2 nodes available:
- 1 node will run 2 clones (1 master, 1 slave), the other 1 master
If there is 1 nodes available:
- 1 node will run 2 clones (1 master, 1 slave), the clone will not be started

FAQ

Will the master clone instances always be an unbroken sequence?
- No.
Will the master clone instances always start at 0?
- No.
Does the CRM keep track of the clone instances' state?
- No.
  It is the responsibility of the OCFResourceAgent[4] to know what state the clone instance is in. The CRM knows what state it thinks the resource should be in, but in all cases it is the script which must understand the resource and determine it's true state.
Why does monitoring stop after my resource is promoted to master?
- You must define a monitor action for every role (e.g. Master, Slave, Started, Stopped) you wish to monitor. Please refer to the examples given in ClusterInformationBase/Actions[5].

Writing a Clone Resource Agent

Which Resource Instance is Promoted

During the start operation, Resource Agent scripts should call the crm_master command-line tool. This tool automatically detects both the resource and host and should be used to set a preference for being promoted. Based on this, master_max, and master_node_max, the instance(s) with the highest preference will be promoted. Instances with a negative or zero preference will never be promoted; a positive preference must be set for all eligible instances.

BeekhofG5::beekhof pengine # crm_master 
usage: crm_master [-?VQ] -(D|G|v) [-l]
Options
        --help (-?)     : this help message
        --verbose (-V)  : turn on debug info. additional instances increase verbosity
        --quiet (-Q)    : Print only the value on stdout (use with -G)
        --get-value (-G)        : Retrieve rather than set the attribute
        --delete-attr (-D)      : Delete rather than set the attribute
        --attr-value (-v) <string>      : Value to use (ignored with -G)
        --lifetime (-l) <string>        : How long the preference lasts (reboot|forever)

Requirements for ResourceAgents

This items listed here are in addition to the requirements for Clones[2].

ResourceAgents[6] are required to accurately report how many instances are active, inactive, masters and slaves. In monitor operations:

Active masters should return $OCF_RUNNING_MASTER
Failed masters should return $OCF_FAILED_MASTER
Inactive instances should return $OCF_NOT_RUNNING
All other active instances should use
- $OCF_SUCCESS if they are active, or
- one of the $OCF_ERR_* values if they have failed

All OCF variables can normally be found in /usr/lib/ocf/resource.d/heartbeat/ocf-shellfuncs which should be the first thing included by a resource agent.

Extra Operations

A Master/Slave Resource Agent must implement the promote and demote operations. The implementation is up to the user.

Extra Environment Variables

In addition to the environment variables added for clones[7], these variables are available to notify actions.

OCF_RESKEY_notify_operation (start|stop|promote|demote)
OCF_RESKEY_notify_{x}_resource
OCF_RESKEY_notify_ {x}_uname
{x} : promote, demote, master, slave

Starting with 2.0.4, they will also be available to start, stop, promote and demote actions if notify="true" is set.

Proper Interpretation of Master Environment Variables

Pre-notification (demote)
- Active resources: $OCF_RESKEY_notify_active_resource
- Master resources: $OCF_RESKEY_notify_master_resource
- Slave resources: $OCF_RESKEY_notify_slave_resource
- Inactive resources: $OCF_RESKEY_notify_inactive_resource
- Resources to be started: $OCF_RESKEY_notify_start_resource
- Resources to be promoted: $OCF_RESKEY_notify_promote_resource
- Resources to be demoted: $OCF_RESKEY_notify_demote_resource
- Resources to be stopped: $OCF_RESKEY_notify_stop_resource
- Resources that were started: NA
- Resources that were promoted: NA
- Resources that were demoted: NA
- Resources that were stopped: NA
Post-notification (demote) / Pre-notification (stop)
- Active resources: $OCF_RESKEY_notify_active_resource
- Master resources: $OCF_RESKEY_notify_master_resource
  - minus $OCF_RESKEY_notify_demote_resource
- Slave resources: $OCF_RESKEY_notify_slave_resource
- Inactive resources: $OCF_RESKEY_notify_inactive_resource
- Resources to be started: $OCF_RESKEY_notify_start_resource
- Resources to be promoted: $OCF_RESKEY_notify_promote_resource
- Resources to be demoted: $OCF_RESKEY_notify_demote_resource
- Resources to be stopped: $OCF_RESKEY_notify_stop_resource
- Resources that were started: NA
- Resources that were promoted: NA
- Resources that were demoted: $OCF_RESKEY_notify_demote_resource
- Resources that were stopped: NA
Post-notification (stop) / Pre-notification (start)
- Active resources: $OCF_RESKEY_notify_active_resource
  - minus $OCF_RESKEY_notify_stop_resource
- Master resources: $OCF_RESKEY_notify_master_resource
  - minus $OCF_RESKEY_notify_demote_resource
- Slave resources: $OCF_RESKEY_notify_slave_resource
  - minus $OCF_RESKEY_notify_stop_resource
- Inactive resources: $OCF_RESKEY_notify_inactive_resource
  - plus $OCF_RESKEY_notify_stop_resource
- Resources to be started: $OCF_RESKEY_notify_start_resource
- Resources to be promoted: $OCF_RESKEY_notify_promote_resource
- Resources to be demoted: $OCF_RESKEY_notify_demote_resource
- Resources to be stopped: $OCF_RESKEY_notify_stop_resource
- Resources that were started: NA
- Resources that were promoted: NA
- Resources that were demoted: $OCF_RESKEY_notify_demote_resource
- Resources that were stopped: $OCF_RESKEY_notify_stop_resource
Post-notification (start) / Pre-notification (promote)
- Active resources: $OCF_RESKEY_notify_active_resource
  - minus $OCF_RESKEY_notify_stop_resource
    plus $OCF_RESKEY_notify_start_resource
- Master resources: $OCF_RESKEY_notify_master_resource
  - minus $OCF_RESKEY_notify_demote_resource
- Slave resources: $OCF_RESKEY_notify_slave_resource
  - minus $OCF_RESKEY_notify_stop_resource
    plus $OCF_RESKEY_notify_start_resource
- Inactive resources: $OCF_RESKEY_notify_inactive_resource
  - plus $OCF_RESKEY_notify_stop_resource
    minus $OCF_RESKEY_notify_start_resource
- Resources to be started: $OCF_RESKEY_notify_start_resource
- Resources to be promoted: $OCF_RESKEY_notify_promote_resource
- Resources to be demoted: $OCF_RESKEY_notify_demote_resource
- Resources to be stopped: $OCF_RESKEY_notify_stop_resource
- Resources that were started: $OCF_RESKEY_notify_start_resource
- Resources that were promoted: NA
- Resources that were demoted: $OCF_RESKEY_notify_demote_resource
- Resources that were stopped: $OCF_RESKEY_notify_stop_resource
Post-notification (promote)
- Active resources: $OCF_RESKEY_notify_active_resource
  - minus $OCF_RESKEY_notify_stop_resource
    plus $OCF_RESKEY_notify_start_resource
- Master resources: $OCF_RESKEY_notify_master_resource
  - minus $OCF_RESKEY_notify_demote_resource
    plus $OCF_RESKEY_notify_promote_resource
- Slave resources: $OCF_RESKEY_notify_slave_resource
  - minus $OCF_RESKEY_notify_stop_resource
    plus $OCF_RESKEY_notify_start_resource
    minus $OCF_RESKEY_notify_promote_resource
- Inactive resources: $OCF_RESKEY_notify_inactive_resource
  - plus $OCF_RESKEY_notify_stop_resource
    minus $OCF_RESKEY_notify_start_resource
- Resources to be started: $OCF_RESKEY_notify_start_resource
- Resources to be promoted: $OCF_RESKEY_notify_promote_resource
- Resources to be demoted: $OCF_RESKEY_notify_demote_resource
- Resources to be stopped: $OCF_RESKEY_notify_stop_resource
- Resources that were started: $OCF_RESKEY_notify_start_resource
- Resources that were promoted: $OCF_RESKEY_notify_promote_resource
- Resources that were demoted: $OCF_RESKEY_notify_demote_resource
- Resources that were stopped: $OCF_RESKEY_notify_stop_resource

Advanced Example

Location Constraints for Masters

It is sometimes desirable to preferentially constrain the master instance of a master/slave resource to run on a particular node or set of nodes in the cluster.

The CIB snippet below will add prefer the node ace to be master of the resource ms-drbd1 by 100 points.

<rsc_location id="loc:ms-drbd1_likes_ace" rsc="ms-drbd1">
  <rule id="rule:ms-drbd1_likes_ace" role="master" score="100">
    <expression  attribute="#uname" operation="eq" value="ace"/>
  </rule>
</rsc_location>

Location Constraints for Slaves

It is sometimes desirable to preferentially constrain the slave instance of a master/slave resource to run on a particular node or set of nodes in the cluster.

However, since all resources go through slave status before becoming promoted, what we have to do is constrain it to avoid becoming master.

The CIB snippet below will prefer to promote any node but fred to be master of the resource drbd1 by 100 points. This is effectively the same as saying "we want fred to avoid becoming master if possible".

<rsc_location id="loc:drbd1_likes_fred" rsc="drbd1">
  <rule id="rule:drbd1_likes_fred" role="master" score="100">
    <expression  attribute="#uname" operation="ne" value="fred">
  </rule>
</rsc_location>

Co-locational and Ordering Constraints related to Masters

It is very common to want to have a certain resource run only on a node which is master for a given resource.

You can either make sure the resource is run on the master, which is the propably more common approach:

<rsc_order id="ms-drbd0_before_fs0" from="fs0" action="start" to="ms-drbd0" to_action="promote"/>
<rsc_colocation id="fs0_on_ms-drbd0" to="ms-drbd0" to_role="master" from="fs0" score="infinity"/>

Translation:

Start fs0 after ms-drbd0 has been promoted
Only start fs0 when ms-drbd0 is running as master

Or you can make sure the resource is not run on the slave:

<rsc_order id="ms-drbd0_before_fs0" from="fs0" action="start" to="ms-drbd0" to_action="promote"/>
<rsc_colocation id="fs0_not_on_stopped_ms-drbd0" to="ms-drbd0" to_role="stopped" from="fs0" score="-infinity"/>
<rsc_colocation id="fs0_not_on_slave_ms-drbd0" to="ms-drbd0" to_role="slave" from="fs0" score="-infinity"/>

Translation:

Never run fs0 on a node where ms-drbd0 is stopped.
Never run fs0 on a node where ms-drbd0 is running as slave.
Only start fs0 after promoting ms-drbd0 to master.

Since the the total set of states for a master/slave resource is {stopped, slave, master}, this only allows it to run on a node which is running as master.

Co-locational and Ordering Constraints related to Slaves

The idiom to use if you have a resource which you want to run on the slave node is similar to the colocation with master[8] case:

<rsc_colocation id="fs_on_drbd0" to="drbd0-partition"
  to_role="stopped"
  from="mount-drbd0" score="-infinity"/>
<rsc_colocation id="fs_on_drbd0" to="drbd0-partition"
  to_role="master"
  from="mount-drbd0" score="-infinity"/>

Translation:

Never run fs_on_drbd0 on a node where drbd0-partition is stopped.
Never run fs_on_drbd0 on a node where drbd0-partition is running as master.

Since the the total set of states for a master/slave resource is {stopped, slave, master}, this only allows the resource to run on a node which is running as slave.

Question:
I don't know how to write the corresponding ordering rule for this case - to ensure that it only runs after being started OR after being demoted from master. Is it necessary? If it isn't necessary, then probably the ordering rule for the master case isn't needed either.

Finding Masters in the Status Section

References

[1]	http://www.linux-ha.org/v2/Concepts
[2]	http://www.linux-ha.org/v2/Concepts/Clones
[3]	http://www.linux-ha.org/ClusterResourceManager/MultiStateResources
[4]	http://www.linux-ha.org/OCFResourceAgent
[5]	http://www.linux-ha.org/ClusterInformationBase/Actions
[6]	http://www.linux-ha.org/ResourceAgents
[7]	http://www.linux-ha.org/v2/Concepts/MultiState
[8]	http://www.linux-ha.org/CIB/Idioms/ColoWithMaster