This site is a work in progress — you can help! Please see the Site news for details.

CTDB (resource agent)

From Linux-HA

Jump to: navigation, search

Contents

Introduction

This resource agent manages CTDB, allowing one to use Clustered Samba in a Linux-HA/Pacemaker cluster.

Those familiar with CTDB will be aware that it can handle node failover, and includes event scripts for managing services other than Samba (e.g. NFS, HTTPD, etc.). This is fine if you want CTDB to manage the cluster, but these features are not appropriate to use in a Linux-HA/Pacemaker cluster, because these abilities are already present in Pacemaker and the underlying messaging layer. To configure a system such that some HA resources are managed by CTDB, and some are managed by Pacemaker starts to become confusing, to say the least.

This CTDB RA will, by default, start and stop CTDBD only. Samba and Winbind should be configured as separate resources, colocated with and ordered after the CTDB resource (or put in a cloned group). Currently (2010-11-30) we don't have Samba or Winbind OCF resource agents, so it is still possible to configure the CTDB RA such that CTDB will in turn manage Samba and Winbind. This mode will be deprecated in future.

Any other resources you need (IP addresses, clustered filesystems, NFS serving, HTTPD, etc.) must be configured within Pacemaker as usual, with appropriate ordering and colocation constraints.


Usage

  • Configure a shared filesysem (e.g. OCFS2). For the purposes of these instructions, we assume it's mounted at /shared-fs.
  • Ensure ctdb, smb, nmb and winbind services are disabled (chkconfig service off or similar).
  • Make directory for CTDB lock on the shared filesystem:
# mkdir -p /shared-fs/samba
  • Create /etc/ctdb/nodes on all nodes, containing a list of the private IP addresses of each node in the cluster, e.g.:
# cat /etc/ctdb/nodes
192.168.101.14
192.168.101.15
  • Add a share (or shares) to /etc/samba/smb.conf on all nodes:
[myshare]
path = /shared-fs/myshare
# ...other options here, e.g.: read only = no etc...
  • Add a CTDB resource to the cluster (assuming clustered filesystem clone is named fs-clone):
# crm configure
# primitive ctdb ocf:heartbeat:CTDB params \
    ctdb_recovery_lock="/shared-fs/samba/ctdb.lock" \
    ctdb_manages_samba="yes" \
    ctdb_manages_winbind="yes" \
    op monitor timeout=20 interval=10
# clone ctdb-clone ctdb \
    meta globally-unique="false" interleave="true"
# colocation ctdb-with-fs inf: ctdb-clone fs-clone
# order start-ctdb-after-fs inf: fs-clone ctdb-clone
# commit

IP Addresses

There are at least two ways to do this:

Clustered IP Address

  • Add a clustered IP address:
# crm configure
# primitive ip ocf:heartbeat:IPaddr2 params ip=192.168.100.222 \
    clusterip_hash="sourceip-sourceport" op monitor interval=60s
# clone ip-clone ip meta globally-unique="true"
# colocation ip-with-ctdb inf: ip-clone ctdb-clone
# order start-ip-after-ctdb inf: ctdb-clone ip-clone
# commit
  • Result should be something like:
# crm status
 ...
 Clone Set: dlm-clone
     Started: [ node-0 node-1 ]
 Clone Set: o2cb-clone
     Started: [ node-0 node-1 ]
 Clone Set: fs-clone
     Started: [ node-0 node-1 ]
 Clone Set: ctdb-clone
     Started: [ node-0 node-1 ]
 Clone Set: ip-clone (unique)
     ip:0       (ocf::heartbeat:IPaddr2):       Started node-0
     ip:1       (ocf::heartbeat:IPaddr2):       Started node-1

This will give a single IP address, connections to which will be handled by one of the nodes in the cluster.

One or More Distinct IP Addresses

Configure multiple separate IPaddr2 resources (non-cloned). It should be possible to combine these with the Tickle ACK feature newly added to the portblock RA. Primitives should be similar to:

 # primitive block-1 ocf:heartbeat:portblock \
     params ip="192.168.100.201" protocol="tcp" \
       portno="137,138,139,445" action="block"
 # primitive ip-1 ocf:heartbeat:IPaddr2 \
     params ip="192.168.100.201" \
     op monitor interval="60s"
 # primitive unblock-1 ocf:heartbeat:portblock \
     params ip="192.168.100.201" protocol="tcp" \
       portno="137,138,139,445" action="unblock" \
       tickle_dir="/shared-fs/tickle"               <--- ideally on shared storage
     op monitor interval="10s"

Then, constraints need to be configured such that the resources start in the following order:

block-1 -> ip-1 -> ctdb -> unblock-1

(Obviously, this still requires some more fleshing-out)

Available RA Instance Parameters

OCF_RESKEY_ctdb_recovery_lock
required, location of lock file on shared storage
OCF_RESKEY_ctdb_manages_samba
optional, default=no (will be deprecated in future)
OCF_RESKEY_ctdb_manages_winbind
optional, default=no (will be deprecated in future)
OCF_RESKEY_ctdb_service_smb
optional, will usually be auto-detected, only necessary if CTDB is managing Samba
OCF_RESKEY_ctdb_service_nmb
optional, will usually be auto-detected, only necessary if CTDB is managing Samba
OCF_RESKEY_ctdb_service_winbind
optional, will usually be auto-detected, only necessary if CTDB is managing Winbind
OCF_RESKEY_ctdb_samba_skip_share_check
optional, default=yes
OCF_RESKEY_ctdb_monitor_free_memory
optional, default=100
OCF_RESKEY_ctdb_start_as_disabled
optional, default=yes
OCF_RESKEY_ctdb_config_dir
optional, default=/etc/ctdb
OCF_RESKEY_ctdb_binary
optional, default=/usr/bin/ctdb
OCF_RESKEY_ctdbd_binary
optional, default=/usr/sbin/ctdbd
OCF_RESKEY_ctdb_socket
optional, default=/var/lib/ctdb/ctdb.socket
OCF_RESKEY_ctdb_dbdir
optional, default=/var/lib/ctdb
OCF_RESKEY_ctdb_logfile
optional, default=/var/log/ctdb/log.ctdb
OCF_RESKEY_ctdb_debuglevel
optional, default=2
OCF_RESKEY_smb_conf
optional, default=/etc/samba/smb.conf
OCF_RESKEY_smb_private_dir
optional, directory for smbpasswd, secrets.tdb, etc. (deprecated - do not use with CTDB > 1.0.50)
OCF_RESKEY_smb_passdb_backend
optional, default=tdbsam, only used if CTDB is managing Samba
OCF_RESKEY_smb_idmap_backend
optional, default=tdb2, only used if CTDB is managing Samba

Notes

  • The CTDB RA is new in agents-1.0.2 and hasn't yet seen widespread use.
  • This RA will automatically generate a new, minimal /etc/sysconfig/ctdb file when started, based on the parameters specified for the CTDB resource.
  • If configured to manage Samba, this RA will automatically add a few settings to the [global] section of /etc/samba/smb.conf, to enable CTDB. These lines will be removed when the CTDB resource is stopped. If Samba is configured as a separate resource, the smb.conf file must have the appropriate settings added manually (passdb backend = tdbsam, clustering = yes, idmap backend = tdb2, ctdbd socket = /var/lib/ctdb/ctdb.socket)
Personal tools