This site best when viewed with a modern standards-compliant browser. We recommend Firefox Get Firefox!.

Linux-HA project logo
Providing Open Source High-Availability Software for Linux and other OSes since 1999.

USA Flag UK Flag

Japanese Flag

Homepage

About Us

Contact Us

Legal Info

How To Contribute

Security Issues

This web page is no longer maintained. Information presented here exists only to avoid breaking historical links.
The Project stays maintained, and lives on: see the Linux-HA Reference Documentation.
To get rid of this notice, you may want to browse the old wiki instead.

1 February 2010 Hearbeat 3.0.2 released see the Release Notes

18 January 2009 Pacemaker 1.0.7 released see the Release Notes

16 November 2009 LINBIT new Heartbeat Steward see the Announcement

Last site update:
2014-07-23 03:42:28

Howto get started with DRBD 0.7

aka

Quick start guide for DRBD 0.7

Started by Lars Ellenberg; feel free to extend this, or write the ComprehensiveDRBDGuide ...

  • <!> Note that DRBD 0.7 is seriously outdated and no longer maintained.

    • You should be using our current stable release series, which come in two flavours:
      • feature frozen drbd 8.0.x, and
      • open for new features 8.2.y
      Please also have a look at the excellent Documentation

      on the drbd.org site.

That being said, some of the information given here may still be useful for someone, so I just leave it there.

You can also find a Red Hat specific (should work on Fedora, RHEL, CentOS or any other Red Hat based distro) How To in the IBB Wiki.

If you install via tarball, you want to read the README, INSTALL, and DRBD/HowTo/Install first.

You want to also read the upgrade*.txt files in the drbd src directory, or online in the drbd subversion repository.

Along with the DRBD tarball or your installed DRBD package, you should find a well commented example configuration file. In the tarball this is in ./scripts/drbd.conf, in installed binary packages it most likely ends up somehwere in /usr/{shared/,}doc/packages/drbd.

You edit that to your needs, copy it to /etc/drbd.conf on both nodes, and check that the meta-disk points to the right location.

If you are setting up multiple resources, make sure that your /etc/drbd.conf file uses different ports (or different IPs) for each resource.

<!> Caution

  • Double check, because if the next step will not find any valid DRBD meta data block where it expects it to be, it will create a new one. If you pointed it to the wrong place, it will overwrite a few KB to several MB of possibly valuable data!

    If you want to use meta-disk = internal;, make sure you shrink an existing file system first!.

Currently DRBD meta-data reserves 128MB, regardless of actual physical data storage. This allows for a maximum storage size of a single DRBD device of approximatly 4TB.

Ok, now, we have that drbd.conf, copy it to /etc/drbd.conf of both nodes. Then, on both nodes, do a drbdadm up all. It should come up as both nodes Secondary and Inconsistent.

The latter is because the underlying storage is not yet in-sync, and DRBD has no way to know whether you want the initial sync from left-to-right, or right-to-left. So you have to tell it. If you have no data below it, it does not matter. But if you have an existing filesystem on the lower level device, and now you sync in the wrong direction, you'd need to get your latest backup ready afterwards...

You choose which node shall become Primary for the first time (the one with the good data, if you have some), then you do drbdadm -- --do-what-I-say primary all (and you better be sure you mean what you say). The result is a full sync of the underlying devices (initial full sync).

The device is usable right away, so if you have no file system already, you should create one now.

You now mount the DRBD on the current Primary, edit some files, unmount it, make that node Secondary, the other one Primary, mount the DRBD there, and see your edits have been mirrored.

Thats it, now you integrate it with heartbeat.

Happy DRBDing.

Again, on the command line

  • This example already uses /dev/drbdX instead of /dev/nbX, what you might have expected. For historical reasons DRBD used to hijack NBDs device major (43), and device nodes. We now have an officially assigned DRBD major (147), therefore starting with 0.7.1 we default to use our own major numbers and device nodes. (Unfortunately the assignment was a few days late for 0.7.0; so your distribution may have decided to stay with the deprecated /dev/nbX # major 43)

    If your system does not know about /dev/drbdX yet, you should create these device nodes like for i in $(seq 0 15) ; do mknod /dev/drbd$i b 147 $i ; done. See also the upgrade*.txt files, mentioned earlier.

# administration ips of the nodes:
left=10.0.0.1
right=10.0.0.2

vi drbd.conf
# double check.
scp drbd.conf $left:/etc/drbd.conf
scp drbd.conf $right:/etc/drbd.conf

cmd='modprobe drbd; drbdadm up all; dmesg | tail; cat /proc/drbd'
ssh root@$left -- "$cmd"
ssh root@$right -- "$cmd"

The excerpt from dmesg (or syslog) should look like this ( from an example run with only 5MB of storage ... )

drbd: initialised. Version: 0.7.0 svn $Rev: 1442 $ (api:74/proto:74)
drbd: registered as block device major 147

  nb: to have it register as 43 (NBD) you can say
  modprobe drbd use_nbd_major

drbd0: Creating state block
drbd0: resync bitmap: bits=1250 words=40
drbd0: size = 5000 KB
drbd0: Assuming that all blocks are out of sync (aka FullSync)
drbd0: 5000 KB now marked out-of-sync by on disk bit-map.
drbd0: Handshake successful: DRBD Network Protocol version 74
drbd0: Connection established.
drbd0: I am inconsistent, but there is no sync? BOTH nodes inconsistent!
drbd0: Secondary/Unknown --> Secondary/Secondary

and /proc/drbd one should look like this:

# cat /proc/drbd
version: 0.7.0 svn $Rev: 1442 $ (api:74/proto:74)

 0: cs:Connected st:Secondary/Secondary ld:Inconsistent
     ns:0 nr:0 dw:0 dr:0 al:0 bm:1 lo:0 pe:0 ua:0 ap:0

Now, I'll chose $left to be the primary.

ssh root@$left -- "drbdadm primary all"
# output is:
ioctl(,SET_STATE,) failed: Input/output error
Local replica is inconsistent (--do-what-I-say ?)
Command line was '/sbin/drbdsetup /dev/drbd0 primary'
drbdsetup exited with code 21

NOTE: This will cause the system to reboot (!)

  • Well, actually this will cause the configured "incon-degr-cmd" to be executed by drbdadm. Since in the example configuration this include a "halt -f", you asked for that reboot. So until you have a running setup, you should probably configure something less brutal, maybe rather configure in your drbd.conf

     incon-degr-cmd "echo 'DRBD: primary requested but inconsistent!' | wall; sleep 300000"; 

    an other nice command to configure there is  killall -9 heartbeat ipfail ccm 

# ok, this was expected.
# so double check that $left is the correct node.
# then force it:
ssh root@$left -- "drbdadm -- --do-what-I-say primary all"
# which will succeed silently.

## Bryce Porter suggests that:
## At this point, you need to connect the resource(s)
# ssh root@$left -- "drbdadm -- connect all"
#
## well, they should have been connected all along, see /proc/drbd excerpt above,
## so if this was neccessary, something "unexpected" happend already...
##     -- lge

ssh $left -- 'dmesg | tail ; cat /proc/drbd'
# output is:
drbd0: Resync started as SyncSource (need to sync 5000 KB [1250 bits set]).

version: 0.7.0 svn $Rev: 1442 $ (api:74/proto:74)

 0: cs:SyncSource st:Primary/Secondary ld:Consistent
    ns:9276 nr:0 dw:0 dr:9404 al:0 bm:2 lo:0 pe:915 ua:32 ap:0
        [=========>..........] sync'ed: 50.0% (4380/5000)K
        finish: 0:00:05 speed: 620 (620) K/sec

# or, to give an example from a larger device:
 0: cs:SyncSource st:Primary/Secondary ld:Consistent
    ns:12940824 nr:0 dw:87492 dr:13690591 al:109 bm:1668 lo:1000 pe:1876 ua:1000 ap:0
        [========>...........] sync'ed: 44.4% (15858/28487)M
        finish: 0:09:20 speed: 28,933 (25,160) K/sec


# whereas on the other node:
ssh $right -- 'dmesg | tail ; cat /proc/drbd'
drbd0: Resync started as SyncTarget (need to sync 5000 KB [1250 bits set]).

version: 0.7.0 svn $Rev: 1442 $ (api:74/proto:74)

 0: cs:SyncTarget st:Secondary/Primary ld:Inconsistent
    ns:0 nr:15000 dw:15000 dr:0 al:0 bm:6 lo:0 pe:0 ua:0 ap:0
        [=========>..........] sync'ed: 50.0% (5000/5000)K
        finish: 0:00:12 speed: 5 (5) K/sec

# or, to give an example from a larger device:
 0: cs:SyncTarget st:Secondary/Primary ld:Inconsistent
    ns:0 nr:27311780 dw:27311780 dr:0 al:0 bm:3447 lo:134 pe:493 ua:134 ap:0
        [==================>.] sync'ed: 93.7% (1818/28487)M
        finish: 0:01:07 speed: 27,482 (25,008) K/sec

see that it is working:

# if you have no file system yet, create one
# ssh root@$left -- 'mkreiserfs /dev/drbd0'

ssh root@$left -- 'mkdir -p /mnt/ha0; mount /dev/drbd0 /mnt/ha0 && touch /mnt/ha0/been_there'

# 'switch over'
ssh root@$left -- 'umount /mnt/ha0 && drbdadm secondary all'
# even during sync!
ssh root@$right -- 'drbdadm primary all'
ssh root@$right -- 'mkdir -p /mnt/ha0; mount /dev/drbd0 /mnt/ha0 && ls -l /mnt/ha0/been_there'
  • <!> Note that even though you can do so, it is a bad idea to have the SyncTarget be Primary, it will panic on a network failure, because it then has no more access to good data ...


And don't forget to support the DRBD project. If you use it, it will likely save you money. You could invest part of that savings in the future of it, and of Open Source Software in general.

  • ;)


Back to the main DRBD page...