Heartbeat Test Plan

This page documents the heartbeat[1] (aka linux-ha) test plan.

Document Control

Maintained online at "DevCorner/TestPlan". URL is: http://wiki.linux-ha.org/DevCorner/TestPlan[2], or http://linux-ha.org/DevCorner/TestPlan[3].

• Retention Level: Valid until updated online. See wiki for current information.

  Initial Author: AlanRobertson[4]

  • Distribution List
    • Distributed to wiki page subscribers on every update.

      • Subscribers must include Stephanie Glass[5]. Distribution of new versions is automatic via Wiki change tracking system, including facilities for comparing any two versions.

Change Control

Maintained automatically by Wiki change tracking system.

I. Introduction

I.A. References/Related Documents

I.B. Line Item

II. Test Plan Overview

The test plan consists running and evaluating the results from of a set of automated tests which are shipped with the Linux-HA software, plus a set of manually performed GUI tests, as docmented in section VII.

There are two suites of tests which ship with Linux-HA:

• cts[6] tests

• BasicSanityCheck[7]

Each of these tests is simple to run, and simple to evaluate. In addition, BasicSanityCheck[7] does not require any special setup to make it work correctly and runs very quickly (in approximately a minute). cts[6], on the other hand requires a significant amount of setup before running it, and normally runs for many hours.

We do not yet have a set of automated GUI tests, so they are (for the time being) manual, and described in section 7.

III. Assumptions and Dependencies

III.A. Additional Program Products

III.B. Hardware

BasicSanityCheck[7] can run on any single Linux machine without any special preparation or architectural constraints.

cts[6], on the other hand, requires three machines to run it on. The tests are controlled from the test monitor machine, and they are run on the other two machines. The Linux-HA package must be installed (in a consistent version) on all three machines. Detailed information about setting up cts[6] is supplied by the cts/README file.

It is highly desirable that the clocks on all the test machines be closely synchronized in order to track down any errors easily.

III.C. General

cts[6] executes in the context of a particular configuration, but does not create different configurations. As a result, it is often desirable to run it once with each of several possible configurations. Most often, the main variant used in the configurations is the value of the auto_failback directive[8]. If two or more configurations are to be considered, one should have auto_failback on and one with it off.

III.D. History

IV. Test Goals and Objectives

The BasicSanityCheck[7] software verifies that many of the major paths in heartbeat andsome associated programs are exercised. This is useful to find problems where the linux-HA software might not be running for reason of some significant platform-related bug.

The cts[6] package is a suite of random tests which eventually run the system through many possible configurations and thoroughly test a configuration with a random sequence of tests chosen from its battery of tests.

IV.A. Hardware and Software Configuration

In order to do any testing, the heartbeat, heartbeat-pils, and heartbeat-stonith packages must all be installed with the version which is to be tested. For the cts[6] tests this means on all three machines. For the BasicSanityCheck[7] test, this means only the single machine to be tested.

Software configuration for the cts[6] tests is complex and is described in detail in the cts[6] README file. This README file can be found at /usr/lib/heartbeat/cts/README[9]. The latest greatest version of the README (which might not match the version you're running) can be found in CVS[10].

The GUI is a program which can be run on any Linux system with the proper set of libraries installed. It is X-Windows based, so that tester must have access to an X-Windows server to view the results on.

IV.B. Test Approach and Methodology

IV.C. System Operation

??

IV.D. Performance

N/A

IV.E. Standards Compliance

N/A

IV.F. Stress

The CTS tests create significant system stress.

IV.G. Regression

Both tests perform a version of regression testing. The GUI regression tests are described in section VII.

IV.H. Ship Test

N/A

IV.I. Installation Documentation

??

IV.J. Installation/Configuration Test

??

IV.K. Reliability, Availability, and Serviceability

??

IV.L. Usability

N/A

V. Quality Goals

V.A. Goals

All tests should pass with zero errors when properly configured.

V.B. Measurements

Run the tests. If any cts[6] tests fail when a reliable message transport is configured for syslog as described in the README, then this is a bug. If it is configured with UDP transport for syslog, it is possible that certain messages might get lost in a busy networking environment. Then the logs on the original machines can be examined to see if the missing message was actually issued but is missing. This is very difficult unless the system clocks are closely synchronized.

VI. Status Information

VII. Testcase Descriptions

Each CTS test case is described in the file CTStests.py. Each test is a class definition similar to this one:

• class FlipTest(CTSTest):

The tests which are currently in the test suite are described in the CTS page[6].

This tests DRBD to see if it is maintaining proper integrity of the disk data. This requires that you configure DRBD into your configuration. If you do not, then it will not be run.

GUI Testcase Descriptions

We are developing a set of testcases for manually tests of GUI. These testcases should cover most functions of GUI. GuiTest[11]

VII.A. Naming Conventions

N/A

VII.B. Testcase Location

The BasicSanityCheck[7] script can be found at /usr/lib/heartbeat/BasicSanityCheck[12]. The cts[6] package is located in the /usr/lib/heartbeat/cts[13] directory.

VIII. Functional Coverage Matrix

IX. Approval Criteria

---

This information provided courtesy of the Linux-HA project at http://linux-ha.org/