This site best when viewed with a modern standards-compliant browser. We recommend Firefox Get Firefox!.

Linux-HA project logo
Providing Open Source High-Availability Software for Linux and other OSes since 1999.

USA Flag UK Flag

Japanese Flag

Homepage

About Us

Contact Us

Legal Info

How To Contribute

Security Issues

This web page is no longer maintained. Information presented here exists only to avoid breaking historical links.
The Project stays maintained, and lives on: see the Linux-HA Reference Documentation.
To get rid of this notice, you may want to browse the old wiki instead.

1 February 2010 Hearbeat 3.0.2 released see the Release Notes

18 January 2009 Pacemaker 1.0.7 released see the Release Notes

16 November 2009 LINBIT new Heartbeat Steward see the Announcement

Last site update:
2017-12-17 15:37:19

Heartbeat Test Plan

This page documents the heartbeat (aka linux-ha) test plan.

Contents

  1. Heartbeat Test Plan
  2. Document Control
  3. Change Control
  4. I. Introduction
    1. I.A. References/Related Documents
    2. I.B. Line Item
  5. II. Test Plan Overview
  6. III. Assumptions and Dependencies
    1. III.A. Additional Program Products
    2. III.B. Hardware
    3. III.C. General
    4. III.D. History
  7. IV. Test Goals and Objectives
    1. IV.A. Hardware and Software Configuration
    2. IV.B. Test Approach and Methodology
    3. IV.C. System Operation
    4. IV.D. Performance
    5. IV.E. Standards Compliance
    6. IV.F. Stress
    7. IV.G. Regression
    8. IV.H. Ship Test
    9. IV.I. Installation Documentation
    10. IV.J. Installation/Configuration Test
    11. IV.K. Reliability, Availability, and Serviceability
    12. IV.L. Usability
  8. V. Quality Goals
    1. V.A. Goals
    2. V.B. Measurements
  9. VI. Status Information
    1. VII. Testcase Descriptions
      1. GUI Testcase Descriptions
    2. VII.A. Naming Conventions
    3. VII.B. Testcase Location
  10. VIII. Functional Coverage Matrix
  11. IX. Approval Criteria

Document Control

Maintained online at "DevCorner/TestPlan". URL is: http://wiki.linux-ha.org/DevCorner/TestPlan, or http://linux-ha.org/DevCorner/TestPlan.

  • Retention Level: Valid until updated online. See wiki for current information.

    Initial Author: AlanRobertson

    • Distribution List
      • Distributed to wiki page subscribers on every update.
        • Subscribers must include Stephanie Glass. Distribution of new versions is automatic via Wiki change tracking system, including facilities for comparing any two versions.

Change Control

Maintained automatically by Wiki change tracking system.

I. Introduction

I.A. References/Related Documents

I.B. Line Item

II. Test Plan Overview

The test plan consists running and evaluating the results from of a set of automated tests which are shipped with the Linux-HA software, plus a set of manually performed GUI tests, as docmented in section VII.

There are two suites of tests which ship with Linux-HA:

Each of these tests is simple to run, and simple to evaluate. In addition, BasicSanityCheck does not require any special setup to make it work correctly and runs very quickly (in approximately a minute). cts, on the other hand requires a significant amount of setup before running it, and normally runs for many hours.

We do not yet have a set of automated GUI tests, so they are (for the time being) manual, and described in section 7.

III. Assumptions and Dependencies

III.A. Additional Program Products

III.B. Hardware

BasicSanityCheck can run on any single Linux machine without any special preparation or architectural constraints.

cts, on the other hand, requires three machines to run it on. The tests are controlled from the test monitor machine, and they are run on the other two machines. The Linux-HA package must be installed (in a consistent version) on all three machines. Detailed information about setting up cts is supplied by the cts/README file.

It is highly desirable that the clocks on all the test machines be closely synchronized in order to track down any errors easily.

III.C. General

cts executes in the context of a particular configuration, but does not create different configurations. As a result, it is often desirable to run it once with each of several possible configurations. Most often, the main variant used in the configurations is the value of the auto_failback directive. If two or more configurations are to be considered, one should have auto_failback on and one with it off.

III.D. History

IV. Test Goals and Objectives

The BasicSanityCheck software verifies that many of the major paths in heartbeat andsome associated programs are exercised. This is useful to find problems where the linux-HA software might not be running for reason of some significant platform-related bug.

The cts package is a suite of random tests which eventually run the system through many possible configurations and thoroughly test a configuration with a random sequence of tests chosen from its battery of tests.

IV.A. Hardware and Software Configuration

In order to do any testing, the heartbeat, heartbeat-pils, and heartbeat-stonith packages must all be installed with the version which is to be tested. For the cts tests this means on all three machines. For the BasicSanityCheck test, this means only the single machine to be tested.

Software configuration for the cts tests is complex and is described in detail in the cts README file. This README file can be found at /usr/lib/heartbeat/cts/README. The latest greatest version of the README (which might not match the version you're running) can be found in CVS.

The GUI is a program which can be run on any Linux system with the proper set of libraries installed. It is X-Windows based, so that tester must have access to an X-Windows server to view the results on.

IV.B. Test Approach and Methodology

IV.C. System Operation

??

IV.D. Performance

N/A

IV.E. Standards Compliance

N/A

IV.F. Stress

The CTS tests create significant system stress.

IV.G. Regression

Both tests perform a version of regression testing. The GUI regression tests are described in section VII.

IV.H. Ship Test

N/A

IV.I. Installation Documentation

??

IV.J. Installation/Configuration Test

??

IV.K. Reliability, Availability, and Serviceability

??

IV.L. Usability

N/A

V. Quality Goals

V.A. Goals

All tests should pass with zero errors when properly configured.

V.B. Measurements

Run the tests. If any cts tests fail when a reliable message transport is configured for syslog as described in the README, then this is a bug. If it is configured with UDP transport for syslog, it is possible that certain messages might get lost in a busy networking environment. Then the logs on the original machines can be examined to see if the missing message was actually issued but is missing. This is very difficult unless the system clocks are closely synchronized.

VI. Status Information

VII. Testcase Descriptions

Each CTS test case is described in the file CTStests.py. Each test is a class definition similar to this one:

  • class FlipTest(CTSTest):

The tests which are currently in the test suite are described in the CTS page.

This tests DRBD to see if it is maintaining proper integrity of the disk data. This requires that you configure DRBD into your configuration. If you do not, then it will not be run.

GUI Testcase Descriptions

We are developing a set of testcases for manually tests of GUI. These testcases should cover most functions of GUI. GuiTest

VII.A. Naming Conventions

N/A

VII.B. Testcase Location

The BasicSanityCheck script can be found at /usr/lib/heartbeat/BasicSanityCheck. The cts package is located in the /usr/lib/heartbeat/cts directory.

VIII. Functional Coverage Matrix

IX. Approval Criteria