This site best when viewed with a modern standards-compliant browser. We recommend Firefox Get Firefox!.

Linux-HA project logo
Providing Open Source High-Availability Software for Linux and other OSes since 1999.

USA Flag UK Flag

Japanese Flag

Homepage

About Us

Contact Us

Legal Info

How To Contribute

Security Issues

This web page is no longer maintained. Information presented here exists only to avoid breaking historical links.
The Project stays maintained, and lives on: see the Linux-HA Reference Documentation.
To get rid of this notice, you may want to browse the old wiki instead.

1 February 2010 Hearbeat 3.0.2 released see the Release Notes

18 January 2009 Pacemaker 1.0.7 released see the Release Notes

16 November 2009 LINBIT new Heartbeat Steward see the Announcement

Last site update:
2018-12-19 16:46:02

NetStrings Implementation

NetStrings were invented by Bernstein as a self-describing way to safely transmit data over the internet. They are used by the HeartbeatProgram as its newest OnTheWireDataFormat.

netstring wire format is item 55 the linuxHA TODO list (http://www.linux-ha.org/HATodo.html).

1. general introduction to netstring:

  • A netstring is a self-delimiting encoding of a string. Netstrings are very easy to generate and to parse. Any string of 8-bit bytes maybe encoded as [len]":"[string]",". Here [string] is the string and [len] is a nonemptry sequence of ASCII digits giving the length of [string]

    in decimal, For example, the string "hello world" is encoded as <31 32 3a 68 65 6c 6c 6f 20 77 6f 72 6c 64 21 2c>, i.e., "12:hello world!,".

    A more detailed introduction is http://cr.yp.to/proto/netstrings.txt

2. why netstring in linuxHA

  • There are couple of reasons we want to use netstring in linuxHA
  • In netstring it gives the length of the string at the beginning, therefore a program can allocate a buffer with exact size. This prevents possible bufferover flow attacks.
  • To transmit binary data and to implement other cool features. Lots of application want to send/receive binary data. However, with string only we will have to do binary-to-string and string-to-binary conversions. With netstring it is trivial to send binary data. The argument is the same for other features, e.g., recursive message encoding.

3. netstring implementation details

  • struct ha_msg modifications:

    A message is essencially many <name,value> pairs. Previously names and values can only be null ended strings. Now values can be binary. An integer array is added in struct ha_msg to represent each <name, value> type, and another integer is added to represent netstring length: struct ha_msg {

    • ..... size_t netstringlen;
    • ... int * types;
    };
  • The type for each <name, value> pair can be FT_STRING, FT_BINARY or FT_STRUCT. FT_STRING: value is a normal string. FT_BINARY: value is binary data. FT_STRUCT: value is a netstring of a child message.

  • to generate a netstring:

    Each <name, value> pair is encoded like the following [namelen]":"[name]","[typelen]":"[type]","[valuelen]":"[value]"," since type is one digit, [typelen] is always 1. [value] can be a normal string, binary data or a netstring of a child message. an authentication number is computed against the netstring containing all the

    <name, value> pairs. That number is attached after the last <name, value> pair. Finally there is a MSG_START_NETSTRING at the beginning, and MSG_END_NETSTRING at the end of the netstring. The whole netstring looks like the following: MSG_START_NETSTRING[name-value-pairs-encoding][authentication]MSG_END_NETSTRING note: a client's message is encoded without authencation because a client does not have access to the heartbeat's authentication computation function.

  • to decode a netstring: This is a reverse work to generating a netstring. First MSG_START_NETSTRING is removed, then it works like a state machine: if the next one is a name-value pair, go to the beginning; if the next one is authentication, try to verify it, then exit successfully or fail, depending on the verification; if the next one is MSG_START_NETSTRING, exit successfully or fail, depending on whether this message is required to authenticate. A successful exit returns an ha_msg, otherwise it returns NULL.
  • to compute netstring length if containing child messages: Because of child messages the netstrng length for

    a message must be sum of all its child messages and its local <name,value> pair netstring length. This length is computed in function get_netstringlen()

4. string implementation of same functionanlitis

  • We need have string implemenation of the same functionalities for back compatibility.
  • string implementation of binary data: Binary data is converted to base64 ASCII string when generating a string for a ha_msg and converted back when decoding a string to an ha_msg.
  • string implementation of recursive messages: A value for a child message is the string generated from the child message when generating a string from a ha_msg containing child messages. Since a child message string contains new lines and it will mix with its parent message string's new lines, a child message's new line is converted to some special symbol. Which Special symbol to convert to depends on the depth of the child message. All special symbols are converted to new lines in decoding a string to an ha_msg.