Table Of Contents
Use Case Four - Iterating through devices using roles¶
Writing general-purpose test definitions¶
How to test that the test definition assumptions are valid.
Requirements¶
- Multi-Node job with at least two devices with usable network interfaces.
- Run operations on each device with a specified role.
- Scale the tests as the group size changes.
Background¶
YAML test definition files are commonly reused between jobs and scalable tests need to cope with changes in the size of the MultiNode job. Tests should use iteration whenever any kind of comparative performance, saturation or load measurements are required. This allows someone to later amend the JSON to increase the number of clients per server with the valid expectation that all clients will operate equally and that the test definition will not fail when it finds six clients instead of one or two.
It is particularly important that test writers should use iteration whenever any kind of comparative performance, saturation testing or load measurements are required. These are just the kind of jobs where someone will later modify the JSON to increase the number of clients per server, expecting all clients to operate equally and for the YAML to not care that it suddenly has six clients rather than one or two.
Recommendations¶
Test the defined roles¶
Always test that any role required by the YAML does exist in the MultiNode job.:
- lava-test-case check-server-role --shell lava-group server
- lava-test-case check-slave-role --shell lava-group slave
This test will fail (lava-group returns non-zero) if the specified role has not been defined for this MultiNode job.
Iterate over each role¶
#!/bin/sh
for role in `lava-role list`; do
echo $role
done
Iterate over each device with a specified role¶
Using set -e will allow the script to fail if the requested role is not defined:
#!/bin/sh
set -e
for device in `lava-group backend`; do
echo $device
done
Iterate over all devices by role¶
#!/bin/sh
for role in `lava-role list`; do
if [ "${role}" = "exception" ]; then
continue
fi
for device in `lava-group $role`; do
echo $device
done
done
Clearly indicate test definitions with fixed group sizes.¶
If the test definition assumes that the group is a particular size or a particular group composition, make this clear to other test writers. To make it obvious when test writers are editing JSON, rename the test definition YAML file to include the word static. To make it obvious to test writers are viewing result bundles and job lists, include the same indication in the test definition name.
Creating an alias in /etc/hosts based on the role¶
lava-network has optional support via the alias-hosts command to create an alias for each device based on the role of that device. The device names are unique and will vary on each run but are guaranteed to be usable as hostnames. The alias only uses the alphanumeric characters within the role - [a-z0-9] - and appends a two digit count suffix for each device assigned that role.
If the device fails to return a fully qualified hostname, lava-network uses the unqualified hostname (which is the same as the device name). In this example, staging-kvm02 was the only device in the group to return a value for hostname -f:
10.1.1.2 staging-kvm01 slave01
10.1.1.6 staging-kvm02.localdomain box01
10.1.1.2 staging-kvm03 slave02
10.1.1.3 staging-kvm04 slave03
Caution
Resist the temptation to hardcode aliases into scripts
YAML and the scripts called by the YAML should not make assumptions about the group size or group constitution as those are defined in the JSON. The same YAML file should be to be usable in multiple groups of varying sizes. e.g. if the test definition relies on two roles, server and client, the YAML and the scripts called from the YAML must not fail if there is no server03 or client7 - equally the YAML must still test server08 and client12 if those exist.