What is LAVA?
LAVA is the Linaro Automation and Validation Architecture.
LAVA is a continuous integration system for deploying operating
systems onto physical and virtual hardware for running tests.
Tests can be simple boot testing, bootloader testing and system
level testing, although extra hardware may be required for some
system tests. Results are tracked over time and data can be
exported for further analysis.
LAVA is a collection of participating components in an evolving
architecture. LAVA aims to make systematic, automatic and manual
quality control more approachable for projects of all sizes.
LAVA is designed for validation during development - testing whether
the code that engineers are producing “works”, in whatever sense that
means. Depending on context, this could be many things, for example:
- testing whether changes in the Linux kernel compile and boot
- testing whether the code produced by gcc is smaller or faster
- testing whether a kernel scheduler change reduces power consumption
for a certain workload
- etc.
LAVA is good for automated validation. LAVA tests the Linux kernel on
a range of supported boards every day. LAVA tests proposed android
changes in gerrit before they are landed, and does the same for other
projects like gcc. Linaro runs a central validation lab in Cambridge,
containing racks full of computers supplied by Linaro members and the
necessary infrastucture to control them (servers, serial console
servers, network switches etc.)
LAVA is good for providing developers with the ability to run customised
test on a variety of different types of hardware, some of which may be
difficult to obtain or integrate. Although LAVA has support for emulation
(based on QEMU), LAVA is best at providing test support for real hardware
devices.
LAVA is principally aimed at testing changes made by developers across
multiple hardware platforms to aid portability and encourage
multi-platform development. Systems which are already platform independent
or which have been optimised for production may not necessarily be able
to be tested in LAVA or may provide no overall gain.
Note
This overview document explains LAVA using
http://validation.linaro.org/ which is the official
production instance of LAVA hosted by Linaro. Where examples
reference validation.linaro.org, replace with the fully
qualified domain name of your LAVA instance.
What is LAVA not?
- LAVA is not a set of tests - it is infrastructure to enable
users to run their own tests. LAVA concentrates on providing a range
of deployment methods and a range of boot methods. Once the login is
complete, the test consists of whatever scripts the test writer
chooses to execute in that environment.
- LAVA is not a test lab - it is the software that can used in a
test lab to control test devices.
- LAVA is not a complete CI system -
it is software that can form part of a CI loop. LAVA supports data
extraction to make it easier to produce a frontend which is directly
relevant to particular groups of developers.
- LAVA is not a build farm - other tools need to be used to prepare
binaries which can be passed to the device using LAVA.
- LAVA is not a production test environment for hardware - LAVA is
focused on developers and may require changes to the device or the
software to enable automation. These changes are often unsuitable for
production units. LAVA also expects that most devices will remain available
for repeated testing rather than testing the software with a changing
set of hardware.
See also
Continuous Integration which covers how LAVA relates to
continuous integration (CI) and covers the consequences of what
LAVA can and cannot do with particular emphasis on how automation
itself can block some forms of testing.
Architecture
LAVA installations consist of two primary components - a master and a
worker. The master has all the code required to run a worker and can
support multiple remote workers to increase the number of devices
available on any one instance.
Elements of the master
- web interface - apache and uwsgi interfacing with django as well as
providing XML-RPC access and REST API.
- database - postgresql, local to the master with no external access.
- dispatcher-master daemon - controls messages from the master to the
worker(s) using ZMQ.
Elements of the slave
- lava-slave daemon - receives control messages from the master and
sends logs and results to the master, optionally uses authentication
and encryption using ZMQ.
- dispatcher - the lava-dispatch process, started by the
lava-slave when instructed to do so by the master. This process
manages all the operations on the device under test, according to the
job submission and device parameters sent by the master.
- device under test. Note that all the configuration for how the
dispatcher interacts with the device is sent from the server.
Features
- Automated validation - designed for automated processes to create,
submit and process results of test jobs to validate the development
process.
- Parallel scheduling - multiple test jobs run at the same time
across multiple devices.
- Multinode test jobs - test jobs can be run as a single group of
tests involving multiple devices.
- Hardware sharing - uncommon hardware is shared between disparate
groups to maximise usage
- Wide device coverage - a large number of types of device can be
supported with instances ranging from one to more than a hundred
devices available for test jobs.
- Data export for customisation - transform the data using custom
interfaces to make the validation output directly relevant to specific
teams.
- Privacy support - test jobs or types of device can be kept private
to selected groups, individuals or teams.
- Live result reporting - if a test job does fail, all results
up to the point of failure are retained.
- UNIX and Android test support - Test jobs can be run on systems
running various UNIX flavours or using the Android Debug Bridge to
interface with mobile devices.
- Complex network testing - reconfigurable networking across multiple
devices using multiple network interfaces.