See also
To run the unit tests, a range of dependencies need to be installed:
$ sudo apt -y install lava-dev
To reduce the number of unit tests which are skipped, a number of other
packages are listed as recommended by lava-dev and these should be
installed as well:
$ sudo apt -y install lxc u-boot-tools tftpd-hpa telnet nfs-kernel-server img2simg simg2img
Important
For security reasons, each installation of
lava-server sets the permissions of
/etc/lava-server/instance.conf to only be readable by the
lavaserver user and users in the lavaserver group.
(Some older instances may have a different username for this user
and group - check the value of LAVA_DB_USER in
/etc/lava-server/instance.conf using sudo.)
To run the unit tests, the user running the unit tests needs to be
in the lavaserver group. For example:
$ sudo adduser <username> lavaserver
If you are an administrator, you may think the previous link sent you to the wrong section. However, administrators need to understand how device-type templates operate and how the template engine will use the template to be able to make changes.
Important
Device type templates are more than configuration files
- the templates are processed as source code at runtime. Anyone
making changes to a device-type .jinja2 template file must
understand the basics of how to test templates using the same tools
as developers.
Jinja2 provides a powerful templating engine. Templates in LAVA use several standard programming concepts:
In addition, LAVA templates need to always render to valid YAML. It
is this YAML which is sent to the worker as device.yaml. The worker
does not handle the templates. All operations are done on the master.
The simplest check is to render the new template to YAML and check that it contains the expected commands. As with test job files, there are common YAML errors which can block the use of new templates.
See also
lava-server manage device-dictionary --hostname <HOSTNAME> --review
All templates are checked for basic syntax and output using:
$ python3 -m unittest -vcf lava_scheduler_app.tests.test_base_templates.TestBaseTemplates.test_all_templates
A more rigorous test is to use the dedicated unit test which does
not require lava-server to be installed, i.e. it does not
require a database to be configured. This test can be run directly from
a git checkout of lava-server with a few basic python packages
installed (including python-jinja2).
Individual templates have their own unit tests to test for specific elements of the rendered device configuration.
The number of unit tests and templates has increased, so there are dedicated unit test files for particular types of template unit tests:
$ python3 -m unittest -vcf lava_scheduler_app.tests.test_fastboot_templates
$ python3 -m unittest -vcf lava_scheduler_app.tests.test_grub_templates
$ python3 -m unittest -vcf lava_scheduler_app.tests.test_qemu_templates
$ python3 -m unittest -vcf lava_scheduler_app.tests.test_uboot_templates
Most changes to device-type templates take effect immediately - as
soon as the file is changed in
/etc/lava-server/dispatcher-config/device-types/ the next testjob
for that device-type will use the output of that template. Always test
your templates locally before deploying the template to the master.
(Test jobs which have already started are not affected by template
changes.)
This cannot be stressed enough. ALL admins need to keep device-type templates in some form of version control. The template files are code and admins will need to be able to upgrade templates when packages are upgraded and when devices need to implement new support.
As code, device-type templates need to develop alongside the rest of the codebase. The best way to maintain support is to Contributing Upstream so that new features can be tested against your templates and new releases can automatically include updates to your templates.
Some individual device files exist in the codebase in
lava_scheduler_app/tests/devices but these are only for use in the
existing unit tests. There is no need to contribute individual device
dictionaries unless there are new unit tests which use those device
dictionaries.
Note
LAVA is developed using Debian packaging to ensure that
daemons and system-wide configuration is correctly updated with
changes in the codebase. There is no official support for pypi or python
virtual environments or installing directly from a git directory.
python-setuptools is used but only with sdist to create the
tarballs to be used for the Debian packaging, not for install.
Some dependencies of LAVA are not available with pypi, for example
python-guestfs would need installing using another method.
See also
Developing LAVA on Debian and a summary of the Debian LAVA team activity
Developers can update the installed code on their own systems manually (by copying files into the system paths) and/or use symlinks where appropriate but changes need to be tested in a system which is deployed using the Developer package build before being proposed for review. All changes must also pass all the unit tests, unless those tests are already allowed to be skipped using unittest decorators.
Mixing the use of python code in /usr/local/lib and /usr/lib on
a single system is known to cause spurious errors and will only
waste your development time. Be very careful when copying files and
when using symlinks. If in doubt, remove /usr/local/lib/python*
and ~/.local/lib/python* then build a local developer
package and install it.
If your change introduces a dependency on a new python module, always
ensure that this module is available in Debian by searching the Debian
package lists. If the
module exists but is not in the current stable release of Debian, it
can be backported but be aware that this will delay testing and
acceptance of your change. It is expressly not acceptable to add a
dependency on a python module which is only available using pypi or
pip install. Introducing such a module to Debian can involve a
large amount of work - talk to us before
spending time on code which relies on such modules or which relies on
newer versions of the modules than are currently available in Debian
testing.
The dependencies required by LAVA are tracked using the
./share/requires.py script which is also available in the
lava-dev package as /usr/share/lava-server/requires.py.
Merge requests which need extra modules which already exist in Debian
can add the relevant information to the share/requirements/debian
files.
Note
For the CI to pass, the extra module(s) must be available for stable and testing. Pay particular attention to the version available in buster and buster-backports. If a minimum version is required, this can be specified in the requirements, as long as that version or newer is available in buster-backports. talk to us if your change involves new files that may need changes in the packaging code. All CI tests must pass before any new code can be merged, including building working packages containing the new support.
All the code for the lava-server and lava-dispatcher support
exists in the single LAVA repository:
https://git.lavasoftware.org/lava/lava
Includes:
lava_scheduler_app
lava_results_app
lava_server
lava
lava_common
linaro_django_xmlrpc
lava_dispatcher
lava_test_shell
debian
See also
See also
The compatibility mechanism allows the lava-master daemon to prevent issues that would arise if the worker is running older software. A job with a lower compatibility may fail much, much later but this allows the job to fail early. In future, support is to be added for re-queuing such jobs.
Developers need to take note that in the code, compatibility should
reflect the removal of support for particular elements, similar to
handling a SONAME when developing in C. When parts of the submission
YAML are changed to no longer support fields previously used, then the
compatibility of the associated strategy class must be raised to one
more than the current highest compatibility in the lava-dispatcher
codebase. Compatibility does not need to be changed when adding new
classes or functionality. It remains a task for the admins to ensure
that the code is updated when new functionality is to be used on a
worker as this typically involves adding devices and other hardware.
Compatibility is calculated for each pipeline during parsing. Only if the pipeline uses classes with the higher compatibility will the master prevent the test job from executing. Therefore, test jobs using code which has not had a compatibility change will continue to execute even if the worker is running older software. Compatibility is not a guarantee that all workers are running latest code, it exists to let jobs fail early when those specific jobs would attempt to execute a code path which has been removed in the updated code.
The Jinja2 templates can be found in
lava_scheduler_app/tests/device-types in the lava-server
codebase. The reason for this is that all template changes are checked
in the unit-tests. When the package is installed, the device-types
directory is installed into
/etc/lava-server/dispatcher-config/device-types/. The contents of
lava_scheduler_app/tests/devices is ignored by the packaging, these
files exist solely to support the unit tests.
See also
Unit-tests and Testing the design for examples of how to run individual unit tests or all unit tests within a class or module.
Individual instances will each have their own locations for the device
dictionaries of real devices. To allow the unit tests to run, some
device dictionaries are exported into
lava_scheduler_app/tests/devices but there is no guarantee that
any of these would work with any real devices, even of the declared
device-type.
For example, the Cambridge lab stores each device dictionary in
git at https://git.linaro.org/lava/lava-lab.git and you can look at the
configuration of staging as a reference:
https://git.linaro.org/lava/lava-lab.git/tree/staging.validation.linaro.org/master-configs/staging-master.lavalab/lava-server/dispatcher-config/devices
Device dictionaries can also be downloaded from any LAVA instance using the XML-RPC call, without needing authentication:
server.scheduler.devices.get_dictionary(hostname)
The lava-dispatcher codebase also has local device configuration
files in order to support the dispatcher unit tests. These are not
Jinja2 format, these are YAML - the same YAML as would be sent to the
dispatcher by the relevant master after rendering the Jinja2 templates
on that master. There is no guarantee that any of the device-type
or device configurations in the lava-dispatcher codebase would work
with any real devices, even of the declared device-type.
The best way to protect your investment on LAVA is to contribute your changes back. This way you don’t have to maintain the changes you need by yourself, and you don’t run the risk of LAVA changed in a way that is incompatible with your changes.
Upstream uses Debian, see Developing LAVA on Debian for more information.
The LAVA Software Community Project uses GitLab for all development and planning for new features and concepts. Discussion happens on the Mailing lists.
Many older git commit messages within the LAVA codebase contain
references to JIRA issues as LAVA-123 etc., as the LAVA project
used to use Linaro’s JIRA instance to track issues. All references
like this can be appended to a basic URL to find the details of that
issue: https://projects.linaro.org/browse/. e.g. the addition of
this section on JIRA relates to LAVA-735 which can be viewed as
https://projects.linaro.org/browse/LAVA-735
If you have comments or questions about anything visible within the LAVA project, please subscribe to one of the mailing lists and ask your questions there.
The LAVA Software Community Project uses GitLab for all bugs, issues, feature requests, enhancements and problem reports. For more general questions and discussion, use the Mailing lists. It is often useful to discuss the full details of the problem on the lava-users mailing list before creating an issue in GitLab.
Note
The old Bugzilla and JIRA systems are both deprecated and reporting bugs in the old Bugzilla system will not be tracked by the LAVA team.
The LAVA software team use GitLab to manage contributions. For details, please read the LAVA Software Community Project Contribution Guide
See also
Release process and LAVA development and LAVA Software Community Project Contribution Guide for detailed information on running the unit tests and other static code analysis tools before submitting the review.
You are welcome to use the bug tracker of your chosen distribution. The maintainer for the packages in that distribution should have an account on https://git.lavasoftware.org/lava/lava to be able to forward bug reports and patches into the upstream LAVA systems.
GitHub has mirrors of the GitLab repository but merge requests need to be run through the GitLab CI tests. This can be done by changing the git remote of the GitHub branch and pushing to GitLab. GitHub users can create GitLab accounts on https://git.lavasoftware.org using their GitHub credentials.
See also