Users can receive notifications containing information about submitted test jobs and the accompanying results by using settings in job definition yaml. Users can also compare the job results within notifications against previous results which satisfy certain conditions.
The basic setup of the notify
block in job definitions will have
criteria, verbosity, recipients and compare blocks.
The basic email template includes job details, device details and results,
based on the level of verbosity (verbose
, quiet
, status-only
).
verbose level job information will only be included if the job finished as
complete or incomplete, not when the job was canceled.
The method and destination of each notification can be set for each recipient. Currently, two notification methods are supported, email and IRC.
Recipients can be specified using the LAVA username or in full.
If the recipients section is omitted in the notify block, the system will send an email to the job submitter only, provided the criteria is satisfied, and there is no callbacks section.
In addition to sending email and IRC messages to recipients, the system can send multiple URL callback actions. This will do a GET or POST request to the specified URL in the callbacks subsection. This can be used to trigger some action remotely. If a callback uses the POST request, the system will attach job data as described below. The callbacks section supports list of the following options:
Content-Type:
application/json
header.Example callback usage:
notify: criteria: status: finished callbacks: - url: https://my-url/callback method: GET - url: https://remote-site/callback/lava?status={STATUS}&status_string={STATUS_STRING} method: POST dataset: all token: ci-callback content-type: json
The job data can also be retrieved using the REST API (which supports authentication). For example:
$ wget -O job_data.gz http://localhost/scheduler/job/2126/job_data
$ wget -O job_data.gz "http://localhost/scheduler/job/2127/job_data?user=neil&token=22yj3ls...."
Returns a gzip file containing the job data as JSON.
Note
Only test jobs which are configured to use the notification callback will create notification callback data for later retrieval. Other jobs will generate a 404 error.
LAVA users can configure the IRC settings
or email address in their own
profile data in the instance. This allows the recipient to be specified using
only the LAVA username.
If the user has not configured their own profile data, the recipient details must be specified in full.
Examples for user vs manual addressing:
notify: recipients: - to: method: irc user: neil.williams - to: method: irc handle: codehelp server: irc.freenode.net - to: method: email user: neil.williams - to: method: email email: neil.williams@linaro.org
IRC and email notifications use different templates since emails allow for more
verbosity, so some options which are present in the notify
block for email
recipients do not apply for IRC recipients, like comparing options and
verbosity.
Users can use notifications to compare the job status against the last few jobs and also compare job results, like test case/suites difference. The notification will report if there are new test cases compared to previous test jobs, if there are some missing and how the said results differ.
Which test jobs the current one is compared to is determined by the LAVA Queries system, where those previous jobs will be taken from the query specified in the notification block. Custom queries (Query by URL) can also be specified.
Blacklisting test cases is also an option. Ignoring some of the test cases in the comparing section of the notification is possible through an option in the notification block.
Here are some comparing setup examples from test definition excerpts:
notify: recipients: - to: method: email user: neil.williams criteria: status: finished verbosity: verbose compare: query: name: query_name username: query_owner blacklist: [lava, singlenode-advanced]
notify: recipients: - to: method: email user: neil.williams criteria: status: finished verbosity: verbose compare: query: entity: testjob conditions: submitter: stevan.radakovic requested_device_type: qemu blacklist: [singlenode-advanced]