We've been working on AutoQA 0.5.0 for a while and it has finally been released and deployed! One of the features that we've been working on for this release is reducing the amount of email sent to package maintainers through Bodhi.
Currently, every test in AutoQA creates a comment in Bodhi which in turn sends an email to the package maintainer. This produces a lot of unwanted email noise that annoys maintainers and dilutes the meaning of our results.
With AutoQA 0.5.0, we're taking advantage of a Bodhi API change in 0.8.0 that allows comments without sending an email in order to reduce the spam created by AutoQA. Specifically, emails sent by AutoQA through Bodhi are currently limited to:
- An email is sent ONLY if a test failed and ONLY when all expected tests for the current repository (testing, stable) are completed
- An email is sent if the UPDATE changes state
- An update's current state is determined by the tests that have been run on it. If all of the tests have passed, the state is PASSED. If any of the tests have failed, the state is FAILED. If any of the tests have not completed, the state is INCOMPLETE
- If one test transitions from FAIL to PASS but another test is still FAIL, there is no state change and no email will be sent.
- More detailed examples can be found on the AutoQA wiki.
This will dramatically reduce the number of emails sent as a result of AutoQA tests and increase the meaning of the emails that are sent.
As always, if you have comments, suggestions or find bugs - let us know. You can find us in #fedora-qa on freenode or on the autoqa-devel mailing list.