I've been rather silent about Taskotron progress as of late - mostly because we're still not ready to start accepting tasks from people. Despite the lack of noise, we have been making forward progress and are still on track to support more types of testing this year.
These updates will start being more regular as we get closer to the current end goal of enabling contributers to write their own tasks without any assistance on our part.
Recent Features
ExecDB
ExecDB is something that we deployed recently and while it doesn't affect what's run or reported publicly, it does enable us to start keeping better track of a task's execution lifecycle.
Each entry in ExecDB represents the entire lifecycle of a task from triggering to execution and reporting in ResultsDB. This data is accessible via a single unique ID instead of relying on the tuple of (buildmaster, builder, buildid) from buildbot which is unique as long as nothing changes but isn't very helpful from a system-wide UUID.
Task Artifacts
Task artifacts are another feature that seems small on the surface but is pretty much required before we can have tasks that output something other than TAP or text on stdout/stderr.
The general idea is that any output from a task is archived and hosted from a central location. This can be log files, html-ified log files that are easier for a human to parse than the current text-based logs, pictures, images or just about anything else that could be generated in the course of running a task.
At the moment, upgradepath is the only task that is using artifacts much. Looking at an example upgradepath run, we see the following directory structure:
task_output/ FEDORA-2015-7306.log FEDORA-2015-8565.log FEDORA-2015-8620.log FEDORA-2015-8849.log FEDORA-2015-9048.log FEDORA-2015-9063.log FEDORA-2015-9285.log dos2unix-7.2.2-1.fc22.log dovecot-1:2.2.18-2.fc22.log groonga-5.0.3-1.fc22.log libinput-0.15.0-4.fc22.log libtiff-4.0.3-20.fc22.log libunicap-0.9.12-15.fc22.log nss-3.19.1-1.0.fc22.log nss-softokn-3.19.1-1.0.fc22.log nss-util-3.19.1-1.0.fc22.log taskotron.log
Unsurprisingly, the taskotron.log entry is the log file generated during that execution but that's secondary to the actual artifacts contained in the task_output directory. In the case of upgradepath, each update tested is split off into a separate log file for easier parsing and linking to specific updates instead of just saying "here's the whole log, have fun reading through all of it to find what's releveant to you".
We still need to update how we're reporting results to actualy use these more specific links but that should come along with our upcoming migration from reporting results via bodhi1 comments to fedmsg emission.
Near-term Roadmap
We're currently working on two features that should be done in the not-so-distant future: disposable clients and results via fedmsg.
Disposable Clients
The next big thing that we've been working on is disposable clients which may not sound like much but it's one of the most critical bits needed before we can start running semi-arbitrary tasks and something we've wanted for years :)
Production Taskotron is currently set up with several clients deployed as long-running VMs. This works fine when we're running the same non-destructive tasks over and over again but becomes problematic when we start talking about tasks that:
- May render the client unbootable
- Have not been extensively tested in Taskotron
- Need to install task-specific packages that aren't generally needed for anything else
Unfortunately, this pretty much covers the entire spectrum of tasks that we want to start running.
Our solution to this is to start running tasks in short-lived VMs to guarantee that all tasks will start in a clean, stable, known-working state. By spawning a new VM before a task starts and destroying the VM when that task ends, whatever the task does, it cannot affect tasks executed at a later time.
Once we have disposable clients complete, deployed and working well, we can get started on the other, smaller bits needed to start doing dist-git style checks and Fedora contributors can start submitting checks that run on specific packages or just about any kind of fedmsg that's emitted.
Results via Fedmsg
Speaking of fedmsgs, the other thing that we've been working on is support for result notifications through fedmsg (and indirectly through Fedora Notifications).
One of the most difficult parts of this work is making sure we do it right - emit enough messages at the right times. In particular, making sure that we don't spam people to death with depcheck and upgradepath notifications. Due to the way that checks like depcheck and upgradepath work, they create far more results than show up in Bodhi comments and we need to be careful about making sure the right results end up as notifications for maintainers.
I expect this to land before Fedora 23 testing gets into full swing.
Longer-Term Roadmap
The main long term goal that we're working towards right now is what we're calling "dist-git style tasks". The main idea is to enable Fedora maintainers to create and maintain checks (or any kind of task that can be done in taskotron) for their packages by simply pushing bits to a git repository much like how dist-git works for package changes. The scheduling, execution and result reporting for those checks will happen automagically whenever the associated package is built.
There are dependencies for this and plenty of details that need to be worked out before we'll be able to offer dist-git style tasks but we'll be talking about it more as we get those details figured out.