Rejecting QA as Quality Assurance

Fri 24 May 2013 By Tim Category: post

This is something I've said a few times before but not so much in a formal sense. It plays into some stuff I've been working on as of late and I figured it was about time to write about it.

As far as I know, the QA in Fedora QA is supposed to be short for "Quality Assurance". I completely reject the concept of "Software Quality Assurance", especially in the context of Fedora QA. The idea that we can somehow assure users that software is of a certain level of quality implies that we have control over the quality of that software. This implication fallacious at best and irresponsible at worst.

An Example

Note: This is a completely contrived example, don't read too far into it or take the example as what I may or may not think of Nouveau, X or Dell. This is an extreme example that I have a very hard time seeing actually happen. The hyperbole is for illustrative purposes only.

Say we have a bug in nouveau that makes it impossible to install Fedora on 50% of all Dell laptops with Nvidia graphics. If the Nouveau devs didn't think they could fix the bug for 6-9 months or if they refused to fix the bug (say they thought it was something in X and the X devs should fix it but the X devs claimed it was a nouveau issue and we went into an eternal game of fingerpointing), what control do I have as a tester? Block the release of Fedora and keep re-testing until a fix magically falls out of the sky and makes everything better? Incessantly pester the Nouveau and/or X devs until the bug gets fixed? Sit on my thumbs and cry about it until <insert deity here> fixes the problem? No, I document the issue as best I can, warn users about the problem and hope that the issue with foo can either be worked around or fixed in the future.

At some point, a decision about whether to release with the bug, rollback to an earlier version which doesn't have the bug or any number of workaround which may exist at that point. However, that decision isn't strictly a testing or a QA issue.

The only position which could be thought of as "Quality Assurance" is the entity which controls development resources (usually the developer(s) or the management of said developer(s)). The developer is the only one who can fix the bug and the entity which controls the developers' time decides whether not a specific bug gets fixed and thus "assures" quality.

Holding the Flashlight

I think of it along the lines of a testing motto : "We hold the flashlight". If the software we're interested in is a house that is nice other than an unlit basement full of bugs, the developers are the exterminators and the testers are people who have flashlights. The exterminators could wander around in the dark basement trying to get rid of the bugs but without light, they're probably going to miss a bunch of bugs and will spend quite a bit of time in places that don't really have bugs. The people with flashlights could wander around the basement, trying to squish small bugs with their shoes but you can't get rid of the really problematic infestations that way and trying to squish all the bugs with only shoes will take forever, if that's even possible.

The best case scenario would be for the exterminators to team up with the flashlight people so that the flashlight people find bugs and hold the flashlight so that the exterminators can better see what they're doing and get rid of more bugs. (Yes, I know the analogy isn't perfect - they almost never are but that gets beyond the point I'm trying to make)

So, What is QA?

So if testers can't do "Quality Assurance", what is QA? I think of the QA in Fedora QA as being "Quality Assistance". Our job is to "hold the flashlight" and work towards making Fedora better. Does that mean that we're somehow lesser beings because we can't "assure" quality? No, not in the least - it's just a better description of what we, as testers, can do and how we can make Fedora better.

To take this a bit farther, I don't think that "Quality Assistance" is limited to just "testing" or running through validation matrices. I'm not saying that we should start trying to do everything, just that it might make sense to think a little more "outside the box" (please excuse the buzzword) for things that may not traditionally be within the realm of testing but could help improve the quality of Fedora.

In my mind, we're already doing this for the most part - look at the blocker tracking app and the whole blocker bug process. How many other "testing" organizations participate in and to a certain extent drive a process which more traditionally falls under the responsibility of project or program management?

Regardless of how we define QA or testing, the job is still the same: help make Fedora better.