I want to discuss here some essential characteristics of Quality Management (Quality Assurance, Quality Control) as it applies to IT projects - especially software development projects. It is not my intention to go into standards and other formalities that exist in the world of Quality Management.
The term "Software Engineering" was coined to emphasise the parallel between software construction and civil (physical) construction. It is meant in turn to emphasise the systematic, structured and modular approach to software development. Methodologies relating to software development are based on this general idea: the idea that the principles applying to the building of a bridge (clear structured approach and phasing, modular design, use of standard or custom components) are also applicable to the development of a software application. Concepts of software quality have also evolved against this background.
The application of the structured, modular approach to software design and development has been of undoubted real or potential benefit, especially in its current guise of Object Orientation (see my paper on this subject). But what about the application of quality management to software engineering? Here I have some critical remarks.
The idea that assigned tasks are not accepted as complete only on the basis that the person perfoming the task says it is complete, but rather only after appropriate checks or inspection have been carried out, is clearly sensible and desirable. Specifically, the following elements form the core of this approach:
(etc.) All this sounds fine in theory - but tends to be never as easy as it sounds in practice. Well, nothing ever is, you might object. But I mean that the problem with applying these principles to project-based software development is such that the underlying principles tend to be seriously weakened. The problem in essence concerns the difficulty of formulating sensible quality criteria. It is in practice very difficult to formulate quality criteria that are not either trivial or general and vague. By "trivial" I mean things like document layout standards, which, whilst also important in their way, are also mere formalities which are not directly relevant to the intrinsic quality of the product. With "general and vague" I have in mind possibly very important principles which nevertheless can be judged only subjectively (e.g. "loosely coupled modules", "simple interfaces"). Why is this the case?
First of all, the intangible nature of software and the creativity involved in its production mean that the concept of a 'defect' is a somewhat softer concept than with physical components. More specifically, the project-based software production process involves a number of preparatory steps, the results of which are not directly a part of the end-product, and we attempt to catch deficiencies in these intermediate products. A functional defect in a software system is often clear enough when the software comes to be tested and used - however, the quality management is very much orientated towards trying to catch defects at an earlier stage in the process. At these earlier stages, defects may be not noticeable or are much 'softer' in nature.
Secondly, the software development project is not a production line in which quality criteria can be based on a concept of repeatability. The project is one-off and largely unique. The techniques used probably will not be new, but a unique functional system must be produced. Specific quality requirements (other than that the user and system requirements must be met) are then difficult to find.
Thirdly, following largely from the second point, the concept of 'Defect Trend Analysis' is unlikely to have any meaning except possibly for an exceptionally large project. Defect Trend Analysis is in principle a useful tool for finding deficiencies in the process - but there has to be a sufficient statistical basis (repetition) for this to work, and the typical development project does not provide this. (It might be applied in the context of more than one project, but then there would need to be sufficient management and administrative continuity.) The inability to apply this tool seems to illustrate the overall problem relating to one-off projects. Each project begins in many respects without precedent.
It is important to recognise these problems and limitations because they lead to a certain ineffectiveness of quality procedures and this in turn leads to a lack of motivation to seriously apply such procedures. This often results in quality procedures being regarded as an obligatory but not useful formality, which really leads to wasted effort. Does it then also result in reduced quality? Moot point! If the perception is that neglect of quality procedures actually does not have an adverse affect on quality, then IT professionals are surely going to be demotivated in their application of such procedures. Partly, though, it has to do with risk. There is no guarantee that the application of quality procedures will increase the quality of the end-product or that their neglect will result in deficiencies. But not having good quality procedures will constitute a risk - and actually, a large risk in most cases.
This latter point - risk management - is actually at the core of quality management as applied to a software development project. A large amount of effort (and hence costs) in such projects goes into the preparatory or intermediate products (specifications, design documents and the like). It is clearly desirable therefore to detect any defects or flaws in these products at as early a stage as possible.
So what is the conclusion? For the reason just stated, it is clear that software development (and similar) projects should have the safeguard of good quality procedures built in. However, one must be careful not to overdo it, by for instance, formulating a lot of detailed but artificial quality criteria, or applying the same thoroughness unnecessarily at different levels (low-level and higher-level products). One should formulate quality requirements appropriate to the product and then derive realistic quality criteria. In many cases, this will mean using rather general criteria (general criteria are better than artificially detailed criteria). This inevitably means in turn that the assessment of some products will be based to an important degree on subjective judgement, but this judgement should of course be based on the relevant experience.
This last point is important: it means accepting that simple black-and-white criteria which can be ticked off on a checklist are not always possible and that a proper evaluation of an intermediate product may involve the judgement of an experienced practicioner. And in accepting this, there may be cultural obstacles to overcome!
By way of a slightly different angle on the same material, it's worth pointing out that quality control procedures should be expected always to cause a certain tension in the work environment. There are always those who seem to pay lip service to Quality Management, but when applied to themselves seem to think they form an exception or treat it as a formality which is not expected to cause them any grief. If many people take this attitude, then the Quality System might as well be discarded. And of course, what always exacerbates this problem is the pressure to meet deadlines. Have you ever been asked to QC a document 30 minutes or a half-day before it is due at the client? Clearly, this cannot be taken seriously as constituting meaningful Quality Control.
It goes without saying that quality procedures should be planned in. It seems to me that the assignment of tasks or work packages should be accompanied by a clear indication of what the quality requirements are, including how QC is to be done. In many cases, it is worth discussing these explicitly with the person carrying out the task. This should include making clear that any foreseen but previously unexpected time constraints should be signalled and discussed as soon as possible. Also, if there are any other doubts as to whether quality requirements will be met (whether because the requirements or criteria are unclear, or for any other reason), then these too should be discussed with the project manager or the quality controller or both. Early interaction with the quality controller may also have the advantage that the final formal control is reduced to a quick formality (whilst not sacrificing quality requirements).
A specific problem that often arises concerns the task that is performed at the client site and in close interaction with, or even with co-operation of, the client. If contractual conditions make clear that the resulting product can be accepted by the client as meeting his requirement, then this is no problem. If this is not the case, a formal responsibility rests with the supplier, whose management then naturally has an interest in first making sure that the product appears to meet the client's requirements. Fulfillment of this interest will then be met by an internal QC procedure. The problem arises because the person preparing the report will understandably believe that the report already meets all requirements (the client has probably already seen it 'unofficially' in its 'final' form) and is not likely to attach much value to the comments of an internal quality controller who doesn't have the intimate understanding of the client problem and environment. This situation should be recognised and a practical solution for it sought. Perhaps instead of a separate supplier control, a procedure can be agreed whereby the supplier's quality controller contacts the client in order to independently verify that the client is satisfied. Or the supplier's quality controller could perform a control but, instead of discussing comments with the producer alone, discuss them also with the client. It is in any case important that, if the client has already seen the draft product, any further changes as a result of supplier QC are discussed with him (and then preferrably not by an embarrassed producer having to explain why the 'final' report has changed).
It is of course important that a good relationship exists between producer and quality controller. The producer should not be over-sensitive to criticism! And the controller should be constructive and helpful in his or her comments. Concerning reports, one can perhaps recognise three levels of comment: detailed, medium level and high level, where the medium level will be easiest to 'swallow' and the other two have potential for conflict.
But at all three levels, judgement and discussion are necessary. And then the question of culture arises again. Can the quality controller be taken to have sufficient expertise and general sound judgement? And, if so, does the producer recognise and accept this? If it is not practical to always have a sufficiently 'heavy' quality controller involved, is there an acceptable arbitration procedure? This may sound exaggerated, but is worth considering if we are serious about quality and want to make the quality system work.