Time for an overhaul?

Concerns over certification credibility are currently more rife than ever. From an auditor certification perspective, IRCA director Simon Feary suggests that the root cause can be found in the accreditation system

This article examines the causes behind the current concerns over the credibility of certification and, from an auditor certification perspective, suggests where the solutions lie. Few involved in the certification industry can have failed to note a change in recent times in the way the markets regard accredited certification.

The situation might be confused slightly by the differences in the way certification is perceived between markets that are mature and those that are developing. Look to the east and all appears rosy. But back in those regions where certification had its origins, a different picture is evident.

It is easy to identify positives. The number of accredited certificates worldwide – for EMSs and QMSs – at last count was well over 600,000. The growth in the five-year period to December 2003 was a very healthy 80 per cent. Certification is now global, with few if any regions of the world exempt from its influence. More standards are being accommodated within the certification infrastructure. On the face of things, it is a healthy scenario.

Lifting the lid

But there are negatives and these are of sufficient magnitude to cause concern. The growth in 2003 was supported almost entirely by two economies, China and Japan, while all the other majors were at best flat with some showing a retraction in percentage terms of double figures. One high-profile manufacturing industry, automotive, has declared no confidence in ‘mainstream’ certification – although maintaining its confidence in ISO 9001 – and has walked away to create its own sector-specific structure for accreditation and certification. Another major sector, the aerospace industry, now requires having auditors accompanied in order to ensure confidence in their findings.

China, a rare bright spot, has seen fit to impose some severe caveats in the way it allows certification to operate within its borders. While some argue their action is no more than a clever expression of ‘the Chinese way’, where legitimate competition is kept at arm’s length, others recognize it to be a reasonable response to damaging and out of control practices that are symptomatic of a malaise within the industry.

So what has caused the brakes to be applied so abruptly? Expressed in business jargon, diplomatically, the reasons are that demand has outstripped the ability of the infrastructure to control the quality of output.

Free markets, an over-abundance of providers and the voluntary nature of certification have allowed short-term commercial interests to exploit opportunities offered by naive markets and complicit or disinterested governments. But put another way, simply and bluntly, accreditation has failed to do its job adequately and the certification product has leaked credibility.

Accredited certification has a simple and coherent structure. This was outlined in the original declaration developed by the UK government in the early 1980s and remains valid today. Government delegates authority for control (control, not regulation – an important distinction that reflects the voluntary nature of this enterprise) at arm’s length to the accreditation body, one per country. The accreditation body has oversight of the certification bodies, ensuring rigour of quality and consistency of those certification bodies’ practices.

Shades of grey

It is simple in concept, but in practice it is not working well enough. Accreditation bodies, invariably monopolies within their own countries, are often accused of lacking the commercial and technical competence found within the certification bodies they are tasked with controlling.

Interpretations of standards differ from one accreditation body to the other, sometimes significantly, depending to an extent on the lobbying skills of the petitioning certification body, and often they vary within an accreditation body, depending on the extent of assessment competence of the accreditation personnel.

It also tends to be forgotten that, as with certification, accreditation is also a business operation. Accreditation is not disinterested financially, being dependent on revenues from those bodies it is controlling. And in some parts of the world, accreditation has overtly political drivers. All these serve to complicate and frustrate what should be a straightforward exercise in surveillance and control.

From the top

So where’s the solution? Apparently, it lies with the accreditation industry’s governing body, the International Accreditation Forum (IAF). Nothing is ever simple or speedy with governing bodies that are consensus-orientated and composed of a membership approaching 50 from across the very broad spectrum of economic situations and cultures.

But credit where credit is due, and taking a historical perspective, the IAF has to be regarded as making progress, from a standing start a few years ago, it has now gripped its brief and has shown competence at producing guidance on a range of issues, all geared toward best practice and harmonization. But it is slow, and its decision-making process subject to the inertia of consensus.

It has reacted with reasonable speed to the specific threat drawn attention to by ISO’s late secretary-general, Lawrence D Eicher. In late 2001, he raised the issue of ‘disreputable practices’ by some conformity assessment operators and indicated firmly that that the remedy lay with the conformity assessment community itself, and not with the writers of standards. ‘Police yourselves!’ was his message.

At the end of 2001, during the IAF plenary meeting in Kyoto, Japan, a proposal was made for the establishment of a multi-lateral task force comprising representatives of the IAF, ISO/TC 176 (the ISO technical committee responsible for the ISO 9000 family) and ISO/CASCO, the committee on conformity assessment. One of the objectives of the new group is to monitor real or potential threats to ISO 9001 posed by the use made of it in certification.

In May 2002, the first meeting of the group, known as the ISO 9000 Advisory Group (IAG) was held in Denver, Colorado, US. The group conducted a 'warts and all' review of the certification industry, its practices and various future scenarios and looked at ways to tackle problems identified. And by all accounts, the IAG is vigorously and seriously pursuing its brief.

Root cause: ISO 17024?

But from the perspective of auditor certification – and it is from this direction that this critique comes – there is unease. There is a fear that the focus of the IAF and other initiatives will be on the symptom, leaving the cause unaddressed. The culprit will be determined to be the auditor, or more precisely, the incompetent auditor and recommendations will lean toward addressing this aspect rather than structurally, where this article argues the problems originate.

The reasons for this unease lie with the considerable interest shown in ISO/IEC 17024:2003, ‘Conformity assessment – general requirements for bodies operating certification of persons’, for the approval of auditors. It is almost as if this is seen as the ‘what we’ve been waiting for’ solution. And that would be incongruous, as ISO/IEC 17024 implies a third-party determination of auditor competence, in other words, auditor certification, and auditor certification has historically been relegated to the periphery of accredited certification.

Although auditor certification provides evidence of competence, it has never gained more than an anecdotal respect from the accreditors. It is very definitely the poor relation. The accreditors and certification bodies have always argued that competence is a matter to be determined themselves and whether or not an auditor is certified is of no great relevance.

Some accreditation bodies accord it some status, others do not. This is understood by (most) auditor certification bodies, and accepted. While having no formal mandate to contribute represents something of an irritation, it does serve to keep auditor certification honest and focused on providing added value.

While it is understandable that there is interest in ISO/IEC 17024:2003 – it is a new and an international standard – the danger is that it will be seen as the answer: not just part of the answer, but the only answer. Already, one major auditor certification body is presenting it as exactly that.

The message this organization is promoting is that only accredited certified auditors are properly competent. All others, it says, are suspect and the current criteria and methods for evaluating auditors count for little. This is dangerous nonsense.

It is nonsense because there are thousands of extremely competent, certified auditors exercising their profession to good effect, and dangerous because it distracts from the root cause of the industry’s loss of credibility – the lack of effective control at the accreditor level.

Fixing the machine

The current mantra is ‘auditor competence’, but it should more properly be ‘audit competence’. To place all the ills at the foot of the individual in the field is disingenuous. The entire audit process, from the advertising literature, the sales pitch, the audit team competence, performance in the field to the certification decision are all valid factors in determining whether the resulting certification provides value to the organization and its customers, is consistent with the purpose for which certification was intended, or whether it is just commercial opportunism.

Auditors are just part of the certification process. If the process itself is not well controlled or is based on false premise, then it is unreasonable to expect the auditor to shoulder the blame.

Two issues give cause to be realistic about the impact that ISO/IEC 17024 and accredited, certified auditors will have on certification and both, unsurprisingly, relate to competence. The first is the way auditor certification is regarded by certification bodies. The majority, especially those that take a responsible, long-term view, see auditor certification as not much more than an indication of basic competence. For them, it represents an assurance that the auditor has a working knowledge of ISO 19011:2002, ‘Guidelines for quality and/or environmental management systems auditing’, and the ability to audit simple, generic processes.

Certification bodies accept that the real training and acquisition of competence comes via the certification bodies’ own monitoring, training and appraisal processes, where the auditor learns, amongst other things, the organization’s own practices.

A unique trade

These certification bodies make sure that no auditor is let loose on a client until he or she is able to demonstrate the required auditing competence and approach specific to that certification body. An expectation that all certification bodies will now regard accredited certified auditors as ‘off the shelf, ready to audit’ just because they are accredited is misguided and naive.

 

No doubt some will, in the same way that they do now with auditors qualified under the current system, but those who recognize that from a risk perspective, the performance of their auditors in the field is one of the most critical in terms of managing client relations, will continue to exercise close control, through training, performance monitoring and calibration. Using accredited auditors will not change that aspect.

The second issue is context-specific competence. Auditor certification provides an indication of generic auditing competence only. There are few if any auditor certification bodies that certify competence to audit within specific industry contexts to any great extent, and it is this context-specific aspect that has caused arguably the greatest negative reaction from client organizations.

This gives rise to the ‘our auditor didn’t know anything about my business/industry/processes’ protest. And unless there is a sea change in the attitude of certification bodies toward costs and they are prepared to absorb the additional cost burden associated with sector-specific auditor certification, this is one aspect of improving audit competence that auditor certification, accredited or otherwise, will not address. Determining competence requirements of a specific audit and matching these with individual auditors’ competences is a responsibility that lies entirely within the remit of those who manage the audit process. That it is too often neglected is neither an indictment of auditor competence, nor of auditor certification. It is a reflection that certification bodies are cutting corners and the accreditation process is not picking this up.

Identifying the remedy is simple. Accreditation must be made to work properly. The key to credible certification does not lie with auditor certification bodies adding complexity, cost and gimmickry to what ought to be a simple process. There are aspects of competence that are best addressed by those who carry the most risk. And those are the certification bodies that employ and use the auditors.

With the best will in the world, the attributes of honesty, integrity, tenacity and most of the others listed in ISO 19011 are best determined in the field, in real-life conditions and, most usually, over time.

Attempting to resolve these aspects through an artificial, third-party process only serves to add a layer of complexity, cost and mystique to what ought to be a simple, value-adding process. So let us leave the gimmickry aside and focus on the real cause of the market’s growing disenchantment with certification – the ineffective and poor performance of the accreditation system.

Some may be tempted to consider this as falling within the ‘too difficult’ category, but left unaddressed, all other approaches will amount to little more than sticking plaster. There is an old adage in business, applied when organizations are not working effectively, indicating where the cause lies – ‘the fish rots from the head’. In this analogy, the auditors quite clearly represent the tail. If the certification industry wants a viable, long-term solution to the problem of audit competence, then leave the tail alone and address the problems at the other end.

For a live panel discussion on auditor competence at the American Society for Quality’s recent ISO 9000 Summit in Atlanta, US, click here. The discussion was moderated by QSU Publishing’s president, Paul Scicchitano. For more information about QSU Publishing, click here.

About the author

Simon Feary has been director of IRCA since 1994. He currently serves on the board of IPC, the successor to IATCA.

This article first appeared in July/August ISO Management Systems. Visit http://www.iso.org/iso/en/iso9000-14000/ims/ims.html for more information.

©2005 IRCA. All rights reserved www.irca.org Contact Abbreviations