Up to the job?
While much has been published about internal auditing and auditing by certification bodies, a vital link in the chain of confidence has received scant publicity: how do accreditation bodies establish that certification body auditors are up to the job? Brian Henry presents a straightforward approach
It has become clear that many certification bodies have found it difficult to make the transition from the old qualification approach for auditors that was previously specified in withdrawn standards to the competence approach taken by ISO 19011, ‘Guidelines for quality and/or environmental management systems (EMS) auditing’, which is now the unique auditing standard in the ISO 9000 and ISO 14000 families.
This problem has led to nonconformances being raised when the accreditation body has audited the certification body, whose management has not understood the competence approach. As a result, ISO and the International Accreditation Forum (IAF) have set up a task force with the job of writing clearer auditor competence requirements in the draft ISO/IEC 17021, ‘Conformity assessment – requirements for bodies providing audit and certification of management systems’. In the meantime, the following straightforward approach may be of some help.
First steps
Examine the contract review that the certification body has carried out for certification of an applicant organization. This, among other things, should have resulted in a profile being created for the organization to be audited, especially in terms of the proposed scope of certification activities to be considered for a particular IAF scope code. You should then expect to see an auditor/audit team profile from the existing certification body pool of auditors that matches the corresponding IAF scope code.
The two profiles should match. If they don’t, consideration needs to be given to acquiring additional specialist external auditor resources or making use of a technical expert(s) to fill the gap.
At this stage, the focus is on the audit team, which may consist of one individual auditor, or perhaps several auditors. Don’t make the mistake of diving straight into table one in section seven of ISO 19011, ‘Examples of levels of education, work experience, auditor training and audit experience for auditors conducting certification or similar audits’. Table 1 does not list levels of competence – it lists levels which historically have been useful as a coarse filter for qualification and which undoubtedly will usefully contribute to the development of auditor competence.
ISO 19011 defines competence as being the demonstration of personal attributes and knowledge and skill. Nine personal attributes, together with four generic areas of knowledge and skills and two specific clusters of knowledge and skills, are specified for QMS auditors in section seven of ISO 19011. Three specific items, knowledge and skills are given for EMS auditors.
It is reasonable to expect that the certification body should have identified each of these within the documented processes and assigned quantitative or qualitative criteria against each for a specific audit situation. The criteria stated needs to be the minimum necessary to ensure satisfactory auditor performance and deliver audits that are sufficiently thorough, consistent and provide added value.
Personal attributes
Individual auditor records should show that the nine personal attributes have been addressed and evaluated. The evaluation can range from simple observation to more complex methods such as psychometric testing. It is vital that any shortfall is either corrected through further development, or can be safely mitigated and managed by assigning the auditor to situations where the audit will not be compromised.
Generic knowledge and skills
The certification body should assign quantitative or qualitative criteria for each of the following criteria for a generic audit situation and that the resulting individual conformity and performance has been demonstrated, tested and accepted.
Audit principles, procedures and techniques
Knowledge of these factors has been acquired from successful completion of formal auditor training courses, certification bodies' specific training, and skills developed from role play, or from audits carried out under guidance and supervision. The demonstrative ability to carry out conformity audits using the process approach is probably the most important.
Management system and reference documents
This is perhaps the most important and controversial. Research is confirming that many auditors do not really understand the standards and do not apply them correctly, often resulting in invalid or inappropriate findings. It is expected that the certification body should be able to demonstrate that the levels of knowledge have been appropriately set.
This is likely to have been acquired from successful completion of formal auditor training courses, certification bodies' specific training, and skills developed from role play or from audits carried out under guidance and supervision.
Organizational situations
This covers, for example, avoidance of ‘disasters’ caused through assigning an auditor without sufficient knowledge of the structure and operations of a large corporate multi-site organization, and not having the requisite skills. The converse case is the auditor who is competent with the large organizations, but is then unable to adjust expectations when dealing with small or medium-sized enterprises where each employee has many roles, but with mitigated conflicts of interest.
It is expected that the knowledge and skills to handle these situations effectively can be gained from exposure to these organizations through training, workplace experience and auditing experience under training and supervision. Some auditors may not be able to acquire the full range of knowledge and skill and the certification body should have precautions built into the assignment processes to confine activity accordingly.
Applicable laws, regulations and other requirements
Although listed under generic knowledge and skills, this set of criteria will inevitably have a slant on a QMS-specific context and it is expected that the auditor(s) should be aware when it is appropriate to ‘back off’.
Specific knowledge and skills
These are quality related methods and techniques. In order to audit effectively in a QMS context, it is vital that the certification body has clearly set the levels needed. It is expected that these relate to modern quality management tools and their applications and are maintained and updated through continuing professional development.
Processes and products
This area is almost certainly the most difficult knowledge and skill set and the most troublesome for certification bodies. What is expected here is that the certification body has recognized that the auditor(s) will encounter sector-specific terminology and jargon, technical characteristics of processes and products including services and, very often, unique sector-specific processes and practices.
The certification body should have carried out some type of risk analysis for the criticality related to the product in the specific sector. It may well be that in situations of high-risk criticality, it decides that the auditor will always be accompanied by a scope-specific and currently practicing technical expert. Conversely, with a low-risk criticality product, an auditor with limited competence may operate alone, but with the support of briefing notes.
What is not acceptable is a situation where the certification body may have mistakenly considered that the process and product knowledge and skills that are needed for a surveillance audit can be somewhat less than for an initial certification and re-assessment audit.
Whatever the approach taken, it is vital that the certification body has defined the parameters to be met in terms of the specific product knowledge and skills for use when establishing the requisite audit team profile.
Migration of competence
It is a well-established fact that many auditors are able to acquire satisfactory levels of knowledge and develop satisfactory levels of skill to audit effectively in other IAF scope codes that are sometimes quite different from their mainstream industrial or commercial disciplines. This should be neither dismissed, nor accepted at face value.
What needs to be considered is that the grounds and justification have been satisfactorily provided by the certification body. This is perhaps the time when criteria such as table 1 in section seven of ISO 19011 may be useful and assist the certification body in setting the levels which, if attained, may lead to the requisite auditor competence. However, it needs to be emphasized that there is no single 'table 1' to suit all situations. The accompanying notes in the standard are quite clear that the levels can be higher or lower, depending on the scope and nature of the specific audit.
Experience has shown that outdated workplace experience is often more dangerous than having none at all. Claims of having gained product and process knowledge through operating as a consultant need to be carefully considered – especially if there is no satisfactory evidence that the individual had the ability to undertake consultancy in the first place and accepted the consultancy project through misrepresentation of his or her qualifications and competence.
Having worked in other areas as an auditor when supported by a technical expert is a common and acceptable way of developing product and process knowledge and skills. It is again expected that the certification body is able to demonstrate that the minimum audit-specific levels of knowledge and skills have been met and have been demonstrated and tested.
Auditor certification
This is now required to be taken into account, but consideration needs to be given about the credibility of the auditor certification body. Members of associations such as the International Personnel Certification Association (IPC), linked by multilateral agreements and subject to regular peer review, and bodies accredited by IAF accreditation bodies to provide auditor certification to ISO/IEC 17024, ‘Conformity assessment - general requirements for bodies operating certification of persons’, should inspire confidence. Even so, auditor certification may not focus on product and process knowledge and skills and therefore does not absolve or take away from the certification body the responsibility for auditor competence.
Generic knowledge and skills of audit team leaders
This is no different to the generic knowledge and skills for auditors with provision for the certification body to set the levels needed to develop the competence needed by QMS audit team leaders and make use of the methods specified in ISO 19011 for testing and demonstrating this.
Evaluation processes
ISO 19011, section seven, figure 5, shows the relationship between the stages of the evaluation process and then goes on to describe four main steps that are needed. This or a similar model should have been documented and implemented by the certification body. Guidance on the use of a variety of evaluation methods is also specified in ISO 19011. These, or a selection of these, should also have been used. Witnessed audit is a powerful technique, but by its very nature is artificial, intrusive and with potential to influence the outcome of the actual audit being undertaken. An alternative technique is ‘post audit review’, which has been found to be equally effective without the undesirable side effects.
Care was taken when preparing this article to maintain confidentiality and not to divulge any unique techniques that have been developed by an individual certification body. What has been cited is strictly a generalization of what is considered as being some of the better approaches that have been seen to be used around the world and would be acceptable to accreditation body members of the IAF.
About the author
Brian Henry is an independent accreditation body lead auditor who is contracted to represent several international accreditation bodies that are members of the IAF. He is the UK principal expert on auditing to ISO/TC 176 and ISO/TC 207 joint working group for ISO 19011 and participated in the development of this standard and its predecessors. For further information contact Brian Henry on e: bhenry@btconnect.com
This article first appeared in July/August ISO Management Systems. Visit http://www.iso.org/iso/en/iso9000-14000/ims/ims.html for more information.
 |
|
 |
|
 |
 |
 |
|
 |
|
 |