Auditing electronic-based management systems

Organizations are increasingly dependent on electronic media for the operation and control of their management systems. This means that certification bodies and their auditors must look at new approaches to ensuring the effectiveness of audits. The Auditing Practices Group believes that the method of evaluation of processes and their related documents and records must be reconsidered.

NEW! Test your knowledge

1. What is an electronic-based management system (EBMS)?

a) A system developed by certification bodies for managing audits.

b) A system which allows audits to be conducted off-site (ie from the auditor’s PC).

c) A system that is dependent on electronic documents, data and software applications for its normal operation.

d) All of the above.

2. When planning an audit of an EBMS you should consider:

a) How the audit team will gain access to the EBMS.

b) The safeguards to ensure that auditors protect the confidentiality of electronic documents.

c) The auditee’s policies for use of its IT infrastructure.

d) All of the above.

3. In order to confirm if process requirements are being met, you find that some audit time must be spent at a computer workstation which is not located near the actual process. In this case you should:

a) Highlight this situation as a non-conformity.

b) Reduce actual auditing time at the physical location of the process.

c) Allow extra time for travelling to and from the physical location of the process.

d) All of the above.

4. When auditing the control of electronic documents you should:

a) Ensure that all documents are in the same format, such as text, HTML or PDF.

b) Understand the organization’s policies and procedures regarding user privileges.

c) Ensure that no obsolete documents are controlled within the EBMS.

d) All of the above.

5. When reviewing the role played by the organization’s IT function you should consider:

a) The controls for external as well as internal software.

b) The environmental factors that may have a bearing on the EBMS.

c) The competence of personnel to operate hardware and software.

d) All of the above.

To find out how you scored, read on (answers at the end).

Planning the audit of an EBMS

An organization may have in place a management system that is dependent on electronic documents, data and software applications for normal operation. This is known as an electronic-based management system (EBMS).

During the audit initiation phase (stage one audit) the auditor should determine the structure of the organization to be audited and the degree to which its management system is electronically-based. For example, it may be a multi-site organization with a centralized EBMS, or even a virtual organization. Auditors should, therefore, verify whether the controls over a multi-site management system are appropriately addressed and established within the organization’s policies and procedures.

Auditor checklist:

  • ensure the audit team has an opportunity to familiarize itself with the auditee’s EBMS
  • review the auditee’s policies for the use of its IT infrastructure
  • review the instructions for accessing records and obtain necessary security clearances
  • check there are safeguards to ensure that the auditors protect the confidentiality of electronic documents
  • check the competence of the audit team to carry out an effective assessment of the EBMS; auditing organizations should provide training with regard to general IT trends and audit-specific considerations

  • ensure the document review can be carried out off-site, either online or by downloading documentation received by email. If technical or security factors prevent this, a review of electronic documents would need to occur at the facilities of the auditee during the stage one audit

On-site realization activities

The auditor’s trail should typically include the physical location of the process being audited. However, with an EBMS, the time needed to confirm whether or not requirements are being met may be spent at a computer workstation which is not located near the actual process. In this case the actual auditing time at the physical location of the process may be reduced, and special consideration should be given to the time required for travelling to and from the physical location of the process. The auditor should evaluate the methods employed for interaction between the physical process and electronic media to ensure the accuracy of the associated information. 

Auditing the control of electronic documents

Electronic documents that establish management system policies and procedures can exist in a variety of file formats such as text, HTML, PDF, etc. Spreadsheets and databases are also considered to be electronic documents subject to the control elements of the management system to being audited. 

Auditor checklist:

  • ensure that policies governing the controls that apply to management system documentation in general are also employed for electronic documents
  • confirm that there are effective methods within the electronic environment for ensuring the adequate review, approval, publication and distribution of management system documentation
  • understand application-specific controls to the degree that these are utilized as a basis for conformance to the applicable management system standard
  • pay particular attention to control elements such as document identification and document revision level
  • verify that the controls being employed for the management of obsolete documents are considered
  • verify that EBMS documentation exists to provide orientation to users with regard to the functional and control aspects associated with electronic documents
  • understand the organization’s policies and procedures regarding user privileges
  • verify the degree to which electronic documents exchanged with third parties are formally introduced and controlled within the EBMS

Auditing the control of electronic records

Electronic records consist of the process output data combined with the electronic formats that house the data. Control elements for electronic forms are not necessarily the same as those which apply to electronic records.

For example, with respect to identification, in the case of electronic forms, the term refers to the categorization of the electronic form itself. When identification is considered in the case of an electronic record, this refers to the unique use of the electronic form for a given data set.

Given that the knowledge-base and the performance of the organization may be almost entirely in electronic records, auditors should review the organization’s approaches for securing the information contained in electronic means. For more information on information security see ISO/IEC 17799.


IT organizational resources

As organizations migrate to using an EMBS, the IT function’s role becomes vital.

Auditor checklist:

  • has the organization dedicated appropriate IT resources to ensure the EBMS operates effectively
  • is the level of interaction, support and involvement of IT personnel properly defined
  • how does the organization address the competence required of personnel to operate hardware and software to run the EBMS
  • is the EBMS being assimilated and utilized by the organization’s personnel
  • what are the system maintenance policies and procedures for the organization’s IT platform
  • does the organization have formal backup systems and are they periodically reviewed
  • are the controls established for internal software, external software, software licensing and software updates adequate
  • has the organization taken into account environmental factors that may affect operation of the EBMS, such as facility maintenance, temperature and humidity

Internal and external electronic communication

When intranets, email and instant messaging are utilized for satisfying the requirements of the EBMS, auditors should verify that policies and procedures address the circumstances under which these means would be employed.

When the organization relies on its IT infrastructure for electronic communications with its customers (ie for e-commerce), suppliers (ie for e-procurement), external sites and other interested parties, the auditor should verify that the methodology, policies and procedures for these communications and associated transactions are formally addressed within the EBMS.

Quiz answers: 1 (c); 2 (d); 3 (b and c); 4 (b); 5 (d)

The ISO 9001 Auditing Practices Group is an informal group of quality management system (QMS) experts, auditors and practitioners drawn from the ISO Technical Committee 176 quality management and quality assurance (ISO/TC 176) and the International Accreditation Forum (IAF). It has developed a number of guidance papers and presentations that contain explanations about the auditing of QMSs. These reflect the process-based approach that is essential for auditing the requirements of ISO 9001.

This article is an edited version of ‘Auditing electronic-based management systems (EBMS)’ from the website of the ISO 9001 Auditing Practices Group, and is reproduced courtesy of ISO and the IAF. These papers were developed on current best practice and therefore have not been formally endorsed as IAF guidance or ISO TC176 interpretations. Follow the link for further information about the Auditing Practices Group.