Uncategorized
How to Understand and Follow Audit Trails in ISO9001
Understanding and following audit trails in ISO9001 is an essential skill of auditing, and also, an important skill for being an auditee. By understanding audit trails, you will write better procedures, be a better internal auditor, and be able to put yourself into the shoes of the auditor.
Definition of Audit Trail
Come to find out, even the committee within ISO that knows everything can’t precisely define what an audit trail is. The most clear definition of the term is in the above linked document, which is:
A systematic approach to collecting evidence based on specific samples, that the output of a series of inter-related processes meets expected outcomes.
And the Committee that Knows Everything goes on to say that the failure of an auditor to do this is the single most important cause of an audit not being effective.
Alternate Definition
A chain of one or more apparent requirements that, when audited, may lead to an incidence of non-conformity (or, conformity… we don’t know until we follow it)
What this means is that by following the path of a given process we can answer the very simple question: good or not good.
Process Approach
We’re supposed to be using this. A process is any activity that accomplishes anything. It can be a manufacturing process, a service process, or any other activity in which something gets done. With some brain work we might be able to come up with a process that does not do anything, but that is a spare time issue, and that is a rarity.
A process has inputs, activities and sub-activities, and outputs. There may also be measurements of whether the process is effective. Just so we’re all on board.
There is further such a thing called the Ishikawa Diagram or Turtle Diagram. This breaks the process inputs down into components: Man, Machine, Method, Materials, and Mother Nature but nowadays we have to be gender neutral so we can call it Human and Father Nature.
Identifying the Process
So the first and most important step of all of this is identifying the process. In an ISO audit, the client is supposed to have done all of this for you (per clause 4.4), including the process inputs, process outputs, process owner, and measurements of process effectiveness, but even 2 years after the implementation of the ISO9001:2015 standard a lot of companies still have not done this. If they haven’t, this isn’t going to slow you down too much, but this is the first non-conformance because it is a requirement not only to define the process, but retain it in writing somewhere.
Identifying the Process Inputs and Outputs
This should not be too hard, most of the time, if you identify the process, it is easy to identify the man, machine, methods, materials and external factors that make it up. Don’t forget to include measurements, and also, any by-product, like trash, which is most certainly a process output in some industries.
In a service business, the output is not physical. About a third of my customers have no physical process output. If that is the case, the output is all about the records of conformity i.e. results, however they may be recorded.
Starting the Process Audit
The ISO standard requires that the processes meet the planned arrangements, and so what we are going to do is pick one of the inputs and ask this very simple question: Good or not good.
Here is the basic principle: Clause 8.5 of the standard requires you to produce product or services under “controlled conditions”. That means the process inputs and process outputs have to be “good”. Anything other than “good” needs to be investigated somehow.
Good or Not Good
This is a two part problem: One is the definition of “good”, that is, what are the requirements? The other is a determination of whether or not a given part of the process meets the requirements. So this is a two part job: Determination of the requirements, and determination of whether the component you are auditing meets the requirements.
Example
Here is an example: Let’s do the human one:
The standard states that employees have to be competent to do their jobs, as determined by the client and not the auditor. The standard further states that there has to be “appropriate documented information” as evidence of competence.
Evidence
I should say that evidence is something we can all agree on. It can be a record, it can be a computer record, it can be a file, it can be a chart. The client also determines what the objective evidence is. If you as a client have not determined what the evidence of conformity is, this will make the auditor start asking a lot of questions, and my number 2 rule of auditing is don’t have the auditor start asking questions. Define the record of conformity up front.
Start your Audit Trail
So you are going to ask those two questions: What is the requirement, and what is the evidence that the client is meeting the requirement. In the “human” case, we are going to be looking for a definition of what “competence” is, and some record that says a given employee is “competent”.
Sampling
Side question: Are we going to check everybody in the process as to whether or not they are competent? No we are not. We are going to do sampling, which means we are going to select a few employees, check and see if they are competent, and make a determination on that basis. Sampling is worthy of a post in and of itself.
Determination of Requirements
So how are you going to determine the requirements? You are going to look for a job description or something similar that lists the requirements for competence for the person you are sampling.
Typically this is found in an HR file, but may be found elsewhere in the system such as a position content document. This may even be a want ad.
But, this is a document, and in most companies there is actually a rule for document control, so this is a chance to determine whether this document meets the rules for document approval. Is it valid? Is it signed off by the document approval authority? This means that even though we started on the trail of Human, we may end up writing a non-conformance on documents.
Evidence of Conformity
Once you’ve figured out the requirement, you should be able to figure out whether the employee you are auditing is in “conformity”.
There is a definition for this:
Where are we going to find this? There may be training records and an HR file. There may be a matrix. There may be a memo that says “this employee is competent.”
The method of evidence that an employee is competent “should” be specified somehow. One of the major causes of a question during the audit is a lack of information, or difference of opinion, about either of those two issues: good or not good, and how it is documented.
Auditor Overreach
This actually happens enough for it to be annoying: An auditor, particularly an external auditor, will make up requirements and expectations of a given system that the customer does not have.
Example: In the case above the auditor may feel that an appropriate qualification for a job is “ability to read and write English”. The company, on the other hand, may not feel that English speaking is a particularly important skill.
The way to prevent Auditor Overreach, and also to prevent lack of clarity in terms of what a requirement is, is to fill that information gap with a specific requirement, which is kept in writing according to the written document rules. Almost all cases of auditor overreach can be prevented by better writing of procedures.
Farther down the trail
Once you have established the requirement, and determined what the evidence is, you can ask more questions: Is the evidence valid? Was it signed off by an authorized signer? Is there an “expiration date” for competence where people need to be trained? Is there an annual review process, and if so has the employee been reviewed, and where is the evidence?
An Opportunity to Celebrate Success
It is possible for the auditor to make it all the way to the end and find out that the employee is qualified. If that is the case, this is a reason to celebrate, because this is a successful completion of the requirement.
I like to tell the joke that this is like the Coyote running into a brick wall, or being dragged into cactus because he gets to the end of the search and comes up empty. The mindset of being curious and determined is valuable, but I don’t really feel bad if the client succeeds since that is what we are after.
‘Sometimes there is no objective evidence of conformity. Records get lost. It is not clear what the record even is. There are defects in the document (not signed off). There may be other issues. If that is the case, you can raise a non conformity (we will have a post later on how to do that).
Summary
This is just one of the examples. We can go through similar examples of the Ishikawa Diagram and develop audit trails for those too, and we may do so at some point.
Keep in mind that this example used the process input, “human”. I could have just as easily used the process output “the product” and conducted an audit trail on whether the product was good or not good, and what is the supporting background information that backs up that statement.
Feel free to contribute. I have a lot of material on this. The Internal Auditing course is imminent.
