In addition to my various auditing jobs, I also “get audited” once a year. I have far from a perfect record in this regard, but when I get a CAR (Corrective Action Request) I use it as a learning experience. This particular case is interesting in that it illustrates best practices, or lack of, with documenting areas of non-conformance.

Back Story

Along with my job of doing audits for two global registrars, I have a nice job two days a month, maintaining the quality system in a distribution operation. This family owned business sells boring industrial supplies into the aerospace industry. So, the client is certified to the ISO9001 and AS9120 standards.

Because it is a small operation, the registrar we use has a hard time finding auditors. In the last 8 years, we have had 6 different auditors. That in itself is probably a topic for another blog post.

I will not tell you who the company is, or who the auditor or registrar are. I am, however, going to send this to the technical manager of the registrar to see what happens.

The “Poorly Written CAR”

During our recent surveillance audit, the auditor issued three “Corrective Action Requests.” The internal audit (that I did) supposedly didn’t audit all of the clauses of the international standard. The second was for our supplier on-time delivery being unacceptably low. The last was for not having completed truck inspection forms for our remote operation.

The Internal Audit Issue

According to the first Poorly Written CAR there was no objective evidence that I audited all of the clauses of the standard. This is actually a requirement of 9.2 of both the ISO9001 and AS9120 standards.

The auditor’s main objection is that I didn’t have an “audit matrix”.

The Audit Matrix

One of my Poorly Written CAR was to make me have an audit matrix, which is not a requirement of the standard.

Here is one. It lists the processes along the top, and the clauses of the standard along the side. It is a work tool that states where, in all of the processes, the clauses were audited.

The Problems with the first CAR

The first problem is, that the auditor did not clearly state which clause of the standard I didn’t audit. The legal profession recognizes this as an “assertion”.

My Actual Audit

What I actually do, in my internal audit, is that I use the client’s procedures. I go line-by-line through the manual. I identify procedures that are not being followed.

In this year’s internal audit, I identified 21 areas of potential non-conformity, while thoroughly auditing all of the processes.

I’ve never had an audit matrix because I know that the client’s system addresses all of the requirements of the standard. The previous 6 auditors that have audited this place have not mentioned this as an issue.

More Problems with This Poorly Written CAR

To start, there is no requirement in the standard that I have an “audit matrix” or any other specific form of documentation. Clause 9.2.2 only states that I need to “retain documented information as evidence of the implementation of the audit program and the audit results.” The word “matrix” does not appear in the standard anywhere.

Here is my audit documentation.

My Poorly Written CAR was for my internal audit documentation, and this is it.

I have a high level audit schedule. There is an audit plan, and the sign in sheet for the opening and closing meeting. I have the notes where I went through each of the processes, I have a list of findings, which is a requirement. I most certainly do have the evidence of implementation of the audit program and the results.

A Better Way to Audit This

What the auditor “should” have done, is to sample some of the clauses and look for evidence that I did or did not audit it. That way, he would have been able to point to a specific clause of the standard that I missed. That way, there would be lack of evidence of me not auditing a specific clause, and I would be OK with the NCR. In fact, there may well be a clause I missed, since I am imperfect.

Whether or not I have an audit matrix does not actually provide the information. I could have checked the box and not audited the clause. In fact, in my response , I will make an audit matrix, stick it into my system somewhere, and go on with my life. It will do nothing to ensure that my next year’s audit actually addresses all of the clauses.

Auditor Back Story

I have actually had this happen before. I have had some of my internal audits questioned by certain auditors because I actually do a good job and find stuff.

A lot of this is “who the auditors are”. A lot of the aerospace auditors are former military people. These guys, and they are always guys, are personally threatened by people that do a better job than they do. Therefore they feel compelled to “pull rank” and hammer down the offending overachiever.

This guy took a lot of audit time to explain that he was a former Navy and NASA guy. He had an extensive firearm collection, and suggested who he voted for in the last election.

None of this has any place in any audit.

I actually have another blog post on this topic. (Click Here)

The Second NCR

The client I work for is a little distribution business. There are some specific AS9120 requirements for distribution that they need to meet.

This business has a lot of specialty and/or one-off products that they cannot carry in inventory. The suppliers often need to fabricate this stuff and deliver it for re-distribution.

So, quite often, the items they carry show up “late” as defined by their promise dates. The downstream problem is that some fraction of the client’s deliveries also show up “late”.

In fact, the incoming on-time delivery is X percent, and the outgoing on time delivery is Y percent. I am not going to tell you what X and Y are, because it doesn’t matter.

The auditor knew nothing about the client’s business before he walked in the door. He thought that X and Y were too high. The auditor wrote the second NCR for the purchasing department not meeting the purchasing KPI.

” The process is not delivering the planned results and appropriate action is not being taken. “

The Can of Worms

The problem with this is that there is no KPI for our purchasing department for incoming on-time delivery. The KPI we have, which is required, is for “vendor units correct” that is, incoming order correctness. In fact, 95% of the suppliers are specified by the customers and we don’t have the option to drop them.

The auditor is basically imposing a supplier on-time delivery KPI on us. This is despite him being a stranger to this company, and not knowing anything about the business.

We are doing fine in the only documented KPI which is “vendor units correct.”

The standard does require us to “periodically review…. on time delivery performance” as part of 8.4.1.1 of the AS9120 standard. We’re doing that annually and had objective evidence that we discussed it in the management review.

The Real Can of Worms

I am not arguing that the X and Y above are high or low. In fact, Marcus Lemonis would have a field day in this place on the topics of inventory control and a lot of other things. It is not my place to say. It is not my business, and neither is it the auditor’s.

https://www.cnbc.com/video/2019/01/09/marcus-lemonis-finds-issues-in-bens-gardens-inventory.html

The auditor is going way out on a limb by suggesting that the values X and Y above are, in his opinion, deficient. If the client improves their on-time delivery “some” but not “enough” as determined by him, will we now get a major NC?

Furthermore, is the auditor now liable for changes we make in the business? Example: One possible solution to this problem is to supply only “Commercial off the shelf” items (COTS). That way the supply chain will be orderly and delivery will be on time, but profitability will be lower. Surely he does not want to own any of that.

The Third Poorly Written CAR

This one is the silliest of all. The client owns an operation in another state which has zero employees. They own a little warehouse, and have trouble keeping staff.

The auditor inspected this place, and found that the daily checklist for inspecting the delivery truck was not completed.

So when the auditor audited the warehouse, there were some number of days that the daily checklist was not filled out by the nonexistent employees.

The Requirement

Is it a requirement that employees inspect these trucks daily? I suppose it is. The insurance company imposes this requirement on us. The procedure lacks the detail that if there are no employees, and the truck does not get used, the inspection is not required that day.

There is now an app for this. I have been in bigger and fancier places that have the same requirements and there is an electronic way to ensure that the forklifts and deliver trucks get inspected, or they won’t run.

But in this little family business, this is not considered a good business investment.

Side note: This is a chronic problem of “stuff flowing downhill” and it complicates the employees’ jobs to fill out a checksheet every day. I have discovered this exact non-conformity in the internal audit every year for the last three years.

We also recognized this as a problem with the forklift inspection at this place. We issued a memorandum that stated that we were accepting the risk of skipping these inspections while there were no employees in the shop.” Evidently the auditor didn’t apply this theory to the truck.

Other side note; The writer did not correctly document this CAR either. It should have said something like “there is no objective evidence for a completed truck form for October 9. ”

The Irony

I suppose the corrective action for this will be to hire an employee, correctly train him or her to inspect these, and audit for effectiveness.

The real irony is that for the purposes of this company, daily inspection of these things is probably unnecessary since they are lightly used.

The Problem with All of These

Actually there are several problems:

One is that because these are poorly written CAR and going out on a limb, I, who have seen 1500 of these am hard pressed to second guess how to fix them. If I have a hard time, I can’t imagine putting myself in the place of someone less experienced.

I suppose my best hope is to address the problems as they appear and hope the auditor accepts them. This is the exact opposite of what we want which is addressing specific problems which we know can be fixed.

The bigger problem is that it causes my clients, who are paying the bills, to lose faith in the system. None of these add value to their business. All three of these NCR will cost time and money to respond to, but not actually address the problems that they actually have.

How am I going to respond?

For CAR-1 I am going to produce an audit matrix, and put it in my system. I will also downgrade my checklist to one that is less effective. As part of my work with the registrars, I already have a nice system to do this. It helps me touch all of the bases, but it does not allow me to spend time on the places where there are real problems. But, it may keep the auditor out of my hair. One can only hope. This is the exact opposite of what I want to do to reduce risk in my system, but it will fix the CAR.

For the second Poorly written CAR, I have already done the analysis as to who the worst offenders are on incoming on-time-delivery, and we will issue supplier corrective actions and see what happens. A lot of these companies are famous, Fortune 500 companies, and the likelihood of a favorable response is close to zero.

The “real” problem is that the client does not have a good system to go back through their promise dates and update them when the suppliers tell them that a shipment will be late.

The third one? As suggested above, when a worker is hired, we will include forklift and truck inspections along with the training. When the internal audit happens, I will audit the forms for completeness. I will also add to the form that the completed form is not required when the truck is not used.

A New Hope

None of these things solve the real underlying issues in this business. I will be a good sport document them just to avoid violating my 2nd rule of ISO auditing. That’s because it is less work than going through the appeal process.

At least I can use this as a blog post and maybe get a few clicks out of it.

I would love to hear your comments on this. Registrar, you know who you are. Maybe you will review your auditor screening and/or selection processes. None of these Poorly written CAR should have passed your expert reviewer.

Tags