Recently there was a concerted effort among PB auditors and corporate leadership to collaborate on developing an electronic process for performing quality audits. This effort included:
• Standardizing PB's quality procedures
• Changing requirements to allow for digital records
• Developing an electronic tool to conduct the audits.
The new system represents a major departure from the past, when most of us used pen and paper and some used semi-electronic methods—either word processing, spreadsheets or e-mail forms. These early electronic methods also resulted in paper, however, because audit records had to be signed and stored, and became problematic because of version changes in software.
Auditing Process
The practice of performing an audit in the past was long, laborious, repetitious, time consuming, and error prone. It could easily take 6 to 8 hours complete the initial audit process alone; not including the time to close the non-conformance reports (NCRs).
Typically the auditor visited a project manager (auditee) with a checklist of questions and requirements. The auditor penciled-in next to each question whether the auditee met the requirements or not, with notes to indicate which records could be verified. If the requirements were not met, the auditor documented why and created separate documents NCRs to describe and track corrective actions to be taken. The marked checklist (paper), along with the NCRs (paper), became the audit record and were filed in the office quality files.
Each NCR was populated by transcribing the requirements that were not met and the reason the non-conformance occurred. This document was given to the auditee, who filled in his or her assessment of why the non-conformance occurred (the “root cause”), plans for correcting the defect, the responsible party and the date these planned actions would be completed. Next the non-conformance documents were returned to the auditor for review. After the date the corrective actions were supposed to be would be completed, the auditor would verify that they were and close the non-conformance report. All paper records from this process were then stored in the office's quality files .
The primary problems with the paper process were not related to time, however, but to recording the history of the audit. The paper audit reports were accessible only to the original auditor, the project manager (auditee) who answered the questions, and the area quality manager who maintained the official quality files. Without additional major record-keeping efforts, there was no easy way to see the progression or history of the audits and NCRs over time.
Figure 1: Quality Award Presentation. Pictured are (left to right): Bill Smith, Ray Crawford, Drew Gilstad, Nevin Garabaglu, Rich Fischer, Stewart Gordon, Tiny Babin, and Greg Kelly |
ISO Audit System Development Team
• Ray Crawford, Director of Quality
• Mel Stein, Metro District Quality Manager
• S t e w a rt Gordon,Metro Deputy District Quality Manager
• Nevin Garabaglu, Princeton Area Quality Manager
• Drew Gilstad, New York Area Quality Manager
• Aileen Read, North West District Quality Manager
• David Wagoner, Rockville Are Quality Manager
• Alan Johnson, Director of Intranet Development
• Brian Baldwin, Intranet Developer
• Mark Cipolloni, Director of IT Support
• Fred “Tiny” Babin, IT Support |
PB ISO AUDIT SYSTEM CREATION
After consolidation of the PBQD quality system program in July 2002, it became apparent to us in the Princeton office that an automated internal audit program would be helpful to all PBQD offices.
ISO AUDIT SYSTEM DEVELOPMENT TEAM
• Ray Crawford, Director of Quality
• Mel Stein, Metro District Quality Manager
• Stewart Gordon, Metro Deputy District Quality Manager
• Nevin Garabaglu, Princeton Area Quality Manager
• Drew Gilstad, New York Area Quality Manager
• Aileen Read, North West District Quality Manager
• David Wagoner, Rockville Area Quality Manager
• Alan Johnson, Director of Intranet Development
• Brian Baldwin, Intranet Developer
• Mark Cipolloni, Director of IT Support
• Fred “Tiny” Babin, IT Support
With a corporate commitment made and a development team established (Figure 1 ), the functions the new tool should provide were defined between December 2002 and late January 2003. The basic ideas were that it should:
• Collect data for project and system audits based on the current quality procedures
• Extract certain project and employee information from Oracle
• Be flexible enough to accept continued changes to the requirements
• Automatically generate nonconformance or observation reports from the data collected
• Be able to be used from any PB standard desktop or laptop on or off the network environment.
In February 2003, system development got underway. Working with the intranet group, we used the Oracle data connect that PB WorldNet already had access to—a move that saved hours of development time and removed the need to replicate the existing data connection to Oracle.
We developed a Windows-based application that stores the collected data in Microsoft Access and SQL databases. In mid January 2004, after a number of beta tests, the official PB ISO Audit System was released to all PBQD offices. It has been well received by its target users, and its features have grown by leaps and bounds since its original conception.
Basic Features
Data Collection. The program collects audit data using a graphical user interface (GUI), which is very similar to common Windows-based applications with respect to how users populate data fields on a form. Data can be typed in, selected from a list of options or picked from pop-up tables for project, employee, office or cost center information. This information is downloaded periodically by the area quality manager from PB WorldNet's connection to Oracle.
Queries and Reports. The program offers a number of tabular queries and reports that are commonly used in the quality system. These reports can be filtered and sorted as needed, then printed or exported to an Excel file.
Graphs. The program has an option to generate graphs of the existing audit data. This option creates a new Excel file containing a number of graphs driven by a number of pivot tables.
Automated Functions
The ISO Audit System has a number of built-in automation (auto) functions, including those described below.
Automatic Answers to Questions. A parent/child relationship was built into the list of audit questions such that if a parent question is answered with NO or N/A, then all child questions are automatically answered with N/A and are grayed-out The gray-out makes it apparent to the auditor that the answer to the parent question caused the other answers to be N/A. Currently, this function can be used to auto-answer up to 119 of the 147 project audit questions.
The system audit form uses parent/child relations also, but has an additional gray-out function. This system audit specific function is based on the type of system audit being performed, either office, district, discipline or company. The questions that are not related to the type selected will gray-out and auto populate with an answer of N/A. This again reduces the number of questions that an auditor needs to ask during a system audit.
Auto-Generation of Non-Conformation and Observation Reports. If questions asked during the audit are answered with “NO” or “Observation,” then a separate report needs to be generated for each of those questions. The auditor can generate these reports fly by simply clicking the menu option; Data à Create NCR/OBS Reports from Audit Rec..
Auto-Population of E-mail Notifications. Auditors can inform the auditee and others of any non-conformance or Observation reports that were created during the audit by simply clicking on a menu option that will auto-populate an e-mail form with the subject, TO, CC & BCC list and instructions for the auditee to follow. Likewise, the auditee can use a similar menu option to auto-notify the auditor that the required actions have been completed, which will indicate the NCR/OBS reports can be closed.
Auto-Population of Oracle-Related Data. When starting a project audit, the auditor can pick the project number from the project pop-up table and the program will auto-populate the project number, name, office location, manager, client name, start date, contract value, percent complete, area manager and principal in charge.
Benefits
The PB ISO Audit System provides a number of benefits users are realizing as they go through the audit process. In addition, significant tangible and intangible benefits being realized
User Benefits. The PB ISO Audit Program:
• Provides an accurate centralized database covering all internal audit activities
• Maintains a centralized database for project and system quality audits.
• Meets and exceeds ISO and PB's quality system requirements
• Serves as an excellent training tool for both auditor and auditee
• Provides quick and simple internal audit results for tracking quality system effectiveness on an office, district or company-wide basis
• Assists senior management in establishing and tracking quality system goals and objectives.
Tangible Benefits . The following savings will result from use of the new system:
• The quality area managers expect to see a 50 percent savings in the performance of quality audits due to the efficiency of having a centralized, accurate, paperless, and automated tracking system.
• A paperless quality system will save reproduction costs.
Tangible benefits are calculated with the following assumptions:
• Average number of projects greater than $100K = 2,000
• Internal auditor average hourly rate = $35
• Overhead multiplier = 150%
• Average time spent during previous paper internal audits per project = 8 hours
• Average time spent for automated internal audits per project = 4 hours
• Average number of pages per audit report including NCR and Observation = 15
• Number of copies distributed = 5
• Average reproduction cost per page = $0.10
Annual direct labor cost savings = 2,000 projects X 4 hours @ $35/hour = $280,000/year.
Indirect overhead savings = $280,000 X 1.50 = $420,000/year.
Total DL + OH estimated savings per year = $280,000 + $420,000 = $700,000/year
Reproduction cost savings = 2,000 projects x 15 pages x 5 copies @ $0.10/page = $15,000/year
We realize that individual savings need to be based upon actual operations at each office and/or location. We also have not attempted to estimate other savings associated with reduced efforts by quality managers, auditors and employees (auditees) in preparing summary reports on quality system effectiveness and completing nonconformance and observation reports, respectively. We expect with a more accurate and comprehensive PB ISO Audit system, savings in maintaining the system records will be achieved as well.
Intangible Benefits. We conservatively estimated that benefits realized from improved quality and efficiency due to a better and vigorous internal audit system = ~$1,000,000/year, knowing that:
Improved quality ? client satisfaction? repeat business.
I At the time of writing, the audit system was available for PBQD offices and handled only ISO 9001; however, the team was in the process of incorporating ISO 14001 questions .
A demonstration copy is available to all PB users, but an internal security list limits who can create or modify records. To access the ISO Audit system:
• Go to http://www.pbworldnet.com
• On green bar on top, click on Departments
• Scroll down and click on Quality Americas
• On left side of your screen, go down to Audit Program, and on the pop-up menu click on Automated Audit Program
• Click on Documentation for the user's guide or system installation instructions.
|
