Six Sigma Metrics: An Evolving Indicator of Quality Assurance for Clinical Biochemistry
Correspondence Address :
Dr. Sibasish Sahoo,
Assistant Professor, Department of Biochemistry, All India institute of Medical Sciences, Kalyani, West Bengal, India.
E-mail: drsibasish18@gmail.com
Introduction: The analytical phase of the total testing process is the one in which the clinical biochemist can directly intervene to improve the quality of tests reporting. The sigma metrics and Operational Process Specification (OPSpec) chart can specify to which category the laboratory belongs.
Aim: To apply sigma metrics to analytical process of testing, do the root cause analysis and apply the corrective measures according to Westgard Rules to improve laboratory performance towards the quality assurances.
Materials and Methods: This was a retrospective-prospective study carried out in a clinical laboratory of MKCG Medical College and Hospital, Berhampur, Odisha, India, from July 2020 to March 2021. A retrospective secondary data analysis of six months duration was carried out in a clinical chemistry laboratory with a follow-up prospective study for three months. During this period, 16 analytes were tabulated to analyse the Internal Quality Control (IQC). External Quality Control (EQC) for the same analytes were obtained on monthly basis and the sigma metrics was calculated for each analytes. For analytes with sigma value <3, appropriate measures were taken according to Westgard rules to improvise the quality of laboratory investigations. The statistical analysis of sigma metrics was performed in “R” v-3.6.3.
Results: Out of total 16 analytes, three analytes at level 1 and two analytes at level 2 Quality control (QC) showed a world class performance whereas four analytes showed a poor performance at both the QC levels with sigma metrics value <3. From Quality Goal Index (QGI) and root cause analysis, the source of error was detected and corrected.
Conclusion: The inaccuracy and imprecision of different parameters in the analytical phase of the testing process can be addressed by calculating the sigma metrics and do the root cause analysis. Application of corrective measures according to Westgard rule can improve the laboratory performance towards the quality assurance.
Imprecision, Inaccuracy, Quality goal index, Quality control, Root cause analysis
The present era of globalisation has driven the medical science into a newly established platform, which is of laboratory investigations with the highest sensitivity and more specificity. But ensuring the accuracy of the report has always been challenging for the Clinical Biochemist, who steers the treating physician towards the next level of treatment for a patient’s wellbeing (1). To stay upbeat, the current clinical laboratory utilises both internal and external quality control for better quality assurance, which varies for different biochemical analytes. However, the development of precision-based medicine has arisen the number of challenges regarding quality management. A system that integrates accurate evaluation, problem-solving, and process improvement is required, and thus Six Sigma quality management methodology has attracted public attention (1).
The initialisation of sigma metrics in clinical chemistry laboratory has become an important aid in evaluating the errors in quality control of a laboratory system. Sigma quantifies the performance of a process at a rate of defects per million. The sigma value indicates how often errors are likely to occur (2). The higher sigma value, lowers the chance of erroneous test results by the laboratory. It can easily quantify the exact number of errors by combining bias, precision, and total allowable error (TEa). The model for six sigma management includes five processes, i.e. Define, Measure, Analyse, Improve and Control (DMAIC) (3).
The detection capacity of a laboratory will become “world class” if the analytes performance of the laboratory achieves a level of six-sigma resulting in minimising the errors as low as 3.4 per one million tests (4). Hence, application of six-sigma for a clinical laboratory is essential for an error-free reporting.
Studies have been carried out to elicit the individual laboratory performance (5),(6). Mao X et al., in their study analysed 20 parameters over a period of five months and found “Six Sigma metrics can serve as a self-assessment method in guiding clinical laboratories to make QC strategy and plan QC frequency”. Implementation of the sigma metrics into the laboratory’s analytical processes can be helpful to produce accurate test results (5). Similarly, Westgard JO and Westgard SA in their study concluded that the EQC validation process will be greatly improved with the application of Six-Sigma principal and metrics, and recommendations can be provide on the amount of QC scientifically which are needed for the laboratories (6).
The MKCG Medical college is the oldest and only medical college in the southern part of Odisha, India. The clinical laboratory in the Department of Biochemistry of this tertiary care hospital caters nearly 530 outpatient samples and 250 inpatient samples in a day. Hence, for proper clinical diagnosis of the patients, accuracy in the biochemical report is very much needed. In that context, the present study was undertaken to apply sigma metrics to analytical process of testing, do the root cause analysis and apply the corrective measures according to Westgard Rules to improve laboratory performance towards the quality assurances.
The present retrospective-prospective study was conducted at the Biochemistry Laboratory of MKCG Medical College, Berhampur, Odisha, which is a tertiary care hospital in southern Odisha, India, from July 2020 to March 2021. A retrospective secondary data analysis of six months duration (July 2020 to December 2020) was carried out in a clinical chemistry laboratory with a follow-up prospectively for three months (January 2021 to March 2021). The study was approved by Institutional Ethical Committee (IEC) with approval number: 804/ Chairman-IEC MKCG Medical college, Berhampur, Odisha, India.
Inclusion criteria: 16 routine chemistry parameters processed on the same platform were included in the study.
Exclusion criteria: Hormone assay and other Enzyme Linked Immunosorbent Assay (ELISA) tests of other platform which are performed in the clinical biochemistry laboratory were excluded from the study.
The IQC data for the 16 analytes (Sr. Albumin, Sr. Alkaline Phosphatase (ALP), Sr. Alanine Transaminase (ALT), Sr. Amylase, Sr. Aspartate Transaminase (AST), Sr. Bilirubin (T), Sr. Cholesterol, Sr. Creatinine, Plasma Blood Glucose, Sr. High Density Lipopprotein (HDL), Sr. Potassium, Sr. Total Protein, Sr. Sodium, Sr. Triglyceride, Sr. Urea and Sr. Uric Acid) were analysed. These parameters were processed in the instrument TOSHIBA 120 FR with Agappe Reagent. The daily IQC materials at two different levels i.e. Level-1 which constitute the Normal Biological Reference range and Level-2 which is Pathological Reference range were obtained.
For the calculation of bias%, the required EQC data for each analyte were collected and calculated by using formula (2)
Bias%= (Measured Mean-Target Mean) / Target Mean ×100
Where, measured mean is the mean calculated for each parameter from the six months collected data and Targeted mean is the mean value obtained from EQC measurement. It was calculated by considering the monthly EQUAS reports form CMC, Vellore as targeted mean. The bias% is an indicator of accuracy and systematic errors in analysis (2).
The EQC data were analysed for inaccuracy and the IQC data for imprecision. For all the parameters, imprecision was estimated using coefficient of variation (CV%) which is a measure of variability of an assay and indicator or random error (7). It is measured by the formula:
CV%=SD *100 / Mean
Where, SD: standard deviation of the analyte and Mean is the measured mean of each analyte (3).
The Total allowable error (TEa) or the tolerance limit is the total allowable variation for the performance of an analyte. The TEa value for the each analytes was obtained from Clinical Laboratory Improvement Amendments (CLIA) proficiency testing criteria for acceptable analytical performance as printed in Federal Register (8). Sigma metrics were calculated using TEa as per the CLIA guideline from US and the biological variation database specification (9). This was calculated by using formula:
Sigma (σ)= (TEa-Bias%) / CV%
Where, TEa: Total allowable error, CV%: Coefficient of Bias (3)
The standardised sigma values were categorised into six categories, i.e. world class (σ ≥6), excellent (5≤ σ <6), good (4≤ σ <5), marginal (3≤ σ <4), poor (2≤ σ <3), and unacceptable (σ <2) (4). For each analyte, the sigma value was calculated and the quality of measurement was group according to sigma value. Further, for the analytes having lower sigma value (i.e. σ <3), the QGI was calculated in order to achieve quality improvement of automated analytic tests, to understand the test-specific reasons for their quality shortcomings, be it either excessive imprecision, excessive bias or both and the reason for the lower sigma level of analytes (10). It is calculated with formula:
QGI= (Bias% * CV%) / 1.5
With the reference from the imprecision and inaccuracy, Root Cause Analysis (RCA) were performed and appropriate corrective measures were taken for the specific analytes ensuing Westgard rules. The probability for error detection is improved through selection of Westgard rules that are particularly sensitive to random and systematic errors (6). RCA was performed with plotting of cause-effect diagram of the various process performed at clinical laboratory i.e. from beginning of sample collection to report delivery to the patient. Following the analysis, corrective measures were taken as per problem identified.
Following corrective measures, the sigma value of those analytes were measured by calculating the mean and bias% by obtaining the data for next three months i.e. January 2021 to March 2021.
Statistical Analysis
The collected information for each analyte were tabulated in Microsoft Excel (Version-16) on daily basis for the six months period and advance analysis (QC sigma chart and RCA) were done in R version 3.6.1 (Foundation for Statistical Computing, Vienna, Austria). The mean and standard deviation (SD) was calculated for each analyte.
The laboratory means, targeted mean, SD and the CV% of 16 parameters for two controls levels (L-1 and L-2) were summarised in (Table/Fig 1). The average CV% for the analytes ranges from the 0.9% (Potassium) to 4.25% (SGOT) for Level-1 and 1.07 (Potassium) to 5.12 (Triglyceride) at Level-2.
Similarly, (Table/Fig 2) illustrated the calculated Bias% and sigma matrices of each analytes for the QC levels (Level-1 and Level-2). The average Bias% for the analytes range from 1.2% (Potassium) to 17.81% (Amylase). Regarding sigma value, the lowest value (2.09) was observed for AST/SGOT and highest value (6.84) was observed for Creatinine at Level-1. Similarly, at Level-2, the lowest sigma value (2.17) was observed for ALT/SGPT and highest value (8.26) was observed for HDL.
The standard QC sigma chart for each analyte (both the levels) were constructed to evaluate the performance of each analytes. For the QC level-1, three out of 16 analytes showed a performance of world class (>6 sigma) i.e. Glucose, Creatinine and ALP; nine analytes (Bilirubin, Albumin, Amylase, HDL-Cholesterol, Potassium, Total Protein, Sodium, Triglycerides and Urea) showed sigma value of >3 and ≤ 6. Four analytes i.e. AST, Uric acid, ALT and cholesterol showed sigma value <3 (poor) (Table/Fig 3).
For the QC level-2, two out of 16 analytes (Bilirubin and HDL- Cholesterol) showed world class performance, nine analytes (Albumin, ALP, Amylase, Creatinine, Glucose, Potassium, Total Protein, Sodium and Urea) showed sigma value of >3 and ≤ 6 and the performance of five analytes i.e. ALT, Uric acid, AST, Triglyceride and Cholesterol was found to be poor with <3 σ in the IQC analysis (Table/Fig 4). Reasons for the potential cause and effect for the low sigma value(<3) of some analytes are illustrated in Cause-effect Chart (Fish-bone Diagram) (Table/Fig 5).
The list of analytes with low sigma value for accuracy and precision problems were tabulated in (Table/Fig 6). As per their QGI values at both the levels. At Level-1, out of five analytes, inaccuracy (QGI >1.2) was the problem detected for the three analytes (AST/SGOT, Uric Acid and ALT/SGPT) and for Cholesterol both inaccuracy and imprecision (0.8< QGI <1.2) was major problem for lower sigma value. Similarly, at Level-2, inaccuracy was problem of four analytes (AST, Uric Acid, ALT and Triglyceride) and Cholesterol had both inaccuracy and imprecision issue for low sigma values.
The RCA was performed for the 5 analytes having poor sigma value (<3 σ) in IQC and the corrective measures were performed accordingly. Findings of the analysis and corrective measures taken were illustrated in (Table/Fig 7).
(Table/Fig 8) illustrated the post corrective measurement of sigma value of the poorly performed electrolytes. The sigma values for ALT, Uric acid, AST was raised from 2.39, 2.24, 2.09 to 3.76, 3.40, 4.06 respectively at Level-1 after corrective measures. Similarly, post corrective values of all the five analytes were raised from 2 sigma value to more than 3 sigma value at Level-2.
In context to Good Laboratory Practice (GLP) every individual laboratory needs to design an Individualised Quality Control Plan (IQCP) with application of six-sigma methodology which is considered as a gold standard (11). It will not only help in maintaining the precision and accuracy of the tests performed for various analytes but also prognosticate the upcoming error(s), which are expected to be reflected in the patients test results or control values of the laboratory. Regulating the level of sigma to 6 SD will reduce the defects as low as 3.4 Defects Per Million (DPM) (12). A sigma level more than 3 SD is always desirable across the all industries (10). To make a clinically important decision, the TEa for the various analytes has been set universally and are used to measure the quality control for the analytes in the laboratory (13).
For low sigma values showing wide variation, the methodology should be re-evaluated along with a strict compliance of Westgard multirule and number of QC run should be increased to avoid the discrepancy (6). Like the Total Quality Management, the sigma model pursues a Plan, Do, Check, Act cycle. The salient features of six sigma metrics are Define, Measure, Analyse, Improve and Control which are dominant in current quality management ensuring superior patient care by ruling out the recurrence of defects (6).
In this study, 16 biochemical analytes were analysed on sigma metrics and standardised the QC sigma charts for both the levels. A world class performance for Creatinine, ALP and Glucose at QC level-1 and for HDL and Bilirubin at QC level-2 were observed with sigma metrics of more than six sigma. Similar world class performance at level-1 for ALP, Glucose and Creatinine was found in the study conducted by Maksane SN et al., (14) and for HDL and bilirubin was found in the study conducted by Emekli DI et al., (15). The analytes with poor performance i.e. <3 σ were AST, Uric acid, ALT and Cholesterol at., QC level-1 and AST, Uric acid, ALT, Cholesterol and triglycerides at QC level-2. A like analysis of poor performance of AST, ALT and total cholesterol was found in the study conducted by Verma M et al., (16) and Kumar BV and Mohan T (2).
To extemporise the performance of the parameters showing sigma value less than 3, present study used a strategy that combined QGI analysis with RCA. QGI with value >1.2 was seen for AST, uric acid and ALT, in both QC levels, which indicates inaccuracy in measurement. For Cholesterol QGI value between <1.2 and >0.8 for both the QC levels, shows the problem lies in accuracy and precision. For Triglyceride, the QGI value between <1.2 and >0.8 in QC level-2 indicates there is both inaccuracy and imprecision in the measurement while in level 1 the sigma value was satisfactory. The corrective action was taken for those parameters following the Westgard rules (6). The RCA was carried out with considering 5 factors as shown in Cause-effect Chart (Fish-bone Diagram) for the determination of potential cause and effect on the low sigma values of some analytes.
RCA in terms of QGI for poor performers (AST, ALT, Uric Acid, Cholesterol, Triglycerides) revealed inaccuracy and impressions as the cause for poor performance (13). Temperature fluctuation was also found to be a major culprit as we used enzymatic reagents like AST and ALT as in the study conducted by Goel P et al., (17). For parameters like Uric Acid, Cholesterol and Triglyceride a very stringent IQC is for at least 90 QC data points omitting the outliers and new lab mean and SD was setup which was narrowed than the previous LJ charts.
Working conditions and instrument proficiency could also affect measurement quality, and these problems cannot be ignored, as the environmental temperature stood to be a game changer (constant indoor ambient temperature) during the study, as this will impact both the efficiency of the instruments and used enzymatic methods for the analysis.
So, to overcome these issues and increase the proficiency of instrument, the frequency of calibrating these analytes could be increased from once a week to every two days in the laboratory (3). The study conducted by Hens K et al., also concluded that the importance of application of six sigma matrices as there is lack of inconsistent TEa targets values for many analytes (18).
When a laboratory quality performance is validated against Westgard rules or any other quality criteria guidelines, the two vital factors to be picked up for consideration are probability for rejection and probability of error detection. For achieving world class quality, it is desirable to have a high probability of error detection and a low probability of false rejection (19).
The probability of false rejection (Pfr) describes the only inherent impression or random error of the method whereas the probability of error detection (Ped) describes the analytical errors along with inherent random errors (20). Thus, it is quite important to choose the specific QC procedure which will minimise the false rejection and maximise the error detection. The primary concern of all clinical laboratory is to provide the accurate test results. This can be achieved by implementation of six sigma metrics in everyday analytical process as it will be helpful for the laboratory to make their QC strategy and plan QC frequency to produce the accurate test results (5).
The study reveals that sigma metrics is a reliable quality tool to assess the analytical performance of a clinical chemistry laboratory; even though the result of proficiency testing material values were within statistical limit, there were some poor performances detected (by 5 parameters) by using of Six Sigma metrics. These parameters would have been the outliers in the upcoming quality control program which got detected under below 3 sigma value.
Limitation(s)
All biochemical parameters, other than routine clinical testing were not included in the study due to low sample load size and cost-effectiveness.
Sigma metrics has proved to be an excellent self-assessment tool in addition to the IQC program for better analysis of various test parameters to meet the performance potential of biochemistry analysers in the clinical laboratory. The inaccuracy and imprecision of different parameters in the analytical phase of the testing process can be addressed by applying Westgard Rules, after calculation of the sigma metrics and do the RCA. In addition to CV% of clinical parameters, all clinical laboratories should implement Sigma metrics to enhance the laboratory quality performances. A larger study including all parameters of Central laboratory Investigation, starting from raise of tests investigation till the initiation of patient treatment should be analysed for Six-Sigma metrics for enhancing the quality of treatment.
DOI: 10.7860/JCDR/2022/56691.16718
Date of Submission: Mar 27, 2022
Date of Peer Review: Apr 29, 2022
Date of Acceptance: Jun 13, 2022
Date of Publishing: Aug 01, 2022
AUTHOR DECLARATION:
• Financial or Other Competing Interests: None
• Was Ethics Committee Approval obtained for this study? Yes
• Was informed consent obtained from the subjects involved in the study? No
• For any images presented appropriate consent has been obtained from the subjects. NA
PLAGIARISM CHECKING METHODS:
• Plagiarism X-checker: Apr 05, 2022
• Manual Googling: Jun 06, 2022
• iThenticate Software: Jul 29, 2022 (13%)
ETYMOLOGY: Author Origin
- Emerging Sources Citation Index (Web of Science, thomsonreuters)
- Index Copernicus ICV 2017: 134.54
- Academic Search Complete Database
- Directory of Open Access Journals (DOAJ)
- Embase
- EBSCOhost
- Google Scholar
- HINARI Access to Research in Health Programme
- Indian Science Abstracts (ISA)
- Journal seek Database
- Popline (reproductive health literature)
- www.omnimedicalsearch.com