ASTM D6299 Evaluate Analytical Measurement System Performance
ASTM D6299 Standard Practice for Applying Statistical Quality Assurance Techniques to Evaluate Analytical Measurement System Performance
7. Quality Assurance (QA) Program for Individual Measurement Systems
7.1 Overview - A QA program (1) can consist of five primary activities: (1) monitoring stability and precision through QC sample testing, (2) monitoring accuracy, (3) periodic evaluation of system performance in terms of precision or bias, or both, (4) proficiency testing through participation in interlaboratory exchange programs where such programs are available, and (5) a periodic and independent system validation using VA samples may be conducted to provide additional assurance of the system precision and bias metrics established from the primary testing activities. At minimum, the QA program must include at least item one.
NOTE 8 - For some measurement systems, suitable check standard materials may not exist, and there may be no reasonably available exchange programs to generate them. For such systems, there is no means of verifying the accuracy of the system, and the QA program will only involve monitoring stability and precision through QC sample testing.
7.2 Monitoring System Stability and Precision Through QC Sample Testing - QC test specimen samples from a specific lot are introduced and tested in the analytical measurement system on a regular basis to establish system performance history in terms of both stability and precision.
7.3 Monitoring Accuracy:
7.3.1 Check standards can be tested in the analytical measurement system on a regular basis to establish system performance history in terms of accuracy.
7.3.2 For measurement systems where calibration is established by using multiple standards of known values, such as materials certified by or traceable to the national certification bodies such as NIST, JIS, BSI, and so forth, and where the total number of standards used exceed the number of parameters estimated by the calibration equation, an alternative approach (instead of check standard testing) to infer system accuracy is to compare the statistics associated with the calibration equation to previously established measurement system precision and to standard errors of the calibration standards used. Coverage of this type of statistical techniques for accuracy inference is beyond the scope of this practice. Users are advised to enlist the services of a statistician when using this approach to infer system accuracy instead of check standard testing.
7.4 Test Program Conditions/Frequency:
7.4.1 Conduct both QC sample and check standard testing under site precision conditions.
NOTE 9 - It is inappropriate to use test data collected under repeatability conditions to estimate the long term precision achievable by the site because the majority of the long term measurement system variance is due to common cause variations associated with the combination of time, operator, reagents, instrumentation calibration factors, and so forth, which would not be observable in data obtained under repeatability conditions.
7.4.2 Test the QC and check standard samples on a regular schedule, as appropriate. Principal factors to be considered for determining the frequency of testing are (1) frequency of use of the analytical measurement system, (2) criticality of the parameter being measured, (3) established system stability and precision performance based on historical data, (4) business economics, and (5) regulatory, contractual, or test method requirements.
NOTE 10 - At the discretion of the laboratory, check standards may be used as QC samples. In this case, the results for the check standards may be used to monitor both stability (see 7.2) and accuracy (see 7.3) simultaneously. If check standards are expensive, or not available in sufficient quantity, then separate QC samples are employed. In this case, the accuracy (see 7.3) is monitored less frequently, and the QC sample testing (see 7.2) is used to demonstrate the stability of the measurement system between accuracy tests.
7.4.3 It is recommended that a QC sample be analyzed at the beginning of any set of measurements and immediately after a change is made to the measurement system.
7.4.4 Establish a protocol for testing so that all persons who routinely operate the system participate in generating QC test data.
7.4.5 Handle and test the QC and check standard samples in the same manner and under the same conditions as samples or materials routinely analyzed by the analytical measurement system.
7.4.6 When practical, randomize the time of check standard and additional QC sample testing over the normal hours of measurement system operation, unless otherwise prescribed in the specific test method.
NOTE 11 - Avoid special treatment of QC samples designed to get a better result. Special treatment seriously undermines the integrity of precision estimates.
7.5 Evaluation of System Performance in Terms of Precision and Bias:
7.5.1 Pretreat and screen results accumulated from QC and check standard testing. Apply statistical techniques to the pretreated data to identify erroneous data. Plot appropriately pretreated data on control charts.
7.5.2 Periodically analyze results from control charts, excluding those data points with assignable causes, to quantify the bias and precision estimates for the measurement system.
7.6 Proficiency Testing:
7.6.1 Participation in regularly conducted interlaboratory exchanges where typical production samples are tested by multiple measurement systems, using a specified (ASTM) test protocol, provide a cost-effective means of assessing measurement system accuracy relative to average industry performance. Such proficiency testing can be used instead of check standard testing for systems where the timeliness of the accuracy check is not critical. Proficiency testing may be used as a supplement to accuracy monitoring by way of check standard testing.
7.6.2 Participants plot their signed deviations from the consensus values (exchange averages) on control charts in the same fashion described below for check standards, to ascertain if their measurement processes are non-biased relative to industry average.
7.7 Independent System Validation - Periodically, at the discretion of users, VA samples may be submitted blind or double blind for analysis. Precision and bias estimates calculated using VA samples test data can be used as an independent validation of the routine QA program performance statistics.
NOTE 12 - For measurement systems susceptible to human influence, the precision and bias estimates calculated from data where the analyst is aware of the sample status (QC or check standard) or expected values, or both, may underestimate the precision and bias achievable under routine operation. At the discretion of the users, and depending on the criticality of these measurement systems, the QA program may include periodic blind or double-blind testing of VA samples.
7.7.1 The specific design and approach to the VA testing program will depend on features specific to the measurement system and organizational requirements, and is beyond the intended scope of this practice. Some possible approaches are noted as follows.
7.7.1.1 If all QC samples or check standards, or both, are submitted blind or double blind and the results are promptly evaluated, then additional VA sample testing may not be necessary.
7.7.1.2 QC samples or check standards, or both, may be submitted as unknown samples at a specific frequency. Such submissions should not be so regular as to compromise their blind status.
7.7.1.3 Retains of previously analyzed samples may be resubmitted as unknown samples under site precision conditions. Generally, data from this approach can only yield precision estimates as retain samples do not have ARVs. Typically, the differences between the replicate analyses are plotted on control charts to estimate the precision of the measurement system. If precision is level dependent, the differences are scaled by the standard deviation of the measurement system precision at the level of the average of the two results.
