The industry must exercise caution and understanding of basic procedures which have been part of the testing world for decades. All other specifications for other petroleum products make reference to ISO 4259 for dispute resolution. Even the jet fuel specification is not based on absolute limits. To prevent confusion and embarrassment, the industry should provide clear procedures for dealing with single results falling within the grey area as defined above. With the high cost of bunker fuel nowadays, the stakes remain high.
The function of any specification is to control the quality of the fuels manufactured and to ensure the supply of a grade reasonably constant in its characteristics. The role of any specification or regulation is to measure independent properties by the prescribed test method and with adequate precision, which is of paramount importance. Test method precision and accuracy have important implications in product quality control and quality assurance.
Laboratory Accuracy and Precision and Fuel Quality:
To explain the difference between accuracy and precision of any test method, please see the four figures opposite. The dots indicate the numerical value of test results, with the centre being the true value of a fuel’s property tested.
-
Fig.1: All results (dots) are in the centre, which means good accuracy. Also dots are very close together, which means good precision.
- Fig.2: The dots are close together, as in Fig.1, so there is good precision. However, the dots are to the left and off the central point which means that there is poor accuracy, with results away from the true value.
- Fig.3: The results are almost in the centre, so the accuracy is good. However they away from each other, which means that differences between test results are very wide, giving poor precision.
- Fig.4: This shows the worst scenario, where both the accuracy and the precision are poor.
Accuracy is defined as the ability to get the correct result. Precision is the ability to produce repeated results with small differences between them. Accuracy can also be defined as the agreement of the average value of a set of measurements on the same material with an accepted value for that material (i.e. a minimum amount of biased deviation). Precision is the agreement of each value in a set of measurement on the same material with the average value of the set (i.e. a minimum amount of standard deviation).
Standard ISO 4259: 1992 ‘Petroleum Products – Determination and application of precision data in relation to methods of tests (incl.Tech Cor 1:1993)’ governs the
interpretation of the test results obtained against specification and regulation limits taking into account the precision data. The most important aspect of everyday testing is the fact that in the majority of cases the laboratory would produce only one result, called by ISO 4259 a single result. Even if one laboratory produced two or three results, we still would not really know which of the above four figures the laboratory performance would be matching. A particular laboratory may have a consistent bias, which can be due to operator or equipment error. In such cases, the results obtained will be repeatable but far from a true value (i.e. not accurate like they appear in Fig.2). The possible bias of a laboratory can be established only when another laboratory produces another result. Ideally we want good precision and good accuracy, which is not always easy with some test methods. None of the test methods produce absolute or true values or true results. True result, as defined by ISO 4259, represents the average of an infinite number of single results obtained by an infinite number of laboratories.
Occasional human mistakes or omissions will introduce a random error, which can be quantified for every test method and are called
Repeatability (r), being the maximum allowable difference between the two results obtained by the same operator, on the same sample, using the same test method and equipment with the sample tested at the same time, would, in the long run, in the normal and correct operation of the test method, exceed the allowable difference only in one case in twenty (95% confidence).
Reproducibility (R), being the maximum allowable difference, between two single and independent results obtained using the same test method by different operators, working in different laboratories using identical test sample, would, in the long run, in the normal and correct operation of the test method, exceed the allowable difference only in one case in twenty (95% confidence).
The precision data statement with 95% confidence simply says that 95% of the time the differences between duplicates can be expected to be less than a certain figure. Consequently it implies that 5% of the time the differences can be expected to exceed that figure.
So the little (r) is the statement intended to answer the question ‘How well can I expect to agree with myself?’ and the large
(R) is the statement intended to answer the question ‘How well can I expect to agree with other laboratories?’ These are mathematical statements couched in the language of probability, which can be easily misunderstood and misinterpreted. In the real world we do not expect to get exactly the same results when we repeat a test, any more than we would expect to draw exactly the same card from a deck on two successive draws.
Each test method contains precision data consisting of the values for repeatability and reproducibility. Without the precision data the test method would be rendered useless and could not be used in any specification or regulation, as without the precision data there would be no guarantee, promise or assurance that the results produced are acceptable.
The International Maritime Organization’s (IMO) MARPOL 73/78 Annex VI Sulphur Regulation 14 (1), implemented on 19 May 2005, stipulates the maximum allowable global sulphur limit for marine fuels of 4.5% m/m. Regulation 14 (4a) stipulates the maximum allowable sulphur limit in marine fuels burned in a Sulphur Emissions Control Area (SECA) of 1.5% m/m.
No standard test methods are specified in MARPOL Annex VI. It seems logical and correct to suggest that the assessment of sulphur content should be carried out in accordance is ISO 8754:2003 ‘Petroleum Products – Determination of sulphur content – Energy-dispersive x-ray fluorescence spectrometry’, which is stipulated in marine fuel specification ISO 8217.
Testing Annex VI samples for sulphur content must be carried out by one internationally agreed test method, as different test methods produce different results and have different precision data. In view of the fact that in the real world a laboratory would produce a single result, some examples of calculation are presented here to show the interpretation of a single test result in relation to specification/ regulation limits using stipulations of ISO 4259.
Clause 9 of ISO 4259, entitled – ‘Quality
Good Accuracy Good Precision
control against specification’, provides information to allow the buyer and the supplier of fuel to judge the quality of a product by introducing a quality safety margin, with regard to the specification or regulation limit when a single result is available.
Calculated Single Results:
A buyer, who has no other source of information on the true sulphur value than a single test result, shall consider that the fuel fails the Annex VI stipulations, with 95% confidence, only if the test result is such that
Test result > Regulation limit + (0.59 x Reproducibility).
Example 1: Fuel supplied to the maximum
sulphur limit of 4.5% m/m,
ISO 8754 reproducibility.
- R = 0.055 (x + 0.8), where x = average result (a single result).
R = 0.2915% m/m. Test result > 4.5 + (0.59 x 0.2915) = 4.6719 = 4.67% m/m.
Consequently, if the single test result is greater than 4.67% m/m, then the buyer can consider the sample to fail the regulation with 95% confidence.
Example 2: Fuel supplied to the maximum sulphur limit of 1.5% m/m ISO 8754 reproducibility R = 0.055(x + 0.8).
- R = 0.1265% m/m.
Test result > 1.5 + (0.59 x 0.1265) = 1.5746 = 1.57% m/m.
Consequently, if the single test result is greater than 1.57% m/m, then the recipient can consider the sample to fail the regulation with 95% confidence. A supplier, who has no other sources of information than a single result shall consider that the fuel meets the regulation maximum limit with 95% confidence, only if the test result is such that
Test result < or = regulation limit – (0.59 x Reproducibility).
Example 3: A supplier has delivered fuel to Annex VI regulations with a maximum sulphur limit of 4.5% m/m.
- R = 0.2915% m/m (as calculated above).
Test result < or= 4.5 – (0.59 x 0.2915) =
4.5 – 0.1720 = 4.3280 = 4.33% m/m.
Consequently, if a single test result is less than or equal to 4.33% m/m, then the supplier can consider the result to meet the regulation with 95% confidence.
Example 4: A supplier has delivered fuel to Annex VI with maximum sulphur limit of 1.5% m/m.
- R = 0.1265% m/m (as calculated above).Test result < or = 1.5 – (0.59 x 0.1265) =
1.5 - 0.0746 = 1.4254 = 1.43% m/m.
Consequently, if a single test result is less than or equal to 1.43% m/m, then the supplier can consider the result to meetthe regulation with 95% confidence.
Calculated Two Results:
If the supplier and buyer cannot reach an agreement about the sulphur content of a fuel, Clause 10 of ISO 4259, entitled ‘Acceptance and rejection rules in case of dispute’, gives the appropriate procedures to be followed. The procedures only cover the situation where each laboratory is analysing fuel samples that are subdivisions from one representative sample.
In addition to provision of the supplier’s safety margin calculations, clause 9.1 of ISO 4259 states that ‘a reported value between the specification value and the calculated limit is not proof of non-compliance’. Such cases would need to be further investigated by additional testing of the original representative sample.
In this scenario, Lab One and Lab Two would produce one result each, marked here as L1x1 and L2x1. Both results are accepted when their difference is less than or equal to test method reproducibility: L1x1 – L2x1 < or = to R. The average of these two results shall be considered as the estimated value of the sulphur content.
However, when L1x1 – L2x1 > R, both results are suspect and each laboratory will need to obtain at least three other results.The procedure and calculation for the resolution of a dispute based on further testing is fully described in ISO 4259.
Single Result Grey Area:
Fuel supplied to the maximum sulphur limit of 4.5% m/m:
ISO 4259 stipulations clearly indicate that there is a grey area in the range of results between 4.5% m/m and 4.67% m/m, where the single result cannot indicate the true value. It is recommended therefore that all results falling into the grey area should be re-tested by another laboratory. The supplier should be informed about the first single result obtained on testing an Annex VI sample and the sample should be re-tested to confirm the first result.
Fuel supplied to the maximum sulphur limit of 1.5% m/m:
ISO 4259 stipulations clearly indicate that there is a grey area in the range of results between 1.5% m/m and 1.57% m/m, where the single result cannot indicate the true value. It is recommended therefore that all results falling into the grey area should be re-tested by another laboratory. Again, the supplier should be informed about the first single result obtained on testing an Annex VI sample and appropriate action taken.
Conclusions:
Using the ISO 4259 standard for the interpretation of analytical test results for compliance with the Annex VI maximum sulphur limit, the following can be concluded:
For 1.5% m/m maximum sulphur limit In accordance to ISO 4259, the buyer can consider the sample complies with MARPOL 73/78 Annex VI Sulphur Regulations with 95% confidence, if the single test result is less than 1.57% m/m.
For 4.5% m/m maximum sulphur limit Similarly, the buyer can consider the sample complies with the Annex VI Sulphur Regulations with 95% confidence, if the single test result is less than 4.67% m/m.
It seems logical that since compliance with Annex VI regulations involves the use of a test procedure which does not produce absolute values, then it would be incorrect and statistically flawed to use the sulphur limit as an absolute limit when judging it on the basis of a single result.
The statements contained above address the real life situation and are meant to create a debate leading to a solution that would leave the industry without any ambiguity whether the ship is or is not compliant
the regulations. Currently, it would seem that some in the industry are thinking of the sulphur limit in absolute terms, some are aware of the meaning of the test method’s precision data and some others are not even aware of this issue at all.
In order to resolve any future disputes, when the Port Authority finds a suspicious sulphur result, it is suggested that the line of compliance process should involve confirmation that the sample tested is representative, establishing the accreditation status of the testing laboratory, evaluation of the laboratory certificate for the correctness of the test method used, and arranging re-checking of the initial result by another laboratory.
Please contact Wanda Fabriek for more information. Visit to www.bunkerspot.com see the original article.
|
Prefer to call us?
