ILT - Collection of the results andd statistical analysis

The collection of the results and the statistical analysis

The collection of the results

All the laboratories were requested to perform the measurements strictly according to the Guidelines supplied by the Coordinator and more generally to the indications of ISO 140 part 3.
The laboratories were supplied with a floppy disk including a form sheet to be used for reporting the results and further information. The following data were requested, for each specimen:

Name of the laboratory and of the operator who carried out the test;
Date of the test;
Area of the test opening and volume of the two test rooms;
5 series of measurements for the 1/3 octave bands in the interval 100-5000 Hz (eventually also for the 50, 63 and 80 Hz bands);
Maximum sound reduction index R’max for the same frequency bands;
Reverberation times in the receiving room for the same frequency bands;
Environmental conditions (air pressure, air temperature, air humidity);

Following the indications of Annex G of ISO 140 part 3, the reports of the results for each laboratory participating in the I.L.T. were composed (in anonymous form). Here is an example:

The statistical analysis

1) Computational procedure (ISO 140/2 and ISO 5725)

The method of analysis adopted follows the indications of ISO 140 part 2, and more generally of ISO 5725-1986, for a uniform level experiment. In this I.L.T. the levels are the 1/3 octave frequency bands from 100 to 5000 Hz.
Each combination of a laboratory and a level is called a cell of the precision experiment. Ideally in each cell of this I.L.T. there should be 5 replicate results.
It was decided to calculate the repeatability r and the reproducibility R only among the 21 laboratories in which the mounting of the specimens had been overseen by the I.L.T. supervisor.
Regarding the remaining three laboratories, their repeatability and reproducibility values with respect to the other laboratories shall be verified.

2) Preliminary considerations

The following tables must be filled in, as indicated by ISO 5725:

- The collation of the original data;
- The measures of spread for each cell of the previous tables. The formula used for computing the cell spread s_ij is:

where y_ijk is one of the results of that cell.
- The cell averages. The formula used for calculating the cell averages is:

3) Statistical tests for outliers

Outliers are entries that deviate so much from the comparable numbers in a table that they can be considered as irreconcilable with the other data.
According to ISO 5725-1986, the following statistical tests were performed:

Cochran’s test

Cochran’s test is based on homogeneity of variance, under repeatability conditions, and hence it has been applied to Tables 6a & 6b.
It tests only the highest value in a set of standard deviations or ranges and is therefore a one-sided outlier test. If the test statistic on a certain item lies between its 5% and 1% critical values, then the entry is called a statistical straggler; if the test statistic is greater than its 1% critical value then the item is called a statistical outlier. The critical values for Cochran’s test are reported in Annex A of ISO 5725-1986.

Dixon’s test

Dixon’s test applies to a series of data and verifies if the extreme values (the highest or the lowest) are compatible with the other ones.
As for Cochran’s test, if the Dixon’s test statistic on the extreme value lies between its 5% and 1% critical values, then the entry is called a statistical straggler; if the test statistic is greater than its 1% critical value then the item is called a statistical outlier. The critical values for Dixon’s test are indicated in ISO 5725-1986.

4) Repeatability values r

The repeatability value r is the value below which the absolute difference between two single results obtained under repeatability conditions may be expected to lie with a probability of 95 %.
Assuming that the distribution of the random errors occurring in every single test result is approximately normal, the repeatability r can be calculated as follows:

where s_r² is the repeatability variance, that is, the average of the within-laboratory variances s_i² , computed as follows (for convenience, the subscript j which indicates the level is omitted):

Regarding the repeatability of single numbers R_w, it has been computed among the single number quantities related to each series of results.

5) Reproducibility values

The reproducibility value R is defined as the value below which the absolute difference between two single results obtained under reproducibility conditions may be expected to lie with a probability of 95 %.
Assuming that the distribution of the random errors occurring in every single test result is approximately normal, the reproducibility R can be calculated as follows:

where s_r² is the reproducibility variance and s_L² is the between-laboratory variance calculated as follows (for convenience, the subscript j which indicates the level is omitted):

Concerning the reproducibility of single numbers R_w, it has been calculated using the single numbers quantities related to the within-laboratory averages.

6) Verification of repeatability and reproducibility values

According to ISO 140 part 2, it is possible to verify that the repeatability and reproducibility values just calculated actually correspond to a 95% probability, as required by the definition, by means of the data from which they have been computed.

It must be noted that the repeatability and reproducibility values for single number quantities are computed on integer numbers determined by the procedure indicated in ISO 717-1, consequently it is expected that in this case the basic model will not be very suitable.

Checking repeatability values

For each frequency band, 10 absolute differences between two single results y_iand y_k obtained by a laboratory can be calculated and compared with the repeatability r of that frequency band. For 21 laboratories and 18 frequency bands, a total of 3780 comparisons is obtained, and it is expected that the corresponding repeatability values be exceeded in no more than 5% of the cases, i.e. no more than about 190 times.

Checking reproducibility values

The critical difference CrD95 to be used is this case is different from the reproducibility value R, because there are 5 determinations for each laboratory and the verification has to be performed among the within-laboratory averages.

The correct critical difference to be used for each frequency band is (where n is the number of determinations):

For n=1, this reduces to the reproducibility value R as expected.

The critical differences thus calculated for each frequency band are compared with the observed differences of the averages in that frequency band. The overall number of comparisons is 3780 (coincidentally the same as for the repeatability verification).
It is expected that the critical differences be exceeded in no more than 5% of the cases, i.e. no more than about 190 times.

7) Verification of excluded laboratories

According to ISO 140 part 2, the three laboratories which were excluded from the calculation of the repeatability and reproducibility values may verify the proper operation of their own test procedure by using the I.L.T. results.

Verification of repeatability r

The criteria for considering the internal repeatability values of the laboratories to be satisfactory is that (for all frequency bands and for both specimens):

where s_x is the standard deviation of the 5 results and r is the repeatability value determined in I.L.T.

Verification of reproducibility R

The verification of the reproducibility values R of the three laboratories has to be done by comparing their own averages with the total average of I.L.T. in each of the frequency bands and for both specimens.

The appropriate critical difference for this case is:

These critical differences should not be exceeded in more than 5% of the cases, i.e. not more than twice.

Back to the ILT Procedure Page