Latest
News		 Data Monitoring LAQM Research
   Home > Data > Verification and Ratification Process

The Data Verification and Ratification Process

It is important to Defra and the Devolved Administrations that end users of the UK Air Quality Archive have access to the best possible data at all times. To this end we have put in place a comprehensive system of both automatic and manual data reviews and updates for the UK Automatic Urban and Rural Monitoring Network (AURN) as described below.

Hourly mean monitoring results from the AURN are uploaded as provisional data every hour. These figures undergo some basic screening criteria in order to exclude clearly faulty data as far as possible. However, the objective of the exercise is to provide data for human health concerns on a near real-time basis, so the checks have to be essentially automatic and rapid. This means that full Quality Assurance and Quality Control (QA/QC) procedures cannot be applied and the data are therefore likely to be of lower accuracy and reliability than that required for final reporting. Provisional data and statistics are clearly marked with a flag in the database to indicate their status.

Following the publication of initial provisional data, there are at least a further two stages which all automatic monitoring data are required to go through to meet the standards required for National Air Quality Monitoring networks.

Data Verification is carried out on an ongoing basis and is nominally a process to "clean-up" the initial provisional data. Any corrections to the data made during the verification process are automatically uploaded (still as PROVISIONAL at this stage) to the Air Quality Archive for end-users to access. The process includes:

  • Further manual review of the data to exclude any data from instrument malfunctions or faulty calibrations.
  • Incorporation of any data which were initially missing due to communications failure with a monitoring station.
  • Updates to data scaling following application of the most recent calibration factors.

Data Ratification is a detailed manual check of the data set carried out on a quarterly basis for the AURN .It requires a longer-term view of the dataset incorporating the results from independent QA/QC audits of the monitoring stations.

Data ratification reviews all calibration data, information from analyser services and repairs and any other information available for the particular site or analyser over the whole ratification period. In addition, the results from the independent QA/QC audits are incorporated to take account of any problems detected during the QA/QC audits such as:

  • Long-term drift in an ozone instrument calibration.
  • Faulty NOx converters.
  • Drifts in calibration cylinder concentrations.
  • Instrument leaks or flow faults.
  • Faulty instrument configuration.

Incorporation of the QA/QC audits ensures that ratified data are traceable to UK national and international gas calibration standards.

In addition, data ratification also requires the judgement of experienced air quality scientists who will have to consider the validity of data in the light of many things including:

  • Relationships between pollutants.
  • The impact of air pollution episodes.
  • The context of the results in the overall UK pollution climate.
  • National and regional pollutant patterns.
  • Long-term trends.

Once all the ratification checks and corrections have been made then the data are re-loaded to the Air Quality Archive with a new status flag of "Ratified".

It should however be noted that there are occasionally circumstances where data which have been flagged as "Ratified" could be subject to further revision. This may be for example where:

  • A QA/QC audit has detected a problem which affects data back into an earlier ratification period.
  • Long-term analysis has detected an anomaly between expected and measured trends which requires further investigation and possible data correction. This was the case with 2000-2008 gravimetric particulate monitoring data in the UK national network.
  • Further research comes to light which indicates that new or tighter QA/QC criteria are required to meet the data quality objectives. This may require review and revision of historical data by applying the new criteria.

Any further necessary corrections to an annual data set are, as far as possible, made before the UK results are sent to the European Commission in September of the following year.

In the event that there is a strong case for modifying datasets already sent to the European Commission, this will usually require widespread consultation and agreement before implementation.

An example is the correction of UK gravimetric PM10 monitoring data from 2000 to 2008 which was widely consulted on. The corrected data are now on the Air Quality Archive database and the revised dataset will be submitted to the Commission in September 2009.