Repeatable and reproducible

Example control chart

Written by Andy Connelly. Published 15th May 2017.

Introduction

When you are starting out on any set of experiments it is important to know whether the results you achieve on one day will be the same the next. If your results vary daily due to the weather or are sensitive to your mood then you might want to alter your method. In scientific parlance – you need to ensure that your method is repeatable and possibly even reproducible”.

DISCLAIMER: I am not an expert in analytical chemistry. The content of this blog is what I have discovered through my efforts to understand the subject. I have done my best to make the information here in as accurate as possible. If you spot any errors or admissions, or have any comments, please let me know.

Repeatability

Repeatability is a measure of the changes in precision (and accuracy) over repeated measurements determined by:

  • The same analysts
  • In the same laboratory
  • Using the same instruments
  • Usually on different days
  • Potentially with different raw materials, eluent, etc.

If your measurements are not repeatable and vary day to day then there is serious issue with your experimental set up. If you’re unsure about changes over time it is useful to run a reference material (or check standard) each time you undertake the experiment. The results of this check standard can then be plotted with time. A more formal version of this monitoring can be performed using control charts.

Control charts

These were introduced by Shewhart in 1931 and originally for industrial manufacturing processes. They allow you to monitor for suddenly occurring changes and for slow but constant changes (see Figure 1). This can act as an early warning system for issues with a process. The basic principle is as follows:

  1. Take control samples during the process.
    • a. Assign a target value – e.g. a certified value of a Certified Reference Material (CRM) (if available).
  2. Measure a quality indicator (e.g. concentration).
  3. Mark the measurement in a chart with warning and action limits.
  4. If data are ‘normally distributed’ the warning / action limits are normally taken as (s = standard deviation):
    • ± 2s is taken as warning limits
    • ± 3s is taken as action limit

Clearly, the results for CRMs will still be subject to random error but your results should still be within the specified uncertainty. If they are not repeated measurements may be required to trace the source of this error. The standard could be a CRM or just a representative sample which you have a lot of.

Using these values as limits there is a probability of only 0.3 % that a (correct) measurement is outside the action limits (3 out of 1000 measurements). Therefore the process should be stopped immediately and errors searched for. There is a probability of 4.5% that a (correct) value is outside the warning limits and so this should be monitored closely.

Other common warning indicators are seven successive values on one side of the central line (mean) or seven successive increasing or decreasing values.

Presented here is only a brief overview – you will need to find out more before using control charts.

Example control chart
Figure 1: Example of control chart with acceptable variation.

Reproducibility

If your method is going to be used across laboratories (or you are using a method from another laboratory) you need to check the reproducibility. Reproducibility is a measure of the changes in precision (and accuracy) determined by:

  • Different analysts
  • In different laboratories
  • Using different instruments
  • Usually on different days

A further one is often added to this list – ‘But all using the same stock solution’. However, this will not be relevant in all cases.

Statistical calculations can be carried out to determine how reproducible results are between laboratories. These calculations should be agreed before such testing occurs.

Summary

Repeatability and reproducibility are a small, but important, part of method development for any set of experiments. It is not difficult to check repeatability and will give any reader extra confidence in your data.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s