An introduction to expressing uncertainty in measurement

 

Table of contents

Source: Content taken from a Powerpoint presentation used for conferences
Author: Mr. Ouellette
Date: 2002-10-25

Uncertainty basics: an introduction to expressing uncertainty in measurement

Overview

  • Why the big deal about uncertainty?
  • Background stats: The basics — nothing fancy
    • Minimal stats jargon
    • Minimal algebraic notations
    • No system modeling
    • No calculus
    • Promise!
  • How to apply it (on the back of an envelope)
  • A worked example

Why the big deal?

  • 17025
  • VIM (re. Traceability)
  • No stated uncertainty = No traceability!

Don't get sucked in!

  • ESTIMATE uncertainty only to the extent that you need, don't make a science of it.
  • Accept that you'll never be certain about uncertainty.
  • Focus mostly on the "biggies"; don't sweat the small stuff (much).

The background stats: the standard deviation

  • Repeat measurements aren't identical.
  • Standard deviation tells us how widely they are dispersed about the mean.
  • Approx. 2/3 (i.e., 68%) of all readings fall within 1 standard deviation of the mean.
Differences from the mean
 
  • Approx. 95.5% of all readings fall within 2 standard deviation of the mean.
Differences from the mean
to calculate standard deviation
to calculate standard deviation continued

Experimental standard deviation of the mean

  • The standard deviation describes the spread of values in an individual set of measurements. What if we took several different sets of measurements?
  • The mean of each set of measurements would vary.
  • The spread of the means is given by the experimental standard deviation of the mean (stdm).
  • To predict stdm from a set of n replicate measurements, divide the standard deviation by √(n).
    Stdm = stdev /√(n)

Experimental standard deviation of the mean: an example

  • If standard deviation, Stdev, is 5.93, and
  • if number of replicate measurements, n, is 10, then
  • experimental standard deviation of the mean, stdm, is

Stdm

= Stdev / square root(n)
= 5.93 / square root(10)
= 1.88

Experimental standard deviation of the mean: So what?

The stdm feeds directly into our uncertainty budget whenever we estimate uncertainty in the mean of a set of random repeated measurements.

Very useful! More on that later

Distributions of Measurements

The spread of a set of measurements can take on different frequency distributions.
Examples:

Distributions of measurements: normal distribution gaussian
Distributions of measurements: uniform distribution rectangular
The spread of a set of measurements can take on different frequency distributions

Distributions of measurements: normal distribution (Gaussian)

  • Most values fall near the mean.
  • Progressively fewer values falling further from the mean.
  • Very common distribution in nature.
Distributions of measurements: normal distribution gaussian

Distributions of measurements: uniform distribution (rectangular)

  • Measurements distributed equally across the interval.
  • Reasonably good choice when you don't have a clue!
Distributions of measurements: uniform distribution rectangular

Estimating uncertainty step 1

Identify the contributors to uncertainty

Make a list of contributors. Consider:

Reference standards & measurement equipment; e.g.

  • Uncertainty in their calibration
  • Long term drift
  • Stability during measurement
  • Resolution & quality of their scales
  • Linearity
  • Digitization
  • Sensitivity to transportation & handling
  • Design issues (such as unequal lengths of equal arm balances)
  • Parallaxes
  • Interpolation between calibration points
  • Reading system
  • Transporation

Not a complete list. Not all necessarily apply.

Environmental conditions; e.g.

  • Absolute temperature
  • Time variance in temperature
  • Spatial gradiant in temperature
  • Heat radiation from operator, etc.
  • Humidity
  • Noise, vibration
  • Contamination
  • Illumination
  • Air pressure, composition, flow
  • Gravity
  • Electromagnetic interference
  • Transients in power supply

Measurement setup; e.g.

  • Warming up
  • Cosine errors
  • Optical aperture
  • Parasitic voltages
  • Current leakage
  • Loading effects
  • Cabling, shielding, filtering
  • Stiffness/rigidity of mechanical systems
  • Properties of measurement probes

Measurement object; e.g.

  • Sensitivity to stresses of measurement
  • Surface roughness
  • Conductivity
  • Weight, size, shape
  • Magnetism
  • Stability
  • Cleanliness
  • Internal strength
  • Distortion during measurement
  • Self heating (e.g., current measurement)
  • Number of terminals
  • Orientation

Measurement process; e.g.

  • Repeatability, conditioning
  • Number & order of measurements
  • Duration of measurements
  • Choice of principle of measurements
  • Magnetism
  • Alignment
  • Choice of reference & apparatus
  • Clamping, fixturing, probing
  • Drift check
  • Reversal measurements
  • Multiple redundancy checks
  • Strategy

Software & Calculations; e.g.

  • Rounding
  • Algorithms
  • Number of significant digits in calculation
  • Sampling
  • Filtering
  • Interpolation
  • Extrapolation
  • Outlier handling

Definition of the Measurement Characteristic; e.g.

Definition of the Measurement Characteristic

E.g., definition of "Diameter" for dimensional measurements of objects that are not perfectly round.

D= 0.5 (Max + Min)?
D= Mean of many measurements?
D= Other? (e.g., Bullets —> Max diameter?)

 

Physical Constants and Conversion Factors; e.g.

  • Uncertainty in knowledge of the physical values used
  • Temperature coefficient
  • Power coefficient
  • Various properties of the working, measuring instrument
  • Various properties of ambient air
  • Local force of gravity

Metrologist Effects; e.g.

Differences in judgements of different operators.

Has system stabilised
How would you integrate shoulder peaks?

How would you integrate shoulder peaks?

Note: Shouldn't need to include uncertainty due to "Operator Error." This is an aberration that should be corrected before measurement.

Summary of Some Contributors to Uncertainty

  • Reference Standards & Measurement Equipment
  • Environmental Conditions
  • Measurement Setup
  • Measurement Object
  • Measurement Process
  • Software & Calculations
  • Definition of the Measurement Characteristic
  • Physical Constant & Conversion Factors
  • Metrologist Effects

Step 2: Decide on the Uncertainty Units

Units may be:

  • ppm or % of result
  • units of measurement
  • other

Generally doesn't matter what you pick; just be consistent: Apples + Apples = Apples.

Everything gets boiled down to a standard deviation of the mean or equivalent (called standard uncertainty).

Step 3: Estimate the Magnitude of the Uncertainty Contributors

1) Type A, Normal

Type A contributors to uncertainty are those that you have statistical data for. Use this data if you have it.

E.g., For Standard Uncertainty in the mean of repeated measurements (preferably 10 or more) use stdm, the experimental standard deviation of the mean.

2) Type B

Type B contributors to uncertainty are those that you have no statistical data for; e.g.,

  • Manufacturer's specification
  • Professional judgement
  • Uncertainty in cal certificate for your reference standard

GUM: There's "no substitute for critical thinking, intellectual honesty, and professional skill."

Gather information from wherever we can.

Boil down the uncertainty info to a standard uncertainty.

But info doesn't always come at level of confidence of 68.3%. It usually comes at a higher level of confidence. It's called an Expanded Uncertainty when coming from a normal distribution.

2) Type B, Normal

For a normal distribution, no problem!

Simply divide the Expanded Uncertainty by a Divisor, depending upon the level of confidence at which the Expanded Uncertainty is given.

Divisor Level of confidence
0.676 50%
1 68.27%
1.645 90%
1.960 95%
2 95.45%
2.576 99%
3 99.73%

E.g., Cal certificate for our mass standard states:

"The uncertainty in the reported mass is ±26 mg at a level of confidence of 95% assuming a normal distribution."

The standard uncertainty is … ±26 mg ÷ 1.960 = ±13 mg

Divisor Level of confidence
1.960 95%
2 95.45%
2.576 99%
3 99.73%

E.g., Cal certificate for our resistor states:

"The expanded uncertainty in the reported resistance is ±30 Ω with a coverage factor, k, of 3."

The standard uncertainty is…

±30 Ω ÷ 3 = ±10 Ω

Here, the "coverage factor" is another word for "divisor".

2) Type B, Uniform

But not all distributions are normal.

Rectangular (I.e., Uniform) distributions also very common.

Equal probability that true value is anywhere between interval a- to a+.

Distributions of measurements: uniform distribution rectangular

For Rectangular (I.e., Uniform) distributions, the Standard Uncertainty, u, is the area shaded.

Standard Uncertainty, u, is the complete interval, a, divided by √3 or ˜1.73.

Distributions of measurements: uniform distribution rectangular

E.g., Gauge readability (resolution).

gauge readability

If smallest subdivision is 5 measurement units, and if we can "eyeball" the gauge to within 1/2 a subdivision, then the gauge's true value can be anywhere in the interval a = ±5 / 2 = ±2.5 with equal probability.

Standard Uncertainty,

u, = ±2.5 / √3 = ±1.4 meas. units
or, if you prefer,
u = ±5 / √12 = ±1.4 meas. units.

E.g., Ambient temperature

Approximately equal probability that temperature is anywhere between control limits a+ = 24 °C and a- = 22 °C.

Standard Uncertainty, u,±= ±1 °C / √3 = ±0.58 °C

Equal probability

But remember that we need to express all uncertainties in the same units!

If we're expressing uncertainty in ppm of reading of a resistor, and if temperature coefficient of resistor is 0.5 ppm/ °C, then std uncertainty becomes:

u = ±0.58 °C x (0.5 ppm per °C) = 0.29 ppm of Ω reading.

standard uncertainty

2) Type B, Unknown Distribution

Clueless about the distribution?

Usually safe to treat it as a Rectangular (Uniform) distribution

Standard Uncertainty, u, = ±(Max Interval width) / √3

E.g., An incomplete instrument specification:

"The anvils of the micrometer are certified to be parallel within ±0.02 mm."

No coverage factor or level of confidence,

No distribution is specification. based on a standard deviation?

Standard Uncertainty, u, = ±0.02 mm / √3 = ±0.012 mm

Step 4: Document, Document, Document

Your uncertainty budget is a living document that is revised as your measurement process changes and as your understanding of it improves. Therefore…

Write down how you arrived at each of the uncertainty estimates that you have listed.

File this info with the uncertainty budget.

Recap on Steps 1 to 4

1. Identify the contributors to measurement uncertainty.

2. Decide on consistent uncertainty units.

3. Estimate magnitude of each uncertainty contributor, and express each as a standard uncertainty.

4. Document the basis for your estimates.

The hardest part is done!

Step 5: Combine the standard uncertainties into one number

Use the RSS method:

  • Take the square of each standard uncertainty, u
  • Add up the squares
  • Take the square root of the sum

This gives combined standard uncertainty, uc, of your measurement, with ˜68% confidence.

E.g., standard uncertainty, u, in ppm of reading:

  1. Cal of ref standard: 0.75 ppm
  2. Long term drift of ref standard: 1.16 ppm
  3. Repeatability, n=5: 0.07 ppm

Combined standard uncertainty, uc, = square root (0.752 + 1.162 + 0.072) = 1.4 ppm

i.e., ˜68% confident that true value is within ±1.4 ppm of the reported result.

Step 6: Expand the combined the standard uncertainty

What if we want to report uncertainty with a different level of confidence rather than ~68%?

No problem! Simply multiply combined standard uncertainty, uc, by an appropriate coverage factor, k. The product is called Expanded Combined Uncertainty, Uc.

Coverage factor, k Level of confidence
0.676 50%
1 68.27%
1.645 90%
1.960 95%
2 95.45%
2.576 99%
3 99.73%

Exercise: Combined standard uncertainty uc, is 1.4 ppm of rdg;

We want to report uncertainty with a level of confidence of "approximately 95%."

Expanded Combined Uncertainty,
Uc, = 1.4 ppm x 2 = 2.8 ppm

Coverage factor, k Level of confidence
0.676 50%
1 68.27%
1.645 90%
1.960 95%
2 95.45%
2.576 99%
3 99.73%

Step 7: Reality check

The preceding simplified approach is based upon several assumptions, including:

Dominant contributor(s) to uncertainty are known with reasonable certainty;

  • For Type A: Have enough data (≥10 points)
  • For Type B: No wild guessing

So identify the dominant contributor(s).

  1. Calibration of reference standard: 0.75 ppm
  2. Long term drift of reference standard: 1.16 ppm
  3. Repeatability, n=5: 0.07 ppm

Dominant contributor is drift. If wild guess or only few data points, then get a more reliable (otherwise a more conservative) estimate if possible. Otherwise follow detailed steps in the GUM on degrees of freedom.

  1. Calibration of reference standard: 0.75 ppm
  2. Long term drift of reference standard: 1.12 ppm
  3. Repeatability, n=5: 0.07 ppm

Small n (of 5) for repeatability is good enough because this contributor isn't dominant.

Focus on the biggies: Long term drift & reference standard calibration.

Step 8: Reporting the result

Not Recommended…

"The measured result is 10,000.051Ω ±2.8 ppm for a level of confidence of approximately 95%, k=2."

Better…

"The measured result is 10,000.051Ω ±0.028Ω for a level of confidence of approximately 95%, k=2."

Better yet…

"The measured result is 10,000.051Ω ±0.028Ω. The reported uncertainty is expanded using a coverage factor k=2 for a level of confidence of approximately 95%, assuming a normal distribution."

Recap on all Steps

  1. Identify the contributors to measurement uncertainty.
  2. Decide on consistent uncertainty units.
  3. Estimate magnitude of each uncertainty contributor, and express each as a standard uncertainty.
  4. Document the basis for your estimates.
  5. Combine the standard uncertainties.
  6. Expand combined std uncertainty to represent desired confidence.
  7. Reality check.
  8. Report the result.

A more Complete Example Uncertainty Budget

For cal of a 10 kΩ resistor by voltage intercomparison in oil
Source of uncertainty Value
±ppm
Probability
distribution
Divisor Standard uncertainty
± ppm
Repeatability 0.071 Normal 1 0.071
Calibration uncertainty of standard resistor 1.5 Normal 2 0.75
Uncorrected drift of standard resistor since last calibration 2.0 Rectangular √3 1.155
Effect of temperature of oil bath 0.5 Rectangular √3 0.289
Effect of voltmeter on measurement of standard resistor 0.2 Rectangular √3 0.115
Combined standard uncertainty, uc empty space Assumed normal
(k=1)
empty space 1.418
Expanded combined uncertainty, uc empty space Assumed normal
(k=2)
empty space 2.9

Where the numbers came from

  • Repeatability from 5 random repeat measurements in ppm from nominal: +10.4, +10.7, +10.6, +10.3, +10.5 —> stdm of 0.071 ppm.
  • Cal of standard resistor from cal certificate saying uncertainty in reported result is ±1.5 ppm at a level of confidence of approximately 95% (k=2); 1.5 ppm/2 = 0.75 ppm.
  • Drift of ref standard is based upon several years of calibration history and comparisions with a check standard. A trend line was fitted to the historical data and was used to correct the ref std's value at the date that it was used. The estimated uncertainty in the correction is ±2 ppm based upon an analysis of the regression residuals.
  • Temperature effects were estimated from the control limits of the stirred oil bath (±0.2 °C) and the manufacturer's specs for the resistors' temperature coefficient (worst case 2.5 ppm/°C). Bath homogeneity was measured and found to be negligible (‹ 0.05 °C).
  • Voltmeter effects are limited to linearity and resolution because it is used as a transfer device only. According to the manufacturer's specifications, the combined effects of linearity & resolution are ±0.2 ppm in the range of measurement. The uncertainty is considered twice: once for each resistor.