At the PMG conference in 2018, Professor Rory O’Connor from the University of Leeds delivered a thought-provoking session on current issues in the application of outcome measures in clinical settings. He presented an overview on principles of measures, reasons for measuring outcomes, the risks of errors, and gave an overview of modern measurement theory.

When reading research papers, clinicians look for the 'gold standard' in healthcare research which are the multicentre, randomised, placebo-controlled trials with an intention to treat analyses, powered to reject the null hypothesis. Clinicians refer to this gold standard on how to conduct the research, but rarely are the measures that are used in the research based on sound evidence.

In the clinical setting, healthcare professionals often administer health-related quality of life (QoL) measures, which are assessments of a person’s experience of the impact of the health condition on their life. One person’s perception of QoL can be different to another's, even if they are living with the same health condition, as each person’s experience of that condition, their expectations and their goals are all different. So how can clinicians compare individuals’ QoL given the differences in their experiences?

Have clinicians been taking measurements for granted?

A quick look at the history and introduction of measurements shows that it was never a straightforward process. It took several years to introduce common measures we take for granted today, for example the metric system and the measurement of temperature. It took a long time for these to be established as physical measures.

It is important to understand the process of standardising measures to ensure repeatability. This standardisation ensures data can be collected consistently and classified appropriately according to the type of measurement scale  -  nominal, ordinal, interval or ratio. However, measurements can differ because each of us measures in a slightly different way. For example, when measuring a wall for hanging a picture, some people will use a ruler or tape measure, and others will use a rough estimation. Even when using the same measuring device, everyone will use it slightly differently

When choosing an outcome measure for a project, audit, or service development, clinicians should question what it is they intend to change, and match it as closely as possible to what they're going to measure.

The key properties of scales (acceptability, targeting, reliability, validity or responsiveness) should be in place to support basic assumptions but, most importantly, any scale used needs to be relevant. When conducting surveys, clinicians must analyse the variables that can actually be measured. These might range from very easy-to-extract information such as height and weight to more complex variables that are dependent on different assessors and their expertise, such as information on disability, cognitive function, or quality of life.

Clinicians must systematically examine the elements within outcome measures, and consider how to apply these within clinical areas, highlighting the benefits and limitations of a particular measure. For example, Kurtzke (1983) developed a scale to measure the disability status of people with multiple sclerosis. The scale measures from 0 to 10:

• point 0 was considered a normal neurological examination
• 1 - 4.5 measured symptoms
• 4.5 – 9.5 measured signs
• 10 was recorded if the subject died due to MS

Thus the Kurtze scale mixes clinicians’ examination findings, patient self-report and pathological events within the same measure, meaning that different things are measured at different points on the same scale.

Professor O’Connor used the well-known visual analogue scale (VAS) to explain the confusion that exists when trying to measure change without understanding what to measure. When administering the VAS, many people cannot agree what 'normal' is, and many scales are only numbered 1-10, meaning the intervals between points are not defined. These difficulties can also be recognised when administering VAS to record pain levels, tone, spasticity, and difficulties with personal activities of daily living. 

"The outcome of encounter between a person and an item is governed by product of the ability of the person and easiness of the item, and nothing more" (Wright & Panchapakesan, 1969)

Reflection points:

• When completing measurements, there is a wealth of information, plus details and protocols about conducting research, but not what to measure
• Selection of outcome measures - do the questions and items measured actually match what you expect to see in that population?
• Outcome measures - are the questions clear enough to be answered by the measure?
• Are the measurements used appropriate for the setting?
• Are they matching the diversity of the population such as country, language of the population, disabilities, education level, function and level of ability?
• What are the properties of the measure? 
• What are the knowledge, skills and attitudes of the clinicians?

References

Kurtzke, J. 1983. Expanded Disability Status Scale (EDSS). [Online]. [1 June 2019]. Available from: https://www.mstrust.org.uk/a-z/expanded-disability-status-scale-edss 

Wright, B & Panchapakesan, N. 1969. A procedure for sample-free item analysis. Educational and Psychological Measurement. 29, pp 23-48.