old man willow

Should we take their word for it? Patient-reported vs observed functional outcomes


Pain is a subjective experience. There’s no direct means to measure what it is like to be in pain. Disability, or the impact of pain on what we do in daily life is, on the other hand, able to be observed as well as reported on by people with pain. A question that has always fascinated me is the relationship between what an individual says they can or can’t do, versus what they can be observed to do.

One of the more common ways to determine disability by observation is the “functional capacity evaluation” – and readers of my blog will know that I’m not especially fond of them when they’re used to determine “work fitness”. There are instances though, where they make sense – particularly if the individual with pain is able to gain confidence about what he or she can do. Just don’t use them to predict whether a person can carry out a full working week.

Alternatively, and for reasons of convenience and cost, self-report measures of disability are often used. These are thing like the Oswestry Disability Index (Fairbank & Pynsent, 2000), or the Oxford Scale for hip and knee function (Dawson, Fitzpatrick, Carr & Murray, 1996).  These pen and paper assessments are completed by patients, have excellent psychometric properties, and are often used for both assessment and outcome measurement.

The question is: do they represent the real world experience of the person with pain? How accurate are they, really?

In the Orthopaedic Surgery Department where I work, physiotherapists conduct a functional assessment of all people who have been referred for hip and knee joint replacement. The activities used are aligned with the Oxford hip and knee questionnaires. What’s interesting is how poorly the two are correlated. Physiotherapists recognise that many people report difficulty performing things like going up steps, kneeling and so on, but when they are asked to do those same movements in the clinic, they have excellent motor function and manage them well. Why? and which version of the “truth” do we use?

I don’t have the answer, because physical performance and self-reported disability are influenced by many different factors, but I am interested that in a recent study conducted by the IMPROvE Study Group in Denmark, researchers found that using AMPS (Assessment of Motor and Process Skills – an assessment approach developed and used by occupational therapists) and the SF-36 Mental Composite Score showed distinct differences in outcomes six months after a two-week interdisciplinary pain management programme for people who met the 1990 American College of Rheumatology classification for fibromyalgia.

What does this mean?

The first point is that the programme these people attended demonstrated good outcomes in the things that matter most: daily activities, self care and motor performance. Participants became more able to do the important occupations that you and I need to do each day.  Group content doesn’t look any different from the pain management programmes I’ve seen around the world. Lots of information, sharing experiences with one another, exercise including how to increase daily exercise, developing ways to return to usual activities of daily living, and some psychology sessions as well as an opportunity to meet with the rheumatologist. Interestingly, two weeks is not enough time to develop changes in muscle strength – yet function and activity levels increased. It’s not a “training” effect, at least not physical training. Two weeks, 35 hours a week, is not an enormous amount of time in which to deliver a programme of this type – but it’s clear it can and does make a difference for people who had fibromyalgia diagnosed roughly 1 – 3 years ago.

The second is that the two outcome measures are not measuring the same thing.  Using the SF-36, change was minimal, and not clinically significant, neither were changes at six months statistically significant. Using AMPS, however, individuals demonstrated a clinically relevant improvement in performance in both motor ability and process ability (ADL). Interestingly, the researchers report that there were large interindividual differences – something that grouped data cannot show.  Self-report measures show what the person believes he or she can do.  AMPS provides external validation of performance, or what the person will do, or does do when observed. And from this study, it looks like AMPS is more sensitive to change, and to individual differences, than the well-known SF-36. AMPS also provides information on the quality of performance, something that is missing from the SF-36.

In terms of the research design, the assessors, all occupational therapists trained in AMPS and who had just recently been recalibrated (checking to see how consistent and how closely they remain true to the original training) did not know which of the two groups individuals had come from (control or programme).

From this study, I draw a couple of conclusions:

  1. Using both observed performance and self-report gives a broader picture of the impact of pain on daily life. That the two measures don’t correlate well is fine – they’re measuring different things. It’s good to be reminded that what a person says they have trouble with doesn’t necessarily reflect what they can do in usual setting, nor the quality of the performance. Both measures are useful, and it might be worthwhile using both. An unanswered question, of course, is what the participants thought. How did they view their performance? It would have been nice to see something like the Canadian Occupational Performance Measure included in the outcome measures.
  2. While group analyses may not show much difference, within the group, some individuals responded very well. It’s worthwhile considering some different analyses that can show the variation within a group – and then start the search to find out the characteristics of those people who responded well, and this is another reason I like the single subject experimental design approach to clinical outcome measurement.
  3. Occupational therapists, trained to understand occupation made a significant and unique contribution to this study. Many disciplines fail to recognise the specialist expertise of occupational therapy: this study shows that by assessing and focusing on activities that matter to individuals, and directly measuring the impact of interventions on occupation, a more detailed analysis of progress is achieved. Time to wave the occupational therapy flag I feel!

In answer to my question, should we take their word for it? I think the answer is – not exactly. We need to spend time understanding the differences between self-report and observed performance, not as a way to ‘catch a person out’, but as a way to more adequately assess what matters to people who experience chronic pain.

Amris, Kirstine, Wæhrens, Eva E., Christensen, Robin, Bliddal, Henning, & Danneskiold-Samsøe, Bente. (2014). Interdisciplinary rehabilitation of patients with chronic widespread pain: Primary endpoint of the randomized, nonblinded, parallel-group IMPROvE trial. PAIN®, 155(7), 1356-1364. doi: http://dx.doi.org/10.1016/j.pain.2014.04.012

 

Dawson J., Fitzpatrick R., Carr A., Murray D. (1996). Questionnaire on the perceptions of patients about total hip replacement surgery. Journal of Bone and Joint Surgery, 78:185-90

Fairbank JCT & Pynsent, PB (2000) The Oswestry Disability Index. Spine, 25(22):2940-2953

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s