Self report or functional assessment – or both?

One of the more vexing problems in pain management is how to measure functional performance. I’ve written before about the problems with functional capacity evaluations (lack of predictive validity, poor reliability and so on), and these problems also apply to assessments of the ability to carry out activities of daily living.

Amongst some of the problems are that there can be a discrepancy between what a person reports he or she can do and what her or she actually does do – and I don’t think this occurs because of some deliberate intent to deceive on the part of the person with pain! I think it’s simply that for so many activities, people are doing them automatically and without thought (habit-driven), or they haven’t done them for so long they simply don’t know what they can or can’t do.

Now I don’t often directly assess activities of daily living.  My focus is on returning to work, so I suppose I assume (yes, I know what that word can mean!) that personal activities of daily living are manageable.  I rely mainly on questionnaire results and self report to estimate the functional limitations both before and after treatment.  After reading this paper, a preprint of an interesting study by Wæhrens, Amris, and Fisher, maybe it’s time to review my practice!  They point out that several authors have concluded that self-report and performance-based measures provide distinct but complementary information about functional ability.

In this study, a group of women with widespread pain, some reaching threshold for fibromyalgia, who had been referred for a pain management programme had their activities of daily living performance assessed in two ways: one was the typical pen and paper questionnaire, this time the Fibromyalgia Impact Questionnaire (Burckhardt, Clark, and Bennett, 1991); and the other an occupational therapy specific measure AMPS, or Assessment of Motor and Process Skills (Fisher, 1993). All participants were assessed prior to participating in the outpatient interdisciplinary pain management programme, and once a completion of the programme and again at follow-up.

AMPS requires that the person carry out two of 85 standardised personal ADL and domestic ADL tasks, calibrated to be at the level appropriate for the person (as well as meaningful and relevant to that person). Some of the activities are preparing meals, shopping, cleaning, doing outdoor home maintenance.

An occupational therapist observes the quality of both motor performance actions and process actions to arrive at a score using a four-point ordinal scale to determine the ease, efficiency, safety and independence observed, for each of the activities. Therapists then use the Rasch-based AMPS computer-scoring software to convert the person’s raw ordinal ADL item scores into two overall linear ADL ability measures, one for ADL motor ability and one for ADL process ability. These overall ADL ability scores are adjusted to take into account the task difficulty and ‘rater severity’ (or how hard the marker marks) using logits and provide a motor score (how much effort or fatigue the person demonstrated) and process score (how efficient the person was observed to be). The process also reveals aspects of independence, and safety, and the scores have been used to determine how much assistance an individual needs to remain safe in that environment.

The AMPS has been standardized on more than 100,000 persons between 3 and 103 years of age internationally and cross-culturally, and psychometric properties of the process have been established. The AMPS has also been shown to be a sensitive outcome measure in intervention studies. I am not AMPS trained, but I have seen therapists using AMPS, and know that it’s been used in pain management for many years as an outcome measure and also for problem detection.

The FIQ consists of 10 items, made up of several subscales including ADL, and including ratings of fatigue, stiffness and ability to work.

The purpose of this study was to establish whether AMPS could be used as an outcome measure in this group of people.  A good measure will be able to distinguish between people with a certain characteristic (in this case, widespread pain or fibromyalgia) and those without – this study found that the majority of the women with CWP/FM had ADL motor ability measures lower than those of healthy women of same age.

The authors make a very important point: this is the first study, based on observation of ADL task performance, to document that women with CWP/FM, overall, do perform ADL tasks with increased effort, inefficiency and/or need for assistance. While women with FM often report that everything they do ‘takes longer’, and they feel ‘old before time’, this is not always captured by simple disability measures like the Pain Disability Index or even the Functional Impact Profile.   The AMPS findings seem to mirror those found in daily diary studies, and reports from couples in focus groups who indicate that the burden of activities of daily living is shared between a couple because of the effect of fatigue and the need for additional breaks.

The authors also found that scores before treatment were stable – this is an important aspect of any measurement process.  If the scores vary before any intervention is carried out, it will be difficult to establish whether the intervention itself had any effect.   At the same time, the measure also needs to be sensitive to real change – the authors indicate that when they “examined the ability of the AMPS to detect change due to intervention, we were able to document a significant improvement in ADL motor ability, providing evidence for the effectiveness of the 2-week group-based interdisciplinary rehabilitation program. A similar change, however, was not seen in ADL process ability.” They point out that this could be because the intervention primarily influenced motor behaviour, such as taking breaks in between actions, which will influence the ‘efficiency’ of the activity – but at the same time, reduce fatigue.

Now one aspect that intrigues me is that there was little to no correlation between self reported ability on the FIQ, and the scores on AMPS.  The authors suggest that this is because the two measures are tapping into two different constructs.  Maybe there is a real difference between what a person reports he or she can/will do, and what he or she actually does do.  I suspect so.  The authors suggest that this means the two different assessments should be used because they measure different things.  I’d have to agree – this finding occurs not only in fibromyalgia and AMPS, but in almost any physical performance measure (yes, even FCE) and any self report disability measure.

The only problem with this is – calibrating people to measure accurately is expensive and time consuming.  Observing people doing REAL tasks is also time consuming and energy demanding.  Do you think health funders will pay for this?  I hope so – because measuring the wrong thing very reliably does not equate to measuring the right thing, even if it takes a bit to make sure it’s done accurately.

Wæhrens, E., Amris, K., & Fisher, A. (2010). Performance-based assessment of ADL ability among women with chronic widespread pain Pain DOI: 10.1016/j.pain.2010.06.008

Burckhardt CS, Clark SR, Bennett RM. The Fibromyalgia Impact Questionnaire: development and validation. J Rheumatol 1991;18:728–33.
Fisher AG. The Assessment of IADL Motor Skills: an application of many-faceted Rasch analysis. Am J Occup Ther 1993;47:319–29.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s