I’m trying to read around the idea but I’ve found that the topic is not easy to locate in peer-reviewed journals. Perhaps I’m not using the right search terms, but I have found around 3 relevant articles so far… Clearly I’m no health informaticist, and given one of the papers I’ve found, I’m no health economist either!
What I have found today is an article written by Rory O’Connor and Vera Neumann, dating from 2006 and published in the Journal of the Royal Society of Medicine. The authors of this paper describe some of the history around measuring outcomes in health and relate it to changes in the NHS in the UK.
Some of the points they make are about the difficulty with defining ‘quality’, suggesting that there is a natural tension between attention to ‘technical management of illness’ and ‘prevention, rehabilitation, coordination and continuity of care’ – too much focus on the former and the latter suffer with ‘poorer outcome for the overall health community’.
I guess that means that we can design the best treatments – but unless they’re placed into context, they are simply the ambulance at the bottom of the cliff.
I don’t think any health care provider would disagree with the notion that biomedical measures such as laboratory results don’t provide a complete picture of treatment outcome – they need to be supplemented with other aspects of health that are important to the patient. O’Connor and Neumann identify that the problem with patient-centred constructs is that these are ‘fuzzy’ (OK that’s not their word, but you get what I mean!). As they say ‘compared with concrete measures such as blood pressure, constructs such as pain relief, walking ability or depression are complex to measure.’
I couldn’t agree more with their statement that ‘measuring outcome remains the ultimate validation of the effectiveness and quality of healthcare’. I also agree with their next sentence ‘payment by results will fail to meet …objectives… if the wrong outcomes are measured’. I think they need to add that outcomes are not dependent on the actions of only one party – not when it comes to self management, and not when the person is ambivalent about taking action.
The history of interest in outcome measurement in medicine is not new – outcomes have been recorded as far back as the treatment of scurvy by James Lind (1716 – 1794), and the steps taken by John Snow (1813 – 1858) to locate the source of the Broad Street pump cholera outbreak. Early health outcome measurement was limited to recording deaths – I’m glad we’ve moved a little on from here!
But how far have we moved? And who determines the appropriate outcomes to measure?
After all, there are four stakeholders involved in the management of a bout of back pain, especially if it has meant time off work for the injured worker… The person with the pain wants to measure how much his or her pain has dropped and how much he or she can do; the health provider wants to know whether pain has reduced and hopefully that the person is happy with his or her treatment; the employer wants to know whether the person can manage his or her tasks at work; and the funder? Hopefully, the funder wants to know that what is being paid for has been provided, and in New Zealand if it’s ACC, that the person is well on the way to recovering full function.
O’Connor and Neumann discuss the concept of ‘quality’ as being influenced by societal values – as I’ve mentioned, each one of the stakeholders has a slightly different idea of a quality outcome, depending on their own values. Once some common ground has been established on what ‘quality’ is – then comes an even more contentious problem: how to map that concept onto something that can be measured. Something that is robust in terms of psychometrics and can be readily used. Something that takes into account patient preferences, motivation and adherence to the intervention.
The timing of outcome measurement is important – there is a distinct post-intervention ‘honeymoon’ phase for many interventions, and as time goes on, the effects become less distinct. And as one clinician used to say time and again ‘two swallows do not a summer make’ – two measures of outcome won’t give a robust measurement over time, it’s important to have at least three to account for the temporal changes that occur especially with respect to improved mood.
Audits are pretty common across all health care services. These typically focus on the processes used to achieve an outcome. Certainly in the EQuIP4 process currently underway in New Zealand, the whole approach is process-driven. This approach is based on the idea that by making sure all the steps in delivery of a service are carried out well, the best outcomes should achieved – but has anyone checked that these steps are relevant? Are they associated with good outcomes or are they about doing things that are visible, able to be remedied – but are actually tangential to the outcome?
The remainder of this paper is focused on the NHS strategy. The points they make could well apply here in New Zealand too. Some pertinent points:
- volume of activity and speed of service delivery can become surrogates for quality healthcare provision
- limited resources within healthcare facilities to record outcomes make it difficult to obtain relevant measures
- individual departments that do make improvements to service outcome need to receive the benefits of their improvements
- payment by outcome is promoted as one way to increase the choice available to patients – but sadly, it often means that services that address chronic health problems such as chronic pain, made more complex by multiple other health problems or social disadvantages may lose funding. Why? Because their patients don’t attain the kind of results that relatively straightforward patients and conditions may reach. As has happened already with elective surgery funded by private insurers here in New Zealand, private providers choose low risk patients, perform relatively straightforward procedures, leading to reasonably ‘good’ outcomes. Comparing apples with apples? or apples with pears?
I won’t go through all the remainder of this article apart from these points made by O’Connor and Neumann. To make a process like this work, they state the following need to be put in place:
- Data concerning healthcare interventions must be recorded accurately
- measures must be appropriately chosen for the outcome under consideration
- outcome measures must be recorded accurately at the appropriate times
- health care costs should be realistic. Complex multidisciplinary care is expensive. Trying to substitute simplified versions of best practice through fragmenting multidisciplinary teams will only result in inferior care for patients.
- adequate numbers of expertly trained staff are needed to collect and analyse these data.
I thoroughly agree with outcome measurement, and that providers need to be monitored to ensure the treatments are appropriately provided. At the same time, the various viewpoints of each of the stakeholders need to be taken into consideration. AND finally, this needs to be a consultative process, informed not just by the purchasers, not just by the patients, but also by health providers and academics knowledgeable about pain management (for example) who can help ensure measures and processes work well.
O’Connor, R., & Neumann, V. (2006). Payment by results or payment by outcome? The history of measuring medicine. Journal of the Royal Society of Medicine. 99:226-231.