Researchers and clinicians are drawn to studies with many participants. Especially randomised controlled trials, where two groups are randomly divided and one gets “the real thing” while the other does not. The joy comes from knowing that results from these kinds of studies suggest that, all things being equal, the differences between the groups is “real” and not just by chance.
When we come to analyse the graphs from these kinds of studies, what we hope to see are two nice bell-shaped curves, with distinct peaks (the arithmetic mean) and long tails either side – and a clear separation between the mean of one group (the experimental one) and the control group.
It should look a bit like this:
Now one of the problems in doing research is that we can’t always guarantee a large sample – for example, it’s difficult to find enough people with a relatively rare problem like complex regional pain syndrome to randomly split the groups to iron out major differences between them. And, this kind of research design presumes the principle of ergodicity – here for more information from Wikipedia, or here for a more detailed examination relating to generalising from groups to individuals.
This research design also struggles to deal with distributions that don’t conform to the lovely bell curve – things like bimodal distributions, or skewed distributions. And if we draw only on the mean – we don’t get to see these delightful peaks and troughs – or the people at either end of the curves.
The more variables we add to analysis, the more complex the statistics needed – so in the case of pain research, we often have to simplify the research question, do complex maths to “normalise” the results, and ultimately we get research that doesn’t look the slightest bit like the people we see in clinical practice. No wonder we get results that don’t look nearly as nice as the research studies!
Now I don’t mind statistics at all, but I do mind research papers that don’t declare the assumptions made when using analyses. Many papers assume the reader knows these assumptions – unlike qualitative research where the authors philosophical assumptions are openly stated, and where epistemology and ontology are considered part of the research design.
So why might lots of data points be cool?
Most of us working in a clinic will be seeing an individual. One person, with all their unique history, attributes, vulnerabilities, preferences and values. When we extrapolate the findings from RCTs especially, and apply them to this unique person, we risk failing to acknowledge that we’re violating the principle of ergodicity, and that our person may be one of those falling at the tails of that bell curve: or worse, in the middle of a bimodal distribution. Given that most pain problems, particularly persistent pain, are multifactorial, applying a single “solution” no matter how many studies showing a positive effect there are, may not cut it.
For years I’ve been pointing out the value, both in research and in clinical practice, of single case experimental designs. There are loads of reasons for using this approach, and it’s a method with a long history. Essentially, the person serves as their own control, they take lots of measurements before introducing a treatment, the treatment is applied and changes in the measurements are closely monitored. If there’s a change in the expected direction, we can test whether it was the treatment by withdrawing said treatment, and closely monitoring any changes in the measurements. Of course, there are challenges to using this approach – we have to be able to withdraw the treatment, and that doesn’t work if it’s something like “information”. But there are ways around this – and the method of intensive longitudinal repeated measures is becoming a rich source of information about change processes.
Change processes are changes that mediate the final outcome. In other words, when we do a treatment, either the treatment directly causes the end outcome – eg give someone a raised toilet seat, and they can get off the toilet because the toilet is at a good height for them – or via some other process – eg by giving the raised toilet seat, the person gains confidence to get on and off the toilet so it’s not the toilet seat per se, but enhanced confidence that mediates the outcome.
Change processes matter because once we’ve identified them, we can develop ways to work with them more effectively. We can also measure the progress a person makes on more than one change process, and refine what we do in our treatments in response. The more data points we collect from that one person, the more we can track their trajectory – and the better we can understand what’s going on in their world to influence their responses.
Technology for repeated measures in real time has become much smarter and more invisible than it used to be. We can still employ some of the simpler techniques – a pen and paper diary still has used! But we then have to rely on the person remembering to fill them in. Passive data collection using wearable technology is something many of us use to track fitness, diet, sleep, travel, heart rate variability and so on. Set the parameters, and as long as you’re wearing the gadget, your data is captured.
Before anyone leaps in to tell me the gadgets are prone to measurement error, believe me I know! For example, monitoring sleep using a phone (or even a smartwatch) doesn’t monitor sleep depth, it monitors movement (and records snoring LOL). However – and this is important – it is more likely to get used than anything requiring me to do something extra in my day. And we can integrate both passive data collection and active data collection using similar technologies. For example, it’s not difficult to send an SMS (instant text message) at random times during the day to ask someone a brief and simple question.
Where these repeated measures approaches get a bit gnarly is in analysing the data – but even this doesn’t mean it can’t be done. The analyses require a good understanding of what it is being measured (and why), and how best to use complex statistical analyses to understand how one factor (variable) might influence another.
The advantages of using intensive measures in clinic lie with understanding how, for example, one day of additional activity (measured using the step counter combined with GPS tracking) might directly influence mood the next day (or pain, or energy levels or whatever). We still need to apply critical thinking to uncover the causal mechanisms (is it plausible for factor X to directly cause a change in factor Y?) and to check whether the results are stable over time (or just a chance fluctuation). Another advantage is that we can quickly step in to experiment with an intervention – and watch what happens. For example, if we think being very active on one day has an effect on mood the following day, we can test this out: try experimenting with a day of lots of activity, and monitor what happens the next day, or the converse, do very little and monitor what happens with mood the following day. Rinse and repeat until we’re certain that for this person, activity level has an effect on mood.
And the study that made me think about all this? It’s this one by Whibley, Williams, Clauw, Sliwinski and Kratz (2022) – click
If we want to really develop excellent clinically-relevant research-based ways to understand what might be going on for the one person in front of us, and not for the large mixed group of people included in a randomised controlled trial, we could be inspired to look at intensive repeated “micro-longitudinal” research strategies as models for clinic-based research.
Whibley, D., Williams, D. A., Clauw, D. J., Sliwinski, M. J., & Kratz, A. L. (2022). Within-day rhythms of pain and cognitive function in people with and without fibromyalgia: synchronous or syncopated? Pain, 163(3), 474-482. https://doi.org/10.1097/j.pain.0000000000002370
HealthSkills Blog 
I think this is the first time I have got excited about statistics and research designs. I’m working with someone to make a topical treatment for CRPS including evidence-based (for treating CRPS) DMSO and vitamin C in the ingredient list. I was wondering how to test this product. Perhaps this is the way? Thanks again Bronnie – these are simply brilliant blogs.
I’ll post more about this in the New Year, including some analytic strategies, and some relatively simple ways to randomly prompt people to make an entry. It will give more insights into what happens in daily life, and offer clinicians and researchers a much richer understanding of relationships between what happens in context and what people do in response.