It’s true that ‘unconventional’ studies of any kind don’t get published as readily as conventional RCTs even if those studies are under-powered, have errors in their construction and don’t tell us much of anything. Grrr. Publishing studies from my PhD has been fraught because I chose a form of grounded theory that doesn’t conform to the conventional constructivist or Straussian approach. What, then are we to do?
Two things strike me: first we always need to select a research method to give us the best answer to our research question, not something that will ‘get published’ easily. There are many research questions and RCTs simply don’t answer them all. A quantitative method doesn’t lend itself to ‘why’ questions and inevitably require assumptions about the factors thought to be relevant, the measurement strategy, the underlying theory explaining what’s going on. This doesn’t really help us when we have a new field of study to look at, where there is no clear theoretical explanation, where measures don’t measure what’s relevant. Hence drawing on different designs like mixed methods and qualitative approaches. From a pragmatic perspective, the numbers needed for an RCT are much greater than most clinicians can find unless they’re working in a large research setting (and have a bit of funding!). Nevertheless, ‘pilot’ studies using RCT methods do get published even when they don’t have huge explanatory power, partly because they’re familiar to the reviewers.
The second thing that strikes me is: we need to have good exemplars. These give us a template of sorts to learn how to conduct good research, how to communicate why a particular ‘unconventional’ method is the best way to answer the question, and how to write the results/findings in a way that is compelling.
I’ve written before about the failure of much research in human behaviour and experience to understand that ergodic theorum is violated in grouped statistics. This means we can deeply question the results as they apply to the person we see in the clinic. Ergodicity implies that all people in a group will ultimately follow the same trajectory, develop in the same way over the same time, respond to treatment in the same way and follow the same processes. But clinicians know that some people respond very quickly to a component in a programme, while others don’t.
I recently found this example from Tarko (2005) and cited in Lowie & Verspoor (2019)

OK, ’nuff said. Ergodicity matters.
Choosing the right research strategy begins with having a good research question, and most clinicians have a very good research question: what is the best treatment I can offer this person presenting in this way at this time? The follow-up question is: is this treatment helping? or… to be more precise, which component of my treatment/s are helping?
It’s this question that N=1 or single case experimental designs are intended to answer, and they do it very well.
Here are some great examples of published studies using intensive repeated measures – and we need more of these!
Lydon-Staley, D. M., Zurn, P., & Bassett, D. S. (2020). Within-person variability in curiosity during daily life and associations with well-being. Journal of Personality, 88(4), 625-641. https://doi.org/10.1111/jopy.12515
I included this one because it’s not about pain! And yet it sheds light on something important in pain management. Curiosity is about being intrigued by novel, unfamiliar situations. Curiosity doesn’t flourish when a person is anxious, but does when people are wanting to increase their knowledge and skills, and it’s associated with greater well-being. So it’s something clinicians might want to foster – especially for someone who has become afraid to rely on their body and body sensations. In this study, people were asked to complete a daily diary and do some internet browsing (yay! my favourite thing to do!). After some fairly complex statistical analysis (described in good detail in this paper), the results from 167 people who completed 21 days of daily diary measures and a one-off set of measures showed that being consistently curious is associated with feeling good – AND that doing physical movement practices might enhance curiosity via improving mood. Now that’s worth knowing.
Mun, C. J., Thummala, K., Davis, M. C., Karoly, P., Tennen, H., & Zautra, A. J. (2017). Predictors and social consequences of daily pain expectancy among adults with chronic pain. Pain, 158(7), 1224-1233. http://dx.doi.org/10.1097/j.pain.0000000000000903
Now this study is a big one – 231 people in study one, and 220 people in study two. Cutting to the chase, these researchers found that people who expected high pain in the evening experienced greater pain the next day, even when controlling for current pain intensity. The study also found that morning pain predicted next afternoon social enjoyment but not social stress. And what this means is…. clinicians need to promote joy/fun/positive affect, and to help people reduce their expectations that their pain will inevitably increase or ‘be bad’ – it’s anticipation that seems to influence pain intensity and avoidance. These study designs allow researchers to tease apart the factors contributing to experiences over time. We need more of them!
Hollander, M. D., de Jong, J., Onghena, P., & Vlaeyen, J. W. S. (2020). Generalization of exposure in vivo in Complex Regional Pain Syndrome type I. Behaviour Research and Therapy, 124. https://doi.org/https://doi.org/10.1016/j.brat.2019.103511
And from a large study to a much smaller one with – wait for it – 8 participants! That’s more like the numbers we see in clinic, right? This study examined whether it’s more fruitful to expose people to many activities they’ve previously avoided, or instead, to limit the number of activities each person was exposed to. This is SUCH an important component of therapy where people have avoided doing things that bother them because they anticipate either that their pain will go to untolerable levels (or interfere with other important things like sleep) or because they’re worried they’ll do harm to themselves. Why? Because doing things in one safe space is not life. We do lots of activities in lots of different spaces, and most of them are unpredictable and we don’t have a ‘safe person’ to rely on. It’s perhaps one of the reasons exercise carried out in a gym might not transfer into greater reductions in disability in daily life – and why involving occupational therapists in pain management as ‘knowledge translation experts’ is such a good thing.
Caneiro, J. P., Smith, A., Rabey, M., Moseley, G. L., & O’Sullivan, P. (2017). Process of Change in Pain-Related Fear: Clinical Insights From a Single Case Report of Persistent Back Pain Managed With Cognitive Functional Therapy. Journal of Orthopaedic & Sports Physical Therapy, 47(9), 637-651. https://doi.org/10.2519/jospt.2017.7371
Lucky last – a single case study exploring the process of change experienced by one person undergoing cognitive functional therapy. While recent meta-analyses suggest CFT is ‘no better’ than any other treatment for people with persistent pain, what meta-analyses can’t show is those for whom it’s highly effective. Why? Because individual responses don’t show up in meta-analyses, and the mean or even the confidence intervals don’t show those people who do extremely well – or those who don’t do well at all. And yet as clinicians, we deal with each individual.
Now I chose these four studies because they are all published in highly respected and ‘highly ranked’ journals. I don’t care a fig about the supposed rank of a journal, but there’s no doubt that getting into one of these journals requires research of a very good standard. And somehow these ones snuck through!
Am I suggesting that RCTs shouldn’t feature in research? No – but I do think a much more careful analysis of these is needed, so we can understand the golden question: what works for whom and when? And to answer these questions we need far more detailed analysis. Oh – and evidence-based healthcare has always been a synthesis of THREE elements – research yes, clinician’s experience AND the person’s preferences and values. ALL THREE not just ‘research’ and out of research, not just RCTs.
Lowie, W. M., & Verspoor, M. H. (2019). Individual Differences and the Ergodicity Problem. Language Learning, 69, 184-206. https://doi.org/10.1111/lang.12324
Tarko, V. (2005, December 29). What is ergodicity? Individual behavior and ensembles. Softpedia News. Retrieved from https://news.softpedia.com/news/ What-is-ergodicity-15686.shtml