What are methods and what are results?

We’ve been having a discussion about where the PRISMA flow sheet belongs.  The answer to this is quite easy on the one hand.  PRISMA org state that the flow sheet belongs in the results section of your report. The rationale for this is that it contains the results of the search.

PRISMA results section

The confusion probably arises because the PRISMA flow sheet is also a useful method of helping your reader understand your methods – how you got the set of results. It seems easier to me to separate out what is methods and what is results if I think about writing the methods in the future tense.  If I write about what I will do I can’t include results in there because they didn’t happen yet.

Other things that creep into the method section quite commonly are the results of the critical appraisal.  The method describes how you will undertake the critical appraisal, the results tells me what the critical appraisal revealed.  Similarly the data extraction: in the method I tell you how I will extract the data, in the results i provide you with a summary of that data.

Impact, generalisability and transferability

This blog was triggered by a student who observed…

What I am sensing from my papers is that culture has a huge influence how spiritual care is perceived and carried out. Although, for instance, my study from Uganda may not be directly transferable to a rural village in England, it still has value in causing us to reflect, expand our understanding and develop our practice.

The goal of much nursing research is to improve the lot of our clients and patients.  Sometimes it is appropriate to use a quantitative approach, designed to measure the effect of a treatment or process, to gauge opinion from a large group, or to assess knowledge and attitudes. At other times it is better to use a qualitative approach to explore the experiences of people with regards to their health, illness, treatment or caring experiences.

Research often benefits those who participate in it directly, but it also comes with costs.  Participants give their time and energy, usually without recompense. They may experience an emotional burden as a result of participation. In deciding whether to take part in research a participant must balance the costs and benefits to them and to others.  This is why we provide a participant information sheet that includes information about what will happen to the results of the study afterwards, as well as what the potential benefits and harms to the individual might be.

The value of a piece of research is related to its impact.  Impact can be demonstrated in terms of academic, and economic and societal contributions. Academic impact can be seen in advances in methodology, understanding, and developments of theory.  Economic and societal impact includes the contribution the research makes to increasing the effectiveness of public services, policy, quality of life, and health.  The importance of this is made clear in application for research funding, which must include an outline of the likely beneficiaries of the work, as well as a detailed discussion of how the researchers intend to make sure that those benefits are realised.

The importance of impact, and the ethical and legal protection of potential participants to prevent them from being exploited is connected to the generalisability and transferability of research findings. Research that has no benefit to anyone is unlikely to be ethical or to attract funding.  However, there are big differences between generalisability and transferability, and inequities in how ‘impact’ is assessed when quantitative and qualitative approaches to research are compared.

Generalisability is determined by the researchers as a function of how they conduct and report their study.  It is used to describe how well the findings of a quantitative study can be applied to similar populations.  Ideally the sampling strategy and rigorous methods used in a quantitative study, as well as the precise demographic data reported, mean that a reader can determine that these results are likely to apply to their group of interest (or not).  When a clinician or manager is making a decision that involves redirecting resources, spending money, introducing an innovative treatment, or even discontinuing a service, it is important to have robust evidence. Generalisability therefore can be relatively easily determined.  The population you have is similar, the methods used a rigorous, the results are trustworthy – therefore the results can be used as part of the evidence in making your decision.

Transferability is a bit more of a slippery concept. Transferability is determined by the reader of the research rather than the researcher, and rather than applying to an entire study it may apply more or less to the component parts of the study.  It will help the reader to make a judgement about the transferability of the relevant results of the study if the report provides a good description of the context in which the research was conducted, the methods that were used, and enough raw data, then the reader can make a determination about transferability. Importantly, transferability has a subjective component.  If you, the reader, appreciate the findings are relevant to your situation and context, then they are.

So, in response to the student’s observation.  Yes, the results are transferable, even if your populations are vastly different.