Call Us: US - +1 845 478 5244 | UK - +44 20 7193 7850 | AUS - +61 2 8005 4826

Appraisal of evaluative research

Appraisal of evaluative research used in evidence-based health care centres on three major questions. Firstly, is the research good enough to support a decision on whether or not to implement an intervention? Secondly, what are the research outcomes? Thirdly, is the research transferable to the potential recipients of the intervention (individuals or populations)?1

In this paper we ask whether (or to what extent) evaluative research on public health interventions can be adequately appraised by applying well established criteria for appraising evidence about prevention and treatment in clinical practice.2–5 We adduce that these criteria are very useful in evaluating some important aspects of evidence. However, there are other important aspects of evidence relevant to public health interventions that are not covered by the established criteria. We draw attention to these additional aspects of evidence and explain their importance in the assessment of public health interventions. We emphasise the distinction between the appraisal of evidence and the process of making policy or operational decisions on the implementation of interventions. Research-based evidence is only one of several factors to be taken into account in these decisions.

Public health interventions tend to be complex, programmatic, and context dependent. The evidence for their effectiveness must be sufficiently comprehensive to encompass that complexity. The evaluation of evidence must distinguish between the fidelity of the evaluation process in detecting the success or failure of an intervention, and the relative success or failure of the intervention itself. Moreover, if an intervention is unsuccessful, the evidence should help to determine whether the intervention was inherently faulty (that is, failure of intervention concept or theory), or badly delivered (failure of implementation).6 Furthermore, proper interpretation of the evidence depends upon the availability of adequate descriptive information on the intervention and its context, so that the transferability of the evidence can be determined.

To fulfil these requirements, we suggest an expansion of the criteria that are used in clinical medicine for appraising research. We draw on evidence-evaluation schema that were developed for epidemiological and qualitative research, health promotion programme evaluations and health economic evaluations.