Research waste is universal

What is research waste?

Research waste refers to avoidable design flaws, non-publication and inadequate reporting of research. It includes research undertaken without any form of prioritization, research that duplicates previous research unnecessarily, does not assess important outcomes, and is limited by significant biases that could have been overcome by better planning. In other words, it is poor quality research that is not needed and introduces “noise” rather than reducing uncertainties that are needed for patient care.

Contrary to previous assumptions ascribing this phenomenon to a few (or, a lot of) bad apples, it is wiser to acknowledge it as a universal problem, stemming from multisystem failure, the contours of which were systematically outlined by Chalmers and Glasziou in 2009.¹ As one of us (H.C.W.) has delineated recently, clinical dermatological research is no exception as far as research waste is concerned.²

Why should it concern us?

This problem of research waste has particularly pernicious implications for a developing country like India. Taking note of the paucity of funds for clinical dermatological research, the Indian Association of Dermatologists, Venereologists & Leprologists (IADVL) started providing research grants since 2013 on the basis of calls for applications by the IADVL Academy. Let us take a look at the outcome of the IADVL research grants over the years: Out of the 28 projects completed up to December 2021, only 16 have been published. The fate of the IADVL L’Oreal grants, which were initiated in 2010, is hardly any better: Out of 54 completed projects, 35 have been published. Granted that some of the projects are in the publication pipeline, the figure of around 60% of completed projects getting published can hardly be a cause of comfort.

What may be the reason for such worrying numbers for research in India? As someone (S.P.) who has been actively adjudicating these research grants almost for a decade now, the major problems that one has noticed regarding the proposals are: inadequate design; lack of formal estimation of sample size so that one does not have any idea whether the studies have the necessary power to reach valid conclusion; and, finally, lack of clinical relevance so that the studies do not get a high priority for publication in peer-reviewed journals.

Not a very happy picture. Research waste is an undesirable problem wherever it happens, but if and when it plagues a country like India with meagre resources for research, alarm bells should start ringing.

Factors behind research waste

Factors behind accumulation of research waste have been generally identified as lack of prioritization by researchers and funders, use of unvalidated outcome measures,³ duplication of studies (e.g., multiple placebo-controlled studies instead of comparing with an established active agent,)⁴, unnecessary proliferation of systematic reviews and meta-analyses (earning them the unflattering sobriquet of ‘sausage machines’⁵), poor design (in interventional studies, no allocation concealment, no intention-to-treat analysis etc), lack of publication of negative studies leading to publication bias, lack of enforcement of prospective trial registration by journals and regulatory bodies leading to selective reporting outcome bias,⁶ and the use of spin that distorts the interpretation of results.⁷

If we analyze these factors, it is clear that all stakeholders hold responsibility for these failings: untrained or poorly trained researchers; funders failing to properly identify priority areas of research; the wrong approach of academic bodies to reward quantity rather than quality leading to a culture of ‘publish or perish’ and emergence of a section of authors displaying publication addiction⁸; journals failing to root out inadequate scientific reporting; commercial interests vitiating research; and, lack of capability of research users or readers to critically appraise clinical research.

How to minimize research waste

Researchers must make a habit of taking stock of existing research before rushing headlong into undertaking new research. High-quality systematic reviews (e.g., Cochrane reviews) or guidelines are a useful source for identifying research gaps, supplemented by updated searches of bibliographic databases and trial registers.⁹

Just identifying a workable clinical research question is not enough–the results need to be of value to a range of stakeholders including patients and healthcare providers, in our context primarily the Indian dermatological patients and Indian dermatologists. Methods on how to work effectively with a range of stakeholders to identify and prioritize research questions have been developed over the last two decades through organizations such as the James Lind Alliance (JLA). The Alliance produces a guidebook and provides help on how to set up a priority setting partnership (PSP), gathering and verifying uncertainties, and how to prioritize the final ‘top ten’ topics.⁹

Funders can also play a more active role in prioritizing research rather than working solely in response mode. For example, the National Institute for Health Research Health Technology Assessment (NIHR HTA) Programme in the United Kingdom (UK) actively identifies and prioritizes clinical research questions that need answering. It then invites teams to apply in open competition to deliver such commissioning briefs.⁹ Research funders in India, like the IADVL Academy or the newly founded Society for Eczema Studies (SES), could adopt similar commissioned approaches for funding priority topics.

Outcomes in clinical studies need to measure something important to patients, and need to be reliable, valid, sensitive to change and clinically interpretable. Core sets of valid and reliable outcomes that should be used in all clinical trials on a particular skin disease are needed so that studies can be compared and combined. The pioneering work of the Harmonising Outcome Measures for Eczema (HOME) group has been instrumental in this regard.¹⁰ Over 20 other groups are working on developing core outcome sets in diverse dermatological conditions such as skin cancer and incontinence-associated dermatitis, using up-to-date methods supported by the Cochrane Skin Core Outcomes Set Initiative.⁹ Senior Indian researchers–faculty in the academic departments who train would-be researchers and the principal investigators of studies and trials–need embrace such initiatives and incorporate these core outcomes in the studies they design.

Working with methodologists and biostatisticians is key to minimizing bias at the design stage. The Dermatology Clinical Research Special Interest Group (SIG DCR) of the IADVL Academy is providing invaluable resource in this regard since 2013, but more is needed. Understanding basic methods and knowing how to critically appraise a clinical study that could benefit patients is a core competency for clinical dermatologists, every bit as important and as fundamental as learning how to make a clinical diagnosis, how to evaluate a basic histopathology slide or how to remove a viral wart. Unfortunately, this is still not being done satisfactorily in many postgraduate departments in our country. Better training - of the trainers as well as the trainees - is needed.

The easiest way to minimize publication bias is to make it mandatory for studies such as clinical trials to have pre-registered protocols. This journal made it a compulsory requirement for interventional studies as early as in 2017. Every clinical trial submitted in the Indian Journal of Dermatology, Venereology & Leprology (IJDVL) from that time onwards had to have a Clinical Trials Registry–India (CTRI) registration number (or, registered to similar registries elsewhere in case of overseas authors). At the same time, IJDVL made the submission of the Consolidated Standards of Reporting Trials (CONSORT) checklist and flow diagram mandatory for clinical trials. It may sound bureaucratic, but the principle is simple: “Place your bet and show us your hand”.¹¹ Since last year, when the IJDVL opened a new article category of ‘systematic reviews and meta-analyses’, being overwhelmed by the huge number of submissions such articles, it made compliance to the 27-item Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist mandatory and prospective registration of the protocol a desirable criterion. To the best of our knowledge, among the Indian dermatology journals, the Indian Journal of Dermatology (IJD) also has pre-registration of clinical trials and compliance to CONSORT as mandatory criteria for submission. But here the impediment to quality control is, once again, the quantity - this time, of journals. Latest figures indicate that the total number of dermatology journals in India is 22 and worldwide it is over 300. It is now almost impossible not to be able to publish any data, however poor quality, error-ridden, opaque or misleading it be. In order to obviate publication bias, IADVL and the Academy may take a leaf out of the NIHR HTA Programme, making it a contracting requirement for projects to be withheld a certain proportion of the grant funds until full online publication of the reports is completed within a reasonable time frame after completion of the project in peer-reviewed journals of repute.

Conclusion

The purpose of this editorial is to raise awareness about the universality of research waste in dermatology clinical research—a phenomenon that is not only the concern of advanced research societies, but perhaps even more so to the wider clinical dermatology community, where there is palpable lack of systemic encouragement of research, clinical research in particular. Any waste in such resource-poor settings is particularly unconscionable. All the stakeholders—researchers, institutions, funders, journals and research users - need to actively engage in simple measures to reduce research waste. The problem is reversible, and we have indicated a certain road map for solving it. Sometimes, less is more — perhaps, most tellingly, in contemporary dermatology clinical research, where less quantity and better quality and relevance might make it more impactful for our patients and for our society.