Accuracy of references cited in articles published in Indian Journal of Dermatology, Venereology and Leprology: A pilot study

Introduction

Science advances gradually by the hard work of many investigators. Usually the work of an investigator builds on that of others. Sir Isaac Newton famously said, ′If I have seen further, it is by standing on the shoulders of giants.′ The lists of references at the end of articles testify to this fact. In order to build on others′ work, it is important that their work is correctly noted and interpreted. Only then it would be possible to draw reasonable conclusions. In the present work, we analyzed references cited in articles published in the Indian Journal of Dermatology, Venereology and Leprology (IJDVL) for their accuracy with regard to citation (bibliographic accuracy) and quotation (whether the claimed assertion was correct) as one indicator of the quality of published articles. Such analysis has not been performed for this journal earlier.

Methods

In the May-June 2008 issue of IJDVL, a total of 440 references have been cited. We did not include references cited in the E-IJDVL section (net study, net case, net letters, net quiz) of the journal. Out of the 440 references cited, we randomly selected 50 references using an online random number generator. ^[1] For these references, we were able to obtain 44 complete articles and two abstracts from local library, DELNET facility of the India International Center Library and the Internet. Additionally, in one case, only the citation precision could be verified from the Internet, as the abstract and paper were not obtainable. Three citations referred to books, which were unobtainable. Thus, we examined citation precision in 47 references and quotation precision in 46 references cited in the journal.

We identified and defined different citation and quotation errors [Table - 1] and [Table - 2]. These errors were partially based on and modified from earlier similar work done in Public Health journals. ^[2] Among the citation errors, errors C1-C4 were considered major because their presence made locating the cited paper difficult. Following were not considered errors: missing ′-′ between two words, and use of capital letters. Among the quotation errors, errors Q1-Q4 were considered major because they led to misinterpretation of the cited reference. All citations and quotations were verified against the original. If a single reference was cited more than once in the paper, all quotations were verified.

Results

Results are shown in [Table - 1] and [Table - 2]. Out of the cited references examined, there were 13 references (28.3%; 95% confidence interval [CI] 15.3-41.3%) that did not contain any error. More than one error was present in 22 citations. Fifteen cited references contained both citation and quotation errors, 13 contained only citation errors, and 5 contained only quotation errors. There were a total of 50 citation errors (three major) and 25 quotation errors (21 major) [Table - 3]. Major citation errors were present in two (4.3%, 95% confidence interval [CI] -1.5-10.0%) references and major quotation errors in 18 (39.1%, 95% CI 25.0-53.2%).

The most common citation imprecision was C8 (error in author′s initial or title), which was present 19 times. This was followed by C12 (spelling or grammatical mistake in title or journal name, present 15 times), and C10 (incomplete title of paper, present seven times). Out of the quotation imprecision, Q1 (assertion not in the cited reference) was present most frequently (11 times). This was followed by Q4 (oversimplification or overgeneralization, present six times); and Q2 (assertion contradicted by reference) and Q5 (secondary reference), each present four times. With regard to Q5, we did not check the primary reference and, therefore, cannot tell whether the assertion attributed to the secondary reference was correct or incorrect.

Discussion

List of references in the end of an article is a very important part of any publication. This is so because the cited references form the basis on which the reported work intends to build on. Therefore, it is very important that the cited references are correct bibliographically, as well as assertions attributed to them are accurate. Unfortunately, cited references have been found to be inaccurate in every journal in which they have been examined. In the present work, we analyzed a randomly selected sample of reference citations in papers published in May-June 2008 issue of IJDVL. We examined 47 cited references for citation errors and 46 for quotation errors. We found that 13 cited references (28.3%; 95% CI 15.3-41.3%) were error-free.

Previous studies have also found a high error rate in reference citations. These studies have analyzed references from different medical specialties, such as Anatomy, ^[3] Public Health, ^[2] Ophthalmology, ^[4] Radiology, ^[5] General Medicine, ^[6] Emergency Medicine, ^[7] Surgery ^[8] and Dermatology, ^[9],[10] among others. There is no reason to believe that reference citations are likely to contain more errors in one medical specialty than others. Also, as these papers have aimed to detect somewhat different errors, the error rates cannot be compared. However, in the only two studies published in dermatology literature, one showed that only 36% references were error-free. ^[9] The overall rate of citation errors was 41% and that of quotation errors was 35%. In the other study, ^[10] the overall rate of citation errors was 29% and that of the quotation errors was 21%. Obviously the problem of reference imprecision is not confined to IJDVL, but is widespread. Although the presence of errors in references is one indication of the quality of published articles, the quality of a journal cannot be evaluated or compared with other journals by these numbers only. These numbers do, however, tell that a problem exists and that it is important to improve the reference accuracy.

If a reference contains citation errors, it becomes difficult for the interested reader to obtain full text of the paper. On the other hand, if a quotation error is present, the base on which the entire work intends to build on becomes questionably quoted. The conclusions drawn from such a paper may lose some of their meaningfulness. Consequently, it becomes clearly important that references are correctly cited and quoted.

Probable causes of errors in references may include oversight, rush to publish, or the creating of pressure by believing in the concept of ′publish or perish′. However, these reasons hardly justify the presence of errors. Despite the persistent problem of reference errors, no effective solutions appear to have emerged. Although it may be impossible to develop a foolproof system that ensures reference accuracy, checks can be performed at the following points: the authors, the editors, and the referees. To minimize such errors, the International Committee of Medical Journal Editors asks authors to verify references against the original documents. ^[11] It would be really hard on referees if one expects them to check every reference from papers they are reviewing. One solution could be that, on receiving an article, editors randomly select a few references cited in the submitted paper and ask authors to send full texts of these papers to them after highlighting assertions quoted in the submitted paper. Then the submitted paper, along with these full texts of selected cited references, may be sent to referees. This system is likely to reduce errors as authors would be more careful in citing and quoting references. Other probable solutions could be to ask authors to submit the first pages of cited references, to limit the number of references and to electronically check citations from the respective journal websites, rather than PubMed, which being a secondary source may contain a few errors. Efforts are desirable to improve the accuracy of references cited in the scientific literature.