Journal Club: screen, select, probe & evaluate
2 Department of skin and sexually transmitted diseases, PSG Medical College Hospitals, Peelamedu, Coimbatore, Tamil nadu, India
G R Kanthraj
"Sri Mallikarjuna", # HIG 33 Group 1, Phase 2, Hootagally KHB extension, Mysore -570018, Karnataka
|How to cite this article:
Kanthraj G R, Srinivas C R. Journal Club: screen, select, probe & evaluate. Indian J Dermatol Venereol Leprol 2005;71:435-440
AbstractPostgraduate dermatology training programs like seminars, panel discussions, and case presentations help residents to acquire knowledge. Journal club (JC) exercises help residents to update themselves with the current literature. What article a resident should choose and how a resident should evaluate and analyze an article or critically appraise a topic are issues that are most relevant for the success of a JC. Little guidance is available in the biomedical literature on how to deal with such issues. The objective of this article is to provide guidance to neophytes on dealing with JC exercises in a way that helps them in learning the critical appraisal skills. A review of the literature and of the author's experience in JC exercises will be presented. Knowing the methodology of rapid screening of articles along with the art of evaluating them, coupled with a sound knowledge of epidemiology and bio-statistics, helps a resident to select appropriate articles and discard poorly conceived or designed topics that may not generate interest in JC attendees. Hence, such an approach helps the resident in acquiring new knowledge in the shortest time. Choosing the right topic and then applying the newly obtained information to clinical practice, participants succeed in making the JC a valuable learning experience. Further, such well-formatted JCs help residents to improve the quality of health care delivered to patients.
Journal Club (JC) is an established academic exercise in the residency training curriculum and a method of continuing medical education for over 100 years. The concept of the JC originated in Europe, and in 1875, Sir William Osler organized the first North American JC at McGill University, Montreal., This exercise was basically aimed at sharing the expense of costly periodicals with his colleagues. Today JC is an established method to train residents to obtain critical appraisal skills, update knowledge on current literature and enhance proficiency in clinical practice. ,, Why do clinicians have to keep abreast with current medical literature? A recent judgement of Supreme Court of Israel on a malpractice issue resulting from the failure to keep updated illustrates the importance of continuing medical education. A wrongly diagnosed case of multiple sclerosis underwent surgery for what was thought to be a brain tumor. The court ruled that the physicians were negligent for not updating themselves with current medical developments and such an update would have led them to the correct diagnosis in the instant case.
The mere fact that research reports are published even in the most prestigious journals is no guarantee of their quality. Critical appraisal refers to the skill of presenting a paper in an objective and structured pattern giving emphasis to the quality and validity of the evidence. A resident should be able to differentiate between what is already known on a particular topic and what the given study adds. Residents who know the art of critical appraisal are well versed with methods and conclusions of published articles and begin to view them with a critical eye and gain good knowledge on epidemiology and biostatistics.,
What are the types of JC? There are three ways in which a JC can be organized: current journal style, topic-based style and evidence based style. In the current journal style of JC, the presenter chooses a list of current journals and presents relevant papers, while in the topic based style the most important papers on a chosen subject in the past five years are presented. In the evidence-based style of JC, a clinician describes a clinical problem at the end of one JC meeting. At the next JC, usually 4-6 systematic reviews are presented on the subject and the evidence is evaluated with the result that a topic is critically appraised.
A member of the faculty identifies a clinical problem from a recently seen case. A resident is asked to select articles that address the clinical problem. The faculty and residents then critically appraise the selected articles. Under close faculty supervision, the resident leads the discussion and at the end, they come with suitable suggestions for managing the given case. Consequently it helps in delivering the highest quality of health care.
How should a resident rapidly read and select an article for a JC? The selection of articles for a JC is a not too easy for a resident. The challenge is to quickly choose a relevant article from the numerous dermatology journals displayed on the library racks. Currently there are more than 50 journals in dermatology, and many libraries stock only the more important ones. The resident may not be able to read all the journals or may find it difficult to trace all the articles published on a particular topic. Therefore, residents must develop the necessary skills to keep up with the vast amount of information available. As a resident advances in his training level, the number of journals they scan also increases., The choice for an article varies from one medical center to another.
Choosing an article by a resident depends on topics of ongoing research projects or most frequently encountered cases in that center. It may be an endemic disease like leprosy, an advanced investigation like phototesting, or subspecialty interests like dermatosurgery or pediatric dermatology. Apart from scanning dermatology journals, a resident should not miss dermatology articles published in non-dermatology journals like the Lancet, British Medical Journal, New England Journal of Medicine, and Journal of the American Medical Association. As pointed by Kanthraj and Siddalingappa, a periodical Medline or Internet search using Pubmed to locate all recent articles on a particular disease or treatment modality in all journals helps prevent missing some dermatology articles published in non-dermatology journals. This is a rapid and revolutionary aid to teaching and patient care. Now with the help of the internet a resident can easily and freely access the tables of contents of all important journals.
Bigby and Gadden have developed a shortcut method of reading scientific articles. The steps involved in the rapid selection for an ideal journal article are shown in [Figure - 1]. In the elimination phase the resident reads the titles of all articles; if they are interesting he will continue to read the abstract, which summarizes the various sections of the article concisely. If the abstract is interesting, he should look at the figures and tables, where the most convincing data are represented. The graphical representation helps residents to summarize the results in a short time. The way the data has been summarized and displayed can be verified by looking at the figures and tables and then reading the results. This involves noting the appropriateness of the control group, relevancy of the outcome with reference to the clinical and biological importance, and statistical analysis of the data. Further, the resident should also note any treatment complications, and the follow-up and compliance.
After reading the results, a resident should ask himself: Does the article teach the resident the critical appraisal skills? Does the article have an impact on clinical practice? Does the article help to keep up with the current literature? If the answer is "no", he should stop reading the article, but if "yes", he should proceed to read the materials and methods.
While reading the methods he should note the eligibility criteria, methods of allocation, sample size, variables like the control group, blindness to treatment and statistical methods. If he is convinced he should note the conclusions, and read the discussion and introduction in that order [Figure - 1]. In this way, a resident can choose a high-impact article in a very short time and discard a poorly conceived or designed study or a topic that may not generate interest to the faculty. Such a monthly exercise of choosing such articles itself will be an art and in turn help in his continued reading.
How should a resident probe and evaluate an article?
An imperfect study that is well described in its limitations, and well presented and discussed, can result in an excellent report. Conversely, a model experiment can be misunderstood because of a poor presentation. A resident should be able differentiate between the quality of research conducted and the quality of the research report. One has to analyze the various sections of the article. We have modified the manuscript quality assessment instrument proposed by Goodman et al. This quality assessment instrument aids to identify whether the authors have described their research in enough detail and with sufficient clarity under each section of the article, so that the reader can independently judge the strengths and weaknesses of the data and conclusions. The manuscript quality assessment instrument serves as a guideline that provides information to identify the necessary items that have to be described under each section of the article [Table - 1].
Evaluating the scientific quality of an article is notoriously difficult. The CONSORT statement (consolidation of standards for reporting a clinical trial) has been proposed to ensure that randomized controlled trials are properly reported., Randomized trials allow valid inference of the cause and effect and effectively eliminate bias. Bigby and Gadanne have modified the scoring system proposed by Nyberg for evaluating a clinical trial [Table - 2]. A resident has to score the article to reconfirm his selection. Out of a maximum score of 20, a score of 16 or more indicates a good or very good study, 11 to 15 indicates a fair study whose results have some credibility, while less than 11 is not acceptable and means that the results need to be confirmed by a better designed study.
Statistical analysis is a vital area where medical research reports are concerned. According to a recent study, dermatology journals infrequently perform statistical reviews of submitted manuscripts. Therefore one has to evaluate such manuscripts carefully before coming to conclusions. Hence the need for skepticism. Residents should be encouraged to go through publications on "Guidelines for statistical reporting in articles for medical journals,", clinical epidemiology and research methods, and meta-analysis of the literature.
We have designed a protocol for evaluating non-clinical trials as well [Table - 3]. While the maximum score is 100, a score of 80 or more indicates a good or very good study, 55 to 79 indicates a fair study whose results can be given some credibility, while a study with a score less than 55 is not acceptable and the results need to be confirmed by a better designed study. We also propose a similar scoring system for evaluating case reports [Table - 4].
Based on the complete review of the article, the cardinal questions a resident should ask and analyze at the end of JC have been proposed and illustrated earlier by Kanthraj and Siddalingappa. Choosing a timely topic and then applying the newly acquired knowledge in an actual work situation, participants succeeded in making the JC a valuable learning experience. A properly organized JC helps residents in delivering quality care to their patients and can make Sir William Osler proud!
Linzer M, Brown JT, Frazier LM, DeLong ER, Siegel WC. Impact of a medical journal club on house staff reading habits, knowledge and critical appraisal skills. A randomized control trial. JAMA 1988;260:2537-41.[Google Scholar]
Linzer M. The Journal Club and Medical Education: over 100 years of unrecorded history. Postgrad Med J 1987;63:475-8.[Google Scholar]
Dirschl DR, Tornetta P, Bhandari M. Designing, conducting and evaluating journal clubs in orthopedic surgery. Clin Orthop Relat Res 2003;413:146-57.[Google Scholar]
Sidorov J. How are internal medicine residency journal clubs organized, and what makes them successful? Arch Intern Med 1995;155:1193-7.[Google Scholar]
Gibbons AJ. Organizing a successful journal club. Br Med J 2002;325:137-8.[Google Scholar]
Fishman RH. Israel's doctors obliged to read journals. Lancet 1998;352:1765.[Google Scholar]
Goldschmidt PG, Colton T. The quality of medical literature: an analysis of validation assessments. In: Bailar JC, Mosteller F, editors. Medical uses of statistics. Waltham MA: New Eng J Med Books; 1986. p. 370-91.[Google Scholar]
Grant J, Cottrell R, Cluzeau F, Fawcett G. Evaluating "payback" on bio medical research from papers cited in clinical guidelines: applied bibliometric study. Br Med J 2000;320:1107-11.[Google Scholar]
Patrick C, Alguire MD. A review of journal clubs in postgraduate medical education. J Gen Int Med 1998;13:347.[Google Scholar]
Sandifer D, Lo SV, Crompton PG. Evaluation of a journal club as a forum to practice critical appraisal skills. J R Coll Physic 1996;30:520-2.[Google Scholar]
Arndt KA. Information excess in medicine: overview. Relevance to dermatology, and strategies for coping. Arch Dermatol 1992;128:1249-56.[Google Scholar]
Valentini RP, Daniels SR. The journal club. Postgrad Med J 1997;73:81-85.[Google Scholar]
Burke DT, DeVito M C, Schneider JC, Julien S, Judelson AL. Reading habits of physical medicine and rehabilitation resident physicians. Am J Phys Med Rehabil 2004;83:551-9.[Google Scholar]
Bernhard JD. Read the Lancet, or else! Arch Dermatol 2000;136:602-5.[Google Scholar]
Kanthraj GR, Siddalingappa K. Journal Club: Can faculty be role model for residents? Indian J Dermatol Venerol Leprol 2002;68:115-7.[Google Scholar]
Bigby M, Gadenne AS. Understanding and evaluating clinical trials. J Am Acad Dermatol 1996;34:555-90.[Google Scholar]
Yancey JM. Ten rules for reading clinical research reports. Am J Surg 1990;159:533-9.[Google Scholar]
Anderson BE, Marks JG, Miller JJ. Continue reading. Arch Dermatol 2001;137:1105-6.[Google Scholar]
Goodman SN, Berlin J, Fletcher SW, Fletcher RH. Manuscript quality before and after peer review and editing at Annals of Internal Medicine. Ann Intern Med 1994;121:11-21.[Google Scholar]
Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 1996;276:637-9.[Google Scholar]
Altman DG. Better reporting of randomized controlled trials: the CONSORT statement. BMJ 1996;313:570-1.[Google Scholar]
Katz KA, Crawford GH, Lu DW, Kantor J, Margolis DJ. Statistical-reviewing policies in dermatology journals: Results of a questionnaire survey of editors. J Am Acad Dermatol 2004;51:234-40.[Google Scholar]
Bailar JC, Mosteller F. Guidelines for statistical reporting in articles for medical journals. Ann Intern Med 1988;108:266-73.[Google Scholar]
Ferraris VA, Ferraris SP. Assessing the medical literature: Let the buyer beware. Ann Thorac Surg 2003;76:4-11.[Google Scholar]
Lozano JM, Ruiz JG. Reading a journal article. Indian J Pediatr 2000;67:55-62.[Google Scholar]
Zodpey SP. Meta-analysis in medicine. Indian J Dermatol Venereol Leprol 2003;69:416-20.[Google Scholar]
Seago BL, Turman LU, Horne AS, Croom P, Cary K. Journal club with a mission. Bull Med Libr Assoc 1994;82:73-4.[Google Scholar]