Three-part scoring system (tripartite) for teledermatology versus International Contact Dermatitis Research Group criteria to interpret patch test readings: A comparative, observational study

Introduction

The art of patch testing and its interpretation underwent several modifications after its introduction by Joseph Jadassohn in 1895. Patch testing is done to identify the allergens causing allergic contact dermatitis (ACD). The standard method of analysing patch test readings is by the International contact dermatitis research group (ICDRG) criteria, a six-part scoring system. An average patch test reading requires four to five visits: (a) to apply the allergen, (b) to remove the allergen and 48^th h reading, (c) 96^th h reading, (d) delayed readings (on day 5 or day 7) and (e) >7th day in special situations.¹ Teledermatology (TD) provides dermatology care using information technology in remote areas and reaches the unreached.² It may reduce the number of visits.³

Teledermatology follow-up care is provided in psoriasis,^4–6 acne,⁷ leg ulcers^8,9 and also in emergency care,^10,11 dermatosurgery¹² and COVID-19.^13–16 Capture and transfer of images is easily performed by smartphones using mobile messenger apps.^17,18

A scoring system is required that suits images and facilitates teledermatology (TD) to interpret patch test readings. The tripartite scoring system (TPSS) is a three-part scoring system that determines the presence of inflammation. This scoring system is based on visual interpretation rather than palpation. This renders it suitable for evaluating the patch test readings based on images. A simplified TPSS grades patch test reactions as either positive, negative or irritant reactions (IR).¹⁹ It minimised interpretation errors and inter-dermatologist variations.¹⁹ Interpretation of patch test results using TPSS may play a vital role in situations like the COVID-19 pandemic. A comparative study between face-to-face (FF) and store-and-forward teledermatology (SAFT) was done to interpret patch test reading. A single dermatologist did the readings at the 48^th h, 96^th h and on the 7th day by using the North American Contact Dermatitis Group (NACDG) criteria.²⁰ We demonstrate tripartite scoring¹⁹ by comparing FF versus TD in a COVID-19 setting.

The present study was performed to compare the tripartite scoring system (TPSS) in tele-dermatology (TD) versus ICDRG grading done face-to-face and assess the feasibility of TPSS by TD to interpret patch test results.

Methods

This comparative study included all clinically suspected cases of ACD seen at our institute, irrespective of age and sex, completing at least two patch test readings. The study was conducted from November 2019 to October 2021 at the Department of Dermatology, Venereology & Leprosy, JSS Hospital, JSSAHER, Mysuru [Figure 1]. A detailed history was noted with respect to the occupation, duration of disease and previous treatment received. Consent form and proforma for TD consultation was prepared as per Indian Academy of Dermatology, Venereology and Leprosy (IADVL).²¹

Close

Figure 1:

Flow chart of the study. ICDRG: International Contact and Dermatitis Research Group, TPSS: Tripartite scoring system

Export to PPT

Confidentiality, medical ethics, security of data and other regulations for TD practice were followed according to the guidelines framed by the National Medical Council and issued by the Ministry of Health and Family Welfare, Government of India.²² Institute Ethical Committee clearance was taken for the study.

Patch testing was done for all the patients using the Indian standard series approved by the Contact and Occupational Dermatoses Forum of India (CODFI). The details of procurement of allergens were as follows: Manufacturer and supplier – Creative Diagnostic Medicare Pvt. Ltd., Mumbai, India; Dilution – 0.5%–100%; Vehicle – Petrolatum; Chambers – Aluminium; Application period – 48 h; Timings of readings – 48^th h, 96^th h and 7^th day.

Two investigators were needed for the study to compare FF readings versus TD readings. The co-investigator (CI) took the FF readings at the 48^th h, 96^th h and on the 7^th day, and assessed them according to ICDRG criteria. The CI captured a baseline photograph of the patch test grid (48 hr. reading), taken under standard light settings, and then transferred this image to the principal investigator (PI) on the same day [Figure 2]. The PI then assessed the image and interpreted them based on the TPSS, which was:

• +ve Erythema, infiltration, papules, vesicles, coalescing vesicles

• −ve Negative

• IR Irritant reaction

Close

Figure 2:

Illustrates the tripartite scoring system (TPSS) to interpret patch test reading for teledermatology practice. Step1 – patient with patch test applied over the back (day 1). Step 2 – patch test grid (read at 48 h). Steps 3–5: Co-investigator captures the images of the patch test grid and transfers to the principal investigator using mobile messenger apps (WhatsApp) at 48 h, 96 h and on the 7^th day. Principal investigator interprets the patch readings as positive (+), negative (-) and irritant reaction (IR). In this case, it is positive (+).

Export to PPT

This process was repeated for the 96^th h and 7^th day visits for each case [Figure 1].

The prevalence of ACD was found to be around 4% (1.7–6%). To calculate the number of cases for the study, we used the sample size calculator provided by https://wnarifin.github.io/ssc/sssnsp.html. With the expected sensitivity and specificity of 0.90 and 0.80, respectively, prevalence rate of 0.04, precision of +0.3, confidence level of 90% and expected dropout rate of 10%, the sample size was calculated to be 76.

Summary statistics are presented by means of proportions for categorical/binary variables and mean, median, standard deviation, and inter quartile range (IQR) for continuous variables.

Inferential statistics calculations were done by using kappa for percentage agreement and sensitivity, and specificity. Negative predictive value (NPV) and positive predictive value (PPV) were measured with 95% confidence intervals (CI) for validity measurement. All the statistical analysis was done using the SPSS 21.0 version for windows. P < 0.05 was considered statistically significant. Kappa statistics is used to compare two categorical outcomes of a diagnostic test/a method through two methods. It gives the percentage agreement between two observers or two methods (when outcome is categorical), taking the chance factor into consideration. This value can lie between 0 and 1. This is classified as excellent if >0.8, good if 0.6–0.8, fair if 0.4–0.6 and poor if <0.4.

Results

There were 78 patients [(Males: 34 (43.6%) and females: 44 (56.4%)]. They were evaluated for both FF and TD on the same day and asked to come for patch test reading at the 48^th h, 96^th h and on the 7^th day.

The number of patients who came for the 48th h reading was 78 (100%). A total of 77 patients (98.71%) came for 48^th h and 96^th h readings. There were 56 patients (71.79%) who completed all three readings – 48^th h, 96^th h and 7^th day.

Only 78 clinically suspected cases of ACD were taken up for the study. Out of 78 patients, 28 had positive reactions, of whom 6 had positive reactions to more than 1 allergen giving rise to 38 positive results (22 were positive to only 1 allergen, 3 were positive to 2 allergens. 2 were positive to 3 allergens, 1 positive to 4 allergens. There was 1 irritant reaction (IR) and 49 negatives leading to a total of 88 patch test results.

In the ICDRG category, at 48 h, a total of 17 patch test results (19.3%) showed positive reaction, 20 (22.7%) were doubtful, 49 (55.7%) were negative and 2 (2.3%) were IR. At the 96^th h, there were 32 positives (36.4%), four doubtful (4.5%), 49 negatives (55.7%) and 2 IR (2.3%). One patient was lost to follow-up (1.1%). On the 7^th day, out of the 87 patch tests that were followed up, there were a total of 30 positives (34%), no doubtful (0%), 35 negatives (39.8%) and 1 IR (1.1%). Twenty-two of 78 patients were lost to follow-up (28.2%) [Table 1].

In the TPSS category, at the 48th h, there were 33 positives (37.5%), 54 negatives (61.4%) and 1 IR (1.1%). At the 96th h, there were 37 positives (42%), 49 negatives (55.7%) and 1 IR (1.1%). One patient was lost to follow-up (1.1%). On the 7th day, there were 29 positives (33%), 36 negatives (40.9%) and 1 IR (1.1%). Same as the ICDRG category, 22 patients were lost to follow-up (25%) [Table 1].

There were 20 allergens in the Indian standard series. In both ICDRG and TPSS categories, the positive allergens detected were 15 [Table 1]. Paraphenylenediamine in 13 patients (14.8%), nickel in 6 (6.8%), neomycin sulphate in 4 (4.6%), parthenium and fragrance mix in 3 (3.4%), parabens and nickel sulphate in 2 (2.3%), thiuram mix, potassium bichromate, nitrofurazone, epoxy resin, cobalt sulphate, Chlorocresol and black rubber mix in 1 each (1.1%) were detected in both methods. An angry back reaction was noted in one patient (1.1%).

The results obtained from both methods were charted and sensitivity and specificity of TPSS were calculated against a confidence interval of 95%. Based on our results, the sensitivity and specificity of TPSS was 100% and 93.4% respectively. The PPV and NPV were 91.7 and 100 with a diagnostic accuracy of 96.5% [Table 2].

The degree of agreement between both investigators was estimated using Cohen’s Kappa, which gave a value of 0.909 (lower limit 0.815 and upper limit 0.978) [Table 2]. The degree of concordance between both investigators with respect to matching of readings (YES/NO) was calculated. A value of 93.2% on YES and 6.8% of NO match were obtained.

Discussion

Applications of various scoring systems for psoriasis along with modifications have been described for TD consultations.^23–25 Digital image–based PASI (DIB-PASI) has been established and demonstrated in a field setting.²³ PASI scores derived from digital images showed good agreement with those determined in-person. DIB-PASI enables TD-based clinical trials for patients with psoriasis and accurate, remote monitoring of disease severity and therapeutic response.

We performed a comparative observational study where TPSS (TD) was compared with ICDRG (FF) to interpret patch test readings. Teledermatology is compared to face to face (in-person examination-gold standard). The image of a patch test grid was transferred by the co-investigator who scored ICDRG face-to-face, to the principal investigator who scored by tripartite readings. The scoring was done by each of the investigators by their respective methods and scores were not known to each other. The current ICDRG scoring system highlights the importance of palpation. Palpation is a limitation for TD. A simplified scoring system for patch testing to be used in TD needs to be designed which gives more importance to visual changes. The doubtful category present in ICDRG serves a limited purpose in TD readings as even the faint erythema could sometimes be detected as a positive reaction from a standard image taken with good lighting.

ICDRG and NACDRG have given criteria to interpret patch test readings. We compared the ICDRG reading criteria which is considered the gold standard and widely used worldwide and in India with the TPSS.

Ivens et al. proposed the tripartite scoring system in 2007.¹⁹ In this study, 5 dermatologists were asked to score 55 patch tests from 15 slides based on the gold standard ICDRG scoring system, which yielded very different results. The pattern of scoring by the dermatologists was almost similar when scored by tri-partite scoring. They came to the conclusion that minimising and simplifying the number of categories in a scoring system to positive reactions, negative reactions (doubtful included) and IR yielded good results. As this scoring was performed on photographic images, they felt that this simplified tripartite scoring can be better utilised for research and TD.

In a study,²⁰ the same dermatologist performed patch test readings for both FF and TD (photographic images) 4–8 weeks apart to prevent recall bias. The scoring here was done by the National American Contact Dermatitis Grading (NACDG).²⁰ The outcomes were “Success” (54%), “Indeterminate” (40%) and “Failure” (6%).²⁰ The NACDG scoring in TD could be a viable option for grading patch test reactions. However, due to the failure rate of 6%, they concluded that the clinics where a large number of tests are performed may face some practical limitations to the widespread implementation.²⁰

Goodier et al. compared the agreement of two in-person dermatologists and eight tele-dermatologists for 70 patch test sites in 101 patients.²⁶ Reading was done using NACDG criteria by both parties. A failure rate of 13.4% was noted. There was a decline in positive percentage at 48 h (5.4%), with a raise for the final interpretation (29.1%). In our study, we noticed an increase from the 48^th h (19.3%) to the 96^th h (36.4%) in the ICDRG category and from 37.5% (48^th h) to 42% (96^th h) in the TPSS category. Goodier et al.²⁶ concluded that the disagreement was correlating with both low confidence in TD by patients and the quality of images provided.²⁶ They observed that the lack of palpation, varying image quality and poor patient confidence in TD were the limitations that contributed to the failure rate. Our method overcomes these shortcomings by using a scoring system purely dependent on visual inspection alone. Counselling of the patients with regard to technique and taking good-quality images with standard lighting and benefits of TD needs to be explained to the patients.

In another study, the sensitivity of assessing patch test reactions by digital images procured on day 7 and its comparison to clinical FF readings have been highlighted.²⁷ The readings were done using European Environmental and Contact Dermatitis Research Group (EECDRG) criteria. Further, they analysed the number of positive patch test reactions obtained at day 7. Digital images were forwarded by the patient on day 21 to the clinician and delayed reactions were verified. The readings were done using European Environmental and Contact Dermatitis Research Group (EECDRG) criteria. Further, they analysed the number of positive patch test reactions obtained at day 7. When FF was replaced with TD, they missed 26% of the day 7 reactions. We missed only 1.7% of the day 7 reactions as opposed to the 26.3% that was missed by them when FF was replaced with TD. In our study, we observed only 1% of positive reactions were missed on Day 7 (34% versus 33%).

In this study, at the 48^th h, we detected 37.5% positives in TPSS, which was 11.5% more than ICDRG because of the doubtful reactions (20 doubtful at 48 h became 14 positives, 3 negatives and 3 doubtful at the 96^th h of FF readings). These were read as positives in TD. At the 96^th h, 42% positives were read, which was 5.6% more than the ICDRG group owing to the fact that two results were read as positive instead of IR. The final day reading showed only a small discrepancy between 33% (TD) versus 34% (FF-gold standard).

Bruze et al.²⁸ used multiple scoring systems and used a three-part scoring system (1+, 2+ and 3+) to evaluate the degree of positivity. It showed 82% agreement between observers.²⁸ In their study, they only graded the positive reactions, as the study subjects selected were known to be allergic to the known allergens. Also, only non-irritating concentrations of allergens were used for the patch test in their patients. The selection of patients and methodology used in this study makes it difficult for us to compare our study with that of Bruze et al.²⁸ For practical purposes, the degree/severity/categorisation of positive reactions can result in inter-observer variations. A summary of various studies on TD patch testing is provided in Table 3.

In tri-partite scoring system (TPSS), only positive, negative and IR will be identified thereby eliminating the inter-observer variation that may occur due to a six-category scoring system. The interpretation using TPSS is comparable to ICDRG.

Six out of the 78 patients showed positive reaction to more than one allergen, leading to a total of 88 patch test results. All 88 were analysed by two investigators (ICDRG by CI and TPSS by PI) who were blinded to each other’s readings. It was observed that the doubtful (?) reactions read at the 48th h under ICDRG by FF were interpreted as both positive and negative from the images in TPSS. While the CI reported 20 (22.7%) doubtfuls (?) and 17 (19.3%) positive reactions (1+ and 2+), face-to-face by ICDRG criteria, the PI obtained a total of 33 positive reactions through TPSS. At the 96^th h, 3 of the doubtful were interpreted as negative, 14 as positive and 3 remained doubtful in the ICDRG group. The TPSS was deliberately designed to have only positive, negative or IR readings to help avoid the uncertainty inflicted by the doubtful category in TD. Doubtful (?) in ICDRG representing faint erythema indicates a weak positive reaction, which was missed a few times in TPSS readings. In TPSS, a sensitivity of 100% and specificity of 93.4% was observed. The PPV and NPV were 91.7% and 100%, respectively. Diagnostic accuracy of 96.5% was noted.

Limitations

This is a single centre study. However, we intend to undertake a multi-centre study in the future.

Conclusion

TPSS is comparable to ICDRG and an alternative scoring system to interpret patch tests in TD. Further, the variance of positive reactions into subcategories in ICDRG (+, ++ and +++) was noted to be dependent on the observer and may not be suited for TD. TPSS can reduce the number of visits needed for the interpretation of patch test readings by 50% (eliminating patient visits for the 96^th h and 7^th day readings). This may be of significance during periods like the COVID pandemic and beyond. This approach can reduce the expenditure borne by the patient for multiple hospital visits. It may also reduce the numbers of those who are lost to follow-up for patch test readings.