Chinese version of the treatment of autoimmune bullous disease quality of life questionnaire: Reliability and validity

Introduction

Autoimmune blistering disease is a group of uncommon skin and mucosal diseases that can significantly affect the patients' quality of life.^[1] The diseases have variable clinical manifestations and include pemphigus vulgaris, pemphigus foliaceus, bullous pemphigoid, dermatitis herpetiformis, epidermolysis bullosa acquisita, paraneoplastic pemphigus, linear immunoglobulin A bullous disease and pemphigoid gestationis. In the past, these diseases carried a guarded prognosis with secondary infection of bullae culminating in sepsis. However with the advent of immunosuppressive therapies, autoimmune blistering diseases tend to be chronic conditions with a significant morbidity.^[2] Treatments for autoimmune blistering disease carry significant risks of medical complications and can affect the patient's quality of life. Furthermore, it may be difficult to differentiate treatment-related effects on the quality of life from the disease burden or the adverse effects of the treatments themselves.^[3]

The lack of uniform and validated outcome measures hampers progress in evaluating treatment impact on patients with autoimmune blistering disease. Standardization of such measures is pivotal to accurately monitor the patient and to pool results from randomized controlled trials for meta-analysis which will provide better understanding of the optimal autoimmune blistering disease therapies.^[1]

Recently, the scholars of Australia published the Treatment of Autoimmune Bullous Disease Quality of Life and the Autoimmune Bullous Disease Quality of Life questionnaires using standardized methodology in autoimmune blistering disease patients.^[3],[4] The Treatment of Autoimmune Bullous Disease Quality of Life questionnaire was the first validated quality of life tool specific to the adverse effects of treatments in autoimmune blistering disease patients and has been shown to be a reliable and valid instrument to measure treatment burden.^[3]

In this study, we aimed to validate the Treatment of Autoimmune Bullous Disease Quality of Life questionnaire in Chinese patients with autoimmune blistering disease with a view to using it as an outcome measure for treatment evaluation and as an end point in future clinical trials in Chinese autoimmune blistering disease patients.

Methods

Study questionnaire

As a self-administered questionnaire, the formal Treatment of Autoimmune Bullous Disease Quality of Life questionnaire of Chinese version was produced with forward-backward translation, cross-cultural adaptation, pilot questionnaire, review of bullous disease experts and adjusting the arrangement using standard method. Prior to the study, the approval of the Ethics Committee of the Shandong Provincial Hospital for Skin Diseases was obtained.

Patient recruitment

All the patients were from Shandong Provincial Hospital for Skin Diseases. Inclusion criteria were over 18 years of age, a confirmed histopathological and immunohistological diagnosis of autoimmune blistering disease and sufficient education to complete the questionnaire. General patient characteristics were recorded including disease stage, gender, concomitant and past medications and for the pemphigus vulgaris patients the indirect immunofluorescence titers around the time of the survey. As previously recommended, we selected five times the number of items as the minimum necessary sample size.^[5] There are 17 items in the questionnaire. Hence, we aimed to recruit at least 85 patients with autoimmune blistering disease.

All the patients recruited in the study were asked to complete the Chinese version of the Treatment of Autoimmune Bullous Disease Quality of Life questionnaire, the Dermatology Life Quality Index ^[6] and the 36-item Short-Form Health Survey ^[7] without assistance. The outpatients performed the surveys after their clinical appointments in the clinic on day 0 and they were also asked to take home a stamped addressed envelope containing the Treatment of Autoimmune Bullous Disease Quality of Life questionnaire to complete on day 7 and return it to the investigator. The inpatients completed the questionnaires on days 0 and 7 while in the ward.

Statistical analysis

All statistical analyses were performed using SPSS 10.01 software (SPSS Inc., Chicago, IL, USA). P< 0.05 was considered statistically significant.

Reliability

Internal consistency was measured using Cronbach's alpha coefficient which also tested for construct validity.^[8] Test-retest reliability was assessed by comparing the responses of a subset of patients who completed the questionnaire on both days 0 and 7. The intraclass correlation coefficient was calculated to determine the ability of the tool to give concordant results at different times.

Validity

Validation methods included assessment of face, content, construct, convergent and discriminant validities.^[3] Face and content validities were established by forward-backward translation of the Treatment of Autoimmune Bullous Disease Quality of Life and by review of the questionnaire by the panel of bullous disease experts. Convergent validity was determined by calculating Spearman's correlation coefficients for the Treatment of Autoimmune Bullous Disease Quality of Life and Dermatology Life Quality Index or the Treatment of Autoimmune Bullous Disease Quality of Life and 36-item Short-Form Health Survey. Discriminant validity was evaluated by comparing the questionnaire scores of (i) outpatients and inpatients, (ii) patients on steroids and steroid-sparing medications, (iii) men and women, (iv) indirect immunofluorescence-positive and indirect immunofluorescence-negative patients with pemphigus vulgaris (all using Student's t-test) and (v) the three patient subtypes (pemphigus vulgaris, bullous pemphigoid and all others [pemphigus foliaceus, dermatitis herpetiformis, linear immunoglobulin A bullous disease, paraneoplastic pemphigus, epidermolysis bullosa acquisita and pemphigoid gestationis]) using one-way analysis of variance.

Factor analysis

Construct validity was also assessed through factor analysis. Exploratory factor analysis and principal component analysis followed by Oblimin rotation with Kaiser normalization were performed to assess the dimensionality of the 17 items and to validate the structure of the Chinese version of the Treatment of Autoimmune Bullous Disease Quality of Life questionnaire. Significance was defined as a loading >0.4 or <−0.4. Item complexity occurred if an item loaded >0.4 or <−0.4 on more than one factor. If an item was complex and the difference in loading was at least 0.1, it was assigned to the factor on which it loaded more heavily.^[3]

Results

A total of 86 patients were recruited and completed the surveys between March 2015 and December 2015. All patients completed the day 0 questionnaire and 59 (68.6%) completed the day 7 questionnaire. None of the patients reported difficulties in understanding the questions.

Demographics

Patient demographics are presented in [Table - 1]. Of the 86 patients recruited, 56 (65.1%) were men and 30 (34.9%) were women. The patients' ages ranged from 18 to 77 years with a mean of 49.9 years. There were 72 outpatients and 14 inpatients. The number of school years completed ranged from 4 to 18 years with a mean of 9.3 years. Most of the patients had pemphigus vulgaris (n = 51), followed by bullous pemphigoid (n = 19), pemphigus foliaceus (n = 8), dermatitis herpetiformis (n = 3), linear immunoglobulin A bullous disease (n = 2), epidermolysis bullosa acquisita (n = 1), paraneoplastic pemphigus (n = 1) and pemphigoid gestationis (n = 1). The clinical disease stages were complete remission minimal therapy (n = 29), control of disease activity (n = 27), complete remission during tapering (n = 23), time to control of disease activity (n = 3), complete remission off therapy (n = 2), partial remission minimal therapy (n = 1) and relapse/flare (n = 1).^[9] Concomitant and past medications included (in order of frequency) prednisone, methylprednisolone, cyclophosphamide, topical steroids, tetracycline, niacinamide, dapsone, azathioprine, cyclosporine, intravenous immunoglobulin and rituximab.

Table 1: Demographic characteristics of the patient cohort

Patient scores

The Treatment of Autoimmune Bullous Disease Quality of Life total scores ranged from 1 to 47 out of a possible 51 points (original study; 0–45). The mean ± standard deviation scores were 17.43 ± 9.77 for all patients (original study; 16.3 ± 10.3), 17.22 ± 9.59 for pemphigus vulgaris, 17.74 ± 11.46 for bullous pemphigoid and 17.75 ± 8.73 for all other patients. The scores for the three disease subgroups are shown in [Figure - 1]. The item “dangerous medications” scored highest (2.05 ± 1.13), indicating the greatest impact on quality of life and the item “blood tests” scored lowest (0.49 ± 0.73). The scores for the individual items are shown in [Table - 2].

Figure 1: The scores (mean ± standard deviation) of the three subtypes of patients (pemphigus vulgaris, bullous pemphigoid and other autoimmune blistering diseases). The left column: pemphigus vulgaris (n = 51), the middle column: bullous pemphigoid (n = 19), the right column: others (n = 16), the black column: Mean and the gray column: standard deviation

Table 2: Mean score, standard deviation and test-retest reliability of each item of the Treatment of Autoimmune Bullous Disease Quality of Life questionnaire

Reliability

The internal consistency and construct validity were acceptable with a Cronbach's alpha of 0.883 and this value was unaffected by deletion of any one item [Table - 2]. The test-retest reliability of the questionnaire was also acceptable with a mean intraclass correlation coefficient value of 0.871 [Figure - 2] and [Table - 2].

Figure 2: The test-retest reliability of the total Treatment of Autoimmune Bullous Disease Quality of Life scores (n = 59). The X-axis: day 0 scores, the Y-axis: day 7 scores

Validity

With regard to convergent validity, the correlation coefficient for the Treatment of Autoimmune Bullous Disease Quality of Life and Dermatology Life Quality Index was 0.664 (P< 0.001) and for the Treatment of Autoimmune Bullous Disease Quality of Life and 36-item Short-Form Health Survey was – 0.577 (P< 0.001). For pemphigus vulgaris patients, indirect immunofluorescence titers and Treatment of Autoimmune Bullous Disease Quality of Life scores showed a poor Spearman's rank correlation (r = 0.081, P = 0.572). Discriminant validity testing showed no significant differences in the Treatment of Autoimmune Bullous Disease Quality of Life scores of patients on steroids and steroid-sparing medication (t = 0.672, P = 0.503), outpatients and inpatients (t = 0.447, P = 0.656), indirect immunofluorescence-positive and indirect immunofluorescence-negative patients with pemphigus vulgaris (t = 1.270, P = 0.210) and men and women (t = 0.251, P = 0.802), all determined by Student's t-test. There was also no significant difference between the scores of the three patient subtypes (pemphigus vulgaris, bullous pemphigoid and others), as determined by one-way analysis of variance (F = 0.030, P = 0.971). Statistical analyses of validity and reliability measures are presented in [Table - 3].

Table 3: Validity and reliability of the Chinese version of the Treatment of Autoimmune Bullous Disease Quality of Life questionnaire

Factor analysis

Bartlett's test of sphericity (P< 0.001) and the Kaiser–Meyer–Olkin measure of sampling adequacy (0.818) suggested that factor analysis of the data was appropriate.^[3] The Kaiser criterion suggested that five factors had eigenvalues >1, representing 66.8% of the cumulative variance of the 17 items. The Cattell scree plot suggested that three factors should be retained, representing 53.3% of the cumulative variance. The rotation matrix indicated that questions 2, 5, 6, 7, 12, 14, 15 and 16 were loaded on factor 1; questions 8, 9, 10, 11, 13 and 17 were loaded on factor 2 and questions 1, 3 and 4 were loaded on factor 3. The three dimensions represented are shown in [Table - 4].

Table 4: The principal component factor analysis of the Chinese version of the Treatment of Autoimmune Bullous Disease Quality of Life questionnaire

Discussion

We were unable to find any previous validation study of the Treatment of Autoimmune Bullous Disease Quality of Life questionnaire in Chinese patients with autoimmune blistering disease. In this study, the Treatment of Autoimmune Bullous Disease Quality of Life scores for pemphigus vulgaris patients (17.22 ± 9.59) were similar to the initial validation study in Australia (17.35 ± 8.80), whereas the scores for bullous pemphigoid patients (17.74 ± 11.46) were higher than that in the original Australian study (15.1 ± 12.38). In the original study in Australia, the patients were all outpatients in dermatology clinics, whereas in this study, the patients were recruited from both inpatient and outpatient departments. The patients in the ward were in severe condition, perhaps this may be one reason why some patients scored more highly in our study. Other factors also influenced the scores, for example, different cultures, education and the subtypes of autoimmune blistering disease.

When evaluating the reliability of patient-reported measures for internal consistency, Cronbach's alpha should ideally be above 0.70.^[10] In our study, Cronbach's alpha was 0.883, indicating high internal consistency compared with a value of 0.892 in the Australian study. Our test-retest reliability coefficient was 0.871, compared with 0.988 in the original study, suggesting that the Chinese version of the Treatment of Autoimmune Bullous Disease Quality of Life questionnaire will yield consistent results under similar conditions. The Cronbach's alpha of 0.883 also indicates satisfactory construct validity for the Chinese version of the Treatment of Autoimmune Bullous Disease Quality of Life questionnaire.

With respect to convergent validity measures, the correlation values are considered to indicate poor correlation when <0.20, to indicate fair correlation when 0.21–0.40, to signify good correlation when 0.41–0.60, to indicate very good correlation when 0.61–0.80 and to indicate excellent correlation when >0.81.^[11] We found very good convergence between the Treatment of Autoimmune Bullous Disease Quality of Life and Dermatology Life Quality Index instruments (r = 0.664) which is slightly higher than that found in the original study (0.64). The Treatment of Autoimmune Bullous Disease Quality of Life and 36-item Short-Form Health Survey correlation coefficient, −0.577, was also higher than that observed in the original study (0.22).

The discriminant validity was not ideal, as expected. For example, the inpatients' disease status was generally more serious than that of the outpatients, but their Treatment of Autoimmune Bullous Disease Quality of Life scores were not significantly different. Similarly, although indirect immunofluorescence titers correlate with pemphigus vulgaris clinical disease activity, we found no significant difference in the Treatment of Autoimmune Bullous Disease Quality of Life scores of indirect immunofluorescence-positive and indirect immunofluorescence-negative patients.^[12] Thus, treatment-related quality of life may be independent of disease severity. This is not unexpected as the Treatment of Autoimmune Bullous Disease Quality of Life is able to capture quality of life effects in patients whose clinical state were in remission. There are still treatment quality of life issues for the patients who need take medications to keep the disease in remission.

The exploratory principal component analysis followed by Oblimin rotation with Kaiser normalization and the Catell scree plot suggested that three factors should be retained and the 17 items in the Chinese questionnaire must be distributed on three factors. However, in the original Treatment of Autoimmune Bullous Disease Quality of Life study, all items except item 1 loaded on factor 1, suggestive of a strong single-dimensional questionnaire. This discrepancy might be explained by the different patient cohorts, customs or cultures tested between the two studies.

There are several limitations to this study. First, this is a patient-reported questionnaire and many patients were excluded because some patients were illiterate or could not finish the questionnaire by themselves. In a study of the 36-item Short-Form Health Survey in Morocco, the high illiteracy rate was overcome by having a single investigator administer the questionnaire to the patient.^[13] The same approach could be taken in clinical practice when measuring the treatment burden in patients with autoimmune blistering disease. Second, the patients were recruited from a single hospital which might have resulted in some patient selection bias. Third, most of the recruited patients had relatively stable disease and the number with acute diseases was low. Further validation of the Chinese version of the Treatment of Autoimmune Bullous Disease Quality of Life questionnaire would ideally involve more hospitals and include more extensive patient populations.

Conclusion

The Chinese version of the Treatment of Autoimmune Bullous Disease Quality of Life questionnaire could be used to measure treatment burden and to serve as an end point in clinical trials in Chinese autoimmune blistering disease patients.

Acknowledgments

We thank the patients with autoimmune blistering disease who completed the questionnaires and the doctors and medical students who contributed to this study.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.