Previous results of the Dermatology-Life-Quality-Index (DLQI) validation in Colombia based on the classical test theory (CTT) perspective have showed the need to delve into its measurement properties. Therefore, we aimed to assess the structural validity, internal consistency and item response analysis of the DLQI through the item response theory (IRT) or the Rasch model.
Material and methodsWe assessed the dimensionality of the DLQI, determined its difficulty, discrimination and differential functioning and went on to evaluate its internal consistency and discriminative validity among patients with inflammatory and non-inflammatory skin disease. We recruited Colombian patients with different skin diseases.
ResultsData of 865 patients (mean age, 49.3 years; 61% females) were included. DLQI-scores ranged from 0 up to 30. Internal consistency for dichotomous item #7 showed a McDonald's Omega coefficient of 0.85 (95%CI, 0.84–0.87) and a Cronbach's α coefficient of 0.86 (95%CI, 0.84–0.88). For ordinal item #7, the McDonald's Omega coefficient was 0.87 (95%IC: 0.85–0.89) and Cronbach's α coefficient, 0.84 (95%CI, 0.82–0.87). Horn's parallel analysis revealed the presence of DLQI unidimensionality with a 1-factor solution. Only item #1 fitted the PCM model. The remaining items, including item #9, did not show adequate fit due to overlapping responses and order changes.
ConclusionsThe validity of DLQI-COL based on Rasch analysis proved not to be a suitable instrument to evaluate QoL in our dermatologic population. A refined version of the scale with an adequate cross-cultural adaptation and validation process is needed to obtain an appropriate and reliable version of the instrument for QOL measuring in Colombian patients with skin diseases.
Los resultados previos de la validación del Índice de Calidad de Vida en Dermatología (DLQI) en Colombia basados en la perspectiva de la teoría clásica de las pruebas (TTC) han mostrado la necesidad de profundizar en sus propiedades de medición. Por lo tanto, nuestro objetivo fue evaluar la validez estructural, la consistencia interna y el análisis de respuesta al ítem del DLQI a través de la teoría de respuesta al ítem (TRI) o el modelo de Rasch.
Material y métodosSe evaluó la dimensionalidad del DLQI; determinamos su dificultad, discriminación y funcionamiento diferencial y evaluamos su consistencia interna y validez discriminativa entre pacientes con enfermedad cutánea inflamatoria y no inflamatoria. Reclutamos pacientes colombianos con diferentes enfermedades de la piel.
ResultadosSe incluyeron datos de 865 pacientes (edad media: 49,3años); 61% mujeres. Las puntuaciones del DLQI oscilaron entre 0 y 30. La consistencia interna para el ítem dicotómico 7 mostró un coeficiente Omega de McDonald de 0,85 (IC 95%: 0,84–0,87) y un coeficiente α de Cronbach de 0,86 (IC 95%: 0,84-0,88). Para el ítem ordinal 7, el coeficiente Omega de McDonald fue de 0,87 (IC 95%: 0,85-0,89) y el coeficiente α de Cronbach de 0,84 (IC 95%: 0,82-0,87). El análisis paralelo de Horn indicó unidimensionalidad del DLQI con una solución de un factor. Solo el ítem 1 se ajustaba al modelo PCM. Los ítems restantes, incluido el ítem 9, no mostraron un ajuste adecuado debido a la superposición de respuestas y al cambio de orden.
ConclusionesLa validez del DLQI-COL basado en el análisis de Rasch demostró no ser un instrumento adecuado para evaluar la calidad de vida en nuestra población dermatológica. Se necesita una versión refinada de la escala con un adecuado proceso de adaptación y validación transcultural para obtener una versión apropiada y confiable del instrumento para medir la calidad de vida en pacientes colombianos con enfermedades de la piel.
Quality of life (QoL) in the dermatological field has been measured with 2 instruments: the SKINDEX and the Dermatology-Life-Quality-Index (DLQI). Between these two instruments, only Skindex-29 has had a formal process of validation in Colombia based on the classical test theory (CTT).1,2 Although the CTT has been one of the most widely used approaches for QoL scales validation in dermatology,3 some of its assumptions are problematic as it is sample dependent and does not take into account that scoring can be affected by different variables of the patient and the measurement process.4–6 To date, few studies have assessed the structural validity of the DLQI with the use of the IRT or the Rasch model.7–10 In this regard, the report by Rencz7 focused on analyzing the answer option “not relevant” whereas the study conducted by Marilia10 assessed unidimensionality and internal consistency, resulting in difficulties in Rasch model adjustment, which is consistent with Nijsten's report that also showed variations of the model across countries.8 A different study of 9845 patients with various dermatological diseases in China revealed that DLQI did not meet proper measurement requirements.9 Additionally, the original 10-item DLQI has been shown to be influenced by gender and age and presented measurement difficulties in patients with atopic dermatitis and psoriasis.11 Also, preliminary results of the DLQI validation in Colombia based on the CTT perspective have showed the need to delve into the measurement properties of the instrument (item number and order of categories or possible response values, their capacity for discrimination and difficulty, and their differential functioning) all of which can be approached through the IRT. Therefore, we aimed to: (1) assess the dimensionality of the DLQI; (2) determine the difficulty, discrimination and differential functioning of the scale; (3) evaluate its internal consistency and (4) assess its discriminative validity between patients with inflammatory and non-inflammatory skin disease.
Material and methodsStudy populationIn this cross-sectional study, recruited patients attended the dermatological consultation of private practice offices, health provider centers and hospitals of 6 different Colombian regions: North-East; North-West Central area; and South-West. This study was conducted in full compliance with the Declaration of Helsinki of 1975, as revised in 1983 and the ethical standards of the School of Medicine Bio-Ethics Committee and Fundacion Hospital San Vicente Research and Ethics Committee (Medellin, Colombia). Informed consent was obtained from all adult patients and adolescents between 16 and <18 years-old, as well as a signed assent form from their parents or guardians. Individuals older than 16 years-old of any gender and with any skin disease and who signed informed consent were included too. Patients with mental disabilities were excluded.
InstrumentThe DLQI in Colombian Spanish applied to all patients is a 10-item instrument divided into six aspects: activities of daily living, work/school, personal relationships, leisure, symptoms and feelings and therapy. Its scoring ranges from 0 up to 30: 0–1=no impact; 2–5=mild impact; 6–10=moderate impact; 11–20=substantial impact and 21–30=extreme impact on the patient's QoL. The higher the score, the worse the QoL. Each DLQI item is answered on a 4-point scale scored as follows: ‘not at all’=0, ‘a little’=1, ‘a lot’=2 and ‘very much’=3. Non-relevant response (NRR) options are available for items 3 through 10.
Statistical analysisQuantitative variables were expressed as means and standard deviations or median and interquartile range (Quartile 1 and Quartile 3) while qualitative variables were expressed as absolute numbers and their relative frequencies in percentages. Skin diseases were stratified as inflammatory and non-inflammatory. Response options are expressed as percentages for each item. Floor and ceiling effects were examined by the proportion of patients who achieved the DLQI best and worst scores, with cutoff values >15%, as described by McHorney et al.12
Item #7 was assessed either as a dichotomous variable (“Within the last week, has your skin problem prevented you from working or studying?”) and as an ordinal variable (“If “Yes”, over the last week how much has your skin been a problem at work or school?”). For all items except for item #7, the matrix of polychoric correlations across items was determined, and for item #7, as well, in its dichotomous version, the tetrachoric relationship with the other items was calculated. Internal-consistency was determined using McDonald's Omega coefficient and Cronbach's alpha (α) coefficient with their respective 95% confidence intervals (CI). Values>0.70 were considered adequate.13,14
For Rash analysis, we used Partial Credit Model (PCM) for polytomic items. The assessment of the assumption of unidimensionality and dimensionality of the instrument among non-inflammatory and inflammatory skin diseases was conducted using Horn's parallel.15 Then we assessed the structural validity of the one-dimensional model with a confirmatory factor analysis while the goodness-of-fit measures was evaluated using the root mean square error of approximation (RMSEA) with values <0.05 in addition to the comparative fit index (CFI) and the Tucker–Lewis index (TLI) with values>0.95.16
We examined differential item functioning in items across subgroups of patients with non-inflammatory and inflammatory diseases and between gender and age ranges categorized either as: “Younger”≤45 and “Older”>45 years-old and into 3 groups: <30, 30–60, >60 years-old) through the Hedges G test based on the standardized mean differences. Fritz et al.17 classification was used with 0.20, 0.40 and 0.80 considered as a low, medium, and high differences,17,18 respectively. In addition, differential item functioning was evaluated using Mantel–Haenszel model19 for both genders, inflammatory and non-inflammatory diseases. Differential functioning was stablished with p values<0.05 as reported by Holland et al.20
All analyzes were performed using the R program version 4.3.0 and the mirt, psych, lavaan and MBESS packages.21–25
ResultsPatients’ characteristicsOverall, 865 patients with a mean age of 49.3 years (SD, 21; 61% females) were included. Around 52% of patients were not on any therapies, 54% of patients suffered from comorbidities and most patients included had non-inflammatory skin diseases (n=482) (Table 1).
Sociodemographic features of included patients.
Non-inflammatory skin diseases (N=502) | Inflammatory skin diseases (N=363) | Total (N=865) | |
---|---|---|---|
Current treatment | |||
Yes | 309 (62%) | 103 (28%) | 412 (48%) |
No | 193 (38%) | 260 (72%) | 453 (52%) |
Comorbidities | |||
Yes | 318 (63%) | 148 (41%) | 466 (54%) |
No | 184 (37%) | 215 (59%) | 399 (46%) |
Gender | |||
Male | 189 (38%) | 147 (40%) | 336 (39%) |
Female | 313 (62%) | 216 (60%) | 529 (61%) |
Age | |||
Mean (SD) | 57.2 (±18.2) | 38.4 (±19.8) | 49.3 (±21.0) |
Median (Q1, Q3) | 59.0 (43.0, 71.0) | 34.0 (21.0, 53.0) | 50.0 (32.0, 66.0) |
Marital status | |||
Single | 116 (23%) | 202 (56%) | 318 (37%) |
Married | 89 (18%) | 62 (17%) | 151 (17%) |
Living in common-law | 111 (22%) | 75 (21%) | 186 (22%) |
Divorced | 27 (5%) | 10 (3%) | 37 (4%) |
Widowers | 53 (11%) | 13 (4%) | 66 (8%) |
Socioeconomic status | |||
Low | 208 (41%) | 119 (33%) | 327 (38%) |
Medium | 205 (41%) | 166 (46%) | 371 (43%) |
High | 83 (17%) | 75 (21%) | 158 (18%) |
Education level | |||
None | 24 (5%) | 16 (4%) | 40 (5%) |
Elementary school | 164 (33%) | 83 (23%) | 247 (29%) |
Middle/high school | 129 (26%) | 78 (21%) | 207 (24%) |
Technician | 8 (2%) | 73 (20%) | 81 (9%) |
College | 76 (15%) | 81 (22%) | 157 (18%) |
Posgraduate | 59 (12%) | 14 (4%) | 73 (8%) |
Health insurance | |||
Contributive | 339 (68%) | 291 (80%) | 630 (73%) |
Subsidiated | 80 (16%) | 30 (8%) | 110 (13%) |
Beneficiary | 52 (10%) | 24 (7%) | 76 (9%) |
Othera | 24 (5%) | 18 (5%) | 42 (5%) |
Dermatology-Life-Quality-Index scores ranged from 0 up to 30, with a median of 5 (percentile25=1; percentile75=13) The distribution of item scores was asymmetrical with positive skewness for total score and response distribution for each item (Figs. 1 and 2 of the supplementary data). Floor effect12 was present in all items (percent of response in option “not at all”>29%). Ceiling effect was not present in any of the items (Table 2).
Floor and ceiling effect.
Porcentage | ||||
---|---|---|---|---|
Not at all | A little | A lot | Very much | |
Item 1 | 37.5% | 27.2% | 22.1% | 13.3% |
Item 2 | 61.2% | 15.7% | 10.8% | 12.4% |
Item 3 | 79.3% | 8.3% | 6.2% | 6.1% |
Item 4 | 65.1% | 12.3% | 9.9% | 12.7% |
Item 5 | 70.1% | 11.1% | 9.1% | 9.7% |
Item 6 | 77.1% | 9.2% | 6.6% | 7.1% |
Item 7 | 90.3% | 5.3% | 4.4% | 0% |
Item 8 | 80.6% | 8.6% | 4.9% | 6% |
Item 9 | 85.1% | 4% | 5.2% | 5.7% |
Item 10 | 74% | 9.6% | 7.1% | 9.4% |
Internal consistency for the version with dichotomous item #7, showed a McDonald's Omega coefficient of 0.85 (95%CI, 0.84–0.87) and a Cronbach's α coefficient of 0.86 (95%CI, 0.84–0.88). For the version with item #7 as ordinal, McDonald's Omega coefficient was 0.87 (95%CI, 0.85–0.89) and Cronbach's α coefficient of 0.85 (95%CI, 0.82–0.87). Correlation matrix is presented in Supplement 3.
DLQI unidimensionality and structural assessmentHorn's parallel analysis indicated DLQI unidimensionality with a one factor solution. A two-factor model alternative only included item #7 in the second factor. Therefore, we chose the one factor solution model (Fig. 4 of the supplementary data). In addition, when analyzing the structural validity under a one-dimensional model, the goodness-of-fit indices showed better results for the model that had an ordinal item #7 with CFI=0.999, TLI=0.998 and RMSEA=0.012 (90%CI, 0.000–0.028) compared to the dichotomous item #7 which showed CFI=0.999, TLI=0.999 and RMSEA=0.000 (90%CI, 0.000–0.024).
When Horn's parallel analysis was performed stratifying by inflammatory and non-inflammatory skin disease, we found that in the former, the first dimension would include item #2, item #3, item #4, item #5, item #6, item #7, item #8, item #9, item #10 while the second dimension would only include item #1 as seen in Fig. 1. The internal consistency by Cronbach's Alpha and McDonald's Omega was 0.83 (95%CI, 0.80–0.87) and 0.86 (95%CI, 0.84–0.89), and 0.58 and 0.58 for the first and second dimensions, respectively. Regarding Inflammatory skin diseases, the single dimension included all items, except for item #7, as seen in Fig. 2. The internal consistency by Cronbach's Alpha and McDonald's Omega was 0.73 (95%CI, 0.66–0.81) and 0.80 (95%CI, 0.75–0.84), respectively.
Regarding non-inflammatory skin diseases, the first dimension would include item #2, item #5, item #6 and item #8 and the second, item #1, item #3, item #4, item #7, item #9 and item #10, as seen in Fig. 2. The internal consistency by Cronbach's Alpha and McDonald's Omega was 0.71 (95%CI, 0.65–0.77) and 0.61 (95%CI, 0.53–0.69) and 0.72 (95%CI, 0.66–0.78) and 0.73 (95%CI, 0.68–0.78) for the first and second dimensions, respectively.
Partial credit model (PCM)Figs. 3 and 4 show the characteristic curves of the responses of items #1 and #9, respectively. Only item #1 fitted the PCM model.
Fig. 5 of supplementary data describes the characteristic curves of the items using the Partial Credit Model (PCM) for polytomic items. On the other hand, Table 3 shows the discrimination and difficulty parameters for the response options of each item where it is observed that the options “A little” and “A lot” do not preserve the order of the responses in any of the items except for item #1.
Parameters estimated for the PCM.
Partial credit model | ||||
---|---|---|---|---|
a | b1 | b2 | b3 | |
Item 1 | 1 | −0.14522 | 0.61389 | 1.61022 |
Item 2 | 1 | 1.27951 | 1.11757 | 1.16409 |
Item 3 | 1 | 2.50180 | 1.36104 | 1.59776 |
Item 4 | 1 | 1.63860 | 0.98036 | 1.07375 |
Item 5 | 1 | 1.91210 | 1.07760 | 1.35421 |
Item 6 | 1 | 2.32738 | 1.36796 | 1.47132 |
Item 7 | 1 | 3.32220 | 1.58605 | NA |
Item 8 | 1 | 2.51885 | 1.67058 | 1.39105 |
Item 9 | 1 | 3.37687 | 0.89670 | 1.54873 |
Item 10 | 1 | 2.17915 | 1.25432 | 1.17784 |
a: Discrimination, b1: Difficulty of response option between “Not at all” and “A little”, b2: Difficulty of response options between “A little” and “A lot”, b3: Difficulty in response options between “A lot” and “Very much”.
Furthermore, in the items information functions, a slight shift to the right was observed in all items toward individuals with impaired QOL close to +1 standard deviation (Fig. 6 of the supplementary data).
Differential item functioning between patients with non-inflammatory and inflammatory skin disease based on gender and age rangesDifferences between non-inflammatory and inflammatory patients was found with G Hedges=0.89 (95%CI, 0.73–1.04), but no gender differences were ever found: 0.07 (95%CI, 0.13–0.15) (Table 4).
DLQI differential item functioning according to disease.
Disease | |||
---|---|---|---|
Stat | p-Value | Effect | |
Item 1 | 0.087 | 0.7681 | Large |
Item 2 | 6.9689 | 0.0083 | Large |
Item 3 | 2.9774 | 0.0844 | Moderate |
Item 4 | 0 | 0.9989 | Negligible |
Item 5 | 10.5272 | 0.0012 | Large |
Item 6 | 0.5841 | 0.4447 | Negligible |
Item 7 | 0.5014 | 0.4789 | Moderate |
Item 8 | 18.4554 | 0 | Large |
Item 9 | 3.3495 | 0.0672 | Negligible |
Item 10 | 9.5376 | 0.002 | Large |
Differential item function (DIF) analysis showed that items #2, #5, #8 and #10 had large DIF in non-inflammatory and inflammatory patients (Table 4). Item #2 had a large DIF between male and female patients (Supplement 7). In addition, when age was categorized in patients younger and older than 50 years (median), items #3 and #6 presented large DIFs (p<0.05), but when age was categorized into 3 groups (<30, between 30 and 60 and >60 years-old) we saw that only item #3 presented a large DIF (p<0.05) (Table 5).
DLQI differential item functioning according to age ranges.
Age (≤45 and >45 years-old) | Age (<30, 30–60, >60 years-old) | |||||
---|---|---|---|---|---|---|
Stat | p-Value | Effect | Stat | p-Value | Effect | |
Item 1 | 0.3003 | 0.5837 | Negligible | 0.0852 | 0.7704 | Negligible |
Item 2 | 0.2180 | 0.6405 | Negligible | 0.3529 | 0.5525 | Moderate |
Item 3 | 11.0973 | 0.0009 | Large | 13.238 | 0.0003 | Large |
Item 4 | 2.9401 | 0.0864 | Negligible | 0.9883 | 0.3202 | Large |
Item 5 | 0.0026 | 0.9591 | Negligible | 0.4986 | 0.4801 | Moderate |
Item 6 | 5.2734 | 0.0217 | Large | 0.0131 | 0.9088 | Negligible |
Item 7 | 0.0060 | 0.9381 | Negligible | 0.1221 | 0.7268 | Negligible |
Item 8 | 2.8166 | 0.0933 | Moderate | 0.3228 | 0.5699 | Negligible |
Item 9 | 0.0004 | 0.9848 | Negligible | 4.9822 | 0.0256 | Large |
Item 10 | 0.0008 | 0.9768 | Negligible | 0.0059 | 0.9387 | Negligible |
In this Colombian first study the psychometric properties of the DLQI were examined with Rasch analysis in pooled data obtained from several cross sectional studies that assessed QOL in Colombian patients with different non-inflammatory and inflammatory skin diseases, resulting in several psychometric deficiencies as has been found by others11,26: First, overall, one of the most important constraints of the DLQI is that it includes several compound questions (several questions in 1 item), which causes confusion when only one or two questions apply and the other do not, which may have also contributed to our floor effect results in all items with the “not at all” response option. This is relevant because such high floor effect suggests that patients at the most mildest ends of the scale are undistinguishable from one another, a finding that has also been reported by others when performing Rasch analysis.10,11
Another interesting finding in our population is that according to the PCM, it seems that Colombian patients had difficulties distinguishing between the “a little” and the “a lot” response options, a result that was also been reported by Rencz et al.7 These findings lead us to question whether a modified Colombian version of the instrument either with dichotomous answers (“yes” or “no”) or disaggregated questions would improve its measurement properties.
In contrast to other studies, in this work, we classified dermatological diseases as inflammatory and non-inflammatory to differentiate skin diseases that exhibit more or less symptoms. As expected, DLQI performed better when measuring the impact on QOL of symptomatic (inflammatory) cutaneous diseases, whereas scale responsiveness to asymptomatic (non-inflammatory) diseases was lower. In fact, we found differential item functioning between non-inflammatory and inflammatory skin diseases, which could be explained by the greater number of symptoms present in cutaneous inflammatory diseases. In addition, although we found no differences in gender as reported by Nijsten et al.,8 we certainly found age-based DIF, which was consistent for age categorizations in patients younger and older than 50 years, in items #3 and #6 and only in item #3 when age was categorized into 3 groups. In this regard, we consider that for these 2 items these results were expected because regarding item #3, there are very few people in our country who can have a garden at home and there are also few who can go shopping due to the lack of financial resources of most of the population. Similarly, and in reference to item #6, differential responses may have been influenced by the percentage of our population that practice sports rather that the impact of such activities caused by skin disease as it has been reported that as individuals age there they practice fewer sports or are not involved in physical activities as much in Colombia.27
Although the DLQI has been the most widely used dermatology-specific QOL instrument, its dimensionality has been debated as some studies support its unidimensionality,28 others describe a scale with a multidimensional structure.11,26,29 Interestingly, in one of our initial IRT tested models we found a two-factor solution with very low variability percentages while only item #7 was included in the second factor, a result that has also been reported among Chinese patients26; therefore we leaned toward the solution of one factor as has been proposed with the original instrument.30 In addition, our dimensionality results stratified by non-inflammatory vs inflammatory skin disease denoted an unclear plausibility and a lack of clinical meaning in grouped items, which reinforces the theory of an instrument measuring a single construct and the adequacy of a summary score as reported by others.7,9,10
Overall, our study describes the misfit of all DLQI items except for item #1, which indicates that patients did not answer the questionnaire according to the predicted model. In this regard, it was striking to see that item #9 (related to changes in sexual life) suited the model the least, which suggests that due to cultural taboo issues, our population might evade a real answer to this question and chooses to answer it as a dichotomous question with a “very much” or “not at all” answer, which is consistent with a previous report conducted by Liu et al.26
As far as we know, this is the first Colombian study ever conducted that tested the validity of the DLQI using a Rasch analysis (PCM) in a large Colombian sample of patients with a variety of dermatological diseases (instead of focusing on a specific disease) with and without symptoms recruited from across the country. Limitations of the study are related to its cross-sectional design, which is more prone to selection bias, which does not allow us to draw directional causal inferences or skin disease severity or QOL fluctuations and heterogeneity related to the variety of the dermatological diseases that were included.
ConclusionsIn conclusion, the validity of DLQI based on Rasch analysis proved to be an unsuitable instrument to evaluate quality of life in our dermatologic population as it does not provide accurate information on this outcome. Therefore, a refined version of the scale (either with a dichotomous item #7 or disaggregating the whole instrument and reducing it) with an adequate cross-cultural adaptation and validation process are needed to obtain an appropriate and reliable version of the instrument for measuring QOL in Colombian patients with skin diseases.
We wish to thank all the patients for their willingness to participate. We are also grateful to Lina Marcela Barrera for her help in the logistic coordination of the study.