Actas Dermo-Sifiliográficas (English Edition) Actas Dermo-Sifiliográficas (English Edition)
Actas Dermosifiliogr 2017;108:650-6 - Vol. 108 Num.7 DOI: 10.1016/j.adengl.2017.06.002
Original Article
Reliability of the MDi Psoriasis Application to Aid Therapeutic Decision-Making in Psoriasis
Fiabilidad de una aplicación de ayuda a la toma de decisiones terapéuticas en el paciente con psoriasis (MDi Psoriasis®)
D. Moreno-Ramíreza,, , , J.M. Herrerías-Estebanb, T. Ojeda-Vilaa, J.M. Carrascosac, G. Carreterod, P. de la Cuevae, C. Ferrándizc, M. Galánf, R. Riverag, L. Rodríguez-Fernándezh, R. Ruiz-Villaverdei, L. Ferrándiza
a Unidad de Gestión Clínica de Dermatología, Hospital Universitario Virgen Macarena, Sevilla, Spain
b Subdirección de Tecnologías de Información y Comunicaciones, Consejería de Salud, Sevilla, Spain
c Servicio de Dermatología, Hospital Universitario de Gran Canaria Dr. Negrín, Las Palmas de Gran Canaria, Spain
d Servicio de Dermatología, Hospital Universitario Germans Trias i Pujol, Badalona, Spain
e Servicio de Dermatología, Hospital Universitario Infanta Leonor, Madrid, Spain
f Unidad de Gestión Clínica de Dermatología, Hospital Universitario Reina Sofía, Córdoba, Spain
g Servicio de Dermatología, Hospital Universitario 12 de Octubre, Madrid, Spain
h Unidad de Gestión Clínica de Dermatología, Hospital Universitario Virgen del Rocío, Sevilla, Spain
i Unidad de Gestión Clínica de Dermatología, Complejo Hospitalario Universitario de Granada, Granada, Spain
Received 07 October 2016, Accepted 26 February 2017
Abstract
Introduction

Therapeutic decisions in psoriasis are influenced by disease factors (e.g., severity or location), comorbidity, and demographic and clinical features.

Objective

We aimed to assess the reliability of a mobile telephone application (MDi-Psoriasis) designed to help the dermatologist make decisions on how to treat patients with moderate to severe psoriasis.

Method

We analyzed interobserver agreement between the advice given by an expert panel and the recommendations of the MDi-Psoriasis application in 10 complex cases of moderate to severe psoriasis. The experts were asked their opinion on which treatments were most appropriate, possible, or inappropriate. Data from the same 10 cases were entered into the MDi-Psoriasis application. Agreement was analyzed in 3 ways: paired interobserver concordance (Cohen's κ), multiple interobserver concordance (Fleiss's κ), and percent agreement between recommendations.

Results

The mean percent agreement between the total of 1210 observations was 51.3% (95% CI, 48.5–54.1%). Cohen's κ statistic was 0.29 and Fleiss's κ was 0.28. Mean agreement between pairs of human observers only, excluding the MDi-Psoriasis recommendations, was 50.5% (95% CI, 47.6–53.5%). Paired agreement between the recommendations of the MDi-Psoriasis tool and the majority opinion of the expert panel (Cohen's κ) was 0.44 (68.2% agreement).

Conclusions

The MDi-Psoriasis tool can generate recommendations that are comparable to those of experts in psoriasis.

Resumen
Introducción

Las decisiones terapéuticas en el paciente con psoriasis están influidas por factores de la enfermedad (gravedad, localización, etc.), de las comorbilidades y de otras circunstancias demográficas y clínicas asociadas.

Objetivo

Evaluar la fiabilidad de una aplicación móvil, MDi-Psoriasis®, como instrumento de ayuda al dermatólogo en la toma de decisiones terapéuticas en pacientes con psoriasis moderada-grave.

Método

Estudio de concordancia interobservador entre las recomendaciones terapéuticas emitidas por un grupo de expertos y MDi-Psoriasis® sobre 10 casos clínicos complejos de psoriasis moderada-grave. Para cada uno de los casos los expertos fueron preguntados por el tratamiento más adecuado, posible y no adecuado. Los mismos 10 casos clínicos fueron sometidos a la aplicación MDi-Psoriasis® y se calcularon las concordancias interobservador pareada (kappa de Cohen) y múltiple (kappa de Fleiss), y el porcentaje de acuerdo entre las recomendaciones.

Resultados

Sobre un total de 1.210 observaciones el porcentaje promedio de acuerdo fue del 51,3% (IC 95%: 48,5-54,1%), con una concordancia pareada de kappa=0,29 y múltiple de kappa=0,28. El acuerdo promedio entre pares de observadores, sin MDi-Psoriasis®, fue del 50,5% (IC 95%: 47,6-53,5%). La concordancia pareada entre las recomendaciones emitidas por MDi-Psoriasis® y la opinión mayoritaria de los expertos fue de kappa=0,44, con un acuerdo del 68,2%.

Conclusiones

MDi-Psoriasis® puede emitir recomendaciones comparables a las emitidas por un experto en psoriasis.

Keywords
Psoriasis, Comorbidity, Health applications, eHealth, Biologics, Reproducibility
Palabras clave
Psoriasis, Comorbilidad, Apps, eSalud, Biológicos, Reproducibilidad psoriasis
Introduction

Psoriasis is a chronic disease with a prevalence of 2.3% in the general population of Spain; the prevalence is 2.7% in men and persons over the age of 30 years and increases gradually until the age bracket of 60–69 years.1 Although this skin disease is benign, up to 60% of patients develop concomitant diseases that can affect other organs and systems and even reduce life expectancy.2,3 Conditions that are commonly associated with psoriasis are metabolic (obesity, diabetes, hypertension, hyperlipidemia), inflammatory (arthritis, chronic inflammatory bowel disease, multiple sclerosis), psychiatric (anxiety, depression), or organ-specific (nonalcoholic fatty liver disease, for example). Certain behavioral traits, such as alcoholism, are also associated with psoriasis. These comorbid conditions not only affect the patient's general health (by conferring cardiovascular risk, for example) but also affect therapeutic management by increasing the toxicity of systemic treatments as well as drug resistance.2 Besides associated diseases, other factors such as age, desire to reproduce, and concurrent medication also have significant effects on therapeutic decision-making. They must also be taken into consideration when the clinician is choosing the most appropriate treatment for a particular patient.

Applications for internet-connected mobile phones (smartphones) are now widely available, offering the general population and health professionals unprecedented access to information anywhere, anytime.4 The health care sector has been among the targets favored by developers, whose applications link patients to professionals (mobile phone telemedicine) and can even assist with clinical decision-making. However, few studies have assessed the usability, effectiveness, reliability, and efficiency of health care applications.4

We aimed to assess the reliability of a mobile telephone application (the MDi-Psoriasis) designed to help the dermatologist decide what treatments to propose for patients with moderate to severe psoriasis.

Methods

This cross-sectional analysis of agreement, or concordance, compared the therapeutic decisions suggested by the MDi-Psoriasis application to those made by a group of 10 dermatologists who are experts in the disease.

The MDi-Psoriasis, which runs on iOS and Android devices, guides the dermatologist making therapeutic decisions for patients with moderate to severe psoriasis.5 After a patient's demographic characteristics are input, the application reports the most appropriate, possible, and inappropriate treatments (Fig. 1). The algorithm of the MDi-Psoriasis includes 60 variables related to patient demographic, clinical, and medication characteristics that combine with 11 possible systemic treatments currently available for moderate to severe psoriasis.5 More than 600 individual scenarios and over 10000 combined scenarios are generated.5 The recommendations are based on currently accepted clinical practice guidelines used regularly by various psoriasis groups (the European Academy of Dermatology, German Society of Dermatology, and American Academy of Dermatology), on consensus papers and interest group recommendations (of the National Psoriasis Foundation and the Psoriasis Working Group of the Spanish Academy of Dermatology and Venereology [AEDV]), and on the approved summary of product characteristics for each medication. The MDi-Psoriasis includes all currently approved treatments for moderate to severe psoriasis: UV-B phototherapy, psoralen plus UV-A phototherapy, acitretin, methotrexate, ciclosporin, adalimumab, etanercept, infliximab, ustekinumab, secukinumab, and apremilast.

Figure 1.
(0.29MB).

Screenshot of recommendations made by the MDi-Psoriasis application for patient number 5 (Table 1). NB–UV-A refers to narrow-band UV-A phototherapy; PUVA, psoralen plus UV-A phototherapy. Translator's note: The English texts from this Spanish-language application have been translated for information purposes.

The participating experts had to have more than 5 years’ work experience in psoriasis management units of general hospitals in a Spanish autonomous community (represented were Andalusia, Catalonia, the Canary Islands, and Madrid). The selected experts (authorsJ.M.C., G.C., P.delaC., C.F., L.F., M.G., T.O.V., R.R., L.R.F., R.R.V.) had a median of 15 years’ experience (range, 7–30 years; thus >5years in all cases) and treated at least 50 patients per month (range, 50–200). These experts were asked to comment on 10 fictitious cases of moderate to severe psoriasis that typified different clinical forms, concomitant conditions, prior treatment profiles, and diverse factors that might potentially modify therapeutic decisions (e.g., drug interactions, pregnancy, breastfeeding) (Table 1). The experts answered a set of questions about each case (Table 2).

Table 1.

Clinical Cases Evaluated by the Psoriasis Experts and the MDi-Psoriasis Mobile Telephone Application.

Woman, 28 y. Reports trying to become pregnant. Plaque psoriasis without arthritis (current PASI 8.50). Nail involvement makes office work difficult. Unresponsive to prior NB–UV-B phototherapy. 
Man, 58 y. Plaque psoriasis (PASI 9). Severe scalp involvement (DLQI 14). Onset of peripheral arthritis 5 y ago. Unresponsive to prior treatment with corticosteroids. Nonalcoholic fatty liver disease (AST×3, ALT×2, AST/ALT>1), moderate obesity (BMI36). Basal cell carcinoma operated 3 y ago. 
Man, 45 y. Plaque psoriasis (PASI 15), no arthritis. Previous treatment with NB–UV-B, ciclosporin, and etanercept (poor response). Colon cancer operated 7 y ago. Patient and partner are trying to have a child. 
Girl, 14 y. Guttate psoriasis, extensive, onset 2 months ago; scarce adherence to topical corticosteroid therapy. 
Woman, 33 y. History of epilepsy. Pustular psoriasis affecting general health and function. Previous treatment with methotrexate for >5 y. Reports desire to become pregnant once the current flare-up is over. 
Man, 65 y. Onset of psoriasis 25 y ago, previously treated with acitretin, PUVA, NB–UV-B, and methotrexate. Arthritis, with severe enthesitis. Currently in treatment with adalimumab, moderate response (PASI50); intends to use a combination therapy. Slightly obese (BMI 31), hyperlipidemia, hyperuricemia; excessive alcohol consumption (>20g/d). 
Woman, 49 y. Plaque psoriasis in treatment with etanercept, poor response (currently PASI9 after response; previously PASI 40). Arthritis, mainly axial. Previous treatment with methotrexate, ciclosporin, NB–UV-B, and infliximab with poor response. Moderate renal-prerenal insufficiency due to ciclosporin (FG 50mL/min/1.73m2). Obese (BMI35). Breast cancer operated 3y ago, currently in remission. 
Man, 39 y. Erythrodermic psoriasis treated with ciclosporin during previous flare-ups. Hemodynamically and metabolically stable. Arthritis, mainly axial, significantly limits function. Currently diagnosed with multiple sclerosis for which he is not being treated. 
Man, 74 y. Pustular psoriasis, localized HBV infection. Previous treatment with NB–UV-B for occasional episodes of plaques. Hand dactylitis. 
10  Woman, 51 y. Plaque psoriasis, moderate to severe. Previous treatment with NB–UV-B, PUVA, ciclosporin, methotrexate, etanercept, and adalimumab with scarce response and/or toxicity. On oral antidiabetic medication for type 2 diabetes, latent tuberculosis infection under treatment 

Abbreviations: ALT, alanine transaminase; AST, aspartate transaminase; BMI, body mass index; DLQI, Dermatology Quality of Life Index; FG, fibrinogen; HBV, hepatitis B virus; NB–UV-B, narrow-band UV-B phototherapy; PASI, Psoriasis Area and Severity Index; PUVA, psoralen plus UV-A phototherapy; TB, tuberculosis.

Table 2.

Questions the Experts Answered About Each Case.

Which of the various treatments available for psoriasis seems most appropriate for this patient? 
Which of the various treatments available for psoriasis are possible candidates for use in this case even though they might not be the best choice? 
Which of the various treatments available for psoriasis seem inappropriate for the patient? 
  The experts were asked to select from the following therapies: narrow-band UV-B phototherapy, psoralen plus UV-A phototherapy, acitretin, methotrexate, ciclosporin, etanercept, infliximab, adalimumab, ustekinumab, secukinumab, and apremilast. 
  Each available treatment had to be classified as appropriate, possible, or inappropriate in each case. 

Information about the same 10 cases was input into the MDi-Psoriasis application. The recommendations (most appropriate, possible, inappropriate) were then compared to those of the 10 experts.

The main outcome measure was agreement between the application and the experts according to Cohen's κ statistic (paired interrater agreement) and Fliess's κ statistic (agreement among multiple raters). The secondary outcome measure was the level of agreement between the recommendations of the experts and the application for most appropriate treatment (first choice). We first calculated paired and multiple concordance rates between the experts without considering the recommendations of the MDi-Psoriasis application.

Because the ratings were qualitative variables, we calculated both the percentages of absolute agreement and their 95% CIs as well as both κ statistics. The κ statistics were interpreted according to the following cutoffs: poor agreement, <0.20; weak, ≥0.21 and <0.40; moderate, ≥0.41 and <0.60; good or substantial, ≥0.61 and <0.80; almost perfect, ≥0.81 and <1.0.6 XLSTAT for Mac and ReCalc3 software was used for these analyses.

Results

The 10 experts and the MDi-Psoriasis application produced a total of 1210 evaluations. The mean level of absolute agreement between the application and the experts was 51.3% (95% CI, 48.5%–54.1%) (Fig. 2). The κ statistic for agreement between the MDi-Psoriasis and each of the experts was 0.29 (κ=0.28 between the application and the group of experts). Absolute agreement for paired expert observers was moderate to low, ranging from 34.6% to 67.3%, giving a mean percent coincidence of 50.5% (95% CI, 47.6%–53.5%) (Fig. 2). The κ statistic was 0.29 for paired observers and 0.28 for multiple observers. The κ statistic for paired agreement between the recommendations of the MDi-Psoriasis tool and the majority opinion of the expert panel was 0.44 (68.2% absolute agreement).

Figure 2.
(0.26MB).

Percent agreement for paired human observers A–J and the MDi-Psoriasis application.

The first row shows the percent agreement between the application and each of the psoriasis experts.

Percent agreement between the first-choice recommendation of the majority of the experts and the MDi-Psoriasis application was 87.3% (κ=0.59). When the MDi-Psoriasis application was excluded, percent agreement on this outcome was 79.3%, but the κ statistic for multiple-observer agreement was 0.33. Apremilast and secukinumab were the first and second choices most often made by the MDi-Psoriasis application. Apremilast and ciclosporin were the first and second choices most often made by the expert consultants (Fig. 3). Absolute paired agreement on the cases ranged from 75.4% (case 10) to 36.2% (case 9) (Fig. 4).

Figure 3.
(0.15MB).

Frequency with which the experts and the MDi-Psoriasis application recommended each of the available treatments.

ACI refers to acitretin; ADA, adalimumab; APR, apremilast, CIC, ciclosporin; ETN, etanercept; INF, infliximab; MTX, methotrexate; SEC, secukinumab; UST, ustekinumab.

Figure 4.
(0.24MB).

Percent agreement between the experts and the MDi-Psoriasis application on each of the 10 cases. Pso refers to psoriasis; HBV, hepatitis B virus.

Discussion

Agreement on a series of complex psoriasis cases between an automatically executed algorithm and a group of psoriasis experts was studied with concordance statistics. However, beyond the issue of the application's reliability, this study also yielded interesting information about the level of uncertainty about how to treat psoriasis. We found that experts varied in the decisions they made in the complex cases presented. This variability, demonstrated by weak interobserver agreement (Fleiss's κ of 0.28) can be attributed to explanations ranging from differences in the experts’ preferences for and experiences with each of the medications, on the one hand, to the relative ease of availability of each treatment in the hospitals where they work. The influence of a physicians’ preference for a specific, more familiar, medication can be appreciated in the differences between the frequencies of recommendations for apremilast and secukinumab by the experts and by the MDi-Psoriasis application (Fig. 3). Because these 2 drugs have only recently been incorporated into the therapeutic arsenal, the experts have had less experience with them. An automatically executed algorithm, such as governs the MDi-Psoriasis, would be unable to take that into consideration.

All the experts reported they had access to all the currently available medications for psoriasis and could prescribe them. Phototherapy equipment was also at their disposal. The scientific evidence the MDi-Psoriasis algorithm was based on, summarized in various guidelines and consensus papers,7–10 also reflects a certain degree of variability, particularly regarding differences in the absolute or relative weighting of contraindications that are built into the MDi-Psoriasis. For example, attitudes toward factors such as liver disease, alcoholism, or desire to reproduce can vary, as can the approach to recommendations for treatments appropriate for difficult-to-treat locations (scalp, palms and soles, nails) or pustular forms.11 Pustular psoriasis figured in 2 of the 3 cases on which agreement was weakest (with rates of 36.2% and 41.2%) (Fig. 4). In contrast, interrater agreement (including the MDI-Psoriasis application's recommendations) reached the highest level (79.3%) only regarding the first-choice treatment because possible relative or absolute contraindications did not enter into that decision.

Finally, the MDi-Psoriasis application achieved a paired interrater agreement equivalent to that of the experts, and agreement was higher in the comparison between the application's opinion and the majority opinion. The fact that the MDi-Psoriasis agreed more often with the consensus recommendation suggests it is able to give advice that coincides with options that are most widely accepted by experts and that are therefore more robust.

A methodological limitation was the difficulty of reaching high κ values in studies with multiple observers. Whereas percent agreement only reflects the number of identical decisions made, the κ statistic incorporates both the percentage and the number of random agreements expected to occur.6 This statistic is therefore considered the most appropriate one for studying interrater agreement.

This study of reliability provided evidence that the MDi-Psoriasis application gives recommendations at least comparable to those given by an expert on psoriasis management. Use of this automatically executed algorithm based on current guidelines, developed as a smartphone application for the dermatologist, can help prescribers with therapeutic decision-making in the complex scenarios psoriasis patients sometimes present.

Ethical DisclosuresProtection of human and animal subjects

The authors declare that no experiments were performed on humans or animals for this investigation.

Data confidentiality

The authors declare that they followed their hospitals’ regulations regarding the publication of patient information and that written informed consent for voluntary participation was obtained for all patients.

Right to privacy and informed consent

The authors declare that no private patient data are disclosed in this article.

Conflicts of Interest

The authors have participated in training events and done research and served as consultants for the following laboratories: Abbvie, Amgen, Celgene, Gebro, Janssen-Cilag, LEO-Pharma, Lilly, MSD, Novartis, and Pfizer.

References
1
C. Ferrándiz,J.M. Carrascosa,M. Roro
Prevalencia de la psoriasis en la era de los biológicos
Actas Dermosifiliogr, 105 (2014), pp. 504-509 http://dx.doi.org/10.1016/j.ad.2013.12.008
2
L. Puig
La psoriasis ¿una enfermedad sistémica?
Actas Dermosifiliogr, 98 (2007), pp. 396-402
3
E. Daudén,S. Castañeda,C. Suárez,J. García-Campayo,A.J. Blasco,M.D. Aguilar
Abordaje integral de la comorbilidad del paciente con psoriasis
Actas Dermosifiliogr, 103 (2012), pp. 1-64 http://dx.doi.org/10.1016/S0001-7310(12)70001-7
4
M.N. Boulos,A.C. Brewer,C. Karimkhani,D.B. Buller,R.P. Dellavalle
Mobile medical and health apps: State of the art, concerns, regulatory control and certification
Online J Public Health Inform, 5 (2014), pp. e229
5
Moreno-Ramírez D, Ferrándiz L, Herrerías-Esteban JM. MDi-Psoriasis [cited 2016 Sept 27]. Available from: www.mdipsoriasis.es.
6
López de Ullibarri I, Pita S. Medidas de concordancia: el coeficiente kappa [cited 2016 Sept 27]. Available from: http://fisterra.com/mbe/investiga/kappa/kappa2.pdf.
7
D. Pathirana,A.D. Ormerod,P. Saiag,C. Smith,P.I. Spuls,A. Nast
European S3-guidelines on the systemic treatment of psoriasis vulgaris
J Eur Acad Dermatol Venereol, 23 (2009), pp. 1-70 http://dx.doi.org/10.1111/j.1468-3083.2009.03389.x
8
E. Daudén,L. Puig,C. Ferrándiz,J.L. Sánchez-Carazo,J.M. Hernanz-Hermosa
Consensus document on the evaluation and treatment of moderate-to-severe psoriasis: Psoriasis group of the Spanish Academy of Dermatology and Venereology
J Eur Acad Dermatol Venereol, 30 (2016), pp. 1-18 http://dx.doi.org/10.1111/jdv.13542
9
A. Nast,W.H. Boehncke,U. Mrowietz,H.M. Ockenfels,S. Philipp,K. Reich
S3-Guidelines on the treatment of psoriasis vulgaris (English version). Update
J Dtsch Dermatol Ges, 10 (2012), pp. S1-S95 http://dx.doi.org/10.1111/j.1610-0379.2012.08019.x
10
L. Puig,J.M. Carrascosa,G. Carretero,P. de la Cueva,R.F. Lafuente-Urrez,I. Belinchón
Spanish evidence-based guidelines on the treatment of psoriasis with biologic agents, 2013. Part 1: on efficacy and choice of treatment. Spanish Psoriasis Group of the Spanish Academy of Dermatology and Venereology
Actas Dermosifiliogr, 104 (2013), pp. 694-709 http://dx.doi.org/10.1016/j.adengl.2013.04.013
11
M. Sánchez-Regaña,M.J. Aldunce Soto,I. Belinchón Romero,M. Ribera Pibernat,R.F. Lafuente-Urrez,J.M. Carrascosa Carrillo
Evidence-based guidelines of the Spanish psoriasis group on the use of biologic therapy in patients with psoriasis in difficult-to-treat sites (nails, scalp, palms, and soles)
Actas Dermosifiliogr, 105 (2014), pp. 923-934 http://dx.doi.org/10.1016/j.ad.2014.02.015

Please cite this article as: Moreno-Ramírez D, Herrerías-Esteban JM, Ojeda-Vila T, Carrascosa JM, Carretero G, de la Cueva P, et al. Fiabilidad de una aplicación de ayuda a la toma de decisiones terapéuticas en el paciente con psoriasis (MDi Psoriasis®). Actas Dermosifiliogr. 2017;108:650–656.

Corresponding author. (D. Moreno-Ramírez david.moreno.ramirez.sspa@juntadeandalucia.es)
Copyright © 2017. Elsevier España, S.L.U. and AEDV
Actas Dermosifiliogr 2017;108:650-6 - Vol. 108 Num.7 DOI: 10.1016/j.adengl.2017.06.002