Psoriasis is a chronic disease with a prevalence of 2.3% in the general population of Spain; the prevalence is 2.7% in men and persons over the age of 30 years and increases gradually until the age bracket of 60–69 years.1 Although this skin disease is benign, up to 60% of patients develop concomitant diseases that can affect other organs and systems and even reduce life expectancy.2,3 Conditions that are commonly associated with psoriasis are metabolic (obesity, diabetes, hypertension, hyperlipidemia), inflammatory (arthritis, chronic inflammatory bowel disease, multiple sclerosis), psychiatric (anxiety, depression), or organ-specific (nonalcoholic fatty liver disease, for example). Certain behavioral traits, such as alcoholism, are also associated with psoriasis. These comorbid conditions not only affect the patient's general health (by conferring cardiovascular risk, for example) but also affect therapeutic management by increasing the toxicity of systemic treatments as well as drug resistance.2 Besides associated diseases, other factors such as age, desire to reproduce, and concurrent medication also have significant effects on therapeutic decision-making. They must also be taken into consideration when the clinician is choosing the most appropriate treatment for a particular patient.

Applications for internet-connected mobile phones (smartphones) are now widely available, offering the general population and health professionals unprecedented access to information anywhere, anytime.4 The health care sector has been among the targets favored by developers, whose applications link patients to professionals (mobile phone telemedicine) and can even assist with clinical decision-making. However, few studies have assessed the usability, effectiveness, reliability, and efficiency of health care applications.4

We aimed to assess the reliability of a mobile telephone application (the MDi-Psoriasis) designed to help the dermatologist decide what treatments to propose for patients with moderate to severe psoriasis.

This cross-sectional analysis of agreement, or concordance, compared the therapeutic decisions suggested by the MDi-Psoriasis application to those made by a group of 10 dermatologists who are experts in the disease.

The MDi-Psoriasis, which runs on iOS and Android devices, guides the dermatologist making therapeutic decisions for patients with moderate to severe psoriasis.5 After a patient's demographic characteristics are input, the application reports the most appropriate, possible, and inappropriate treatments (Fig. 1). The algorithm of the MDi-Psoriasis includes 60 variables related to patient demographic, clinical, and medication characteristics that combine with 11 possible systemic treatments currently available for moderate to severe psoriasis.5 More than 600 individual scenarios and over 10000 combined scenarios are generated.5 The recommendations are based on currently accepted clinical practice guidelines used regularly by various psoriasis groups (the European Academy of Dermatology, German Society of Dermatology, and American Academy of Dermatology), on consensus papers and interest group recommendations (of the National Psoriasis Foundation and the Psoriasis Working Group of the Spanish Academy of Dermatology and Venereology [AEDV]), and on the approved summary of product characteristics for each medication. The MDi-Psoriasis includes all currently approved treatments for moderate to severe psoriasis: UV-B phototherapy, psoralen plus UV-A phototherapy, acitretin, methotrexate, ciclosporin, adalimumab, etanercept, infliximab, ustekinumab, secukinumab, and apremilast.

Figure 1.

Screenshot of recommendations made by the MDi-Psoriasis application for patient number 5 (Table 1). NB–UV-A refers to narrow-band UV-A phototherapy; PUVA, psoralen plus UV-A phototherapy. Translator's note: The English texts from this Spanish-language application have been translated for information purposes.

(0.29MB).

The participating experts had to have more than 5 years’ work experience in psoriasis management units of general hospitals in a Spanish autonomous community (represented were Andalusia, Catalonia, the Canary Islands, and Madrid). The selected experts (authorsJ.M.C., G.C., P.delaC., C.F., L.F., M.G., T.O.V., R.R., L.R.F., R.R.V.) had a median of 15 years’ experience (range, 7–30 years; thus >5years in all cases) and treated at least 50 patients per month (range, 50–200). These experts were asked to comment on 10 fictitious cases of moderate to severe psoriasis that typified different clinical forms, concomitant conditions, prior treatment profiles, and diverse factors that might potentially modify therapeutic decisions (e.g., drug interactions, pregnancy, breastfeeding) (Table 1). The experts answered a set of questions about each case (Table 2).

Information about the same 10 cases was input into the MDi-Psoriasis application. The recommendations (most appropriate, possible, inappropriate) were then compared to those of the 10 experts.

The main outcome measure was agreement between the application and the experts according to Cohen's κ statistic (paired interrater agreement) and Fliess's κ statistic (agreement among multiple raters). The secondary outcome measure was the level of agreement between the recommendations of the experts and the application for most appropriate treatment (first choice). We first calculated paired and multiple concordance rates between the experts without considering the recommendations of the MDi-Psoriasis application.

Because the ratings were qualitative variables, we calculated both the percentages of absolute agreement and their 95% CIs as well as both κ statistics. The κ statistics were interpreted according to the following cutoffs: poor agreement, <0.20; weak, ≥0.21 and <0.40; moderate, ≥0.41 and <0.60; good or substantial, ≥0.61 and <0.80; almost perfect, ≥0.81 and <1.0.6 XLSTAT for Mac and ReCalc3 software was used for these analyses.

The 10 experts and the MDi-Psoriasis application produced a total of 1210 evaluations. The mean level of absolute agreement between the application and the experts was 51.3% (95% CI, 48.5%–54.1%) (Fig. 2). The κ statistic for agreement between the MDi-Psoriasis and each of the experts was 0.29 (κ=0.28 between the application and the group of experts). Absolute agreement for paired expert observers was moderate to low, ranging from 34.6% to 67.3%, giving a mean percent coincidence of 50.5% (95% CI, 47.6%–53.5%) (Fig. 2). The κ statistic was 0.29 for paired observers and 0.28 for multiple observers. The κ statistic for paired agreement between the recommendations of the MDi-Psoriasis tool and the majority opinion of the expert panel was 0.44 (68.2% absolute agreement).

Figure 2.

Percent agreement for paired human observers A–J and the MDi-Psoriasis application.

The first row shows the percent agreement between the application and each of the psoriasis experts.

(0.26MB).

Percent agreement between the first-choice recommendation of the majority of the experts and the MDi-Psoriasis application was 87.3% (κ=0.59). When the MDi-Psoriasis application was excluded, percent agreement on this outcome was 79.3%, but the κ statistic for multiple-observer agreement was 0.33. Apremilast and secukinumab were the first and second choices most often made by the MDi-Psoriasis application. Apremilast and ciclosporin were the first and second choices most often made by the expert consultants (Fig. 3). Absolute paired agreement on the cases ranged from 75.4% (case 10) to 36.2% (case 9) (Fig. 4).

Figure 3.

Frequency with which the experts and the MDi-Psoriasis application recommended each of the available treatments.

ACI refers to acitretin; ADA, adalimumab; APR, apremilast, CIC, ciclosporin; ETN, etanercept; INF, infliximab; MTX, methotrexate; SEC, secukinumab; UST, ustekinumab.

(0.15MB).

Figure 4.

Percent agreement between the experts and the MDi-Psoriasis application on each of the 10 cases. Pso refers to psoriasis; HBV, hepatitis B virus.

(0.24MB).

Agreement on a series of complex psoriasis cases between an automatically executed algorithm and a group of psoriasis experts was studied with concordance statistics. However, beyond the issue of the application's reliability, this study also yielded interesting information about the level of uncertainty about how to treat psoriasis. We found that experts varied in the decisions they made in the complex cases presented. This variability, demonstrated by weak interobserver agreement (Fleiss's κ of 0.28) can be attributed to explanations ranging from differences in the experts’ preferences for and experiences with each of the medications, on the one hand, to the relative ease of availability of each treatment in the hospitals where they work. The influence of a physicians’ preference for a specific, more familiar, medication can be appreciated in the differences between the frequencies of recommendations for apremilast and secukinumab by the experts and by the MDi-Psoriasis application (Fig. 3). Because these 2 drugs have only recently been incorporated into the therapeutic arsenal, the experts have had less experience with them. An automatically executed algorithm, such as governs the MDi-Psoriasis, would be unable to take that into consideration.

All the experts reported they had access to all the currently available medications for psoriasis and could prescribe them. Phototherapy equipment was also at their disposal. The scientific evidence the MDi-Psoriasis algorithm was based on, summarized in various guidelines and consensus papers,7–10 also reflects a certain degree of variability, particularly regarding differences in the absolute or relative weighting of contraindications that are built into the MDi-Psoriasis. For example, attitudes toward factors such as liver disease, alcoholism, or desire to reproduce can vary, as can the approach to recommendations for treatments appropriate for difficult-to-treat locations (scalp, palms and soles, nails) or pustular forms.11 Pustular psoriasis figured in 2 of the 3 cases on which agreement was weakest (with rates of 36.2% and 41.2%) (Fig. 4). In contrast, interrater agreement (including the MDI-Psoriasis application's recommendations) reached the highest level (79.3%) only regarding the first-choice treatment because possible relative or absolute contraindications did not enter into that decision.

Finally, the MDi-Psoriasis application achieved a paired interrater agreement equivalent to that of the experts, and agreement was higher in the comparison between the application's opinion and the majority opinion. The fact that the MDi-Psoriasis agreed more often with the consensus recommendation suggests it is able to give advice that coincides with options that are most widely accepted by experts and that are therefore more robust.

A methodological limitation was the difficulty of reaching high κ values in studies with multiple observers. Whereas percent agreement only reflects the number of identical decisions made, the κ statistic incorporates both the percentage and the number of random agreements expected to occur.6 This statistic is therefore considered the most appropriate one for studying interrater agreement.

This study of reliability provided evidence that the MDi-Psoriasis application gives recommendations at least comparable to those given by an expert on psoriasis management. Use of this automatically executed algorithm based on current guidelines, developed as a smartphone application for the dermatologist, can help prescribers with therapeutic decision-making in the complex scenarios psoriasis patients sometimes present.

The authors have participated in training events and done research and served as consultants for the following laboratories: Abbvie, Amgen, Celgene, Gebro, Janssen-Cilag, LEO-Pharma, Lilly, MSD, Novartis, and Pfizer.